Question: (Closed) What Does P<0.05 Mean?
4
gravatar for Dole
3.2 years ago by
Dole200
Dole200 wrote:

hi there

I'm new in the field, and I read some plos articles. I meet the term p<0.05, but I don't know what it means. Can you give me a keyword or something to help me understand what that p is about?

During the cranberry period, 6 subjects had 7 UTI, compared with 16 subjects and 21 UTI in the placebo period (P<0.05 for both number of subjects and incidence).

ADD COMMENTlink modified 2.3 years ago by brentp17k • written 3.2 years ago by Dole200
3

Off topic, general statistics question. See http://en.wikipedia.org/wiki/P-value or post to http://stats.stackexchange.com/. Should this one be closed?

ADD REPLYlink written 3.2 years ago by Casey Bergman15k
2

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I had this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is not asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

ADD REPLYlink written 3.2 years ago by Bio_X2Y2.8k
1

@dole "What is a p-value anyway?: 34 Stories to Help You Actually Understand Statistics" http://www.amazon.com/p-value-Stories-Actually-Understand-Statistics/dp/0321629302

ADD REPLYlink written 3.2 years ago by Khader Shameer14k

Could you quote the sentence in which it was used, or describe the model they were testing please. The keywords I'd go for are 'hypothesis testing', 'null hypothesis', 'probability'.

ADD REPLYlink written 3.2 years ago by Russh1.1k

@Casey Bergman: TY, that's the answer I was looking for.

ADD REPLYlink written 3.2 years ago by Dole200

@Casey Bergman: TY. That's the answer I was looking for. The wikipedia link is very valuable.

ADD REPLYlink written 3.2 years ago by Dole200

Since it was answered in the comments, I think we can close as "no longer relevant".

ADD REPLYlink written 3.2 years ago by Neilfws41k

How is babby formed?

ADD REPLYlink written 3.2 years ago by Casbon2.7k

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I have this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

ADD REPLYlink written 3.2 years ago by Bio_X2Y2.8k

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I had this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

ADD REPLYlink written 3.2 years ago by Bio_X2Y2.8k

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I have this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is not asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

ADD REPLYlink written 3.2 years ago by Bio_X2Y2.8k

I only closed it because the questioner stated "That's the answer I was looking for" in a comment thread, before any real answers were posted. If people think it's useful to leave open, no problem.

ADD REPLYlink written 3.2 years ago by Neilfws41k

Absolutely, completely off topic, and answered as well as it's going to be be answered. So back to closed ;)

ADD REPLYlink written 2.3 years ago by Daniel Swan10k
8
gravatar for Sequencegeek
3.2 years ago by
Sequencegeek640
UCLA
Sequencegeek640 wrote:

From Wiki: http://en.wikipedia.org/wiki/P-value

"the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed"

The purpose of a p-value is to determine if a value you observed is significant. Usually your p-value comes from an observation that you assume is governed by events whose probabilities can be represented by a control distribution. Usually your hope is that your observed value differs significantly from the mean of the control distribution.

Maybe its best explained with an example:

Suppose you were trying to prove that a coin you had was weighted towards flipping heads 70% of the time. We know that an un-weighted coin will give heads or tails with equal probability, so we need to show that your coin will give heads more than tails if flipped X amount of times. A weighted coin will produce (number of times flipped)(probability of being heads ) = (100)(.5) = 50 heads in 100 flips on average. Your coin will produce (100)*(.7) = 70 heads in 100 flips on average. A p-value is a statistic that answers the question "How much does my observed value (in this case 70 heads) need to differ from a control value (in this case 50) in order for me to conclude that my coin is weighted?"

ADD COMMENTlink written 3.2 years ago by Sequencegeek640
4
gravatar for Istvan Albert
3.2 years ago by
Istvan Albert ♦♦ 39k
University Park, USA
Istvan Albert ♦♦ 39k wrote:

I have reopened this questions for the following reason, I have just finished interviewing 5 bioinformatics candidates selected from 25 applicants. I prepared 10 questions for each of them; one question was in essence: What does a p-value mean?

To my great shock not a single candidate was able to answer or was even on the right track! The better ones believed that p-values are some sort of a similarity measure. Yet most of them had multiple publications in selective journals (using p-values of course).

The current science education in general does a horrible job when it comes to educating researchers in basic descriptive statistics. The "p-value abuse" is rampant in peer reviewed publications. When a simple concept is misunderstood by so many the problem lies in the education not with the individual.

So I'd welcome this question I think it is relevant and important, and for everyone that thinks this is too easy or obvious, walk around your lab and ask a few collaborators what a p-value is, you'll be surprised (or saddened).

ADD COMMENTlink written 3.2 years ago by Istvan Albert ♦♦ 39k
2

Here it is for the first: awk '{print $3}' table.txt | tac :) I won't put the python code for Fizz Buzz here, but it's a 213 caracter program.

ADD REPLYlink written 3.2 years ago by Eric Normandeau7.1k
1

Well, really shocking. btw, I knew the answer, could I have the job ? ;D

ADD REPLYlink written 3.2 years ago by Michael Dondrup27k
1

@Michael only after passing the live coding test where you'd have print out the third column of a tab delimited file in reverse order no less ... for the record I am doing the so called FizzBuzz testing to filter candidates http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html

ADD REPLYlink written 3.2 years ago by Istvan Albert ♦♦ 39k

Shocking indeed! Of all the courses I had during my university education, the one that has been most useful to me since was a statistics course.

ADD REPLYlink written 3.2 years ago by Lars Juhl Jensen8.5k

Cool, both are one-liners in R...

ADD REPLYlink written 3.2 years ago by Michael Dondrup27k

Didn't know about tac so far, thanks Eric :)

ADD REPLYlink written 3.2 years ago by Michael Schubert5.9k

Also, 213 characters seems a little long for Python ;) print [e[2] for e in line.split() for line in open(file)].reverse()

ADD REPLYlink written 3.2 years ago by Michael Schubert5.9k

@Michael: Indeed, it would be too long for the 3 column problem, but this was for the FizzBuzz problem :)

ADD REPLYlink written 3.2 years ago by Eric Normandeau7.1k

Interesting indeed. But i'd like to know the other 9 questions as well.

ADD REPLYlink written 2.3 years ago by Israel Barrantes320

For the 3rd column problem, cut -f3 table.txt | tac would also work (and it's slightly shorter than Eric's solution!).

ADD REPLYlink written 2.3 years ago by Frédéric Mahé1.6k

Is it allowable to answer in the form of an xkcd comic?

http://xkcd.com/882/

It's not a complete answer, but it gets to the heart of why assuming that a low p-vlue = being done is a bad idea.

ADD REPLYlink written 2.3 years ago by Swbarnes21.1k
2
gravatar for brentp
2.3 years ago by
brentp17k
Denver, Colorado
brentp17k wrote:

Given a test of some effect where p = 0.05 according to some test-statistic t, you expect to see a test-statistic greater than t (or equivalently, a p-value < 0.05) on a test of random data 5% of the time.

By accepting p = 0.05 for a single test, you're accepting that there's a 5% chance that effect or difference may be due to random variation--and that there may not be an actual "effect" at all.

A consequence of this is that, if the magic number for publication is p <= 0.05, then, the expected number of publications which have erroneously rejected their null hypothesis is:

p_average * N_publications

so, given a fixed p-value cutoff, we can expect the number of falsely rejected null hypothesis to increase as more papers are published.

ADD COMMENTlink written 2.3 years ago by brentp17k
1

false rejection at a fixed alpha level is only pertinent to studies that test the same null hypothesis, so it is not N_publications_total that matters but N_publications_that_test_hypothesis_X

ADD REPLYlink written 2.3 years ago by Casey Bergman15k
1

Imagine papers reporting 100 tests of null-1 and 100 papers reporting tests of null-2 at alpha=0.05. You would expect 5 false rejections of null-1 and independetly 5 false rejections of null-2 collectively in these 200 papers. You would not expect 10 false rejections of null-1 and 10 false rejections of null-2, as you imply by assuming that the total number of publications is the relevant quantity. It is only the number of publications that test a specific null hypothesis (e.g. null-1) that matters.

ADD REPLYlink written 2.3 years ago by Casey Bergman15k

Casey, really? I'd say that by definition, the number of tests (publications) is a reasonable multiplier. I'm not sure how you're using alpha here.

ADD REPLYlink written 2.3 years ago by brentp17k

the alpha level sets the acceptable type I error (in this case 0.05) and is the arbitrary cutoff chosen such that when replicating a test of the same null hypothesis, one expects to reject that particular null hypothesis falsely at a rate = alpha (see http://en.wikipedia.org/wiki/Type_I_error#Type_I_error). It is not the total number of tests in all of science that matters, but the number of tests for a particular null hypothesis that will give you an estimate of the number of false rejections.

ADD REPLYlink written 2.3 years ago by Casey Bergman15k

From the wikipedia link: "Suppose, the probability for a type I error is 1% , then there is a 1% chance that the observed variation is not true. This is called the level of significance, denoted with the Greek letter α (alpha)." Not sure how you think that differs from what I've written. If 100 tests have a probability of type I error of 0.01, then the MLE of false rejections is 1. Do you disagree? (I'm really trying to understand, not argue)

ADD REPLYlink written 2.3 years ago by brentp17k

Maybe we are saying the same thing. In your example, you have a total 200 tests at alpha == 0.05. 0.05 * 200 == 10 == 5 + 5 false rejections in total.

ADD REPLYlink written 2.3 years ago by brentp17k

I'm not sure we are saying the same thing...but in this example you would expect 10 total false rejections (as would be expected by multiplying by N_publications_that_test_hypothesis_X for each null), not 20 total (as would be expected by multplying N_publications_total for each null). The key correction in you reply is to change your N_publications to be conditioned on a particular null, eg. N_publications_for_hypothesis_X.

ADD REPLYlink written 2.3 years ago by Casey Bergman15k

My thought is that once you have a p-value, it's just a p-value, so it doesn't matter what test it came from. So, it doesn't need to be conditioned on a particular h0 , that's just where it came from.

ADD REPLYlink written 2.3 years ago by brentp17k

Just to clarify, I'm only suggesting to do n_publications * pvalue once.

ADD REPLYlink written 2.3 years ago by brentp17k

Now I'm sure we are not saying the same thing, and I do disagree with your claims. P-values are not exchangable objects that have the same meaning across null hypotheses or statistical tests. A P-value is tied explicitly to a specific null hypothesis and a specific statistical test. Even when using data aggregation methods that combine P-values (e.g Fisher's method http://en.wikipedia.org/wiki/Fisher%27s_method) they must still all pertain to the same null hypothesis.

ADD REPLYlink written 2.3 years ago by Casey Bergman15k

My point is that two p-values of 0.05 from different tests tell you the same thing about the likelihood of that test being a false positive.

ADD REPLYlink written 2.3 years ago by brentp17k
Please log in to add an answer.
The thread is closed. No new answers may be added.

Help
Access
  • RSS
  • Stats
  • API

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.0.0
Traffic: 380 users visited in the last hour