Question: (Closed) What Does P<0.05 Mean?
4
6.0 years ago by
Dole200
Dole200 wrote:

hi there

I'm new in the field, and I read some plos articles. I meet the term p<0.05, but I don't know what it means. Can you give me a keyword or something to help me understand what that p is about?

During the cranberry period, 6 subjects had 7 UTI, compared with 16 subjects and 21 UTI in the placebo period (P<0.05 for both number of subjects and incidence).

p-value statistics • 21k views
modified 5.1 years ago by brentp21k • written 6.0 years ago by Dole200
3

Off topic, general statistics question. See http://en.wikipedia.org/wiki/P-value or post to http://stats.stackexchange.com/. Should this one be closed?

2

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I had this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is not asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

1

@dole "What is a p-value anyway?: 34 Stories to Help You Actually Understand Statistics" http://www.amazon.com/p-value-Stories-Actually-Understand-Statistics/dp/0321629302

Could you quote the sentence in which it was used, or describe the model they were testing please. The keywords I'd go for are 'hypothesis testing', 'null hypothesis', 'probability'.

@Casey Bergman: TY, that's the answer I was looking for.

@Casey Bergman: TY. That's the answer I was looking for. The wikipedia link is very valuable.

Since it was answered in the comments, I think we can close as "no longer relevant".

How is babby formed?

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I have this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I had this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

@all. I think this is a valid question on a bioinformations site, and I think a simple answer like "it's a statistical concept - do some background reading about hypothesis testing" would be valid answer. I have this exact question within a day or two of starting in bioinformatics. I don't see why it should be closed, or even downvoted. The author is not asking for a huge explanation - to quote "can you give me a keyword or something to help me understand what p is about".

I only closed it because the questioner stated "That's the answer I was looking for" in a comment thread, before any real answers were posted. If people think it's useful to leave open, no problem.

Absolutely, completely off topic, and answered as well as it's going to be be answered. So back to closed ;)

8
6.0 years ago by
Sequencegeek700
UCLA
Sequencegeek700 wrote:

From Wiki: http://en.wikipedia.org/wiki/P-value

"the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed"

The purpose of a p-value is to determine if a value you observed is significant. Usually your p-value comes from an observation that you assume is governed by events whose probabilities can be represented by a control distribution. Usually your hope is that your observed value differs significantly from the mean of the control distribution.

Maybe its best explained with an example:

5
6.0 years ago by
Istvan Albert ♦♦ 69k
University Park, USA
Istvan Albert ♦♦ 69k wrote:

I have reopened this questions for the following reason, I have just finished interviewing 5 bioinformatics candidates selected from 25 applicants. I prepared 10 questions for each of them; one question was in essence: What does a p-value mean?

To my great shock not a single candidate was able to answer or was even on the right track! The better ones believed that p-values are some sort of a similarity measure. Yet most of them had multiple publications in selective journals (using p-values of course).

The current science education in general does a horrible job when it comes to educating researchers in basic descriptive statistics. The "p-value abuse" is rampant in peer reviewed publications. When a simple concept is misunderstood by so many the problem lies in the education not with the individual.

So I'd welcome this question I think it is relevant and important, and for everyone that thinks this is too easy or obvious, walk around your lab and ask a few collaborators what a p-value is, you'll be surprised (or saddened).

2

Here it is for the first: awk '{print \$3}' table.txt | tac :) I won't put the python code for Fizz Buzz here, but it's a 213 caracter program.

1

Well, really shocking. btw, I knew the answer, could I have the job ? ;D

1

@Michael only after passing the live coding test where you'd have print out the third column of a tab delimited file in reverse order no less ... for the record I am doing the so called FizzBuzz testing to filter candidates http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html

Shocking indeed! Of all the courses I had during my university education, the one that has been most useful to me since was a statistics course.

Cool, both are one-liners in R...

Didn't know about tac so far, thanks Eric :)

Also, 213 characters seems a little long for Python ;) print [e[2] for e in line.split() for line in open(file)].reverse()

@Michael: Indeed, it would be too long for the 3 column problem, but this was for the FizzBuzz problem :)

Interesting indeed. But i'd like to know the other 9 questions as well.

For the 3rd column problem, cut -f3 table.txt | tac would also work (and it's slightly shorter than Eric's solution!).

Is it allowable to answer in the form of an xkcd comic?

http://xkcd.com/882/

It's not a complete answer, but it gets to the heart of why assuming that a low p-vlue = being done is a bad idea.

2
5.2 years ago by
brentp21k
Salt Lake City, UT
brentp21k wrote:

Given a test of some effect where p = 0.05 according to some test-statistic t, you expect to see a test-statistic greater than t (or equivalently, a p-value < 0.05) on a test of random data 5% of the time.

By accepting p = 0.05 for a single test, you're accepting that there's a 5% chance that effect or difference may be due to random variation--and that there may not be an actual "effect" at all.

A consequence of this is that, if the magic number for publication is p <= 0.05, then, the expected number of publications which have erroneously rejected their null hypothesis is:

``````p_average * N_publications
``````

so, given a fixed p-value cutoff, we can expect the number of falsely rejected null hypothesis to increase as more papers are published.

1

false rejection at a fixed alpha level is only pertinent to studies that test the same null hypothesis, so it is not N_publications_total that matters but N_publications_that_test_hypothesis_X

1

Imagine papers reporting 100 tests of null-1 and 100 papers reporting tests of null-2 at alpha=0.05. You would expect 5 false rejections of null-1 and independetly 5 false rejections of null-2 collectively in these 200 papers. You would not expect 10 false rejections of null-1 and 10 false rejections of null-2, as you imply by assuming that the total number of publications is the relevant quantity. It is only the number of publications that test a specific null hypothesis (e.g. null-1) that matters.

Casey, really? I'd say that by definition, the number of tests (publications) is a reasonable multiplier. I'm not sure how you're using alpha here.

the alpha level sets the acceptable type I error (in this case 0.05) and is the arbitrary cutoff chosen such that when replicating a test of the same null hypothesis, one expects to reject that particular null hypothesis falsely at a rate = alpha (see http://en.wikipedia.org/wiki/Type_I_error#Type_I_error). It is not the total number of tests in all of science that matters, but the number of tests for a particular null hypothesis that will give you an estimate of the number of false rejections.

From the wikipedia link: "Suppose, the probability for a type I error is 1% , then there is a 1% chance that the observed variation is not true. This is called the level of significance, denoted with the Greek letter α (alpha)." Not sure how you think that differs from what I've written. If 100 tests have a probability of type I error of 0.01, then the MLE of false rejections is 1. Do you disagree? (I'm really trying to understand, not argue)

Maybe we are saying the same thing. In your example, you have a total 200 tests at alpha == 0.05. 0.05 * 200 == 10 == 5 + 5 false rejections in total.

I'm not sure we are saying the same thing...but in this example you would expect 10 total false rejections (as would be expected by multiplying by N_publications_that_test_hypothesis_X for each null), not 20 total (as would be expected by multplying N_publications_total for each null). The key correction in you reply is to change your N_publications to be conditioned on a particular null, eg. N_publications_for_hypothesis_X.

My thought is that once you have a p-value, it's just a p-value, so it doesn't matter what test it came from. So, it doesn't need to be conditioned on a particular h0 , that's just where it came from.

Just to clarify, I'm only suggesting to do n_publications * pvalue once.

Now I'm sure we are not saying the same thing, and I do disagree with your claims. P-values are not exchangable objects that have the same meaning across null hypotheses or statistical tests. A P-value is tied explicitly to a specific null hypothesis and a specific statistical test. Even when using data aggregation methods that combine P-values (e.g Fisher's method http://en.wikipedia.org/wiki/Fisher%27s_method) they must still all pertain to the same null hypothesis.

My point is that two p-values of 0.05 from different tests tell you the same thing about the likelihood of that test being a false positive.