p-Values Are Random Variables
p-Values Are Random Variables
We consider -values for or simulations using a random sample of size from a normal distribution with mean and unit variance to compute the two-sided -values for the test of the null hypothesis, :μ=0 versus :μ=δ using the -distribution method as implemented in the Mathematica function MeanTest. When , the -values are uniformly distributed on . With simulations, the result is obtained very quickly but there is more random variability in the histogram. Increasing to simulations takes less than three seconds on most modern computers and provides a more accurate result.
p
3
10
4
10
n
δ
n
p
H
0
H
a
n
t
δ=0
p
(0,1)
3
10
4
10
Mouseover the first rectangle to see the estimate of the probability of the power of a 5% test; the area of this rectangle represents observed probability of -values in the interval .
p
(0,0.05)
The slider changes the alternate hypothesis. When , the -values are no longer uniform on . The area under the first rectangle gives an estimate of the probability that the -value is less than 0.05. This is the estimated power of a two-sided test at the 5% level for :μ=δ
n
δδ>0
p
(0,1)
p
H
a
n
.The distribution of the -values may also be visualized using a Q-Q plot in which the quantiles of the -values are plots against the corresponding quantiles from a uniform distribution.
p
p
(0,1)