Interpret p-values, significance levels and errors
Read the p-value from a hypothesis test on the TI-84 Plus CE, Casio fx-CG50, Casio fx-CG100 or TI-Nspire CX, compare it with the significance level, and write the conclusion the examiner wants.
The p-value is the probability of a result at least as extreme as the one observed, if is true. Compare it with the significance level : if , reject ; if , do not reject . The calculator gives the p-value but never the decision, so write the comparison and the conclusion in the context of the question.
Example
Sixty customers each choose one of three juice flavours. The numbers choosing each flavour are 28, 18, 14. A shop owner claims that the three flavours are equally popular. Carry out a goodness-of-fit test at the 5% significance level. State the p-value and your conclusion.
If the flavours are equally popular, the expected count for each is . The test gives with degrees of freedom, and the p-value is the shaded area under the curve to the right of the statistic: . Since , do not reject : there is insufficient evidence at the 5% level that the flavours are not equally popular. The same p-value would lead to rejecting at the 10% level, because . The conclusion depends on the significance level.
Choose your calculator
How to read a p-value on a TI-84 Plus CE (chi-squared goodness-of-fit test)
Run the test, then compare p with the significance level | |
Press stat and choose 1:Edit.... Type the observed counts in L1 and the expected counts in L2, pressing enter after each. |
L1 L2 28 20 18 20 14 20 |
Press stat, press ▶ twice to reach TESTS and choose D:χ²GOF-Test.... With the Stat Wizards on, set Observed to L1 (2nd 1), Expected to L2 (2nd 2) and df to 2, then choose Calculate. |
χ²GOF-Test Observed:L1 Expected:L2 df:2 Calculate |
The screen shows χ², p and df. The value after p= is the p-value. Compare it with the significance level: if p is smaller, reject ; if it is larger, do not. and to 3 significant figures. Since , do not reject . |
χ²GOF-Test χ²=5.2 p=0.0742736 df=2 CNTRB={3.2,0.2,1.8} |
The calculator never says reject or do not reject. It gives only p. You compare p with the significance level yourself and write the conclusion.
How to read a p-value on a Casio fx-CG50 (chi-squared goodness-of-fit test)
Run the test, then compare p with the significance level | |
Press MENU and choose Statistics. Type the observed counts in List 1 and the expected counts in List 2, pressing EXE after each. |
List1 List2 28 20 18 20 14 20 |
Press F3 (TEST), then F3 (CHI), then F1 (GOF). Set Observed to List 1, Expected to List 2 and df to 2. Leave CNTRB as it is, or choose a spare list to store the contribution of each count. |
χ² GOF Test Observed:List1 Expected:List2 df:2 CNTRB:List3 Execute |
Move to Execute and press F1 (CALC). The result shows χ², p and df. The value after p is the p-value. Compare it with the significance level. and to 3 significant figures. Since , do not reject . |
χ² GOF Test χ²=5.2 p =0.0742736 df=2 |
A p-value near 0 may be shown in standard form, such as 3.2E-5. Read it as a number: 3.2E-5 is 0.000032, which is smaller than any usual significance level.
How to read a p-value on a Casio fx-CG100 (chi-squared goodness-of-fit test)
Run the test, then compare p with the significance level | |
Press HOME and choose Statistics. In the List Editor, type the observed counts in List 1 and the expected counts in List 2, pressing EXE after each. |
List1 List2 28 20 18 20 14 20 |
Open the Setup tab and choose Hypothesis Test. Highlight line two, press OK and choose χ² Test → χ² Goodness-of-Fit Test. Set Observed to List 1, Expected to List 2 and df to 2. |
χ² Goodness-of-Fit Test Observed:List1 Expected:List2 df:2 |
Press ▶, or highlight Execute and press OK. The Results tab shows p:, χ²: and df:. The value after p: is the p-value. Compare it with the significance level. and to 3 significant figures. Since , do not reject . |
p:0.0742736 χ²:5.2 df:2 |
The same Results tab shows the p-value for the t tests, so the reading is identical for any test: find p, then compare it with the significance level.
How to read a p-value on a TI-Nspire CX (chi-squared goodness-of-fit test)
Run the test, then compare p with the significance level | |
Press home and choose Lists & Spreadsheet. Type the observed counts down column A and the expected counts down column B, pressing enter after each. |
A B 28 20 18 20 14 20 |
Press menu → Statistics → Stat Tests → χ² GOF. In the box, set the observed list to a[], the expected list to b[] and df to 2. Move between the fields with tab, then press enter on OK. |
χ² GOF Observed List:a[] Expected List:b[] Deg of Freedom, df:2 |
The results appear in two columns of names and values. PVal is the p-value: the smallest significance level at which can be rejected. Compare it with the significance level in the question. and to 3 significant figures. Since , do not reject . |
Title χ² GOF χ² 5.2 PVal 0.07427358 df 2 |
PVal is a probability, so it always lies between 0 and 1. If you read a value greater than 1, you have read the test statistic, not the p-value.
Sources. Key presses follow the manufacturers' documentation: the TI-84 Plus CE eGuide, the TI-Nspire eGuide, the Casio fx-CG50 Software User's Guide (version 3.30) and the Casio fx-CG100 online manual.
What to write in the exam
- State the hypotheses first. Write and in words that fit the situation, for example : the flavours are equally popular.
- Write the p-value and the significance level side by side. For example . The comparison is the step that earns the mark, so do not leave it implied.
- Give the decision, then the conclusion in context. Say reject or do not reject , then say what that means for the shop owner's claim. A bare reject or do not reject is not enough.
- Use careful wording. Say there is sufficient or insufficient evidence at the 5% level. Do not say is proved true or that is proved.
- Check which papers allow a calculator. In AI SL and AI HL a calculator is allowed on every paper (check your syllabus).
Reading the p-value
- What it measures. The p-value is the probability of getting a result at least as extreme as the one observed, assuming is true. It is not the probability that is true.
- Small p means unlikely under . The smaller p is, the stronger the evidence against .
- Two-tailed tests. A calculator that offers a two-sided alternative () already includes both tails in p. Compare that p directly with ; do not double it or halve it again.
- The test. The p-value is the area to the right of the statistic, so a large gives a small p.
| Comparison | Decision | Conclusion |
|---|---|---|
| Reject | Sufficient evidence at the level to support | |
| Do not reject | Insufficient evidence at the level to support |
The significance level is set before the test, usually 5% or 1%. It is the probability of rejecting when is true. The guides to the test, the t-test for a mean and the two-sample t-test show where each calculator puts p.
Type I and Type II errors
- Type I error: rejecting when is actually true. Its probability is the significance level .
- Type II error: not rejecting when is actually false.
- In the example, a Type I error would mean concluding that the flavours are not equally popular when they are. A Type II error would mean failing to notice that they are not equally popular.
- A lower significance level makes a Type I error less likely and a Type II error more likely.
Calculating the probability of a Type II error is examined in AI HL. It needs the distribution of the test statistic under the true value, which you find with the normal or binomial commands, as in normal distribution probabilities.
Error messages and what to do
| Calculator | Message | Usual cause | What to try |
|---|---|---|---|
| TI-84 Plus CE | ERR:DIMENSION MISMATCH | The observed list and the expected list have different numbers of values. | Check that L1 and L2 each have one value for every category. |
| Casio fx-CG50 | Dimension ERROR | The observed list and the expected list have different numbers of values. | Check that List 1 and List 2 each have one value for every category. |
| Casio fx-CG100 | Dimension ERROR | The observed list and the expected list have different numbers of values. | Make the number of values in List 1 and List 2 the same. |
| TI-Nspire CX | Dimension mismatch | The observed column and the expected column have different numbers of values. | Check that columns A and B each have one value for every category. |
Common mistakes
- Reading the test statistic as the p-value. A p-value is between 0 and 1. The values labelled , t or z are the test statistics.
- Reversing the comparison. Small p rejects . Write or in full so the direction is clear.
- Saying is true. If p is large, you do not reject . That is not proof that is true, only that the data give insufficient evidence against it.
- A conclusion with no context. Finish with a sentence about the real situation, not only reject or do not reject.
- Doubling p twice. For a two-tailed test the calculator's p already covers both tails, so compare it directly with .
Try it yourself
Real questions from the IB Maths Help question bank on Type I and Type II errors and the chi-squared test. Type your answer and press Mark it: the worked solution and mark scheme then appear.
Practise with your calculator
Choose your course to see the practice sets that use this skill. You can also try practice mode or browse every GDC guide.
Quick questions
What is a p-value in simple terms?
It is the probability of getting a result at least as extreme as yours if the null hypothesis is true. A small p-value means your result would be unlikely under the null hypothesis, which is evidence against it.
How do I use the p-value to decide?
Compare it with the significance level. If p is less than the significance level, reject the null hypothesis. If p is greater than or equal to the significance level, do not reject it. Then state the conclusion in the context of the question.
Does the calculator tell me whether to reject the null hypothesis?
No. It shows the test statistic, the p-value and the degrees of freedom. You make the comparison with the significance level and write the decision.
What is the difference between a Type I and a Type II error?
A Type I error is rejecting the null hypothesis when it is true, and its probability is the significance level. A Type II error is failing to reject the null hypothesis when it is false.
Does a large p-value prove the null hypothesis is true?
No. It means the data give insufficient evidence against the null hypothesis. Write do not reject, and avoid saying the null hypothesis is accepted as true.