Errors in hypothesis tests and confidence intervals AI HL
A hypothesis test can go wrong in two ways: rejecting a null hypothesis that is true (a Type I error) or keeping one that is false (a Type II error), and you must be able to describe each in context and find its probability. A confidence interval gives a range of plausible values for a mean, found with the calculator from raw data or from summary values, using the normal distribution when the standard deviation is known and the t-distribution when it is not. The first section deals with the errors and the second with intervals, and each has a worked example before the questions.
Formulas at a glance
The formula booklet has no result for either topic. The error probabilities, the decision rule and the two interval formulas are not in it, so learn them; the calculator builds the interval for you.
Probability of a Type I error
, the significance level
Not in the bookletProbability of a Type II error, and power
, power
Not in the bookletDecision rule with a p-value
Reject if
Not in the bookletMean of a sample, normal data with known standard deviation
Not in the bookletConfidence interval for the mean, standard deviation known
Not in the bookletConfidence interval for the mean, standard deviation unknown
, with degrees of freedom
Not in the bookletWhich method to use
Decide first whether the question is about a decision and its errors or about estimating a mean. For an error question, say which hypothesis is true in reality before choosing the distribution to use.
| The question has | Use | Notes |
|---|---|---|
| A description of a Type I or a Type II error | Type I: reject when it is true. Type II: do not reject when it is false | Write it in the words of the question, not only with the symbol |
| A p-value and a significance level | Reject if , otherwise do not reject | The probability of a Type I error is , so a larger level makes it more likely |
| The probability of a Type I error for a stated critical region | Assume is true and find the probability that the test statistic falls in the critical region | For a discrete test this can be less than the stated significance level |
| The probability of a Type II error when the true mean is given | Use the distribution with the true mean and find the probability of landing outside the critical region | Power is ; a larger sample lowers |
| A confidence interval, and the standard deviation is given | The -interval , or the calculator's Z interval | Use the standard deviation given, whatever the size of the sample |
| A confidence interval, and is not given, so comes from the data | The calculator's T interval with the data list, or with , and | Use a interval whatever the size of the sample |
| An interval given, and the mean or the margin of error wanted | The mean is the midpoint and the margin of error is half the width | The two ends are symmetrical about |
| A request to say what an interval means, or to compare two levels | A sentence in context, and a higher level gives a wider interval | About 95% of intervals built this way contain the true mean; it does not say where individual values lie |
Section 1 of 2 AHL4.18
Type I and Type II errors
A Type I error is rejecting when it is true, and a Type II error is failing to reject when it is false. The probability of a Type I error is the significance level , and the probability of a Type II error is written . Questions ask you to describe these errors in context and to say how changing the test affects them. To find the probability of a Type I error, assume is true and find the probability that the test statistic falls in the critical region. To find the probability of a Type II error, use the true value of the parameter instead, and find the probability that the test statistic falls outside the critical region. Given a p-value, reject if and do not reject it otherwise. For a discrete variable, such as a binomial or Poisson count, the critical region is the largest one whose probability under is less than the stated significance level, so the probability of a Type I error can be less than .
Key terms
What is the significance level?
The significance level is the probability of rejecting when it is true. It is chosen before the test and gives the probability of a Type I error.
e.g. A test at the 2% level has , so the probability of rejecting a true is .
How do you describe an error in context?
Say what was concluded and what was actually true, using the words of the question rather than only the symbol .
e.g. For , a Type I error is concluding that the mean is not when it is in fact .
What is the critical region?
The critical region is the set of sample results for which is rejected. It is chosen so that, when is true, the probability of landing in it is no more than .
e.g. At the 2% level, the critical region has a probability of at most when is true.
What is the power of a test?
The power is , the probability of correctly rejecting when it is false. A higher power means the test is better at detecting a real difference.
e.g. If , the power is .
Worked example, step by step
The question Calculator needed [5 marks]
The lengths of metal rods made by a machine are normally distributed with standard deviation 0.6 cm. The mean length is claimed to be 20 cm. A manager suspects that the mean has increased, and tests against , where cm is the mean length. A random sample of 9 rods is taken, and is rejected if the sample mean is greater than 20.4 cm.
(a) Find the probability of a Type I error.
(b) The mean length has in fact increased to 20.5 cm. Find the probability of a Type II error.
(c) Explain what a Type II error means in this context.
Worked solution: worked example
Answer: (a) (b) (c) concluding the mean length has not increased when it has
Step 1 - Find the distribution of the sample mean when is true.
A Type I error is rejecting when it is true, so assume . The standard deviation of is , so and the probability needed is . M1 - Use with
Why? The error probability is found assuming the true situation that makes the error possible. For a Type I error that is , which is why the mean is 20.
Step 2 - Use the calculator.
Use the normal cumulative distribution function with lower bound 20.4, upper bound (a very large number), mean 20 and standard deviation 0.2. The calculator shows A1 -
Why? This is the significance level of the test as it is set up. A critical value of 20.4 is two standard deviations above the mean, which gives roughly 2.3% in the upper tail.
Step 3 - Find the distribution when the true mean is 20.5.
A Type II error is not rejecting when it is false, so now and Not rejecting means , so the probability needed is . M1 - Use with
Why? The standard deviation of stays the same, because it depends only on and . Only the centre of the distribution moves.
Step 4 - Use the calculator.
Use the normal cumulative distribution function with lower bound , upper bound 20.4, mean 20.5 and standard deviation 0.2. The calculator shows A1 -
Why? The value 20.4 is half a standard deviation below the true mean, so about 31% of samples fall in the "do not reject" region even though the mean has increased.
Step 5 - Describe the error in context.
A Type II error means concluding that there is no evidence the mean length has increased, when the mean length has in fact increased. R1 - Concluding the mean length has not increased when it has
Why? The answer must say what was concluded and what was actually true, in the words of the question, not only "failing to reject when it is false".
M1 Use with
A1
M1 Use with
A1
R1 Concluding the mean length has not increased when it has
Now you try: enter your answers to be marked
These are real practice questions. Type your answer in each box and press Mark it. The worked solution and mark scheme appear straight away, and your first attempt is the one that counts. Use the keyboard for powers, roots and trig, and the working box for your notes.
Worked solution: practice question 1
Worked solution: practice question 2
Worked solution: practice question 3
Your score so far: 0 out of 7 marks
Calculator steps: Interpret p-values, significance levels and errors, Normal distribution probabilities
Common mistakes
- Defining the errors in the abstract instead of in context. "Rejecting a true null hypothesis" alone often loses the mark - the answer should say what that means for the actual scenario, e.g. "concluding the mean has changed when it hasn't."
- Confusing which error the significance level controls. is set before the data is collected, and for a normal test statistic it is the probability of a Type I error. For a binomial or Poisson count, the probability of a Type I error is the probability of the critical region, which can be less than .
- Assuming lowering always improves the test. A smaller significance level reduces the Type I error rate but increases the Type II error rate (lower power) - the two trade off against each other unless the sample size also increases.
Section 2 of 2 AHL4.16
Confidence intervals
A confidence interval for the mean is a range, calculated from a sample, that is likely to contain the true mean . It has the form , built with the normal distribution when is known and with the -distribution when it is not. The result must be interpreted in context.
Choosing your interval
Choose the interval from what the question gives you, then interpret the result in context.
z-interval vs t-interval
or
These aren't given as algebra in the formula booklet - your calculator builds them directly from raw or summary data, so the real skill is spotting which routine to use. If the question gives the standard deviation , use a -interval; if the standard deviation has to be found from the data, as , use a -interval, however large is.
Confidence level
90% → 95% → 99%
A higher confidence level needs a larger critical value, which widens the interval.
Key terms
What is the margin of error?
The margin of error is the distance from the sample mean to each end of the interval, so the interval is plus or minus this amount.
e.g. An interval has centre and margin of error .
What does 95% confidence mean?
If many samples were taken and an interval built from each, about 95% of those intervals would contain the true mean. It does not describe where individual data values lie.
e.g. Out of intervals built from samples, about would be expected to contain .
How does sample size affect the interval?
A larger sample size reduces the spread of the sample mean, so the interval is narrower and the estimate more precise.
e.g. With fixed, the margin of error is proportional to , so increasing from to halves it.
What is the critical value?
The critical value is the multiplier of (or the equivalent for the -distribution) needed for the chosen confidence level. For a 95% interval using the normal distribution it is about .
e.g. With and , the margin of error is .
Worked example, step by step
The question Calculator needed [4 marks]
The times, in minutes, taken by 8 volunteers to complete a puzzle were
12.4, 15.1, 13.8, 14.6, 12.9, 15.5, 13.2, 14.0.
The times may be assumed to come from a normal distribution with unknown standard deviation.
(a) Find a 95% confidence interval for the mean time to complete the puzzle.
(b) Interpret the interval in context.
Worked solution: worked example
Answer: (a) minutes (b) 95% confident that the mean time lies in this interval
Step 1 - Choose the interval.
The standard deviation of the times is not given, so it is estimated from the data and a -interval is used. With 8 values there are degrees of freedom.
Why? A -interval needs a known standard deviation. When the standard deviation is unknown the -distribution is used whatever the size of the sample.
Step 2 - Enter the data and run the interval.
Enter the 8 times in a list. Choose the -interval routine with the data option, the list just entered and confidence level 0.95. The calculator finds the mean and the standard deviation itself. M1 - Use a -interval with the data and 95% confidence
Why? Entering the raw data avoids rounding the mean and standard deviation before they are used. The calculator uses the divisor for , which is the value a -interval needs.
Step 3 - Read the interval.
The calculator shows the interval from 13.03 to 14.84, so to three significant figures A1 - Lower end A1 - Upper end
Why? The interval is , a margin of error of about 0.907 either side of the mean.
Step 4 - Interpret the interval.
We are 95% confident that the mean time to complete the puzzle lies between 13.0 minutes and 14.8 minutes. If many samples of 8 volunteers were taken and an interval found from each, about 95% of the intervals would contain the true mean time. R1 - 95% confident the mean time lies between 13.0 and 14.8 minutes
Why? The interval is about the mean, not about individual volunteers: it does not say that 95% of volunteers finish within these times.
M1 Use a -interval with the data and 95% confidence
A1 Lower end
A1 Upper end
R1 95% confident the mean time lies between 13.0 and 14.8 minutes
Now you try: enter your answers to be marked
Worked solution: practice question 1
Worked solution: practice question 2
Find a 95% confidence interval for the mean mass.
Worked solution: practice question 3
Your score so far: 0 out of 11 marks
Calculator steps: Confidence interval for a mean (t-interval)
Common mistakes
- Reaching for by default. Unless the question explicitly gives you the standard deviation , you should be using a -interval with , found from the data - it does not matter how large is.
- Assuming a higher confidence level is always "better". A 99% interval is wider, not more accurate - it trades precision for a greater chance of containing the true mean, which is worth stating explicitly if a question asks you to compare levels.
- Losing the units or context in the final answer. A confidence interval is only fully answered once you've stated it in context - "the mean height lies between 171.2 cm and 174.8 cm" - not just as a bare pair of numbers.
Using your GDC
The exact key presses for the TI-84 Plus CE, Casio fx-CG50, Casio fx-CG100 and TI-Nspire CX, with what the screen shows. Pick your calculator.
Interpret p-values, significance levels and errors
Used in: Type I and Type II errors
How to read a p-value on a TI-84 Plus CE (chi-squared goodness-of-fit test)
Run the test, then compare p with the significance level | |
Press stat and choose 1:Edit.... Type the observed counts in L1 and the expected counts in L2, pressing enter after each. |
L1 L2 28 20 18 20 14 20 |
Press stat, press ▶ twice to reach TESTS and choose D:χ²GOF-Test.... With the Stat Wizards on, set Observed to L1 (2nd 1), Expected to L2 (2nd 2) and df to 2, then choose Calculate. |
χ²GOF-Test Observed:L1 Expected:L2 df:2 Calculate |
The screen shows χ², p and df. The value after p= is the p-value. Compare it with the significance level: if p is smaller, reject ; if it is larger, do not. and to 3 significant figures. Since , do not reject . |
χ²GOF-Test χ²=5.2 p=0.0742736 df=2 CNTRB={3.2,0.2,1.8} |
The calculator never says reject or do not reject. It gives only p. You compare p with the significance level yourself and write the conclusion.
The full guide: Interpret p-values, significance levels and errors
Normal distribution probabilities
Used in: Type I and Type II errors
How to find a normal probability on a TI-84 Plus CE
normalcdf (between two bounds) | |
Press 2nd vars (DISTR) and choose 2:normalcdf(. With the Stat Wizards on, a screen asks for each value in turn: type it and press enter each time. Enter the lower bound 140, the upper bound 165, the mean 150 and the standard deviation 12, then press enter on Paste. |
normalcdf( lower:140 upper:165 μ:150 σ:12 |
Press enter to run it. If the Stat Wizards are off, type to 3 significant figures. |
normalcdf(140,165,150,12) 0.6920218 |
For more than 170, there is no upper limit, so use a very large number as the upper bound: type 1 2nd , (EE) 9 9 for 1E99. to 3 significant figures. |
normalcdf(170,1E99,150,12) 0.0477904 |
For less than a value, use −1E99 as the lower bound. If you give only the two bounds, the calculator uses mean 0 and standard deviation 1.
The full guide: Normal distribution probabilities
Confidence interval for a mean (t-interval)
Used in: Confidence intervals
How to find a confidence interval for a mean on a TI-84 Plus CE
TInterval (from a list of data) | |
Press stat and choose 1:Edit. Empty L1 if it holds old numbers (cursor on the name L1, then clear enter), then type the data into L1, pressing enter after each value: 23.1, 25.4, 22.8, 24.9, 26.2, 23.7, 25.0, 24.3, 22.5, 26.1. |
L1 23.1 25.4 22.8 24.9 26.2 23.7 25.0 24.3 22.5 26.1 |
Press stat, then ▶ ▶ to move past CALC to TESTS, and choose 8:TInterval. On the Inpt line choose Data. |
TInterval Inpt:Data Stats List:L1 Freq:1 C-Level:.95 Calculate |
Set List to L1 (2nd 1), leave Freq as 1, and type the confidence level as a decimal: . 9 5 for 95%. Move down to Calculate and press enter. |
|
The first line is the interval as (lower, upper). Below it are the sample mean x̄, the standard deviation Sx and the sample size n. The 95% confidence interval is , each end to 3 significant figures. |
TInterval (23.44321,25.35679) x̄=24.4 Sx=1.337494 n=10 |
TInterval (from summary statistics) | |
Use this when the question gives the mean, the standard deviation s and the sample size, for example a sample of 10 with mean 24 and standard deviation 3. Press stat, ▶ ▶, choose 8:TInterval, and on the Inpt line choose Stats. Enter x̄ as 24, Sx as 3, n as 10 and C-Level as . 9 5, then choose Calculate and press enter. The 95% confidence interval is , each end to 3 significant figures. |
TInterval Inpt:Data Stats x̄:24 Sx:3 n:10 C-Level:.95 Calculate |
ZInterval (σ given) | |
Use a z interval when the question gives the standard deviation σ of the population, for example σ = 4 with a sample of 16 values and mean 62.5. Press stat, ▶ ▶, choose 7:ZInterval, and on the Inpt line choose Stats. Enter σ as 4, x̄ as 62.5, n as 16 and C-Level as . 9 5, then choose Calculate and press enter. The 95% confidence interval is (60.5, 64.5), each end to 3 significant figures. |
ZInterval Inpt:Data Stats σ:4 x̄:62.5 n:16 C-Level:.95 Calculate |
On the Stats screen, Sx must be the n - 1 standard deviation s. The σx from 1-Var Stats is a different, smaller number and gives an interval that is too narrow.
The full guide: Confidence interval for a mean (t-interval)
See the full list of GDC guides for every calculator task.
Syllabus content covered
| Code | Syllabus content |
|---|---|
| AHL4.16 | Confidence intervals for the mean of a normal population. Use of the normal distribution when is known and the -distribution when is unknown, regardless of sample size. Students should be able to interpret the meaning of their results in context. |
| AHL4.18 | Critical values and critical regions. Type I and II errors including calculations of their probabilities. Applied to normal with known variance, Poisson and binomial distributions. For discrete random variables, hypothesis tests and critical regions will only be required for one-tailed tests. The critical region will maximize the probability of a Type I error while keeping it less than the stated significance level. |
Official IB source: Mathematics: applications and interpretation subject brief (PDF, ibo.org).
Quick questions
What is a Type I error?
A Type I error is rejecting when is actually true. For a normal test statistic its probability equals the significance level ; for a binomial or Poisson count it is the probability of the critical region, which can be less than .
When do I use a z-interval instead of a t-interval for the mean?
Use a -interval only when the standard deviation is given directly. Use a -interval whenever the standard deviation has to be found from the data, as , regardless of how large is.
How can the chance of a Type II error be reduced?
Increasing the sample size reduces without changing . Raising the significance level also reduces , but at the cost of a larger chance of a Type I error.
Why does a 99% confidence interval come out wider than a 95% one?
A higher confidence level needs a larger critical value, which widens the margin of error and therefore the interval - you are trading precision for a higher chance of capturing the true mean.
Why can't both errors be made impossible?
A decision based on a random sample always carries some risk. For a fixed sample size, lowering shrinks the critical region, which makes larger.
How do I write a confidence interval as an answer?
Give both end points, as or , to the accuracy asked, with units. Add a sentence in context saying what the interval means.
How do I find the probability of a Type I error for a given critical region?
Find the probability, assuming is true, that the test statistic falls in the critical region. For a discrete test this can be less than the stated significance level.
How do I find the sample mean from a confidence interval?
Add the two end points and halve: . The margin of error is half the width, .
Why is a t-interval wider than a z-interval for similar data?
The t-distribution has heavier tails to allow for the extra uncertainty from estimating the standard deviation. Its critical value is larger, so the interval is wider.
Ready to practise properly?
Practice questions on every section, marked instantly like the real exam.