Markov Chains (AI HL)
A Markov chain models a system that moves between a fixed set of states, where the probability of the next move depends only on the current state - not on how it got there. This page covers setting up the transition matrix \(T\) from a described situation and using it to project probabilities forward several steps, with worked examples and the mistakes that most often cost marks. It's part of the broader Transition Matrices & Markov Chains topic.
16 questions on this sub-topic.
Building and applying the transition matrix
Covered under IB syllabus reference AHL4.19: transition matrices and powers of transition matrices, with transition diagrams used to represent transitions in discrete dynamical systems.
State vector after \(n\) transitions
\(s_n = T^n s_0\)
In the formula booklet. \(T\) is the transition matrix and \(s_0\) is the initial state (probability) vector - raise \(T\) to the power \(n\) on the GDC's matrix menu rather than multiplying it out by hand.
Columns = "from" state
\(T_{ij}\): from \(j\) to \(i\)
Reading down a column shows where a system currently in that state can go next - and every column must sum to 1, since the system has to be somewhere after the transition.
Need the full syllabus wording and formula-booklet reference table? See Transition Matrices & Markov Chains. For calculator-specific steps, see the parent topic's GDC guidance.
Worked examples
Each day is sunny (S) or rainy (R). If today is sunny, tomorrow is sunny with probability 0.8; if today is rainy, tomorrow is sunny with probability 0.4.
Write the transition matrix \(T\) with columns representing today's state.
Worked solution
Columns are 'from', rows are 'to'. M1
\[T=\begin{pmatrix}0.8&0.4\\0.2&0.6\end{pmatrix}.\] A1 A1 A1
A particle moves between states 1, 2, 3 with transition matrix \(T=\begin{pmatrix}0.5&0.2&0.1\\0.3&0.6&0.4\\0.2&0.2&0.5\end{pmatrix}.\) Starting in state 1, find the probability it is in state 2 after 3 steps.
Worked solution
\(s_0=\begin{pmatrix}1\\0\\0\end{pmatrix}\), compute \(T^3 s_0.\) M1
State-2 probability \(\approx0.449.\) A1
Common mistakes
- Mixing up rows and columns. \(T_{ij}\) is the probability of moving from state \(j\) (the column) to state \(i\) (the row) - reading a transition matrix the wrong way round reverses every probability in the model.
- Building rows that sum to 1 instead of columns. With the IB's convention, it's each column of \(T\) that must sum to 1, not each row - a matrix built the other way round will look plausible but give the wrong answer at every step.
- Multiplying in the wrong order. \(s_n = T^n s_0\) means \(T\) acts on the state vector, so it's \(Ts_0\), not \(s_0T\) - matrix multiplication isn't commutative, so reversing the order gives nonsense (or an error) on the GDC.
Ready to practise properly?
16 Markov chain questions, marked instantly like the real exam.
Quick answers
What is a transition matrix in a Markov chain?
A square matrix \(T\) where entry \(T_{ij}\) is the probability of moving from state \(j\) (the column) to state \(i\) (the row) in one step - every column must sum to 1.
How do you find the state after n transitions?
\(s_n = T^n s_0\), where \(s_0\) is the initial state (probability) vector - raise \(T\) to the power \(n\) and multiply by \(s_0\), usually done on the GDC's matrix menu.