Transition matrices and Markov chains AI HL

A Markov chain moves between a fixed set of states with probabilities that depend only on the current state, and a transition matrix stores those probabilities with one column for each starting state. Multiplying the matrix by a column of probabilities gives the next state, so the state after any number of steps comes from a matrix power on the calculator, and the long-term behaviour comes from a steady-state vector that the chain no longer changes. The marks are for reading the matrix the right way round, setting up the product in the right order and reading the result. Work through the two sections in order, because the second uses the first.

Formulas at a glance

The booklet gives the state after nn transitions. The meaning of an entry of the matrix, the rule that each column sums to 11, the steady-state equation and the large-power method are not in it, so learn them.

State after nn transitions

sn=Tns0s_n=T^ns_0

In the booklet

Meaning of an entry of the transition matrix

TijT_{ij} is the probability of moving from state jj to state ii

Not in the booklet

A valid transition matrix

Each column of TT adds to 11

Not in the booklet

Steady-state distribution

Tπ=πT\pi=\pi, with the entries of π\pi adding to 11

Not in the booklet

Steady state by repeated multiplication

For a large power nn, every column of TnT^n is close to π\pi

Not in the booklet

Which method to use

Decide first whether the question is about a particular number of steps or about the long run. A stated number of steps needs a matrix power; the long run needs the steady state.

The question hasUseNotes
Probabilities of moving between states, given in words or as a diagramA transition matrix TT with the starting state as the column and the new state as the rowEach column adds to 11, so a missing entry is 11 minus the others in its column
The state, or the share of each state, after a stated number of stepssn=Tns0s_n=T^ns_0 in matrix modeRaise the matrix to the power first, then multiply by the starting column on the right
The probability of one particular state after nn stepsThe entry for that state in sns_nWhen the chain starts in one state, s0s_0 has a single 11 and the rest zero
The long-term probability of each stateThe steady state π\pi, from Tπ=πT\pi=\pi with the entries adding to 11, or from a large power of TTGive decimals unless the question asks for exact values
A check that a vector is the steady stateMultiply it by TT: it must come back unchanged, with its entries adding to 11The steady state is the eigenvector of TT for the eigenvalue 11
A number of items, such as bikes or customers, rather than probabilitiesMultiply the probability by the total numberRound to a whole number only at the end

Section 1 of 2 AHL4.19

Markov chains

A Markov chain moves between states with fixed probabilities that depend only on the current state, and these probabilities are stored in a transition matrix TT whose columns each sum to 11. The state after nn steps is sn=Tns0s_n=T^ns_0. Use it to predict how a system, such as the weather, changes step by step.

Key terms

What is an initial state vector?

The initial state s0s_0 is a column of the starting probabilities of each state, and its entries sum to 11. If the system definitely starts in one state, that entry is 11 and the others are 00.

e.g. Starting in state 2 of three states gives s0=(010)s_0=\begin{pmatrix}0\\1\\0\end{pmatrix}.

How do you read an entry of a transition matrix?

Entry TijT_{ij} is the probability of moving from state jj to state ii. The column is where you start and the row is where you end.

e.g. In (0.50.30.50.7)\begin{pmatrix}0.5&0.3\\0.5&0.7\end{pmatrix}, the entry T12=0.3T_{12}=0.3 is the probability of moving from state 2 to state 1.

What is a transition diagram?

A transition diagram shows the states as points, with arrows labelled by the probabilities of each move. The probabilities on the arrows leaving each state sum to 11.

e.g. If state A has only two arrows leaving it, one to B labelled 0.30.3 and a loop back to A, the loop is labelled 0.70.7.

How do you find a missing entry in a transition matrix?

Each column sums to 11, so subtract the other entries in that column from 11.

e.g. In the column (0.5p0.2)\begin{pmatrix}0.5\\p\\0.2\end{pmatrix}, p=1−0.5−0.2=0.3p=1-0.5-0.2=0.3.

Worked example, step by step

The question Calculator needed (matrix mode) [5 marks]

A bike-share scheme has 200200 bikes and three docking stations: Harbour (H), Station (S) and University (U). Each evening every bike is at one of the three stations. From one evening to the next, the probabilities of a bike moving between stations are given by the transition matrix T=(0.60.30.20.30.50.30.10.20.5),T=\begin{pmatrix}0.6&0.3&0.2\\ 0.3&0.5&0.3\\ 0.1&0.2&0.5\end{pmatrix}, where the columns are the station a bike is at (in the order H, S, U) and the rows are the station it is at the next evening (in the same order). On Sunday evening, 40%40\% of the bikes are at H, 25%25\% at S and 35%35\% at U.

(a) Write down the probability that a bike at S on one evening is at U the next evening.

(b) Find the proportion of the bikes at each station on Tuesday evening.

(c) Find the number of bikes expected at U on Tuesday evening, to the nearest whole number.

Worked solution: worked example

Answer: (a) 0.20.2 (b) 0.3890.389 at H, 0.3700.370 at S, 0.2410.241 at U (c) 4848 bikes

Step 1 - Read the entry.

The columns give the starting station and the rows the new station. S is the second column and U is the third row, so T32=0.2.T_{32}=0.2. A1 - 0.20.2

Why? The entry TijT_{ij} is the probability of moving from state jj to state ii. Reading along the row instead of down the column would give the probability of arriving at S from U, which is 0.30.3.

Step 2 - Set up the calculation.

The starting state is s0=(0.400.250.35)s_0=\begin{pmatrix}0.40\\0.25\\0.35\end{pmatrix}, and Tuesday is two evenings after Sunday, so s2=T2s0.s_2=T^2s_0. M1 - Use s2=T2s0s_2=T^2s_0 with the starting column on the right

Why? The shares add to 11, as they must. The matrix acts on the column from the left, and the power is taken before the multiplication.

Step 3 - Enter the matrices.

In matrix mode, store TT as a 3×33\times3 matrix and s0s_0 as a 3×13\times1 matrix, then enter T2×s0T^2\times s_0. The calculator shows s2=(0.3890.3700.241).s_2=\begin{pmatrix}0.389\\0.370\\0.241\end{pmatrix}. A1 - s2=(0.389, 0.370, 0.241)s_2=(0.389,\ 0.370,\ 0.241)

Why? Check that the three entries add to 11: 0.389+0.370+0.241=10.389+0.370+0.241=1. If they do not, a column of TT has probably been typed in as a row.

Step 4 - Convert to a number of bikes.

The share at U is 0.2410.241, so the expected number at U is 200×0.241=48.2.200\times0.241=48.2. M1 - Multiply the share at U by 200200

Why? A proportion of the whole fleet is the proportion times the total. The answer is not a whole number yet, so the rounding comes in the next step.

Step 5 - Give a whole number.

Bikes are counted in whole numbers, so about 4848 bikes are expected at U. A1 - 4848 bikes

Why? The answer is rounded only at the end, after the multiplication.

Marks (5):
A1 0.20.2
M1 Use s2=T2s0s_2=T^2s_0 with the starting column on the right
A1 s2=(0.389, 0.370, 0.241)s_2=(0.389,\ 0.370,\ 0.241)
M1 Multiply the share at U by 200200
A1 4848 bikes

Now you try: enter your answers to be marked

These are real practice questions. Type your answer in each box and press Mark it. The worked solution and mark scheme appear straight away, and your first attempt is the one that counts. Use the keyboard for powers, roots and trig, and the working box for your notes.

1
Medium
Calculator
[2 marks]
A particle moves between states 1, 2, 3 with transition matrix T=(0.50.20.10.30.60.40.20.20.5).T=\begin{pmatrix}0.5&0.2&0.1\\0.3&0.6&0.4\\0.2&0.2&0.5\end{pmatrix}. Starting in state 1, find the probability it is in state 2 after 3 steps.
[2]
Fractions & powers
Other

Worked solution: practice question 1

2
Hard
Calculator
[4 marks]
A Markov chain has three states with T=(0.60.10.30.20.80.10.20.10.6)T = \begin{pmatrix}0.6 & 0.1 & 0.3 \\ 0.2 & 0.8 & 0.1 \\ 0.2 & 0.1 & 0.6\end{pmatrix} and starts in state 1: s0=(100)s_0 = \begin{pmatrix}1\\0\\0\end{pmatrix}.

Find s2.s_2.
[4]
Fractions & powers
Other

Worked solution: practice question 2

3
Medium
Calculator
[3 marks]
Brands A and B have transition matrix T=(0.90.20.10.8).T=\begin{pmatrix}0.9&0.2\\0.1&0.8\end{pmatrix}. Initially 60% use A and 40% use B.

Find each share after 2 periods.
[3]
Fractions & powers
Other

Worked solution: practice question 3

Your score so far: 0 out of 9 marks

Common mistakes

  • Mixing up rows and columns. TijT_{ij} is the probability of moving from state jj (the column) to state ii (the row) - reading a transition matrix the wrong way round reverses every probability in the model.
  • Building rows that sum to 1 instead of columns. With the IB's convention, it's each column of TT that must sum to 1, not each row - a matrix built the other way round will look plausible but give the wrong answer at every step.
  • Multiplying in the wrong order. sn=Tns0s_n = T^n s_0 means TT acts on the state vector, so it's Ts0Ts_0, not s0Ts_0T - matrix multiplication isn't commutative, so reversing the order gives nonsense (or an error) on the calculator.

Try some questions on markov chains

Section 2 of 2 AHL4.19

Steady state and equilibrium

A steady state is a probability vector π\pi that is unchanged by a transition, so Tπ=πT\pi=\pi, and whose entries sum to 11. To find it, solve the equations from Tπ=πT\pi=\pi together with the sum condition, or raise TT to a high power. It gives the long-term probability of being in each state.

Key terms

What is a regular Markov chain?

A regular chain has some power of its transition matrix with every entry positive. A regular chain settles to a single steady state whatever the starting state.

e.g. T=(0.50.30.50.7)T=\begin{pmatrix}0.5&0.3\\0.5&0.7\end{pmatrix} is regular because every entry is already positive.

What does the long-term probability mean?

It is the probability of being in each state after many transitions. For a regular chain it does not depend on the initial state.

e.g. A steady state of (0.70.3)\begin{pmatrix}0.7\\0.3\end{pmatrix} means that in the long run the system is in the first state 70%70\% of the time.

Why is the eigenvalue 1 involved?

The equation Tπ=πT\pi=\pi says that π\pi is an eigenvector of TT with eigenvalue 11. Find that eigenvector, then scale it so its entries sum to 11.

e.g. An eigenvector (35)\begin{pmatrix}3\\5\end{pmatrix} has entries summing to 88, so the steady state is (3858)\begin{pmatrix}\dfrac38\\\dfrac58\end{pmatrix}.

How can repeated multiplication show the steady state?

Raising TT to a large power makes every column of TnT^n approach the steady state. Multiplying by any initial state with entries summing to 11 then gives that same vector.

e.g. If every column of TnT^{n} is close to (0.3750.625)\begin{pmatrix}0.375\\0.625\end{pmatrix}, the steady state is approximately (0.3750.625)\begin{pmatrix}0.375\\0.625\end{pmatrix}.

Worked example, step by step

The question Calculator needed (matrix mode) [5 marks]

At the start of each day a machine is found to be working (W), under maintenance (M) or broken (B). The transition matrix for one day to the next is T=(0.850.700.500.100.200.300.050.100.20),T=\begin{pmatrix}0.85&0.70&0.50\\ 0.10&0.20&0.30\\ 0.05&0.10&0.20\end{pmatrix}, where the columns are today's state and the rows are tomorrow's state, both in the order W, M, B.

(a) Find the steady-state distribution, giving each probability to 3 significant figures.

(b) The machine is checked on 250250 days in a year. Find the number of those days on which it is expected to be broken, to 3 significant figures.

Worked solution: worked example

Answer: (a) π=(0.808, 0.126, 0.0662)\pi=(0.808,\ 0.126,\ 0.0662) (b) 16.616.6 days

Step 1 - Choose the method.

The steady state π\pi is the vector that TT leaves unchanged, Tπ=πT\pi=\pi, with entries adding to 11. Every entry of TT is positive, so the chain is regular and every column of TnT^n settles to π\pi. Use a large power of TT, such as T50T^{50}. M1 - Use a large power of TT, or solve Tπ=πT\pi=\pi with the entries adding to 11

Why? A regular chain reaches the same steady state from every starting state, so the columns of a high power are all the same. Solving Tπ=πT\pi=\pi with the sum condition gives the same vector.

Step 2 - Calculate the power.

In matrix mode, enter T50T^{50}. Every column shows 0.8080.808, 0.1260.126 and 0.06620.0662. So the probabilities of working and of maintenance are 0.8080.808 and 0.1260.126. A1 - 0.8080.808 and 0.1260.126

Why? The columns agree to more than three significant figures, so the power is large enough. A larger power would not change the answer.

Step 3 - Read the third entry and check.

The probability of being broken is 0.06620.0662. The three unrounded entries add to 11, and as a check TπT\pi on the calculator returns the same vector. A1 - 0.06620.0662, with the three entries adding to 11

Why? The entries of a probability vector add to 11; the rounded values give 1.00021.0002, which is 11 to the accuracy used. The vector is also an eigenvector of TT for the eigenvalue 11, which is why it is unchanged by TT.

Step 4 - Find the expected number of days.

The expected number of days broken is 250×0.066225…=16.556…,250\times0.066225\ldots=16.556\ldots, using the unrounded value of the probability from the calculator. M1 - Multiply the probability of being broken by 250250

Why? In the long run the machine is broken on a fraction 0.0662…0.0662\ldots of days, so the expected number of days is that fraction of 250250. Using the unrounded value keeps the third significant figure safe.

Step 5 - Round.

expected days broken=16.6.\text{expected days broken}=16.6. A1 - 16.616.6 days

Why? The question asks for 3 significant figures. An expected number need not be a whole number.

Marks (5):
M1 Use a large power of TT, or solve Tπ=πT\pi=\pi with the entries adding to 11
A1 0.8080.808 and 0.1260.126
A1 0.06620.0662, with the three entries adding to 11
M1 Multiply the probability of being broken by 250250
A1 16.616.6 days

Now you try: enter your answers to be marked

1
Medium
Calculator
[2 marks]
Find the steady-state distribution for T=(0.80.40.20.6).T=\begin{pmatrix}0.8&0.4\\0.2&0.6\end{pmatrix}.
[2]
Fractions & powers
Other

Worked solution: practice question 1

2
Hard
Calculator
[3 marks]
Use technology to find the steady-state distribution of T=(0.50.20.30.30.60.30.20.20.4).T=\begin{pmatrix}0.5&0.2&0.3\\0.3&0.6&0.3\\0.2&0.2&0.4\end{pmatrix}.
[3]
Fractions & powers
Other

Worked solution: practice question 2

3
Easy
Calculator
[2 marks]
Customers switch between brands X and Y. The transition matrix is T=(0.70.30.30.7)T = \begin{pmatrix}0.7 & 0.3 \\ 0.3 & 0.7\end{pmatrix} (columns = current brand).

Find the steady-state distribution.
[2]
Fractions & powers
Other

Worked solution: practice question 3

Your score so far: 0 out of 7 marks

Common mistakes

  • Forgetting the normalisation condition. The equation Tπ=πT\pi=\pi alone only fixes the ratio between the state probabilities - you still need ∑πi=1\sum \pi_i = 1 to pin down exact values, not just a proportional relationship.
  • Rounding the steady state before scaling it. When a long-term probability is multiplied by a total, such as a number of days or customers, use the unrounded value from the calculator. Rounding 0.06620.0662 first and multiplying by a large total can change the third significant figure.
  • Reading the steady state from a power that has not settled. Raise TT to a power large enough that the columns agree to the accuracy you need, such as T50T^{50}. Examination questions state when an exact answer is required: then solve Tπ=πT\pi=\pi with the entries adding to 11 and keep fractions.

Try some questions on steady state and equilibrium

Using your GDC

The exact key presses for the TI-84 Plus CE, Casio fx-CG50, Casio fx-CG100 and TI-Nspire CX, with what the screen shows. Pick your calculator.

Matrix powers (transition / Markov chains)

Used in: Markov chains, Steady state and equilibrium

How to raise a matrix to a power on a TI-84 Plus CE

The state after n steps

Enter the transition matrix as [A], with the same layout as in the question (each column adds to 1), and the starting state as a one-column matrix [B]. For the example, [A] is 2 × 2 with rows 0.7 0.4 and 0.3 0.6, and [B] is 2 × 1 holding 200 and 100. The steps for the matrix editor are in matrix operations and solving systems.

Press 2nd mode (QUIT) to return to the home screen. Press 2nd x⁻¹ (MATRIX) and choose 1:[A] from the NAMES list. The name must be pasted from this list, because it cannot be typed. Then press ^ 4 ×, press 2nd x⁻¹ (MATRIX) again and choose 2:[B].

[A]^4*[B]

Press enter. The answer is a column holding the number in each state after 4 steps.

After 4 weeks there are 171.66171.66 customers of P and 128.34128.34 of Q.

[A]^4*[B]
[[171.66]
[128.34]]
The long-term state

Use the same entry with a large power. Press 2nd enter (ENTRY) to bring the last line back, change the 4 to 50, and press enter.

The numbers settle at 171.4285714171.4285714 and 128.5714286128.5714286, which is 171171 and 129129 to 3 significant figures.

[A]^50*[B]
[[171.4285714]
[128.5714286]]

To see the long-term proportions, enter [A]^50 on its own: every column is the same, and each column is the steady state. The power can be any whole number from 0 to 255.

Check the answer by adding the two numbers: they should still total the starting number, here 300. If they do not, the columns of the matrix probably do not add to 1.

The full guide: Matrix powers (transition / Markov chains)

Matrix operations and solving systems

Used in: Markov chains

How to use matrices on a TI-84 Plus CE

Enter a matrix

Press 2nd x⁻¹ (MATRIX), press ▶ twice to reach EDIT, choose 1:[A] and press enter.

Type the size, 2 enter 2 enter for 2 rows and 2 columns, then type the numbers row by row, pressing enter after each one. Use (−) for a negative number.

MATRIX[A] 2 ×2
[[2  1]
[1  3]]

Enter the second matrix the same way in 2:[B] and the right-hand side as a 2 by 1 matrix in 3:[C]. Press 2nd mode (QUIT) when you have finished.

MATRIX[C] 2 ×1
[[5]
[10]]
Add, multiply, determinant and inverse

On the home screen press 2nd x⁻¹ (MATRIX), choose 1:[A] from the NAMES list and press enter. Then press +, paste [B] in the same way, and press enter. A matrix name can only be pasted from this menu, not typed. Use × instead of + to multiply.

A+B=(3347)A + B = \begin{pmatrix}3 & 3 \\ 4 & 7\end{pmatrix} and AB=(581014)AB = \begin{pmatrix}5 & 8 \\ 10 & 14\end{pmatrix}.

[A]+[B]
[[3  3]
[4  7]]
[A]*[B]
[[5  8]
[10  14]]

For the determinant, press 2nd x⁻¹ (MATRIX), press ▶ to reach MATH, choose 1:det(, paste [A] from NAMES, type ) and press enter.

detA=5\det A = 5.

det([A])
5

For the inverse, paste [A], press x⁻¹, then enter. The answer is in decimals. To see fractions, press math, choose 1:▶Frac and press enter.

A−1=15(3−1−12)A^{-1} = \dfrac{1}{5}\begin{pmatrix}3 & -1 \\ -1 & 2\end{pmatrix}, which is (0.6−0.2−0.20.4)\begin{pmatrix}0.6 & -0.2 \\ -0.2 & 0.4\end{pmatrix} in decimal form.

[A]-1
[[0.6  −0.2]
[−0.2  0.4]]
Ans▶Frac
[[3/5  −1/5]
[−1/5  2/5]]
Solve a system with the inverse

To solve Ax=bA\mathbf{x} = \mathbf{b}, paste [A], press x⁻¹, then ×, paste [C] and press enter. The inverse must come first: A−1bA^{-1}\mathbf{b} is not the same as bA−1\mathbf{b}A^{-1}.

x=1x = 1 and y=3y = 3.

[A]-1*[C]
[[1]
[3]]

Paste matrix names from the MATRIX menu rather than typing them, and put the inverse on the left when you solve a system.

The full guide: Matrix operations and solving systems

Eigenvalues and eigenvectors

Used in: Steady state and equilibrium

How to find eigenvalues and eigenvectors on a TI-84 Plus CE

No eigenvalue command: characteristic equation and rref

Press 2nd x⁻¹ (MATRIX), press ▶ to the EDIT menu and choose 1:[A]. Enter the size, 2 enter 2 enter, then the entries row by row, pressing enter after each: 6, 2, 1, 5. Press 2nd mode (QUIT) to leave the editor.

MATRIX[A] 2 ×2
[ 6 2 ]
[ 1 5 ]

The manual lists no eigenvalue or eigenvector command for this calculator, so the eigenvalues come from the characteristic equation. For a 2×22\times2 matrix it is λ2−(trace)λ+det=0\lambda^2 - (\text{trace})\lambda + \det = 0. Here the trace is 6+5=116+5 = 11 and the determinant is 6×5−2×1=286\times5 - 2\times1 = 28, so it is λ2−11λ+28=0\lambda^2 - 11\lambda + 28 = 0.

Press apps and choose PlySmlt2, press any key, then choose 1:Polynomial Root Finder. Set ORDER to 2, press graph under NEXT, and enter the coefficients 1 enter, (−) 1 1 enter, 2 8 enter. Press graph under SOLVE.

The eigenvalues are λ=4\lambda = 4 and λ=7\lambda = 7.

X1=4
X2=7

For an eigenvector, row-reduce M−λIM - \lambda I. Quit the app, then on the home screen press 2nd x⁻¹ (MATRIX), ▶ to MATH, and choose B:rref(. Type [A] from 2nd x⁻¹ (MATRIX) 1:[A], then − 7 ×, and choose 5:identity( from the same MATH menu, followed by 2 ) ) and enter. This is rref([A]-7*identity(2)) for λ=7\lambda = 7.

Each row of the result is an equation in xx and yy. (1−200)\begin{pmatrix}1 & -2\\0 & 0\end{pmatrix} means x−2y=0x - 2y = 0, so x=2yx = 2y and an eigenvector for λ=7\lambda = 7 is (21)\begin{pmatrix}2\\1\end{pmatrix}. Repeat with 44 in place of 77: the result (1100)\begin{pmatrix}1 & 1\\0 & 0\end{pmatrix} means x+y=0x + y = 0, so an eigenvector is (1−1)\begin{pmatrix}1\\-1\end{pmatrix}.

rref([A]-7*identity(2))
[[1 -2]
[0 0]]

Any non-zero multiple of an eigenvector is also an eigenvector, so (2, 1) and (4, 2) are equally correct. The identity size must match the matrix: identity(2) for a 2 by 2.

The full guide: Eigenvalues and eigenvectors

See the full list of GDC guides for every calculator task.

Syllabus content covered

CodeSyllabus content
AHL4.19Transition matrices and powers of transition matrices. Use of transition diagrams to represent transitions in discrete dynamical systems. Regular Markov chains and initial state probability matrices. In general, the column state matrix sns_n after nn transitions is given by sn=Tns0s_n = T^n s_0, where TT is the transition matrix (with TijT_{ij} the probability of moving from state jj to state ii) and s0s_0 is the initial state matrix. Calculation of steady state and long-term probabilities by repeated multiplication of the transition matrix, or by solving a system of linear equations - with awareness that the solution is the eigenvector corresponding to the eigenvalue equal to 11.

Official IB source: Mathematics: applications and interpretation subject brief (PDF, ibo.org).

Quick questions

What is a transition matrix in a Markov chain?

A square matrix TT where entry TijT_{ij} is the probability of moving from state jj (the column) to state ii (the row) in one step - every column must sum to 1.

What is the steady-state distribution of a Markov chain?

The state vector π\pi that satisfies Tπ=πT\pi=\pi and sums to 1 - once the system reaches it, further transitions leave the distribution unchanged.

How do you find the state after n transitions?

sn=Tns0s_n = T^n s_0, where s0s_0 is the initial state (probability) vector - raise TT to the power nn and multiply by s0s_0, usually done in the calculator's matrix mode.

How do you find a steady-state distribution by hand?

Set Tπ=πT\pi=\pi, write the resulting equations, add the normalisation condition that the entries of π\pi sum to 1, then solve the system - or raise TT to a high power on the calculator and read off the converged columns.

What does the Markov property mean?

The probability of the next state depends only on the current state, not on any of the earlier states.

Does the steady state depend on the starting state?

For a regular chain it does not. Any initial state leads to the same long-term probabilities, although the number of steps needed to get close may differ.

How do I find the probability of being in a state after 2 steps?

Calculate s2=T2s0s_2=T^2s_0 and read the entry for that state. You can also find s1=Ts0s_1=Ts_0 first and then s2=Ts1s_2=Ts_1.

How do I solve for the steady state of a 2 by 2 matrix by hand?

Write π=(p1−p)\pi=\begin{pmatrix}p\\1-p\end{pmatrix}, form Tπ=πT\pi=\pi, and solve one of the equations for pp. Using the sum condition leaves only one unknown.

Can the rows of the transition matrix sum to 1 instead?

In the IB convention the columns sum to 11 and the state is a column, so sn=Tns0s_n=T^ns_0. Mixing this with a matrix whose rows sum to 11 changes the order of multiplication, so keep to one convention.

How can I check my steady state is correct?

Multiply TT by your vector: the result must be the same vector, and its entries must sum to 11. Both checks together confirm the answer.

Ready to practise properly?

Practice questions on every section, marked instantly like the real exam.