Markov Cohort Models in Cost-Effectiveness Analysis: States, Transitions, and the Cohort Trace

A Markov cohort model is the workhorse structure behind most published health-economic decision models. Instead of tracking individual patients, it moves a hypothetical cohort — typically 1,000 or 100,000 people, the number is arbitrary and cancels out in the ratios that matter — through a small set of mutually exclusive health states over a series of discrete time steps called cycles. At each cycle, a fixed set of transition probabilities determines what fraction of the cohort moves from one state to another. Costs and quality-adjusted life-years (QALYs) are attached to time spent in each state, summed across the cohort and across cycles, and the running total is what feeds the incremental cost-effectiveness ratio a reimbursement body such as NICE ultimately evaluates.

The approach is standard enough that ISPOR and the Society for Medical Decision Making jointly published a dedicated methodology report on it: Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, Kuntz KM, “State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3,” Value in Health 2012;15(6):812-20 (also published simultaneously in Medical Decision Making 2012;32(5):690-700). That report, alongside the broader seven-part ISPOR-SMDM Modeling Good Research Practices series, is the reference this page follows for structure and terminology.

The core structure: states, cycles, and transition probabilities

Three design choices define a Markov cohort model before any numbers are entered:

• Health states. A finite set of mutually exclusive, collectively exhaustive states that capture the clinically and economically relevant status of a patient — for example, in an oncology model: Progression-Free, Progressed Disease, and Dead. Every member of the cohort occupies exactly one state at any given time.
• Cycle length. The discrete time step the model advances by — a week, a month, a year — chosen to match the clinical rhythm of the disease and the resolution of the available transition-probability data. A cycle length that’s too coarse smooths over meaningful within-cycle events; this is precisely the problem half-cycle correction (below) exists to partially address.
• Transition probabilities. The probability of moving from each state to each other state (including staying put) during one cycle, usually arranged as a transition probability matrix. These are typically derived from trial survival curves, published natural-history data, or registry follow-up, and every row of the matrix must sum to 1.

Running the model means repeatedly multiplying the cohort’s current state-occupancy vector by the transition matrix, cycle after cycle, until either a fixed time horizon is reached or the cohort has effectively emptied into an absorbing state such as Dead.

The Markov assumption: why the model is “memoryless”

The defining — and defining-limiting — property of a Markov process is that the probability of transitioning out of a state depends only on the state currently occupied, not on how the cohort got there or how long it has already spent there. A patient who has been in Progression-Free for one cycle and a patient who has been there for twenty cycles are, under a plain Markov model, assigned exactly the same probability of progressing next cycle. This is the “memoryless” property, and it is what makes the model computationally tractable: the entire future evolution of the cohort is fully determined by its current state distribution, so the model never needs to track individual trajectories or history.

The practical limitation

Memorylessness is a modeling convenience, not a clinical fact, and it becomes a real limitation whenever the true transition risk is time-dependent rather than state-dependent — for instance, when recurrence risk declines the longer a patient has already remained disease-free, or when a treatment’s benefit accrues cumulatively rather than resetting each cycle. Applying a constant, state-only transition probability in those situations misrepresents the underlying process and can bias the resulting cost and QALY estimates in either direction depending on which way the true risk trends.

Tunnel states and other workarounds

The standard fix, described in the ISPOR-SMDM Task Force-3 report, is to expand the state space rather than abandon the Markov framework: tunnel states split a single clinically-defined state into a sequence of time-indexed sub-states (e.g., Progression-Free, cycle 1, Progression-Free, cycle 2, …), each carrying its own transition probability, so that a patient is effectively forced through a “tunnel” that reintroduces elapsed-time dependence without breaking the underlying memoryless mechanics of the model. Other established workarounds include adding entirely separate states to represent distinguishable histories (e.g., separating first-line from second-line treatment as different states rather than one “on treatment” state), or moving to a patient-level microsimulation, which tracks individual histories explicitly and removes the memoryless constraint entirely at the cost of Monte Carlo noise and materially higher computational and reporting complexity. Tunnel states are usually the more proportionate fix when only a handful of transitions are genuinely time-dependent; microsimulation earns its added complexity when time- and history-dependence pervades most of the model.

Building the cohort trace

The cohort trace is the table produced by running the model: one row per cycle, one column per health state, with each cell holding the proportion (or absolute number, out of the starting cohort size) occupying that state at that cycle. It is the single artifact that makes a Markov cohort model auditable — a reviewer can look at the trace and see exactly how the cohort redistributes across states over time, independent of the costs and utilities layered on top.

A simplified, illustrative three-state trace (starting cohort of 1,000, hypothetical transition probabilities, cycle length one year) looks like this:

Costs and QALYs are attached to the trace by multiplying each cycle’s state-occupancy figures by a per-cycle cost and utility weight for that state, then summing across states within a cycle and across cycles for the full time horizon — usually with future costs and QALYs discounted back to present value per the analysis’s chosen discount rate. The Progression-Free column, for example, might carry a utility weight closer to full health while Progressed carries a lower one, so the trace directly determines how many QALYs the model accumulates for the cohort. Utility weights themselves are typically sourced from a validated preference-based instrument such as the SF-36 or a similar health-related quality-of-life measure, mapped onto a utility index.

Half-cycle correction

Because the cohort trace only records occupancy at the start (or end) of each discrete cycle, a plain trace implicitly assumes that every transition into or out of a state happens exactly at the cycle boundary — which overstates or understates time spent in each state depending on convention. Half-cycle correction is the standard adjustment: rather than summing full-cycle state occupancies, the model assumes transitions occur, on average, halfway through each cycle, so it weights the first and last cycles’ contributions by one-half and effectively shifts the accumulated area under the trace to better approximate a continuous-time process. It is a modest correction — its effect shrinks as cycle length shortens relative to the disease’s natural time scale — but it is treated as a standard refinement in published cohort models specifically because the alternative (uncorrected, boundary-only accounting) introduces a small, systematic, and avoidable bias into every cost and QALY total the trace produces. Reporting whether half-cycle correction was applied, and how, is exactly the kind of methodological detail the CHEERS 2022 reporting checklist expects a published economic evaluation to disclose.

Where a Markov cohort model fits

A Markov cohort model is one point on a spectrum of decision-analytic approaches, not the only option. Simple decision trees suit short time horizons with no need to represent recurring events; Markov cohort models suit chronic or recurring conditions over a longer horizon where a manageable number of health states captures the clinically relevant distinctions; individual-level microsimulation suits models where patient history genuinely drives future risk throughout, not just at a few points a tunnel state can patch. Once a cohort model is built, its uncertainty is explored the same way any decision-analytic model’s is: one-way deterministic sensitivity analysis presented as a tornado diagram, and probabilistic sensitivity analysis summarized through measures like the expected value of perfect information. One naming note worth flagging explicitly: a Markov cohort model has nothing to do with Markov chain Monte Carlo (MCMC) — both borrow the same underlying “memoryless” Markov-chain mathematics, but MCMC is a computational sampling technique used in Bayesian statistical estimation, not a health-state cohort simulation, and the two are not interchangeable despite the shared name.

Frequently asked questions

What’s the difference between a Markov cohort model and microsimulation?

A cohort model tracks proportions of an aggregate cohort moving between a fixed set of states, governed only by the current state (the Markov assumption). A microsimulation tracks individual simulated patients one at a time, letting each patient’s accumulated history — not just their current state — influence future risk, at the cost of Monte Carlo noise and materially more computation and reporting complexity.

How do you choose a cycle length?

Cycle length should match the clinical rhythm of the disease and the granularity of the available transition-probability evidence — short enough that clinically meaningful transitions aren’t averaged away within a single cycle, but not so short that it multiplies computational burden and parameter-estimation uncertainty for no added accuracy. Half-cycle correction narrows, but does not eliminate, the bias from choosing a cycle length that’s coarser than ideal.

What is a tunnel state, in plain terms?

A tunnel state splits one health state into a sequence of numbered sub-states a patient passes through one cycle at a time, so the model can assign a different transition probability at each step — reintroducing elapsed-time dependence into an otherwise memoryless model without abandoning the cohort-trace structure.

Why does half-cycle correction matter if the bias is small?

Because it is systematic rather than random, the uncorrected bias doesn’t cancel out across a probabilistic sensitivity analysis — it shifts every cost and QALY estimate the trace produces in a consistent direction. Applying it is inexpensive, which is why it is treated as standard practice rather than an optional refinement in published cohort models.

Do transition probabilities have to stay constant across all cycles?

No — time-varying transition probabilities (age-dependent mortality, a hazard that changes over the model horizon) are compatible with a Markov cohort structure as long as the probability applied in a given cycle depends only on the current cycle and current state, not on the cohort member’s own history before reaching that state; this is sometimes called a semi-Markov extension when probabilities vary by time already spent in the current state specifically.