Direct comparison
Bayesian vs Frequentist Network Meta-Analysis
How Bayesian and frequentist network meta-analysis differ on priors, MCMC diagnostics, SUCRA vs P-score ranking, and use in NICE DSU-guided HTA submissions.
Written and maintained by CASRAI Editorial Board
Last updated
Ask CASRAI · included with Regulatory Radar
Ask about Bayesian vs Frequentist Network Meta-Analysis
Ask CASRAI answers research-administration questions and cites the passages behind every claim — and says so when the corpus does not cover something, instead of guessing. It comes with a Regulatory Radar subscription at $29 a month, alongside the daily digest of regulatory changes and the dashboard of what changed.
150 questions a day, on this site, over the API, or inside your own tools through the CASRAI MCP server.
Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays free to read, with no account and no card.
How do Bayesian NMA, Frequentist NMA compare side by side?
The table below compares Bayesian NMA, Frequentist NMA across 14 procurement-relevant dimensions, from statistical framework through sensitivity to sparse data.
Side-by-side comparison
| Dimension | Bayesian NMA | Frequentist NMA |
|---|---|---|
| Statistical framework | Full probability model: treatment effects and nuisance parameters (e.g. heterogeneity) are treated as random variables with prior distributions, combined with the trial data via Bayes’ theorem to produce a posterior distribution. | Effects estimated via likelihood-based methods -- generalized linear/mixed models or generalized least squares -- with inference based on the sampling distribution of the estimator, not a posterior. |
| Prior distributions | Required. Vague or minimally informative priors are the common default for treatment effects, but NICE DSU TSD2 flags that results, especially in sparse networks, can be sensitive to the prior chosen for the between-study heterogeneity parameter. | Not used. Heterogeneity is estimated directly from the observed data (e.g. via REML-type or DerSimonian-Laird-generalized estimators), with no prior specification step at all. |
| Model fitting | Fit by MCMC simulation (e.g. Gibbs sampling): the algorithm draws thousands of samples from the joint posterior distribution rather than solving for a closed-form estimate. | Fit by iterative optimization or generalized least squares, typically converging to point estimates in a fraction of the time, with no simulation step required. |
| Convergence checking | MCMC convergence diagnostics are required before trusting output: trace plots, the Gelman-Rubin (Brooks-Gelman-Rubin) potential scale reduction statistic, effective sample size, and running multiple chains from dispersed starting values. | No MCMC convergence step exists. Model-fit checks instead concern deviance, residual heterogeneity, and goodness-of-fit -- a different kind of diagnostic entirely. |
| Treatment ranking output | Produced directly: at every MCMC iteration each treatment gets a rank, so a full posterior distribution of ranks -- and probability statements like "probability Treatment A is best" -- falls straight out of the model. | Derived, not modeled directly: point estimates and confidence intervals for each pairwise comparison come first, and a ranking metric is calculated from those afterward. |
| Ranking metric | SUCRA (Surface Under the Cumulative RAnking curve), introduced by Salanti et al. (2011), summarizes the simulated rank-probability distribution as a single 0-100% score per treatment. | P-score (Rücker & Schwarzer, 2015), SUCRA’s frequentist analogue -- computed analytically from the normal distributions implied by each treatment’s point estimate and standard error, with no simulation needed. |
| Effect-estimate reporting | Posterior median or mean with a credible interval (e.g. 95% CrI) -- interpreted as the range containing the true effect with a given probability. | Point estimate with a confidence interval and, where relevant, a p-value -- interpreted via the long-run frequency properties of the estimator, not a probability about the true value. |
| Complex network structures | Hierarchical/graphical models handle multi-arm trials’ internal correlation and unbalanced, sparse networks in a comparatively unified way, and inconsistency can be assessed within the same modeling structure (Cochrane Handbook Ch. 11). | Historically needed extra adjustment for multi-arm-trial correlation and network complexity; graph-theoretical methods (e.g. netmeta) have closed most of that gap, though Bayesian hierarchical modeling is still often considered more flexible for very sparse or structurally unusual networks. |
| Common software | WinBUGS/OpenBUGS or JAGS, often called from R via a package such as gemtc, or increasingly Stan-based tools. NICE DSU TSD2’s own worked-example code is written for WinBUGS. | The netmeta package in R (Rücker/Schwarzer), plus general mixed-model or GLS-based implementations -- no MCMC engine required. |
| Computational demand | Heavier: MCMC runs need enough iterations, burn-in, and chain diagnostics to be confident of convergence, though modern hardware makes this fast for a typical-sized NMA. | Lighter: near-instantaneous point estimates and intervals with no sampling step and no burn-in to tune. |
| Use in HTA submissions (NICE DSU) | NICE DSU’s Technical Support Documents, particularly TSD2, present network meta-analysis within a Bayesian framework -- historically the more commonly seen approach in NICE technology appraisal submissions that reference this guidance. | Methodologically accepted and increasingly used, e.g. via netmeta, particularly for exploratory analyses or sensitivity checks alongside a Bayesian base case -- not the historical default in NICE DSU-referenced HTA work. |
| Reporting standard | PRISMA-NMA (Hutton et al., 2015, Annals of Internal Medicine) applies regardless of framework: network diagram, description of network geometry, and an inconsistency assessment are all expected. | Same PRISMA-NMA extension applies -- the reporting checklist is framework-agnostic, so switching estimation approach does not change what must be reported. |
| Inconsistency assessment | Commonly assessed with the node-splitting method or an unrelated mean effects (inconsistency) model fit alongside the consistency model, comparing model fit statistics (e.g. residual deviance) between the two within the same Bayesian workflow. | Commonly assessed with the design-by-treatment interaction test or a separation of evidence into direct and indirect estimates for comparison -- conceptually the same consistency check, computed without a posterior distribution. |
| Sensitivity to sparse data | Vague priors on heterogeneity can behave poorly in very sparse networks (few trials per comparison); NICE DSU TSD2 recommends checking results against alternative, more informative heterogeneity priors as a sensitivity analysis in that situation. | Estimation can also become unstable with sparse data (wide confidence intervals, difficulty estimating heterogeneity), though there is no prior-sensitivity dimension to check specifically -- the instability shows up directly in the interval width instead. |
Common questions
Common questions about Bayesian NMA vs Frequentist NMA
Is Bayesian network meta-analysis inherently more accurate than frequentist?
+
Not inherently. With vague, minimally informative priors and enough data, Bayesian and frequentist NMA models estimate the same underlying quantities and, in most applications, produce very similar point estimates and interval widths. The practical difference is in what output each naturally produces -- probability statements versus p-values/confidence intervals -- and how ranking is derived, not raw accuracy. The right choice usually comes down to what the research question or reviewing body needs to see, and what software and expertise are available.
Does NICE require Bayesian network meta-analysis for technology appraisal submissions?
+
NICE’s Decision Support Unit Technical Support Documents, particularly TSD2, set out network meta-analysis within a Bayesian framework, and Bayesian NMA has historically been the more commonly seen approach in submissions that reference that guidance. This is a strong convention rather than an absolute mandate -- frequentist methods are methodologically acceptable and do appear in submissions, especially as sensitivity analyses -- but analysts targeting a NICE-facing HTA submission should expect Bayesian NMA to be the default expectation.
What is SUCRA and how does the frequentist P-score differ from it?
+
SUCRA (Surface Under the Cumulative RAnking curve) is a 0-100% summary of a treatment’s rank-probability distribution from a Bayesian NMA’s MCMC output -- higher values mean a treatment is more likely to rank near the top. P-score is the frequentist analogue: instead of being derived from simulated posterior rank probabilities, it is calculated analytically from the normal distributions implied by each treatment’s point estimate and standard error. The two measures are numerically very close for the same network and support the same practical read of "which treatments tend to rank well," but P-score needs no MCMC simulation to compute.
What MCMC convergence diagnostics should I check before trusting a Bayesian NMA?
+
At minimum: trace plots for each parameter (checking visually that multiple chains mix well with no trend), the Gelman-Rubin / Brooks-Gelman-Rubin potential scale reduction statistic, and effective sample size for key parameters. NICE DSU TSD2 outlines this practice for networks submitted to NICE, and running multiple chains from dispersed starting values is standard for catching non-convergence a single chain would miss.
Can frequentist network meta-analysis handle multi-arm trials and complex networks as well as Bayesian methods?
+
Modern frequentist approaches, particularly the graph-theoretical method implemented in the netmeta R package, have substantially closed the historical gap -- multi-arm-trial correlation and network inconsistency can both be handled directly. Bayesian hierarchical models are still often considered somewhat more naturally flexible for very sparse or structurally unusual networks, since the correlation structure falls out of the model itself rather than needing a separate correction step, but this is now more a difference of convenience and tradition than of what is technically achievable.
Do I need specialized software to run a Bayesian network meta-analysis?
+
Yes, typically. Bayesian NMA is usually run through an MCMC engine such as WinBUGS, OpenBUGS, or JAGS, often called from R via a package like gemtc, or increasingly through Stan-based tools -- NICE DSU TSD2’s own worked-example code is written for WinBUGS. Frequentist NMA has a lower software barrier: the netmeta package in R runs a full network meta-analysis, ranking, and inconsistency assessment without any MCMC setup.
Should I run both Bayesian and frequentist NMA on the same network?
+
It is common practice, and often reassuring rather than redundant: because both approaches are estimating the same underlying network of treatment effects, a Bayesian base-case analysis with a frequentist (netmeta) cross-check -- or vice versa -- is a reasonable way to confirm results are not an artifact of the estimation framework. Materially different conclusions between the two are worth investigating rather than reporting only the more favorable one; genuinely large discrepancies usually trace back to a sparse network, an influential prior choice, or an inconsistency problem rather than the frameworks disagreeing on well-supported evidence.
Going deeper
