December 24, 2024 - 11:00am

A recent study has criticised the reliability of Covid-19 projections made by SPI-M, a subgroup of the UK’s Scientific Advisory Group for Emergencies (SAGE), suggesting that some forecasts were so inaccurate as to be ineffective for planning and decision-making purposes.

The paper, published in the journal Global Epidemiology, examines two key failures in SAGE’s predictive modelling: one in July 2021 during the Delta wave, and another in December of that year with the emergence of the Omicron variant. In both cases the forecasts, widely relied upon by policymakers, were either too vague or significantly off target.

Ahead of the so-called “Freedom Day” in July 2021, SAGE forecast that daily hospitalisations could range from 100 to 10,000, and warned cases would “almost certainly remain extremely high for the rest of the summer”. Instead, hospitalisations peaked at about 1,000 per day, while cases began to decline shortly after restrictions were lifted, more than 10 times below the upper bound, diverging sharply from predictions, the study found.

Meanwhile, in December 2021, SAGE warned that under “Plan B” restrictions — which the Government ultimately maintained — daily deaths could peak between 600 and 6,000. The eventual peak was just 202, falling far below the lower bound of the prediction.

The study’s authors attribute these failures to SAGE’s over-reliance on mechanistic modelling, which simulates disease dynamics based on theoretical assumptions. While mechanistic models are useful in assessing intervention impacts, they depend heavily on high-quality data, which was often unavailable or inconsistent during the crisis.

In contrast, the authors cite the South African Covid-19 Modelling Consortium (SACMC), which adopted a more flexible and diverse approach, using simpler descriptive models alongside mechanistic ones. Despite being far less resourced than SAGE, the authors claim that SACMC delivered significantly more accurate projections, particularly during the Omicron wave.

Over the course of the pandemic, the UK experienced fluctuating infection rates, hospitalisations, and deaths, with notable peaks during both the first and second (Alpha) waves. At the height of the third Delta wave in mid-2021, the country saw daily case numbers exceeding 40,000, with hospital admissions rising in some areas. By the time of the Omicron variant’s surge in late-2021, infections soared once again, yet hospital admissions and death rates remained much lower than the first two waves.

The findings highlight shortcomings that impacted not only UK policymaking but also global public health strategies, given Imperial College London’s influence as the World Health Organization’s sole collaborating centre for infectious disease modelling. “Had SAGE adopted a methodologically pluralistic approach, many of these errors could have been avoided,” the study argues.

“The failure to adapt and use diverse methods during a pandemic is not just a missed opportunity,” the researchers write. “It is a risk we can’t afford to repeat in future global health crises.”