Canada’s Perplexing Pandemic Forecasting
by Paul Minshull, CEO and Founder, COVID-19 Solution Architect
November 5, 2020
As we move into late Fall and watch the Wave II acceleration, we can see the growing impact of Canada’s lack of a cohesive pandemic modeling. Using recently published Ontario and Public Health Authority of Canada (PHAC) updates as examples, it’s obvious Canada can and must do better.
The good news is, with at least another year of the COVID-19 battle ahead of us, there is time to improve the quality of Canadian decisions. Forecasting remains one of the greatest weapons in the Canadian response to the pandemic. We have a dwindling opportunity to clean up and align our forecasting to save lives, protect the economy and support the daily wellbeing of Canadians.
At a glance:
- Ontario’s forecast for November is too conservative, impacting healthcare system capacity
- PHAC is communicating critical warnings to Canada
- Ontario’s new control guidelines, in many ways, contradicts the PHAC message
- Both Ontario and PHAC need a more robust approach to COVID-19 forecasting
- Canada desperately needs a national forecasting system
PHAC communication is on point!
The PHAC forecast [i] below in Figure 1 communicates a clear message about the significant risk posed to Covid-19 management in Canada for the next 60 days. Based on our modeling we fully support and applaud PHAC in reinforcing this critical message.
This modeling methodology takes a much higher level approach to the relationship between contacts and Covid-19 spread. Great news, there is significant opportunity to enhance this based on the Scarsin DSS approach forecasting the 92 health regions and consolidating upward. The other quantum enhancement of the Scarsin DSS is to manage the contact impact at a more discrete infection location (school, university/college, homes, congregate settings, community, workplace, travel…). This facilitates a more realistic understanding of potential interventions. We agree that an overall reduction in contacts of 25% would have a significant impact, although not as dramatic as what is presented. The main reason is that a 25% reduction would not have a homogenous impact across health regions given their stage of acceleration. That is the advantage of health region insights consolidated to a national level.
The updated Canada forecast from the Scarsin DSS in Figure 2 (below) for November 30 provides a consistent story to the grey line. What differs in the Scarsin DSS model is that we took into consideration all trend changes in community & workplace mobility in addition to announced changes in public health interventions. For example, the Ontario announcement of a new control system [ii] with its related guidance [iii] is acting as a spread accelerator in our forecasting vs previous forecasts. We are not alone in our concerns.
Infectious diseases physician Dr. Isaac Bogoch pointed to the measures for communities in the province’s orange category — which allows for up to a ten per cent positivity rate in a community — as potentially too loose. “Some of the metrics are too forgiving,” he said on Twitter. “These would allow for significant community spread without imposing measures to curb transmission.” iii
It is extremely important to note that positivity % as a metric is a formula:
Positivity rate % = Positive Tests / Tests Conducted
This means that changes in the denominator (tests conducted) can influence the key warning signs if the testing is not representative of the total population. If this were to occur, it could create additional decision risk.
Dr. Michael Warner, the medical director of critical care at Michael Garron Hospital, said the new rules enable economic reopening at the expense of health and safety. “The framework proposed by (Premier Ford) enshrines exponential spread as acceptable,” he wrote in a post online. iii
Canada and Ontario are inextricably connected through forecasting results. Canada needs a coherent, cohesive, coordinated response to the pandemic. Anything less produces a fragmented Whack-a-Mole response to an existential public health threat.
Figure 2 represents the current Scarsin DSS forecast for Canada. The epidemiological based models are predicting an acceleration beyond the statistical forecast. It is however not moving in a full exponential behavior as the 92 health regions are trending on different paths. Many have maintained control and therefore are not exhibiting exponential spread. The November projection ranges between 3,867 for lower, 4,564 for base and 5,424 for the upper. This compares to ~6,300 cases in the PHAC grey line.
Ontario forecast misaligned to new policies
Ontario released its modeling update to the public on October 29 [v], Figure 3 (below) is the forecast analysis. The relevant aspects of this graph are the purple line, which is the average daily cases for seven days, along with the three dashed red lines, which represent Ontario’s forecast.
The other lines represent potential comparisons (or benchmarks) and are meant to provide additional insight. The problem is, these choices may actually mislead the reader. Let’s take Michigan as an example, since it is referenced in the title. The time period from the Michigan data was selected from the Summer and time-shifted to the Fall in this graph.vi
The case count for Michigan on November 1, adjusted for Ontario’s population (10m vs 14.7m for Ont), would be the equivalent of >5,000 cases per day. Using Michigan as a proxy for Ontario purposes today is not helpful or appropriate.
The three red dashed lines are forecasts for Ontario. While it’s unclear what methodology was used to create these lines, using growth-based trend models is not ideal at this point in the pandemic, particularly if they are used to communicate critical risk and uncertainty to the public.
More good news: the best practice is clear! Using health region-based forecasts with an appropriate epidemiological methodology, we can begin modeling interventions (e.g. school closures, increased testing) directly, based on their current or projected implementation.
The forecasts in Figure 3 (above) provide a common base sloped line then flattening at three levels of daily case counts: 800, 1,000 and 1,200. But arbitrary peaks do not change the spread of COVID-19 in the health regions. The spread of the virus in the 34 health regions creates a provincial peak. This is an immutable fact.
We mapped the Ontario forecast to the Scarsin COVID-19 decision support system (DSS) in Figure 5 (below). Using 7-day smoothing for cases, based on the reported date, provides less lag in insights in comparison to date-to-case reporting system. The Scarsin DSS forecasts represent a range of upper, lower and base. The risk range in the Scarsin DSS is driven by local health region variances in the contact spread coefficient by age cohort.
Ontario’s worst case is actually much worse
Disturbingly, the Scarsin DSS forecast range exceeds the province’s 1,200 worst case scenario and shows an acceleration from the current trends, not the slower growth trend in the Ontario forecast. We cycled new assumptions for the health regions based on the November 3 new control system. They resulted in an increase to the Ontario forecast.
Another note of concern is around testing volumes. The Scarsin DSS forecast is predicated on testing productivity of ~37,000 per day. Figure 6 (below) shows that daily tests in Ontario are trending downwardvi. The 37,000 tests are equivalent to ~250 tests per 100,000 populations. This sounds significant but ranks below October testing volumes in Quebec and Alberta, rates of 280 and 294 per 100,000 populations, respectively. Precipitous drops below the 37,000 will result in fewer reported cases and additional unreported spread.
Healthcare system impacts
Forecasts matter in many ways. Assessing supply and demand on the healthcare system is a critical example. The additional demand forecasted in the Scarsin DSS shows incremental supply pressure on ICUs, over and above the 150 beds per day Ontario flags as a worst case. The risk above the worst case scenario is a critical ICU capacity challenge for November and even greater challenges in December.
The Scarsin DSS uses an epidemiology-based ODE (ordinary differential equation) compartmental model. This custom designed solution supports health region forecasts based on current and future potential interventions, which can include not only masks and social distancing, but also interventions at the source of potential infection such as schools, workplaces and long-term care facilities.
The 34 health regions in Ontario, and the 92 regions across Canada can accommodate wide ranging dynamics to expose the different timing of spread acceleration and deceleration. If these 34 forecasts are consolidated to the provincial level, they provide more insightful forecasting; at the national level, the insights can be game-changing.
The Scarsin DSS cycles health region forecasts twice a week, providing a constantly updating picture of potential outbreaks and successful mitigation and suppression strategies. Figure 9 shows the Scarsin DSS forecast for Ontario based on the bottom up consolidation.
Figure 8 (below) shows an Ontario heat map for November 30. This highlights major differences in spread throughout the 34 health regions based on their reported per 100k. The table is sorted by 5 day smoothed cases.
Here are a few selected health region forecasts to showcase the differing dynamics as well as how the epidemiology based forecasts (blue line) differ from the time series statistical forecast (orange line) with 80% prediction interval (dashed yellow).
Conclusions
At Scarsin, we are ready to help Canadian public and private sector stakeholders make better, evidence-based decisions that will save lives and support our economic survival.
References/Endnotes
[i] Update on COVID-19 in Canada: Epidemiology and Modelling Oct 30th, 2020, https://www.canada.ca/content/dam/phac-aspc/documents/services/diseases-maladies/coronavirus-disease-covid-19/epidemiological-economic-research-data/update-covid-19-canada-epidemiology-modelling-20201030-eng.pdf
[ii] Ontario Releases COVID-19 Response Framework to Help Keep the Province Safe and Open, https://news.ontario.ca/en/release/59051/ontario-releases-covid-19-response-framework-to-help-keep-the-province-safe-and-open [1] Coronavirus: Ontario classifies municipalities in new, more targeted COVID-19 category system, https://globalnews.ca/news/7439021/ontario-government-covid-19-coronavirus-proposed-measures/
[iii] Coronavirus: Ontario classifies municipalities in new, more targeted COVID-19 category system, https://globalnews.ca/news/7439021/ontario-government-covid-19-coronavirus-proposed-measures/
[iv] The label on the forecast scenario says “Forecast – October 24 data”. The name was not updated in time for this blog. It represents “Forecast – Oct 30 data” as this is the data last utilized for the forecasts, but we provided an additional data load to Nov 2 to ensure the actual data reporting of the west coast provinces which only publish data on the Monday.
[v] Covid-19: Modelling Update, https://covid19-sciencetable.ca/sciencebrief/covid-19-modelling-update-presentation/
[vi] Johns Hopkins Covid Map – Michigan, https://coronavirus.jhu.edu/data/new-cases-50-states/michigan
[vii] COVID-19 (Coronavirus) Updates: Canada, https://www.covid-19canada.com/graphs