Understanding Risk - Ordered Weighted Averaging and Relative vs Absolute Risk Reduction
One of my students has just completed an individual research paper on "Investigating the use of GIS-Based Ordered Weighted Averaging in Wildfire Restoration". Ordered Weighted Averaging (OWA) is a multi-criteria decision analysis technique that allows the decision-maker to mathematically define their attitude towards risk. The available strategies range from risk-averse to risk-taking approaches, based on a simple parameter setting. That parameter determines whether we emphasize the negative or positive aspects of a possible solution. Under a risk-averse strategy, we focus on the worst aspects and choose the solution that has the "least bad" outcomes. Under a risk-taking strategy, we focus on the best aspects and choose the solution with the "greatest good" outcome. The OWA also allows any number of strategies between these extremes, including an intermediate approach under which emphasis is balanced between the strong and weak outcomes.
While it is a fascinating conceptualization of risk, the OWA technique needs to be used with caution in practice. For example, a risk-averse strategy might be too much focused on the negative outcomes of a solution and miss thereby miss important positive outcomes that could more than compensate for the weaknesses. Similarly, a risk-taking strategy may accept a weak solution on the basis of only one, very strong aspect. The extreme low and high risk strategies do not allow for any tradeoff between strong and weak criteria, while the intermediate strategy, marked as weighted linear combination (WLC) in the above diagram, includes full tradeoff.
Now, think of the corona crisis. Are we pursuing a balanced response, as recommended by a group of Canadian infectious disease experts in July 2020? Many readers will probably think of government restrictions as a risk-averse strategy—designed to slow the spread of SARS-CoV-2 and reduce the expected strain on the health-care system. Given that large-scale lockdowns are an unprecedented public health measure, and the collateral damage was either not expected or willingly taken into account, this approach however appears more risk-taking than risk-averse to me. The obvious inefficiency and illegitimacy of lockdowns, and the mission creep from "flattening the curve" to achieving the unattainable goal of #CovidZero, support their assessment as an extremely risk-taking strategy.
The concept of risk, and our ability to assess risk, has also made the headlines in the context of the COVID-19 vaccine trials. Using data from a Nov 26 opinion piece in the British Medical Journal (BMJ), we can see that vaccine efficacy in terms of the relative reduction of the risk of getting ill is around 95%. For example, in the Pfizer trial, assuming an equal split of the 44,000 participants into the vaccine and placebo groups, 0.74% of the placebo group fell ill but only 0.04% of the vaccinated participants did. The relative risk reduction is calculated as the difference between these two incidences (0.7%) divided by the placebo value (0.74%), arriving at the conclusion that 95% of COVID-19 could be avoided if people got immunized. However, there is another way of looking that the same data: The risk reduction in absolute terms is only 0.7%, from an already very low risk of 0.74% to a minimal risk of 0.04%. Thus, risk reduction is 95%, but it also is just 0.7%.
Which one of these measures should we base an individual vaccine decision on? This is a very personal decision (or should be!), but in terms of publicly presenting and discussing the vaccine trial results, a 2019 article in the Drug and Therapeutics Bulletin is revealing. Box 3 explains the difference between absolute and relative risk reduction. Note the author's statement "Relative risks, then, can exaggerate the perception of difference, and this is especially prominent when the absolute risks are very small." What would you think if the headlines about the trial successes had read "Shot Reduces COVID-19 Risk by 0.7%" instead of "COVID-19 Shot 95% Effective"? The author of "How to communicate evidence to patients", Dr. Alexandra Freeman, advocates for reporting multiple metrics for better context. With respect to the ongoing pandemic, the powers that be should ensure transparent communication of scientific evidence. This also includes a host of other issues with the vaccine trials that Dr. Peter Doshi raises in the above-cited BMJ commentary.