Vital Signs. The evidence that lockdowns work may not be gold standard, but it's good

As Australia begins gradually to relax some of its containment measures (so-called nonpharmaceutical interventions or NPIs) we are confronted with a number of questions

• when will children be back at school full time?

• when might restaurants and pubs be open and will the public continue to respect social-distancing rules when they do?

• do we have adequate testing and contact tracing to extinguish flare-ups in infections without reverting to harsher containment measures?

And more. They are hard and important questions.

But another one being asked is surprising: do lockdowns work?

The question has underpinned the position of lockdown sceptics such as Henry Ergas writing in the Australian and Christopher Joye,

The Australian’s Adam Creighton gave explicit voice to the question this week, writing that our excellent performance in keeping COVID-19 at bay had blinded some to the lack of evidence for lockdown restrictions

The human desire for ritual, and our laudable want to do the right thing, appear to underpin social distancing restrictions more than science.

It is odd that anyone should be seriously asking this question, and it is important to understand that the evidence most certainly does suggest that lockdowns work.

If we fail to accept it, our response to a second-wave outbreak, or to the next global pandemic, will be badly compromised.

It is also important to consider what constitutes “good evidence” when high-stakes decisions need to be made quickly.

The evidence is fairly clear

The gold standard is high quality evidence from randomised controlled trials subjected to peer review, but in emergencies there’s no time to wait for it.

It allows sceptics to point to uncertainly and cherry-pick low-quality evidence that supports their case.

As it happens, the cross-country evidence is fairly clear. Countries that locked down earlier have had better health outcomes, as the following chart from the Financial Times shows.

FT analysis for European Centre for Disease Control data. Financial Times

We don’t yet have the final word on this crisis, so it is also worth looking back to an earlier pandemic.

A 2007 study in the Journal of the American Medical Association examined what happened in 43 United States cities during the 1918-19 Spanish Flu pandemic.

It found measures such as school closures and bans on public gatherings were (statistically significantly) associated with fewer excess deaths.

Cities that implemented NPIs earlier had lower peak mortality rates and lower total mortality. Also the cities that kept the NPIs in place the longest had the lowest mortality.

As the authors put it,

these findings demonstrate a strong association between early, sustained, and layered application of nonpharmaceutical interventions and mitigating the consequences of the 1918-19 influenza pandemic in the United States

A more recent study by economists at the Massachusetts Institute of Technology and the Federal Reserve finds the cities that adopted the most aggressive social distancing measures had the highest economic growth after the restrictions were lifted.

Moreover, cities that implemented NPIs 10 days earlier than other cities increased their manufacturing employment by 5% more than those that imposed restrictions later. The difference lasted through to 1923.

Overall they found an additional 50 days of social-distancing was associated with a 6.5% percent increase in manufacturing employment.

Sure, it’s not gold standard

Creighton cited a World Health Organisation report that summarised a number of studies of previous pandemics as evidence that there was no evidence to support the measures Australia has taken.

In fact, the WHO report simply said two things.

One was that randomised controlled trials are the best, but that they are not always possible or available.

No surprises there.

The second was its recommendations, based on the evidence it assessed.

World Health Organisation

In summary, the World Health Organisation looked at the evidence and recommended social distancing measures to avoid crowding, school and workplace closures, and internal travel restrictions.

It is true that at that stage it recommended against border closures, but if Creighton or his newspaper want to defend the docking of the Ruby Princess on March 19 on the basis of the report, they can go ahead.

In a crisis, evidence is never gold standard

In a crisis, decision makers don’t have the luxury of time. They have to act based on the best evidence they have.

Even if that evidence falls short of perfectly identified causal effects, it deserves to be interpreted in a scientific and dispassionate way.

Or to paraphrase the late US Senator Daniel Patrick Moynihan, “commentators are entitled to their own opinions but not to a wilful and purposeful misinterpretation of the facts”.

Those who misinterpret the evidence there is have COVID-19 on their hands.