What Most People Still Get Wrong About COVID Management

When I read COVID retrospectives, my hopes for humanity die a little bit. Today, I want to zero in on the most crucial lessons people have not learned. Later this week in the premium article, I will do a retrospective on my own performance: What did I get right and wrong? I’ll look back at some articles I’ve written, with a special focus on the big predictions. Subscribe to read it!

I’ve written in the past about takeaways from COVID: Mental Pitfalls of COVID, Top 24 Policy Mistakes, how most of the West failed at managing it… What I haven’t done, though, is distill the most fundamental and important ways of thinking that people consistently get wrong in the context of COVID. Three years in, these are the main lessons that most people haven’t learned.

Judging Past Decisions Based on New Information

BAILEY: “Should we go to the pool or to the beach?”

ANANDA: “We went to the pool yesterday. The weather is supposed to be great at the beach, and it’s Tuesday so there shouldn’t be that many people. Let’s go to the beach?”

BAILEY: “OK let’s do it!”

Later that day…

BAILEY: “What is this traffic jam?!”

ANANDA: “Waymo says there’s been an accident. Wow, we’ve been stuck here for two hours! It was the wrong call: We should have stayed at the pool.”

BAILEY: “No. Based on the information we had, it was the right call to go to the beach. We looked at the pros and cons, how long it would take with the usual traffic, and based on what we knew, we made the right choice. It turned out poorly, but we did the right thing.”

If you can bet $100 and have a 55% chance of doubling your money to $200, and a 45% chance of halving it to $50, should you take that bet? Yes, all day long, as many times as you can, because the expected value of that bet is quite positive. You might still lose money, but in the long term, odds are you’ll make money.

Should you leave your money in the bank, certain that you’ll earn 2% per year? Or should you invest in the stock market, knowing that you might lose a lot of money or make a lot, but on average you’ll make ~5% per year? Theory says you should invest¹. Yes, some people invested in 2008 and lost most of their money before making it back a few years later,while some Japanese invested in the 1990s and barely recouped their losses later on. But investing was probably still the right call, based on the information these people had.

Here are some statements I’ve read about COVID management:

“We shouldn’t have closed schools during COVID.”
“We shouldn’t have had lockdowns.”
“We shouldn’t have obsessed about soap and hand sanitizer.”
“We shouldn’t have stopped people from going outdoors.”
“We shouldn’t have tried to go for an Asian-style lockdown.”
“We shouldn’t have tried to do contact tracing.”
“We should have closed our borders earlier and kept them closed longer.”
“We should have opened up just after old people got vaccinated.”
“We should have locked down old people and left the rest free.”
“We should have mandated masks earlier.”
“We should have stopped social gatherings immediately.”
“We should have only tried to flatten the curve.”

For many of these, we had enough information in March 2020 to accurately predict the outcome. For others, new information that came later meant that decisions that seemed right at the time had bad outcomes. We must differentiate between them.

For example, we knew what would happen if we fully opened up the country just after old people got vaccinated: There were dangerous alpha and delta variants running in 2021; many more people would have died, and many more would get Long COVID.

We knew around March or April 2020 that we could hang out outdoors without masking, and it would be mostly OK. It’s wrong most countries maintained restrictions on gathering for outdoor events until well into 2022.

We knew in March 2020 that masks worked. We should have mandated them as soon as we did.

Extract from The Hammer and the Dance, published on March 18th 2020

Right Decisions with Wrong Outcomes

Conversely, we initially thought the virus spread mostly through direct interaction, either speaking / breathing / singing, or through touch. It made sense to keep a distance of 2 meters, wash hands², and disinfect surfaces.

Then we learned that aerosol transmission is much more significant than transmission through surfaces, which changes everything. Indoor spaces are dangerous, whereas standing close to strangers outdoors is not a danger. In 2021, it made no sense to keep the onus on social distancing and hand-washing instead of masking and ventilation.

Another example is closing borders. In March 2020, the most prominent paper circulating on border closures stated that, at most, they only delayed an epidemic by a few days. Back then, this was consistent with what was happening across most of the world—except for Taiwan and Japan, which fully closed their borders and which were able to fend off COVID. 

Chart from Coronavirus: Why You Must Act Now, March 10th 2020, showing how reducing travel (staying in the same band, going one row lower) didn’t really help to stop transmission rates.

Sidenote: Empirical >> Theoretical

This deserves its own aside. Many people claimed during COVID that some measures could or could not work, despite real-time evidence about them being available. 

For example, you can’t categorically claim that border closures don’t work when countries like Japan or Taiwan are using them successfully. 

That said, since it was unclear why exactly Taiwan and Japan were successfully stopping COVID, or if it was sustainable, and at the same time some papers were pretty clearly against it, multiple different border decisions were reasonable: to fully close, to remain open for countries with similar prevalences of disease, or to quarantine visitors. 

We now know that full closure like in Taiwan or Japan can work, but that most countries would have a hard time pulling it off. Regardless, if your country tried and failed to fully close, or was in a position to succeed but didn’t even try—those were both acceptable situations³.

Lack of Leadership

There’s a completely different type of decision where failure was not due to the laws of epidemics, but rather to lack of leadership.

A very clear example is masking. Where most European countries had no problem adopting it, in the US it became a culture war. I don’t think Americans failed to mask “because it’s a free country.” In that free country, you are still not allowed to buy Kinder Surprise eggs, you can’t drink alcohol on the street, you can’t walk naked in most cities⁴, and you still get a ticket if you drive without a seatbelt or while texting. Freedom is always relative. Americans failed to mask because Republican leadership thought it would be a good political battleground. 

Most Western governments didn’t even get into the privacy issue. They were very open about destroying the economy and freedom to save lives; but for some reason, a small infringement on people’s privacy that would allow fantastic contact tracing—a proven epidemic mitigation measure—was never on the table. 

I think this is because privacy was an intense conversation topic in the 2010s⁵, so people learned that privacy=good, always. But it ain’t so. A proper leader would have realized this, and explained how some specific mobile tracing in some specific situations, with specific boundaries, could have allowed a much earlier reopening of economies. 

But that leadership was nowhere to be seen. I think this is a symptom of our times—somehow, the democratic processes we follow in Western countries don’t usually produce strong leaders. The occasional exception—like New Zealand’s Jacinda Ardern⁶—only highlights the contrast with most other leaders.

Ms. Ardern unified the country on strict border closures, masking, quarantines and other measures, thereby making their implementation possible—while many other heads of state tried similar measures and failed miserably. 

Missing the Value of Time

But by far, the concept people most miss is the value of time. The best example is flattening the curve vs. full Zero COVID vs. not doing anything.

It was clear in the past that not doing anything was a bad idea. But many countries tried to go for Zero COVID for months or years, failing miserably. That leaves “flattening the curve” looking like the only reasonable option. There are many problems with that.

First, Zero COVID makes much more sense before vaccines than after. Since vaccines divide deaths by 10 and reduce transmission⁷, trying to save nearly everybody until the vaccines arrived made sense. There was value in Time, and a policy adapted to one time might not be adapted to another. 

Conversely, Zero COVID after vaccines makes little sense: If you’re in a world where the entire planet is infected, most countries can’t keep the virus under control, and people in developed countries are getting antsy about lockdown measures… Were developed countries supposed to keep attempting Zero COVID on for decades⁸? Even China had to accept that it couldn’t pull it off. What worked early on didn’t work later on.

More importantly—and this is the single most important factor that most people miss—time early on allowed us to learn.

What if the infection fatality rate turned out to be 5% instead of ~1%?
What if it affected young people?
What if a second infection was deadly?
What if 80% of infected children developed Long COVID?
What if masking stopped the virus completely?
Or hand-washing?
What if hydroxychloroquine or ivermectin reduced fatalities by 99%?
What if the virus mutated even faster and became deadlier with every variant?
What if a combination of testing, contact tracing, isolation, and quarantine was an easy package to deploy, and eliminated nearly all transmission?

If any of these situations had turned out to be true, the entire management of the pandemic would have been radically different. A Zero COVID measure is unacceptable today, but if 80% of children developed Long COVID, I can assure you we would have made Zero COVID work. 

Conversely, if hand-washing worked flawlessly, any lockdown would have been unacceptable—once we learned that fact.

The “once we learned that fact” is the key here. We didn’t know the answers to this list of questions in March 2020. As a result, a total lockdown was necessary back then, so we could learn how the virus behaved and what worked and what didn't work against it. The biggest value of initial lockdowns was not just to flatten the curve to reduce cases. It was to eliminate all cases because we had no clue what we were dealing with. 

The moment we knew—around April-May 2020 we knew a lot already—lockdowns became much less useful. They could still be useful enough to maintain them. But their value in our learning process was unbeatable early on. 

In computer science, we call this the “explore–exploit” problem: First, you want to learn. Then, you exploit this learning. You need to be thoughtful about when you’re doing which. 

In the case of a pandemic, when billions of lives are at stake, you should be cautious and give yourself the time to explore. 

Many people miss this, including Nobel laureates⁹, so don’t blame people who do. But from now on, I hope you don’t forget it. 

Nobel Prize winner Michael Levitt, jumping from Chemistry to Epidemiology with a tad too much confidence. Source.

How to Improve The Confidence of Your Predictions

These are the biggest lessons from COVID about how to think better. But they’re not the only ones. Indeed, getting it right is hard. In Uncharted Territories, we frequently explore ways to think better. But as important as getting things right is knowing how confident you are about your answers.

If you’re right 90% of the time playing roulette, but you go all in every time, you are sure to lose all your money. You should try to be right, but also know when you’re sure or not.

The only way to know this is by answering lots of questions along with your confidence in the answer, and then checking your score. For example:

You answer a hundred questions like this, and you get a graph like this:

I answered hundreds of questions, while also stating how confident I was about them (meaning, what % of the time I would get such an answer right) and then compared them with actual answers. 

The confidence is in the horizontal axis and the accuracy in the vertical one. You can see that for most questions, my confidence matches my accuracy, but this isn’t true when I’m half sure (I am right 70% of the time when I think I’m 50-60% right), or when I’m very sure (when I say I’m 95% sure, I am only 80% right¹⁰). 

I didn’t start this way. The first time I tried this, I was completely off. But I played this game for an hour or so, and I got much more aware of how confident I should be in my answers. If you want to try it yourself, go play the Calibration game. It’s free.

Why is this useful? Because without it, most people tend to be overconfident. They think they know the answers to things, but it turns out they’re not really sure. This way, they learn to introspect and realize how sure they are of their answers.

This is a thing that would have been incredibly valuable to most COVID pundits. If everybody was trained with this tool, instead of dealing in categoricals like vaccines kill or lock down now, we could have more nuanced debates.

Takeaways

1. Don’t judge decisions from the past based on their outcome. This will inevitably lead to frustration and bad thinking habits. Instead, think back at what you knew back then, and see if you could have made a different decision by seeking the right information or thinking better. Don’t try to improve the accuracy of the decision, but rather its quality.

2. If somebody has been able to do something, you must conclude it’s possible to do, even if your theory says otherwise.

3. Good leaders can lead their people in the right direction, even if nobody has led them before. They must be both right and good at communication—a rare combination in political leaders nowadays.

4. Always consider time in your decisions. Sometimes, you can’t wait and must make a decision immediately. Sometimes, you must delay it. Your decision might also change over time.

5. It’s not only important to be right. You must also know how likely you are to be right. Otherwise, you’ll tend to be underconfident or overconfident, both of which lead you to worse decisions. 

I hope these lessons make sense to you. In the next (premium) article, I will show how I applied these concepts in some COVID predictions I made. Subscribe to read it!

1. This is not investment advice. I am not a financial advisor. This is to illustrate the point only.

2. For a really long time, many people washed them 40 times a day!

3. Acceptable science-wise and regarding public opinion.

4. Although the first time I drove through Berkeley, CA, I stopped at a red light and a naked guy proceeded to cross the street in front of my car while dancing with maracas. In retrospect, this was a hardcore introduction to Berkeley politics, but coming from Europe, it might have just helped my US soft landing.

5. This is the decade of Edward Snowden, Cambridge Analytica, or the GDPR law.

6. I don’t want to judge her general policies, because I don’t know them. But her leadership of NZ through the pandemic was exceptional.

7. Both in total number of cases and, more importantly, in viral load, which makes the illness much less dangerous.

8. Since now COVID is endemic, and would have become endemic anyways in emerging economies.

9. Michael Levitt was a strong proponent of just letting the virus run.

10. This, however, might be because of a format issue. For example, “What two letters did the singer Hammer lose?” Answer: “MC”. “Wrong! It’s M.C.” I’m 70% confident in that diagnosis.