The Signal and the Noise

"The next book is called The Signal and the Noise and it’s by Nate Silver. Silver used to be an American poker player. He was also a baseball analyst. He had lots of different jobs and projects in the past. He became famous as a blogger because he succeeded in predicting the American presidential elections in 2008 and 2012 very accurately. Based on this success, he then launched a site called FiveThirtyEight and published this book, The Signal and the Noise. This book comes back to what I was saying before. As the title suggests, it’s about differentiating between ‘the signal’ and ‘the noise.’ Silver starts with the premise that yes, in the last 10-15 years, we’ve had huge amounts of data coming in and we’re now able to analyse it. But his view is that we should not focus too much on the noise but rather focus on the signal. That means making sure that we don’t just look at data in a random and haphazard way, but in an insightful way. The book discusses the practical application of this in a variety of fields, including baseball, elections, weather forecasting, climate change, economics, and of course poker. It also emphasizes the importance of properly expressing uncertainty in statistical statements and the need to consider a range of probable outcomes rather than just single-point estimates. This can take various forms, but probabilities are a big one. A lot of the book is about that. Silver describes perfectly what probabilities are, and I think that’s an important question that many more people in our societies should be able to answer: what do probabilities actually mean? Famously, there was—I’m not sure if scandal is the right word—but a big issue around 2016 and the Trump election. Nate Silver and FiveThirtyEight published a model that predicted a 30% chance of Trump winning. And still, to this day, there are many people who say that Silver was wrong because Trump was elected. A lot of this book is about explaining why that is a terribly wrong understanding of probability. If you say that something has a 30% chance of happening—which is one in three—then it can definitely happen. If you roll a die, it has a roughly 30% chance of landing on one or two, and surely that’s not impossible? What 30% means is that if we were able to run an endless number of American elections in 2016, a third of them would have ended up with Trump winning, and we just happened to live in one of those branches. “What do probabilities actually mean?” Another big thing about this book—which is a theme in all the books I picked: I think probably every book I chose at some point explains it, except maybe for Factfulness —is Bayes’ theorem. Thomas Bayes was an 18th-century statistician and philosopher. He came up with this important theory about how to understand the world analytically. The actual equation of the theorem doesn’t matter; it’s a little bit complicated, and it’s not important. The reason why all these books describe Bayes’ theorem is that doing Bayesian statistics means doing statistics and understanding data in a way that is trying to reason iteratively, based on the evidence available to you over time. Let’s take an example. Let’s try to figure out, ‘What are the chances that I have COVID right now?’—which is a question all of us have had a lot. What a lot of people tend to do is go back and forth between extreme versions of their view: they start by thinking they could never have COVID (so roughly a 0% chance), but then one day they start coughing and instantly think, ‘Oh, I must have COVID.’ Under Bayes’ theorem, the correct way to do this would be to start with a baseline probability, which could be an estimate of how many people have COVID in the UK right now, which you can derive from government statistics. For example, this could be 3%. After this, when you get a new piece of information—for example, you start coughing—the idea underpinning Bayes’ theorem is that you should update your probability that you have COVID based on this new information. Maybe that probability goes up to 20%. That 20% is based on the fact that if you have COVID, there’s a good chance you’re going to cough… but coughing also happens for lots of other reasons! So instead of going straight to the worst-case scenario, under Bayes’ theorem, you simply update your view according to the available evidence. And with each new piece of information—for example, you start feeling feverish—you keep updating until you get to the latest and most accurate estimate of the probability of the event. Bayes’ theorem can work for lots of things—in your personal life, to think about current events or the probabilities in an election. In an election, you’d start with the probability of a Democrat or a Republican winning, you could start with a 50/50—or maybe something a little more fine-tuned, for example, based on which party is the incumbent. Then, with each poll that you see, you can slightly update the probability based on it, to get more and more accurate estimates. Just because you see one new poll that contradicts your previous estimates, you shouldn’t radically change your view. You should update your view a little bit based on each poll, but not too much."