Facebook
Twitter
Pocket
Reddit
Email
Since the midterm elections, a feud has been raging on Twitter between Nate Silver, founder of FiveThirtyEight, and Nassim Nicholas Taleb, hedge-fund-manager-turned-mathematical-philosopher and author of The Black Swan. It began, late last year, with Silver boasting about the success of his election models and Taleb shooting back that Silver doesn’t “know how math works.” Silver said Taleb was “consumed by anger” and hadn’t had any new ideas since 2001. The argument has gotten personal, with Silver calling Taleb an “intellectual-yet-idiot” (an insult taken from Taleb’s own book) and Taleb calling Silver “klueless” and “butthurt.” Here is a recap of what they’re fighting about so you can know who’s right (Silver, mostly) and who’s wrong (Taleb).
The origin of Taleb’s ire can be found in Silver’s success since 2008—and his some-time failures. As I described in Nautilus last month, evaluating probabilistic election forecasts can be conceptually slippery, made especially difficult by the counterintuitive properties of mathematical probability. Historically, Silver has received considerable credit, probably too much, for “calling” elections correctly. As his most recent analysis (incorporating at least some of my suggestions!) shows, the more meaningful questions are: 1) How often does something that he gives an n percent chance to actually happen? and 2) How bold are his predictions, in the sense of probabilities being closer to 100 percent rather than long-run averages? In both respects, his models appear to be doing pretty well.
Yet Taleb maintains that Silver (whom he has taken to calling “Bullshit Nate”) is “ignorant of probability,” a “fraud,” and “a total impostor.” Why?
Critiquing—often rudely—the ways people think about uncertainty is the essence of Taleb’s brand. A thesis repeated in his books, backed up by his strategies as an investor, is that we understate the uncertainty concerning rare and extreme events in complex systems. By modeling randomness with assumptions gleaned when conditions are mild, we expose ourselves to unanticipated crises like market crashes. But a savvy investor, taking advantage of a market under-pricing those risks, can make a profit betting on the extremes.
A “black swan”-like critique of the FiveThirtyEight forecasts is likely possible and even valuable.
His complaint is that Silver’s forecast probabilities, particularly in 2016, fluctuated too much leading up to the election. Raising the stakes even more, Taleb has repeatedly hate-tweeted a paper of his, saying Silver’s forecasts were guilty of “stark errors” that, if committed on Wall Street, would have allowed investors to clean him out by means of what’s called “arbitrage.” Taleb, drawing on his financial background, proposed his own model that he claimed was immune to such an attack.
Why does Taleb think finance has anything to say about election forecasting? It has to do with the sometimes-complex relationship between probabilities and prices. Imagine we were placing bets on an event, say, Trump being impeached, in the form of lottery tickets that pay $1 if he is. Suppose I’m willing to pay up to $0.25 for such a ticket. Financial theory can’t say whether that price is right or wrong, but it can say that, were I then offered the complementary ticket paying $1 if he isn’t impeached, the only rationally coherent price I could pay for it is $0.75, because if my price were, say, $0.77 instead, then someone could earn a sure profit of $0.02 by selling me both bets for $1.02. Similarly, if I am required to offer the bets at the same price I quote, then any price less than $0.75 would allow someone to earn a riskless profit by buying both from me.
Today we would call any one of these combinations of bets an arbitrage, meaning an investment with no probability of loss and some probability of profit—a theoretically infinite money pump. The general assumption is that real arbitrage opportunities are, at most, very short-lived, because market forces act immediately to shut off the pump.
Taleb says that Silver’s probabilities would make him a target for arbitrage if they gave the price he would offer on an election bet. Unlike the static version we just considered, the arbitrage would happen as time unfolds. Maybe an investor could “buy” an election forecast when it’s cheap (it claims a low probability of a given candidate being elected) and then “sell” it when the price goes up.
Taleb argues the only right way to do election forecasting is to use the derivative-pricing techniques from mathematical finance. Derivatives are certain kinds of bets on the performance of an underlying security, like a stock. In this case, the “stock” is the voter support for a political candidate, and the bet is on whether that support will exceed a winning threshold on election day, similar to a “binary option” in the financial world.
So, should Silver be scared of losing millions of dollars if he takes Taleb’s bait? In short, maybe but not certainly.
In the 1930s, the Italian statistician and actuary Bruno de Finetti noticed something interesting about these kinds of bets: In order to avoid arbitrage, the prices must obey the same equations as the mathematical rules of probability, meaning such relationships as Price[A] + Price[NOT A] = 1, and so on. Taleb’s arbitrage-through-time was also known to de Finetti. Suppose, for example, on Monday a forecaster assigns a 40 percent probability to the event CLOUDY = “Tuesday’s sky will be cloudy” and a 30 percent probability to the event CLOUDY & RAINY = “Tuesday’s sky will be cloudy, and it will rain,” and these probabilities are interpreted as prices.
Then we ask the forecaster/bookie: “If we wake up tomorrow and the sky is cloudy (and we have no other new information), what price will you give us for a bet that it will also rain?” De Finetti showed that the only arbitrage-free price for this conditional bet “RAINY GIVEN CLOUDY” is the one consistent with Bayes’ rule of conditional probability:
Price[RAINY GIVEN CLOUDY] = Price[CLOUDY & RAINY] / Price[CLOUDY]
Taleb’s wires get crossed, and as a result his argument borders on nonsense.
In this example, the price should be (0.3)/(0.4) = 0.75. If the forecaster gave too high a price, say 0.90, then through a clever combination of transactions:
-
BUY a bet of $1 on CLOUDY & RAINY
-
BUY a bet of $0.75 on NOT CLOUDY
-
SELL a bet of $0.15 on CLOUDY
-
If we see that it is cloudy, SELL a bet of $1 on RAINY
we could make a profit of $0.06 no matter what scenario unfolds. We could just as well make that $6 million, or however much we like, by increasing the amounts until we had emptied the forecaster’s bank account.
Derivative-pricing in a consistent way that avoids arbitrage is a difficult problem (worthy of a Nobel prize)! An extended form of de Finetti’s argument is at the heart of the modern solution, in what’s now called the “Fundamental Theorem of Asset Pricing”: if prices are consistent, then they form a set of probabilities, meaning the equations of probabilities are satisfied.
Unfortunately, Taleb’s wires get crossed here, and as a result his argument borders on nonsense. The converse to the statement above is also true: if prices form a set of probabilities, then they are consistent. Since Silver’s forecasts begin with probability models, it’s safe to assume they obey all the rules, including Bayes’, and would be arbitrage-free. Taleb’s procedure is therefore sufficient but not necessary, making his choice of model doubly unnecessary.
Taleb’s real discomfort with Silver’s bouncing probabilities is less about arbitrage than it is about the relationship between past and future volatility. As I illustrated last month, even for a simple “random walk” model of elections—where a candidate’s support goes up or down every day according to a coin-flip—if the past and future polling volatilities are the same, then a candidate’s chance of winning will have the same amount of variability whether the polls are fairly stable or bouncing around. In both cases, the chance of winning may fluctuate dramatically. Again, since these are exact probabilities computed in the same way Taleb’s are, there is no possibility for arbitrage.
In order for Taleb’s desired steady forecast probabilities to manifest, the uncertainty in the future polls would need to always be greater than the observed volatility of polls in the past. This is a valid “black swan”-like criticism that he could have made but didn’t: that we should not understate future uncertainty given our past experience observing mild fluctuations in the polls; otherwise we might get blindsided by an event like former director of the F.B.I. James Comey’s letter to Congress.
In mathematical finance, derivatives depend on a traded asset like a stock with a known market price. What’s uncertain is whether that price will reach the threshold necessary for a derivative to pay off. In the world of election forecasting, the “price” before election day can only be guessed at based on polling data, so there is uncertainty both around how voters’ opinions will change and also what those opinions are at any given time. Figuring out the latter is what Silver’s models excel at, and this complication makes the whole framework of derivative-pricing arguably inapplicable here, which is no great loss since there was no need for it in the first place.
A “black swan”-like critique of the FiveThirtyEight forecasts is likely possible and even valuable. It may be that investors like Taleb, who think Silver’s probabilities are wrong, could earn a profit on average by betting on them—like taking advantage of a casino offering the wrong odds—but they could not claim those profits were risk-free. But by accusing Silver of arbitrage and trying to apply his financial expertise to elections, Taleb has overplayed his hand this time and is left looking, well, klueless.
Aubrey Clayton is a mathematician living in Boston.
WATCH: This astrophysicist priced the planet Kepler-186f at $655.
Get the Nautilus newsletter
Cutting-edge science, unraveled by the very brightest living thinkers.
