Bayesians, Frequentists, and Lance Armstrong

Citation:

Paxton, Nathan A. 2012. “Bayesians, Frequentists, and Lance Armstrong.” The Monkey Cage. Copy at https://tinyurl.com/y3ye7evb

Full Text

Lance Armstrong has returned to the news, and the Tour de France is upon us in just a few more days. For those of you who don’t follow professional cycling, most likely the first thing that comes to mind about the sport is doping. Indeed, that’s why Armstrong popped up in sports pages again, since the United States Anti-Doping Agency (USADA) has begun an investigation of him (after the Justice Department dropped its case against him last February).

Rather than consider whether all these charges, counter-charges, and counter-counter charges are true, let’s talk about a couple of different ways that social scientists might think about l’affaire Armstrong. (As justification for the break in election programming, I’d like to note that one of the founders of this blog was just a bit of a cycling nut.)

As social scientists, at least as regards what we can empirically assess, we tend to make statements of probability rather than fact. So rather than say that Armstrong did or did not use performance enhancers, we would talk about how likely versus not likely it is that he used the substances. It frustrates many people that we rarely make categorical statements, but we’re trying to be honest about what we know and don’t know.

The two major approaches to this sort of reasoning are probabilistic/statistical in nature, and we generally refer to them as “frequentist” and “Bayesian.” A “frequentist” viewpoint is the basis of almost any basic statistics class you took in college or grad school (unless you became a social methodologist or statistician). Basically, it asks, “Given an infinite number of trials or experiments or tests, what is the probability that the results I am getting are true?” At a sufficiently small, agreed-upon threshold (1 out of 20 or 0.05 in the social sciences), a frequentist would accept or reject the “null hypothesis” (the proposition that nothing actually happened even if the data said otherwise). They’re called “frequentists” because they assess how frequently a phenomenon “should” occur.

In the case of a doping test, a frequentist would look at the number of performance enhancing substance (PES) tests that Lance Armstrong has taken (lots, and all negative, so far as we know), seen that they are all negative, and say, “The probability of a false negative is small but possible. Under repeated sampling (which is what each drug test is in essence), we become more and more sure that we are getting the ‘true’ result.” This understanding of probability is what underlies the case from Armstrong’s camp: he’s had ALL of these tests over years, they have ALL been negative, and so it is virtually impossible that he could have been using performance enhancers. The frequentist perspective relies upon holding probabilities for some event constant, like getting positive test results, but those probabilities are based upon specific conditions or assumptions that may not hold.

Bayesians, on the other hand, look at evidence differently. The world, for them, can be divided into “priors” (what you know or educatedly guess the world is like), “data” (information you collect and assess), and “posteriors” (your revised beliefs about the world, which can be thought of as the combination of priors and data). Posteriors come from the combination of priors and data.

Bayesians also like to “iterate.” Posteriors beliefs from a situation can become the prior beliefs for another round of data examination.

Bayesian-inclined cycling fans might look at the brouhaha this way. Armstrong has never failed a drug test, of which he has taken more than 500. Armstrong was at the very top of this grueling sport for many, many years (7 Tour de France titles). Armstrong’s greatest, most consistent competitors — like Italian Ivan Basso, German Jan Ullrich — and potential American heirs — like Tyler Hamilton or Floyd Landis — have all been found to have used PES. If these are the only people who have been able to keep up with Armstrong over the years, and they have been found to use some form of performance enhancement, then we may have more reason to think that something doesn’t quite add up.

    • Prior: LA did not use performance enhancers because the tests show super-low probability.

    • Data: All significant competitors used performance enhancers and tested negative, until they were caught (sometimes via a test, sometimes via old-fashioned police work).

    • Posterior: Perhaps the tests’ probability of sussing out those who use performance enhancers are wrong. Revise those probabilities, in light of what we now know, make them the new “priors”, run the tests on the data, and assess how probable it is that Armstrong used those substances.

My own interpretation: If Armstrong did not use performance enhancers, then that means he’s even more extraordinary than previously reported, since he won unassisted against those withPES assistance. It seems more plausible that there’s either something wrong with the tests or that LA used performance enhancing substances than that he becomes an even greater statistical anomaly.

Why can we say this? Given what you might call the “hard” results of LA’s tests and the “soft,” circumstantial results of his competitors, iterating the posteriors and making them priors is exactly what we want to do.

The tests that these cyclists take tell us something not only about the cyclists, but about the tests too. Since we know that many of these riders were being regularly tested and passed the testsuntil they did not, and we gain a better appreciation of how well the tests detect people who are doping. It’s pretty commonly accepted in sports discussion that the EPO test has a high false negative rate (athlete tests “clean” even if using PES), even if no one seems to know the exact rate. The updated/iterated prior confirms that idea.

A Bayesian perspective can more easily contend with a world where athletes are actively trying to mislead, hide, and evade detection. When we know that competitors are trying to hide, that they did use banned PES, and that previous tests didn’t catch them, we can use the collective information about the population of athletes to make more real-world accurate  statements about the probability a particular athlete’s results are right.

Importantly, I think, this “light” Bayesian perspective on cycling and Armstrong gives us better leverage for thinking through the problem. Adopting the perspective does not mean that you think Armstrong used PES. But it does mean that one has to take into account more information about the world (of cycling, at least) than just how likely a particular test result is — one also needs to know how well the system is being gamed, and how well the tests caught demonstrated users. We can get some indication of that by looking at Armstrong’s peers.

Even if it turns out that Armstrong used performance enhancers, it may not diminish his Tour de France accomplishments. If he won using these substances while all his significant competitors did too, that still may mean that he’s the better cyclist. On the other hand, given how much he has made of never using banned substances and competing clean, it could significantly hurt his charitable works on behalf of cancer patients.
URL: http://themonkeycage.org/2012/06/26/bayesians-frequentists-and-lance-armstrong/