[Article] The fallacy of broken records

[RandBlade]

Sometimes in debates on various issues, either side of the debate can claim a broken record as strong evidence/proof of their point of view - whether we're talking about politics, economics, science, sport or anything else. The reality is that we have enough events and ways of measuring them that simply from a normal distribution records will regularly be broken in something, without being anything else than just normal. If you highlight the specific issue after the fact then the odds of it happening may have been extremely low - but we don't however notice the absence of anything special happening, that gets taken for granted, so just the extremes get noticed and assumed to mean something strange is happening.

As a sporting aside (to avoid tarnishing this with a specific political debate), I am a fan of Test Cricket. Watch any Test series, like The Ashes, and you can virtually guarantee that in any series a lot of records will be broken - though the types of records can vary and sometimes get grounded upon specifics eg to give just three from memory: "Highest sixth-wicket English partnership in Brisbane", "Most runs scored by an individual over a series since 1922", "First time ever the top 3 batsmen in an Innings have all scored a century". All no doubt meant much to the individuals afterwards and are entertaining to know but it is one of those quirks. Had any not happened, nobody would have asked beforehand whether those would have happened. The more caveats you add the easier it is to reach a record; expand it into including a record by being a "top 10" of an event historically and you reach far more. Every Olympics someone sets a World Record in something.

Records can obviously be evidence for something, perhaps. But they are not proof and may not even be significant.

[Crowheart]

Repeating things over and over like a broken record as proof, is also a fallacy.

[wiggin]

I'll ask it again for everyone: how would you determine whether a record is significant? Certainly some unusual events are paradigm-changing, but others fade into obscurity. How might we assess whether a given extraordinary occurrence is indicative of a new trend or a changing reality, or just a statistical fluke? Might the existence of a statistical fluke itself (e.g. an unlikely but very strong earthquake) change our behavior even if it isn't indicative of a trend? Our society is often geared against smoothing out statistical uncertainty - testing products to resist failure even one in 10,000,000 times, establishing building codes for once-in-a-millenium events rather than just day-to-day activities, buying insurance for unlikely but devastating personal or societal calamities, etc.

I would agree that an unusual event can't be used as an existence proof for a new trend (though of course it can serve as an existence proof for itself), but they are far more important for changing the way we think about and respond to the unlikely rather than the 'new normal'.

[Nessus]

Certainly some unusual events are paradigm-changing

This is usually only obvious well after the fact, and in our hysteria-driven culture a whole bunch of things are hyped up as something they're really not (just look at the unfortunate events around Fukuyama's End of History), and while certainly some people would want these things to change how large segments of society behave, it's just not going to happen. Hitler's Germany was consigned to the lowest Hell by the people who proceeded to raise the number of victims in their coming war to end all wars to ten to the power of eight. What was the change, and in which direction?

[Loki]

The problem with paradigm-changing events is not just that you can rarely see them ahead of time, but that whether they are indeed paradigm-changing is a matter of perception. For example, we've just seen the first popular toppling of an Arab ruler in modern history (perhaps all of history). What does this mean? If a sufficient amount of Arab rulers and subjects decide that this event was paradigm-changing and signal the end of Arab autocracy, then we might very well have many revolutions on our hands. If, however, such a view does not become widely held, the event will prove to have little significance (the likely result is that some will believe the former and some the latter, which is going to lead to violence between the two groups).

[Aimless]

I think it's safe to say that if the future of my world may hang in the balance it's more significant than anything that happens in test cricket.

[wiggin]

I think it's safe to say that if the future of my world may hang in the balance it's more significant than anything that happens in test cricket.

That's poor logic. In American debate (as a sport, that is) there's a frequent reductio ad absurdum argument where by some convoluted process they explain that their opponent's position may lead to nuclear war. Thus, you'd have to be crazy to support their position, right? The problem is that the chain of events getting there might be wildly unlikely, and that a similar chain of logic could be used for the opponents.

In reality, we have to look at probabilities. Say you have a binary choice between averting a 0.1% of nuclear war and a 40% chance of a worldwide recession that starves 10 million people to death (along with increasing the number of resource wars, yadda yadda yadda). I'd go for the 0.1% chance of nuclear war, personally.

On most policy decisions the future of the world does hang in the balance, at least theoretically, but that's all the more reason to be careful in our analysis rather than making unsubstantiated claims one way or another.