8 Obvious Signs Statistics Are Lying To You

8 Who's Sharing It?

The first way to check data is to check who's sharing it. This sounds unscientific, as we're meant to respect the data instead of being biased by who's presenting it, but it's already been biased by their presentation of it. Science only works when you use the whole thing. You have to consider all the results, not just the ones that support things you like. But pundits Google titles that match what they already believe, then wave quotes out of context to claim the power of SCIENCE. It's like ripping the shiniest part out of a Lamborghini engine and claiming you have the ability to sprint at 300 kilometers per hour.

Properly prepared scientific data is free of bias, but most of that preparation is boiling out those biases, intentional or otherwise. The whole point of numerical analysis is not screwing up the second we get emotional. Which is why you have to ignore people who are already frothing at the mouth when they start screaming about the statistics backing them up. Even if it sounds interesting, you should take the title of the paper and go read it at the source. If a study has truly proven that 100 percent of all pies are infected with diarrhetic toxodeathmosis, you're going to hear about that shit on the news, not just on Crazy CakeFan Karl's "Rather Die Than Pie" Cake Hour on 106.4 CAKE FM.

7 Who's the Source?

There isn't a special "A Study" committee convening to make sure that only eminent scientists are allowed to release bar charts and percentages. "A study" just means "some people wrote some shit down." A child with a calculator can calculate up some numbers just as easily, and you might be 5,318,008 times as compelled to look at them when they're twisted around, but using numbers doesn't mean they're smart. It's your responsibility to check the source before getting upset about what they say.

The average Internet infographic is a full page of graphic design, one sentence of numbers, and a tiny bottom-right corner of source written in sky-blue letters on an aqua blue background. You don't see who's actually making the claims until the very end. Which is weird. In real life, we check who's speaking before listening to their entire spiel. We don't evaluate a half-hour presentation on the problems with current capitalism and how it's the fault of the foreign carp weevil infesting the ears of our politicians through imported furniture in government buildings, and only then check whether we're listening to the British Broadcasting Corporation on the radio or Big Bearded Carl after a bottle of homemade bleach-wine.

First, check if the figure provides a source. If it doesn't, it's garbage. Then, check the source. It might be the World Health Organization, or it might be the American Medical Liberty League for Living Whole. One of those is a specialized expert agency of the United Nations. The other is just a collection of words hoping you think they sound nice. But even the nicest name could be claimed by Bond villains. Talking about wholeness and liberty doesn't mean they're not fundamentalist sects dedicated to the incubation and release of biological plagues on the civilian population.

6 What's the Sample?

The latest study of modern work environments revealed that 33 percent of staff displayed increased productivity when they drank Manhattans, and 100 percent improved productivity when they weren't wearing pants. I know it's the latest study, because I just did it. The sample was me and two cats.

If the sample size isn't given in the same breathlessly excited paragraph as the results, those results are potentially garbage. If the sample size isn't given at all, those results are definitely garbage, rubbish intended to bury your brain under spoofed input and steer you toward the writer's opinions instead of informing you to make your own.

There have been some amazing cohort studies over decades of medical professionals, gigantic efforts to gather priceless medical and sociological data. There have also been wastes of potential toilet paper where bored students walked around campus one sunny morning and decided, "We won't just ask the people who can afford to come here, we'll ask the ones who don't even understand that this has to be paid for and are screwing around instead of being in class." Lo and behold, the resulting survey shows that "people" don't really need food stamps. When all it really reveals is that arseholes are involved in every step of that paper's publication. From the source that thought like that, to the writer who wrote about it, to the publication which accepted the paper for a submission fee instead of for peer review.

5 What Are the Error Margins?

How large are the error bars on the figure? If they're small, the data could be useful; if they're large, the data is less accurate but still useful; and if they're zero, the data is garbage. Exact numbers sound more scientific, but at a sociological level they only reveal innumeracy. We can quantify the precise percentage points on particle physics experiments, but anyone claiming that anything on the human scale is exactly 37.8 percent is trying to garner your respect by reciting sacred digits. You couldn't measure exactly 37.8 percent of anything human even if you used only one of them, a good ruler, and an extremely sharp knife. The act of cutting would cause extra percentage points to squirt and slop all over the place.

I used to compile data on the depth of holes blasted in material by lasers. A clear length under literally microscopic examination, and laser-blasted craters are the most definite mark you can make on anything with modern technology. We still included error bars in the results. Because it's not possible to measure things perfectly. It's still possible to get useful results by accounting for error, but it's also possible -- and much, much easier -- to mislead the world by ignoring it. For more detail, here's a great post on the importance of error bars and a detailed follow-up, explaining how they help you avoid being fooled out of $20,000 by bananas.

Or here's an example: I offer you a million dollars to wear a bulletproof vest with a 100 percent bullet-stopping record while I fire one bullet at it.

Would you take it? Oh, didn't I mention that my measurements had an error of 90 percent? Because I'm sure as hell not going to tell you that if you don't ask.

So, the effectiveness might be as low as 10 percent, and even then, that's only if I'm telling the truth. How good does the job look now? People assume that not mentioning any errors means it's accurate. Not mentioning any errors means they don't want to show you the errors. And you should really want to know why.

4 "Latest" Does Not Mean "Best"

An easy way to generate contrary headlines is to act like the latest study overturns all previous knowledge instead of adding to it. Studies aren't iPhones: The latest ones aren't automatically (if only slightly) superior. If you've already got a million studies suggesting something, one claiming the opposite isn't a sudden shattering of that misconception. It's making an extraordinary claim in the face of everything we've ever known, and had better present extraordinary proof.

You'll most often see this with food. Tabloid sites will scream about eggs being good for you, then bad for you, then good for you again, as if chickens were laying alternately good and evil eggs just to mess with their masters.

Opponents of science (aka "people who know it disproves their bullshit") try to use this to discredit science. They use alternating studies to erode confidence in scientific progress, claiming that the "boffins" are flip-flopping back and forth and can't make up their minds. But changing your opinions based on new data is the whole point of having a mind in the first place. Absolute faith in a single idea without accepting new input is how you use your higher functions to simulate brain damage.

You can only overturn previous assumptions with proof: Develop a theory which explains the results, design an experiment where the assumption does one thing and your theory does the other, and run it. If you claim that gravity is really a species of really fast gremlins pushing everyone down, you can't claim that 100 percent of respondents in your science-tastic institution (bedroom) concur. You have to design an experiment. And if you run it, and it ends with an adamantium claw clutching a gravitational goblin, then yes, we'll all have some serious rethinking to do. But until then you're just a crazy person.

3 Up to No Good

What's the point of numbers if you can't use SCARY numbers? Answer: That is the exact point.

The study in the above image only suggested a possible link between altered sleep patterns and stroke, one which required further study. The headline writer just helped it along to SLEEP WILL DESTROY YOU.

The point of analysis is to get answers immune to emotional contamination. This is where we see headlines with "up to." This thing you do every day could increase your risk of spontaneous genital collapse by up to 100 percent! And you're already too busy being horrified imagining what "spontaneous genital collapse" even is to realize that "up to 100 percent" includes every number below that. Especially zero. Which is nothing to get excited about.

"Up to" is how people justify focusing on outliers and extreme cases, even though that's the opposite of using statistics in the first place. "Sure, the math says that this case has less than a 0.0001 percent chance of ever happening to anyone in the world," says the clickbaiter, "but that's not scary. Besides, if I don't write an article on every bullshit study that crosses my desk, I'd have to do more work to generate enough articles."

2 After-Market Mathematics

Sometimes the numbers just aren't scary enough. But the writer has a calculator right there just below Solitaire. So when a study says there's only a one in a million chance of anyone in the world ever developing the corrupted blood plague for real, the writer multiplies that by 7 billion people and THOUSANDS OF VIDEO GAME PLAGUEMONGERS ARE SHAMBLING IN THE STREETS!

It doesn't matter that that factor was already included in the original number. Or that the original figure was developed by people who knew what the hell they were doing, while most Facebook fear figures are generated by people who rely on spellcheck even though spelling words in a row is their only job.

If you could just multiply statistics by bigger numbers, we wouldn't need statistics. Because we'd all be dead from the thousands of appallingly unlikely things which suddenly killed us. The instant a writer starts expanding on the statistics, you should act like they're talking about turbo-charging their car's engine with some nitroglycerin they found. They're not working with what they think they're working with, they don't understand the terms, and you should not go anywhere they're trying to take you because it'll blow up in their face.

1 Factors of Bullshit

Does the story explain exactly how likely something is, or does it stress how much more likely it has become? Because that makes it more likely it's bullshit.

They extracted that headline from a study where the key factors were "17 people out of 4,045" and "we get a nosebleed even trying to spell correlation or causation."

Every time I stand up, I double my chance of being brutally murdered by an Ultron robot body being punched horizontally through several buildings by the Hulk, leading to severe injuries, pain, and disability for the rest of my life, possibly even death, because I've doubled my target profile compared to sitting. But all that talk of how horrible it would be if it happened doesn't change how amazingly unlikely it is in the first place. It doesn't cause me to spend my life sobbing in a huddle in an adamantium-laced basement.

This is most often seen in health arguments, where it's screamed that something now triples a 0.0000001 percent chance (among a study of 200 middle-aged men who had nothing better to do than fill out an bacon survey).

Statistics can uncover amazing results. They operate at a level far beyond our ability to picture, because the human mind simply cannot deal with the idea that there are 7 billion anything. But stats are a mathematical crowbar: a very useful tool in the hands of smart scientists, but one which also can be misused to hurt people and take things and get into positions people shouldn't.

Follow these rules and you'll be shielded from most of the bullshit. Yes, that will mean ignoring most of the things you see shared online, but that's OK, because the only universally applicable statistic is Sturgeon's law: Ninety percent of everything is crap. The work is finding the other 10 and building a world out of it.

Enjoy the most powerful computer product in existence with The Greatest Gun in Gaming History, or avoid more electronically bad advice with The Worst Romantic Lessons From Video Games.

Enjoy more online idiocy with 5 Online Petitions That Prove Democracy Is Broken or learn how to learn better with The 7 Dumbest Things Students Do When Cramming for Exams.

Luke knows what the other 90 percent of your brain is doing, tumbles, and responds to every single tweet.