Today in bad pop science: "You’re Probably a Psychopath if You Like Your Coffee Black"

TL;DR: A study tests 171 theories on a sample size of 900 individuals and surprisingly finds a “statistically significant” result. Readers digest accepts this result as gospel to write clickbait.

Reader’s Digest recently put out an article which begins confidently:

Do you prefer your morning joe sans cream and sugar? A new study says you’re probably a psychopath with sadistic tendencies.

It’s your typical “every study in pop psychology is literally true!” clickbait. You would think we would know better since the replication crisis is several years in the making at this point, and started with exactly these kinds of studies.

The original paper isn’t quite as confident, concluding that:

The present research has demonstrated that bitter taste preferences are associated with more pronounced malevolent personality traits, especially robustly with everyday sadism.

Let’s see what the study itself has to say about it.

The study tests 10 different personality characteristics on 10 different coffee preferences, which we can see in table 1, pp.39:

If you count, this is 171 comparisons (AKA theories being tested) on n=953 data points. Odds are you’re going to find something to write clickbait about if your “statistically significant” threshold is the classic 5% or 1% p-value.

For the stats nerds out there, there is something you can do called a Bonferonni correction to properly test multiple theories on one dataset – but unsurprisingly “bonfer” appears nowhere in the paper.