It happens to us all: Your survey question uncovered something profound, but the actual data point doesn’t sound exciting enough. The marketer (or tabloid headline writer, or great American novelist) inside you desperately needs to make the headline better.
You might channel your inner Hemingway and write the most concise copy you can muster. Or you might channel Faulkner and use all the ten-dollar words you know. Either way there’s a good chance your statements are no longer faithful to the data — and that means they’re no longer factual.
The two best ways to make sure your data copy is faithful? Verify the sample and verify the statement:
Verify the sample. The sample (or base or population) reveals who or what your data describes. Specificity matters. So if you survey five dozen cooks in Edinburgh and Glasgow, your base could be “60 Scottish chefs.” If you collect data on everyone who makes frites in Paris and Nice, your base might be “French french fry fryers.” In a recent edition of his excellent Gobbledy newsletter*, my old friend Jared Blank channeled Hemingway, removed crucial detail about the base of his data, and so accidentally misled his readers.
The problem: Jared described the chart above by saying “Nearly three-quarters of people now buy a subscription [ski] pass before the season starts.” Except the data doesn’t cover people, or even skiers. As the chart label explains, this data describes the share of skier visits at Vail. (The crucial details: skier visits, at Vail.) Sure, if every skier visited Vail exactly the same number of times each year there would be a 1:1 ratio between skiers and visits, and this chart would also reveal the share of skiers at Vail. But if we guess that skiers with subscriptions visit more often — a good bet — then it’s likely a much smaller percentage of the people who ski at Vail each year purchase an annual pass.
The solution: The safest option is to quote the data: “Nearly three-quarters of skier visits at Vail are made by people who hold a subscription lift pass.” But while faithful, that’s also somewhat boring. A pithier option: “"On any given day, nearly three-quarters of the people stuck in the lift line at Vail hold a subscription ski pass.”
Verify the statement. Last year I helped a software company research their customers’ career prospects. (Side note: If you want to generate lots of leads, publish a white paper revealing your audience’s income and job satisfaction.) The survey asked, “Do you plan to leave your job in the next year?” But when my client drafted their report, the headline screamed “X% of your peers are in their jobs for the long haul.”
The problem: With apologies to the millennials who read this, planning to stay at your job for 12 months doesn’t mean you’re in it for the long haul. (Planning to stay for 10 years? Definitely. Five years? Sure. But if you’ve only committed to your employer for the gestation period of a bottlenose dolphin, you’re not exactly a lifer.) So although that copy made for a great headline, it wasn’t faithful to the data.
The solution: Once again, the most basic fix is to quote the data: “X% of your peers don’t plan to leave their jobs in the next year.” But that’s not very exciting. To change the headline to something that’s both punchy and true, I suggested “X% of your peers don’t plan to go anywhere.”
*Despite this slip, Gobbledy is one of my favorite reads. Jared examines and improves marketing copy the same way I examine and improve data. But he’s significantly funnier. Go subscribe.
Thanks for reading. If you’ve seen other unfaithful data claims lately, post them in the comments below or on LinkedIn, and we’ll suggest solutions. And if you want good data delivered to your inbox, subscribe here.