Research Abuse, from people who ought to know better
I’m on a Scientific American audience panel. Recently, they sent us two cover designs and asked us various questions in hopes of figuring out which one is going to do better in newsstand sales.
Someone needs smarter researchers.
No matter how many different ways you ask the same question—which the survey tried to do, clumsily and ad nauseam--you cannot rely on self-reported data or on self-analysis. Even if Scientific American’s researchers don’t understand that—amazingly, most marketing researchers I have met do not—certainly Scientific American itself should understand that. I mean, it’s a science magazine.
The trick to predictive research is not to ask people to tell you what they think—people haven’t a clue—but to get them to reveal what they think through their behavior (while you surreptitiously watch). The way to find out which cover will sell better at the point of sale is to test both covers at the point of sale. There are a number of ways to do this. One would be to place different covers, in advance, in similar stores in test markets. Another would be to send a free issue offer to test markets, featuring one cover to half of the sample and the other cover to the other half. At the end of the test, count.
If time and budget don’t allow for the above-described tests, there are other, valid, extremely low-budget ways to pretest. I am loathe to give them away here. For one thing, it would make this post way too long. For another, I’d rather you hire us.
—Steve Cuno
I’m on a Scientific American audience panel. Recently, they sent us two cover designs and asked us various questions in hopes of figuring out which one is going to do better in newsstand sales.
Someone needs smarter researchers.
No matter how many different ways you ask the same question—which the survey tried to do, clumsily and ad nauseam--you cannot rely on self-reported data or on self-analysis. Even if Scientific American’s researchers don’t understand that—amazingly, most marketing researchers I have met do not—certainly Scientific American itself should understand that. I mean, it’s a science magazine.
The trick to predictive research is not to ask people to tell you what they think—people haven’t a clue—but to get them to reveal what they think through their behavior (while you surreptitiously watch). The way to find out which cover will sell better at the point of sale is to test both covers at the point of sale. There are a number of ways to do this. One would be to place different covers, in advance, in similar stores in test markets. Another would be to send a free issue offer to test markets, featuring one cover to half of the sample and the other cover to the other half. At the end of the test, count.
If time and budget don’t allow for the above-described tests, there are other, valid, extremely low-budget ways to pretest. I am loathe to give them away here. For one thing, it would make this post way too long. For another, I’d rather you hire us.
—Steve Cuno
RSS Feed
