• Home
  • Lumpy Mail Experts
  • Damn Good Writing
  • Need a Speaker?
  • Clients
  • Links
  • RESPONSE Agency Blog
  • Talk to Us
How to Do Unreliable Research 12/02/2009
2 Comments
 
According to a new Adweek survey, 50% of respondents say that advertising "rarely" motivates them to switch brands, 11% say that advertising "never" influences their brand choices, and people 18-34 say they are most likely to be "swayed by ads." 

I suspect there may be some truth in Adweek's conclusions. But, if so, their methodology doesn't establish it.  

The worst way to learn what influences people is to ask them. People don't know what they do, why they do it, or what makes them do it. But they sure think they know! So, when you ask, they'll give you an answer, and most likely they will mean what they say. Trouble is, they are often mistaken. 

For a valid, reliable way to learn what influences people, put them in a real situation. Do not let them know that a test is afoot or that anyone is watching. Then note what they do. 

Our firm recently conducted such a test for the parent company of two national brands that you have heard of. They wanted to know which of the two brands was stronger. We divided their market into three segments. All three received identical offers; the only difference was whether the offers came from National Brand A, National Brand B, or Unknown Brand C. Then, we counted orders. All segments bought at the same rate. We repeated the test and obtained the same results. Post analysis showed that the brand made no difference; only the offer did.  

I might add that no one in the parent company was pleased. Some had championed one brand, some the other. The finding that neither brand had power offended both camps. Getting to the truth doesn't always make an agency popular. 

I should also add that this test in no way proved that brands are powerless in general. This was but one case. But it certainly suggests that you should challenge whether your brand has as much power as you'd like to believe.

Back to the research. Since I suspect Adweek was on to something, you might ask, why quibble over the methodology? The reason is that I can't be sure of a conclusion, even one that sounds reasonable, that is attained by faulty methodology. When I was in grade school, I heard that mixing yellow and blue paint makes green paint. I verified it by stealing my older brother's water color set (don't tell him—he still doesn't know) and mixing up some green for myself. I could have flipped a coin. Heads, yellow and blue make green; tails, they don't. I might have gotten the correct answer. But there would have been a 50% chance of getting the wrong one. 

I deal with the science behind fallacious versus reliable research in chapters 8 and 9 of my book Prove It Before You Promote It, so I'll stop here in deference to those who have read the book, and to try and influence those who haven't to buy copies for themselves and all of their friends. Right now, if you please.

Steve Cuno
 


Comments

towr
12/04/2009 03:11

If you want to compare the strength of two brands, shouldn't you make people choose between them? i.e. give the same market segment both choices, and see which wins. Because if each market segment gets one choice, then it seem to me that the result you get is the demand for that kind of product, not for a brand. At best you can say any special offer trumps brand loyalty.

Reply
Steve Cuno
12/04/2009 08:26

Good question, and it deserves a longer answer. (I may make this the subject of a future blog.) The short answer is, each approach reveals slightly different things. The approach we chose revealed whether it mattered if the client sold under Brand A, Brand B, or under no brand at all—in the absence of another choice. The approach you suggest is useful for determining which brand would prevail if consumers were asked to choose between them.

Reply

Your comment will be posted after it is approved.


Leave a Reply

    RESPONSE Agency Blog
    RESPONSE Agency - direct mail and lumpy mail experts Salt Lake City Utah

    RSS Feed

     | 
    Follow SteveCuno on Twitter
    Picture
    Picture
    Picture
    Click here to read Steve Cuno's column in Deliver Magazine
    Picture
    Picture

    Archives

    January 2012
    December 2011
    November 2011
    October 2011
    September 2011
    August 2011
    July 2011
    June 2011
    May 2011
    April 2011
    March 2011
    February 2011
    January 2011
    December 2010
    November 2010
    October 2010
    September 2010
    August 2010
    July 2010
    June 2010
    May 2010
    April 2010
    March 2010
    February 2010
    January 2010
    December 2009
    November 2009
    October 2009
    September 2009

    Looking for older posts? CLICK HERE.
    Picture
    Click here to read the official newsletter of the James Randi Educational Foundation, for which Steve Cuno is an occasional writer.

RESPONSE Agency, Inc. • 936 Granite Peak Drive Suite 1100 • Sandy UT 84094 • Phone 801-352-9100 • Email info@ResponseAgency.com