Preaching against survey data

Posted by on Sep 15, 2008 in Uncategorized | 2 Comments


Pic from Orchard Lake

A number of times over the last decade, I've found myself publicly questioning the value of the huge investment marketers and other folk make in Ask/Answer market research (that's surveys and focus groups) – for example on the grounds of its spurious, positivist transparency (it provides very plausible accounts of why folk do stuff while the real mechanisms remain necessarily hidden to those witnesses given how our minds are constructed).

I've weighed in against scientism and the false sense of precision that many vendors promise; I've warned against the cognitive biases in research users' minds that skew the value of whatever data is served up to them; I've argued against using research to tell you what to do (rather than helping you refine your ideas).

It didn't make me very popular – particular in research circles. At MRS a couple of years ago, they ran a wisdom of crowds poll with Brainjuicer on what delegates thought was going to be the most influential paper and I came just about last. Comments included, "wrong, wrong, wrong!". Things got so bad at one point that I even found myself described by Research Magazine as Public Enemy Number One

…which I've always thought a bit harsh: I'm in no way a "touchie-feelie", "it just kinda feels right", "crystals will tell us" kind of marketing thinker. No, I think disciplined and evidence-based stuff is the only sensible way forward – we just need better (and perhaps less) research approaches which harness what we now know about human beings and not more and more of the same old stuff (This btw is why I rate the folks like Brainjuicer and Spring/Mesh so highly – they're trying to provide better insights to their clients by turning the new behavioural science consensus into practical research methodologies. They're not alone of course, but they are unusual)

So nice to see today that the ARF and two big clients seem to be arguing very much along the same lines as I've been arguing for: to be more observational than interogative and in their world and against their agenda and not ours.

"Our consumers have been sending us messages for years that they're not interested in the traditional survey process,"  says Kim Dedeker of P&G. That's right: co-operation rates (folk agreeing to do a survey are at an all time low: 4 out 5 say no)

Makes me seem (spuriously) that I had some kind of influence on this….maybe I've just got my timing wrong?


  1. tom
    September 15, 2008

    Don’t mean to blow smoke up your arse, but I’m sure you had a role to play. I’ve seen the Death of Marketing on many an influential bookshelf. (Herd, not so much.)

  2. Nick Rothstein
    September 15, 2008

    Should we not think even more upstream. Does it not start (always) with the people we hire and how we train them?
    Output being input and all.
    the link shows nicely who would like Mr Earls approach better and who would not..