Wednesday, January 6, 2010

Pollara & the CRIA Part Ways

Michael Geist reports that Pollara, the former chief polling firm for the CRIA (Canadian equivalent of the RIAA), has changed its stance on the interpretation of data it collected for the CRIA on music downloading in past years.

Pollara Executive Vice-President Robert Hutton in this comment on Zeropaid, notes that relying on 2006 research is "dubious at best." he goes on to say (edited for brevity, click on the links for the full text):
There is a significant body of market research from around the time downloading became a significant issue suggesting not only that downloaders were perfectly willing to pay, but also that dowloading to some extent promoted actual purchase of the tangible product.  [...]
That the industry then chose to fight with it’s customers rather than address their needs merely exacerbated the problem. Perhaps they didn’t know any better, and in their desperation to hang on to that CPG model of selling music, they took the wrong route.  [...]
[...] The industry is very much to blame here, because they ignored their customers needs knowing full well that a technological revolution was at hand that would enable their customers to satisfy their needs, with or without them.
And my favorite quote:
Trying to force hamburgers on a group of vegetarians might work in a famine, but long term, just cannot be a successful marketing approach.
Mmmmm.  Hamburgers.

7 comments:

  1. I'm struggling to get the connection between vegetarians and P2P technology... Oh, wait - iTunes = McDonalds?

    ReplyDelete
  2. Well, I am struggling to find any comment I have made that could possibly be construed as saying "Pollara has changed it's stance on the interpretation of data it collected..." or the inference that because Pollara hasn't done any research for CRIA in a few years that means "Part Ways"...

    If Ingmar can enlighten me on that, it would be greatly appreciated.

    ReplyDelete
  3. Robert, thank you so much for your comment. My apologies for not replying more quickly - I went to double check my sources for this article and found that you are a prolific commenter. I had much reading to do.

    Due to the length of my response and the max. character limitations on comments, I'm going to have to break this up into two posts. My apologies.

    Response to Robert Hutton, Part 1

    This originally came to my attention (as cited above) in Micheal Geist's article, "Pollara Changes Its Tune On Music Downloading." In it he writes:

    "What a difference a few years (and a change in client) makes. CRIA and Pollara parted ways soon afterward and current Pollara Executive Vice-President Robert Hutton offers a decidedly different take on the issue."

    Admittedly, I did paraphrase Michael in my article and though I intended no distortion, each paraphrase varies the message in subtle and often significant ways. It's like the old schoolyard game Telephone, except an Internet version of it. Perhaps we ought to call this version game Tubes :P

    So as to your original question - those statements were my own rephrasing of Michael Geist's words. All emphasis is mine and if I did anything to distort or misrepresent the truth, I expect my readers to hold me accountable for that. As such you did, and I appreciate it.

    So as I read your numerous responses to Michael's article, my summary of your comments would be this: The spin is in the interpretation of the data your company provided. Pollara only provides the data. The spin is provided by the clients and critics.

    Perhaps a rather terse summary for such a nuanced issue, but that's why I provide backlinks for readers to get the full story. Please correct me if I'm wrong or feel free to expand your response if you think I missed a salient point.

    continued in next comment …

    ReplyDelete
  4. Response to Robert Hutton, Part 2

    So now that I have your attention, I have a comment and a question for you.

    First, I would like to commend you on your public stance on the broken business model of the recording industry in the face of disruptive technologies. A dispassionate analysis of many issues faced by artists, consumers and the industry is a rare and good thing.

    Second (and this is the tougher bit), my reading of your response to Michael is that you view the data you provide to your clients as neutral (with nuances). Once again I admit that to be a gross oversimplification, but think of it as a lead-in to my question about the neutrality of that data.

    To start, I presume that all reputable pollsters make extensive efforts to minimize the effect of question wording, question order, sample selection, etc. and as such I presume your organization to be reputable, having no evidence to the contrary.

    Late last year Nate Silver exposed Strategic Vision, a major American pollster, for huge statistical variances in their data favoring their client's desired outcomes. More recently he saw similar oddities, though less extreme, in the data coming out of Rasmussen, but to his credit he concluded that making a call of intention bias was marginal at best, even in the face of so many others crying foul.

    Your industry, like most, is competitive and there is no doubt that some firms are willing to 'bend' their results to get clients. Sometimes a little. Sometimes a lot. Clients who hire pollsters seem to be in the habit of selectively releasing polling data in whole or in part to best support their position. It's no surprise that this leads public sentiment to agree with Samuel Clemens.

    So how can we (the consumer, the blogger, the interested party) tell when data is reliable?


    Thank you for your comment and I look forward to your response.

    ReplyDelete
  5. It's an interesting topic and I admit to some reluctance to wade in.

    Let;s start by proposing that even the mere act of asking a question can cause a bias. After all, the fact that a question is asked implies that someone must want an answer - right? That can be a bias in itself. Purely functional factors can introduce bias - time of day, tone of the interviewer, the mindset at that moment of the respondent, the phases of the moon...

    And at the end of the day, facts are a living thing. They are not always static - they live within the current social climate and within people's frame of reference and their own preconceptions and predispositions.

    Mechanically, Pollara as well as any other competent research firm knows how to do random sampling, how to word a question, how to order them and how to deliver those questions in a very unbiased way. But that's data collection mechanics.

    The insights taken from that data are another matter.

    What we are seeing already here is a wide variation in interpretation of the comments I made on Zeropaid. You read that to be Pollara disavowing the data it produces some years ago. Geist interpreted as Pollara changing it's tune. Others interpreted it as Pollara and CRIA being at loggerheads. Others saw it as a Pollara executive come on to give some context to the debate. As you can see, facts are a rather fluid thing.

    Let me give you an example. Some years ago, I recall some data (I don't know if it was Pollara's or another firms) stating that Canadians are heavier downloaders per capita than many other countries. Fair enough - a fact is a fact, right?

    But then again, was the survey conducted by phone or online? I suspect a higher incidence of downloaders would be available to respond online that those responding by phone. But likely there is a bias through either methodology, and to do it right would seem to me to call out for a mixed methodology.

    That finding was widely reported in support of the advocacy of tighter laws protecting the recording industry against illegal downloading in Canada.

    However, around the same time, there was a seismic change in the Canadian retail landscape. Tower in Toronto went out of business, and the dominant Sam's chain went bankrupt, leaving HMV as the far dominant chain in a near monolopy position in many markets.

    HMV quickly took action - they eliminated their loyalty card, dropped their liberal returns policy, upped prices, got rid of the knowledgeable music staff and replaced them with lower cost staff with little knowledge, cut back on stock at the store and so on. They sensed with the demise of their main competitor that they could get away with that.

    How much does that contribute to Canadians being more prone to download? Was their any examination of this?

    And overall, has the recording industry properly addressed the retail channel and it's role in the demise of physical sales?

    So overall, I hope this gives some insight into the role and limitations of survey data.

    I continually see polling results in the media where I just bang my head against the table three times in frustration.

    Recently, I saw a poll on the Afghan torture issue stating that a majority of Canadians believed the Canadian army is mistreating detainees in Afghanistan. That was widely picked up in the media as Canadians being concerned about this and how it is a growing issue. Hard to dispute that - right?

    Wrong. The real question should have been - 'do you care?'. Had that question been asked, I suspect a majority would have said 'no, not too much'.

    Interpret what you read in the media very critically, ask yourself whether the right questions were asked, ask if you are seeing the whole survey, and then put it in the context of what you know and events as they unfold around you.

    ReplyDelete
  6. Robert,

    Thanks so much for your response. It's great to have some inside insights from the polling industry. I feel a little smarter now :)

    I'll post a followup to this article shortly calling attention to your comments here and on Michael Geist's site.

    If you have any more insights or comments, we would be more than happy to provide a forum for you here.

    One more question - how did you ever find this site? This is quite a new blog with relatively little traffic and analytics tells me that, apart from our domain name, we don't yet register in the top 20 hits for any search.

    ReplyDelete
  7. Ingmar, thanks, and I'm glad you and your readers find my ramblings useful.

    I have a feed when our company name comes up, that's how I came to your interesting blog. I'll be happy to chime in from time to time.

    All the best!

    ReplyDelete