Posts Tagged ‘Qualitative Market Research’

Advertising Communication Checks: Valuable or a Necessary Evil?

Posted on: July 26th, 2017 by doyle

adcommchecksFrom the desk of Kathy Doyle

I’ve been conducting advertising communication checks for over 30 years, and one thing has not changed…   most of the parties involved dread them. The agency doesn’t like seeing their creative work questioned based on input from a small number of research participants in an artificial setting.  The client does not like navigating the politics of getting the job done, all the while knowing the agency is less than thrilled. And no one really likes sitting in a back room, or in front of a computer screen, for hours on end listening to the same questions being asked every 20-30 minutes.

Yet there are some very compelling reasons why we continue to conduct communication checks:

  • To make sure we haven’t lost sight of who the target is, keeping our finger on the pulse of how best to communicate with them, and mitigate coming across as pandering or tone deaf
  • To make sure we haven’t missed the mark on messaging, and mistaking what we thought was crystal clear for a totally unintended meaning
  • To make sure the visuals support the message, rather than conflict with it
  • To make sure that brand/product recall is strong. It’s great if people love the ad, but if they don’t know what it’s advertising, what’s the point?
  • To optimize (or eliminate) executions prior to quantitative testing and/or final production. Why not find out if there are ways the execution can be tweaked to strengthen it before spending large sums of money?

Clearly, I’m coming out on the side of considering communication checks valuable.   To maximize their value, here are Six Tips for More Productive Communication Checks:

  1. Limit exposure to three executions per respondent, to prevent fatigue from clouding candid feedback
  2. Video storyboards with audio are acceptable; a complete video (albeit rough cut) is better; don’t make the consumer work too hard to see the idea
  3. Consider exposing the ads in a clutter reel to more closely simulate a real viewing experience and more accurately assess breakthrough
  4. Keep them 1:1 for the most honest commentary. People rarely watch programs or web surf with others, let alone strangers!
  5. Keep them short (20-30 minutes) to prevent over-thinking and to be efficient. We often do 12-18 interviews per day!
  6. Consider conducting the interviews online rather than in-person. When people are at home, they are more relaxed and more likely to provide candid feedback.  Use a platform built for research for the most problem-free experience.

One rule to keep in mind: Avoid using communication checks to kill a creative concept. Not only is the sample size too small, but the research is designed to assess communication not the core concept, so elimination is incongruous in this research context. Follow this one simple rule, incorporate some of the tips above, and the needle can easily be moved from “necessary evil” to truly advantageous!

IIeX 2017: A Qualitative Recap

Posted on: June 21st, 2017 by doyle

iiexgeneral

Once again, I attended what I have come to consider the premiere event for MR’s who are interested in staying abreast of trends, the IIeX Conference in Atlanta.   A combination of excellent presentations as well as a very robust exhibit experience – often with vendors I have not yet seen at another conference – makes it a “must” on my annual conference list.   Here are a few of my takeaways:

  •  There is a sense that the industry has swung too far in our focus on technology at the expense of insights.   It’s not enough to have whiz bang, gee whiz technology unless it is helpful in producing strategic insight.    As one panelist stated, technology should be assisting us in freeing up our intellectual capital, so that only 20% of our time is spent on analysis, and the remaining 80% is on the storytelling.
  • Qualitative seems to be making a comeback, as the antidote to  overwhelming amounts of data that are lacking insight. As a qualitative research consultant, it is both gratifying and a very welcome trend.
  • We have reached the point where there is no longer much meaningful distinction between online research and mobile research. Even when respondents are participating using an online platform, they are highly likely to be accessing it on their smartphone.   Essentially, we have moved into an era where research has become “device agnostic”.  I heretofore resolve to refer to Doyle Research’s online and mobile capabilities as our “digital” methods.
  • The panel and recruitment segment of our industry is struggling with the fact that screeners and surveys are becoming longer and longer, sharply increasing the cost to complete a study. Some vendors are considering charging for Q’s above a certain number; others are taking the approach of refusing to accept more than a certain number of questions.   Clearly, as researchers and clients, we must question the need to ask so many questions.   Do we really think the quality of the insights is going to be improved by surveying respondents who are impatient and fatigued?
  • One thing I heard that disturbed me: some clients reported that they receive deliverables from their MR partners that they need to rewrite before issuing them.   In some cases, they have defaulted to asking only for the raw data and writing it from scratch themselves.   We cannot let that happen!   Our long-term value—the value that cannot be replaced by technology — lies in our ability to deliver insights, as well as the strategies for acting upon those insights, in a clear and compelling manner; and to engage our clients in co-creating solutions.

Once again, I left IIeX exhausted (did I mention that it took 15 hours to fly home from Atlanta to Chicago?) but inspired.    Keep up the good work Lenny and crew!   I’ll see you next year.

Qualitative Research: Can it Produce System 1 Thinking?

Posted on: October 10th, 2016 by doyle

From the desk of Kathy Doylesub-con-mind

Multiple times in recent months, I have been told by clients that they are being advised to forego conducting focus groups because they elicit rational, logical responses – System 2 thinking – rather than tapping into subconscious, emotional responses of System 1 thinking*. These methods include facial coding, biometrics, eye tracking, EEG’s, etc.   However, while Doyle Research has experience with some of these methods, I’d like to go out on a limb and argue that qualitative research, when done well, can actually produce System 1 thinking.

The issue we face is that most traditional research methods ARE tapping into System 2 thinking. We are asking questions, and waiting for answers (“look at this, and tell me what you think”).   What we get are rational, considered, thoughtful responses.   But that is only a portion of what really drives human behavior.  So we need qualitative approaches executed by trained moderators that can tap into System 1 thinking in order to better understand the subconscious influencers and drivers of human behavior.

Here are four ideas to consider:

  • Incorporate projective techniques.    An easy “add” to a focus group session, projective techniques are designed to get below surface responses to uncover subconscious attitudes, feelings and behaviors.    Just one example is the use of picture sorts. Respondents are given a series of pictures totally unrelated to the topic at hand (perhaps scenery or animals for a CPG category) and asked to select the picture that best fits their feelings about a brand, a product, an experience, a situation.  They have to step out of their System 2 thinking in order to do the exercise and emotional, subconscious thoughts are the result.
  • Incorporate an observational component, so that you can observe discrepancies in behavior between what someone says they do vs. what you observe.     A classic example that actually came from a laundry care project many years ago:  as respondents were sorting their laundry during an in-home interview they would say “I sort my clothes into whites and darks”.   But time after time, respondents would generate 5 or 6 different piles that did not appear to fall into either a “white” OR a “dark” pile.  What you are seeing are behaviors that are automatic and, sometimes, totally subconscious. By discussing what we had observed, more nuanced answered were elicited.
  • Capture behaviors over time, in a short-term community, or using mobile ethnography.    An example: we recently followed new moms for a month to understand how feeding decisions were made.  They used their smart phones to complete a guided audio and video diary, sharing their thoughts and experiences as a new mother.     They were not aware that understanding their feeding decisions was our end goal; we wanted to observe the context, influencers (medical personnel, literature, family, employers) and emotions that went into their decision.
  • Capture “In the moment” behaviors using mobile technology.   Example:  mobile intercepts with panel members whose location services are on, and who enter a geo-fenced location. This allows us to capture respondents’ reactions to an experience, as it takes place, and while respondents are literally in- the-moment.

It is our responsibility, as qualitative researchers, to continually seek ways to get below surface responses and gain insights that are grounded in consumers’ actual behaviors rather than their reported ones, and more likely to uncover a more complete story than techniques that rely solely on System 2 processing.

*Thinking, Fast and Slow by Daniel Kahneman.

Costing Qualitative, Part 2: Apples to Apples Comparisons

Posted on: June 10th, 2016 by doyle

project-costing-image1From the desk of Kathy Doyle

This is the 2nd of four blog posts with the goal of de-mystifying the process of obtaining and evaluating a qualitative bid.

Not all cost estimates are created equal. To ensure that apples-to-apples comparisons are made, check that all bids have based their costs on the exact same factors…

Number of recruits—Are all bids assuming the same total number of recruits? Note the difference between recruiting 30 total vs. recruiting 36 for 30 to complete.
Session length—Three 120-minute sessions vs. three 90-minute sessions is the equivalent of one additional 90-minute group discussion. An increase of 30-minutes per session will not only impact facility rental fees and respondents’ incentives, it will also increase the moderation and reporting fees.
Markets—Some markets are simply more expensive than others. If an RFP requests that the research be conducted in New York, a bid that’s based on conducting the sessions in Albany will likely be far less expensive than a bid based on conducting the sessions in Manhattan.
Incentive fees—Incentives are paid to entice consumers to share their experiences. If an incentive is too low, it makes recruiting difficult and tends to yield higher no-show rates. While a bid that includes a lower incentive may appear more budget friendly, in the long run it may cost more in terms of securing a solid recruit. Tip: Some estimates include respondent incentive fees, while other estimates state that incentives are not included and will be billed as a line item on the final invoice. Check this carefully!
Line item expenses—Review the bids to understand what items are included vs. those that will be billed as line items on a final invoice. Items such as client/respondent food, video recording, and shipping expenses, if not included in the project’s costs, can add substantial costs on a project’s final invoice.

For a copy of our whitepaper: “The Inside Skinny: Costing a Qualitative Project” click here.

Why Moderating is the Best Job in the World

Posted on: January 21st, 2016 by doyle

From the desk of Alice Morgan

My husband owns a barbecue restaurant. When people find this out, a spirited and lengthy discussion of barbecue ensues. What sauce? (He makes five – all from scratch.) Dry rub, or no? (Mostly dry, but we offer wet and dry ribs.) What wood is used on the smoker? (Hickory.) It goes on and on.

02When people find out that I moderate focus groups, or (more simply put) interview people, they politely say “Oh, how interesting.” An awkward silence ensues. And I understand why. “Interviewing” is an abstraction. It is hard for people to understand who I interview, why I interview, and in what context the interview occurs. So in this blog post I would like to explain to you, humble reader, what we moderators do, and what is so wonderful about it.

There but for the grace of God go I. We all live in a bubble, whether it be a city, a suburb, or a rural community. In my case, I live in Ann Arbor Michigan, a Midwestern college town. It is a diverse, highly educated, happily sports-obsessed place. Moderating gets me out of my bubble, as I am constantly reminded that there’s a whole world out there of people different than me and my townsfolk. Last year I worked on a project in which I interviewed convenience store employees. Most convenience store employees are not highly educated. Most are just getting by. But many of these clerks, with high school educations at best, provided razor-sharp feedback. The takeaway was crystal clear: what separates people in my community from the convenience store workers I interviewed is opportunity.

Never a dull moment. There is no one way to interview people. Lots of people are successful at it – introverts, extroverts, the whole shebang. What is needed is creativity, and the ability to turn on a dime. We all get there differently. In several instances I have had people start crying as they recounted painful experiences (one was about a horrific experience with a doctor, one was about on-the-job stress). I needed to lighten the mood – fast. And I did. Moderating a group is like conducting. We bring out taciturn, and subdue the loquacious. It is challenging. No two groups are ever the same. This is a job that keeps you on your toes.

All walks of life. There are many jobs in which people are exposed to wide swath of society, but often that exposure is fairly brief. Interviewing people is intense. Moderators get to know people on a profound, personal level. I have interviewed CEOs. I have interviewed plumbers. I have interviewed Moms. During a recent project about car dealerships a participant and I were at a Jeep dealership in Yonkers for 3 ½ hours (during which he actually bought a car). Not to sound too grandiose about it, but moderating is about the human condition.

No job is perfect. The problem with moderating is that moderators spend quite a lot of time, well, not moderating. There is the process of figuring out who to interview, and how (AKA study design). There is the process of determining what to ask (AKA crafting the discussion guide). And there is the process of analyzing what people said and what it all means (AKA writing the report). But the essence of moderating, interviewing people alone or in groups about topics of interest to my clients, is fantastic. And that is what keeps me – and my peers – still crazy about qualitative after all these years.