Archive for the ‘Field Tips’ Category



We can see clearly now! Tips and tricks for making the most of your qualitative webcam interview.

Posted on: October 26th, 2017 by doyle

From the desk of Natanya Rubin

Qualitative webcam interviews are an exciting opportunity to see a respondent in her native habitat, but there are plenty of tech pitfalls that can make that time less rewarding.  Here are some of the best practices that we and our platform partners employ to make sure the interview goes well:

The tech check is key.  We always recommend scheduling a live tech check prior to the interview, rather than relying on the respondent to complete an automated one.  This allows a technician to connect personally with the respondent to test bandwidth in the area where the respondent will do the interview, work with them to confirm that their audio functions well, and help them adjust their lighting (see below for more on both topics).

Separate audio and video allows for flexibility, should something go wrong.  Although advances in VOIP stability and clarity make it tempting to have both video and audio run through the respondent’s computer, we recommend that the audio for the qualitative interview be done through the respondent’s phone line.  That way, if the respondent’s internet is spotty and happens to go down during the interview (which sometimes happens despite best efforts to vet bandwidth in advance), the interview can still be salvaged using the separate audio.  During the tech check, we also instruct respondents to make sure their phones are charged and that the power cord is within reach, to ensure that there are no interruptions to the audio.

Lighting is a make or break proposition.  Ensuring good lighting is critical to the success of a qualitative webcam interview.  If the respondent is just a dark, backlit mass, it’s hard to read emotion or see details of their space.  During the tech check, we often help respondents adjust their lighting set-up by asking them to grab a desk or table lamp to get light on their faces, while closing drapes or turning off the lights behind them so they’re not silhouetted.  Then, on the day of the interview, they’re ready to be seen!

By preparing the respondent for the interview through a live tech check, separating audio and video as a safety net against bandwidth issues, and guiding the respondent to be sure they can be seen on-screen, Doyle Research ensures that each qualitative webcam interview is both technically seamless and rich in insights.

 

 

 

 

 

Advertising Communication Checks: Valuable or a Necessary Evil?

Posted on: July 26th, 2017 by doyle

adcommchecksFrom the desk of Kathy Doyle

I’ve been conducting advertising communication checks for over 30 years, and one thing has not changed…   most of the parties involved dread them. The agency doesn’t like seeing their creative work questioned based on input from a small number of research participants in an artificial setting.  The client does not like navigating the politics of getting the job done, all the while knowing the agency is less than thrilled. And no one really likes sitting in a back room, or in front of a computer screen, for hours on end listening to the same questions being asked every 20-30 minutes.

Yet there are some very compelling reasons why we continue to conduct communication checks:

  • To make sure we haven’t lost sight of who the target is, keeping our finger on the pulse of how best to communicate with them, and mitigate coming across as pandering or tone deaf
  • To make sure we haven’t missed the mark on messaging, and mistaking what we thought was crystal clear for a totally unintended meaning
  • To make sure the visuals support the message, rather than conflict with it
  • To make sure that brand/product recall is strong. It’s great if people love the ad, but if they don’t know what it’s advertising, what’s the point?
  • To optimize (or eliminate) executions prior to quantitative testing and/or final production. Why not find out if there are ways the execution can be tweaked to strengthen it before spending large sums of money?

Clearly, I’m coming out on the side of considering communication checks valuable.   To maximize their value, here are Six Tips for More Productive Communication Checks:

  1. Limit exposure to three executions per respondent, to prevent fatigue from clouding candid feedback
  2. Video storyboards with audio are acceptable; a complete video (albeit rough cut) is better; don’t make the consumer work too hard to see the idea
  3. Consider exposing the ads in a clutter reel to more closely simulate a real viewing experience and more accurately assess breakthrough
  4. Keep them 1:1 for the most honest commentary. People rarely watch programs or web surf with others, let alone strangers!
  5. Keep them short (20-30 minutes) to prevent over-thinking and to be efficient. We often do 12-18 interviews per day!
  6. Consider conducting the interviews online rather than in-person. When people are at home, they are more relaxed and more likely to provide candid feedback.  Use a platform built for research for the most problem-free experience.

One rule to keep in mind: Avoid using communication checks to kill a creative concept. Not only is the sample size too small, but the research is designed to assess communication not the core concept, so elimination is incongruous in this research context. Follow this one simple rule, incorporate some of the tips above, and the needle can easily be moved from “necessary evil” to truly advantageous!

Tackling Tight Timelines, Tough Specs: Is Online Recruiting for Qualitative an Option?

Posted on: June 26th, 2017 by doyle

finger-with-peopleFrom the desk of Natanya Rubin

In a world where the pace of business continues to increase but the demand for actionable, dependable results remains the same, the question in qualitative research becomes, how to deliver?  In the field department, we’re often asked to reconcile a tight timeline with challenging specs, and it’s up to us, with the help of a strong recruiting partner, to accomplish the seemingly impossible.  One option that we sometimes employ is to embrace the online recruit.

Prior to the explosion of online resources, a tough recruit was accomplished mainly by phone.  This was a dependable, but slow, method of finding qualified respondents.  Today, our most common approach to reaching a difficult target quickly is a hybrid of online and telephone recruitment.  Potential respondents are emailed (or even texted!) a preliminary screening survey, containing the study’s foundational specs.  If the respondent qualifies via the survey, they move on to a phone screening to ensure that they truly meet the requirements, and are engaged and articulate.  The online survey saves time and costs, while the telephone screen ensures quality.

Although it’s rare, in the face of a truly compressed timeline we might consider an online only recruit, where respondents complete the entire screener online, and never speak directly with a recruiter.  In that case, embracing the over-recruit is critical to the success of the study.  Although eliminating the hours and manpower that phone screening requires might save some dollars in the budget, an extremely generous over-recruit is needed, sometimes as much as 50% or more.  This ensures enough completes and enough high quality responses to generate reliable findings.  It’s never our preference to recruit this way, but in the face of an urgent need for results, this compromise can deliver.

The challenges of locating hard-to-find respondents are steep enough without adding a compressed timeline to the mix.  But thanks to the possibility of online screening, the chances of success are better than ever.  By deploying a hybrid recruiting method or even an online only method (always being aware of the additional steps needed, in that case, to assure quality completes), a tough timeline can be conquered!

Loving the List: A Qualitative Perspective

Posted on: April 27th, 2017 by doyle


Reading-a-list-225x300From the desk of Natanya Rubin

Recruiting qualitative respondents from a database can be a challenge when the target is very specialized.  The solution is sometimes a list provided by the client.  But list recruits come with their own challenges and it’s important to face them with creativity and realistic expectations.

Factors to consider when determining the viability of a list include:

  • The type of contact information available: Does the list provide the full name of the respondent?  Does the list provide an e-mail address, home phone number, and cell phone number?  Often, e-mail is the most efficient way to reach respondents, but the subject line and body text must be very compelling to break through the general bulk of spam that people receive.
  • The accuracy of the list: How current is the list?  The longer ago the information was collected, the less likely it is to be accurate.
  • The size of the list: Conventional wisdom in the recruiting sphere says that for a “good” list—that is, one with full, current information—there should be 30 names provided for each desired recruit. But that number can go up to 70 or 100 depending on the factors above.
  • The ability to reveal the sponsor of the research: Can the sponsor of the research be identified, or is it a blinded study?  Often, there are compelling reasons to obscure the origin of the study.  But in a world where people are trained not to click on a link from an e-mail address they don’t recognize or pick up a call from an unknown number, it’s an uphill battle to get a reply from respondents not primed for research.

So how to break through and make a list work harder?  It’s important to consider the pitch that you’re going to deliver to potential respondents.  Thought needs to be put into a catchy subject line, a clear and compelling explanation of the study, and an appeal that makes it clear why they, personally, are so important to the study.

It’s also necessary to understand that even with a brilliant pitch, a lot more unprimed respondents are going to say no than yes.  Allowing more time for recruiting and considering sweetening the deal with a higher incentive are both ways to improve the possibility of a favorable response.

Sometimes a qualitative recruit can’t be completed without a list, but the challenges can be considerable!  Knowing the difficulties in advance can prepare you for success through careful planning, patience, and making the best case possible for a respondent to join the study.

Show Me the Money! Four Factors That Impact Qualitative Research Incentives

Posted on: November 22nd, 2016 by doyle

From the desk of Natanya Rubin

How much should we pay respondents to participate in a qualitative research study?    Answering that question is more of an art than a science.  Incentives provide both motivation for respondents to follow through on their commitment to the research, as well as compensation for their time and effort.moneyvote

While there is a basic formula that recruiters use to estimate respondent incentives, there are four key considerations that influence the recommended amount:

  1. The “time and inconvenience” factor:  Is it a 15-minute telephone interview?  A two-hour focus group at a facility that is a 20-minute drive away?   A series of activities over an extended period?
  2. The difficulty of the recruit:  Does virtually anyone qualify?  Or are we looking for the proverbial needle in a haystack?
  3. The time of day.  Are you asking stay-at-home parents with school aged children to participate in research at 3 PM?    Are you asking full-time professionals to participate in research during business hours?
  4. The market where the research is going to be conducted:  Is the research being conducted in coastal urban markets or in small Midwestern towns or rural markets?

The time and inconvenience factor assumes there is a basic cost to get people “out the door,” whether that involves logging onto a website, dialing into a webcam group or interview, or driving to a facility.  After that the other consideration is whether we are asking for a one-time commitment, or a commitment over time.    A one-time commitment might involve visiting a store to complete a diary exercise; or, driving to a focus group facility for a 90-minute focus group.   A commitment over time might involve asking a respondent to shop and prepare a specific food item, or to keep a diary of their workflow or food consumption for an extended period.  An incentive for an over-time commitment factors in the actual hours it will take for respondents’ to complete the tasks, but it also factors in a dollar amount that will keep them engaged and participating for the days, weeks, or months that are required.

The difficulty of the recruit comes into play when it is necessary to talk to a highly trained or very specific, low incidence target market.   If there are a limited number of possible respondents, when recruiters find them they want incentives to be higher than average so the qualified person is more likely to agree to participate.   In this case, increasing the participation rate reduces the number of recruiting hours needed to successfully complete the project.

Don’t underestimate the importance of time of day when determining incentives.   If you need stay-at-home parents to participate in a group or interview at 3 PM, just when the children are coming home from school, you are going to have to factor in the possibility that they will need to hire a babysitter.   If you are asking full-time professionals to participate in research during business hours, they will need to be compensated enough to make up for missing work.

Finally, the market where the research is going to be conducted can directly impact incentives.  Large urban markets command higher incentives than small town or rural markets where parking is free, and the cost of living is lower.

One other time when incentives are adjusted is in the moment, if weather is expected to impact show rates.   When it’s raining hard, or a snowstorm is expected, people are more likely to cancel.   To make the trip in inclement weather worth their while, we may increase incentives at the last minute, or offer a bonus for people who show up on time.

So when determining incentives, remember that there is not a “one size fits all” answer.   The real answer is “it depends!”