Archive for the ‘Field Tips’ Category



Recruiting Online Qualitative: Best Practices from the Trenches

Posted on: February 20th, 2018 by doyle No Comments

From the desk of Natanya Rubin

Online qualitative research is a terrific way to accomplish a national or even international recruit, save time and money on travel, and to gain deep insights from respondents.  But the success of a project can only be assured by getting the details of the recruit right.  We’d like to share some best practices that help ensure that your online project delivers!

Understand your recruiting target.  When you’re deciding whether to conduct mobile diaries, online communities, or webcam interviews, take a good look at your respondent profile and ask yourself, “What is the right method for this target? For example, you might find that with a youth-oriented recruit, you will get richer insights from a mobile journal, where respondents can speak intimately and in their own voice, versus a traditional online bulletin board where they may not have the patience to provide in-depth, written responses.

Be prepared to offer an appropriate incentive. Make sure your incentive is appropriate to the investment of time and effort you’re asking of the respondent, to reduce drop-off during the study and the need for more over-recruits.  Online qual often asks respondents to return to the platform on multiple occasions, since tracking behavior over time is one of the many benefits of the methodology.  The incentive must entice (as well as fairly reward) the respondents in order to get good completion rates.

Screen smart. Screeners for online qual should always include questions around technology ownership and user comfort with their devices, along with questions about online behaviors that ensure that they will be able to navigate the chosen platform. Additionally, we select recruiters who use recruiting methods that meet respondents on their own turf.  For example, a mobile pre-screening link is a natural method of outreach for respondents who spend a great deal of time on their smartphones, and who might be great candidates for a mobile journal.

Choose the right recruiter.  It’s important to select a recruiter with a substantial qualitative database for online methodologies.  Although there are some studies where a panel recruit might be appropriate, it is not recommended for online qual, as respondents who are more used to online surveys might not understand the robust commitment expected for an online qualitative study.

By taking a good look at your target, offering the right incentive, asking the right questions in the screener, and choosing a recruiting partner who can deliver committed respondents, you can ensure that your online study delivers rich, in-the-moment insights!

 

Five Key Factors That Impact Qualitative Costs

Posted on: December 18th, 2017 by doyle

From the desk of Kathy Doyle

When you ask your qualitative partners to provide a cost estimate for your research project, do you know what the key factors are that impact the price they provide? Knowing that information could help you save money, avoid surprises, and provide you with a better idea of how to evaluate competing bids.

In any qualitative bid, there are five key factors that impact price:

1. Screening criteria: The more difficult the screening criteria, and the more questions that need to be asked of respondents during the screening process (which impacts cooperation rates) the greater the cost. For that reason, focus on the “need to know” questions, such as key demographics/psychographics, and category/brand usage. “Nice to know” information can be covered in the research.

2. Size of the recruit: The more participants you recruit, the greater the cost. Because qualitative is designed to provide direction rather than to be projectable to a larger population, be conservative with your sample size. Tip: don’t forget to account for last minute cancellations, no shows and respondents who don’t pass a re-screen. We recommend over-recruiting by 20% to ensure that you end up with the number of respondents requested. When comparing bids, make sure to ask whether the cost estimate includes the cost for over-recruits.

3. Incentives: There are three things that influence how much respondents are paid: the difficulty of the recruit, the location of the research, and the amount of work a respondent is being asked to do. The difficulty of the recruit goes without saying: the harder it is to find respondents (very low incidence, consumers vs. professionals), the more you need to pay to make sure they agree to participate. And if your research is being conducted digitally vs. in-person, incentives will most likely be lower because respondents are able to participate from the convenience of their home or office. Finally, the more you ask respondents to do, the greater the incentive. Do you need them for 30-minutes, 2 hours, 2 weeks? Be realistic about how much time you need to accomplish your research objectives, so that you are not paying more in incentives than is necessary.

4. Facility Rental or Platform License Fees: Time is king. The more time you spend in a facility, or licensing a technology platform, the greater the cost. Know how much time you’ll need to accomplish your goals, and set the length of the research sessions accordingly. Padding, because you haven’t adequately scoped out the research, can cost you. And conversely, not allowing enough time can shortchange results.

5. Deliverables: The type of deliverable required can dramatically impact your costs, easily increasing professional fees by 50% or more. Do you need a detailed report, an executive summary, or just a post-research debriefing session? A multi-media PPT presentation, or a professionally edited video report? Be realistic: contract only for what you know your team will use.

Armed with this information, you can be confident that you have designed your research to maximize results and minimize costs.

We can see clearly now! Tips and tricks for making the most of your qualitative webcam interview.

Posted on: October 26th, 2017 by doyle

From the desk of Natanya Rubin

Qualitative webcam interviews are an exciting opportunity to see a respondent in her native habitat, but there are plenty of tech pitfalls that can make that time less rewarding.  Here are some of the best practices that we and our platform partners employ to make sure the interview goes well:

The tech check is key.  We always recommend scheduling a live tech check prior to the interview, rather than relying on the respondent to complete an automated one.  This allows a technician to connect personally with the respondent to test bandwidth in the area where the respondent will do the interview, work with them to confirm that their audio functions well, and help them adjust their lighting (see below for more on both topics).

Separate audio and video allows for flexibility, should something go wrong.  Although advances in VOIP stability and clarity make it tempting to have both video and audio run through the respondent’s computer, we recommend that the audio for the qualitative interview be done through the respondent’s phone line.  That way, if the respondent’s internet is spotty and happens to go down during the interview (which sometimes happens despite best efforts to vet bandwidth in advance), the interview can still be salvaged using the separate audio.  During the tech check, we also instruct respondents to make sure their phones are charged and that the power cord is within reach, to ensure that there are no interruptions to the audio.

Lighting is a make or break proposition.  Ensuring good lighting is critical to the success of a qualitative webcam interview.  If the respondent is just a dark, backlit mass, it’s hard to read emotion or see details of their space.  During the tech check, we often help respondents adjust their lighting set-up by asking them to grab a desk or table lamp to get light on their faces, while closing drapes or turning off the lights behind them so they’re not silhouetted.  Then, on the day of the interview, they’re ready to be seen!

By preparing the respondent for the interview through a live tech check, separating audio and video as a safety net against bandwidth issues, and guiding the respondent to be sure they can be seen on-screen, Doyle Research ensures that each qualitative webcam interview is both technically seamless and rich in insights.

 

 

 

 

 

Advertising Communication Checks: Valuable or a Necessary Evil?

Posted on: July 26th, 2017 by doyle

adcommchecksFrom the desk of Kathy Doyle

I’ve been conducting advertising communication checks for over 30 years, and one thing has not changed…   most of the parties involved dread them. The agency doesn’t like seeing their creative work questioned based on input from a small number of research participants in an artificial setting.  The client does not like navigating the politics of getting the job done, all the while knowing the agency is less than thrilled. And no one really likes sitting in a back room, or in front of a computer screen, for hours on end listening to the same questions being asked every 20-30 minutes.

Yet there are some very compelling reasons why we continue to conduct communication checks:

  • To make sure we haven’t lost sight of who the target is, keeping our finger on the pulse of how best to communicate with them, and mitigate coming across as pandering or tone deaf
  • To make sure we haven’t missed the mark on messaging, and mistaking what we thought was crystal clear for a totally unintended meaning
  • To make sure the visuals support the message, rather than conflict with it
  • To make sure that brand/product recall is strong. It’s great if people love the ad, but if they don’t know what it’s advertising, what’s the point?
  • To optimize (or eliminate) executions prior to quantitative testing and/or final production. Why not find out if there are ways the execution can be tweaked to strengthen it before spending large sums of money?

Clearly, I’m coming out on the side of considering communication checks valuable.   To maximize their value, here are Six Tips for More Productive Communication Checks:

  1. Limit exposure to three executions per respondent, to prevent fatigue from clouding candid feedback
  2. Video storyboards with audio are acceptable; a complete video (albeit rough cut) is better; don’t make the consumer work too hard to see the idea
  3. Consider exposing the ads in a clutter reel to more closely simulate a real viewing experience and more accurately assess breakthrough
  4. Keep them 1:1 for the most honest commentary. People rarely watch programs or web surf with others, let alone strangers!
  5. Keep them short (20-30 minutes) to prevent over-thinking and to be efficient. We often do 12-18 interviews per day!
  6. Consider conducting the interviews online rather than in-person. When people are at home, they are more relaxed and more likely to provide candid feedback.  Use a platform built for research for the most problem-free experience.

One rule to keep in mind: Avoid using communication checks to kill a creative concept. Not only is the sample size too small, but the research is designed to assess communication not the core concept, so elimination is incongruous in this research context. Follow this one simple rule, incorporate some of the tips above, and the needle can easily be moved from “necessary evil” to truly advantageous!

Tackling Tight Timelines, Tough Specs: Is Online Recruiting for Qualitative an Option?

Posted on: June 26th, 2017 by doyle

finger-with-peopleFrom the desk of Natanya Rubin

In a world where the pace of business continues to increase but the demand for actionable, dependable results remains the same, the question in qualitative research becomes, how to deliver?  In the field department, we’re often asked to reconcile a tight timeline with challenging specs, and it’s up to us, with the help of a strong recruiting partner, to accomplish the seemingly impossible.  One option that we sometimes employ is to embrace the online recruit.

Prior to the explosion of online resources, a tough recruit was accomplished mainly by phone.  This was a dependable, but slow, method of finding qualified respondents.  Today, our most common approach to reaching a difficult target quickly is a hybrid of online and telephone recruitment.  Potential respondents are emailed (or even texted!) a preliminary screening survey, containing the study’s foundational specs.  If the respondent qualifies via the survey, they move on to a phone screening to ensure that they truly meet the requirements, and are engaged and articulate.  The online survey saves time and costs, while the telephone screen ensures quality.

Although it’s rare, in the face of a truly compressed timeline we might consider an online only recruit, where respondents complete the entire screener online, and never speak directly with a recruiter.  In that case, embracing the over-recruit is critical to the success of the study.  Although eliminating the hours and manpower that phone screening requires might save some dollars in the budget, an extremely generous over-recruit is needed, sometimes as much as 50% or more.  This ensures enough completes and enough high quality responses to generate reliable findings.  It’s never our preference to recruit this way, but in the face of an urgent need for results, this compromise can deliver.

The challenges of locating hard-to-find respondents are steep enough without adding a compressed timeline to the mix.  But thanks to the possibility of online screening, the chances of success are better than ever.  By deploying a hybrid recruiting method or even an online only method (always being aware of the additional steps needed, in that case, to assure quality completes), a tough timeline can be conquered!