Archive for the ‘Online Qualitative’ Category



Quant, Qual, and Quant+Qual: Addressing Data Quality

Posted on: August 8th, 2018 by doyle No Comments

My good friend and colleague Jim Bryson at 20|20 recently shared his thoughts in a blog post about qualitative research and its value for protecting sample and data quality.  I agree with Jim’s key point, that data fraud is a huge industry problem.  Another concern I have is that despite massive panel sizes the reality is that only a fraction of those panelists are active.  That means we are relying more and more on a smaller and smaller pool of participants for the information on which huge business decisions are based.  It seems to me that the panel industry is ripe for disruption. 

As for qualitative and the role it can play in addressing both issues, there is no question that qualitative recruiting can be more precise, and more accurate, because the smaller numbers required allow for a high-touch recruit. It is much harder to be a “cheater or repeater” when you are “face-to-face” with the recruiter and the researcher.  However, qualitative research is not always the solution our clients are seeking.  And is in no way a 1:1 replacement for a true quantitative study.  Which is why hybrid research is surging in popularity.

The ability to scale qualitative, quickly and efficiently, makes it a viable alternative or supplement to more traditional quantitative research. For example, 30-50 online chat interviews might just meet your need for a small-scale quant study, while still getting you the VOC insights that qualitative provides.  Best of both worlds.

Or, something the Doyle team is doing more of is conducting a full-scale quantitative study supplemented by a small number of qualitative interviews.   Respondents meeting key criteria can be routed to the qualitative exercise immediately after participating in the survey or they can be hand-picked after the fact to amplify specific quantitative findings.

While qualitative and quantitative research serve distinctly different purposes, the increasing capabilities available for blending them produces multiple benefits worth considering.

Research Methods: A Deeper Dive Into Online Qualitative

Posted on: July 9th, 2018 by doyle

From the desk of Kathy Doyle

Over the past few years, the qualitative toolkit has vastly increased with the addition of online and mobile methods.  These additions have brought significant benefits and the ability to conduct research that was simply not possible before.

With that in mind, we thought we’d take a closer look at some of our favorite online methodologies – webcam interviews, online communities and digital chats.

Webcam Interviews

Webcam interviews are online interviews in which the participant and moderator can see each other as well as any stimuli (storyboard, video, website, etc.) utilizing webcams and a split screen.

Webcam interviews take geography out of the equation, allowing you to conduct interviews across the country in a single day. You get a close-up of participants’ facial expressions as they view the stimulus, something that is difficult to achieve in a focus group facility. This makes them ideal for web usability studies and advertising communication checks.

Online Communities

Online communities take place over time (short term or long term, ranging from two days to two weeks or longer) with respondents answering questions or reacting to stimulus posted by the moderator. They allow for anonymity, unlimited response time, and iterative or longitudinal learning. Respondents also have the ability to upload photos and video and complete homework assignments before or during the session.

Online communities are ideal for exploratory research when rich, nuanced feedback is needed. They also work well for message development or evaluation, and whenever a topic is sensitive.

One-on-One Chats

One-on-one chats function much like an instant message or text chat, but one that is led by a moderator who engages with the individual participant in real time. A great way to get beneath surface-level insights, with this technique the moderator can also share stimuli for respondent feedback.

One-on-one chats are ideal for creative and message testing and allow anonymity that also makes it well suited for discussion of sensitive topics. Furthermore, this approach can be used as a stand-alone qualitative exploration or added seamlessly to a quantitative survey, routing qualified participants from the survey instrument to a moderated chat to provide greater depth and context.

If you’d like to explore whether, or how, these methods could be used to answer your business questions, feel free to contact me at kdoyle@doyleresearch.com.

 

Recruiting Online Qualitative: Best Practices from the Trenches

Posted on: February 20th, 2018 by doyle

From the desk of Natanya Rubin

Online qualitative research is a terrific way to accomplish a national or even international recruit, save time and money on travel, and to gain deep insights from respondents.  But the success of a project can only be assured by getting the details of the recruit right.  We’d like to share some best practices that help ensure that your online project delivers!

Understand your recruiting target.  When you’re deciding whether to conduct mobile diaries, online communities, or webcam interviews, take a good look at your respondent profile and ask yourself, “What is the right method for this target? For example, you might find that with a youth-oriented recruit, you will get richer insights from a mobile journal, where respondents can speak intimately and in their own voice, versus a traditional online bulletin board where they may not have the patience to provide in-depth, written responses.

Be prepared to offer an appropriate incentive. Make sure your incentive is appropriate to the investment of time and effort you’re asking of the respondent, to reduce drop-off during the study and the need for more over-recruits.  Online qual often asks respondents to return to the platform on multiple occasions, since tracking behavior over time is one of the many benefits of the methodology.  The incentive must entice (as well as fairly reward) the respondents in order to get good completion rates.

Screen smart. Screeners for online qual should always include questions around technology ownership and user comfort with their devices, along with questions about online behaviors that ensure that they will be able to navigate the chosen platform. Additionally, we select recruiters who use recruiting methods that meet respondents on their own turf.  For example, a mobile pre-screening link is a natural method of outreach for respondents who spend a great deal of time on their smartphones, and who might be great candidates for a mobile journal.

Choose the right recruiter.  It’s important to select a recruiter with a substantial qualitative database for online methodologies.  Although there are some studies where a panel recruit might be appropriate, it is not recommended for online qual, as respondents who are more used to online surveys might not understand the robust commitment expected for an online qualitative study.

By taking a good look at your target, offering the right incentive, asking the right questions in the screener, and choosing a recruiting partner who can deliver committed respondents, you can ensure that your online study delivers rich, in-the-moment insights!

 

We can see clearly now! Tips and tricks for making the most of your qualitative webcam interview.

Posted on: October 26th, 2017 by doyle

From the desk of Natanya Rubin

Qualitative webcam interviews are an exciting opportunity to see a respondent in her native habitat, but there are plenty of tech pitfalls that can make that time less rewarding.  Here are some of the best practices that we and our platform partners employ to make sure the interview goes well:

The tech check is key.  We always recommend scheduling a live tech check prior to the interview, rather than relying on the respondent to complete an automated one.  This allows a technician to connect personally with the respondent to test bandwidth in the area where the respondent will do the interview, work with them to confirm that their audio functions well, and help them adjust their lighting (see below for more on both topics).

Separate audio and video allows for flexibility, should something go wrong.  Although advances in VOIP stability and clarity make it tempting to have both video and audio run through the respondent’s computer, we recommend that the audio for the qualitative interview be done through the respondent’s phone line.  That way, if the respondent’s internet is spotty and happens to go down during the interview (which sometimes happens despite best efforts to vet bandwidth in advance), the interview can still be salvaged using the separate audio.  During the tech check, we also instruct respondents to make sure their phones are charged and that the power cord is within reach, to ensure that there are no interruptions to the audio.

Lighting is a make or break proposition.  Ensuring good lighting is critical to the success of a qualitative webcam interview.  If the respondent is just a dark, backlit mass, it’s hard to read emotion or see details of their space.  During the tech check, we often help respondents adjust their lighting set-up by asking them to grab a desk or table lamp to get light on their faces, while closing drapes or turning off the lights behind them so they’re not silhouetted.  Then, on the day of the interview, they’re ready to be seen!

By preparing the respondent for the interview through a live tech check, separating audio and video as a safety net against bandwidth issues, and guiding the respondent to be sure they can be seen on-screen, Doyle Research ensures that each qualitative webcam interview is both technically seamless and rich in insights.

 

 

 

 

 

Costing Qualitative, Part 4: Can You Save Money by Conducting Your Research Online?

Posted on: August 3rd, 2016 by doyle

From the desk of Kathy Doyleproject-costing-image1

There is a perception among clients that online research is cheaper to conduct than in-person research. Is this really true? The answer is “It depends”. Let me explain…

Both online and in-person research studies include all of the same cost components: recruiting, incentives, moderating, report, and location fees (what you save on facility costs is spent on software licensing). So when comparing an in-person group to an online group, the costs are about the same.

Where the cost-savings come into play is in study design. Let’s say, for example, that you want to conduct research in 3 different regions of the country. With in-person research, you must travel to each of the 3 regions, and conduct a minimum of 2 groups per market (you should never rely on a single group), in order to achieve your research objectives. With online research, each group can be recruited to include a regional mix. So you may be able to conduct only 2-3 groups in order to achieve your research objectives. And therein lies the savings. Six in-person groups compared to only 3 online groups saves you almost 50% while still achieving your objectives.

So, can you save money by conducting your research online? It depends!

Wrapping up this 4-part series:

In sum, there is no standard qualitative bid.   Each project is unique and complex, with many variables to consider.   Understanding the variables that impact price can help you better design research projects that meet your budget constraints and minimize surprises.

To read the other posts in the series:

Part 1: Four Key Factors That Impact Price

Part 2: Apples to Apples Comparisons

Part 3: Hidden/Forgotten Costs