Recently, we discovered that Doyle Research had been listed on a website that aggregates recruiting facilities (which we definitely are not!). This site, in addition to linking potential respondents directly to facilities’ online registration pages, also lists the details of upcoming studies, in various degrees of detail, often including the incentive amount. For those of us whose job it is to ensure that our recruits are honest, accurately recruited, and legitimate, it was an uncomfortable thing to see.
Posting studies online can facilitate specific recruits, and is used by many reputable companies. But it obviously opens up opportunities to cheat. With no limit on the number of projects for which a user completes an initial screener, sites like the one we found make it all too easy for people to become professional respondents. Reliable recruiters have to have strategies to screen out cheaters who reply to these types of ads.
I reached out to Kathryn Harlan, CEO of 20|20 Facilities, a longtime recruiting partner of ours, to get her take on the issue. She told me that 20|20 approaches these concerns in several ways:
Blind the study as much as possible. “Ideally, we craft postings that are applicable to the target audience, but where the exact screening specifications are omitted. You want to be considerate of respondent time. You want them to take surveys that they might qualify for, so that you don’t burn out their engagement. But you don’t want to lead them directly to the screening criteria. It’s a fine line, and we’re always thinking about where it is.”
Database maintenance is key. Prior participation should be scrupulously tracked by topic and by client, so that there isn’t a chance of respondents double dipping in a topic area. Although there’s no way to guarantee that they aren’t participating in studies with other companies, a strong database makes sure that they’re being honest about their participation with an individual recruiter.
Screen multiple ways, multiple times. Through an initial online screener, a detailed phone screener, and a re-screen upon confirmation or at the facility, the recruiter should keep a sharp eye out for inconsistencies in the answers. If the respondent answers consistently, without being led, there’s a greater chance that they have been honest in their responses.
Online recruiting is undeniably a great way to reach fresh sample. Following a few simple best practices can go a long way to keeping studies honest.