How to avoid biased responses due to screening survey inclusion/exclusion criteria: if possible, don't spell out the criteria🔗
If you have a screening survey that determines whether someone is or is not eligible for a follow-up study, we recommend not making it obvious that something is a screening survey. This will help you to avoid people trying to give responses that they think will qualify them for the study.
Example of a question that doesn't reveal itself as being about an exclusion criterion🔗
Suppose you're conducting a screening survey to see who would qualify for a study you want to run involving night shift workers. As part of a set of questions which are not specifically labeled as screening questions, you could ask the following:
*question: Do you do night shift work?
Yes
>> eligibleForFollowUpStudy = 1
No
>> eligibleForFollowUpStudy = 0
Because participants are not told what kind of answer you are expecting or wanting, this would hopefully maximize the chance that they will answer honestly. It is always best to avoid creating an incentive to lie.
Example of what not to do: making the desired answer obvious to participants🔗
*question: Only people aged 30 to 50 who are night shift workers are eligible for our follow-up study. Can you confirm that you are in that age range and that you do night shift work?
Yes
>> eligibleForFollowUpStudy = 1
No
>> eligibleForFollowUpStudy = 0
In this example, participants have a clear idea of the "correct" answer (if they want to be invited to the follow-up survey), which might influence the less honest participants.
Next: How to progress from a screening/initial to a full/follow-up survey