GuidedTrack Docs

Comprehension checks🔗

When your experiment or study includes a very important instruction (or set of instructions) that participants must understand in order to participate, it is advisable to include a comprehension check. Please see this section for tips on how to choose the content for comprehension and attention checks in general. In the current section, the focus is on the cases where you need to check that participants can understand a specific set of instructions prior to allowing them to participate in your study. In such situations, we recommend using comprehension checks prior to randomization or group assignment. Otherwise, you'll run the risk of having systematic differences in the level of task comprehension between experimental groups.

Example of how to check participants' comprehension prior to participation:🔗

In the example below, the participant's comprehension of the study requirements is tested prior to randomization. This allows participants who don't yet understand the experiment to be redirected to an explanation of the study instructions. (If the participant repeatedly fails to understand the study, an experimenter could choose to eliminate them. And in rarer cases, participants who do not pass the comprehension check the first time could be automatically eliminated prior to randomization. In these scenarios, though, you would still need to reimburse them for the time they have taken on that survey, of course.)

--quiz page

*header: Quiz

It's *very important* that we make sure that you understand what we're asking of you if you agree to complete the second part of the study. Please *carefully* answer the following question, as *getting the correct answer here is necessary for being included in the second part of the study*.

>> comprehensionCheck = 0

*question: What are you expected to do each day of the study, except on the first and last days?
	*tip: *You must answer correctly here in order to qualify for the second study* (though you'll still be paid for the survey that you're *currently* doing, regardless of your answer).
	*shuffle
	Nothing. I'll wait until I'm contacted to check-in.
	I'll be preparing for a test on my knowledge of sleep disorders which I'll be contacted to take.
	I'll be tracking my sleep habits and then answering questions about them during a follow-up survey.
	I'd like to read the explanation of this study again, please.
		*goto: explainTheStudy
	I'll be completing the daily check-in that I receive a link to in the emails and I'll be practicing my selected sleep techniques each night.
		That's correct!
		>> comprehensionCheck = 1

*if: comprehensionCheck = 0
	Sorry, that's not correct! Please review the instructions again.
	*goto: explainTheStudy

Although this example highlighted how you can check for comprehension prior to study randomization occurring, you may also want to check for comprehension and active engagement at multiple other timepoints during your study. In that scenario, you are unlikely to stop the study if someone fails a comprehension check (since screening has already been completed), but you can use their comprehension checks as a way to gauge data quality.

You may wish to compare the answers of those who passed particular comprehension checks versus those who did not pass, or you may wish to exclude all participants from analyses if they did not demonstrate a sufficient level of understanding; but in both cases, you will still need to reimburse those participants for the time they spent on the survey. Please see this section (as mentioned previously) for tips on how to choose the content for comprehension and attention checks in general.