GuidedTrack How should you choose when to shuffle and when to use a specific order (1) GuidedTrack Docs

How should you choose when to shuffle and when to use a specific order?🔗

If your answer options cannot be ordered into a logical or intuitive order and as long as the list of answer options is not so long that participants would struggle to process it, we recommend using the *shuffle keyword to ensure that the order of presentation does not unduly affect the responses that people are more likely to choose on average. (Since people read from the top, they may stop at the first answer they think is acceptable, even if there is a better answer they haven't seen yet. This "satisficing" tendency may bias people towards answers at the top of the list.) On the other hand, if your list of answer options is particularly long (e.g., a long list of locations, diseases, or drugs), then it may be advisable to place the options in alphabetical order (despite the risk of satisficing), so that participants can more quickly check for the answer(s) that most closely reflects their answer(s) to the question.

If answer options follow a natural progression or ranking (e.g., from one end of a scale to another), we recommend keeping the ordering of answer options in alignment with people's natural assumptions and intuitions. For questions involving quantities, we recommend placing higher numbers toward the top of the scale, and lower numbers toward the bottom. For questions including answers that range on a scale from "good" or "agree" to "bad" or "disagree," we recommend placing the "good" or "agree" options at the top of the scale.

Good answer option shuffling examples🔗

Here's a good example of shuffling a question's answer options:

*question: Have you ever had any of the following experiences without them being due to the use of psychedelics or other drugs?
	*tip: Please tick all that apply, and leave all unchecked if you have had NONE of these experiences.
	*type: checkbox
	*shuffle
	*save: psychosisWithoutDrugs
	Losing touch with reality so you see, hear or believe things that aren't physically real
	A persistent combination of several of the following factors: difficulty thinking, feeling the world is unreal, seeing unreal things, fear of external organisations, and unconventional beliefs
	A manic episode
	A depressive episode followed by a somewhat manic episode
	Less intense rapid emotional cycling between negative and positive emotions
	A traumatic childbirth experience
	A spiritually powerful experience while giving birth to a child

In the above example, since there is no natural ordering to the question categories, it is advisable to use the *shuffle keyword so that, on average, satisficing tendencies do not affect the same question across the whole sample (since the presentation order is randomized across people).

Here are some good examples of ordered quantity and frequency-related questions:

*question: How often do you visit the dentist?
	Less than once per year
	About once per year
	About once every six months
	More than once every six months

*question: How many times have you gone camping in your life?
	Never
	Once or twice
	Three to five times
	More than five times

In the above questions, there is a natural presentation order, and this will help people to quickly locate the answer that most closely approximates the answer they are thinking about after reading the question.

Bad answer option shuffling examples🔗

Here's a bad example of an inappropriately non-shuffled question:

*question:	Have you ever had any of the following experiences without them being due to the use of psychedelics or other drugs?
	*type: checkbox
	A traumatic experience
	A period of rapid emotional cycling between negative and positive emotions
	A depressive episode followed by a hypomanic episode
	A depressive episode without a hypomanic episode
	A manic episode

The above categories should be shuffled to reduce the effects of satisficing systematically affecting the study results.

Here's a bad example of a frequency question that doesn't employ natural ordering:

*question: How often do you visit the dentist?
	*shuffle
	Less than once per year
	About once per year
	About once every six months
	More than once every six months

In the above example, categories that naturally progress from one end of a scale to another (that would be easier to read and sort between if they were left in order) have been unnecessarily shuffled.

Next: