Test participant biases.
Test participants are humans, and just as researchers have cognitive biases. As a UX researcher, if you are unaware of them, you can fall into the trap of flawed conclusions. Therefore, a systematic error in thinking caused by cognitive biases can affect the feedback you receive from the test participant.
Participants’ bias happens when the participants involved in the research respond in a manner that suggests they are trying to match up with the desired result of the researcher. This means that the respondent starts to exhibit unusual characteristics from what they would normally do or how they would normally react.
As a user researcher, it is crucial to be aware of your participants’ potential biases.
Being aware of participants’ biases and finding control measures from the beginning of the research can be beneficial to the positive outcome of the study, same as for all other research biases or errors.
You should know that no study will be perfect, but being prepared and cautious can get you close to your desired result. Let’s see how participant bias occurs and what you can do to reduce its effects.
Framing bias:
The framing effect is a cognitive bias where people decide on options based on whether the options are presented with positive or negative connotations, e.g., a loss or a gain. People tend to avoid risks when a negative frame is given but seek risks when a positive frame is presented.
How to avoid it?
- Write biased questions and then rewrite them by using neutral language;
- Use open-ended questions;
- Summarize what they said to make sure you understand it correctly;
- Let test user fully explain their point of view before moving to the next question.
Hindsight bias:
Also known as the known-it-all-along phenomenon or creeping determinism. This bias occurs when people tend to perceive past events as more predictable than they actually were.
How to avoid hindsight bias?
- Ask about the current process user is going through;
- Get test users to recall recent memories.
Social desirability bias:
The tendency of survey respondents to answer questions in a manner that others will view favorably is social desirability bias. It can take the shape of over-reporting “good behavior” or under-reporting “bad” or undesirable behavior.
How to avoid this?
- Tell participants that you are not associated with the product, just asking questions;
- Let them know there are no right or wrong answers;
- Ask what they would do to improve the product for their friends/family/other users.
Social position bias:
Social position bias happens because people tend to remember or choose the items at the beginning or the end of the list. It is just the way our brain works.
How to avoid it?
- Vary the order of questions for each participant;
- Don’t test every prototype at once;
- Focus on 1-2 ideas per participant.
Illusion of transparency bias:
The illusion of transparency is a cognitive bias in which people tend to overestimate how well their mental state is understood and observed by others. The observer’s illusion of transparency is the overestimation of how well a person understands the mental conditions of others.
How to avoid transparency bias?
- Use participant’s language;
- Be aware of body language;
- Ask them for clarifications;
- Repeat what they said and ask if you understand them correctly.
Recap of test participant biases:
As a user researcher, it’s essential to be aware of your participants’ potential biases. We hope that we have given you some food for thought.
If you plan to test your product with real users, it’s much better to know in advance what could potentially influence the results. After all, bias can be present even when we try to hide it. So do everything in your power to minimize the potential negative impact on research results.
