New! We wrote a guide to building apps set for success from the beginning by being focused on value. Get a free copy now to learn tips used by the market leaders.

Opens a new tab
NEW! Get a free copy of our new  guide to building apps driven by value! Learn from our experience and tips used by the market leaders.
Opens a new tab
Orbitum Insight

Interview questions for non-biased user testing

How to structure your questions for useability app testing and user research.

How to structure your questions for useability app testing and user research.

Preparing for software testing is no mean feat. Software testing life cycle includes many stages such as functional testing based on the predetermined functional requirementsis. But one of the most important steps to veryfy your previous UX research and and development quality lies in non-functional testing field, and that is user testing.

You've got to gather materials, set up a testing environment, find participants, prepare prototypes, and much more. With so much to think about, it's easy to forget that the questions you ask — and the way you ask them — can dramatically impact the results of the testing process.

And in usability testing, if you’re not careful, a little bias will start to sneak in.

Bias can affect how users respond to your app, or it could influence what you take from what they’re saying. Either way, the result is the same: you’ll end up with misleading data, rather than useful, objective feedback.

So how do you avoid research bias while doing application user testing? Follow these quick tips...

Would you like to receive all the latest insights from us?

Software tester

The best user testing questions are open-ended ones

In many UX research methods, there’s a golden rule: never ask a question that could be answered with a simple yes or no. Instead, ask open-ended questions that encourage the test participant to express their opinions in detail.

To make a closed question open, swap words like did, was, would, which, or when for words like how, what, and why. This allows the participant to fully express their thoughts and allows for more natural follow-up questions.

Closed Questions ❌

  • Was the feature easy to use?
  • Did you like the homepage of the app?

Open Questions ✅

  • How would you describe your experience of using this feature?
  • What three words would you use to describe this homepage?

Open-ended questions eliminate the risk of bias by empowering users to share their unbridled views. After all, ask someone “Did you like my app?” and they’ll likely tell you what you want to hear.

If, on the other hand, you ask “What worked well, and what worked not so well in this app?” — then you’re more likely to hear the truth.

Use a neutral tone

When doing user testing, you'll speak with strangers and you might want to break the ice with some friendly humor.

However, you want honest and critical feedback from your test participants, which is difficult to give to someone you’ve instantly warmed to. So try to keep your questions and conversation as neutral as possible, without seeming too stuffy either.

Essentially, keep your language clear, concise, and free of jargon (or flattery!).

Mobile testing interview questions

Avoid leading questions

How you ask questions can encourage the test participant to answer in a certain way. So be sure to phrase your UX research interview questions in a way that doesn't influence the participant’s answer.

Some examples of leading questions to avoid:

  • Can you see how our app would save you time?
  • Does our app make you more productive?
  • Was that feature too slow for you?

Not only are these questions closed, but they also betray what you’re looking for, which will bias the response.

Create realistic tasks

When planning user testing, you'll have a specific outcome you want to evaluate — for example, testing the function of inviting new users to help with editing a document in your application.

So whilst you want to make sure this task gets adequate testing, you don’t want to be too transparent with your objective. This’ll only make users hyper-aware of what you’re testing; eroding the realism of your test conditions, and increasing the risk of bias.

The solution? Write your discussion guide in a way that encourages users to discover the desired task organically. Instead of specifically directing participants to the “invite” button, ask them to perform an activity that would naturally lead to inviting others, such as collaborating on a document.

Need more help with mobile app testing?

We hope these tips help you minimize bias in your user research and useability testing process. After all, non-biased research helps you learn, anything else is such a waste!

If you’re looking to develop a product relying on solid research and testing methodologies, we'd love to hear from you. Get in touch with the Orbitum team today and we’ll help turn your app vision into a reality.

Follow us on social media for more insights

Let's talk

Our goal is to align with your goals and make the technology work for you.

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form

Featured Case Studies

Wink Eye Care

Wink Eye Care

Replacing routine with the power to make an impact in eye care vet practice
TMG Construction

TMG Construction

Streamlining workflow to save resources and minimize risks in Construction industry


Turning pandemic into opportunity for a nationwide hi-tech chain of veterinary clinics
HatchTank Research

HatchTank Research

Helping focus groups catch up with XXI century


Marketing and Growth


Research and Insights


Product Strategy


Product Design


Web Development


Mobile App Development