AI in UX research that aligns with human values
At UX studio more than half of our UX researchers, including myself, are psychologists by degree. Right after enrolling in university we have to review and agree to the Code of Ethics for Psychologists containing basic principles about human dignity and respect, data protection and confidentiality and scope of competence. Always working in line with research ethics is a must for us in our everyday tasks. Working with AI as researchers adds another layer to this, raising a lot of new questions about ethics. Here is how to do research with AI responsibly, that it aligns with human values.
The first step
The very first step while doing research ethically is informed consent. All of our research participants have to know clearly what is going to happen during a study, know that they can withdraw at any time and know what is going to happen with their data. Before each study, we ask them to sign a consent form covering these points.
In our research team we never use AI facilitation during interviews or user tests, but in some cases we turn to AI for help in transcribing our sessions or synthetizing our findings. That is why we — and we recommend this for everyone who uses any type of AI for research — created an AI skill that formulates a short paragraph that we can copy into our consent forms whenever we use AI during a research process. With this AI disclosure statement generator you can state who is going to be the target audience, set the tone of voice of the text and define how exactly AI is going to be used, so the copy is going to be tailor-made every time.
Paying attention to informed consent and disclosing AI usage is a step that can be done with quite a low effort. Now let’s see the responsible AI principles that have an effect on how to do research ethically.
Responsible AI
Responsible AI (RAI) covers the ways how AI systems are built and used so they reflect human values and do good for the people, society and the environment. It is about spotting the harms AI can cause and working to reduce them. The core objective of RAI is also to ensure AI technologies meet ethical standards and this way foster trust in these technologies. Below are the responsible AI principles and some tips on how to make sure to include them within your research processes.
Fairness and non-discrimination
AI systems should treat every person and group fairly. That means no bias or discrimination based on characteristics like race, gender, religion or disability. Decisions and outcomes should be impartial.
In research work, when asking AI to summarize key takeaways, you can make sure fairness and non-discrimination is present by how you formulate your prompts. Be inclusive and specific in your wording by not just asking for user feedback, but asking it to pay close attention to the differing experiences of specific user groups. You should also double check the language of your prompt and data, not to introduce bias, like avoiding gendered or culturally specific terms (unless they are essential for your research goals). Another tip is to embed fairness when defining the role in your prompts, by stating something like “act as a fair and impartial researcher”.
In most of the cases we are blind to our own biases, that’s why before publishing AI assisted findings it’s helpful to show them to colleagues for feedback. Also, always look out for outliers in your data and double check everything so the nuances don’t get missed.
Transparency and explainability
AI systems should be understandable and interpretable. Users need to understand how and why specific outputs were produced. That takes us back to the first step mentioned above in this post.
Users and non-experts need to know what happened, how and why a decision was made. AI disclosure statements should answer in clear, plain language how AI will be used, if they are going to interact directly with an AI, how will their personal data be protected and if it’s going to be used to train models, with the possibility to opt out.
It is important to keep in mind that disclosing AI usage can change participant behaviour and it is possible to negatively affect the perceptions of the researcher and the study itself. By emphasizing the value of AI usage and that a human is always in control, reviewing and validating AI’s work you can minimize that effect.
Privacy and security
AI systems should protect user data and privacy, ensuring confidentiality. With a data checklist you can make sure that your research participants’ data is safe.
First, since less data equals fewer risks, only collect and use data that is absolutely necessary for your research goals. Also make sure to anonymise any personally identifiable information before feeding the data into an AI tool, and never upload sensitive or confidential participant data to unsecured, public AI tools. And lastly make sure that every AI tool you use, you understand its data policies, you know how your data is stored, processed and whether it’s used to train models.
Accountability and responsibility
When using an AI system, it should be clear who is responsible for its use and the resulting decisions. There should also be a clear process for raising concerns and addressing any harm caused by these decisions.
We already mentioned the importance of informing research participants about the use of AI, but to improve transparency and accountability within your organization, include AI usage in your research plans as well. Also, when creating a research report, document which AI tools you used, what you used them for, and the prompts you used. You can also include what steps you did to work in line with the RAI principles.
To ensure accountability and responsibility, establish AI guidelines or policies internally with clear rules about what type of AI usage is allowed while doing research, what is a no-go, how should you handle data and what are the requirements for disclosure.
Human oversight and control
AI should augment human capabilities, not replace human judgment. The higher the stakes, the more important it is that humans can understand, direct, and override AI systems.
In practice, to be always in control, set up a few ground rules while conducting AI assisted research. Firstly establish “human in the loop” checkpoints in the process, where it is a must for a human to review what is happening. At the same time, always review all AI generated output and content. If you are not sure about something in the output, ask a peer for a second opinion.
It is also beneficial to create an AI correction protocol to know what is the backup plan when AI-generated errors occur during the process. And lastly, another simple thing you can do is to continuously provide explicit feedback for the AI on the quality and accuracy of the outputs. It is a good practice to actively try and find where an AI output is false or inaccurate, where the AI gets confused, hallucinates or misunderstands some nuance.
Societal and environmental well-being
AI systems should prioritize human welfare and societal and environmental well-being. While minimizing the environmental and societal impact, AI should serve human needs and shouldn't make environmental harm.
What you can do in practice to minimize this impact while doing research is very simple: first just ask yourself whether the use of AI is really necessary. AI is not needed or suitable for every task. Use impact assessment methods to consider unintended societal and environmental consequences before using AI tools or sharing AI assisted research insights.
If AI is not needed for your specific task, don’t use it. If AI is truly helpful, but it is a smaller, simpler task, use a smaller, more energy-efficient model version. You can also advocate for these sustainable AI practices within your organization, clients, partners and vendors.
Final thoughts
Using AI in a responsible way isn't a checklist you complete once and move away. Principles like human control, clear accountability, privacy and transparency only mean something when they shape everyday decisions: which tools we adopt, how and when we use them, and who answers for the outcome when something goes wrong.
AI can make research faster and broader, but it can't take responsibility for the conclusions. That part stays with us, and it should.
Moving from principles to a working process isn't always straightforward. If you're doing research looking to bring AI into your workflow without compromising ethics, our Applied AI for research service can help.
Get in touch to talk about what responsible AI-assisted research could look like for your team.