Interpretation is the stage of research on which its quality most depends. In analysing qualitative data, researchers decide which statements express the same idea, which patterns are significant, and which accounts will represent a finding in the final report. These judgements shape what a report finds, and they are the judgements that commissioners, reviewers, and readers most need to be able to trust.
Much of the current debate about artificial intelligence (AI) in research focuses instead on writing. Reviewers and commissioners have good reason to watch for fabricated references, generic prose, and conclusions that sound more certain than their evidence allows. These concerns relate to how researchers present their findings rather than how they arrive at them. The more consequential question is what part AI plays in the analytical work that produces those findings.
Leading qualitative analysis packages now include features that suggest codes, propose themes, and summarise transcripts. General-purpose AI tools can perform similar tasks when a researcher uploads interview data. For teams working to tight timelines and limited budgets, these features have obvious appeal, and in some cases, they may help researchers to notice patterns they would otherwise miss. Even so, when a tool performs these tasks, it makes analytical judgements that a researcher would otherwise make. A tool that groups statements into a theme, for example, has decided that those statements express the same idea. The quotation it then selects to illustrate that theme determines whose words will represent the experience of others.
The use of AI in analysis therefore raises a question of research integrity. Readers of research expect the people who present findings to explain how they reached them and to take responsibility for them. When a tool makes some of the analytical judgements, and no one records which ones, that expectation becomes difficult to meet. In this post I consider why this matters for humanitarian research, what disclosure needs to capture, and what commissioners and researchers can do to keep analytical judgements visible and to ensure that someone takes responsibility for them.
Judgements about meaning
The question of who makes these judgements carries weight in humanitarian research, where data often passes through several languages before anyone analyses it. Language interpreting, transcription, and translation each involve judgements about meaning of their own, and qualitative research has had to learn to account for them.
Briony Jones and I made this argument about interpreters in a paper we published in 2009 (under my previous name Ficklin). Drawing on fieldwork in Nicaragua and Bosnia-Herzegovina, we argued that an interpreter changes the dynamic of an interview, and that researchers cannot properly analyse an interpreted interview without accounting for the interpreter’s epistemology, positionality, and capability. I saw this during my fieldwork in Nicaragua, I speak Spanish but worked with an interpreter as a back-up during a confidence wobble. Comparing our interpretations showed different understandings of meaning beyond just what participants had said. Our interpretations were shaped by differences in subject and local contextual knowledge, emotional intelligence, and positionality.
AI tools now make comparable judgements at the stage of analysis, when they group translated statements into themes or select the quotation that will represent a finding. The lesson from language interpreting applies to them as well. Their judgements shape the evidence, and researchers need to account for them rather than treat the tool as a neutral instrument. Researchers can discuss an interview with an interpreter afterwards, but no equivalent conversation is possible with a tool, and those working from its output are often not able to compare it with what participants said.
The selection of quotations raises a related concern. Choosing which account will represent a finding is itself an analytical judgement. That choice shapes how readers understand the people whom the research concerns. In a recent study for the International Center for Transitional Justice, Briony and I noted that selecting research stories to represent harm or experience can risk flattening complex individual accounts. When a tool makes that selection, the same risk applies, and the reasoning behind the choice is more difficult to examine.
These concerns also bear on localisation. As national and local organisations take on more responsibility for research, often with limited resources, some may understandably turn to AI tools to meet the analytical and reporting standards of international donors. However, the contextual and linguistic knowledge that national researchers bring is what allows a team to recognise when a tool has misread the data. If AI replaces that knowledge rather than supporting it, the research loses the people best able to check the tool’s analysis.
Disclosure, authorship, and accountability
Academic publishing has already addressed the question of responsibility. The Committee on Publication Ethics (COPE) states that an AI tool cannot be an author of a paper, because it cannot take responsibility for the work. COPE also asks authors who use AI tools in writing or in collecting and analysing data to state in their methods section which tools they used and how. Responsibility for the content, including any part that AI produced, remains with the authors.
Commissioned research in the humanitarian and development sectors seldom passes through an editorial process of this kind, as I argued in my previous post. Its authorship is also often less clear than in academic publishing. A single report may reflect the work of a commissioning organisation that designed the study, consultants who conducted it, programme staff who reviewed it, and a communications team that edited it for publication. Each of these contributions is legitimate, but together they can make it difficult to identify who is responsible for a particular finding. The use of AI in analysis adds a further contributor to this chain. Unlike the others, it cannot explain its reasoning or take responsibility for its contribution.
Disclosing the use of AI is a necessary step. Disclosure practice in commissioned research is still developing, and what a statement records matters as much as whether a report includes one. A statement that AI tools supported drafting tells a reader something quite different from an account of how a team used a tool to propose themes, which of those themes the team accepted or revised, and who reviewed the result. Only an account of the second kind allows a reader to judge whether the analysis behind the findings is sound. From the perspective of research integrity, what matters is whether each analytical judgement behind a finding remains traceable, and whether a named person is prepared to stand behind it.
Accountability for analytical judgements
AI tools have a legitimate place in humanitarian research. The analytical judgements in which they play a part still belong to the method and deserve the same scrutiny as any other methodological choice. When I quality-assess a research report, I check that its findings, claims, and recommendations follow from its evidence. That check depends on tracing a finding back through the analysis to the data, and on someone who can explain the judgements that shaped it. Where AI played a part in a report’s analysis, I would now also ask which codes and themes the tool proposed and how the team reviewed them.
The audit trail, a record of how codes and themes developed over the course of an analysis, already serves this purpose in qualitative research. Where a team uses AI, the audit trail can show which codes or themes a tool proposed and which the team accepted, revised, or rejected, together with the reasons. A methods section can then describe the role of AI at each stage of analysis, rather than in a single general statement. Together, these two steps would give reviewers and readers what they need to assess how the team reached its findings. Both add some time to an analysis, which commissioners would need to allow. Pretending that AI use only saves time on a research project compromises quality.
Commissioners also have a role in this. When they receive a report, they can ask who made the analytical judgements behind each of its main findings, and what part AI played in them. A research team should be able to name the person or people responsible and explain how they reached those judgements. If no one can do so, the finding is arguably not yet ready for publication.
AI tools are likely to become a routine part of humanitarian and development research, and many of their uses will be helpful. As they do, the fluency of a report will say even less than before about the soundness of the analysis behind it. In humanitarian research, the consequences of that analysis are considerable, since it helps to determine which needs a response recognises and whose voices come to stand for those of others. The people whose experiences the research describes are rarely present when decision-makers read a report, which makes the care that researchers take over its analysis a matter of integrity as well as method.
I would be interested to hear how colleagues are approaching this. If you commission, conduct, or review qualitative research, how do you currently record or ask about the use of AI in analysis?

Join the conversation
Comments
Loading comments…