Factual consistency

Do you trust the answer your LLM gives?
LLMs are now excellent at summarisation and answering questions, but trusting them requires us to be able to check if their answers are correct: do the facts they state agree with the source information we have given them? When the answer is an open-ended text, such as a summary, this is challenging.
We are working on computationally efficient techniques for factual consistency checking, using human-inspired strategies, preference optimisation, and pretrained encoder models. Take a look at our recent work here:

• Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies, ACL 2026, Yuxuan Ye, Raul Santos-Rodriguez, Edwin Simpson
• Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics, EMNLP Findings 2025, Yuxuan Ye, Raul Santos-Rodriguez, Edwin Simpson

AI for Healthcare

How can AI and NLP models help clinicians, patients and researchers to improve healthcare? AI and NLP techniques can be very effective at collating, organising and combining information from different sources. We’re currently using this idea in several active projects.

• Working with the Child Mortality Analysis Unit to develop an early categorisation system for the causes of child deaths in England, providing a much quicker way to identify risks to children and support timely interventions.
• Combining textual and visual information to improve Radiology report generation by following a process inspired by how human radiologists look at Radiology data and clinical notes. See our survey paper in IEE Reviews in Biomedical Engineering, 2024.
• AI scribes, which listen to consultations and draft clinical notes, are being widely adopted by GPs around the world. But how well do they actually work? Do they save time or improve the quality of doctors’ notes? We’re working with clinicians at the Bristol Medical School to develop a framework for assessing the real-world impact of these tools. This work is led by Dr. Peter Edwards in the Centre for Academic Primary Care (see his work on consultation notes in British Journal of General Practice, Letters).
• Using LLMs to automate the process of extracting and synthesising evidence from biomedical research papers, using LLMs to identify candidate studies and assess them according to study design criteria.
• Quantifying and explaining uncertainty in medical codes: what information is making the classifier unsure of the correct label for this patient?

Learning Specialised Models Under Uncertainty

Small models are fast and cheap; they can be run on standard local hardware without transferring sensitive data to the cloud, unlike remote LLMs, which is important in sensitive domains like healthcare. With the right training, they can excel at tasks such as document classification. However, training is made harder by small amounts data and label noise, and the performance of small models tends to collapse when tested on new examples that are very different to their training data – so called out-of-distribution (OOD) examples.
Our research is developing methods for adapting small, specialised models with limited data, detecting OOD cases, and quantifying uncertainty in data, models, and predictions to handle these issues.

• Safe deployment of classifiers requires the system to recognise when an input is out-of-distribution: a novel kind of example, perhaps of an entirely new category, that the model has not been trained to handle. Our work introduces a technique for improving OOD detection and shows its effectiveness for multimodal document classification. Out of distribution detection with attention head masking for multimodal document classification, Nature Scientific Reports 2025.
• A quick way to ‘build’ a text classifier is to write instructions for an LLM, but how can we compress that to a smaller, more efficient program for running at high frequency? We introduce a pipeline for distilling LLM classification to small models, using synthetic data generation, and showcase its effectiveness for building financial sentiment classifiers. Efficient Financial Language Understanding via Distillation with Synthetic Data, LREC 2026.
• We may want to personalise a model, e.g., to generate the kind of summaries we need for a specific use case, but a single user can only provide a tiny amount of training data. Bayesian optimisation provides a way of deciding what to annotate so that we can adapt summarisation and question answering models with just a handful of labels. Efficiently Acquiring Human Feedback with Bayesian Deep Learning, UncertaiNLP 2024.
• Uncertainty in language itself presents a challenge for NLP systems, including LLMs: can they recognise figurative language, such as metaphors? Can they correctly process negation? Work led by Jasmine Owers (look out for the TACL paper to appear shortly!) is helping us to understand how LLMs and other transformer models differ from humans in how they process figurative language.

Automated Essay Scoring

One kind of specialised model is the automated essay scorer: it helps teachers to evaluate long written answers from large student cohorts, but requires adaptation to specific essay questions and marking schemes, and to ensure fair evaluation with human oversight. Data for training the scorer for a particular essay question is scarce.
We have released the first public automated essay scoring dataset for Arabic, shown how specialised classifiers can deliver competitive performance with additional human markers, and teamed them up with human markers to mark fairly, accurately and quickly. Arabic LLMs performed less well (Computers and Education: Artificial Intelligence, 2025) in our study than smaller encoder models, showing there is still some challenge in adapting them to specialised tasks, particularly in languages besides English.