At Flatiron, we work with leading NHS partners that are delivering cancer care to patients across the UK. The data generated through that care is highly sensitive, and using it well means getting both the technology and the oversight right. Machine learning helps us curate cancer data from these partners at a speed and scale that manual review alone can't match. But every model is shaped, tested and checked by clinicians, security and privacy experts.
We sat down with Serena Dover and Marina Kushnir, two of Flatiron's Oncology Clinical Data Specialists, who bring backgrounds in cancer research nursing and clinical trials to this work. Working alongside our machine learning engineers, they write the prompts our models use, investigate every disagreement between the model and our human abstractors, and refine the models until the outputs meet the standard that clinical decisions demand.
They explain why patient safety and data quality always come first, how this work puts our VALID framework into practice, and what it could mean for patients, particularly those with rarer cancers. Our white paper, Responsible AI in UK Health Data, sets out the approach in more detail.
Please could you explain what you and your team do at Flatiron?
Serena: I’m an Oncology Clinical Data Specialist and a cancer research nurse by background and I’m part of Flatiron’s Clinical team. We’re all healthcare professionals with oncology experience—either doctors, nurses or clinical trials professionals. We work to produce ‘golden’ research-ready data outputs from de-identified patient data from our NHS partners. For our team of abstractors, that involves manually going through the de-identified data and using their experience and expertise to answer a set of questions, such as ‘does this patient have breast cancer?’ or ‘what kind of biomarkers do they have?’
Marina: I'm also an Oncology Clinical Data Specialist and my background is in clinical trials and real-world evidence programs. Over the past two years the Clinical team has been working with Flatiron’s machine learning team to evaluate models on the abstracted data. We want to see if the model can answer questions with the same level of accuracy as our human abstractors. Where there’s a disagreement between the model and the abstractors, the Clinical team will review those cases and identify the root cause of the error, and that feeds into modified prompts to make sure the dataset is of the very best quality.
We review patient charts in depth very carefully and identify exactly what's caused the error. And then what’s probably the most interesting new element for our team is providing actionable recommendations for how to then deal with that error, how to optimise the model and improve it? This can be, for example, through modifying the prompt, filtering the document categories that the model sees, or adding post-processing rules, which are rules that polish the rough output into the final usable data. This is something that was completely new for us and it’s a skill we’ve developed over this past year.
Why and how do you use machine learning?
Serena: We use machine learning to help speed up the process of extracting data from patient health records. The machine learning prompts are written by us as Clinical Data Specialists and also approved by Flatiron’s senior medical team. That provides a stable way of extracting the data. Before anything reaches us, identifiers are removed. We will run the model inside our closed UK environment and get an output back which we can match against what the abstractors have done so that we can see how well it’s performing.
Obviously, machine learning does not actually read or understand the words, it's just assigning numbers and probabilities to them. So, it is essential that there is always a human in the loop to check what it is doing. It is super interesting and not something that, coming from a nursing background, I ever really imagined that I’d be involved in.
Marina: When first utilising machine learning, we scrutinised every piece of ML-extracted data meticulously, as clinicians naturally do with any new technology. But what we have found is that machine learning is performing at such a high quality that it's very rare that we do find any issues. Our input is more optimisation and refinement.
Can you give a specific example of some data you’ve worked on recently?
Marina: The first NHS datasets we built using machine learning are from patients with prostate cancer. We asked the model to look at all of the things that we clinicians would look at.
One of the most basic questions we ask is whether it can be proven that a patient has cancer, based on the data in their records. The answer must come from a valid source of evidence for a given data point: so for most solid tumour cancers, this would be from a pathology report. We also look for specific terminology, for example, the type of cancer, any hormones or biomarkers driving the cancer, and the grading of the cancer.
Other factors we ask it to look for are histology results, tumour grade and treatments, or if not treatments if that is due to elective active surveillance or ‘watch and wait’, which are two different ways of observing a patient if they are not going to have an active treatment.
This can happen in some prostate cancer cases if there’s elevated prostate-specific antigen (PSA) at diagnosis. If it's elevated it's an indicator, but it's not a diagnostic in itself. That’s because other factors can raise a person’s PSA, such as a urinary tract infection, or an enlarged benign prostate. So it’s one of those markers that also then gets reviewed as we go.
Two abstractors work on each set of patient data first and then we will run the machine learning model on the same data. If there’s a disagreement between our abstractors and machine learning then we will review those cases and try to rationalise why the disagreement has arisen and determine which is correct. This is a central part of Flatiron’s VALID framework – the first peer-reviewed approach to evaluating the quality and reliability of real-world data extracted by large language models and machine learning, which we published in April 2026, and explained in our white paper on responsible AI in UK health data.
We then go through a rigorous process of testing before we would run the model on the patient data that we would deliver back to the Trusts in a curated format, and to researchers as anonymised datasets.
What are the benefits of using machine learning for this process?
Serena: Machine learning is particularly helpful in supporting the extraction and analysis of much larger cancer populations data, more quickly. This can produce significant outputs, such as speeding up the identification of new treatments.
These datasets are then provided back to our NHS partners so that clinicians can use the outputs to support improvements in care for their own patients. Using machine learning will expand and quicken this process too, delivering more value back to the NHS faster.
Marina: Where I think machine learning is really performing very well is where there's very dense text, and key information buried within that text is like a needle in a haystack. Machine learning is able to pick that out very easily.
Also, when there is a large volume of documents, machine learning is able to see the entire patient journey. It is better able to ‘remember’, in place and in context, the different spans of treatment and where procedures took place during that span. Whereas, when a human abstractor goes through many, many clinical letters, it may be difficult to remember everything that happened to an individual patient.
How do you ensure that the machine learning model outputs are of the very highest clinical quality?
Serena: Away from the modelling, the data all goes through quality assurance at a patient level and at a cohort level. If clinical insights are going to be made from our data then we need to be sure that we are happy with how robust it is. There is so much work that goes into checking before and after the model run as well. You could not just let machine learning run without checking what it is doing.
Marina: The areas where machine learning struggles are the same areas in which abstractors would struggle: ambiguous or sparse documentation, inconsistencies in the data, or nuanced language or wording. So where there's some interpreting or inference needed, we come in to make sure that it's inferring things correctly and that machine learning is not over-assuming and making up things that aren't there. So this is the area in which we provide the most input.
What stage is Flatiron currently at with using machine learning? And what’s next?
Serena: A few of the prostate cancer models are now up and running: for example, the model on real-world progression in prostate cancer in Germany. Research published in April this year has shown that the model enables more complete and accurate tracking of how prostate cancer progresses, which gives clinicians and researchers the real-world evidence they need for research and quality control.
Set for launch in 2027, our longitudinal breast cancer and non-small-cell lung cancer datasets will combine machine learning with clinical expertise to power smarter research decisions.
Marina: I think as time goes by, it's possible that machine learning might even be able to judge its own performance, identify its own errors, and suggest ways to mitigate it.
It’s still going to be essential to have a human in the loop to make sure that it's doing this correctly. But I think Flatiron’s clinical team will shift increasingly towards Quality Assurance. And where the clinical role will become even more important is to ask how do you then apply that data – what questions will be the most important to answer? Because we're going to have such a huge volume of high quality, highly robust data. So this data application to the real world is where I think our team will play an even bigger role.
What do you think is unique about Flatiron’s approach?
Serena: One thing that makes Flatiron special is that so many of us have clinical backgrounds: our medical directors are practising doctors, and a lot of our abstractors are practising nurses and clinical trial professionals.
We hire people who know the true impact that this work can have. We have all seen, on a daily basis, those patients who ask their doctor, ‘What will be the outcome for someone like me?’ Doctors can’t necessarily answer that question right now. Whereas potentially with Flatiron’s datasets we can give them a much deeper level of insight into what’s going on and they might be able to do that with more certainty.
Speaking for myself, it’s important that my work aligns with my personal values of being a nurse at the core of it all. Flatiron is a lovely company to work for because it is guided by its mission every day in trying to improve the lives of people with cancer. Even though I’ve not been in the NHS now for three years, I feel like we can still make a huge difference.
Marina: Focusing specifically on the data product, Flatiron has certain datapoints that simply don’t exist anywhere else. We like to compare our data to national benchmarks, but the reality is that often comparable data just doesn't exist, because we're the first ones to be generating it. Part of that has to do with access to unstructured documents like clinical letters, where there are certain data points that are not available in other databases or registries. We are able to curate these at scale, with high data recency and consistency.
We also capture really complex clinical milestones that are not going to necessarily be recorded as a single entry.
[For example, a patient’s progression to castrate-resistant prostate cancer is often reflected across a sequence of lab results and medication changes rather than a single note. Our team applies consensus clinical guidelines to standardise these timelines, ensuring that the clinical reality is reflected in a consistent, structured way.]
We develop clear definitions and provide detailed guidance to our clinically trained abstractors and constantly optimise our quality control processes.
What do you hope Flatiron’s machine learning approach will achieve for cancer patients in the longer term?
Serena: I would love to see this approach expanded into rarer cancers. For example, pancreatic cancer, which typically gets diagnosed at a late stage, has a poor prognosis of around six months and has only a few treatment options. By using ML to curate data at scale, we can reach statistical significance for research on those rarer cancers and drive treatment change and cancer care improvements for those small populations of people.
Marina: My hope is that as we expand across more regions of the UK, all patients are represented in our data and we can better understand how treatments perform across different patient populations and geographies, identify any disparities in care, spot emerging trends earlier, and carve out sub-cohorts to answer much more specific clinical questions. Ultimately, that can feed into more informed clinical decisions and better-designed clinical trials.
Find out more about our approach
Our white paper, Responsible AI in UK Health Data: Setting Standards for Future Use of LLMs in Clinical Data Extraction, sets out how the VALID framework and robust governance underpin the safe, responsible use of large language models in UK health research.
Download the white paper.
