By Aaron Smith, Co-Founder and Machine Learning Scientist

‍

The FDA’s risk-based seven-step framework for using AI in clinical trials sends a clear message: AI belongs in drug development, but only with the right guardrails and infrastructure in place. On paper, it might sound straightforward. But in practice, building AI that meets regulatory expectations means rallying teams across machine learning, statistics, clinical ops, IT, QA, and regulatory affairs to agree on and execute how the work gets done, documented, and defended. 

This raises a strategic question for biopharma teams: Should you build all of that from scratch or partner with a team that’s already done it?

Most companies aren’t built to do that quickly or to the standard required. The risk isn’t just falling short of the FDA’s bar. It’s burning time, budget, and internal trust along the way.

‍

At Unlearn, we’ve already done the hard part. We’ve been pioneering the use of AI in clinical trials for nearly a decade while earning EMA qualification for our PROCOVA method, holding successful Type C meetings with the FDA, and collaborating with regulators to define best practices for AI credibility.

While the same methodology can also be used to reduce sample sizes, our latest whitepaper, A Risk-Based Approach for Leveraging AI in Clinical Trials, focuses specifically on using digital twins to increase trial power while mapping directly to each of the FDA’s steps.

Here’s a snapshot of how we apply each step in practice:

1. Define the Question of Interest

The goal of any trial is to determine whether a treatment is safe and effective. Digital twins can be incorporated into the trial analysis to make this determination more precise, boosting the power of the trial. By boosting power, digital twins increase the chance of detecting a true treatment effect, reducing the risk of overlooking a real signal.

2. Define the Context of Use (COU)

For our application, the context of use is specific and narrow: we use digital twins to generate prognostic covariates from baseline data. These covariates represent each participant’s likely outcome under control or standard of care, and we incorporate them into a standard ANCOVA statistical model to adjust for baseline variation. The digital twin does not alter trial conduct, endpoints, eligibility, or assessments. Its only role is to improve the precision of treatment effect estimates by accounting for expected variability, thus increasing statistical power without introducing bias.

3. Assess the AI Model Risk

We divide risk into two categories: statistical risk and operational risk. Statistically, PROCOVA introduces no additional risk. Adding a prognostic covariate to an ANCOVA analysis cannot inflate the type I error rate. At worst, if the covariate were uninformative, you’d lose a degree of freedom. In practice, our covariates are strongly prognostic, so power reliably increases. Operationally, we mitigate risks like data leakage and execution errors through strict data lineage controls. Our models only use permitted baseline data, and outputs are generated through a locked, version-controlled pipeline. Every run is auditable and reproducible.

4. Develop a Plan to Establish Credibility

Before the trial begins, we serialize and lock the model, ensuring it is fully pre-specified—architecture, hyperparameters, inputs, and outputs included. Our ML models are deterministic: given the same input, they always produce the same result. This is verified through software tests before use in any trial. We also implement full traceability. Each run of the model is versioned, timestamped, and tracked through a standard software development lifecycle (SDLC). This ensures that sponsors and regulators can independently verify that the locked model was applied as intended and without deviation.

5. Execute the Plan

During trial execution, we generate prognostic covariates using the locked model and a fully automated, pre-specified pipeline. No parameters are changed mid-trial. The system performs automated quality checks to confirm that predictions exist for all expected variables, outputs fall within valid ranges, and no data types are misaligned or missing. Any deviations are flagged before inclusion in the trial analysis. All steps are logged, timestamped, and version-controlled to ensure complete traceability.

6. Document the Results

Every step in our process is documented in a credibility assessment report. This includes system logs, structured quality control artifacts, and audit trails showing how baseline data was transformed into prognostic covariates. We also generate cohort-level summaries and diagnostic outputs to validate the behavior of the digital twins, for example, confirming that predicted disease trajectories align with known reference data from similar patient populations. If we observe anomalies, we investigate and document them before moving forward.

7. Determine Adequacy for COU

Finally, we assess whether the ML model’s application is adequate for its intended use at both the population and individual levels. This includes validating that expected relationships between predicted variables are preserved and that overall disease progression tracks with reference data. We also flag and review participants with atypical digital twin trajectories. If anomalies are traced to unusual baseline inputs or gaps in the training set, we document the rationale for inclusion, exclusion, or further review. The result is a defensible, transparent application of AI that regulators can trace from input to output.

‍

‍

We encourage you to read our whitepaper for a more detailed breakdown of how our digital twin technology applies the FDA’s framework step by step. It’s the operational playbook we use in real trials, with real regulators, today.