High-performing AI isn’t enough. Learn how intended use, validation evidence, change control, and post-market planning shape a successful FDA submission from the earliest stages.
An AI team can spend months improving a model, reach an impressive AUC and still arrive at a difficult question late in development: does the evidence support the device it plans to put on the US market?
That question is larger than model performance. It includes who the device is for, what clinical decision it informs, how the correct answer is established, whether performance holds across the intended-use population and what happens when the model changes after release.
A recent AI-enabled device Pre-Submission, supported by BlueBridge, reinforced the point. The discussion moved quickly from whether the AI worked to whether the proposed evidence could defend the claim. Subgroups affected the shape of the validation dataset. The process for establishing ground truth needed its own justification. The boundary between the AI output, deterministic software and clinical decision-making had to be clear.
For early-stage AI medical-device teams, especially university spinouts and founder-led companies, these decisions can feel like downstream regulatory detail. They are not. Intended use, risk management, validation, labelling, change control and post-market monitoring start to shape the product long before a submission is assembled.
The claim sets the size of the evidence problem
Take a model that analyses chest X-rays. It might quantify a feature for a radiologist, prioritise studies on a worklist, support a diagnosis or make a finding without specialist review.
The underlying model could be similar in each case. The devices would not be. The intended user, clinical workflow, consequences of error and degree of reliance all change. So do the risk analysis, labelling, human-factors questions and evidence needed to support the intended use.
The claim boundary is a product decision. A broad claim may look commercially attractive, but it can bring a broader dataset, a more demanding reference standard and more complex clinical validation with it. A narrower initial claim can sometimes offer a more credible route to market.
In practice, regulatory thinking needs to begin while the intended use and product architecture are still flexible. Leave it too late and a wording problem becomes a study-design problem.
A performance metric cannot defend the dataset
Sensitivity, specificity, AUC, Dice score and error rates matter. Their value depends on the evidence underneath them.
The evaluation population needs to make sense for the intended-use population. Sites, acquisition devices and workflows need to be sufficiently representative. The reference standard needs to be appropriate, reproducible and independent of the model output. The study also needs to test the device in its intended workflow, rather than only testing a model against a convenient, curated dataset.
Subgroup analysis makes the connection particularly clear. If performance needs to be understood across age groups, sites, disease severity or other clinically relevant subgroups, that cannot be left until the analysis is under way. The planned subgroups influence which sites are approached, which patients are enrolled or collected retrospectively and how the validation dataset is constructed. They may also affect what needs to be represented in the training data.
Ground truth deserves the same attention. Saying that expert clinicians labelled the data does not settle the matter. Depending on the device, the defensibility of the reference standard may turn on who those experts are, whether they are independent and blinded, how disagreement is resolved and whether their qualifications fit the intended US use case.
The AI is judged against the reference standard the team creates. Its design needs to be defended with the same care as the performance results.
Change control starts before the model changes
Traditional software changes too, but AI makes the question unusually visible. A manufacturer may retrain a model, alter an input pipeline, adjust a threshold, expand supported hardware or introduce data from new populations.
Ordinarily, a post-market modification must be assessed to determine whether a new marketing submission is required. A Predetermined Change Control Plan, or PCCP, offers a route to describe certain anticipated modifications in a marketing submission, along with how they will be developed, validated and controlled.
Under the FDA’s final guidance, a PCCP includes a Description of Modifications, a Modification Protocol and an Impact Assessment. Each proposed change has to sit within a defined boundary and follow an agreed method of development, validation and control.
The useful work happens before the update. A team needs to decide which modifications are genuinely foreseeable, what evidence would show that an updated device remains acceptable and where the boundary of the authorised device sits. An over-ambitious PCCP may be difficult to justify. Without one, predictable updates may create avoidable regulatory work later.
Those decisions affect the product roadmap as well as the submission.
Plan for performance after clearance
Once deployed, an AI-enabled device may encounter different populations, scanners, protocols, prevalence, workflows and user behaviours. Clinical practice changes too.
The FDA’s draft lifecycle guidance recommends describing plans for monitoring and managing the performance of AI-enabled device software functions. The international GMLP principles likewise call for deployed models to be monitored and for retraining risks to be managed. In June 2024, the FDA, Health Canada and MHRA also published [transparency principles for machine-learning-enabled medical devices], focused on information that supports safe and effective use across the product lifecycle.
The transparency principles do not set detailed submission requirements, and the lifecycle guidance remains draft. Both indicate the questions FDA expects manufacturers to be able to address.
In practice, post-market monitoring needs to be designed into the product and its supporting processes. Teams should know which performance signals matter, what data can realistically be collected, how drift or unexpected use will be investigated and what would trigger corrective action or regulatory assessment.
Finding a site-specific performance problem early may prevent customer complaints, lost clinical confidence or a field action.
Keep the claim, evidence and change controls aligned
Claim boundaries, reference standards, subgroup plans, change assumptions and monitoring arrangements all influence what gets built and what evidence can credibly support it. They need attention while the team can still change the product, dataset strategy and development plan without major rework.
For an early-stage AI-enabled device team, the product claim, the evidence and the change controls need to remain coherent. Better model performance cannot compensate for gaps between them.
BlueBridge helps software and AI-enabled medical-device teams pressure-test these decisions while the claim, dataset strategy and lifecycle plan are still flexible.
