What Training Data Powers Healthcare AI?

Why healthcare AI needs its own training data

A general-purpose model can draft a clinical note that reads like a physician wrote it. Getting the prose right and getting the medicine right are separate problems, and only one of them is solved by fluency. In a survey of clinicians conducted alongside a physician-audited benchmark of model responses to real medical cases, 91.8% reported personally encountering a medical hallucination and 84.7% judged those errors capable of causing patient harm. Physician auditors traced 64% to 72% of the residual errors to failures of causal or temporal reasoning rather than gaps in medical knowledge, meaning the model had the facts and drew the wrong line between them. ^[6]

The same research found something that cuts against the obvious fix. General-purpose frontier models produced hallucination-free responses 76.6% of the time against 51.3% for medically specialized models trained on biomedical corpora. ^[6] Feeding a model more textbook medicine does not reliably make it safer. What closes the gap is grounding: pairing the model with the institution's own records, its own imaging, its own documented outcomes, and a task narrow enough that performance can be measured against a real reference standard. That is a data problem rather than a scale problem, and the data lives inside health systems.

Two forces are pulling that data into the market at once. The first is the documentation burden. A study published in JAMA in April 2026 compared 1,800 clinicians using ambient AI scribes against 6,770 controls across five US hospitals over more than two years, finding daily reductions of 13 minutes in EHR time and 16 minutes in documentation time, with roughly triple the documentation savings among clinicians who used the tool on more than half their visits. ^[4] A separate multicenter study published in JAMA Network Open in October 2025 found burnout prevalence among ambulatory clinicians fell from 51.9% to 38.8% thirty days after ambient scribe deployment. ^[8] Every one of those products is trained on physician-patient conversation audio paired with the note a clinician actually signed.

The second force is regulatory clearance. As of the FDA's March 2026 update, the agency's AI-Enabled Medical Device List contained 1,524 devices authorized for US marketing, with 333 authorized in 2025 alone and 92 more in the first quarter of 2026. ^[4] A device that clears the FDA is a device whose training and test data survived agency scrutiny. That scrutiny is what turns a pile of scans into an asset.

The six data types that actually power clinical AI

Healthcare data is not one commodity. Each modality trains a different class of model, carries a different compliance burden, and prices differently.

• De-identified EHR data and clinical notes. Progress notes, discharge summaries, problem lists, medication histories, and structured encounter data, the substrate for summarization, decision support, risk stratification, and coding automation. Longitudinal depth, meaning whether a patient's record follows them across years and settings, matters more than row count.
• Radiologist- or pathologist-annotated medical imaging. CT, MR, digital radiography, ultrasound, and whole-slide pathology with pixel-level segmentations, bounding boxes, or study-level classifications. The annotation is the product: an unlabeled DICOM archive trains a self-supervised encoder, an expertly annotated one trains a device.
• Physician-patient conversation transcripts. Ambient exam-room audio paired with the clinician's signed note. Audio alone teaches transcription; audio paired with the note teaches the summarization judgment that makes an ambient scribe usable.
• Clinical trial and outcomes data. Protocols, eligibility criteria, enrollment records, adverse event logs, and endpoint results, which train trial-matching and outcomes-prediction models and carry the cleanest consent trail in healthcare because participants signed an informed consent form.
• Lab and genomic data. Structured lab panels with reference ranges and flags, plus sequencing data with variant calls and clinical interpretations. Genetic data is separately enumerated in the EU as its own special category under GDPR Article 9. ^[4]
• Claims and administrative data. Diagnosis and procedure codes, place of service, and payment outcomes: broad, cheap, structured, and semantically thin. Claims record what was billed rather than what the clinician observed, which limits them as a diagnostic ground truth.

The public corpus for all of this is thin. MIMIC-IV, the field's canonical open clinical database, covers 364,627 unique individuals, 546,028 hospitalizations, and 94,458 ICU stays, all drawn from a single Boston medical center between 2008 and 2022, and it requires CITI training plus a signed credentialed data use agreement to access. ^[4] It is an outstanding research resource and a poor foundation for a commercial product that has to generalize across community hospitals, scanner vendors, and patient populations that a single academic center never sees.

Why confirmed-outcome labels are the scarce, high-value asset

Health systems generate imaging and notes continuously. What they generate far less of, and almost never in export-ready form, is the pairing of a case with a confirmed answer: the biopsy result that resolved the ambiguous nodule, the twelve-month follow-up that established the lesion was benign, the adjudicated panel read that settled a disagreement between two radiologists. The FDA has a name for this and treats it as the center of a submission.

The agency's January 2025 draft lifecycle guidance then asks sponsors to document how that reference standard was established, the uncertainty inherent in it, the strategy for equivocal or missing results, and, where clinicians produced it, the grading protocol, the blinding protocol, the number and qualifications of the participating clinicians, and an assessment of intra- and inter-clinician variability. For data annotation specifically, FDA recommends independent assessments by each annotator without knowledge of the other annotators' decisions, along with a documented method for adjudicating disagreements and a plan for handling incorrect annotations. ^[4]

Three structural forces keep this data scarce.

• 1The label requires a clinical event that may never happen. A radiologist's read is an opinion. Confirmation requires a biopsy, a surgical finding, a subsequent scan, or a death certificate, and most imaging studies never get one because the patient was fine, went elsewhere, or was never followed. The confirmed-outcome subset of any imaging archive is a small fraction of the whole, and no amount of retrospective processing enlarges it.
• 2Expert annotation is priced like specialist labor because it is specialist labor. Pixel-level segmentation of a tumor across a multi-sequence MRI takes a radiologist on the order of an hour per patient, and FDA's blinding and adjudication expectations mean the serious version needs several radiologists per case. ^[4] That cost structure caps how much annotated data any one organization can afford to create.
• 3Site independence multiplies the requirement. FDA expects test data sequestered from development data and, generally, drawn from different sites than those used to develop the device. ^[4] A sponsor with 200,000 beautifully annotated studies from one hospital still needs an independent multi-site validation set, a separate procurement problem with its own consent and de-identification trail.

HIPAA de-identification: Safe Harbor versus Expert Determination

Protected health information cannot be licensed for AI training in identifiable form. HIPAA provides exactly two routes to de-identification under 45 CFR 164.514(b), and the choice between them shapes what the resulting dataset can be used for. ^[4]

Safe Harbor: strip 18 identifier categories

Safe Harbor is the mechanical route. A covered entity removes all 18 enumerated identifier categories for the individual and for the individual's relatives, employers, and household members, and must additionally lack actual knowledge that the remaining information could identify someone. ^[4] The categories, in the regulation's order, are: names; all geographic subdivisions smaller than a state, with a limited three-digit ZIP exception for units above 20,000 people; all date elements except year, plus all ages over 89 aggregated into a single "90 or older" bucket; telephone numbers; fax numbers; email addresses; Social Security numbers; medical record numbers; health plan beneficiary numbers; account numbers; certificate and license numbers; vehicle identifiers and serial numbers; device identifiers and serial numbers; URLs; IP addresses; biometric identifiers including finger and voice prints; full-face photographic images and any comparable images; and any other unique identifying number, characteristic, or code.

Expert Determination: a documented risk analysis

The alternative route requires a person with appropriate knowledge of and experience with generally accepted statistical and scientific principles for de-identification to determine that "the risk is very small that the information could be used, alone or in combination with other reasonably available information, by an anticipated recipient to identify an individual," and to document the methods and results of the analysis supporting that determination. ^[4]

Expert Determination costs more and takes longer, and it is what serious clinical AI datasets use, because it permits retaining full dates, granular geography, and unmodified head imaging when the analysis shows residual risk is acceptable. The phrase "by an anticipated recipient" carries weight in a marketplace transaction: a determination made for one recipient under one set of controls does not automatically transfer to a buyer with different adjacent data holdings. Any dataset marketed as expert-determined should ship with the written analysis, the expert's qualifications, and the recipient assumptions the determination rested on.

The FDA framework: SaMD, PCCPs, and what it means for your data

Clinical AI that diagnoses, treats, or drives treatment decisions is regulated as a device, reviewed through 510(k), De Novo, or PMA. The traditional device paradigm assumed a product that does not change after clearance, which fits poorly with models that improve as data accumulates. Congress addressed this in the Food and Drug Omnibus Reform Act of 2022, and FDA implemented it through the predetermined change control plan.

The current instrument is FDA's final guidance, *Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions*, issued August 18, 2025 and revising the December 4, 2024 original. It applies across the 510(k), De Novo, and PMA pathways. ^[4] A PCCP has three parts: the Description of Modifications listing the planned changes, the Modification Protocol describing the methodology to develop, validate, and implement them, and the Impact Assessment weighing benefits and risks. FDA reviews the PCCP as part of the original submission, and modifications executed within it do not require a new marketing submission.

The Modification Protocol is where a data seller's work shows up. FDA asks manufacturers to explain how new data will be collected, annotated, curated, stored, retained, controlled, and used for each modification, how training, tuning, and test sets will be sequestered, whether identifiable subpopulations including intersectional groups are adequately represented, whether the reference standard represents the best available process for determining ground truth, and, notably for anyone valuing a dataset, how older data will be complemented or replaced by newer data so that performance stays representative of the current patient population and standard of care. ^[4]

The practical consequence is that a dataset's provenance documentation is part of its function. A sponsor cannot use a corpus in a submission if the seller cannot answer who annotated it, under what training and blinding, with what inter-reader agreement, and from which sites.

Outside the US: GDPR Article 9 and the EU AI Act

Data from EU or UK patients is governed independently of HIPAA, and the GDPR is stricter in one important respect: health data is not merely personal data subject to a lawful basis under Article 6. Article 9(1) prohibits processing of genetic data, biometric data used for unique identification, and data concerning health outright, unless one of the ten conditions in Article 9(2) applies. ^[4]

For AI training, the realistic paths through Article 9(2) are narrow:

• Article 9(2)(a), explicit consent to the processing "for one or more specified purposes." Explicit and purpose-specified are both doing work. A general consent to treatment does not cover licensing records to train a commercial model, and Member State law can bar consent from lifting the prohibition at all. ^[4]
• Article 9(2)(j), scientific research purposes in accordance with Article 89(1), based on Union or Member State law that is proportionate and provides specific safeguards. This route is genuinely available for research collaborations and genuinely contested when applied to commercial model development, and Member States implement it inconsistently. ^[4]
• Article 9(2)(h), preventive or occupational medicine and healthcare management, which supports care delivery and related operational processing rather than downstream AI licensing. ^[4]

Layered on top, the EU AI Act classifies much clinical AI as high-risk by operation of existing device law. Under Article 6(1), an AI system is high-risk when it is intended as a safety component of, or is itself, a product covered by the Union harmonisation legislation in Annex I and that product must undergo third-party conformity assessment. Annex I Section A lists Regulation (EU) 2017/745 (the MDR) and Regulation (EU) 2017/746 (the IVDR). ^[4] In effect, a device's MDR or IVDR risk class determines its AI Act status: software that needs a notified body under the MDR is high-risk AI, which triggers the AI Act's data governance, technical documentation, logging, human oversight, and accuracy obligations.

The timing changed recently, and stale guidance on this point is now common. The European Commission's Digital Omnibus, proposed 19 November 2025, deferred the AI Act's high-risk obligations. Under the agreement reached in May 2026, standalone Annex III systems move to 2 December 2027 and AI embedded in Annex I regulated products, including medical devices and IVDs, moves from 2 August 2027 to 2 August 2028. ^[6] Anyone assembling an EU clinical dataset should confirm the current Official Journal position rather than relying on any secondary summary, including this one.

Data type, AI use case, and the regulation that governs it

Worked example: evaluating a clinical AI wind-down dataset

Consider a diagnostic-imaging AI company shutting down after failing to raise a Series B. Through research agreements with four health systems it accumulated 1.8 million de-identified imaging studies with paired radiology reports, 240,000 studies with radiologist pixel-level annotations, and a separate ambient-documentation pilot that produced 410,000 physician-patient audio recordings paired with signed notes. The headline is 1.8 million studies. The asset is much smaller and much more specific. Here is how the evaluation actually runs.

• 1Establish which de-identification route was used, and get the paperwork. If the company used Safe Harbor, ask immediately whether head studies were defaced and whether dates survive at anything finer than year. If it used Expert Determination, request the written analysis, the expert's credentials, and the recipient assumptions. A determination made for an academic collaborator does not automatically extend to a commercial AI lab with a large adjacent data holding. ^[4]
• 2Separate weak labels from strong ones. All 1.8 million studies have a report, and report text can be NLP-mined into study-level labels. Those are useful for pretraining and report generation, and they are not a reference standard. The 240,000 annotated studies are a different product with a different price.
• 3Inside the annotations, find the submission-grade subset. Ask how many readers annotated each case, whether readers were blinded to one another, how disagreements were adjudicated, and whether inter-reader variability was measured. Suppose 88,000 of the 240,000 have documented multi-reader adjudication and the rest are single-reader. Only the 88,000 can support the documentation FDA expects in a marketing submission. ^[4]
• 4Find the confirmed-outcome slice. Which studies carry a pathology-confirmed diagnosis, a surgical finding, or documented follow-up establishing the true condition? Suppose 61,000 do. That subset, roughly 3.4% of the headline count, is what supports a diagnostic accuracy claim, and it is the number a serious buyer prices against.
• 5Check site independence, scanner diversity, and representativeness together. Four health systems beats one, but ask how volume distributes. If 80% of the annotated studies come from a single academic center on one scanner vendor, the corpus cannot serve as the independent external validation set FDA generally expects. Demographic distribution, disease severity spectrum, and acquisition protocol variation all appear on FDA's list of characteristics a sponsor must document, so a dataset shipping with that analysis already done removes months from a buyer's timeline. ^[4][6]
• 6Treat the ambient audio as a separate transaction with a separate consent problem. Of the 410,000 audio-note pairs, how many were collected in two-party consent states with documented patient consent to recording, and how many survive de-identification with the note still correctly paired? Suppose 290,000 clear both tests. Voice prints are an enumerated HIPAA identifier, so raw audio needs a different treatment path than the transcript does. ^[4]
• 7Date the corpus. Imaging protocols, reconstruction algorithms, and standard of care all move, and FDA asks how older data will be complemented or replaced so performance reflects current practice. Studies from 2016 are worth materially less than studies from 2024 for the same anatomy. ^[4]

How healthcare training data gets priced

Clinical datasets price against the same seven drivers Dayda applies across every vertical: volume, quality and annotation depth, domain scarcity, metadata richness, recency, licensing terms, and legal cleanliness. Healthcare rearranges their relative weight more than any other vertical does.

• Legal cleanliness is a gate rather than a multiplier. A buyer building a regulated device will not license data whose de-identification method is undocumented, at any discount. The Expert Determination report, the IRB determination or waiver, and the original consent language either exist or the deal does not happen.
• Annotation depth outweighs volume by a wide margin. Single-reader annotation makes a research corpus; blinded multi-reader adjudicated annotation makes a submission exhibit, and the upgrade is unrecoverable once the annotating clinicians have moved on.
• Domain scarcity concentrates in the long tail. Common findings on common modalities are relatively available. Rare disease imaging, pediatric studies, underrepresented populations, and unusual scanner or protocol combinations are what buyers struggle to source, and FDA's representativeness expectations make them necessary rather than nice to have. ^[4]
• Metadata richness means clinical linkage. An imaging study linked to the patient's labs, medications, pathology result, and outcome is worth several times the same image standing alone, because the linkage is what creates the reference standard.
• Recency carries a regulatory floor on top of the modeling one. FDA asks sponsors to plan for older data being complemented or replaced so performance tracks the current standard of care, which puts a documented decay curve on every clinical corpus. ^[4]
• Licensing structure works as it does elsewhere. Non-exclusive licenses let a health system monetize the same corpus repeatedly; exclusive licenses remove it from the market and command a premium. Field-of-use restrictions are unusually common here, where a seller may license imaging for detection research while withholding rights for any patient-facing diagnostic claim.

One healthcare-specific dynamic has no analogue in other verticals. Because clearance data must be defensible years after collection, buyers pay for durability of documentation, not only for the data. A corpus assembled by an organization that no longer exists, with no one left who can attest to the annotation protocol, is worth a fraction of an identical corpus whose protocol documents, annotator qualifications, and site agreements travel with it. Wind-down is precisely the moment that documentation is most likely to be lost and most easily preserved.

The confirmed-outcome data nobody is preserving

Healthcare AI is past the demonstration phase. Ambient scribes are cutting documentation time and burnout in multi-site studies ^[2][4], and the FDA has authorized more than 1,500 AI-enabled devices, a third of them in the last eighteen months. ^[6] The constraint on the next wave is not model architecture. It is the supply of imaging and clinical records that carry a confirmed outcome, a documented multi-reader reference standard, a defensible de-identification analysis, and enough site diversity to validate against.

That combination is created only inside operating health systems and clinical AI companies, and it degrades fast once one stops operating. Retention schedules purge archives, the radiologists who did the annotation leave, and the person who could explain the grading protocol becomes unreachable. The window in which an outcome-linked corpus is still intact, still documented, and still licensable is short. Inside that window it is one of the most valuable assets in the AI data market.