Healthcare LLM Training Infrastructure

Why healthcare LLMs need clinical data

General-purpose LLMs underperform on clinical tasks. Without exposure to real clinical language — EHR notes, discharge summaries, radiology reports, pathology narratives, operative notes — the models default to generic medical knowledge scraped from public sources. The gap between a general-purpose LLM and a clinically useful one is real training data.

But clinical text is the most protected data category in any hospital. Patient identifiers, unstructured provider notes, diagnosis codes, medication histories — the value to an AI model is enormous, and the governance risk of moving it is equally so.

LoRA fine-tuning on clinical text at the edge

Rapha Protocol supports LLM fine-tuning workloads through LoRA (Low-Rank Adaptation) running directly on the edge appliance inside the hospital network. The base model is loaded once. Only the LoRA adapter weights are trained against local clinical text data. The adapter weights — typically a few megabytes — are exported. The raw clinical text remains inside the institution.

This approach is ideal for:

• Clinical note summarisation — fine-tune on real discharge summaries, clinic letters, and handover notes.
• Radiology report generation — train on paired imaging findings and structured report text.
• EHR phenotyping — extract structured clinical variables from unstructured provider notes.
• Medical coding — train ICD-10/SNOMED coding models on real clinical documentation.
• Clinical trial matching — fine-tune NLP models on real eligibility criteria and patient records.

GPU-accelerated edge compute

The edge appliance ships with Nvidia L4-class GPU compute capable of running QLoRA fine-tuning with 4-bit quantisation. Training scripts run in Docker containers with network_mode: none, read-only data mounts, and output validation — only approved file types (.safetensors, .json, .txt) can leave the environment. Raw clinical text, CSV exports, and FHIR bundles are blocked at the filesystem level.

How it compares to other approaches

• vs. HIPAA-compliant cloud — Cloud BAAs shift liability, not data risk. Data still leaves the institution. Compute-to-data keeps it on-prem.
• vs. Federated learning — Federated learning sends gradients, which can leak training data. Rapha sends only trained adapter weights after full local training.
• vs. Synthetic data — Synthetic clinical data is useful for prototyping but cannot capture real clinical distributions, rare disease patterns, or real-world treatment outcomes.
• vs. De-identification — De-identification is probabilistic and field-dependent. Compute-to-data removes the need to de-identify by never exporting data in the first place.