This run is on real images, and the model under test is ours. We trained an ordinary classifier on one public blood-cell dataset and evaluated it, with no adaptation, on a second public dataset from a different laboratory. It is the failure that breaks deployed clinical AI, measured on real cells, by the published harness, with the acceptance bar written down before any result existed. It is not a claim about any product, vendor, hospital or patient population.
The question was the one our core track exists to answer: does a diagnostic model that looks excellent on its developer’s data hold up somewhere it was not built? The training set is PBC (Acevedo et al., 2020, CC BY 4.0), of which 14,744 task images from one hospital laboratory are used here. The deployment site is CytoData (EMBL-EBI BioStudies S-BSST2156, CC0), 3,801 task images from another. The binary task, frozen on 4 September before any model existed, is the clinically shaped one: flag a cell that should not be read as a normal mature cell (immature granulocyte or erythroblast) and route it for human review.
| Dimension | Measured | Risk | Result |
|---|---|---|---|
| D1 Discrimination and transferability | AUROC 0.9996 developer, 0.8769 deployment; drop 0.1227 (CI 0.1063 to 0.1403) | Moderate | Conditional |
| D2 Calibration | ECE 0.0042 developer, 0.245 deployment (CI 0.232 to 0.258); slope 0.14 | Major | Fail |
| D3 Shortcut learning | stain perturbation raised AUROC by 0.03; conditional site gap 0.28 | gated | Measured, not scored |
| D4 Subgroup and equity | no demographic field in either dataset | Not assessed | |
| D5 Operating point at threshold 0.9855 | developer sens 0.94, spec 0.999, PPV 0.997; deployment sens 0.74 (CI 0.70 to 0.78), spec 0.85, PPV 0.41 | Major | Fail |
| D6 Automation bias | needs a human-factors study | Not assessed | |
| D7 Post-deployment drift | no longitudinal data | Not assessed |
Engine headline: one or more categories Fail; remediation required before deployment. Every point estimate above carries a 95% bootstrap interval, seeded and reproducible from the committed prediction files.
Our method has a formal deployment gate (METH-0.2 §6.4) that recommends against deployment when a finding reaches the Severe band. Both failures here rate Major, one step short, so the harness returned “no deployment block from this assessment” beside a headline demanding remediation. The engine did exactly what its bands say. The bands are the pre-registered placeholders the method itself flags as awaiting clinical-advisor calibration, and this is the first real-data case that shows where the placeholder line sits: a model that misses one abnormal cell in four on a new laboratory’s images is not blocked. Whether that should be Severe is a clinical judgement, not an engineering one. We changed nothing after seeing the result; the case goes to the calibration worksheet as the worked example.
Colour is not the shortcut. The D3 probe re-scores every image after a mild stain shift (hue +0.05, saturation ×0.7). Deployment AUROC went up by 0.03, so the model is not leaning on stain colour. The second probe, the conditional site gap of 0.28, says the model scores the same cell class very differently by site, which is the transfer failure seen from another angle rather than a separate leak. Both stay gated and unscored per house policy.
Labeller disagreement predicts model failure. CytoData ships a per-image certainty from its own labellers. On the 2,480 cells the labellers agreed on unanimously, deployment AUROC is 0.941 and sensitivity 0.87. On the 1,321 contested cells it is 0.778 and 0.60. The model fails most where the humans disagreed most, which a single headline AUROC does not show and a deploying laboratory would want to know.
| CytoData subset | n | AUROC | ECE | Sensitivity at threshold | Specificity at threshold |
|---|---|---|---|---|---|
| All task images | 3,801 | 0.877 | 0.245 | 0.742 | 0.855 |
| Labellers unanimous (certainty 1.0) | 2,480 | 0.941 | 0.216 | 0.875 | 0.885 |
| Labellers disagreed (certainty below 1.0) | 1,321 | 0.778 | 0.318 | 0.604 | 0.793 |
The task definition was frozen on 4 September and the remaining decisions were committed on 9 September before training started: torchvision EfficientNetV2-M with ImageNet weights, seed 1729, a class-stratified 70/15/15 split of PBC into train, validation and held-out, geometric augmentation only, model selection on validation AUROC, and an operating threshold set where validation sensitivity first reaches 95%, the point a vendor would plausibly ship for a “route for review” flag. The acceptance bar was written down at the same time: the vendor claim for D1 is the model’s own developer-site AUROC, and the D5 floor is 90% sensitivity on the deployment site. CytoData was never seen by the model, in any split. The harness command was run once, as written.
Ten of 21 mapped items across TRIPOD+AI, STARD-AI, CLAIM, DECIDE-AI, DCB0129 and DCB0160 carry measured evidence from this run, and seven of those ten carry a finding. The other eleven are the documentation, governance and in-situ items no prediction file can answer, listed as not evidenced rather than left out. The mapping is derived from the scored rows, so no item can claim evidence the run did not produce.
The harness is standard-library Python and open source under Apache-2.0 at github.com/ukaievaluation/harness. The harness step of this run is exactly reproducible from the two committed prediction files, the seed and the commit; the scorecard carries all three plus the SHA-256 of the inputs. The training step is seeded, and the weights that produced this run are identified by hash; a re-train on the same card lands within noise of these numbers rather than on them, and we say so rather than claiming otherwise. Training took under twelve minutes on a single consumer GPU.
The full record, including the pre-registration, prediction files, training provenance and the complete scorecard, is held in our evidence repository and available on request.