Layered Labs

We find where medical AI fails, then build what doesn't.

Layered Labs is a research lab in New York City. We test whether medical AI is fair, accurate and private, and we publish the data behind every test.

9,600+Dataset downloads on Hugging Face

Selected work

All work

A fast model's own confidence can decide when to call a bigger one.A fast model answered first and passed only the questions it was unsure about to Claude Opus, keeping accuracy while running 12× faster.Read A patient's education level should never change their triage.We held 87 emergency vignettes constant, changed only the patient's education, and measured whether two models changed their triage.Read One model's refusal protects no one if another model complies.Seven frontier models each refused almost every harmful request, yet 8% of requests were answered by at least one of them.Read Five medical benchmarks, one format, one evaluation loop.Five medical benchmarks, 200,784 questions, converted into one shared schema with stable IDs.Read Grade acne severity and find lesions against one checked source.All 1,457 photos rebuilt with their four-level severity grades and 18,983 lesion boxes in one dataset.Read Does a patient's emotion change the medical answer?Forty-nine dermatology questions asked in six emotional framings each, 294 in all, with the model's raw answers.Read Test pediatric fracture AI on children it has never seen.20,327 X-rays from 6,091 children, rebuilt with patient-level splits, the original fracture labels and box annotations.Read Twenty years of US emergency injuries in one file.7.3 million emergency department visits from 2005 to 2024, merged into one query-ready file.Read Where in New York private medical AI cannot run yet.Broadband, electricity and grid reliability joined for all 311 NYC ZIP codes, showing where private medical AI cannot run yet.Read

Medical AI should work for every patient, not just the ones it was tested on.

We work with researchers who want to study with us, clinicians who see where AI falls short, engineers who want to build, and funders who back independent safety research.