This Nature Medicine published study has a strong warning for AI in healthcare.

Frontier AI in healthcare has a hidden failure mode: it can look medically brilliant while being clinically unready.

The authors tested frontier AI models on health benchmarks, then added stress tests to see whether the models were actually robust or just good at passing exams.

Found that the models were brittle.

i.e the models could give the right answer in a normal test, but fail when the question was slightly changed, when important information was removed, or when the image-text setup was altered.

One strange result was that some models could still guess the correct answer even when key inputs were removed, which suggests they may be using shortcuts rather than truly understanding the medical case.

the models sometimes gave convincing explanations that sounded medical and logical, but the reasoning was flawed.

The final conclusion is not “AI is useless in medicine” but that "benchmark success is not the same as clinical readiness.”