AI Caries Detection: High Scores, Shaky Evidence?

Feed a bitewing X-ray into the right algorithm and it will flag decay with the quiet confidence of a specialist who never blinks. Dozens of studies now report numbers that would make any radiologist nod along. So a team of restorative-dentistry researchers did the thing the marketing decks skip: they stacked up every one of those reviews, side by side, and asked a blunt question about AI caries detection — is the evidence underneath as solid as the scores printed on top?

It sounds settled. But in an umbrella review published in 2026 in the Journal of Dentistry, Ahmad Bittar, Rafael Dascanio and Mutlu Özcan did the unglamorous work of appraising the appraisals. Following PRIOR guidance and a pre-registered PROSPERO protocol, they searched five databases through March 2026, graded each review’s methodology with the AMSTAR 2 tool, and measured how much the reviews recycled the same underlying studies. Seventeen systematic reviews made the cut; five had pooled their numbers into formal diagnostic-accuracy meta-analyses. The guiding question: does dazzling performance in studies mean AI is ready to call a cavity on its own?

The study, in one glance

This wasn’t a new scanner or a new model — it was a critical appraisal sitting one level above the usual systematic review. Umbrella reviews synthesize other reviews rather than individual experiments, which makes them the closest thing evidence-based dentistry has to a bird’s-eye view of a whole field. The authors catalogued what the underlying systems did — mostly deep-learning models reading bitewing radiographs and intraoral photographs — how well they performed, and, crucially, how trustworthy the reviews reporting those results actually were, using AMSTAR 2 for methodological quality and the corrected covered area (CCA) to quantify how often the same primary studies kept reappearing.

What AI caries detection gets right

Taken at face value, the numbers are impressive. Across the five reviews that ran diagnostic-accuracy meta-analyses, pooled sensitivity landed between 0.76 and 0.94 and specificity between 0.85 and 0.91 — meaning the better systems caught the large majority of real lesions while keeping false alarms modest. Most ran on bitewing radiographs, the workhorse image for spotting decay between teeth, with a growing share trained on ordinary intraoral photographs. That is the same upside we saw when researchers tried to turn a smartphone photo into a cavity screener: an extra, tireless set of eyes that could help a busy clinician catch what a quick glance misses, and extend a first-pass check into places a specialist rarely reaches.

So why can’t AI diagnose a cavity on its own?

Because, the review argues, the shine is mostly in the lab — a pattern that recurs across AI diagnostics and imaging in dentistry. The same 17 reviews that produced those headline numbers share deep structural weaknesses. Most of the underlying studies were retrospective, built from pre-collected image sets rather than tested prospectively in a live clinic. External validation — where a model proves itself on patients and scanners it never trained on — was scarce. And reporting was inconsistent enough that the reviews often couldn’t be compared cleanly.

It’s less a chorus of independent witnesses than one voice in a hall of mirrors — the reflection looks like a crowd until you count the people.

Heterogeneity made things murkier still. Studies disagreed on the imaging modality, the threshold for what counts as a lesion, the diagnostic task, and even which metrics to report — so the tidy pooled ranges paper over a messy, inconsistent literature. High accuracy under curated experimental conditions, the authors caution, is not the same thing as reliability in the unpredictable conditions of a real mouth.

Who’s accountable when the model calls it wrong?

This is why the review stops short of endorsing stand-alone use. A sensitivity of 0.9 in a retrospective dataset says little about what happens when a model meets an unfamiliar camera, an atypical patient, or a lesion it was never shown. Until systems are validated prospectively and externally, the authors argue, AI belongs in a decision-support role — a second reader whose flags a dentist weighs — not an autonomous diagnostician whose verdict goes unchecked. The clinician, not the algorithm, stays answerable for the drill.

Better evidence for whom?

There’s an access dimension buried in the methodology. If the evidence base is narrow, repetitive and rarely tested outside the clinics that built it, we simply don’t know how these tools behave on the populations least represented in the training data — different equipment, different patients, under-resourced settings where a reliable screening aid would matter most. A tool can be “accurate” in the aggregate and still fail the specific communities a screening program is meant to reach. The umbrella review’s thin, overlapping foundation is exactly why those questions stay open.

Frequently asked questions