"AI outperforms doctors in Harvard trial of emergency triage diagnoses | AI (artificial intelligence) | The Guardian"

This surprises me not at all. And it probably surprises no one except perhaps doctors.

Back in the '90's while working on my PhD in statistics, I came across some research on how physicians make diagnostic decisions. The most common paradigm was to pick one hypothesis and ride it until it was definitively proven wrong. Then, pick the next hypothesis, and repeat.

It would be much better to list all possible hypotheses, or at least the top k, and keep them in play, gathering data to discriminate them. Continually refine Bayesian posteriors of competing hypotheses.

No one looking carefully at diagnostic decision-making would ever have said that physicians generally perform optimally.

We could have developed diagnostic reasoning tools in the '90's that could have beaten physicians (but without natural language input). But no one would have seriously entertained using it. We weren't allowed. Too many questions about liability.

Now AI is a 500-pound gorilla and it can do what it wants. That's the biggest change. Medical diagnosis isn't really that hard a problem to solve. Robust handwriting recognition is probably harder.