“‘Superhuman’ Isn’t Well Specified” by JustisMills

Author: LessWrong ([email protected])
Published: Sun 04 May 2025
Episode Link: https://www.lesswrong.com/posts/R7r8Zz3uRyjeaZbss/superhuman-isn-t-well-specified

Strength

In 1997, with Deep Blue's defeat of Kasparov, computers surpassed human beings at chess. Other games have fallen in more recent years: Go, Starcraft, and League of Legends among them. AI is superhuman at these pursuits, and unassisted human beings will never catch up. The situation looks like this:[1]

At chess, AI is much better than the very best humans

The average serious chess player is pretty good (1500), the very best chess player is extremely good (2837), and the best AIs are way, way better (3700). Even Deep Blue's estimated Elo is about 2850 - it remains competitive with the best humans alive.

A natural way to describe this situation is to say that AI is superhuman at chess. No matter how you slice it, that's true.

For other activities, though, it's a lot murkier. Take radiology, for example:

Graph derived from figure one of CheXNet: Radiologist-Level Pneumonia Detection [...]

---

Outline:

(00:10) Strength

(02:28) Effort

(04:35) And More

(06:36) Beyond Superhuman

The original text contained 1 footnote which was omitted from this narration.

---

First published:

May 3rd, 2025