Date: October 17th, 2026 10:48 PM Author: The Penis
It's funny because for transformer models it isn't even true. next token prediction describes the training objective. that was the entire big thing with transformer computation that the head can integrate information from widely separated positions in the context. There is more to it than that too. The system develops its own representations and algorithms in order to satisfy the optimization objective, and the final system that emerges still is not fully understood.
Date: October 17th, 2026 10:51 PM Author: Lucy Mercer (Safari)
That is what people literally say about it so the critique is reasonable. It's just false. Its obviously not analyzing a photograph by predicting the next word.