Date: October 17th, 2026 10:48 PM Author: The Penis
It's funny because for transformer models it isn't even true. next token prediction describes the training objective. that was the entire big thing with transformer computation that the head can integrate information from widely separated positions in the context.