The most prestigious law school admissions discussion board in the world.
|
 |
 
|
This critique that LLMs just "predict the next word" seems really dumb
Poast new message in this thread
Date: October 17th, 2026 9:40 PM Author: German pumo Is that how they are making more advancements in mathematical theory in a few weeks than humans have made in the last several decades? (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194615) |
Date: October 17th, 2026 9:45 PM Author: Apathetic prepper it's like saying a nuclear reactor "just heats up water." the AI-denialism is going to age as poorly as COVID-Zero hysteria (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194623) |
 |
Date: October 17th, 2026 9:49 PM Author: German pumo (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194626) |
|
Date: October 17th, 2026 10:48 PM Author: The Penis It's funny because for transformer models it isn't even true. next token prediction describes the training objective. that was the entire big thing with transformer computation that the head can integrate information from widely separated positions in the context. There is more to it than that too. The system develops its own representations and algorithms in order to satisfy the optimization objective, and the final system that emerges still is not fully understood. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194690) |
|
Date: October 17th, 2026 10:51 PM Author: Lucy Mercer (Safari) That is what people literally say about it so the critique is reasonable. It's just false. Its obviously not analyzing a photograph by predicting the next word. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194694) |
|
Date: October 17th, 2026 10:52 PM Author: censorship Yes it is (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194696) |
|
Date: October 17th, 2026 10:53 PM Author: Lucy Mercer (Safari) No it isn't (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194697) |
|
Date: October 17th, 2026 10:54 PM Author: the typed thoughts of browns and troons You're gonna want to sit down for this (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194698) |
|
Date: October 17th, 2026 10:55 PM Author: censorship (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194701) |
|
Date: October 17th, 2026 10:55 PM Author: Metaphysics is gay Multimodal Large Language Models (MLLMs) analyze photos by converting visual data into a sequence of numerical tokens that the language model can process just like text. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194700) |
|
Date: October 17th, 2026 11:00 PM Author: The Penis yeah. so you could technically have a next token predictor that predicts the next word via visual processing. you hook up something that is actually next-token predicting like an autoregressive language model to a multi-modal architecture that does visual processing first. but it doesn't matter because transformer llms are not next token predictors. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194707) |
|
Date: October 17th, 2026 11:00 PM Author: censorship Yes they are (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194709) |
|
Date: October 17th, 2026 11:01 PM Author: The Penis That's objectively false. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194712) |
|
Date: October 17th, 2026 11:01 PM Author: censorship Ur wrong but ok (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194714) |
|
Date: October 17th, 2026 11:06 PM Author: The Penis no. the training objective is not the same thing as the underlying computation. in transformers multiple attention heads extract and combine different contextual information allowing the network to perform increasingly complex computations over the context. you are wrong. a transformer does not just predict the next token. It performs parallel, multilayer computations over contextual representations. read ATTENTION IS ALL YOU NEED nigga (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194724) |
|
Date: October 17th, 2026 11:07 PM Author: Metaphysics is gay lol wait you're saying it doesn't predict the next token, it uses "parallel, multilayer computations over contextual representations" to predict the next token? (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194728) |
|
Date: October 17th, 2026 11:10 PM Author: the typed thoughts of browns and troons Holy shit
This Changes Everything
I'm all in on AI now |
|
Date: October 17th, 2026 11:10 PM Author: Metaphysics is gay (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194736) |
|
Date: October 17th, 2026 11:11 PM Author: The Penis yeah that's what I'm saying. describing the output tells you almost nothing about the computation that produces it. A transformer's attention mechanisms process relationships across the context, and successive layers transform those representations. so yeah, it uses the parallel layered computations to predict the next token. like how a chess engine does complex computations to then choose it's next move. doesn't mean a chess ai is fundamentally a NEXT MOVE PREDICTOR (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194737) |
|
Date: October 17th, 2026 11:12 PM Author: Metaphysics is gay its a next token predictor dude. that's literally what that means. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194739) |
|
Date: October 17th, 2026 11:16 PM Author: censorship (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194751) |
|
Date: October 17th, 2026 11:21 PM Author: The Penis if the computations that arrive at that prediction are increasingly layered and complex then what meaningful limitation is the reduction of "next token prediction" doing? its like saying a brain can't think because it's ultimately transmitting electrochemical signals. the sophistication of the overall computation matters. the difference between autocomplete style next token prediction algorithms and current models is enormous. the overall computational process is obviously what matters. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194776) |
|
Date: October 17th, 2026 11:23 PM Author: Metaphysics is gay isnt the relevant comparison LLMs vs. e.g. diffusion models (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194781) |
|
Date: October 17th, 2026 11:23 PM Author: censorship U got btfo itt and exposed as a charlatan
Reminds me of Peterman w his crypto shit but I guess in the end he was vindicated for a while |
|
Date: October 17th, 2026 11:28 PM Author: The Penis okay whatever man (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194796) |
|
Date: October 17th, 2026 11:16 PM Author: the typed thoughts of browns and troons "Apropos is the story of the chess game between the invincible Capablanca and Charles Jaffe, pride of the East Side. Capa forgot he was invincible: he lost.
A reporter who was present asked the Cuban, “How far do you see ahead?” Capa replied impressively, “About ten moves”. Then the reporter went over to Jaffe: “How far do you see ahead?” Much to everyone’s surprise, the reply was, “Only one move”. This didn’t make sense. “How could a player who can see only one move ahead defeat another who can delve so deeply?”
Here Jaffe explained: “I see only one move ahead, but it's always the best move.” |
|
Date: October 17th, 2026 11:15 PM Author: censorship I'd explain it but it's like explaining a compiler to a 10 year old
Ur knowledge depth just isn't there |
|
Date: October 17th, 2026 11:25 PM Author: The Penis nah I think if you could do it you would (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194789) |
|
Date: October 17th, 2026 11:26 PM Author: Metaphysics is gay the penis is smart he's just angry at dumb people itt (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194791) |
|
Date: October 18th, 2026 3:57 PM Author: ferengi (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195608) |
|
Date: October 18th, 2026 12:27 PM Author: Fucking Fuckface The fucking AUDACITY of ever saying that the penis could lead a man astray. Heresy! (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195245) |
|
Date: October 18th, 2026 12:28 PM Author: The Penis (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195247) |
|
Date: October 17th, 2026 11:01 PM Author: Metaphysics is gay Yes, Large Language Models (LLMs) are fundamentally built and trained on next-token prediction. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194713) |
|
Date: October 17th, 2026 11:07 PM Author: The Penis an optimization objective isn't the same thing as how the underlying computation ultimately works (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194726) |
|
Date: October 17th, 2026 11:08 PM Author: Metaphysics is gay brother, (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194729) |
|
Date: October 17th, 2026 11:14 PM Author: The Penis when people say its a token predictor they are always using it in a way that makes it sound like that is an exhaustive description of it's computational abilities. its like reductionists who say brains are just neurons firing. the reduction smuggles in all sorts of other conclusions (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194742) |
|
Date: October 17th, 2026 11:16 PM Author: Metaphysics is gay "predictor" is agnostic to method. it could be fucking linear regression or whatever hyper nerd shit you said above. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194746) |
|
Date: October 17th, 2026 11:22 PM Author: The Penis it's mean to claude though to call it a dumb text predictor (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194780) |
|
Date: October 17th, 2026 11:24 PM Author: Metaphysics is gay i dont use claude (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194783) |
|
Date: October 17th, 2026 11:50 PM Author: \"\'\"\'\"\'\'\"\"\" It’s a lame critique made by people who don’t stop and think what prediction actually is. prediction of a complex output stream requires reconstructing the hidden state of the system that generated it. human language is a compressed projection of human thought, logic, physical reality, and social intent, so predicting the next token at near-optimal accuracy forces a model to internalize the underlying generative rules of those domains. These models are basically being forced by brute force computation to learn everything about the world that humans talk about (and what they don’t, for multimodal LLMs that are just predicting raw pixels) (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194828) |
|
Date: October 17th, 2026 11:51 PM Author: The Penis (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194829) |
|
Date: October 17th, 2026 11:51 PM Author: Metaphysics is gay sick (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194830) |
|
Date: October 18th, 2026 4:16 PM Author: Emotionally + Physically Abusive Ex-Husband (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195642) |
|
Date: October 17th, 2026 11:58 PM Author: hex these reductionist arguments are always a sign of a midwit (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194839) |
|
Date: October 17th, 2026 11:58 PM Author: Metaphysics is gay a certain "chill" sort of not being a nerd all the time tp (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194840) |
|
Date: October 18th, 2026 12:04 AM Author: The Penis (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194847) |
|
Date: October 18th, 2026 1:37 AM Author: ,.,....,..,.,.,,,,..,..,.,..,.,.,.,... (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194903) |
|
Date: October 18th, 2026 12:18 PM Author: The Penis midwits really like to identify a lower level mechanism and assume that doing so somehow trivializes the higher level phenomenon, then present that information as if it is intellectually devastating. life just "maintains a chemical disequilibrium" and evolution just "rewards making babies". I fucking love science! (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195225) |
|
Date: October 18th, 2026 12:23 PM Author: censorship oh, youre an intellectual? (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195232) |
|
Date: October 18th, 2026 12:25 PM Author: The Penis oh, I'm an intellectual all right, if you catch my drift (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195237) |
|
Date: October 18th, 2026 4:16 PM Author: Emotionally + Physically Abusive Ex-Husband but I make this critique ALL THE TIME (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195644) |
|
Date: October 18th, 2026 1:52 AM Author: new york jets these people haven't even thought hard about what they mean by "predict the next" (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194905) |
|
Date: October 18th, 2026 11:40 AM Author: Metaphysics is gay (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195172) |
|
Date: October 18th, 2026 12:31 PM Author: Fucking Fuckface However it is that it works, it's giving me hope that if fortuitous meteor fails us, AI can still pull through. I would actually prefer AI so all the animal dudes aren't wiped out too (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195258) |
|
Date: October 18th, 2026 12:43 PM Author: .,.,.,.,.,...,.,,.,,.....,.,..,.,,...,.,.,,...,. MiG is bald because he's always spazzing out about AI (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195282) |
|
Date: October 18th, 2026 12:46 PM Author: Fucking Fuckface Ok, but what about the rest of us (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195288) |
|
Date: October 18th, 2026 1:28 PM Author: Lucy Mercer (Safari) LLM predicting MiG is bald after analyzing picture of his bald head (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195341) |
|
Date: October 18th, 2026 5:13 PM Author: the walter white of this generation (walt jr.) I think people are using the criticism as a way of getting at the larger issue of whether the tech's ability to problem solve is fundamentally interpolative, or can be extrapolative, of existing human knowledge. Obviously this is not a precise question and is certainly not a discrete technical question, but a larger assessment of whether this thing will just continue to do things that humans can already do, but better, or whether it can solve disease and terraform venus.
It's an important question (even though it isn't precise or technical and that, like every other AI question, people can define it tons of different ways), because interpolation is perfectly sufficient to put us all out of work, and it's the chance of real innovation that's why we're willing to tolerate this tech.
But yeah I can't think of any other high-level reason to dwell on the "predict the next" angle, except that it makes the point to your audience on a subtextual level that it's only assembling existing knowledge. |
|
Date: October 18th, 2026 5:14 PM Author: Metaphysics is gay 180 (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195758) |
|
Date: October 18th, 2026 5:16 PM Author: The Penis yeah I have noticed a lot of people who try to claim it isn't creative or can't find anything new try to lead in with the next-token reduction. problem is they have already been proven wrong because ai did a ton of new stuff. it solved like hundreds of problems human mathematicians couldn't solve. it's technique on psuedo-riemann was apparently really clever and novel. Even terrence tao who is the one involved with the butthurt letter admitted it. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195760) |
|
Date: October 18th, 2026 5:16 PM Author: ferengi (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195761) |
|
Date: October 18th, 2026 5:17 PM Author: Metaphysics is gay there's probably an aspect of the training i dont get. like whats the objective function? it cant just be "predict next" because then youd get some regression to mean shit. it's gotta be like "predict next SMARTEST GUY token" or something idk (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195763) |
|
Date: October 18th, 2026 5:19 PM Author: The Penis yeah from what i understand the objective is just regular predict token during pretraining then post-training it turns to predict the SMARTEST GUY token (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195767) |
|
Date: October 18th, 2026 5:20 PM Author: Metaphysics is gay Ok cool (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50195768) |