The most prestigious law school admissions discussion board in the world.
|
 |
 
|
This critique that LLMs just "predict the next word" seems really dumb
Poast new message in this thread
Date: October 17th, 2026 9:40 PM Author: German pumo Is that how they are making more advancements in mathematical theory in a few weeks than humans have made in the last several decades? (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194615) |
Date: October 17th, 2026 9:45 PM Author: Apathetic prepper it's like saying a nuclear reactor "just heats up water." the AI-denialism is going to age as poorly as COVID-Zero hysteria (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194623) |
 |
Date: October 17th, 2026 9:49 PM Author: German pumo (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194626) |
|
Date: October 17th, 2026 10:48 PM Author: The Penis It's funny because for transformer models it isn't even true. next token prediction describes the training objective. that was the entire big thing with transformer computation that the head can integrate information from widely separated positions in the context. There is more to it than that too. The system develops its own representations and algorithms in order to satisfy the optimization objective, and the final system that emerges still is not fully understood. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194690) |
|
Date: October 17th, 2026 10:51 PM Author: Lucy Mercer (Safari) That is what people literally say about it so the critique is reasonable. It's just false. Its obviously not analyzing a photograph by predicting the next word. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194694) |
|
Date: October 17th, 2026 10:52 PM Author: censorship Yes it is (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194696) |
|
Date: October 17th, 2026 10:53 PM Author: Lucy Mercer (Safari) No it isn't (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194697) |
|
Date: October 17th, 2026 10:54 PM Author: listening intently as peter thiel stutters You're gonna want to sit down for this (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194698) |
|
Date: October 17th, 2026 10:55 PM Author: censorship (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194701) |
|
Date: October 17th, 2026 10:55 PM Author: Clean your Godhead Multimodal Large Language Models (MLLMs) analyze photos by converting visual data into a sequence of numerical tokens that the language model can process just like text. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194700) |
|
Date: October 17th, 2026 11:00 PM Author: The Penis yeah. so you could technically have a next token predictor that predicts the next word via visual processing. you hook up something that is actually next-token predicting like an autoregressive language model to a multi-modal architecture that does visual processing first. but it doesn't matter because transformer llms are not next token predictors. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194707) |
|
Date: October 17th, 2026 11:00 PM Author: censorship Yes they are (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194709) |
|
Date: October 17th, 2026 11:01 PM Author: The Penis That's objectively false. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194712) |
|
Date: October 17th, 2026 11:01 PM Author: censorship Ur wrong but ok (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194714) |
|
Date: October 17th, 2026 11:06 PM Author: The Penis no. the training objective is not the same thing as the underlying computation. in transformers multiple attention heads extract and combine different contextual information allowing the network to perform increasingly complex computations over the context. you are wrong. a transformer does not just predict the next token. It performs parallel, multilayer computations over contextual representations. read ATTENTION IS ALL YOU NEED nigga (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194724) |
|
Date: October 17th, 2026 11:07 PM Author: Clean your Godhead lol wait you're saying it doesn't predict the next token, it uses "parallel, multilayer computations over contextual representations" to predict the next token? (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194728) |
|
Date: October 17th, 2026 11:10 PM Author: listening intently as peter thiel stutters Holy shit
This Changes Everything
I'm all in on AI now |
|
Date: October 17th, 2026 11:10 PM Author: Clean your Godhead (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194736) |
|
Date: October 17th, 2026 11:11 PM Author: The Penis yeah that's what I'm saying. describing the output tells you almost nothing about the computation that produces it. A transformer's attention mechanisms process relationships across the context, and successive layers transform those representations. so yeah, it uses the parallel layered computations to predict the next token. like how a chess engine does complex computations to then choose it's next move. doesn't mean a chess ai is fundamentally a NEXT MOVE PREDICTOR (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194737) |
|
Date: October 17th, 2026 11:01 PM Author: Clean your Godhead Yes, Large Language Models (LLMs) are fundamentally built and trained on next-token prediction. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194713) |
|
Date: October 17th, 2026 11:07 PM Author: The Penis an optimization objective isn't the same thing as how the underlying computation ultimately works (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194726) |
|
Date: October 17th, 2026 11:08 PM Author: Clean your Godhead brother, (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2\u0026mark_id=5304212",#50194729) |