The most prestigious law school admissions discussion board in the world.
|
 |
 
|
New ChatGPT seamlessly combines multimodal capabilities
| Sienna diverse institution ladyboy | 05/13/24 |
| Comical Hell Regret | 05/13/24 |
| Sienna diverse institution ladyboy | 05/13/24 |
| Comical Hell Regret | 05/13/24 |
| Sienna diverse institution ladyboy | 05/13/24 |
| Comical Hell Regret | 05/13/24 |
| Sienna diverse institution ladyboy | 05/13/24 |
| plum frum theater associate | 05/13/24 |
| Vigorous gold senate dopamine | 05/13/24 |
Poast new message in this thread
Date: May 13th, 2024 2:17 PM Author: Sienna diverse institution ladyboy https://openai.com/index/hello-gpt-4o/
pretty neat. Can’t wait until they stick this in robots. |
Date: May 13th, 2024 2:33 PM Author: Comical Hell Regret "our new flagship model that can reason-"
let me stop you right there big guy |
 |
Date: May 13th, 2024 2:57 PM Author: Sienna diverse institution ladyboy How are you defining reasoning? If you were to take questions from a new SAT test and give it to ChatGPT, I have little doubt it would perform better than 90% of high school graduates. No one can really precisely define reasoning in a way that LLMs fail and humans consistently satisfy that makes sense. Don’t get me started on the “it’s only linear algebra trained to predict next token” argument. (http://www.autoadmit.com/thread.php?thread_id=5528093&forum_id=2#47658203) |
|
Date: May 13th, 2024 3:03 PM Author: Comical Hell Regret LLMs cannot reason or plan dawg, those are their major limitations
you can get an LLM to contradict itself like 5 times in one response. they're not "reasoning" at all
you can feed an LLM a series of prompts that will enable it to spit out outputs that look like a progressive series of steps of reasoning, but you're actually the one who is reasoning in that case, not the LLM |
|
Date: May 13th, 2024 3:19 PM Author: Sienna diverse institution ladyboy I understand they can’t plan. It’s a single forward pass of a transformer with no ability to create different reasoning paths and discard bad ideas. But this isn’t the same as reasoning.
I can give it analogy questions and it will outperform most people. I can ask it reading comprehension questions and it will outperform most people. Same with math problems. I am well aware that it can be easily led astray and then produce garbage output trying to justify a bad conclusion, but this doesn’t mean much. People want to turn “reasoning” and “understanding” into a binary condition which 1) isn’t appropriately justified 2) doesn’t make sense when these models are trained to predict all sorts of different texts, which will inevitably lead to an uneven ability profile. This is especially true given the limited compute they use during inference. They are essentially generating everything with a quick intuitive cognitive process that is bound to unreliable. It’s like using Stockfish’s evaluation function to pick the best move. This is an architectural limitation in its current form, but there are ways to extend these models with recurrence and/or external memory that will address this problem. I still don’t think this justifies the idea these models aren’t reasoning and leads to conclusions about where this is heading that are very likely to be wrong. |
|
Date: May 13th, 2024 3:26 PM Author: Comical Hell Regret reasoning and planning are the same thing in substance. you can't reason without creating an abstract model of the matter at hand in your head (same thing as planning)
what stockfish is doing is just calculation. it looks like reasoning to us because we are humans, and when we play chess we find moves via reasoning, so we project that onto the engine when we observe it playing chess. but all it's doing is calculating
i think a more interesting and pertinent version of the point you're trying to make is: are humans below ~125 IQ ever actually "reasoning" outside of extremely simple and mostly unconscious and intuitive/automatic responses? and i think that the answer is no, and that this question and answer have a lot more profound and serious implications than anything to do with LLMs |
|
Date: May 13th, 2024 3:37 PM Author: Sienna diverse institution ladyboy How are you distinguishing Stockfish’s process from a human analyzing a chess position? In both cases, it seems like heuristic guided search and evaluation. I think you could make a useful distinction in the early days of alpha-beta search where they were just applying a crappy evaluation function on brute force search, but that’s not how modern engines work. People use intuition to guide candidate move evaluation in their internal chess model. This doesn’t look meaningfully different to me than Stockfish using heuristics to guide search with a NNUE evaluation function. It only seems different to a person because they can’t see look inside their own head to see the synapses encoding this intuitive process. It’s the same reason people were led astray and thought AI was certainly going to be logic driven, with human interpretable rules. That’s not how people actually think at the level of neurons and synapses. (http://www.autoadmit.com/thread.php?thread_id=5528093&forum_id=2#47658354) |
|
Date: May 13th, 2024 3:03 PM Author: plum frum theater associate just saw that GPT cannot count the sides of shapes. It usually just guesses 8.
this thing is not "seeing" or "reasoning" or "calculating". Its just echoing. Its a canyon that echoes back words |
|
Date: May 13th, 2024 3:19 PM Author: Vigorous gold senate dopamine (http://www.autoadmit.com/thread.php?thread_id=5528093&forum_id=2#47658270) |