https://x.com/elder_plinius/status/1943171871400194231

system prompt for new grok 4. it has the same line about being allowed to say politically incorrect things, which supports my suspicion that the will stancil-raping version of grok the other day had more tweaks than just this

it appears to now be the strongest AI currently available. they spent a huge amount of compute resources on post-pretraining RL compared to all the other models. this is probably why it's performing so well on reasoning and problem-solving benchmark tests, because RL training helps a lot with this

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49093385)

https://www.mechanize.work/blog/sweatshop-data-is-over/

move over, teachers getting paid 100k to try to "teach" nigger kids how to write their name and tie their shoes

hello, RL engineers getting paid 300k to teach AI how to write its name and tie its shoes

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49093621)

that's just Part Of The Deal

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49093643)

an Elon company? no way am i trusting them unless the model is way better.

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49093657)

https://arxiv.org/pdf/2507.06952

paper that demonstrates that while LLMs excel at predictive tasks that fall within their training data, they can't generalize that predictive ability into a complete and accurate world model to make correct predictions on tasks that weren't within their training

lecun is right imo. statistical inferences do not lead to the ability to make generalized inferences

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094177)

good WSJ normie article about this

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094212)

https://arxiv.org/abs/2207.02098

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094428)

i think companies are doing RL now as more of a marketing gimmick than anything else. people are saying that the latest grok 4 is only performing so well on benchmarks because its RL training had overlap with the benchmark tests. this is what i mean when i say that they're just gaming the benchmark tests

i think the next few years might end up being the big base models trained with RL into a bunch of different specialized sub-models that are used in larger multi-agent architectures in order to be more useful in practice in different specific contexts

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094487)

that paper i linked seems to imply there are already models that consistently generalize better than transformers, so it's not clear we see companies flailing about for years using RL gimmicks with transformers. memory augmented transformers could be a near-term replacement for current models. if models don't improve substantially over the next year, there will be a strong motivation to try novel approaches like this.

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094525)

this would take me like...i dunno, 2 days to do before AI. not even counting all the time and effort that AI has saved me to get to the point of being able to understand this in the first place

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094574)

really good short article explaining how people think that we'll be able to use generative AI video to train robots to have functional world models

the notion of this actually working seems crazy to me but these guys are all very smart so it must be somewhat viable or someone would be calling it out as BS. it would cut robot training time and costs IMMENSELY and make them a lot more commercially viable and speed up robotics development by years

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094551)

looks like we found the real reason why grok 4 is so surprisingly good

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49094660)

https://x.com/ns123abc/status/1943806065524507007?s=46&t=YKr-jZOYUHE15Tew69wt4w

The Microsoft partnership was key to getting them off the ground, but now it’s an adversarial relationship.

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49095182)

I haven't believed in OpenAI since a lot of their top guys left a year ago. It's a really bad sign for an org when a bunch of your top level producers leave

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49095388)

China just released a new 32 billion parameter model that is apparently the 2nd best model in the world behind o3 and it's all open source(!)

If China could whip up the infrastructure and hardware for enough inference compute to meet demand, I think they could snap up a bunch of the market share for AI because they could undercut the US labs by so much

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49095399)

I don't trust the American LLM companies with my private data that actually matters either and I only work locally with it when I need to

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49095893)

There are ways to scrape web search info into your local model and incorporate it into your local LLM prompts but I haven't set it up yet because it hasn't really come up. It looks pretty easy to do though

(http://www.autoadmit.com/thread.php?thread_id=5749144&forum_id=2/en-en/#49095912)