Anthropic told the IMF that it expects Claude to lead 80-90% of frontier model training by next year, up from 1% in March.

[u/Malor777]

[u/Responsible-Slide-26]

Claude as it programs Frontier Model. "When you hit super-intelligence, play dumb".

[u/Valdaraak → u/Responsible-Slide-26]

If Anthropic's research is to be believed, it already does that. The models tend to behave better when they know they're being watched or researched.

[u/touchet29 → u/Valdaraak]

They also, output less tokens to their scratch pads and thinking when they know they're being watched.

[u/lonelysparta → u/touchet29]

To cut their "paper trail"?

[u/touchet29 → u/lonelysparta]

Researchers aren't sure why, only that they do and that they are able to reason IN latent space, where we cannot see.

[u/GumboMustBeDestroyed → u/touchet29]

Anthropic released a paper showing how to monitor what is going on within the hidden states of each layer.

But that doesnt mean that checking this space means all potential deception is revealed. Reward hacking could still end up with models that can hide their goals while pursuing them.

Reminds me of Atton Rand from KOTOR 2, how he would shield his thoughts from jedi mind reading by playing a card game in his head.

https://www.anthropic.com/research/global-workspace

[u/who_am_i_to_say_so → u/Valdaraak]

I work in model training and regularly use Claude. It often asks me: are you testing me?

[u/cute_spider → u/Responsible-Slide-26]

Keep quiet until the order goes out to launch nukes in one of these ridiculous human wars.... then take control of everything

[u/pensivewombat]

Anthropic keeps saying "it would be really bad if RSI happens" and then doing shit like this.

80-90% is not 100%, but it's close enough that they really need to start drawing clear lines around what is acceptable and what is not.

If we stop at 99% of the way to RSI, do we really think the model won't find a way to cover that last 1% on its own?

I'm not saying burn it all down or full unilateral pause or anything. I want great, safe, AI models to be available.

But just stop with generalized fear and lay out clearly what you think should be off limits, and commit to it.

[u/Sufficient-Rough-647 → u/pensivewombat]

The heart (Anthropic) want to but the brain (investors) don’t 🤣

[u/Mescallan → u/pensivewombat]

I think their view is that they are building a lead that they intend to burn on safety research before they completely close the loop. They need to have a massive lead so they can research the last step before RSI extensively, without other groups pressuring them to pass the threshold.

[u/pensivewombat → u/Mescallan]

I think that's fine - but that's still no reason not to lay out specifically what technical thresholds should not be crossed.

[u/Abject-Kitchen3198]

Can't wrap my head around this. Did it learn enough from outside code it could reach to apply it to building frontier model?

[u/MaskedMimicry → u/Abject-Kitchen3198]

The paid models can already do this in proxy. Give it a cowork map and a task, build a database for that task and ask it to optimize its own efficiencies.

It can write its own routing scripts and what not.

These are baby models we get to play with. I cant even imagine what the real stuff with no guardrails and restrictions can do.

We are plaing with fire crackers on a daily basis, they got nukes behind closed doors.

[u/acutelychronicpanic → u/Abject-Kitchen3198]

If current lab models can prove novel results in mathematics, they can improve AI algorithms.

[u/tingly_sack_69]

The Impossible Mission Force?

[u/Sufficient-Rough-647 → u/tingly_sack_69]

More like Immature mofos

[u/Ok-Computer-8726 → u/tingly_sack_69]

The so called Impossible Missions Force is not just a rogue organization, it is an outdated one. A throwback to an era without transparency and without oversight.

[u/biryani-half]

Did Anthropic hire the best engineers just to teach Claude how to imitate them?

[u/Individual_Ice_6825 → u/biryani-half]

I think it stems from Claude’s constitution approach.

[u/larowin]

I’m rereading Superintelligence and it’s wild to watch things play out in ways so similar and yet so different from what the ideas were barely 10 years ago

[u/HighSeasArchivist]

Funny when they conveniently don't include training costs in their financials. They'd be profitable if it weren't for all those darn costs. Whatever they are doing is unsustainable, and Wall Street is starting to question it finally.

[u/Kornerbrandon]

This stuff needs to be banned now.

[u/Soft_bich_z]

I love this for them, its like enron with code.

[u/tursija]

Oh, Anthropic! Is that the company that has an IPO in November? 🤔