r/MachineLearning • u/Shizuka_Kuze Student • Sep 08 '26

News OpenAl Says It Has Cracked One of Math's “Millennium Problems” (Navier-Stokes) [N]

As reported by the New York Times:

https://www.nytimes.com/2026/09/08/science/openai-proof-millennium-problem.html?smid=nytcore-ios-share

OpenAI’s announcement:

https://openai.com/index/navier-stokes-solution/

710 Upvotes

92% Upvoted

283 comments sorted by

View all comments

Show parent comments

697

u/Even-Inevitable-7243 Sep 08 '26

"OpenAI says they would partially credit Tristan for the $1,000,000 discovery (even though Tristan did not solve the $1,000,000 problem) — but only if they remove Levent as an author, as he works for Anthropic."

This says it all. OpenAI knows their "proof" relied on Tristan's work via his conversations with their models.

227

u/SimiKusoni Sep 08 '26

Reading between the lines the model had been actively trained after the work had been added to Codex and OpenAI "cannot rule out that de-identified data derived from their usage of our products helped improve our models⁠."

Ignoring that it's pretty awful that they can't tell what the training corpus was for a specific model I think it's a fairly safe bet that the user's work was in that training data. The conversations described also portray them as being pretty desperate to downplay any human involvement in the work, even on their side and ignoring the question of provenance.

It seems like a pretty sad state of affairs either way that could have been trivially avoided by OpenAI keeping track of training data.

81

u/THE_FUZBALL Sep 08 '26

If they can’t tell what the corpus was and they are also training models based on user chats, doesn’t this open up the models to massive poisoning vulnerabilities? That’s like rudimentary security 101. It appears that either they would be vulnerable to such attacks, or they do have measures in place to control what is contained in corpus.

26

u/nonotan Sep 09 '26

You do realize pretty much all major LLMs are essentially trained on any and all data they can physically get their hands on? You can already poison the data by posting bullshit on here. It's possible that some sort of crude filter is in place to reject obviously malicious text, but as long as you keep it sounding loosely plausible and target something niche enough that it won't be overwhelmed by tons of inputs saying otherwise, it's going to work. They are vulnerable to such attacks, and will continue being vulnerable probably for as long as "LLMs trained on astronomical amounts of data" remains the approach of choice.

3

u/THE_FUZBALL Sep 09 '26

I do realize this for sure. I guess we all need to speak in some kind of silliness cipher. Will be interesting to see where it goes.

11

u/biscuitchan Sep 08 '26

but then it wouldn't be anonymized, no? a bigger problem

22

u/lauradorbee Sep 08 '26

It doesn’t de-anonymize anyone to be able to tell “were this user’s conversations in the set of data used to train this model?” - you know that it’s in there, not what part of it originated from what user. It would only be de-anonymizing if they had metadata attributing each message/conversation back to the user, i.e if they could answer “which user (in particular) did this piece of the training data originate from”.

28

u/SimiKusoni Sep 08 '26

Data being anonymised doesn't prevent it from being found after the fact. In this specific instance for example they know the user's working title for the paper, they could just search their dataset for it and find the training data. Or they could use any other uncommon strings from the work they put into codex.

Also as an aside de-identified data is not anonymised, it has an explicit meaning and they didn't just choose the unusual wording randomly. Genuinely anonymised data falls outside regulatory frameworks like GDPR or various state laws entirely, de-identified data does not because re-identification remains possible.

Given that OpenAI don't specify anywhere (in this blog post or other articles) exactly what they do to "de-identify" training data they may well have a hashed user ID sitting alongside the data to verify the source if they ever need it.

14

u/silva_p Sep 08 '26

It can be anonimized only one way, i.e. can't know who the data belongs to from the data alone but probably can see his chat history and confirm if it is there

3

u/TwistedBrother Sep 09 '26

It’s more that imagine there’s an idea that fits, like learning planes rise because of faster air on the top of a wing. A model doesn’t have to learn that more than a handful of times because it’s reinforced by all the other corroborating evidence.

Models don’t “average out” the things they learn, they built semantic structures topologically (as in shapes of ideas that work together). Some fragile ideas that aren’t (wait for it) load bearing might get forgotten way too quickly but some ideas that just settle all the others can persist through a lot of training.

If this idea was in a few conversations, and it fit (the forced Euler version) that might be the sort of thing that once exposed it settles many other uncertainties in the model such that any downstream model could draw upon that coherent ensemble of ideas.

24

u/usefulidiotsavant Sep 08 '26

The claim that a singe conversation (which, let's suppose, might have even contained the solution) in the training data latter allowed the model to reconstitute the proof is highly dubious.

A model the size of Astra trains on millions of curated conversations, billions of pages and trillions of tokens. A single conversation with the correct answer is essentially quantization noise and should have no measurable effect in any practical scenario where that same problem is involved.

On the other hand, OpenAI could be doing something much smarter that could affect the result, say, a RAG over similar conversations in the past, a self-evaluation of remarkable results that are marked or boosted etc.

If I had the smartest model in the world, as well as a database of the problems and approaches the smartest people in the world are playing with, it would be foolish not to connect the former with the latter and mine the dataset for low hanging fruits in scientific discovery. It's such an unfair advantage that the firm doing it will win in any area, forever.

35

u/faustianredditor Sep 08 '26

The claim that a singe conversation (which, let's suppose, might have even contained the solution) in the training data latter allowed the model to reconstitute the proof is highly dubious.

A model the size of Astra trains on millions of curated conversations, billions of pages and trillions of tokens. A single conversation with the correct answer is essentially quantization noise and should have no measurable effect in any practical scenario where that same problem is involved.

In theory I agree, but in practice I've had an experience that made me doubt theory here. I asked Sonnet 5 about my own work, and it was able to pull the title and core idea of a petty little publication of mine with almost-zero citations, from model weights. That's something I very much did not expect. Granted, it's different from pulling a proof idea from a single conversation, but I would've expected it'd need a few orders of magnitude more training data.

16

u/Stock-Self-4028 Sep 08 '26

The same goes for my (currently unpublished as a paper yet) but not very significant result which I've fed to some of the previous GPT-5 series models (I don't remember if it was GPT 5.3/5.4 series or a newer GPT-5.5).

Anyways what I have been working on was least-squares fitting of trigonometric least squares method through Levinson recursion and NuFFT-1. Nothing groundbreaking, but it's a new computational complexity floor (more precisely superfast Toellitz solvers give the complexity floor, but Levinson recursion happens to be the most practical silver here) for that exact problem. Also I have code using that algorithm in four different public GitHub repositories to be clear.

Either way - older GPT models failed to derive that despite being relatively straightforward, new ones give you the right solution with absolutely no issues just as you ask them for the lowest computational complexity for that exact problem.

And now I am aware that the models are getting smarter, but I also see that as a clear training set "contamination".

And now I am not sure if either my chat with ChatGPT somehow got into training dataset, or if that exact solution got scraped from one of the GitHub repos I have uploaded the algorithm to.

Either way I am practically certain, that that exact algorithm somehow got into OpenAI's training dataset and GPT 5.6 has remembered it flawlessly.

2

u/usefulidiotsavant Sep 09 '26

Frontier models overtrain to the tune of 1000 tokens per parameter. So, while it's not implausible for certain pieces of training to have an outsized effect (perhaps, deliberately so), its impossible that the average recall for random piece of training material to be this good. Even the best imaginable compression can't recover hundreds of tokens from a single scalar.

6

u/faustianredditor Sep 09 '26

Right, but a single conversation can easily have thousands of tokens, recurring conversations even more. So you could easily have an allocation of a few tokens to the researchers' conversations, or similarly a token or three for my paper. Theoretically, of course.

3

u/MathAddict95 Sep 09 '26 edited Sep 09 '26

I think you would be surprised as to how good these LLMs are at learning even from a single data point. A document containing say a proof of NS is not a random piece of training material, rather its a data point of a very narrow domain in the corpus (not many data points would be discussions involving NS). As an extreme example, if your data was a collection of documents filled with random words (essentially noise), except for one special document containing "The proof of NS is" followed by random words, its plausible that the neural net would memorize the random words for the special document, simply because its likely the only document starting with "The proof of NS is", and so conditioning on this prior isolates the document precisely.

I got curious and tested this: I asked ChatGPT to recall a somewhat obscure paper from 2006. It could not recall exactly the algorithm used in the paper, but it was able to recall an important idea in its analysis. I know for a fact that this idea was used only once in literature (since I work in the field), and its also a niche topic, so it really should be the case that this idea appears once in the training data (or at the very least, a single digit number of times).

In contrast, Gemini 3.5 completely failed at recalling the paper. The gap is probably because ChatGPT is on a larger pre-train than Gemini.

We don't really know exactly how these LLMs learn, so it is really hard to rule out whether the independent researcher's data affected the model's trajectory significantly, even if it was one data point.

1

u/usefulidiotsavant Sep 10 '26

I'm not disputing the effects you mention, just raise the fundamental Shannon informational limit against verbatim recall, there is no mathematical way that a single 16 bit parameter could compress hundreds of tokens and allow perfect recall in the average case, as some commenters claim.

In your particular case, was that a paper with zero citations, or did maybe some of the citers rephrase the main approach in their introduction? Could we perhaps imagine a rational path to that approach with the vectors of related research aligning towards it, so that the model makes a "happy hallucination" that happens to match the actual approach, without actually encoding it? maybe aided by a few parameters the training did nudge in the right direction based on that paper? Was COT used, allowing some rational recreation? This would also explain the inconsistencies between models. Could we imagine quality research papers from this field be boosted somewhat in the training, in a way random conversations with customers would not be?

So not disputing it could happen, just questioning the fundamental information limits in the average case.

18

u/THE_FUZBALL Sep 08 '26

They will win until the community gets wise to it and stops leaking secrets to their service if they want to avoid their research being leaked.

I don’t understand how OpenAI could think this is a good move. If they are stealing IP from chats then it will eventually become clear that using their service is akin to pasting your solution to a public forum and expecting it not to be stolen. In fact that’s almost a better idea because then you have some kind of paper trail to prove your authorship. The only explanation is they think their product is so good that people will have no better option, but this is very short term gain pre-IPO strategy.

2

u/En-tro-py Sep 10 '26

I don’t understand how OpenAI could think this is a good move.

AGI/ASI/RSI/$$$ - wherever your personal brand of sociopathic inspiration comes from!

4

u/Smallpaul Sep 08 '26 edited Sep 08 '26

OpenAI specifically denied the RAG accusation.

I also agree with the others that the recall of minor details for these models is remarkable and we can’t rule it out.

4

u/ScottBurson Sep 09 '26

I think you're mistaken. I've seen another case where a single short piece of text in the training data popped out under the correct conditions.

It's true that a single update doesn't change any weight very much — but it can change many of them slightly.

1

u/amebix1221 Sep 10 '26

How so? Models have been shown to reproduce training data verbatim. And did OpenAI rule out the possibility that the data could have been available to the model in other ways?

1

u/redditter208475093 Sep 12 '26

Why? If the conversation is very niche and that's the only data relevant to the input, it will use that data. LLM are just very advanced autocomplete and it's not clear how much entropy is lost between the training data and the model. Frontier models are trillions of parameters.

117

u/purplebrown_updown Sep 08 '26 edited Sep 08 '26

Exactly. Not to mention OpenAI responded with threats like "Why would you ruin your career". Nobody who knows they're on the up and up would do that. What I believe actually happened is that OpenAI was aware of Buckmaster's progress and direction, got backdoor access to the conversations to get a preprint, and then used that information to claim they solved the problem on their own. It's the equivalent of stealing someone's manuscript and publishing before them and claiming credit or shared credit.

6

u/ImNotAWhaleBiologist Sep 08 '26

It’s just as impressive, if not more so, that someone used their model for such a solution and wasn’t affiliated. Seems appropriate to include all as authors, and can credit contributions accordingly.

12

u/PM_ME_YOUR_PROFANITY Sep 08 '26

They may not have used it to actually figure out the solution, but for something as simple as text formatting.

-15

u/Wise-End307 Sep 08 '26

See sebastiens response.

29

u/Even-Inevitable-7243 Sep 08 '26

I did. He admitted that OpenAI offered to make Tristan the lead author of a re-write of OpenAI's "proof", essentially giving him credit for work that they claim he had nothing to do with, since OpenAI's model came up with the NS proof with what OpenAI claims was "little human involvement". You do not pay-off a guy, monetarily or with academic credit, when you are confident that your AI did something with little human involvement, and you certainly would not credit the person who had already come up with a proof for the same problem.

This is like a pop star claiming a songwriter had nothing to do with a hit song, then offering to pay the songwriter royalties and giving him lead songwriter credit.

2

u/Smallpaul Sep 08 '26

Songwriters get royalties. Forever. The stakes are much higher than putting some dude’s name on your AI generated paper to avoid having the precise conversation we are having right now. Corporations “settle” in lawsuits that they do not believe they are at fault in all of the time.

What matters to the OpenAI marketing department is not what they believe but what they think the market will believe.

2

u/G_fucking_G Sep 10 '26

Update from NYTimes:

The Wednesday evening statement from OpenAI was more emphatic: “We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.” The statement added, “After investigating, we can say with full confidence that no user inputs past July 3rd could have influenced this system in any way.”

And also from Tristan Buckmaster that most of their work was also produced by AI models:

In their collaboration, Dr. Buckmaster set up the problem, and then Dr. Alpöge fed that into an A.I. model at Anthropic. Based on what came out, they provided further input and repeated the steps. The process worked well, but for Dr. Buckmaster, it became discouraging. “On one side it’s, like, super exciting to be part of this,” he said. “On the other side, this is frightening, because I felt like the A.I. had overtaken me.” Was he now an interpreter of A.I. slop and no longer the discoverer of new mathematical insights? “What’s the human part of it?” he said.

2

u/aeroumbria Sep 09 '26

A bot doesn't need credit, recognition or profit. Anything more and it is not really working for me.

1

u/florinandrei Sep 09 '26

Like leadership, like company.

Slimy as fuck.

-3

u/Smallpaul Sep 08 '26

Their reaction would be the same whether they know this, suspect it or cannot disprove it. Having their big marketing moment overshadowed by this controversy was worth far more than $1M and they aren’t even claiming the $1M anyhow.

It is only rational for them to try to get Tristan on side regardless of whether his claims have merit. Especially if they cannot prove they do not.

-3

u/willncsu34 Sep 09 '26

To be fair pretty much every math proof relies on someone else’s work. When I published a paper it was built off of previous works and cited them. Citing something that you don’t totally know what it pulled from is going to be tricky here.