r/statistics • u/GayTwink-69 • 16d ago
Research Can AI do mathematical statistics now? [R]
Say you are developing some sort of statistical test and you have run simulations and applications, and now you wanna work on its theoretical guarantees (asymptotics, efficiency, consistency, etc.)
Can you not just feed in the test statistics, assumptions, etc. to your favorite intelligent AI and get it to derive everything?
In this instance, is the focus of statistical theory going to be more on the algorithms/computation side and dealing with issues like scalability going forward, while leaving the math to AI?
Didn't AI solve that big unsolved problem in pure mathematics the other day (Navier-stokes or something, I think)
9
u/Fancy-Animal7704 16d ago
I mean the main issue here is making sure AI knows what question you are trying to answer and knows how to communicate those results to an audience. AI is an answer-generating machine and thus can only answer what you tell it to.
You may be overlooking the fact that the computations are really the easy part of statistics. Yes, the actual computations take time, and the formulas are sometimes quite involved, but ultimately it is just busywork to add / subtract / multiply / divide. We learned how to do those things in elementary school. The real challenge of statistics is to properly collect the data you want, select the data that answers your question, and then be able to effectively communicate what we have actually learned from those results.
I don't entirely trust AI with that responsibility. How can AI know what is good for humanity? How can it know what questions are even important to us and thus know what questions to answer with that data?
3
u/jadexiaohui 16d ago
100% agreeing with this as a biostatistician… Most of the time I’m not stressing about the code, but more about how to handle different edge cases (e.g unequal/low sample sizes, outliers, pseudoreplication). I could spend days stressing about how to handle those even after consulting 3 AI agents
1
u/Kepler___ 16d ago
It is uniquely bad at certain stats concepts though without the right tools, I've had very little luck when trying to feed my paid model larger stochastic calculous problems unless the prompt is so curated that it would have been equally as fast just writing it in R.
1
u/Chemical-Whereas-750 15d ago
Even if the AI could spit out rigorous asymptotics, half the time the real work is figuring out the right assumptions to even feed it in the first place. The gap between "ran some simulations and it works" and "here's a theorem" isn't just algebraic grinding, it's knowing which lemmas to lean on and which dependencies matter. That judgment takes years to develop, it's not just pattern matching on the text of the problem.
1
u/No-Argument9627 12d ago
What recommendations would you have for a new grad in stats looking for work in this new age of AI? Do you think companies are starting to value people who can use AI for the raw technical work while focusing more of their effort on asking the right questions and interpreting the results?
1
u/Fancy-Animal7704 12d ago
If I were you, yes, I would very much emphasize your value as a question-generating machine and how THAT is where your value lies. Your advantage over AI is that you know how to run any analysis just like AI would know, but you have the added benefit of understanding which analyses would be useful in the first place and how those analyses can help answer whatever questions your employer may be wanting to ask.
1
u/No-Argument9627 12d ago
Thank you for your response. Any recommendations on how to do this on a resume given my experience is non-stats related? (helping run operations at a small business, and a legal intern-As I was considering lawschool at one point)
1
u/Fancy-Animal7704 12d ago
Well your resume isn't a place where you can do this and isn't really for that. Your resume is simply where you highlight your education, skills, and experience.
1
7
u/Distance_Runner 16d ago edited 16d ago
Yes it can be useful, but with oversight and heavy direction.
I've been using it for exactly this type of thing. The answer is nuanced. The best way to think about AI in its current form is that of an advanced calculator that can manipulate abstract symbols algebraically, with an encyclopedic knowledge of every obscure theorem, principle, conjecture, etc. in existence.
Look, math follows a set of logical rules. AI is very good at applying mathematical principles and deriving. But it still needs guidance, because it doesn't do a great job of knowing what to derive without you telling it. It rarely makes mathematical mistakes per say, but it often goes off track and starts deriving or proving things that are irrelevant to what your question or goal was. It often smuggles in assumptions that aren't reasoanble, so what it derives is correct under those assumptions, but those assumptions render the results useless. I've often noticed with statistical/probability based derivations, one thing that Claude Fable and GPT Astra have issues with occasionally, is carrying proper conditioning through its derivations. If you're conditioning an estimate on something, you need to walk through its derivations and proofs very carefully. So all of things point to this -- Yes it can be used for theoretical work, but it should be thought of as a highly advanced calculator, and the user should be able to follow and understand the derivations/proofs, explain what's happening, and why.
And the last thing I'll say, is while the top models are damn good at actual derivations, their interpretations of what all of it means, the contextual implications, and explanations in plain terms can still be pretty rough. It often just restates in words what it derived in math terms, and doesn't do a great job turning it into an actual explanation you could write in a paper.
context: I'm a PhD Biostatistician and Associate Professor of Biostats and Data Science. AI has enabled me to pursue my methodological research with more freedom and at a far faster rate than ever before. I'm highly collaborative. Most of my funding comes from grants where I am overseeing studys and practicing applied statistics. AI has enabled me to puruse methods and theory, because its effecitvely like a highly efficient post-doc I can direct and check the work of, that works at a super-human pace. I can check theoretical ideas in the matter of minutes to hours, instead of days and weeks. But I use AI interactively. I probe. I check its work. I don't accept what it gives me blindly. I also won't pretend that the results and work I've achieved with AI in the last year I would have produced alone. Some of the connections it can make because it has access to pretty much all math ever on the internet is really amazing. The important thing, imo, is that if youre a user of it in this way, you need to be able to understand every step. When it pulls something you wouldnt of thought of, your reaction should be, "thats brilliant and makes sense!", not "WTF does that mean, oh well, using it anyway".
2
u/involuntarheely 16d ago
you can also have the ai develop the algorithms and work on the computation side
as it stands today, ai can automate many parts of doing this kind of work but, again as of today, it still needs you to have a goal, an idea, an intention, otherwise it will give you a sort of boilerplate “good enough” answer that’s not very innovative. the analog to this is when it writes obviously ai-generated text, obviously ai-generated pictures, websites, and so on. it’s not like the ai can’t make good text, it’s just that you’re not giving it clear enough instructions.
1
u/GayTwink-69 16d ago
So prompt engineering will be the most important skill to learn?
1
u/involuntarheely 16d ago
i don’t know what the most important skill to know in the future is. certainly now in order to have good ideas you need to know your stuff. you need to know how to verify. prompting is more about details and how much you care the ultimate product not being obviously ai-generated
2
u/Able-Fennel-1228 16d ago
That’s a very good question GayTwink-69. AI is pretty darn good, and it’s reasonable to expect it to only get better now. The “getting better” part is something I don’t see being addressed in these answers.
I can certainly see mathematically adept users being MUCH more productive with it, with less mathematically adept users being less adept but still more productive than before (so the gap between the knows theory guy and the doesn’t know theory guy, remains, and maybe even widens, and maybe continues to widen in the future). One can imagine how publication requirements go up now that everyone is coming up with MANY more papers than before. Perhaps, eventually, only the extremely talented few that are also good at utilising AI will remain.
Another issue is being able to check for correctness. AI can help come up with a solution (and even help figure out a question to ask), but as of today, you still need to be able to check for correctness (AI still bullshits). But what if there comes a time where you don’t need to check? What if it tends to be so “correct” in a particular area of math like analysis and mathematical stat, that we are satisfied with the chance of error being very low, even if that doesn’t qualify as a “proof”. Idk what then…
As of right now. You still need to know mathstat. No way around it. But AI DRASTICALLY helps speed up your own learning, question generation, idea generation and solution generation. Id be more concerned about how this bump in pubs per person will affect the pub requirements now…. You certainly won’t be able to keep up with the already math stat talented guy that jerks AI super hard all the time. Insane transitionary period we are living through….
1
u/Valiant4Truth 16d ago
I mean people are developing MCP agents to do this exact workflow. It'll be here quicker than you think for simple inference. I don't know much about the Navier-Stokes solution but I did see that models generated a simple counter example for the Jacobian conjecture, which is pretty crazy.
1
u/efrique 15d ago
Didn't AI solve that big unsolved problem in pure mathematics the other day (Navier-stokes or something
Not really. Its a pretty sordid story, another tale of theft and attempt to avoid giving credit
If you want to credit AI with something there are better choices, but theres still plenty of theft, albeit less blatant.
Can you not just feed in the test statistics, assumptions, etc. to your favorite intelligent AI and get it to derive everything?
Okay, go ahead and do it.
1
u/eeaxoe 15d ago
Depending on who you ask, maybe. One of my friends over at Stanford had his student put this together: https://causalsmith.org/
Apparently the trick is just to verify everything in Lean.
1
22
u/Initial-Ad6631 16d ago
lol I just knew it was a GayTwink-69 post