Best books to learn everything about statistics like how different things are derived and everything? [Q]

[u/kangjimi]

[u/Jaded_Round5918]

man statistical rethinking by mcelreath is the closest thing to a bible for understanding the guts of what's actually happening

its not a traditional textbook but he walks you through building models from scratch and really forces you to think about the generative process behind the data

for the derivations and math side something like casella and berger statistical inference is the classic but its dense as hell. you have to be ready to sit there for an hour on two pages

i found that mixing a few different resources worked better than one perfect book. nobody explains everything in a way that clicks for everyone

[u/Mitazago]

In addition to books that are recommended, I'd also suggest you build time for broader reflection and thinking.

Sometimes, even years after you learned a concept, say when you are taking a jog and thinking through a statistical problem, clarity will hit you. You will realize a distinction you thought you had understood, or at least, had memorized, and the rational of it only now hits.

Taking time for unstructured more loose thinking, too is important.

[u/efrique]

learn everything about statistics

everything? You wont be learning everything. Thats not even possible, since new stuff comes along faster than anyone can keep up. You could learn some of whats covered in a stats major.

how different things are derived

You need probability (for which you need calculus) and some mathematical statistics; if you want to learn derivations for regression you'll want some linear algebra as well.

The book by Blitzstein and Hwang is fine for probability - you can get it in pdf, go here: https://stat110.hsites.harvard.edu/ and click "book". There's a bunch of other resources there. The video lectures are decent

What books you need next depends on what stuff you cover and at what depth/level. You might start with Siegrist's book here:

https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_(Siegrist)

(ch 6-9 covers some of the math stat basics, the rest of the book is probability and stochastic processes, which is good if you want a second book to go with Blitzstein) There are many good alternatives. You may want more depth after that, but it depends.

Thats roughly a couple of subjects worth, a decent start. Then you'll want other books depending on what stuff you want to know about, but I'd imagine something like: regression, simulation, resampling methods, GLMs, Statistical Learning, Bayesian stats, multivariate stats (including some basic theory and application of copulas), survival models, time series ... to get started.

[u/Able-Fennel-1228]

Learn multivariable calculus and matrix algebra well. Then work through a copy of mathematical statistics by Hogg and Craig (get solutions manual so you can check your work). Then work through Casella and Berger Statistical Inference + David Olive’s statistical theory and inference.
After that, if you have some basic proof based math (preferably real analysis) under your belt then you might wanna look into asymptotic statistics more (Large sample theory by Ferguson), and also see Boos and Stefanski’s book.
Also keep an eye out for Anirban Dasgupta’s upcoming book.

All this will take serious time and effort.

[u/Probstatguy → u/Able-Fennel-1228]

Thanks for all the great suggestions and I would have answered with some of your suggestions at the first! But for your answer, I wouldn't have known that Anirban Dasgupta is bringing out a new book. Looks great 👍

[u/JAMIEISSLEEPWOKEN]

Mathematical statistics?

[u/DigThatData]

what's your mathematical background? have you studied calculus or linear algebra? do you have prior experience with probability? set theory?

[u/PaulSandwich → u/DigThatData]

like how different things are derived and everything

I'm thinking it's a no, based on the phrasing in the title.
I think OP wants a cliffs notes. "How hard could it be?"

[u/seanv507 → u/PaulSandwich]

Op, I would suggest you start with 'how to lie with statistics'

To build intuition

Then assuming you can program, you could try 'think stats' (free online, with GitHub repo of Jupiter notebooks) which allows you to simulate different distributions and their effects

Other introductory books are

Freedman Pisani and Purves, Statistics

Which could be followed up by Statistical Models by Freedman

[u/lamhintai]

I would also like to ask for a recommended book list on multivariate methods. I have a MSc in Stat, so having studied briefly C&B I would appreciate books at or similar to that level.

[u/ThisFlamingo77]

Depends a bit on what you want to achieve. But in general from a pure mathematic approach the international monograph on mathematics are highly recommended. There is a monograph on statistics if I remember correctly. They are quite older ones but they do pure math from pre computer era. Which is imo a plus, later on you scale up in computer stats techniques.

Later on the list on this page could be quite handy too, but once again it depends on which you want to stat

https://www.cambridge.org/core/series/institute-of-mathematical-statistics-monographs/C624C85BFB98FD08F1334F462E49A412

In stats the most interesting question is more how to define the collection you want to stat. Because most of the time, just in that definition of the collection most of them do stats in disguise. ecause quite often, the biggest statistical assumptions are already hidden in that definition. (But that's more sample theory then stats)

[u/mangonada123]

Lectures on Probability Theory and Mathematical Statistics by Marco Taboga and Introduction to Mathematical Statistics by Robert Hogg. I like the first one as a reference guide for proofs. The author has a web version that contains some of the book material for free, but I prefer the book. I think the second one is a good intro book for a beginner.

[u/kangjimi]

Thank you for all the book recommendations but as I checked all of the books I couldn't find any topics on design of experiments.

[u/CreativeWeather2581 → u/kangjimi]

Because you didn’t ask for design of experiments.

[u/Old_Positive2231]

• For intuition and derivations without drowning in proofs: “Statistics” by Freedman, Pisani, Purves.
• For actually using stats in decisions under uncertainty: Douglas Hubbard – “How to Measure Anything” and Sam Savage – “The Flaw of Averages”.
• For distributions, Monte Carlo, real risk modelling: David Vose – “Risk Analysis” (this is the one most qualitative risk managers should read but never do).

[u/LucaJulien]

Andy Field’s stats books are great. I used them for my PhD in psychology for our required stats courses. They are funny too. He slips in jokes so you don’t get too bored. He’s very easy to follow and has a youtube channel. He is a stats professor but I will say it depends on what you’re wanting to learn. He had excellent books to enter your way into different stats programs.

Given that we mainly used SPSS in psych (this was 2015-17), it was perfect for me but he has a series of “discovering statistics using (insert software).” His discovering stats using SPSS and discovering stats using R are books I still look at.

The only stats author to ever make me laugh.

[u/txbytrxtta]

What is your calculus background? Derivation of statistical models, their probability- and cumulative-density function, likelihood functions (just to name a few) all use calculus fundamentals. Unless you’re looking to go into advanced probability, I can’t see you needing Real Analysis, but a strong calculus & reasoning foundation is necessary.

[u/ForeignAdvantage5198]

math statustics by hooge and craig