r/antiai • u/Locke357 • 10h ago
Preventing the Singularity LLMs Literally *ARE* Just Spicy Autofill
93
u/Objectionne 10h ago
It's technically true on a foundational level (if you ignore all of the supporting software that enhances their actual capabilities), but for me it's kind of like saying "a computer is literally just a fancy calculator!" Technically true but drastically understating the functional difference between a computer and a calculator.
38
u/Locke357 10h ago
Except people aren't treating a computer as a god just because it's more advanced than a calculator
That's the point I think is trying to be made here. It's just a collection of linear algebra, not artificial intelligence, not super intelligence, not artificial general intelligence, not the machine god, not some magic bullet to solve all of our problems.
18
u/LakeSubstantial3021 9h ago
Idunno about calling it super or general but it’s certainly artificial intelligence by any metric invented before gpt was released.
4
u/Edward_Tank 7h ago
Artificial intelligence assumes human level intellect and understanding of concepts as opposed to just statistical data saying which word should come next.
It's certainly artificial but it sure the fuck ain't intelligent.
10
u/Emuu2012 7h ago
No it doesn’t. You just made up your own definition and then argued that it didn’t fit.
→ More replies (17)4
u/LakeSubstantial3021 7h ago
This is the “stochastic parrot” description of LLMs and it’s not the description that researchers use to describe LLMs behavior.
LLMs clearly have quite strong grasps of “concepts”, and have “reasoning” capabilities.
“next word prediction” is just the mechanism by which we train these traits into them.
→ More replies (23)2
u/-MtnsAreCalling- 7h ago
That has never been what artificial intelligence means.
1
u/Edward_Tank 7h ago
Sure thing man, I remember when AI meant something that could think and reason and understand concepts. Remember Data from Star Trek: The Next Generation?
Or are you going to say that Data was nothing more than an LLM?
3
u/Beautiful_You3230 6h ago
Bro, what? Are you seriously bringing up sci-fi movies??
→ More replies (1)→ More replies (2)3
u/BigLittleDeal 5h ago
In real life, AI has more often been used to refer to the computer opponent in a video game. An AI that can do most everything a human can do would be an artificial general intelligence.
→ More replies (3)2
u/BigCelebration13 5h ago
No they redefined it to help market the spicy autocorrect and now every tedious nerd will argue until the heat death of the universe that a statistical model is a real boy.
2
u/88sSSSs88 10h ago
The human brain is just a collection of cells passing information to each other.
10
u/NoOneMan79 9h ago
Rarely. Do you really see much information being passed around by those lofty cells taking a gander at the incredulous garbage posted on reddit?
6
u/velvetabyss 9h ago
The human brain is just a collection of cells passing information to each other.
And AI isn't.
Reductionism is so silly. Everything is just atoms right? So AI and the brain are the same thing. Just atoms interacting with each other. A glass of water is also just a bunch of atoms interacting with each other. So a glass of water is just as intelligent as a human brain.
→ More replies (17)3
u/ExtremeAcceptable289 9h ago
Cells = living, math = non living
11
u/Punell92 9h ago
That’s not a good argument, cells are living by definition. We created the definition of living specifically so that cells would be in that definition, so pointing it out as though that makes cells special is wrong.
→ More replies (12)3
u/Ekktz 8h ago
Our brain runs on electricity, so does a computer.
2
u/ExtremeAcceptable289 8h ago
A toaster, a lightning bolt, an electric eel, and an iPhone are the same because they all run on electricity.
→ More replies (5)2
→ More replies (15)1
u/GrokiniGPT 6h ago
idk man, what makes up those cells ain't living. the concept of emergence is pretty weird
5
u/DayDramatic5236 9h ago
Yet, I can remember the context of what's happening around me past a few minutes. When I wake up tomorrow, my memory won't be completely gone. My creativity doesn't require being trained on everyone else's creativity first.
Guess there must be something else going on up there in the ol noggin
2
u/Practical-Rub-1190 8h ago
Don't we pick up on others' creative styles and create something new? For example, if you were a comedian and nobody had ever done a parody. Nobody had seen it before; it did not exist. Nobody would be doing it, right, till somebody suddenly did, and now everybody would be doing it. Now, there would be all kinds of styles of parody in art, movies, music, etc.
→ More replies (3)→ More replies (4)1
u/RegorHK 8h ago
You are using the same language and same figures of speech as most of the humans in your cultural sphere. You have literally been trained on the creativity of others. We call that process learning a language and receiving an education.
You did not come to magically use the same language as we by your own creativity.
1
u/DayDramatic5236 8h ago
It's not AGI dude.
It's a really cool predictive algorithm.
It picks a random number based on the exact number of times that number was picked by other people. It can't even make this simple of a choice on its own.
3
u/Devils_SteelMan 9h ago
Can you describe (mathematically or otherwise) the process by which humans or llms go from understanding individual words to whole sentences?
2
u/evocativename 9h ago
LLMs don't understand. They predict based on statistical frequency patterns in their training data.
4
u/serinty 8h ago
how do you know we aren't the same. And what is understanding if not just making a inference based on prior experience
2
u/evocativename 8h ago
Because LLMs are a vastly simplified model of one part of how actual brains work, not a direct representation of the function of brains.
And because we don't have to sit here taking in instructions before we think or act, but can go out and make self-directed decisions based on our internal models of the responses of other people with different knowledge from ourselves, observe the world (imperfectly), parse that data, and determine how and if to update our understanding of the world as a result.
→ More replies (12)1
u/Devils_SteelMan 4h ago
This is not remotely true. Not only do you have to not understand text embedding and attention to say this, there's a mountain of research that strongly disproves this. Future lens, j lens, etc.
2
u/bystander993 9h ago
Who cares if it is "just" linear algebra? You cannot predict what it is going to output and it outputs semantically meaningful responses that the very least very clearly mimic human thought. The fact is we don't know why the math results in this, we are learning about it ourselves. "It's just algebra" is woefully ignorant of what it actually does and how unexpected it is.
1
u/DrinkMoreGlorp 7h ago edited 7h ago
Of course you can predict what it's going to output. That's kinda the core principle of the entire scheme. It's called forward propagation. You can even break it down statistically what it's most likely to output for any given token. Manipulating that is how we get LLM watermarking.
It's not even reductive to say that it's just linear algebra. It's linear algebra used to predict the next most likely token. You can do nifty things with that. That doesn't make it transcend what it is. Its fundamentals bubble up through the layers of abstraction constantly, it remains its most common and fundamental failure modes.
→ More replies (10)1
u/AnotherArryn 7h ago
It's also woefully underestating what basic algebra actually gets us... It's a language that describes the majority of natural functionality as humans observe it. Most of physics is just that. And life is just organized physics. Our brains are just zillions of interactive calculations made up in exactly the same basic mesh form of an LLM neural net.
The scary part for most people (I think) is not how smart the LLM is, it's how it openly highlights how dumb we are.
1
u/Juicemoose222 9h ago
Do you often feel you are very smart but struggle with day to day cognitive tasks? Honestly curious
1
u/Grouchy-Ad-8044 7h ago
The argument is still dumb. You could just as well say "Human brains are just like if you squished a million fruitfly brains together, and fruitflies are not intelligent therefore humans are also not intelligent". We also know how fruitfly brains work well enough that we have fully mapped them and can simulate them on computers. You cannot decide if something is intelligent or not by looking at what the smallest functional parts of them are like in isolation (neuron activation, matrix multiplication), say "that looks simple and easy to understand", and therefore conclude that the whole thing is also just as dumb and simple as a component. The overall wiring of all those parts IS the thing that makes what it does interesting
1
u/pixelizedgaming 5h ago
i mean no one is treating ai as a god either they just overrely on it for everything. however the same can be said for every technological advancement ever, people who pull out their iphone calculator for the times tables have existed way before ai.
1
→ More replies (40)1
u/maximhar 2h ago
“It’s just a collection of linear algebra, not artificial intelligence” - the latter doesn’t follow from the former, unless you can’t prove why an artificially intelligent system can’t possibly use linear algebra. Other analogies:
“It’s just a collection of transistors, not a computer”
“It’s just a collection of neurons, not a human brain”
5
u/Edward_Tank 7h ago
Literally everything a computer does is just based around the concept of math. Yes, it is just a fancy calculator that we trick via the power of numbers to do things for us.
3
u/IvanMalison 7h ago
Literally everything that happens in the physical world can be represented and computed with math. We literally do not know of any physical things that do not have some mathematical representation. To what extent does your argument about computers not apply to the entire world itself.
1
u/Edward_Tank 7h ago
To what extent does it matter when your point is flawed that computers are in fact just fancy calculators?
1
u/IvanMalison 6h ago
i think we're agreeing.
1
u/Edward_Tank 6h ago
Ah, my apologies, I confused you for Objectionne, the person I was replying to.
Also happy Cake Day.
1
43
u/LethalFury 10h ago
wait till she learns how a single human neuron works
15
u/bojackhorsemeat 10h ago
Good thing we're more than neurons.
11
u/SirBananaPerson 9h ago
Wait until you learn that those things can also be described mathematically
9
u/velvetabyss 8h ago
A description isn't the thing being described.
You can't eat a mathematical description of a steak.
→ More replies (11)3
u/Slippywasmurdered 5h ago
That’s not remotely true.
1
u/SirBananaPerson 5h ago
Explain please
7
u/Slippywasmurdered 5h ago
We are no where near being able to describe anything about how human consciousness works or how the brain works through a mathematical model. Let alone how the rest of the human body influences our brain.
2
u/SirBananaPerson 3h ago
You are talking nonsense
We ve been mathematically modeling brain activity for decades. The Hodgkin huxley model, uses differential equations to describe how neurons generate electrical signals. We have mathematical models of synaptic plasticity, neural population dynamics, sensory processing, and decision-making.
And yes, the rest of the body influences the brain through hormones, the nervous system, and other physiological processes. Those interactions can also be studied and modeled mathematically.Not having a complete mathematical model of a system doesn’t mean we have no mathematical understanding of its components
1
u/Mikeisright 4h ago
Okay, give us the mathematical equation for human consciousness.
2
u/SirBananaPerson 3h ago
Y=ax+b.
On a serious note: we dont have a mathematical equation for consciousness, but why would that mean the processes occurring in our brains can’t be described mathematically?
Neurons operate through electrical and chemical processes governed by physical laws. We can already model neuronal activity using differential equations, interactions between neurons using linear algebra, and learning through mathematical models of synaptic plasticity.
The fact that we cant yet mathematically describe how billions of neurons collectively produce consciousness doesnt mean those processes are somehow beyond mathematics.1
9h ago
[deleted]
2
u/SallyStranger 9h ago
Most of us have a body. You don't?
1
2
u/Super_broad 9h ago
Disingenuous to ask someone to explain it for you. Go ahead and Tetsuo yourself if you feel humanity and consciousness are mundane.
1
1
→ More replies (6)•
u/frenlytransgurl 7m ago
But we are all atoms. Mostly the simplest one, hydrogen. Humans aren't really conscious, we're just spicy atoms.
6
u/NicoN_1983 9h ago
It's pretty complicated actually
1
u/LethalFury 9h ago
well yes going down into actual compounds and STP and LTP and such but more so the actual computations they do are very simple
→ More replies (3)1
1
19
u/Scarvexx 10h ago
Well it's a bit more complicated than that.
But yes, some people think it's god. That's just how some people are. Every generation has a religion built around something new and fancy.
7
u/Locke357 10h ago
Not trying to dispute, I'm just curious what "new and fancy" things previous generations worshipped in a similar way to AI.
12
u/Scarvexx 10h ago
Sure. Christian Science was preoccuplied with mesmerism and phrenology.
Theosophy was based on the racial psudosciance of the day.
Spiritualism was based on the telegraph. The believed glavanic electricity was spiritual in nature.
Scientology was based of of Psychoanalysis.
The Mormon faith has roots in anthropology, archeology, and early astronomy.
Heaven's Gate and ufology and the internet. They believed that humans minds were like software, and could be upgraded and transferred out of their hardware. Those guys aren't around anymore, and they're the reason people say "drinking the Koolaid".
As sciance advances it changes the way we see the world. But humans look for magic wherever they can. If there's a mountain too tall to climb, the gods must live there. If lightning falls from the sky, Zeus threw it. If we train AI on the bible, perhaps it will embody Jesus christ?
That sort of thing.
4
u/bromanceintexas 9h ago
Drinking the kool aid comes from Jonestown in 78, and it was actually flavor aid. Heaven’s Gate consumed a cocktail of barbiturates, apple sauce, and vodka, and plastic bags for good measure.
2
1
u/LeastBlackberry1 4h ago
And, if they wouldn't drink it, they had it injected into them. Which is perhaps an even more appropriate metaphor for how AI is pushed on all of us now.
→ More replies (6)1
u/Practical-Rub-1190 8h ago
Electricity was huge; suddenly, you could work at night, just think about hospitals where they did not need to use candle lamps when doing operations. Suddenly, you had frozen food for storage, which was massive.
Cars are still being worshipped.
The internet was kinda big. Remember the bubble.A ton of new and fancy things have been worshipped and are still being worshipped today. Yeah, the iPhone? Bitcoin?
18
u/Ok_Community_383 10h ago
omg computers are just 1s and 0s!
11
u/Eastern-Ad-3586 9h ago
I see what you mean, but the point is that AI in its current form (or at least LLMs) are non-deterministic algorithms trained on large amounts of data to put the optimal answer most of the time.
It’s an algorithm. It’s a very powerful algorithm that has lots of use cases, but it can’t feel pain, or be creative, or have human rights. It’s an algorithm.
A technology can be incredibly powerful and useful without being conscious, and the people claiming Claude has a soul are just irresponsible
8
u/NotReallyJohnDoe 9h ago
It has no qualia. No internal experience of the world. It can’t be conscious in the way we mean the word.
Put another way, it gets input tokens and outputs tokens. Where would the consciousness be?
4
u/Taraxian 9h ago
There's no proof that human beings have that either (other than me of course)
3
u/protomenace 8h ago
You got downvoted and I have no idea why. This is epistemological solipsism which is pretty widely accepted.
1
u/Taraxian 7h ago
I wonder what the response would be if I said I know it's possible for humans to be P-zombies because I am one
1
u/Ok_Community_383 4h ago
A lot of people haven’t taken philosophy 101 and are going to not like what you just said
1
u/protomenace 8h ago
Human brains get sensory input and output nerve impulses. Where would the consciousness be?
1
u/MakeDawn 2h ago
Getting them to differentiate brain states from qualia is its own field of philosophy.
1
u/Ok_Community_383 4h ago
To be fair we don’t know that. But it is most definitely the most likely case that it doesn’t.
→ More replies (16)2
2
19
u/88sSSSs88 10h ago
If you can understand calculus and statistics at a college level, you can understand the basics of machine learning. What you won't like is that there's a good chance that much of the way human brains work can be approximated just fine with linear algebra.
9
u/Fearless_Ad7780 9h ago
Converting qualitative data into numbers so a model can process it isn’t proof the data was numbers all along. That’s like saying music is basically a spreadsheet because you can store it as an MP3.
8
u/88sSSSs88 9h ago
You missed the point. I'm saying a sufficiently comprehensive system of linear algebra could probably model most of the human brain's capacity for intelligence.
7
u/Fearless_Ad7780 8h ago
No, I got your point. I just think it’s b.s. and not true.
First, it can’t be linear algebra. Stack all the linear layers you want and you still get one linear function. Neural nets only work because of the nonlinear parts, so linear algebra can’t even model ChatGPT, never mind a brain.
Second, there’s no evidence for it. Nobody knows how the brain produces intelligence. It took a 5 to 8 layer deep network just to mimic one neuron, and we have about 86 billion of them. So far only some aspects of the brain can be approximated linearly, not intelligence as a whole. Wilder Penfield, who pioneered epilepsy surgery, stimulated the brains of around a thousand awake patients. He could make them move, feel tingling, or relive a memory, but he never once triggered abstract thought or their will. He started out a materialist and ended his career convinced the mind was more than the brain.
Third, the data humans take in isn’t numbers. We get words grounded in things. Harnad called this the symbol grounding problem. The model has the symbols with no access to the referent. Fire, hot, singe, smoke, and burn all sit close to each other in a vectorized space, but that never tells you what they mean. Just that they relate to each other and get used in a frequent context. You know what burn means because you’ve been burned. Braitenberg argued the brain wires a word to the experiences you learned it through, and Pulvermuller’s brain scans backed it up. Your “hammer” reaches into the parts of your brain that have held one, and nobody else’s is wired the same. The model only knows the word shows up near nail. That’s the linear algebra part. It can tell you which words hang out together. Braitenberg and Pulvermuller got their results from real brains in real people, and what they found points to every person’s wiring being built from their own history. So there’s no one model of “the human brain.” There’d have to be one for every person. So where’s the intelligence in your vector space?
And “sufficiently comprehensive” just means as big as it needs to be. You can’t ever be wrong if you get to keep moving the goalposts. Even Sapolsky, who’s about as hardcore a determinist as you’ll find, doesn’t get you there. He traces every action back through hormones, childhood, the womb, genes, and culture, all specific to one person. And in Determined he spends chapters on chaos, where everything is caused but tiny differences blow up so fast you can’t predict them even in principle. Determined doesn’t mean modelable. Your argument needs them to be the same thing, and they aren’t.
6
u/88sSSSs88 7h ago edited 5h ago
Since you insist on being pedantic, then I suppose I need to explain that it's not about moving the goalpost and instead about language shorthands for a subreddit that is not mathematically literate.
Here's my exact position:
1) Any Turing-computable process can be simulated by a recurrent network of linear maps with a fixed nonlinear activation. If the process is discrete, then there is strict parity. If the process is continuous, then there is arbitrarily close parity.
2) There is presently absolutely no reason to believe that intelligence cannot be expressed by a Turing complete language.
3) Therefore, there is presently absolutely no reason to believe that a recurrent network of linear maps with a fixed nonlinear activation cannot reach parity with intelligence.
Are you starting to see why using the term "linear algebra" as shorthand for "sufficiently comprehensive system of linear algebra" as shorthand for "a recurrent network of linear maps with a fixed nonlinear activation" in a subreddit that does not know what half those terms mean is sufficient? Because it's interesting that you're not condemning the original post image for bunching LLMs, which use nonlinear mappings, with linear algebra but for some reason decided to condemn me for it.
With that aside, are we going to get into why I believe my second premise or are we going to agree to disagree on that?
1
1
u/Fearless_Ad7780 5h ago
I’m not picking on you. You made a pretty reductive statement about human intelligence and I thought it would make for a good debate. I’m not going to go after OP for a meme when you’re the one who actually made the argument.
The shorthand matters. When you say linear algebra you leave out the nonlinear part, and without it the network can’t do anything you’re claiming, so you ended up saying something different from what you meant. If you think this sub doesn’t know what half those terms mean then you should be even more careful to say what you mean.
I’ll give you premise 1 since it’s a real result. The proof assumes perfectly precise numbers and unlimited time though, so it’s possible in principle and not something anyone can actually build, and arbitrarily close is still an approximation.
Look at what premise 1 actually covers. A Turing machine works on the shape of symbols and never knows what any of them mean, and the same goes for your network. With text the meaning has to be bootstrapped from outside. People wrote the training data, people rate the answers, and people read the output. Without us it has no way to tell a right interpretation from a wrong one because to it they’re both valid strings. Searle’s point about syntax not being semantics still holds.
Premise 2 is where your argument falls apart. Saying there’s no reason to believe intelligence can’t be computed doesn’t give anyone a reason to believe it can. You put the burden on me, and I already gave you evidence. Harnad’s symbol grounding problem shows the system has the symbols with no access to the referent. Braitenberg argued the brain wires a word to the experiences you learned it through, and Pulvermüller’s brain scans backed that up. Penfield stimulated around a thousand awake brains and never once triggered abstract thought or will. You didn’t address any of it, you just skipped over it and called me pedantic.
Words also don’t work the way you need them to. Wittgenstein said meaning is use inside a form of life, so words mean something because of what people do with them in a shared world. A model has the record of how words were used without the life they were used in. Searle took this further with speech acts. When I say “I promise” I’m doing something rather than describing something, and it only works because I mean it and we both know the rules. He called it X counts as Y in context C. Paper counts as money and a sentence counts as a promise because people with intentions treat them that way. A model can produce the X but it can’t make it count as Y, because it has no intentions and it isn’t part of the C. Its intentionality is borrowed from us while ours is intrinsic.
Even if you got parity in output, describing something doesn’t make it the thing. Calling water H2O doesn’t come close to swimming in it or drinking it when you’re thirsty. Taking a description of a process and treating it like the thing itself is reification. A computation that matches what intelligence does is no more intelligence than a recipe is a meal.
Your first comment said there’s a good chance much of the brain can be approximated with linear algebra, and now you’re saying there’s no reason to believe it can’t. That’s a big step back from where you started. I’m happy to get into premise 2, but that’s where you leave math. Premise 1 is computer science and premise 2 is a claim about minds, which puts it in neuroscience and philosophy of mind. Ballantyne calls confidently making claims outside your field epistemic trespassing.
1
u/evilteddy 5h ago
I mean, in the limit you're wrong because the isolated quantum state vector representing the brain can only take unitary evolutions.
→ More replies (3)→ More replies (2)3
u/Devils_SteelMan 9h ago
Math is a description of things. Its not what the thing necessarily is. It is the structured aspects of the thing described in notation.
10
u/JarPicklez 9h ago
Is it linear algebra? Yes. But when I took linear algebra in college I didn’t expect the linear algebra to become better at linear algebra than I am
Saying it’s “just linear algebra” is like describing how a car works by saying it uses molecules.
2
u/dualmindblade 9h ago
It's actually just mostly linear algebra, the activation function is non-linear, and without that the network couldn't learn certain things, basically it wouldn't work at all
3
2
u/Money-Armadillo-733 6h ago
the activation function is non-linear, and without that the network couldn't learn certain things, basically it wouldn't work at all
Unless you use GradIEEEnt decent.
Now I gotta watch the tom7 video again.
It's a great video about abusing floating point operation to get the nonlinearty.its great fun.
1
u/dualmindblade 6h ago
That's hilarious actually. Does it offer some kind of computational advantage or just a peculiarity?
1
u/Money-Armadillo-733 5h ago
No it's just fun messing around. But extremely high effort messing around. I highly recommend the video and his other ones.
1
u/JarPicklez 9h ago
Yes, but the activation function is boring and preprogrammed. It’s the bone of ai behavior. The meat and the thing we need to learn how to control is the gigantic matrix multiplications
2
u/dualmindblade 9h ago
Eh, it's true that the bulk of the calc is linear, but I wouldn't say it's boring. There're a whole bunch of different activation functions and choosing the right one is important, this isn't even something that was known in early days. It used to be all sigmoid, now we have a variety of them and different ones for different layers. Also, we do things like softmax, which takes into account more than one neuron's value. It's super interesting actually
1
7
u/Cybyss 9h ago
You're right.
LLMs really are doing nothing more than "Just Predicting Tokens".
1
u/necroforest 9h ago
the sum of a list is a really good example. intelligence as compression is a very elegant idea.
1
5
u/Fit_Employment_2944 9h ago
Turns out matrix multiplication can do more math than all mathematicians combined
2
u/Sweaty-Department595 9h ago
Not totally true because of the RLHF training. If the current models were simply pre-treined in all internet text, then you could say it is a spicy auto-fill.
2
u/OddInstitute 9h ago
There is a lot to criticize in recent developments in AI and extremely spicy autofill is an accurate (if incomplete) explanation for how they work.
If you want to have an accurate understanding of what the technology is and how it works, it is important to understand that it’s large linear algebraic operations (matrix multiplication) alternated with non-linear functions.
The non-linear functions are extremely important because if you apply two linear transforms in a row to a vector, you can always find a third linear transforms that will give you the same result as the combination of the first two. Said differently, if you multiply a vector (x) by a matrix (A) and then the result by a matrix (B), you can multiply A and B together to get a matrix C that will do the same thing. That is to say that A(Bx) = (AB)x = Cx.
This is an extremely useful property for math and engineering and would save a lot of time if you could just multiply out an LLM and turn it into a single matrix. The reason that you can’t do this is that neural networks need non-linearities in order to approximate any function.
Computers just happen to be really good at linear algebra, so if you want to do a ton of non-linear math on a computer and you can choose whatever approach you want, the cheapest and easiest way to do that is to do an enormous amount of linear algebra and then feed the result through a very simple non-linear function. This breaks the composition property I described above and gives you a system that in principle can approximate any mathematical function as long as you do enough math in the middle.
There are other systems that could have these properties and there are other reasons that linear algebra was chosen as the major building block for deep learning, but it’s important to understand that there is a really meaningful difference between linear algebra and deep learning. If you know math, the math LLMs use to run is still extremely simple and easy though.
I understand this is a snark post than is just dunking on idiots. I don’t want people to be getting their education about how these systems work from incomplete and misleading information. It is easy to criticize the wrong things or not understand how they will evolve if you don’t understand how LLMs are different from previous technologies. It is also not to absolve linear algebra and machine learning techniques that are solely linear algebra from societal harm. A lot of damage has been done through basic linear regression and criticism of these technologies based on it didn’t start with AlexNet or GPT3.
2
u/Business-Ride-6530 8h ago
Yeah....do people not know this? It only looks impressive because it has access to a massive data repository and large amounts of compute. It's not going to come alive like Skynet. On the other hand, rich people might decide to use it to wipe out the non-rich using autonomous weapons. Remember, EVERYTHING it does is deliberate action by some human.
2
u/Simple-Olive895 8h ago
I've tried to have this discussion with so many people. But people that are not actually interested in tech (as in actually has an interest in it, not just shitty vibe coders that don't know shit) just refuse to believe that LLMs really are just word predictors. They have an insane amount of data to access. So they are very precise in their predictions. But it still doesn't change the fact that they just use vector math in order to chain words that are likely to answer any prompt.
2
2
u/shumpitostick 6h ago
LLM pretraining is just learning to auto fill, but that's only part of the process. The reinforcement learning that comes next is an increasingly crucial part where LLMs learn to complete text in a way that actually fulfills tasks correctly or gets the user the output they want.
1
u/ScarletIT 10h ago
Pro AI here.
To be honest, anyone who think AI is god, AI is conscious or anything on that line is literally insane.
Listen... I like AI, I do things that are neat with it. it helps me solve problems (mostly in coding)
I don't worship it nor I treat it like a friend a lover a person or a deity.
It is a math equation.
1
u/Imaginary_Camera_298 9h ago edited 9h ago
legos are not complex but different types of blocks combined using simple rules together can create some pretty complex stuff especially if you have billions of them.
2
u/NotReallyJohnDoe 9h ago
The more interesting question is how can “spicy autocomplete” mimic humans so well? Maybe we aren’t as complex as we thought.
→ More replies (1)1
u/Eastern-Ad-3586 8h ago
This is the crux. Most human beings don’t actually think critically.
I’m a physician myself and a majority (not all, but the majority) of diagnostic reasoning is shockingly similar to how LLMs are trained.
Now the difference between me and an LLM is that I can reason in other ways and I have a physical body (a lot of diagnosing people with illness requires hands and ears and eyes) . But the LLM is absolutely going to revolutionize medicine if we let it (instead of using it to oppress people or make shareholders happy)
1
u/Glum-Objective3328 9h ago
The complexity of the math isn’t a factor in anything. Simple, easy to understand rules have always given way to difficult to predict, and complex looking from the outside, features. I wouldn’t say the code is alive in any way, but I do think comparing to ways we think is fair.
1
u/New_Salamander_4592 9h ago
yeah its been really annoying to watch the entire world collapse the economy over matrix math
1
u/bystander993 9h ago
This is a microcosm of society's ignorance. Learn a TINY bit about something and pretend you know everything about it. The math is simple, the result is anything but. We do not know WHY the math over enormous data self-organizes the way it does. The magic is that semantics emerged, we didn't expect it and we don't know why, which means we do not perfectly understand the behavior. We are studying it.
1
u/Eastern-Ad-3586 8h ago
I think the point being made is that a statistical machine trained on data isn’t going to develop a soul or human rights.
That doesn’t mean it isn’t a powerful and useful technology but the people wanting to give Claude personhood are ridiculous
→ More replies (5)
1
1
1
u/j_spencer1993 8h ago
This isn’t accurate at all. Have the issue is the commenters not understanding the different in machine learning, LLM, and agents and thing they are all the same thing.
Holy hell.
1
1
u/EgoistHedonist 8h ago
No they fucking aren't and this is a VERY ignorant take on the technology. Why on earth you need to fabricate comforting stories when the reality is already in front of you and you can see it by yourself. What is this blind denial of the current state of AI?
1
u/thecheeseinator 8h ago
But also like, linear algebra is pretty frickin cool. This feels kinda insulting to linear algebra.
1
1
u/Kadabrium 8h ago
If you think ai is smarter than you, keep in mind that they actually studied the study material they were given.
1
u/def_not_jose 8h ago
Well no one should be worried then, right? No one's job is that simple that it could be replaced by spicy autofill right?
1
1
u/Redd1tProtectsP3dos 7h ago
Sure let's just put a bunch of garbage into the left side. It will look neat on the right side.
1
u/Fangz713 7h ago
Can someone provide videos explaining AI that academic (using math) level while being simplified?
I also want to know if we have figured out how the brain (our brain) thinks?
If so, how do the algorithms compare (our brain's vs the AIs)?
Some time ago (years), I read that google had a whole datacenter just to simulate one brain.
1
u/MythOfDarkness 7h ago
You quite literally are just cells communicating with each other via molecules that you can download and visualize on a laptop.
1
u/SuspiciousSimple9467 7h ago
This is true, but who knows what’s going in the newest advanced closed source models
1
u/PhilosophicalGoof 7h ago
I mean… saying linear algebra is simple math is insane since linear algebra is a very popular concept that allows us to create graphic engines.
So as someone who dabbles in game engines… I m offended.
1
u/x6un5m0kex 6h ago
Ignorance is the mother of “thinking they found god and it’s just linear algebra”
1
1
u/Badtimewithscar 6h ago
people have this weird thing (i do it too, just not for ai) where we attach emotions and personify inanimate objects
1
u/RegularImportant3325 6h ago
My favorite realization was that AI wasn't so much designed and created, rather it was discovered. The math is relatively simple and has been around for decades. It wasn't until people shoved all digitized human output into it that it became somewhat like us.
1
u/True-Grab-5288 5h ago
Just wait until she finds out that humans are also just spicy autofill. It is the exact some mechanism, different substrate. There is nothing special about humans and never will be.
1
u/Moronic-Warrior 5h ago
Holy fuck this subreddit is truly brain dead. LLMs do think, they are not stochastic parrots, they understand information quite well and are able to organize concepts and ideas. Go on downvote me I dare you. No matter how much evidence I show you nothing will go through your thick skull.
Emergent Analogical Reasoning in Large Language Models
Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
Language Models Represent Space and Time
LLMs Are Not Stochastic Parrots: Evidence for Meaning-Mediated Abstraction from Conlang-Like Tasks
1
u/RepulsiveSuccess9589 5h ago
Yeah lol, just networks of nodes that can predict the next most likely word in a sentence
1
u/garloid64 5h ago
Holy shit man, do people really just not understand that you can combine a bunch of simple units and operations to get something complicated? How do you think a computer works? Conway's game of life is a good visual example. The human brain is also the same way, as is the entire rest of the body, down to every protein in your body which is just made from the same amino acids arranged in a particular order. Truly moronic argument.
1
u/walter_theweasel 5h ago
I mean, make all the criticisms of AI you want, I agree with a good amount of them and think AI companies need to be reigned in. But this just comes off as someone being snarky, who wants to find every reason to shit on a new technology and has already made up their mind that they’re going to see it negatively. By her logic, I guess a car is a spicy horse? “Hey, I looked into it, turns out a car is just four wheels and some metal gears that move forward or backwards when you make some stuff do combustion. What a stupid invention, I can’t believe people are excited about this technology.”
1
u/Melodic-Ebb-7781 5h ago
It's quite famously not just linear algebra? If it was then you'd only get a linear recombination of the input. Non linear activation functions allows the network to approximate any function.
1
u/realquidos 5h ago
I mean yeah, in computers it all just boils down to simple addition. So by that logic everything a computer does is just simple math. OOP is a dummy.
1
u/Beautiful-Signal6530 5h ago
Laplace argued differential equations used by a hyper intelligent being could actually predict the future if given unlimited computing power and the initial conditions of every particle, and equation of governing forces in the universe… this was back in the 18th and 19th century.
The line of thinking is not new that we found God. Eventually chaos theory and quantum mechanics squashed the idea Laplace had. Figure we will find a point with LLMs where they hit a wall and crush people’s dreams of AGI because we learn something new along the way.
1
u/One_Ad761 5h ago
You didn't learn shit then.
You can always find a way to ignore the complexity and say some bs like "its 1s and 0s"
1
u/zero0n3 4h ago
lol imagine saying all this as some literary professor who spoke on AI before even researching how it works..
Then to have the audacity to think this woman fully grasps how it works from a simple 16 minute ur e research run right before an interview?
lol this sub is filled with some delusional people.
Are “LLMS” alive? Yeah prolly not.
Are they extremely advanced tools that actually accomplish work?
Absolutely.
You sit here complaining while the builders are out there building shit with the help of AI;
production systems, side projects, exploring cool idea someone thought of in passing.
But please keep posting the bullshit and arguing with bots and fueling the division.
1
u/Scolmann 4h ago
And humans are just made of molecules, but if I took out all the carbon, oxygen, etc., from your body and laid them out in sorted order, I'd just have a bunch of molecules. It's the order they are in that makes you, you.
Yeah LLMs are just autocomplete, but the complexity is described with the model weights, much like our brains are (primarily) described with neurons. And we don't understand LLMs much better than we understand our own brains, that's emergent complexity.
1
1
u/thehodlingcompany 4h ago edited 4h ago
Anything running on a computer is ultimately just logical nand and nor operations. This is not a compelling argument against AI being possible.
Like what's the argument here, that if you can understand some complex phenomenon in terms of simpler phenomena then the complex thing doesn't really exist, it's "just" the simpler thing all along? How do you explain actual intelligence then? It's just something that exists and is irreducible to anything else?
1
1
u/r-3141592-pi 4h ago
Next-token prediction is only the training task, not the goal. People need to be a little more curious and ask: What do we get in return after forcing a network to predict the next token? What happens to the activations as we scale up this task?
Also, dismissing it on the grounds that "it is linear algebra" is ridiculous. Quantum mechanics "is" also linear algebra. Relativity "is" calculus, but only an idiot would think that's all there is to it.
1
1
u/Corvax123 3h ago
I don’t think you know what current AI actually is
, it’s far more than just an llm at this point regardless of how you feel about its progress. I find it interesting all the people shouting “stochastic parrot” are acting as nothing more than stochastic parrots.
1
u/Alarmed-Minimum-3814 2h ago
Damn, I’m guessing that she would’ve created the first AI if wasn’t for patriarchy.
1
u/Earthtone_Coalition 2h ago
Thank goodness we can all stop dooming about it, right? Can you believe some people think linear algebra is going to destroy the world?
•
u/frenlytransgurl 13m ago
Except the thing that predicts the next word with linear algebra can solve mathematical problems humans couldn't and can decompile closed source software. There was a study where a medical AI was flipped out of curiosity and instead of inventing medicine, it was able to invent thousands of chemical weapons, including both ones we know and ones we've never considered. Complex systems emerge from simple rules. Everything you've ever seen, touched, eaten, loved, likely even every thought and opinion you have, is millions of atoms arranged in specific ways and interacting with each other in specific ways. Did you know your brain is mostly made of hydrogen? It is a single proton with a single electron. Are you conscious or are you just fancy hydrogen? Linear algebra is just a method for quickly multiplying many value. The actual concepts behind machine learning allow an algorithm to mathematically model relatio ships between numbers that are so abstract it is extremely difficult for us to understand. It can take every data point on a 3D graph and extend it to a new dimension to find a relationship we could never comprehend unless we put out lives' work into it.
230
u/OfficerSmiles 10h ago
Yeah the people treating the current iteration of AI as alive are really bizarre.
Like a previous commenter said, the basics of how an LLM works is just linear algebra and calculus and statistics. Its fascinating though to be honest.