Is AI a Bubble? Gary Marcus on GPT-5 Hype and the Future of AI
Gary Marcus, the cognitive scientist, bestselling author and founder of Geometric Intelligence, argues that AI is probably in a financial bubble after the underwhelming launch of GPT-5. He says scaling large language models has hit a wall, that hallucinations come from how the models work, and that progress needs neuro-symbolic AI and world models. The hosts debate whether he is right and whether AI is underinvested rather than overvalued.
Transcript
AI skeptics went wild this week when Sam Altman himself was quoted as using the B word - bubble. Is this exciting AI race akin to the 90s dot-com boom and subsequent bust? Are investors and consumers a little bit overexcited about the promise of an AI takeover? One of AI's most outspoken critics says that the shine is wearing off. So what does happen when the hype dies down?
Welcome to Agents of Tech and to our Washington, DC studio. I'm Autria Godfrey, and with me today, here are Stephen Horn and Laila Rizvi. This is a very hot topic. It seems that AI is everywhere right now. It's really hard to miss. But is it really the undeniable next big thing? Or are we overblowing the impact that AI will have? Before I get to both of you and I get both of your thoughts, this, of course, is a very easy one for our viewers and listeners to weigh in on, so make sure that you leave your thoughts below and while you're there, don't forget to like and subscribe. All right, so AI, the B word. What do you both say?
So I was watching the GPT-5 release and demo live. And I will say I was disappointed. So, when I was watching it, it wasn't, you know, fireworks. And it was also just very sterile, the whole thing. But I still don't think just because OpenAI were a little bit limp in the GPT-5 release - and they also need to raise investment in whatever way is possible - I don't think that means that the entire AI train is kind of over and current strategies and paradigms, which have got us to this point have now hit a wall.
Well, can we not hold two things in our heads at one time? You know, I mean, it's just beyond us in this world of AI. But, you know, can we not think that AI can be a revolution and a bubble at the same time, right? You know that we are in the long haul. We are in for a revolution, yeah, but we're in a bubble as things go along. Like we've always been in the bubble in every industrial revolution that we've ever seen. OpenAI is a $500 billion company, right. Let that sink in.
With regard to ChatGPT-5 do you think that people were expecting too much of a complete overhaul and not just some slight improvements?
Well, yeah, because GPT-5 had a lot attached to it. As in you saw those images from when GPT-4 was released that this is how big GPT-4 is and this is how big GPT-5 is. So everyone was expecting a huge jump. Really, GPT-5 should be called GPT-4.6, but that's not going to get OpenAI another 100 billion is it? So they had no choice but to because everyone was waiting for the next thing, which was labeled GPT-5.
So do you think the other problem with ChatGPT now is that everybody has their own view and what they think it should be? ChatGPT-5 the latest thing. So everybody, all of us think it should do this or it should do that. And if it doesn't do what we want it to do all of a sudden, it's rubbish, right?
Yeah. I'm glad that you mentioned the investment component because I would like to have your viewpoint, as you know, a business owner, as an entrepreneur, do you think that we are over investing in AI?
No, we're underinvesting. Okay.
Yeah, I agree, I think a lot of these companies are still undervalued, like that sounds ridiculous, but I actually think that's the case.
I mean, trouble is with a lot of it though, Autria, it's not very sexy, right? It's productivity improvements. I mean, no one gets very hot under the collar about productivity improvement, but with agentic AI like we saw with James, that's what's happening now. So yeah, no, I think we're underinvesting. I think all of us are going to be overinvesting in AI.
All right.
And I think we also have this kind of, tolerance where we expect something of the AI. And if we actually compared it to how GPT-3 was two years ago. Seeing GPT-5 back then, we'd say this is AGI. But now because we're like, oh, actually. We’re a little numb to it. Yeah, exactly. So I think that's going to continually happen and something which resembles AGI will come about and everyone say, well, it’s alright, okay, but why can't it take us to Mars?
Right. Yet. Get Elon on the phone.
It sounds like we might have some fireworks with Gary Marcus, that's coming up. All right, let's get to it.
Gary Marcus is a cognitive scientist, bestselling author and founder of Geometric Intelligence, which has been acquired by Uber. He is also one of the most prominent critics of deep learning and large language models, and calling them for more reliable approaches and stronger regulations. Gary, such a pleasure to have you joining us today.
I appreciate it. Thanks for having me.
All right. So the word skeptic has been thrown around a lot when it comes to your name and your viewpoint on AI. What do you believe? Is this a bubble?
I'm a skeptic in all things in the sense that I'm a scientist. I want to know how things work. I want to know whether the claims that are made around them are true. It so happens that in the generative AI era, there have been a lot of untrue claims. And as a scientist, I call them out. Especially because I think that they have consequences, negative consequences for society. I actually love AI and I wish people would get that part too. It's not that I don't want AI to succeed, I do want it to succeed, but I don't like how it's being done.
Now, in terms of the bubble part, we're probably in a financial bubble. The valuations don't really make sense in terms of two things - the economics and the technology. So from the technological perspective, which is really how I come into this, I can look at how a large language model works and what it does, and I can see that it has problems. In fact, some of those problems I foresaw a quarter century ago literally I wrote a book in 2001, and I pointed out that technologies that are kind of the ancestors of today's technologies would hallucinate. I didn't use that word, but I said that they would inevitably overgeneralize. And that's what's happening. They hallucinate, they overgeneralize, they get things wrong. Yesterday I asked, ChatGPT-5 to draw a picture of me juggling five tennis rackets, and it gave me six. And I said, you know, is there anything wrong with that? And it said, oh, well, there are six. And then I said, okay, can you try again - or it offered to try again - and it made seven.
On the technical side, the reality is they’re like a magic trick, a parlor trick. It's an incredible trick. There's some genuine uses for it, but it's just not as smart as it looks. LLMs are interesting, but they're not the answer to so-called artificial general intelligence.
And so now comes the economic side. In the economic side they don't actually make that much money compared to their enormous costs. The trick people have developed is one that requires training a model on the entire internet. That turns out to be expensive, you know, hundreds of millions of dollars, sometimes billions of dollars, possibly. And it's much more expensive than traditional software. So when your expenses are billions of dollars, you need to make a lot of money. And so far, all that people have been able to make out of LLMs is maybe the entire industry has maybe made 20 billion dollars to date, but has spent like 500 billion dollars. Those numbers obviously don't make sense.
Now, the fantasy was okay, GPT-4 is not that great, but GPT-5 is going to be great. It's going to solve all these problems. So, you know, over the last couple of years, everybody tried ChatGPT. Everybody kind of saw what I'm saying, which is like is really cool, but you can't really trust it. So the theory was that the next model would be so much better, would solve all those bugs, they'd all go away and now it would be worth it, you know? Now you'd be able to make infinite money. You'd have this thing called artificial general intelligence that could solve any problem you had. And of course, people would pay a lot of money for that. I mean, that might actually be worth trillions of dollars.
But the problem is we don't actually have that, and nobody actually knows how to build it. I had been saying for years that GPT-5 would either be late or a disappointment. As it turned out, it was both. That has I think, led to the kind of wake up call that could lead to a crash in valuations.
So, Gary, do you think GPT-5 was underwhelming because the scaling paradigm kind of hit a wall?
Exactly. I mean, those are all words that I actually kind of invented in a 2022 essay called ‘Deep Learning is Hitting a Wall’. Yeah. I said that scaling was not going to be able to continue. What I said was, you have this history of it, but history, it’s like what they say in stock prospectuses past performance is not a guarantee of future performance. And what I said was, you are going to keep getting stuck if you keep using this technology, on hallucinations, on stupid errors and so forth. And that’s what people dismissed, but that’s what they’re realising is in fact, true.
Did scaling even happen with GPT-5? Because Rohan Pandey, he said after GPT-5 was released, that from GPT-2 to GPT-3 to GPT-4, there was a 100 time scale up in pre-training compute. But GPT-5 wasn't 100 times scale up because you know, Stargate, Colossus 2, those data centers, just they haven't been built. So if it wasn't a 100 time scale up, then is it that scaling has hit a wall or scaling just didn't happen for something like GPT-5?
I don't know how much bigger 5 was than 4. Yeah. But scaling is hitting a wall one way or another. One way might be they just can't get the energy together because the needs are so enormous to make 100x. Sure. Or they can't get the chips to talk to each other well enough. There's some communication issues, or it might be that they can't really get the data that they need in order to do this, but whatever it is, this whole procedure of we'll throw everything we've got at it in terms of data and compute just isn't working anymore. And it's not just GPT-5, it's just GPT-5 is most visible. But the same thing happened with Grok 4 and Llama 4, and a model called Behemoth that Meta had and, you know, the latest versions of Claude.
In general, everybody has been trying the same things since GPT-4 came out, which is to put as much in it. Now, I don't know, because they never published what they're doing anymore. They're not really doing science anymore. It's like more marketing in a way. But, you know, so I don't know, I think Elon said that Grok 4 was 100 times bigger than Grok 2. Strangely, he's been one of the few people to be candid about what he's done. Yeah. But none of these experiments, whether they were 50x or 100x or whatever, got to the desired results. So the people in the industry now realize that some of them talk big, but, you know, behind the scenes, they will acknowledge this is not really gotten them to AGI. And it was disappointing. So scaling as best they could do it did not make 5 live up to expectations.
Part of why there's a sea change is because people did place so much weight on GPT-5, when Grok 4 failed, it showed the same thing. I mean, Grok 4 didn’t, it's not terrible, but it's not so much better than its predecessor. So it's 100x, but it wasn't giving you that much more bang for the buck, right? So Grok 4 was a failure, but people let it slide. But there was this mystique, which Altman directly built around GPT-5. With so many rumors and blog posts and jokes and whatever, he just constantly was implying that it was going to be GPT-5.
And at this point, it was late, and we can ask why they released it under that brand name. We know, in fact, that GPT-4.5, we've pretty good reason to believe that GPT-4.5 was supposed to be something called Project Orion that was supposed to be 5, and they realized it wasn't good enough. And so they called it GPT-4.5 to kind of protect that brand name and protect their name and whatever. And for whatever reason, they didn't call this new one 4.6, I would have called it 4.6. It doesn't have, you know, what 5 seems to, you know, what expectations around 5 were.
I've come from a business perspective and, and there's, I'm not quite sure it has to be all or nothing. I'm not quite sure we have to have AGI or the whole thing's written off, you know, it's that, you know, you can get big productivity improvements using a large language model without expecting it to run your entire business.
First of all, in the abstract, you're right. Right. There's got to be some in between. Like something in between GPT-4, which, let's be honest, has not yielded the productivity improvements that people thought. When you do careful studies, you get a very mixed bag. Sometimes you get improvements, sometimes you don't. And sometimes people overestimate.
So there was a study by METR, that tries to do independent benchmarks. And they looked at computer coders and they said, how much is LLM coding going to help you in these problems? And they said about 20%. Or maybe it was after the fact, how much did it help them? And the actual result was it made them 20% slower like it writes code faster, but then it's buggy code. You spend a lot of time trying to debug it. So they weren't really getting the productivity. But when ChatGPT came out people were talking about like 100x-ing the world economy, crazy stuff like that. And the reality is there are no studies that show a dramatic improvement to the world's productivity from these products. It just hasn't happened yet.
So okay, that's what's happened so far. AGI, I think would obviously be a huge improvement to productivity. The question is about in between. What I would say is the key issue is reliability. Right? Right.
You know calculators you type something and you know the answer is right. Spreadsheets. You can mess up the formula, but Excel is not going to mess up its computation. LLMs just aren't like that. And that means that it limits the productivity because you're always worried about hallucinations, some of which can be very expensive, right? If you're running a customer service thing for a bank, you know, a single mistake could cost you a lot of money. You tell the customer, you know, you did this thing, you didn't, and they sue you. And you know, there's bad press and so forth. And so a lot of the productivity gains are only going to come if we have reliability.
In my thesis, which has held strong for a quarter century, is that these technologies are not going to give you the reliability that you need. Yet, nobody really listened to me until a few weeks ago, but in the last few weeks, I think a lot of people have realized that that is in fact the case.
So like, for example, you were saying about, AI kind of code like Cursor, Claude Code hasn't actually in studies shown to improve coding productivity, but Anthropic have claimed that Claude Code wrote 80% of itself. So I mean, it must be pretty reliable if that's true, I mean, I again, I agree that there's a lot of marketing speak they’re trying to use.
Even if it's true. Yeah. I really want to see the asterisks around that. Yeah. Before I would be prepared to believe that it's true. There is boilerplate code and there is a kind of abstraction by which you build that system. Sure. I doubt they did any of it.
So the way to think about these coding assistants is they're kind of like interns. So you can say, go do the boring part for me, but they don't really do the sophisticated part. And like we're not seeing in studies massive gains. METR, I think did the most careful study and saw a 20% loss. But it's in a very particular circumstance. Some people have found 20% gains, but like we're not seeing the 10x or 100x that people were talking about.
Because obviously the whole AI 2027 stuff about AI exponentially kind of improving and using AI to develop, AI machine learning engineers, what do you think of that? Do you think it is on an exponential curve or that's a load of rubbish?
I mean, if you really want to get technical about the math, it depends on what the exponents are in the curve. You make them small enough then you can make anything exponential, but no we're not you know, GPT-5 is late and is not what was expected. And same with all these other models that I mentioned before. We just aren't on that curve anymore. We were in the transition from GPT-2 to 3 to 3 to 4. And we're not even with the so-called reasoning stuff that they've added. It's like, well, we're just not there anymore. And that's what the field is waking up to.
And if you listen to the chatter and so forth, people are like, maybe we should use small models. Some people are adding neuro-symbolic AI which I could talk about which is something I've added for a long time, but the pure scaling is not giving those exponential improvements.
I want to go back to something that you mentioned right off the top when you were talking about, you know, just like the basic facts, getting messed up and these LLMs are being trained though on data, clearly that's incorrect somewhere.
It doesn't even have to be trained on incorrect data you can train an LLM only on correct data, and it will still screw up because of the nature of what it does. So what it does is it breaks everything it sees into small little bits, and it tries to cluster those little bits together and then reconstruct them. But something gets lost in the breaking down into the little bits.
So I'll give you an example. I wrote a whole Substack about my friend Harry Shearer, and what happened with his biography. So Harry is an actor, you know, his voice because he does Mr. Burns on The Simpsons. So he's very well known. There's lots and lots of data about him, many of which say that he was born in Los Angeles, which is true
And still someone sent him a biography, which he passed along to me, because these things become a running joke between us. Somebody sent him a biography, which it said that he was a British comedian, voice over actor, etc. but he's not British, and there's incredible amounts of data in the database to tell you otherwise probably, you know, 1000 or 5000 interviews or something like that. There probably wasn't any data until I shared his example saying that he was born in England, but on the other hand, his mate, so to speak, in Spinal Tap, Christopher Guest, the actor who plays his mate in Spinal Tap is a British actor. And it's just conflating these different pieces in the process of reconstruction.
So in technical terms, there are things called embedding spaces and clusters within them. And it's going to put Christopher Guest and Harry Shearer near together because they were in, I think, 4 or 5 of the same movies. And the system is just conflating them together. So it's not just that the data is bad.
Can we move on a little bit to to what you think some of the answers are. Because, you know, you talked about that a little bit earlier it’s something that you've been studying a long time. You know, you said right at the beginning that you believe in AI, but at the same time are an AI skeptic, I'd be really interested to move on to, you know, from all the work you've done where you think we, you know, where we might head on this.
So I wrote a paper called ‘The Next Decade in AI’, which you can find free on arXiv just look up the title ‘The Next Decade in AI’ - in 2020, explaining what I thought we should do. I still think we should do what I said there, which I'll explain in a second, and we’ve made no progress on any of that. And that's really depressing. So I said, we need to do four things. I won't go into all of them here, but I'll give you two of them.
One is what I call neuro-symbolic AI. This is something I've been focused on for a lot of my career. What that means is combining two approaches. One approach is called neural network. That's what large language models are. They're supposed to look a little bit like the brain. They use all these things that we call nodes. You could think of them as neurons that are very oversimplified. And they basically do pattern matching. So that's one approach. It's very dominant right now. The other approach just looks like a computer program. We call it symbol manipulation. It has its history and algebra and logic. And you know, most of it is recognizable as a typical computer programs. You have functions that you transform and so forth.
And so what I have always said is we need to bring these together. There's a history, a kind of sociology, where people played on these two teams and hate each other. They fought over grant money and graduate students and prestige and have been really, really bitter. And they ridicule each other and so forth. The reality is we need the best of both worlds. So that's what I call neuro-symbolic. It's number one and number four in the paper, but we'll call it number two here,
Is what some of us call cognitive models or world models. And this says that you have kind of like a database in your head of the stuff going on. So, like, I build a world model of the scene I'm looking at. So there's a table, there's three people, there's some microphones, there's a screen that I'm projected on on a tripod, there’s bookshelves. And then I'm also building a model of like your personalities and who asks tough questions and who has the business questions and so forth. Right. So this is really basic to how we think about the world.
We're like, okay, somebody is flying. If it's Clark Kent, I dig it because that's his world. If I see it here on the street, then like, whoa, what is going on there? I need to understand that, etc. So we're always doing this, and our reasoning is relative to those world models that we built. And LLMs, it's just what would somebody say here, given the bunch of words in a particular context, it’s very different. We need to build AI that does that.
We have it in certain limited circumstances. So a GPS system, a navigation system has literally a model of the world. What are the roads? How do they connect to each other? If they make a mistake, it's because the data was wrong. But given the correct data, they make 100% reliable inferences and they get you from point A to point B. And so we do use world models in certain cases, people building robots have understood this for years and years, but a whole lot of people kind of fell in love with the magic of large language models and gave up on that approach. We need to go back to that.
We've got companies in Silicon Valley now who are valued at hundreds of billions of dollars and they're valued at hundreds of billions of dollars around these large language models. What needs to happen for people to stop going down that path and to go in the direction that you're talking when the huge monetary incentive is around the large language model
There’s a book, an old book, and it was about things like the tulip crash. Right. So imagine you're in the middle of Holland. Tulips are going insanely up, and you sit there and you know that this doesn't make sense. I mean, really $10,000 for a tulip, right? But you don't know when the mania is going to end. So you can make a rational argument that this, you know, just does not make sense, but you don't know when people are going to change.
Now, I think the GPT-5 moment of disappointment might change that crowd psychology. I don't know for sure. I mean, they're still talking about getting a $500 billion valuation for OpenAI, which to me makes no sense. You know, given the quality of the product. Also the fact that everybody knows how to build the same thing. So there's no moat, right? If you're the only game in town, you know, maybe. But if other people are doing the same thing, like the economics don't make sense, but people have to wake up and realize that. So, and it's not just the economics, it's also the research. Right? Which is part of what you were getting at.
So in order for people to think about other ideas, they have to realize that the ones they're using are not working that well. There is actually some recognition of that in industry now. So for example, they're using a lot of what they call tools, which actually are like classical computer programs in their systems. They're not making too big a deal of it. But they are, they're actually doing this. And so they are slowly shifting what they're doing because, even if they don't want to admit in public, in private they realize that they're not getting the gains from adding more data that they want. So there's starting to be some pressure realizing from the internal failures, even if they don't make them public, that they do need something new.
You touched on neuro-symbolic AI a minute ago. Is that, in your opinion, the answer to these hallucinations?
To use philosophical terms, it’s a necessary but not sufficient condition. So unless you have symbolic representations, you're going to make these mistakes. Now, that doesn't mean that all of this is going to be solved the day that we have neuro-symbolic AI, we have to build the world models and all the other things.
How would you do that at scale? Because like, how would like that's the challenge, right? And I guess when the large language models kind of came about, the idea of just throwing loads of data at it was controversial itself as in that wasn't the standard path. So like, how do you do that?
Yeah, I think it's a really, really good question. It's so much easier to build a large language model than to put in the hard work, to do the things that I'm talking about. And the only project where somebody did this cost maybe $100 million dollars, which seems small compared to the amount of money thrown around now, but still seems kind of big, and nobody kind of has had the appetite to do it. It is a lot of hard work. As you know, it's relatively straightforward what it means to scale a large language model. I just get more data through it, and we need to make discoveries about how we're going to do this with these alternative architectures. We don't really know.
You know, it's such a good question because it's one we don't know the answer for, like, you know, most of the things you ask me, I do know, and that one, I don't, I know some of the places that we can look, but that's where we need to do the science.
You know, an analogy I would make is like in the age of alchemy, people had some good intuitions that you could kind of take different chemicals and put them together and do cool stuff, but they didn't know which is which. They didn't know that if you actually wanted to make lead into gold, you were going to, you know, need massive amounts of energy. And so, like, they need to understand that they had, you know, learned certain things that work and they were missing certain things. They had not invented the periodic table that organized all of it. We're kind of like in the age of alchemy for AI like people try some stuff. Some of it works, some of it doesn't. They don't tell you about the stuff that didn’t, but then they get a little bit better. But what we're doing is not very principled, and we're going to have to invent things and have to make new scientific discoveries in order to answer your question, how are you going to take a different approach and make it at scale?
Seems like a good place to end. I have one final question. I want one final thought from you. And this is where I think, being an AI skeptic is probably welcomed by the masses. Will AI take all of our jobs?
Not anytime soon. I mean, hundred years from now? Sure. Like, okay, it's almost impossible to predict where AI would be in 100 years. It's certainly possible you will have achieved so-called artificial general intelligence that the economics of it might make sense that it you know, I mean, maybe it'll take so much energy that we can't figure out how to do it well.
But there's a reasonable likelihood in 100 years that AI can take most jobs. In the near term, they can't do physical labor that isn't like highly structured. They're still pretty dumb. You know, they can do some things, not others. Most of the promises about professions being replaced have not come to pass.
Like in 2016. Geoff Hinton, who later won a Nobel Prize, said that we should stop training radiologists. Well, here we are nine years later and no radiologists have been replaced. Instead, what we found is that AI is helpful to radiologists. But radiologists know a lot of things that we don't know how to program into AI. Mostly it's been like that. There are some things where AI really has replaced some people, not typically a whole profession, but some people.
So for example, voiceover actors, famous voiceover actors aren't getting replaced, but not famous ones? People breaking into the profession? That's just not a thing anymore. Because if you just need a voice that doesn't need to be recognized, does not need to be recognizable, AI can do that for you. So, you know, I don't think anybody would have guessed the right professions that were going to place. A lot of people thought drivers would be replaced. We're only starting to replace them a little bit in certain cities, not in general. So a lot of the things people thought would be replaced aren’t, I don't think anybody thought voiceover actors would be replaced but they kind of are, so we're not very good collectively at predicting that.
But I would say in the next ten years, you know, plumbers, carpenters, gardeners are not going anywhere. Actors are not going anywhere, etc. Inexperienced computer programmers might be in trouble. Top computer programmers aren't. It's a sort of complicated, mix, but we're not going to see massive unemployment in the near term. We do need to think about what's society going to be like. How are people going to find meaning when we do reach that? Even if it's 20 years, 50 years or 100 years, we should start to think about it.
All right. Wonderful conversation. Thank you so much for your time today. Thanks for all your insights. Really appreciate it. Thanks so much again. Thank you very much. Thank you.
Okay. So I thought it was interesting. I felt like you and Gary aligned a little more than I expected you to.
Yeah. I mean, we're both disappointed by GPT-5.
100 years? I don't think I'm on that timeline. We went from horses to landing on the moon in, like, 50 years, so. And AI is moving faster than that. Like technology in the last 100 years is insanity, if you think about it. So I don't think it's just going to be yeah, most of us will be replaced by 2100. Yeah. I think meat robots are coming soon. Meaning us doing.
But don't you think, based on what he was saying with the, you know, the factual errors still existing in the hallucinations, like, there's always still going to have to be a human in the room.
To some extent, but one thing that I didn't get the chance to ask him was about the hallucination benchmarks, as in they’re currently for the latest models like 1%, right? Is that less than humans getting things wrong? So I mean, are they a lie? Like, are those benchmarks just a lie? And anecdotally versus, Gary's experiences of the seven tennis rackets.
So, Gary, over the past kind of a couple of years has used the test of a horse riding an astronaut as an image. I did that yesterday with GPT-5 Thinking, did an amazing job. I've not had any problems with GPT-5 Thinking.
So yeah. I mean, he raised some really interesting points like you'd expect, right? It was very interesting. But you know, he sort of said, didn't he, that, you know, ChatGPT is to tulips. You know, we all know what happened to tulips in the 17th century. The whole market went bust. So is he saying that now all these large language models, all the, you know, OpenAI companies are now going to go bust? Because he didn't have an answer for Laila’s excellent question, did he? About how do you do what he wants to do at scale? So these guys trying to do what they're trying to at scale. He hasn't got an answer for that. So.
So how do we transition from large language models to this kind of, you know, different view of the world? Because when we spoke to John Jumper, he said something like they would encounter a problem. And it's surprising how actually quickly that team would innovate to come up with something completely novel to get over that obstacle. And I think people in the entire AI industry are doing that. Reasoning models didn't exist before. o1, o3 but now they do. It wasn't just, you know, brute force pre-training. So I think that that's going to happen. And they are all kind of innovating.
Right. But that for me, what's so fascinating. So the problems that you were talking about at the beginning and he was talking about, I mean, I think everybody can see it, right? Everybody can see the mistakes it makes. But everybody can also see the potential in where this thing is heading. But how do you then get there? How do you then switch from one to the other? Sure. Is the thing that we're all going to to find out. For sure.
Yeah. And one last thing. I do agree with him on the point that, you know, if something is the general consensus and it isn't working, you do have to try something different, because that's exactly how the LLMs started. As in, Geoff Hinton's team just kind of had a hunch that the current paradigm wasn't working. Let's just throw loads of messy data at it and no one thinks it will work. And it worked. So I think that mentality and that approach has to be applied constantly, that it can't just be. Right.
But the problem with that is when your company's worth $500 billion dollars. Of course. Yeah. Then throwing the baby out with the bathwater is a little bit tricky. You have a lot more to answer to. Yeah.
All right. Very fascinating conversation. You've heard Gary's case for not taking AI at face value, and we would love to know your thoughts. Make sure to comment below. Let us know if you agree or disagree, or maybe you're somewhere in the middle. Make sure to like and subscribe while you're there as well. The more feedback we get, the more we can tailor our talks. Until next time, we'll see you later.