The $5 Trillion AI Bet You’re Already Paying For
Amba Kak, co-executive director of the AI Now Institute and a former senior advisor on AI at the US Federal Trade Commission, argues that the bigger danger from AI lies in the monopolies that own it. She tells Autria Godfrey, Laila Rizvi and Stephen Horn why she believes governments are already giving the industry a slow bailout and who would pay if the spending does not pay off.
Transcript
The people building AI say that even with the promise of life-changing progress, it poses an existential risk. But our guest today thinks that the biggest danger isn't actually from the machines, but from the monopolies that own them.
These large bets aren't being made by you and I. They're being made by a small group of billionaires in Silicon Valley.
By 2030, what do you think is needed for these AI expenditures to make sense? What needs to happen?
To make good on the estimated $5 trillion of expenditure that we are already going to see in the next five years, we would need to see, per the Bain study, $2 trillion in annualized revenue, larger than the entire size of the global software enterprise market.
Who's on the hook for it?
They're looking to not just our 401ks, but our insurance plans, our life insurance policies, in these vast amounts of capital expenditures that are being made.
What you're describing sounds to me like the 2008 scenario of too big to fail. Is that the trajectory we're on?
What we're actually seeing is a slow bailout.
Amba Kak helped build Mayor Mamdani's new government in New York City, a place that's home to more billionaires than anywhere else on Earth, and now with a socialist in City Hall. She argues that we might already be seeing the government bailing out these large tech firms slowly, and that it's the ordinary people that will pay the price.
Amba Kak runs the AI Now Institute and has been named one of the hundred most influential people in AI by "TIME" magazine. She says that big tech's circular financing deals are putting us all at risk, and today she joins us to unpack it. So what should we fear more? AI the technology, or the few who control it? Before we hear from her, we want to hear from you. Leave your thoughts in the comments below and be sure to like and subscribe.
So Stephen, I want to get started with you. As CEO of a company, do you worry about this circular financing that we're seeing?
I worry massively about it, and coming from somebody who is very all in on kind of AI, as you know, and the investment returns on AI, 'cause I'm not Gary Marcus, I do believe that we're already seeing these returns, but I worry that no one is writing this down now, that we have lost sight of the amount of investment going. Think about subprime. This is subprime all over again, and that is what worries me.
But what happened to ordinary people during 2008? And are we going to let it happen again or is the government, like how are we going to protect the rest of us?
The problem is it's already happening. So it's not a question of are we going-
So do we just sit back and just hope the train doesn't crash?
Are we already seeing a government bailout of these tech companies?
There's two kinds of worries about AI, one of which is, "Is there a bubble?" And we're all, our 401ks, we're all somewhat implicated and vulnerable in all of that. And in that context, we might feel somewhat reassured that a bailout might also be on the horizon. But then there's a whole entirely different worry, which is, is this even a market that's standing on its own two legs?
And the argument that me and my co-authors have been making is that what we're actually seeing is a slow bailout. So the idea is that we have a market, one which has shown a tremendous amount of value in sort of isolation, but not as much when you think about the eye-watering amounts of capital expenditure that are being made at the same time. So the math still isn't mathing. And in that context, it feels like what the government is doing is playing that very important function of providing both direct and indirect forms of legitimacy and viability to a market that, you know, may not be standing entirely on its own legs just yet. And that's what we've been arguing feels a little bit like a slow bailout.
So Amba, by 2030, what do you think is needed for these AI expenditures to make sense? What needs to happen?
The experts have done the math, and to make good on the estimated $5 trillion of expenditure that we are already going to see in the next five years or that have been committed to be made in the next five years, we would need to see, per the Bain study, approximately close to $2 trillion in revenue and annualized revenue come in. Just to put that number in perspective, that would be larger than the entire size of the global software enterprise market.
There is a real question, you know, you could put it in ways that are familiar, like what is AI's killer app going to be? But I think the more deeply worrying question that is keeping me up at night is the only vision that would make good on this kind of revenue vision, one that does implicate our worst fears around vast displacement. Because often I think that's when it feels like, right, if you're going to put a whole lot of people out of jobs, and AI is really going to be in a position to accrue, you know, the value in the economy that is currently going to labor, that's when, you know, maybe we can start seeing those numbers make sense. But if it's just our trusted helper, I'm less convinced, at least off the bat.
So if it is just our trusted helper and we're spending all these trillions of dollars, who's on the hook for it?
In the beginning, I think I was less worried because it was these big tech firms. They've spent, you know, a decade plus creating extremely deep capital reservoirs, and they were mostly putting that cash back into the AI business, which is kind of their call to make. But increasingly what we're seeing is these firms are looking to debt markets and they're looking to, it is now not just our 401ks, but our insurance plans, our life insurance policies, all of which I think are also, from a financial picture, being implicated in these vast amounts of capital expenditures that are being made.
And then I think that, you know, what we've been arguing in our argument about the slow bailout is underway is that at some point, with enough government and other market sort of backing, you could reach this really uneasy place where the market simply has to make it, or it's the feeling in the market is that we have to make good on this bet, by hook or by crook. And I think that's when you start to see applications of AI that, you know, aren't fit for purpose, or entire sectors getting overhauled in a structural way so the market can be AI-shaped. So the problem that we're looking at will be recast as an AI-shaped problem so that then AI can be the solution. I think then you see a lot of perverse incentives to sort of plug AI in every hole, or make every hole AI-shaped. And to me, those are the warning signs, more than anything else, is when you see a market start acting out of desperation.
But given the money is being spent, I mean, that's the one thing we can all agree on, what do we do about that? We might not be sleeping at night, but we're spending the money.
Well, depends on who we're talking about when we say "we," right, what is troubling to many people, certainly everyday people across the world, certainly people that are, you know, on the hook with data centers in their backyard and the like. The thing that it feels off, or missing in this moment, is that these large bets aren't being made by you and I. They're not being made by our political representatives. They're being made by a small group of billionaires in Silicon Valley with, you know, a very persuasive, by all accounts, vision of where the future should go.
But there's equally a very live conversation based on their own remarks about what that future looks like with a lot of people saying like, "Hey, wait a minute. A, we're not sure we should be headed in that direction because it feels like a future that suits the interests of these tech companies, not so much the rest of us."
What you're describing sounds to me like the 2008 scenario of too big to fail. And, you know, it takes us back to like Lehman Brothers and Bear Stearns and, you know, the government bailout that ensued there because they had propped themselves up to where if they were going to collapse, then they were going to take society down with them. Is that the trajectory we're on?
Yeah, and I think with AI, there's sort of an additional dimension to its too big to failness, and that is that it is also being cast as too strategic from a national security point of view to not pursue to the bitter end, right? Because if these tech guys are right and if we're looking at something resembling super general intelligence, right, the acronyms keep changing, AGI, ASI, if that's a real thing, and I will always approach that with the healthy amount of skepticism it deserves. But if there is a chance that it's a real thing, then what if China gets to it first or what if an adversary gets to it first? And that's what I think ensures that not just the tech companies keep their sort of pedal to the metal, but certainly governments feel like they have no choice but to submit to this paradigm, no matter the cost.
And I think where that is coming up against the everyday person's experience is, you know, they're like, "Okay, this is like big talk about geopolitics and, you know, the visions of super intelligence and killer robots and all of the rest of it, but sort of, what does this mean for me, for my job, for my kids, for, you know, the kind of immediate environment or the material reality in which most people in the world live?" And I think that disjunct between the logics propping up this market and the everyday experience of those that are, you know, going to live with its consequences is getting wider and wider.
All right, we're going to get more into the societal implications coming up in just a second. We're going to take a quick break, and we'll be right back with Amba Kak on the other side.
Coming back to what you're talking about China, what is your response to those that say, "Well, if China gets ahead, it is going to be a huge threat to the US." Like how do you respond to that? Because that is what people like David Sacks have been saying.
One thing to remember is that this argument about "What about China" has been made for, at this point, close to 15 years by the tech industry when faced with any questions of accountability, questions of like basic baseline protections for the public. So there's a little, I guess I do want to say there's something perverse about the zero-sum game way in which this argument is used in DC, right? It's data protection law, but what about China? Basic protection for kids and teenagers online, but what about China? And I think I would be resolute in the fact that we must always ask questions about not just the argument itself, but like, where is it coming from and what is it being used as a shield for? So that's one.
But I do think to take the Chinese example, right, arguably they're worrying about similar questions, whether or not we agree with anything else. If they are in this great power rivalry, surely they're thinking about the same thing. And yet all of the moves that have been reported, at least in Western mainstream media in the last several months, have been about the Chinese government getting A, more and more worried, as they typically do about their industry sort of acting bigger than their boots and sort of trying to constrain their own domestic industry to make it much more responsive and responsible to not just government interests, but also on the labor front, they're very worried about their population and fears and anxieties about jobs. And so a labor court ruled to make sure that AI can't, without due process, take someone's job.
Now, what this tells me is that governments, including the Chinese government, can't have a conversation about AI and its impacts that is solely about a great power rivalry with another nation. It has to at some level be accountable to its own people, right?
If so many things are keeping you up at night, what do you think governments should do? What should regulation look like?
I mean, the good news is, you know, we have a laundry list of what are some kind of basic hygiene, you know, baseline regulations we could put in place, whether that is around, you know, protecting our data, making sure that we have really smart but also clear rules around where AI can and can't be used, whether the ways in which it can be used or can or can't be used in high stakes setting, whether that's healthcare or education or the battlefield. Right, what I want to say is that this is sort of the bread and butter of what we do. There's a long list of what we could do.
It doesn't matter if we have the right ideas, if we don't have political will. And if with any time there is any kind of baseline, like regulatory proposal put before Congress, or frankly anywhere in the world, you know, there's an army of lobbyists standing at the door ready to water it down to, you know, a level that it is almost meaningless or toothless. And that's why I think, you know, you may disagree on the fact or even the impetus for the data center protests, or for that matter, the concerns around children and AI.
But I think what these issues have done is that they have for the first time made AI deeply unpopular at a public level. And finally, even despite the vast inflows of lobbying dollars that are coming from the AI industry, you have political representatives say, "Look, we need to regulate AI." And so we're in a better moment than we've ever been because there's a lot we can do, baseline transparency, accountability for the sector, to, you know, make sure that this sector is eventually accountable in some level to the public.
But when it comes to the concentration of power, do you not think that regulation will increase that, because the big players are the only ones that can navigate compliance like what happened with GDPR? And similarly, do you think that AI as a technology structurally will lead to power concentration, just because of the nature of the scaling paradigms?
I think one of the things that is really worrying about the structure of the AI market, and maybe more importantly, it lends itself to a structure which is more winner-takes-all-esque. Whether that's one winner or five in a cabal, you know, we can debate that, but it seems like it's going to be that kind of a market, certainly every single signal till this moment has pointed and reaffirmed that the market is moving in that direction of consolidation.
This kind of concentration of power is a problem for innovation, it's a problem for the small guy, who, you know, at this point might be happy that he will be one day soon acquired by one of these bigger players. But longer term, I think it speaks poorly to a market where gatekeepers or kingmakers can simply shut the door behind them, or really cannibalize the pipeline for innovation in the first place. I think we're all worse off. And of course, there are follow-on political and other impacts of that kind of concentration that make it deeply worrying.
Now, the question of whether regulation will end up reinforcing this consolidation or relieving us from it, because we spend a lot of our time pushing for fair rules of the road when using competition law to really make this market less rigged. So I think there's A, there's a host of rules that we could put in place today that would make sure that even if Anthropic, OpenAI, and the large hyperscalers are in this potential gatekeeper position, here are a list of things, non-discrimination rules, that mean that they cannot abuse that position in a number of ways.
We could have, as we had certainly during Lina Khan's term at the FTC, really strict merger review or acquisition review to make sure that these firms are not in a position to just eat competition. You know, the foundation models aren't just eating competition downstream, particularly in the application market, which frankly at this point is quite innovative, there's a lot going on, but why wouldn't these firms that have put, you know, billions of dollars on the line, trillions of dollars, why wouldn't they go forward and then acquire these downstream players? So I think without these rules of the road, we're going to have a rigged market.
I think the GDPR question is a important one, but as we brought up even in that context, you can structure laws so that they don't apply the same way to everyone. So the GDPR also did have carve-outs for firms under a certain size, and there are several other ways in which you can make sure that the burden is heaviest on the players that have not just the deepest pockets, but are also positioned in the supply chain at the point where they can actually make a difference.
What is your suggestion, or I guess your proposal, for maybe minimizing these monopolies, or at least trying to rein in some of these monopolies, without disrupting investment and therefore slowing progress?
I'm maybe not as worried about a little slowing down of progress as maybe others would be.
But what about China? But what about China? We have to beat China.
When the DeepSeek moment happened, let's just take that as one example, right? It felt very much like a seismic moment because it was like, "Okay, wait a minute, we've been told this story about bigger is better. We've been told that there's no one but our Silicon Valley tech giants that can lead, and that their way is the only way." And for a tiny blip in time, you know, everyone was like, "Wait a minute, what if all of our assumptions about this market aren't true?" And I think what it tells us that a market with coddled monopolies, a market which is sort of celebrating national champions at any cost, is one where likely the biggest losers will end up being the public, right? Because we aren't actually in a position where we're seeing free and fair competition. We don't know what innovation we're missing.
And certainly we know from the last 10 years that these companies have every incentive to interpret innovation as, you know, behavioral ads targeting. Like that's where we ended up with the last 10 years of the internet and we had kind of accept, "Well, I guess this is how the internet works, and this is what we have."
And so I think that like, now looking ahead, one of the big questions for us is how are we going to create, whether by using regulatory tools or by other means, a market that feels, and is in fact more dynamic, and where we don't have to take the word of a couple of handful of Silicon Valley companies as gospel, and we create structure so that there is, you know, more dynamism in the market and more room for disagreement. I think some of what we're seeing from the open source companies, and from that open source space, hints at that disruptive potential.
You say that like the people haven't chosen this and they're going to be the ones experiencing the worst downsides of this whole market. We have contributed to the demand, and inference is now kind of the bottleneck, because there is such a high demand. So should we be rationing this stuff? Like, should it be used to make AI slop on TikTok, as in, we are contributing to it, ultimately, like, we are part of this equation.
Yeah, I think you're asking two separate things, both of which are very important. The first question on like "Should we be using this for AI slop?" I think gets to this question of like, "How do we measure productivity when it comes to AI?" Like, what are productive uses for this technology? And to me as a policy person, I think it also raises the question of "Does the government, or does the public, have a say in kind of steering this industry towards more productive uses versus less?" And by productive, I mean not extractive, you know, trying to manipulate people on the back end, but more like, "How can we actually make ourselves work more efficiently, or make our healthcare system work better?"
So I think we need better metrics to judge productivity when it comes to AI that are more holistic, and try to, you know, have the public interest at the heart of them. And there are actually like quite interesting debates around this. But I think your other question on whether we should use this stuff less or-
The argument is that this huge infrastructure development is because the demand is so high, and it can't be met, and that five trillion needs to be spent to meet the demand. And so where is that demand coming from?
I think we need to look at where these demand projections really get material, and that's in, you know, the infrastructure build out that we're seeing. And I think that's one place where time and again, when you hear these earning reports, we're hearing that these firms, they are betting on a future that they can't see yet, almost by design. They're not just building to meet existing demand. Is there a compute scarcity in this current moment? Absolutely, yes. Is demand growing for AI? Absolutely, yes.
But I don't think the answer is to kind of responsibilize or put that back on individuals that are demanding these tools to say, "Now it's your job to curtail demand because this stuff is bad," right? Like I think these are macro assessments of whether a technology is net good or net bad for society, and what are the ways in which those that are responsible for protecting the public interest can better insulate the public from its harms. I would worry about a kind of paper straws and the climate type of responsibilizing of the individual actor or even the individual business in response to what are broader societal or environmental concerns.
Where do you think that people will draw the line when it comes to inviting AI into their personal lives?
If it became more apparent that AI was not anthropomorphized, if we were constantly reminded that this isn't just your friendly Claude, but a company with very particular profit incentives and an incentive potentially to mine your data in ways that enrich them, but maybe hurt you.
Yeah.
If there were ways to constantly remind folks that this is a company, not your friend, then I guess what I'm saying is maybe those lines would be drawn more clearly.
What's something that's universally accepted in your specific field that you actually disagree with?
I think maybe the idea that this, that large scale AI is inevitable. That all we can do is tweak around the edges or put regulation in place. I'd like to think of a paradigm in which we can hold onto the idea that innovation, world-changing innovation is still possible, but I'm not convinced, and I don't want to be convinced that this is what it looks like.
What makes you optimistic?
What makes me optimistic is that five years ago we were, you know, staring at a really, really bleak picture when it came to anything regulation-related. And everyone was like, "You're the bad guys. We love ChatGPT, why are you even here?" And now people are like, "I do love ChatGPT and I can still be deeply, deeply worried about this industry being unregulated." And I think that that is now the popular opinion, not anything like the unpopular one. So I'm amazed that this industry has done more for our kind of work than we maybe have.
I'm still stuck on that demand issue, because I know we, like the projections are crazy, but that's kind of what was done with electricity, the railways. It wasn't built out for the demand at the time. It was built out for the future, and look at us now. Not saying that's going to happen necessarily with AI, but it was necessary to take that risk.
Mm-hm.
I think it was fascinating. And the first person we've actually spoken to who wanted to talk about regulation, right? Where no one else has wanted to talk about regulation. But, you know, it's very complex, isn't it? I mean, you're looking at social media companies, how long have they been around, and is there any regulation on social media companies? There's talk of it, but that's about it.
Yeah, it's a slow drip.
I think so. And I think if you look at the balance of power right now between the billions being, you know, turned over by the frontier companies and regulatory efforts, I applaud her optimism.
I mean, the way that she was describing though, you know, the way that the power is being amassed by these select companies and this handful of companies that I feel like what she was describing sounds like are getting to a level of too big to fail, especially if our livelihoods are wrapped up in the success of these AI companies, that's a little bit scary, you know?
I think potentially I would argue even without AI, those same companies have been too big to fail, right? You know, your Microsoft, your Alphabet, your, I mean, what would happen to the American stock market, your Apples, if they failed? I mean, I think we-
So we're already there is what you're telling me.
I think we're long past that point.
Okay, great. All right, well, our thanks to Amba for the conversation today. And still, we would love to hear from you. Be sure to drop your comments in the blocks below, and be sure to like and subscribe while you're there so that you are all tuned up for next week's "Agents of Tech."