Will the U.S. LOSE the AI Race to China? – Helen Toner, ex OpenAI Board Member

Helen Toner, a former OpenAI board member and interim director of the Center for Security and Emerging Technology at Georgetown University, joins the hosts in person in Washington, D.C. to compare the American and Chinese approaches to AI. She argues there is no single race, sets frontier models against adoption and diffusion, and sets out what governments should be doing now in case very advanced AI arrives within 10 to 15 years.

Watch on YouTube

Transcript

The race for AI dominance continues to be neck and neck between the United States and China, but with wildly different approaches. China's bet on open source AI to win friends stands in stark contrast to the United States mission to take full control of Big Tech. So which strategy is the path to success? Our guest today has been to Beijing and back, and she's here in Washington with us to weigh in on who is in the lead and what that means for the future of AI.

Welcome to Agents of Tech coming to you from Washington, DC. I'm Autria Godfrey, and I'm delighted to be joined by my co-hosts Stephen Horn and Laila Rizvi. And guys, we have got a first today on Agents of Tech. Helen Toner will be joining us here in person. She's one of the most influential voices in global AI policy, and she'll be here to weigh in on where the race for AI dominance stands.

But first, I'd like to get started with you both. Let's talk about this split in strategy. Open source in China. Big Tech control here in the United States. Which way do you think sets the pace for innovation? Which one is more dangerous?

I guess what I'm interested in is one there's the narrative around, okay, the US is very dominant when it comes to these Big Tech companies in terms of all of the money that's being poured into it. And China is, you know, looking for this more global kumbaya kind of vibe.

Collaborative. Yeah.

Like AI for everyone and we'll share everything, it's decentralized and everyone can kind of pick it apart. But what's the reality because the US has Hugging Face, the US also has OpenAI's new open source models. So how do they actually compare to something like DeepSeek and DeepSeek's newest model, they tried to train it on Chinese chips, but it was delayed because they couldn't train it as well on Chinese chips compared to Nvidia chips. So what is the actual reality, not just the headline around it, is what I want to know. And I'm sure Helen will be able to tell us.

I think the reality is, is where both stand in terms of access to technologies. I mean, you know, the Chinese famously have not been able to manufacture chips for themselves that measure up to, you know, Nvidia's standard. And now we've got this whole trade negotiation over the quality of chip that goes there. And so, you know, what I'm interested in is kind of how the world order is turned on its head, you know, so it's China that supports sort of multilateralism. It's China that supports regulation. It's China that supports countries working together. And it's America that doesn't. And, you know, in my entire lifetime, which is getting on a little bit now, you know, it's been the other way around.

Yeah. And also, what has always led to American success is taking risks, moving fast and also having kind of a decentralized approach, which is now what the Chinese are doing, whereas now the US has all of this kind of corporate secret culture, all of this ethics talk, a lot of this regulation around, you know, what's Big Tech going to do, whereas China is just building these things and letting everyone else work on them.

Yeah, I think you've also got to follow the money, which is, you know, the US also has companies that are worth $500 billion and not so many of those in China, not in the field of AI anyway.

No, no, I agree with that. No.

Now, it is our pleasure to welcome the Interim Director for the Center for Security and Emerging Technology at Georgetown University, Helen Toner. Such a pleasure to have you with us today.

Thanks so much for having me.

Okay. Before we dive deep, I want you, if you can, to just kind of create a snapshot for us with who is winning the AI race right now, China or the United States.

Two things. One, there's not one race. So there's multiple things that they're competing over. Two, I think it makes a lot of sense to think in terms of competition, the same way that in markets we think of companies competing with each other. I think saying that there's a race actually builds in a bunch of assumptions that there is a finish line. The only thing that matters is who edges over the finish line first, which is not really, I think in most cases, depending on what kind of AI we're talking about, usually it's not quite right.

Helen, I want to talk about more open source strategies and the narrative around that. So the narrative from our perspective is very much that China is winning this open source race and the US isn't. And it's very closed in corporate secrecy. And there's not much sharing with the AI community. So what would you say is the actual reality of it? Because the US does have quite strong open source models in the form of what Hugging Face is putting out, and even GPT-OSS, which OpenAI has recently put out. So what is the reality of the situation?

Yeah, I would definitely say that at a vibes level, the vibes are the U.S. goes for closed and China is going for open, and we are kind of eating our lunch there. I think if you dig in a little more, it is true that the leading U.S. companies are the ones who are really putting out the most advanced models first, which is typically your OpenAI, Google, Anthropic, are kind of the top three companies. They have all gone for a closed first model. So they release their systems through an API or through a consumer app that you can go and use yourself, but they don't put out their models open source so that you can download the whole model.

And in contrast, it is true that many of the best Chinese companies, obviously famously DeepSeek, but also companies like Moonshot with their Kimi models, or Alibaba, a very big Chinese company, has their Qwen series of models, very popular. They're all open source. So there's definitely a lot of truth to the US being kind of more closed, the Chinese leading companies being more open.

But as you say, there's a little bit of subtlety to that as well where the US companies, so, Meta has been very well known for pursuing an open first strategy. And in fact, one of the things that made DeepSeek so impressive in January was that they were kind of beating Meta to the punch. They weren't putting out the world's best AI model, but it was the world's best open model, which used to be Meta's thing. Yeah. And as you said, you know, OpenAI has clearly felt the pressure to start competing in the open source world as well. And they put out their GPT-OSS, a pair of models, and Google, actually, Google often has a lot of really good AI models available that just aren't quite that widely known. So Google's Gemma series is also open source, and they're a little bit smaller, a little bit less capable than their kind of very leading Gemini AI models. But they've been doing that for quite a while, and those models are popular as well.

So when you say that there are different races, what do you mean by that?

I think the clearest one that's really like a race where you're really trying to be like the company at the front is in what gets called sort of frontier AI. So this is really pushing the most advanced systems. This is your GPT-5 or your Gemini 2.5, moving towards artificial general intelligence, AGI, really trying to have the most advanced systems. And that is one where I think this open, closed question is particularly contentious, which is should you have those most advanced models being open sourced immediately, or does it actually make more sense to be a little bit more cautious and say like, well, if we don't quite understand these systems, we don't know what they can and can't do. We don't know what risks they do and don't pose. So that's kind of the tension there.

But there's a whole other competition or multiple other competitions really, more on the diffusion and adoption side of things. So less which company has the very, very best model this month versus last month versus next month and more who is actually using these models? And from a, you know, from a technical perspective, a huge thing there is you don't always want the most advanced model because the most advanced models are often very large, they're very slow, they're very expensive to run. So I think China is really trying to compete in that diffusion and adoption race. And the fact that they're putting out open source models is really helpful for them in getting that adoption.

So how would you characterize the Chinese policy?

So the goal of the Chinese government, I would say, is to be using AI to bolster their national competitiveness broadly, so they know that they're in a demographic crunch. So they're running out of working age population. Partly the one child policy, but also just general economic development. They really need to keep their economic numbers going up, their productivity going up. AI is a big part of their story for how to do that.

What does that look like? From a government policy level, the way China often works is that they will kind of set central goals or targets or kind of priorities, and then they'll let a lot of other actors, whether it's provincial governments, local governments, companies, state owned enterprises, private companies as well, they let them kind of figure out, how do we do this? So what you've seen from the central government is mostly just AI, huge priority. We want it in the economy, we want it in the military. We want to be leading, we want it around the world, but not a lot of sort of step by step, detailed planning of how to do that. Instead, they kind of leave that to a lot of other actors.

I hear that their policy is two pages long.

Yeah. Depends which policy. Oh, yeah. I could pull out dozens. All right. Okay. Many of them are longer.

Because I've also kind of heard that, I guess in line with what you're saying, that their concerns are manufacturing. Like, how are they going to use AI as a tool to improve that, as opposed to this focus on achieving superintelligence and AGI. Whereas that's literally all I hear of on the American side, which is who has AGI first, is ChatGPT-5 going to be AGI?

That was going to be my next question. Okay. I'll let you expand on that. I know you mentioned earlier that there really isn't a defined finish line, but would AGI be the closest thing to one? And if that's the case, is the United States leading when it comes to some of these more advanced systems?

Okay. Yeah. So just to take a step back to Laila's question for one second, which is, I do think something China has done really well at articulating is what they call in one of their many policy documents, the AI Plus strategy, and AI Plus means AI plus something. So AI plus healthcare, AI plus finance, AI plus manufacturing. And that is something they've really clearly messaged is a priority. And that has really incentivized their whole ecosystem, both state based and the more private sector actors, to be not just saying, you know, here we are half an inch closer to superintelligence, but here, here is a useful application.

I do think when it comes to that frontier race or the race to AGI, the race to superintelligence, I do think the US continues to have an edge there. It's a little difficult, you know, how do you count what that edge is? I think the closest you can say, the smallest you would say would be three months, which was the difference between OpenAI releasing their o1 and DeepSeek having their R1 model. But I think you could also say it's more like six months. It's more like a year, depending on kind of what you're looking at.

But I don't think it's clear that that's a race with a clear finish line. Like there's really not a lot of agreement on what would artificial general intelligence look like? My take tends to be, I think we should expect the systems to keep getting more capable in lots of ways, but I don't think it's clear that there's going to be some milestone, some clear endpoint where everyone's like, wow, now we got it. Now, we're there. And also now it doesn't matter that someone else is a week behind us because we got there first. You know, we cross the finish line, we get the trophy. It might play out that way.

One way that I think the main way that there might be kind of a meaningful finish line is one of the kind of more sort of sci fi sounding possibilities, but that is taken really seriously by serious technical experts, is if you have AI systems starting to improve themselves or improve the next generation of AI systems. And so that's the place where you get, could there be some like, you know, first mover advantage where once you get to that point where you're able to improve your own systems fast enough, you kind of take off into a feedback loop. And if someone is a week behind you, they're left in the dust because they didn't get there first. But whether that will happen is very unclear. And so that takes a certain set of assumptions to say there's going be that end point.

Yeah, do you think, I mean, do you think that this kind of, let's say, the lukewarm reception of ChatGPT-5 has changed the goals here a little bit, because are we not slightly further away from AGI, and therefore, is there not maybe now more of a premium on what, let's call it productivity improvements using AI that we have to, to improve a whole kind of bunch of things, from health care through to manufacturing through to everything else.

So what I'm seeing in the ecosystem of people who are trying to figure out when will we have AGI, what does that mean, is I think the releases we've seen this year, especially GPT-5, but also others, is taking a little bit of the edge off some of the most aggressive predictions. So people saying we're going to have AGI in 2027 or 2028, we're going to have superintelligence by 2030. I think people are not saying that's off the table, but even people who thought that was possible are now kind of easing back and saying maybe not quite that soon, but I think it's not quite right, or I think it's been a little bit exaggerated, sort of how much of a letdown GPT-5 should be seen as.

I agree.

Because I think if you look at, I mean, one thing that's going on is, if you've been paying attention to the AI space since ChatGPT, then you saw this sudden leap, you were like, oh my goodness, ChatGPT is so good. And you haven't had another moment since then. But if you've been paying attention since kind of 2019 or 2016 or so, 2014-2016 is when I got interested in this space, then I think what we've seen over and over again is just incremental, incremental, incremental, incremental, incremental. And over time you look back and you're like, well, that's actually been a good chunk of progress.

What I would say, though, it's hard to equate incremental improvement with a $500 billion valuation.

There are people who take AI 2027 as like a Bible in terms of timelines as well. So I guess it's whether you're saying, how realistic is that? And where do you stand on that? Because it terrifies some people. And now people are like, well, that's a load of rubbish because GPT-5 isn't that good?

I mean, my basic perspective on this, I wrote something earlier this year called Long Timelines to AGI Have Gotten Crazy Short. Yeah. Meaning like when I got into the space, it was seen as if you wanted to say we might have AGI in 10 years. 15 years, 20 years. That was like what? Like, really? You think that soon, like, that's crazy. That's a huge deal if true. You know, big if true, as they say. And over the past few years, that number has come down and down and down. So now it really only counts as like a short timeline if you're saying 2027 years, saying like two years, three years.

But I think if you look at a lot of people in this space and you went to a wide range of experts and said, look, is it plausible that we'll have, whatever you want to give the definition, AGI, very advanced systems, human level, you know, whatever. Is it possible that we'll have these very advanced systems in 10 years, in 15 years, a lot of people who seem to disagree, but I think a lot of them would say, yeah, look, not for sure, but certainly likely enough that we should be preparing for that. We should take that seriously as a possibility. And that's crazy. Like if we're going to have, if we're going to have computers that are that smart. Yeah. That would be a huge deal. That would need a lot of preparation, a lot of societal adaptation.

So I think sometimes often in AI, there can be these binary, sort of polarized discussions of like AGI tomorrow versus AGI is a hoax and it's never going to come. And it's all hype. Yeah. And I think the reality is almost always in the middle. Yeah. In this case I think the middle is like a pretty dramatic, like, oh my goodness. We should be thinking about this. Yeah.

Can I just take you back right to the beginning of the conversation about the different roles China and the US, and maybe just the roles they play in the world in terms when it comes to AI. Most of my adult life, you know, I'm used to America being behind sort of multilateral organizations, being sort of, you know, about regulation, about being the sort of adult in the room, if you like, and China, maybe less so. And as it's reported in the press, it's a bit like the other way around at the moment. So A, is that right? And B, you know, why are we where we are now, do you think?

At a rhetorical level, that's absolutely right. So a month ago I was in Shanghai and it was the same week that the US AI Action Plan was released, and that action plan was all focused around US leadership, US dominance. Yeah. The US winning at AI. And this Chinese conference, which was sponsored or sort of closely connected to their Ministry of Foreign Affairs, so very sort of government and government directed, the tagline was Global Solidarity in the AI Era. And so that is like a pretty striking difference where the US is saying we're in it for us. We need to win, we need to dominate. And China saying we're here for the world, you know, developing countries, global majority, like we're here for you. We want to help you. That's the rhetoric.

I think there is some reality in that rhetoric, obviously. Chinese companies are, China has a long running Belt and Road Initiative. Digital Silk Road is another component of that, going out to try and sell Chinese tech. This goes back, you know, 5 or 10 years, and AI slots into that really nicely, going out and trying to help other countries use AI. One of the projects that was on display at the conference was a huge transmission project in Brazil trying to transmit huge amounts of energy from their kind of renewable friendly north into the sort of heavily populated areas further south, and how they're using AI for that. So China really wants to do that. They really want to lead in that. They really want to be seen as a good partner.

At the same time, it's worth mentioning, you know, ChatGPT has 700 million users around the world. So that is real adoption. That is real diffusion. That is real international usage. And likewise, you know, Google, Meta, like these companies have a lot of global penetration. Other US companies that go under the radar more, your Amazons, they're building Amazon Web Services, it's huge. They're building AI tools of all kinds into Amazon Web Services. So, I think certainly China is winning at the rhetorical level. I think they're a really serious competitor in reality, but I don't think the playing field is as skewed as it's sometimes presented as.

Just playing Devil's advocate, though, is there a little bit of a danger that's associated with making everything open source and the concern about some of their technology getting in the hands of bad actors?

Yes. I think it is genuinely confusing to try and think through the pros and cons of making things open source. At a basic level, I mean, I think the starting point should be open sourcing things is great because it allows much more usage and allows much more testing and allows much more probing. It makes it much more accessible and much more, you know, of an even playing field. So I think starting point, open sourcing is great.

I do think coming back to this specific slice of the AI world that is frontier AI systems, the very most advanced systems, we really don't understand how those systems work. We don't really understand how any of deep learning systems work kind of in detail. Usually when these new systems are released, so a GPT-5 or a DeepSeek-R1 or whatever it is, we don't have a clear understanding of the capabilities of those models because often something will be tested, it'll be released. And then 6 months later or 12 months later, we'll figure out, oh, if we actually prompt it in this way, we put it into this kind of agentic software setup, it can do things that we didn't realize that it could do.

So the place where I come down on this is, I think, specifically for those most advanced models, the frontier models, it does make sense to have a little bit of delay, a little bit of a lag, a little bit of time for us to try and understand what are the capabilities of these systems, what are the confidence bounds. So like, we're pretty sure it can only get, you know, 80% on this particular task. But, you know, right now maybe it actually could get up to 99%, we're not sure. Over time can we kind of narrow those confidence bounds, say we think it's 80 but maximum 85. And that's fine. So we're going to open source it. Does that make sense?

Yeah, I think so.

I think having that delay does make sense. And a comparable version to this is you keep your biggest, most advanced model closed and you pull a Google and you, or an OpenAI now, and you release a smaller, somewhat less capable, but also more useful because it doesn't require the huge infrastructure set up. And I think that is a pretty good compromise.

To that point, though, going back to, you know, now OpenAI has their open model. Do you think that they're doing that so that the US doesn't look quite so bad, that all of their models are closed? And they're called OpenAI but they're not open. Like we're really not, you know, so cloak and dagger about it. Yeah. This is global optics here. Right.

For sure. I mean I think there's multiple motivations. Yeah, okay. That's absolutely a part of it. Okay.

And also, I mean, because it's often described as like an arms race between the US and China. But when it actually comes down to compute, because obviously you've got export controls over chips and then they're using them and they're not using them. And it's a back and forth, like, is China really competing when it comes to compute? Like what is the reality?

I do think that access to compute and the sort of quantity and concentration of compute is going to be a meaningful barrier for China at the frontier. So at trying to really push the limits of what the systems can do. Yeah. What is it going to mean for adoption and usage and kind of getting real world value out of the technology? I think that remains to be seen and much more. So it really depends on are we trying to talk about military advantage? Are we trying to talk about overall economic productivity, are we talking about GDP effects? Yeah. Something else.

Because if you're, if you're treading this line in this route of internally developing AIs, which are really good machine learning engineers, particularly given how much of a kind of race there is to poach amazing talent. So if you can just copy that or automate it, then you've got this recursive self-improvement of the system. So, and compute would be essential to that strategic advantage. Right. So I would say that if I was an American AI company, I would just try and build the ultimate machine learning engineer and just deploy thousands of them simultaneously.

You actually have good strategy.

Because that's what they're all doing. Yeah. But it makes it difficult for China to compete on that. Right. Because they can just be choked of that compute from a company like Nvidia, which they're kind of dependent on.

Yeah. I mean, is Nvidia willing to choke them off? It doesn't seem so right now. Yeah. That's true.

I do think that the underlying point of, if it is actually possible to automate that machine learning engineering work, machine learning science work that happens inside these companies. If it's possible to automate that, then access to computing power will become a huge bottleneck. Yeah. And that will probably be an advantage for the US. But, and I mean, that's something that all of the big companies are trying really hard to automate. So I wouldn't bet on them failing.

Okay. Can I bring you back to one point that you made, which is really interesting, when you said that whether it's 10 years away or 15 years away, who knows? But it's not that long and we should be thinking about what the implications for that are. So what are some of the implications we should be thinking through?

I think we don't know. So I think what we should, the kind of planning that we should be doing is trying to think through what those implications could be. I think there's the kind of steps that in my mind would make sense, if we take that seriously, are things like requiring much more transparency from the companies and more disclosure from inside the companies, what they're doing, what they're building, what tests they're running, more access for third party independent auditors or other kind of groups to come in and say, look, the company said that it performed these tests and we can confirm that they did perform this test, or the company says that this model cannot do this. We went in and we confirmed that, you know, what they're saying is right.

I think we need massive AI literacy campaigns. So, for the broad public, for people in government, we need much better hiring of tech talent into government. That's something that we at CSET have worked on a lot. I think we need to be starting to have some of the international conversations that sort of got started in the post ChatGPT moment, and now are kind of faltering.

Oh, another huge category is investing in kind of the measurement science of AI, which sounds really nerdy, but is basically like right now so much investment is going into making AI systems that can do more stuff. So making them more capable, right? And so little, relatively speaking, is going into understanding how do they work, what is going on inside, why do they fail in these weird ways when they fail? How are they reaching these great results when they do reach great results? That kind of more scientific work is really neglected, I think. And if we have two years, that's probably not enough time to make progress on it. If we have 10 to 15, it really could be. So I think there's a lot that we can do to try and give ourselves more clarity, put ourselves on a firmer footing so that if there are really big transformative changes ahead, then we're in a better place.

So one of the things that I've always been really fascinated with, I come from the North of England. I've always been really fascinated with the parallels between what's going on with AI and the first industrial revolution in 1760, and I would argue it took society 100 years to catch up, you know, to have schools, to have local authorities, to actually have any meaningful regulation of the new towns and industries that were created. And obviously we haven't got 100 years to catch up, have we, so it is quite important we get on with that.

Yep. Yeah. No, I totally agree. I think a lot of the stuff here, a lot of the potential risks, a lot of the potential disruptions, if we have 100 years, I just think we'll do fine. There'll be some eggs broken along the way. It could be a rough ride, but I think we'll come out okay on the other side. And the challenge is really if it's crammed into a much shorter period of time, and are we equipped to handle that?

Like how do you feel about where people are at the moment? Do you think that they're underestimating the risk, or is it very much just a tool, yeah, we're going to use it. It's going to increase our productivity. But it's not that bad. Because I feel like you've got two quite stark camps now. Whereas maybe a couple of months ago everyone was like, you know, fasten your seatbelt, you're in for a ride. This is going to be crazy. You're going to lose your entire livelihood. Whereas now everyone's like, ha, it's not so much the case. Where do you kind of sit on it?

Yeah. I mean, I think when we say where people are at, different people are at such different places. Yeah. So it's hard to say. I think it's not right to think that the sky is falling right now, and that disaster, these systems are basically, you know, smarter than humans already. Yeah. I mean, that's clearly not right. I think a position that worries me a lot is in some sort of more like left progressive climate sort of spaces. It's getting really ingrained to just think that AI is just bad and also is useless, like it's bad for the world, and also it's all hype, and useless, and hot air. And I think that is just not a position that's going to hold up well. I think that's not going to let people learn about the technology and understand how it is useful and how it isn't useful and what risks it does and doesn't pose. So I do worry about that position.

I also think there can sometimes be a bit of a dynamic where intuitively, a lot of people, like if you just talk to regular folks, they're like, man, building smarter than human computers, that sounds scary. Like, do we want to do that? But this seems kind of scary. And then there can be sort of a reaction from more of a sort of elite intelligentsia sort of thing that's a little bit pooh poohing that of like, no, no, no, no, like that's just silly science fiction. Yeah. But then you go talk to like the luminaries in the field. They're like, no, no, like building smarter than human computers. Like, oh, is that a good idea? We're really not sure.

Because like Geoff Hinton recently was like, the only way we can get by is if we build AI which is maternal, right? Which is like, because the only instance for his logic is in which a weaker or less intelligent being has control over a more intelligent one is a child and mother. So that said, like, that's apparently his solution.

He's been peddling that for a long time. That like this is awful. This is bad. Like we need to get a handle on this. So yeah. I think it's important that people feel like they have permission to be concerned. Yeah. Which is not to say everyone should be freaking out. Yeah. I think there's a middle ground there. Sure. Yeah.

Well, so kind of to that point, and you mentioned this earlier about, you know, the safety standards and things that are going to have to be implemented as we're kind of like learning this globally, you know, along the way, we know that China has been actively involved in creating some of these AI standards and working with the UN. Do you feel that if the United States continues to prioritize the closed source models and keep all of their, you know, proprietary technology to themselves, then we lose a seat at the table when it comes to implementing standards and putting in place safety protocols?

I think we might. I think it's a little unclear how exactly standards are going to be set in the AI space. And I think often, I think certainly open sourcing technology is really helpful for setting standards because everyone can adopt it, everyone can use it. But you can also become a de facto standard just by having really excellent technology that everyone wants to use. Yeah. And also, you could be maybe not open sourcing your models, but for example, Anthropic, I think one of the clearest sort of emerging standards in the frontier AI large language model space is something called Model Context Protocol, which is released by Anthropic. So they open source this protocol, this sort of outline of how you should structure information that's being fed to an AI system. They can open source that without open sourcing the model itself.

I do think in the kind of standards bodies in the UN, I sometimes think that the US government has a little bit of leftover trauma from the 5G days. I don't know how much any kind of focus on 5G, but that was a space, like telecommunication networks. You really need international standards bodies saying this is the technical protocol we're using to set up these telecommunications networks. There's going to be one. It's going to apply all across the world. It's going to get decided in these standards bodies. You better, like, bring your A-game. Right. I don't think that's how it's going to work for AI. I don't think the standards are comparable. I think it'll be much more bottom up. I think there'll be different kinds of standards for different kinds of things. I think they'll shift over time. So I don't know. I'm in favor of the US playing a solid role in international standards organizations, but I think sometimes that gets kind of a little bit overplayed as an angle here.

Yeah. Because the other thing is that we talk about quite frequently. Compared to the US and China, does anyone else really have a chance? Because especially with the amount of money and the head start that the US in particular has, but obviously China has the kind of might to challenge it. But I mean, we're not doing much in the UK, so can other countries really compete?

I think compete at what is the question. Okay. Yeah. In the frontier space it's going to be tough. I do think it's going to be tough. But there's a lot of room for using AI in practice. Yeah. Making useful things from it. Getting value out of it. Yeah. Of course. Building more dedicated products. And I think that that is something where there's space for plenty of other countries as well.

If you could kind of use a crystal ball for a moment here and make some predictions for us, if we just look down the road just five years. So not, not too terribly long when you're talking about technology, what do you think the AI landscape looks like with regard to the American version and the path that we're on versus the Chinese version?

Do you mean like open and closed?

Yeah, yeah, yeah.

So I don't know is the short answer. I think we're likely to see, I think the motivation to release models openly that are a bit behind the frontier is really strong. I think that the set of incentives there, the kind of ecosystem that that creates is pretty clear. So I think we'll see sort of a continuous, sort of like a bow wave of like the, the leading ones will probably be closed. And then following that, a chunk of many competing open models. I think it's possible that it will become more, more common for the real leading frontier models to be released openly. But I wouldn't bet on it.

In particular, we haven't kind of gone into much depth on this yet, but because of some of the risks that they do pose. One that I'm particularly, you know, concerned about is the sort of cyber security risks. So do you kind of democratize extremely high level hacking abilities, meaning anyone can carry out much more sophisticated cyber attacks than has been possible in the past? Another one people talk about a lot is bio weapons. And, you know, will you enable the creation of viruses that, you know, usually previously you would have needed a PhD for, but now you can do it from your parents' kitchen. So in principle the AI could help both at offense and defense. But in practice, if there's a kind of an adoption lag, do you have this kind of vulnerable period for critical infrastructure? That is something I'm a little concerned about.

Do you think there's any tie in between when we, if and when we get to a quantum computer, and the advancements in AI?

I think quantum computing is definitely something I'm very far from an expert in. I think where, as I understand it, where that might be most helpful is in simulating natural processes. So simulating chemical reaction and simulating biological reactions, that's something that is a real challenge for a classical computer. It's just really, really difficult to have good simulations. And for AI, the way this ties into AI is if you can simulate something, then you can train your AI to predict it.

And right now, people would really love to be simulating, for example, people talk about having a virtual cell. So could you do kind of biological experiments on a cell that is purely virtual? You know, you have in vivo, in vitro, this would be in silico experiments. And people are really optimistic about this. They're trying to build it. They're trying to make it work. I tend to think that's going to be really, really hard. But I think that's the kind of thing that if you had really actually functional, relatively large quantum computers, they might be able to get a handle on that.

When you were talking, obviously, about the risks of open sourcing very powerful models, how concerned are the Chinese about that? Because obviously, if that whole strategy is open source.

It does look like that's a difference between the US government and the Chinese government right now. So yeah, certainly the US government in kind of 2023, I would say, had a big freakout moment about AI and bioweapons. Yeah. And AI biosecurity. Yeah. That I think was at that time in the Biden administration and also in Congress was a really serious concern. I don't think the Chinese government, it's always hard to be sure, because you get such limited information. Yeah. But from everything I've seen, I don't think they have had a similar freak out moment. Yeah. So some people in the space kind of predict that at some point they're going to wake up to that and say, oh, hang on. Oh no. We've been, we've been promoting this open source strategy and that's going to be terrible. We'll see. I think it depends a lot on how the technology evolves and also whether these impacts start to show up in practice or not.

Thank you so much. Thank you so much for coming all the way in. And being our first in person guest.

Yes. My pleasure.

Thank you.

What an incredible first guest to be here in studio with us. I thought she was absolutely fascinating. What did you think?

What I thought was really interesting was how she was talking about the finish lines. And the fact that the finish lines might not be the same. We don't know what the finish lines necessarily are and how they're going to play out. And that's what the future kind of depends on, that. If the finish line is AGI, then okay, maybe the US kind of is winning that race. But if the finish line is global productivity and kind of global economies, and AGI really isn't in the realms of that, then maybe China will win in that game in terms of just getting global, kind of in terms of their relationships. And the US will isolate itself.

Stephen, I actually think that's kind of one of my takeaways as well. And maybe we're not on the same track. What do you think?

Well, I think it's interesting because I think there's clearly a lot of gains to be made in productivity and not just in business terms, but in health care and all of these things that, but I was really struck by what Helen did actually say, that she thought maybe 15 years to AGI. And that's not a lot of time. You know, we're thinking 15 years is a lot of time, but it's not really. And the massive societal impacts that it's going to have that we're really not thinking through. We don't have the structures in place to answer those questions. So that's what really grabbed me with the conversation. But I agree with you. It's a little bit nuanced, more nuanced than some of the conversations we've had.

Yeah. I thought the remarks that she made on cybersecurity. Right. Yeah. Potential for, you know, cyber attacks. I thought that was also interesting. And I'm glad to hear that she shares some of those concerns and even has some of those concerns. And I hope that other people in her space are monitoring that as well.

Yeah. And I think what it shed light on is the fact that you're trying to choose between these two kind of options. I guess the lesser of two evils. Do we stop this rapid progress and then let China, who could potentially do worse and have this kind of military advantage over us, get ahead of us if we slow down, or do we carry on going really fast? And we have these safety concerns where these frontier models are getting in the hands of bad actors, or they're going out of control and they become autonomous and they're, you know, hiding from us and tricking us and things like that.

Right. But that's not really a real, you know, either or, is it? Because, you know, the money that is invested, you know, $1 trillion a year in AI among the main players, the United States, nobody's going to slow down. So that rate of acceleration is going to continue. But what we need is we need government.

Yeah. But you would slow down. If someone, like if you were told that you might get AGI which could fool the entire world and kill you, like I said. Yeah, it sounds ridiculous. It sounds like a movie. Yes, I get it's $500 billion, but I don't know. Rockefeller was investing $500 billion at the time with railways. Like, yeah, it's a lot of money, but there's been loads of money at lots of different points in history.

He didn't slow down.

He didn't slow down. I think that's true.

But she pointed out that, you know, like with OpenAI putting out their open source model, that that's not their best foot forward, essentially. Yeah. Here's some breadcrumbs so that the United States doesn't look so greedy. Yeah. And then they're holding on to, you know.

But I guess my point is not really a criticism at all of the private sector companies, which are doing what private sector companies do. It's just understanding that there is a role for government, and government with a big G and a little g, and Helen's made a great point as well, that right now government is not really equipped to do that role. The people who we need in government to be able to make the regulations to keep an eye on this don't have those skills at the moment, and 15 years is not a lot of time to tool up to make sure that, you know, we have the right safeguards in place. I'd feel better if we had the right safeguards in place.

I think we need to at least start trying to figure out what regulation we need. Yeah. I think at the moment there is no consensus on what regulation is needed.

All right. Well, thank you both so much. Our huge thanks to Helen Toner. From open source in China to Big Tech control in the United States, we also want to hear from you. Be sure to weigh in on the discussion in the comments below. Which path do you think will set the pace for innovation, and which one is a little bit more dangerous? Leave your thoughts below and then remember to like and subscribe while you're at it. Thanks for joining us here on Agents of Tech today. Until next time, have a good one.