유엔의 글로벌 AI 거버넌스: 앞으로의 도전과 기회

### 글로벌 AI 거버넌스의 시급한 필요성: 앞으로의 도전과 기회
* 🌍 거대 기술 기업의 영향력이 점점 더 커지는 세상에서 지정학 전문가인 Ian Bremer는 **인공지능(AI)이 교전, 경제, 심지어 전쟁의 규칙을 좌우하는 '테크노폴'**에 대해 설명합니다.
* 📅 유엔은 **2024년 9월 19일에 발표한 글로벌 AI 거버넌스를 위한 대담한 계획**을 통해 원자력 및 항공을 관리하는 것과 유사한 국가를 위한 가드레일을 구축하는 것을 목표로 하고 있습니다.
* 🔍 브레머는 AI와 관련된 기회와 위험을 정의할 필요성을 강조하며, **AI가 개인의 삶과 산업을 변화시킬 수 있지만 방치할 경우 상당한 위험**을 초래할 수 있다고 강조합니다.
* ⚖ 그는 현재의 자본주의 모델은 기술 기업이 AI를 통해 이익을 얻는 동시에 비용과 위험을 외부화할 수 있다고 경고하며 **수익과 손실 모두에 대한 책임을 보장하는 거버넌스 모델의 필요성**을 강조합니다.
* 💡 이 보고서는 특히 AI 개발에 있어 상당한 장벽에 직면해 있는 글로벌 남부를 위해 기술에 대한 공평한 접근을 보장하기 위해 **AI 역량 개발 네트워크와 AI를 위한 글로벌 펀드**를 설립할 것을 제안합니다.
* 📈 브레머는 **AI의 의미 있는 개방성은 소수 기업의 독점적 지배를 막고 인류 전체에 이익이 되는 다양한 애플리케이션**을 촉진하는 데 매우 중요하다고 주장합니다.
* 🤝 AI 기술이 계속 발전함에 따라 그는 AI 배포의 복잡성을 효과적으로 해결하기 위해 **표준과 거버넌스 프레임워크를 정의하는 데 있어 국제적 협력의 중요성**을 강조합니다.
* 🔗 궁극적으로 AI 거버넌스의 미래는 의미 있는 대화와 행동에 참여하는 우리 모두의 집단적 능력에 달려 있으며, **AI가 단순히 소수를 풍요롭게 하는 것이 아니라 인류를 위해 봉사할 수 있도록 보장**해야 합니다.

AI 거버넌스는 기술이 기업의 이익뿐 아니라 인류의 공동 목표에 기여할 수 있도록 글로벌 협력을 요구합니다.

Imagine a world controlled not by nations but by a handful of tech giants driven primarily by profits holding the reins of artificial intelligence, a technopol world as geopolitical expert Ian Bremer calls it, where algorithms shape our lives, our economies, and even our wars. Now, this isn't some sci-fi movie; it's the future we're barreling towards right now, and the question is, who gets to write the rules? Well, now the UN has stepped into the ring with a bold plan for global AI governance. They've released this plan on September 19th, 2024, and we're giving you the scoop right here. I had a chance to read an advanced copy of the report, and it gave me a lot to think about. So, this plan is a set of guardrails the UN wants to put in place for all sovereign nations to adhere to, just like we do around nuclear power or aviation. Only this time, the stakes are even trickier because it's AI, right? It's a domain tightly controlled by tech giants; it's an ever-evolving

technology that even the most seasoned researchers are still trying to understand. It's a tool that has the potential to solve both humanity's biggest challenges and unleash worldwide chaos. In other words, do global rules of conduct for such a powerful young technology ruled by tech alliances even apply? And is the UN, an almost 80-year-old bureaucracy consisting of sovereign nations, the right institution for the job? I'm Belaval Sido, and this is the TEDi Show, where we figure out how to live and thrive in a world where AI is changing everything. Today, we're joined by Ian Bremer, an American political scientist and one of the co-authors of this report. He's not just a policy wonk but someone who's in the room with world leaders, feeling the pulse of this tech revolution. He's explored the very question of governance in the digital age and its shift in power from sovereignties to Big Tech. He believes the UN is the only international body in a position to hammer out a viable global

consensus for this world-changing technology, and y'all are going to want to buckle up for this one because this is a ride into the future we're all building, whether we're ready or not. Hey Ian, uh, welcome to the show. Thanks, ball. Looking forward to it. Cool. So this report comes at a pivotal time. AI is obviously no longer this futuristic concept that we hear about in movies. It is certainly in the Zeitgeist, but I have to ask what prompted the UN to take the step towards Global AI governance at this specific Moment In Time. Been talking with the Secretary General about this issue for more than 5 years now. He recognized when he became Secretary General, Antonio Guterres, that there were two Global issues where governance was wholly inadequate, that he needed to make utter priorities of his time. One, being climate, where it's happening, it's just happening way too slow, and with far too Limited resources for the majority of people on the planet. And second, disruptive Technologies, notably

artificial intelligence that was coming way too fast, and there was a complete absence of governance. The reason it took so long for it to happen was, in part, because the UN is a bureaucratic creature, and it takes a while to get things set up. But secondly, because they didn't have a senior Envoy for geotechnology in place. But having said that, this has always been a top priority for the Secretary General, and the fact that you then had billions of dollars being poured into this technology, with not just enormous hype but real excitement around real use cases, changing the way humans interact with each other, changing the way we innovate, changing the way society and the global economy works, meant that an absence of global governance was going to leave a lot on the table and was also likely to get us in real trouble. So the urgency has clearly stepped up in the past, I'm going to say, 18 months, and in that regard, I think that this report is landing at a

really critical time. Yeah, one might even say it's coming at just the right time because you're totally right, especially let's call it the chat GPT moment or the large language model moment, where certainly ML has been on a steady trajectory, but really, you know, the fact that the average person in the world can now start experiencing this technology seems like a more recent phenomenon. From machines sort of that can just understand to machines that can create. And so along with that sort of this bucket of technologies that requires governance, you know, the report points out that a shared understanding of the opportunities and risks is key. Um, if you had to distill down for the audience just a couple of the key points um, in terms of what the opportunities and risks are when it comes to this technology, what would they be? Well, the opportunities uh, you mentioned that the average person can now see, can now use artificial intelligence in a way that it affects their daily lives. They will engage with chatbots, they will use

creative AI tools. They'll create art with it. They'll create videos with it. They'll edit with it. They will brainstorm with it. Anyone that has access to a smartphone or a computer now has access to AI in at least a limited way. That is transformative as a tool. It's a helpmate, and it's about to become vastly more than that. In some cases, it's a friend, it's a counselor, it can be an advisor. Those are the opportunities for individual human beings. But then you have the industrial, the corporate, the enterprise use cases. You get innovation, you get new vaccines, you get new batteries, you get more efficient ways of stacking traffic, you get all that kind of thing. And I mean, I've spoken with so many CEOs in global pharma, energy, retail, transport who are talking to me today, this week, this month about the billions of dollars that they are already saving from deploying AI in their industrial use cases. So the opportunities, in my view, this is not

a technology that is being overly hyped in my view. These are technologies that are still radically underappreciated because most people don't yet know or see all the ways that they are about to be deployed and are already being deployed. So, that's the first point. The second point is on the risk side, is that all of the companies, and they're mostly companies, mostly not governments, that are driving the investment in AI are capitalists when it comes to their profits, but they are socialists when it comes to any losses or costs that come from their products. And I don't know about you, but I would like them to be capitalists for both profits and losses, right? That's how we got ourselves in trouble on climate change. We had large numbers of part of industry and government, governments aligned with that industry, that was very happy to be capitalist in making lots of money from exploiting the natural wealth that the planet exists in, and were very happy to push the losses off to be

everyone's problem, which means the global South, your children, you know, the poor, and that's not capitalism. That is rapacious oligarchy. So, what we need is to ensure that the negative externalities that come from deploying AI, like any jobs that are lost, like skill sets that need to be redeployed, like disinformation and the use of AI by malevolent actors or tinkerers in ways that are dangerous for our economy or national security, and also to make sure that the opportunities generated by AI are not simply taken advantage of by small numbers of comparatively wealthy people who have the market serve them most effectively because they have the deep pockets. Those are and those risks will grow if they are not addressed, and they will not be addressed if the marketplace is allowed to capture the governance model. Remember, a functioning free market economy is one where corporations compete against each other in a level playing field with governments and regulators that act as independent and fair arbiters, and that

is not the way the American economic system runs, and that's not the way the global economic system runs. And that's particularly dangerous when you're deploying new technologies where there's an absence, a complete absence of regulation. That is, I think, the context where what the United Nations is presently trying to do is operating it. You know, I think you very nicely outlined why this is sort of a double-edged sword, right? You're making a brilliant point, which is that we're hearing about these large language models that are largely trained off of publicly available data, but all the latent value that is sitting in all these enterprises and even governments across the world. I mean, we haven't yet begun to see the economic impact that's going to have. At the same time, yes, this technology could absolutely obliterate the world of bits, so to speak, and the climate analogy is a very interesting one too. I want to do two things. I want to talk about the players here, and then I want to talk about incentives.

you've stated in the past that we're headed towards this world that you call a technopolar world, right, where these large tech companies are effectively sovereign in this digital world that we're talking about. Yet, it's important for incumbent sovereign powers to govern both of these digital and physical worlds that we spend so much time in and that are getting more and more interconnected. Could you break down the shift in sort of power dynamics that is taking place and how AI plays into all of it? Sure, um, I mean, when you think about artificial intelligence, first of all, um, the bots and the models that are being created that we're all using, they are being developed largely, overwhelmingly, by private sector corporations who have business models and respond to market forces, and they are primarily incented by profitability and to be able to outcompete other companies in this space. That will determine what they create, um, unless, uh, and until, uh, they are incented in other non-market ways, uh, by governments, um, and by other actors, and

overwhelmingly in the first few years, couple years of the explosion of AI. Um, the governments are playing catchup. There's no Global governance model. Um, it's overwhelmingly just a small number of advanced Industrial economies, and further, a lot of it is catching up to the state of where AI is today. Um, where you have no governance at all, uh, as opposed to thinking ahead to where AI is likely to be in three or five years, which is a dramatically different and more capable place. The fact that the um, the models, the data, the priorities, the incentives, the actions are being determined by the companies as opposed to the governments does create an environment which is technopol. In other words, when you talk about the AI environment, at least today in 2024, you're really not talking about an environment where governments are sovereign. Now, I mean ultimately, of course, governments uh, have the ability to enforce laws, and these companies don't exist in the virtual world. They exist in localities. They have Bank accounts, um, and they're not you.

know just all on crypto Off the Grid um, and they live in houses, and they have citizenship, and so they absolutely can be, you know, uh, can be governed. But the de facto reality is that um, the decisions that are being made about what AI is and what is done with it are being made overwhelmingly by the technologists, by a small number of the technologists. So, they're the ones that are deciding, and that means a couple of things. It means, first of all, if you are a government actor or a multilateral organization comprised of government actors, and you want to create effective governance, you actually need to work with the private sector. You can't do this by yourself because you don't have the knowledge, you won't move fast enough, you don't understand what needs to be done. They're the ones doing it, and if you don't do that, uh, they will de facto regulate themselves, and that is a challenging thing because their interests are in treating citizens at best as consumers and at worst as

products not. Yeah, it didn't work out too well for social media, did it? Not as Citizens and governments care at best in people as Citizens. So, I mean, if the government's going to work for the people, the government has to work with the private sector on AI. That is the reality of where we are today. And what we have certainly learned with a lot of humility at the United Nations is that if you want to start governing something effectively, you first need to define it. You need to understand it. You need a common understanding of what AI is, where it's going, both the opportunities and the concerns. And you know, this is where the UN was so effective on climate change. And it took a long time, but we are now in a situation where almost every single member state, no matter if they're poor or rich, small or big, a dictatorship or a democracy or something in between, they all agree that we've had 1.2 Centigrade of climate change. They all agree on how

much carbon and methane is in the atmosphere. They all agree on the implications of that for extreme weather conditions and loss of biodiversity. Impact of plastics in the oceans, you name it, right? In other words, all of the negative externalities from globalization, the world agrees on that. Now, they don't agree on what to do about it. They don't agree on the resources, but my God, if you can agree on the problem and the opportunity, then when you have resources, you'll deploy them more rationally. When you have conversations about global governance, it's much more likely to move in the right direction. It's so much easier to harvest cooperation as opposed to zero conflict when you all agree on what the challenge is. So, we are now at the point in the United Nations on global governance, not where we're calling for, We've got to create an authority that has the arm to compel behavior. It's not that. No, we're at the point where we want to commonly define the opportunities and the risks that come.

from the state of play of artificial intelligence right now and going forward, because doing that will give us the tools we need to govern. I think that is very important for a technology as nebulously defined and many associated technologies that sort of need to be taken in concert to govern this effectively. And of course, the central point in what you're saying seems to be that we cannot leave this to market forces, right, to influence the development and deployment of this technology. We know exactly how that went with social media, not super well for the end user, maybe not even that well for society as a whole. Oh, clearly, clearly horribly for society, totally as a whole. I mean, social media as it stands right now is undermining the civic architecture of our societies, and that is not the intention of social media. It is purely an unintended but logical consequence of the business model. Oh, 100%. If you're optimizing for engagements and that's the dial you're moving things towards, funny things will happen. Now, AI again presents its own challenge and you're kind of

alluding to this, so I have to ask what should be the guiding principles for technology that, as the report states, is one having the world come together to define the state of climate today and how it's been changed by humanity. The world has also come together to define, we have a universal declaration of human rights, and we have sustainable development goals. The world agrees that people should have the opportunity to engage in productive labor. The world agrees that people should have adequate shelter. The world agrees that people should have adequate food. The world agrees that people should have adequate water. The world agrees people should have access to health care. I mean, there's basic things, and yet the world agrees that there should be goals for the 8 billion plus people on the planet, and we should orient towards those goals. So, the baseline for artificial intelligence and governance should be how do you help use and deploy these technologies in service of those goals. There's nothing really bigger than that, right? It's not more complicated than that. We right now, we are not

actually on a path to achieving those goals. In fact, since the pandemic, we've actually backslid on things like hunger and unforced migration. AI is a singular tool available to humanity that could help. I think it will help humanity achieve those goals if it is deployed in service of those goals. There is nothing more noble and nothing more sustainable for our planet than to accomplish that. Um, and so I think those are your principles, and I think the goals of the UN high-level panel on artificial intelligence governance should be in service of that. It really is a question of objective functions, and what you're saying is, Hey, there's already global consensus on these things that we want to achieve, you know, Agenda 2030, the Sustainable Development Goals. Let's put this new technology to bear on, you know, making a huge dent in all of those goals as quickly as possible. And, by the way, I think it's easier um than climate in a fundamental way, and it makes me very excited, which is climate change. There were an awful lot of very rich

very powerful actors, governmental individuals, corporate banking, um, that really saw that addressing the climate agenda, never mind advancing it, just addressing it, was existentially threatening to their well-being. That is absolutely not true of AI. You do not have people in Industry out there that are saying, I need to pull the wool over Collective Humanity's eyes and not have AI used in service of humanity. It's not that. It's actually that if these people focus purely on the business models and the market agenda, they're just not going to get to all of the other opportunities for AI because it's not a priority. So, they won't bother with the global South, they won't bother with a lot of the poor, they won't bother as much with biodiversity because there's so much low-hanging fruit for them to make, you know, working with the Fortune 500 and working with, you know, the deep-pocketed in the Gulf and Europe and the United States. So, the point is, if you just create that knowledge, if you create those opportunities, if you let

everyone knows give you another point bwell if you talk to the owners of these AI companies the technologists the CEOs the founders the shareholders you know they are looking for good things to do they want it's not like they're not charitable people totally right altruism is a huge part of the culture in many of these companies too it it really is but they don't necessarily know what the right thing is to do so if you don't have a global conversation with common standards and with a common Global scientific and policy understanding of here are the ways that AI can have the greatest impact then you are going to waste so much money you're going to waste so much good effort instead of having AI for Humanity you're going to have ai washing you know you're going to have people just saying hey here's what I'm doing with AI and just make sure that at least they have a good story to tell I'm not interested in giving people good stories to tell right so this is something that is completely doable and

It's not a collective action problem; it is just a coordination issue. I love that. Yeah, it's like the saying in Silicon Valley: move metrics that matter. If you can't define the metrics to move, everyone's just going to be running in conflicting directions. Exactly. And so, by having sort of a common set of goals to move towards, I think that could be extremely, extremely impactful. You brought up the global South, and the report emphasizes protecting the interests of the global South, right? And a couple of key ingredients are outlined. I'm paraphrasing here, but roughly, I'd break it down into compute, data, and talent. What specific measures are being proposed to ensure this equitable access to AI technologies and sort of prevent this widening of the digital divide? There's so many languages that are already underrepresented in the modern large language models, but there's so much more than that too, right? Like, how the heck do you even get access to the GPUs that you need to train or fine-tune your

models. So, how is that going to work? Well, I mean, first of all, it's going to start slowly, and then hopefully it picks up. So, you know, you've got two concrete recommendations that we hope to be taken up by the member states. One is an AI capacity development network, and that idea is to have UN-affiliated development centers that make expertise, compute, and AI training data available to the global South. That means, you know, you're going to have sandboxes to test AI solutions and learn by doing. It means you have educational opportunities and resources for university students in the global South, and programs for researchers, social entrepreneurs, and training public sector officials, which they generally don't have right now. You know, a fellowship program, you know, for individuals to spend time from the global South into top programs working on AI in the developed world. These things have to be created. There's a second recommendation which is connected to that, which is a global fund for AI. In other words, that you end up raising the money from the private and

the public sector that would support this development capacity. Now, look, I'm not suggesting that this is easy or done overnight, and when you have half of Africa that doesn't even have access to electricity, capacity development for their data isn't going to get them on AI right. So, there are big issues out there that have to be addressed, but this is a unique tool, and the idea is that every million dollars that goes into this Global fund is going to be oriented in a way that the world believes is most effective to get AI to be used by the world for the development of humanity. It's inconceivable that an individual Corporation would try to do this, or government would try to do this by themselves. It's well said, and it also makes the, I think you also make the point that the incentives are aligned, which is, you know, often not the case. You are basically, by providing this Baseline level of computes and abilities for these markets, you're basically unlocking markets for these companies as well. And so, I think the, the incentives

there are quite well aligned. I have to ask one question related to that, which is sort of the, uh, I would say the, uh, the sort of pushback in Silicon Valley when it comes to all of this stuff, which is, you know, especially AI companies, particularly in Europe, sort of feel that the existing regulatory landscape, right, GDPR and so forth, DMA, are already so restrictive, and it's the large companies that benefit from this, and hey, what's going to happen to all the smaller companies and the smaller players? How do you sort of balance, um, consumer protection while also, uh, enabling innovation? How do you think about that? Um, I, I do fear that, um, at the platform level, the amount of compute that is required, um, the energy to service that compute, the water to service that compute, uh, I mean, you're talking about a small number of corporations and governments that are capable of doing that, and the data, uh, that they have available to them allows them to identify startup corporations very quickly, um, that they can buy, support, or kill, and we've seen

that historically with Facebook/Meta, we've seen it with a lot of organizations, and we're seeing it play out right now. Some of the startups that raised hundreds of millions of dollars are like, We just can't keep paying this compute cost. They can't keep paying the compute cost. So look, I mean, it certainly, there's an argument to be made that maybe, um, if AI, uh, models go open source, um, that um, it's possible that you end up with a whole bunch of players, um, that are using that AI in their own local applications, their own sepal applications that make a big difference. Most people in AI would not make a strong bet that that's where AI is going, right? I mean, that Meta is making that model, uh, making that bet, uh, but most of the money is not heading in that direction. Certainly, that's not true in China, uh, which we haven't talked about, is a wholly different kettle of fish because it's largely self-contained. But on the industrial side, what China does is going to be extremely important. Look

I mean, I could answer this question in a more dystopian way, um, which is for most of my life. Uh, the reason that state capitalism was a failure is because you have kleptocracy, inefficiency, nepotism, um, that makes those companies make very bad decisions. Um, but it might well be that when AI is deployed, um, within real time, with access to massive data sets, that AI making decisions at the enterprise level to drive corporate growth might prove to be more efficient, um, than what the private sector can do by itself. And if that is true, then you will have a very strong anti-competitive impulse because suddenly, um, economies of scale together with efficiencies driven by AI means that the best possible um, outcome you can have economically is non-competitive. But if that's where we're heading, then the role of effective regulation to ensure that individuals have rights that are safeguarded becomes far more important, right? Because people and consumers no longer have the ability to choose A as opposed to B, right? They're going to be in an environment an

AI environment that controls everything. It's like a fish in water, right? Exactly. I mean, it'll be like, I mean, you can, it'll get, I am of the view that consumers within five years will probably use an AI ecosystem for an incredible amount of social, business, educational, health, and other relationships, and they'll probably all be the same AI because there'll be advantages in having that data set together in one place, and you'll have preferences depending on your economic well-being and what kind of service you want and everything else, also your geography. Okay, but if that's the case, there would be real costs in you switching. You already feel this, like, I mean, if you're on Twitter and you've spent a bunch of years building up your following and your connection, and suddenly you want to go off, like, you're locked. You have no, you're locked in. You have no rights. If that's where we're heading on the consumer side, and if on the industrial side we end up with, again, a small number of players that have extraordinary amounts of data that are

able to drive the innovation because they have the data to compute the energy. Um, then the role of effective governance and regulation at the national and at the global level becomes far more urgent. So, I am not answering your question well because your question was like, well, what happens to the small companies? I'm like, I'm more concerned if the small companies end up dying, um, or getting, or as they become small companies that are, or you can have a whole bunch of small companies, but as they become big enough to be noticed, they have to align. What happens to the small creators? They can all create, but if you become big enough that you Mo make a move, you've got to sign up with a record company. You've got to sign up with, like, you've got to be a Facebook and influencer, Instagram, whatever it is, right? I mean, I'm saying I could see that happening in every sector with every corporation and startup, but if that happens, then we've got to spend a lot more attention, uh, making sure that individuals still have these rights. You

bring up this point where the current paradigm, at least that we're experiencing, does seem to be that the companies that control large amounts of compute, large amounts of data, are going to be at a huge advantage. They're going to be creating these products and experiences that really mediate our interactions with the digital world and increasingly the physical world too. And so, you know, in this context, open source is brought up as this sort of counterbalance, if you will, and it's a very hotly debated topic too because there's a spectrum of sort of open source and closed source, right? Like, let's be honest, most of the open source models in the wild today are more like open weights versus fully open source models where you have access to the training data and the source code and all that good stuff. The report advocates for this stand of meaningful openness. Can you just elaborate on that? It means that you are hoping that people, governments, and organizations around the world aren't

purely takers um of AI models, values, preferences, and products that come um from the biggest Global Platforms in the West, in China, that you hope that there is Meaningful openness, enough openness that individuals and Enterprises and governments will be able to actually deploy AI in meaningful ways for their benefit um over time. That is the Hope um that's what you're calling for but there's no there's no power behind that, there's no money, there's no Authority behind that. That's just that's just the collective view of 39 people around the world that happen to have various expertise on AI uh and it's something was fairly easy to agree on now um. I will say because I'm agnostic as to which way this is going to go as opposed to which way I would like to see it go um. I mean you analysis is not preference here right and that's a very important thing to keep in mind when you're talking about these issues. There are different risks that come from these models so if it turns out that

overwhelmingly um we have closed models and a very small number of companies, maybe that need to be aligned with either the United States or China um that end up dominating the AI space. Then the critical governance framework you will need um for National Security and sustainability is a series of Arms Control agreements in AI between the Americans, the Chinese, and relevant private sector actors like what you had between the Americans and the Soviets. But we only got that after the 1962 Cuban Missile Crisis when we almost blew ourselves up. That's a serious problem, right? Because in the first 15 years, the Americans and the Soviets are developing bombs, and then H-bombs, and then missiles to launch them, and all of this capacity, and we're going as fast as we can to assert dominance, and the Soviets as fast as they can to undermine that dominance. And you know, well, you you you don't want to talk to your enemy and let them know what you're doing because, I mean, that's just going to give them information. It's going to constrain you.

well no it turns out you really need to have those agreements because otherwise you will blow yourselves up and so that is essential um I would say for the next us Administration secondarily if it turns out that actually the drivers of Cutting Edge AI turns out more like uh not just meaningful openness but radical openness where you know almost anybody in the world good actor bad actor in between tinker um can take AI models and deploy it for their own purposes that means that the future of AI technology is going to be more like the Global Financial Marketplace it'll be systemically important everyone will use it and everyone will need it for their own purposes but also almost anyone could be a systemic threat to it right and could bring it down like kind of like you know you've got a couple of yahoos on Reddit talking about GameStop and before you know it you've got like a challenge for the global markets right and so then what you need is a geotechnology stability board something like the financial stability board that sees that

Okay, we're actually not politicized. We're independent technocrats with an intention of identifying potential threats to the AI Marketplace, ensuring resilience, and responding to crises, ringing the bell and saying we've got a crisis, and then responding to them collectively as soon as humanly possible. That's a radically different kind of governance than the US-China Arms Control governance, and we presently need to start thinking about both because we're not actually sure which sort of methodology, if you will, is going to be most successful in how AI will be deployed over the coming years. We just don't know. I like the finance analogy a lot. You almost want these parties to sort of be, regardless of sort of military or diplomatic ties, that open lines of communication are there because if you've got this interconnected web of Open Source AI running the world, if you will, it could very well be something where traditional diplomatic channels might be far too low bandwidth or almost stifling.

uh to respond as quickly as possible. Well, it's funny. I'm sure you read Ministry of the Future by Kim Stanley Robinson, but I will after this, the book. Okay, so it's interesting. It's a book on climate, and it's about, you know, kind of medium-term, somewhat dystopian, you know, massive wet bulb climate incident. Millions die in India, lots of terrorism, eco-terrorism, and need to have global technocratic independent governance over carbon emissions and related things because the world just has failed, failed, failed, right? Fascinating concept, but probably not necessary for climate, uh, in part because the money is coming, the technology is there at scale. We are moving to a post-carbon environment, um, that is, you know, I mean, it's later than you'd like it to be with a lot of cost, but nonetheless, it's going to happen, um, in the next two generations. AI may well require a Ministry of the Future. You may need, like you have central bankers today, you may need to have ministers of AI play that kind of a role.

who are truly like in their appointed parts of national governments, but they act in a politically independent way. They're not driven by ideology, they're not driven by political cycle, they're driven to ensure that the system globally continues to work. When you have a major Financial Crisis, you know, the head of the People's Bank of China and the head of the FED have the same tools. You've got fiscal tools, you've got monetary tools, and you've got the same Playbook, you've got the same definitions, and you all want to get the markets working, and you really want to avoid a global depression, right? I could easily see that being what the future of AI needs to be like, but it's much harder. It's much harder because the marketplace, you know, is dealing with exposures and movements of, you know, massive amounts of money, but AI, it's not just about, you know, sort of digital equivalent of money, right? It's, it's like

it's everything. It's National Security. It's an election. It's bio. It's, you know, it's drones. It's anything. So, it's a much more... It is like the central bank issue, but it would be much more powerful. And, you know, for people that are thinking about artificial general intelligence, I mean, the reality is that, you know, humanity could probably never handle artificial general intelligence unless you had vastly more powerful governance to deal with that, and governance at a technocratic and global level. I have to ask about, you know, you ask about the China question. Like, there's this ambitious vision for global AI governance that's been outlined here. A common critique of the UN tends to be, How are you going to enforce these guidelines? Especially with, you know, major tech companies that might play along right now but at some point might want to assert their own independence or countries like China that might be reluctant to comply. Well, uh, that's why I use the climate change example. Um, is you

did not start with compliance. You started with defining the challenge, the opportunity, and the risk. So, you had an intergovernmental panel on climate change, and that's what predated all of the committed, even if non-binding, carbon reductions and targets and everything else. First, you had to have a whole bunch of countries that all agreed these are what the opportunities and challenges are, and that's what we are starting with. We are starting with an international scientific panel on AI, just like the intergovernmental panel on climate change, which is meant to regularly convene and define the capabilities, the opportunities, the risks, the uncertainties, the gaps in scientific consensus on AI development and trends. And the point is, if you get the world to agree on that and exchange standards and talk about opportunities and align with sustainable development goals, well then clearly you'll have your annual summits on AI, and you'll get governments to say, Okay, let's apply some resources to this, and let's make some commitments on this.

and you'll have some countries that'll want to be leaders on it, and they will, and they'll, you know, they'll push, they'll prod, they'll try to get other countries to come on board. Now, the latter is much harder and much slower than the former, but you can't do the latter without the former. My view, which I think has been one that has been shared really by consensus, is the purpose of this group is to help the world define this opportunity and this challenge. And by the way, the opportunity is bigger than the challenge, right? Totally. And we truly think of it that way. This is not coming out there saying we want to have enforceable governance power over what technology companies do and don't do. That was never going to be in the cards. But also, it would be premature if you don't actually know what you're trying to accomplish. So don't put the wagon before the horse. Get the problem right first. Define it. It makes a ton of sense, right? Like, if you're not going to

survey the landscape and build a common map. You're obviously going to have disagreements from get-go um, but what about more challenging topics like for instance in light of the increasing use of AI in both the Ukraine Russian conflict right and the Israel Gaza War. What position does the report take on developing International International Norms or regulations specifically addressing the use of AI and warfare? Well, I mean, there's already been efforts at the United Nations um to ban uh lethal autonomous drones and artificial intelligence um in those areas uh where National security is uh directly um in the Target um. You know, you also have the beginnings of uh the US China and South Korea uh talking about the nuclear decision-making process for example. My view uh is that um governments a small number of governments are probably going to be necessary to drive that uh and not the UN even though the UN can have a say because ultimately this will need to be Poli and the way you police is by having uh willingness to deter and some kind of

actual enforcement sanctioning enforcement they're going to be cost. I mean, the Iranians, you know, are not standing up to the nonproliferation treaty. If they're not developing nukes, it's because they're concerned that the Israelis are going to blow them up with UF support if they actually decide to go full bull nuclear breakout. And that, I think, when it comes to using AI in these Weapons Systems, when you are talking about life and death for the Russians and Ukrainians on the ground, and they have access to those tools to make into weapons, they will make them into weapons. So, you're going to have to have the most powerful countries in the world be willing not just to opine but to govern, to exert Force, um, to ensure um that AI is not used um in uh by itself in targeting the decisions, is not used um by itself um in war fighting that takes decisions in life of death completely out um of the loop. We've already seen what happens um as War

becomes more and more distant from humanity. Is that it becomes much easier for people to pull the trigger and much easier for wars to escalate, and that's not where we want to be. Are you personally more worried about digital attacks or political interference, you know, via the Internet, or sort of these physical manifestations of AI systems when it comes to like drone warfare and things like that? I'm much more worried about the vulnerability of economic systems to AI manipulation. I'd be much more worried about financial markets crash or critical infrastructure crash, especially when you already have antagonists that are deeply inside other systems and just sitting there. In other words, a CrowdStrike-type situation, but instead of a mistake, a malevolent situation. I mean, you saw what happened to, you know, sort of, I mean, cost billions of dollars immediately and shut down all sorts of systems, computers, air travel, you name it, you know, around the West, and that was just a mistake, that, you know, it's just a mistaken patch.

uh what happens if that was done intentionally by an adversary, uh whether a rogue actor or by a government. I think that's a much bigger concern, but that's not my biggest concern. My biggest concern is something actually much more prosaic. What is it? My biggest concern is that you beings are programmable. Um, we are very susceptible to algorithms and um, you know, when I was growing up, uh, it was all about nature and nurture. It was about, you know, sort of, you know, what your genetics say and then how that potentiality is shaped, your environment, by human beings around you, individual human beings, and collections of human beings. That's now shifting to algorithms and, you know, social media is a part of this problem, but when it's Ai and when human beings are developing principal relationships algorithmically, um, and they're no longer engaging principally with other human beings, so nurture is being replaced by algorithm and by AI. I worry that we will lose our Humanity. Uh, I think that's a much more, for me, it's a much more profound, it's a

much more philosophical issue, but it's also a really real near-term threat. You know, we may develop AGI in 20 years, but humans may become algorithmic in five. Um, and I think we should resist that. Uh, I think that, um, at the very least, we need a lot of testing to understand the implications before we want to run real-time experiments on humanity. And right now, what we're doing is running real-time experiments. We need a smaller sandbox. I think you're totally right. It goes back to technology mediating our interactions in the physical and digital world. And that's right. That gives us the sort of the surface area for us to be impacted and influenced is greater than ever before. I'd argue this is already the case today. That's in part why online advertising is just so darn effective. That's why mimetic warfare is a thing right now in information. Yeah, but your relationship is still principally with people. Oh yeah, now that once we have intelligent entities that are sort of

talking to us and oh good lord right. I mean we've already, you know, basically seen AI pass the Turing test in the last year, and now we have a bunch of people in AI that tell us that that's not that important. That's just moving the goalpost. It's moving the goalpost. I think actually it is essential for what matters because what matters for us at least is humanity. And so if we can no longer tell the difference between what is and is not a human being, and we start engaging more principally with non-human actors in service of other goals that have nothing to do with human beings, then we have just unwillingly, unknowingly given up something essential. I'm deeply uncomfortable with that. I'm deeply uncomfortable with that. Once this report is made public, you're obviously going to get a huge amount of feedback from various stakeholders, right? What would you like the audience that's listening to this to do? Should they engage with this report, and how should they give their feedback to which

y'all are proposing um. I think they should certainly um talk about the fact that AI is a tool that needs to be used for all of us. You know, if there's any real message here, it's that um, we are turning much more inwards um, at a nation by nation level um, and uh, we're not thinking about common collective humanity. We are now inventing tools that allow us to improve ourselves together and to redress a lot of the damage that we've done to the planet. I mean, if I look at young people today, we need to be honest with them. You know, we've not been effective stewards of the planet and we need them to make better decisions than we have made. I would argue this report has been an effort over the last year to try to give young people an opportunity to make better decisions and that's the way I want people to respond to this report. Awesome Ian, thank you so much for joining us. My pleasure. Okay, Ian's insights are a

wakeup call. AI isn't just about code and algorithms; it's a global challenge that demands global cooperation, just like climate change or the interconnected web of financial markets. I mean, think about it. When a financial system teeters on the brink, central bankers don't care about borders or political alliances. They pick up the phone because their fates are intertwined, their incentives aligned to prevent disaster. That's the kind of urgency and cooperation we need for AI. What gives me hope is that this isn't some impossible challenge. We have achieved this level of cooperation before. Now the question is, can we replicate it for AI before it's too late? Will we see a world dominated by a few tech giants hoarding the power of AI, or can we foster a scenario where a thousand flowers bloom, where AI empowers individuals and nations across the globe in a truly interconnected yet decentralized AI ecosystem? This conversation with the in has been a reality check. AI isn't just about futuristic hype; it's about the choices.

We make today this UN report is a vital blueprint to build a shared understanding on what exactly it is that we're trying to regulate. But the real work begins with us understanding the complexities, demanding transparency, and pushing for responsible AI development, and doing that without stifling innovation and nimble startups. Because ultimately, the future of AI isn't predetermined; it's an interconnected tapestry and a global story we're all writing together.

The UN Is Speaking Up About AI — Here’s What They’re Saying | Ian Bremmer and Bilawal Sidhu | TED

The TED Talks channel features the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes (or less). Look for talks on Technology, Entertainment and Design -- plus science, business, global issues, the arts and more. You're welcome to link to or embed these videos, forward them to others and share these ideas with people you know.

TED's videos may be used for non-commercial purposes under a Creative Commons License, Attribution–Non Commercial–No Derivatives (or the CC BY – NC – ND 4.0 International) and in accordance with our TED Talks Usage Policy: https://www.ted.com/about/our-organization/our-policies-terms/ted-talks-usage-policy. For more information on using TED for commercial purposes (e.g. employee learning, in a film or online course), please submit a Media Request at https://media-requests.ted.com

글로벌 AI 거버넌스를 위한 유엔의 대담한 계획

유엔은 주권 국가들이 따라야 할 가드레일을 설정하는 것을 목표로 하는 글로벌 AI 거버넌스 계획을 발표했습니다.

이 계획의 시급성은 AI 기술의 급속한 발전과 사회를 혼란에 빠뜨릴 수 있는 잠재력으로 인해 더욱 커졌습니다.

이 보고서의 공동 저자인 이안 브레머는 AI가 개인과 산업에 제공하는 혁신적 기회를 강조하는 동시에 그 위험을 관리하기 위한 효과적인 거버넌스의 필요성을 강조합니다.

AI의 힘의 역학 관계와 그 영향력

민간 기업이 AI 기술을 빠르게 도입하면서 정부와 기업 간의 힘의 불균형이 발생하고 있습니다.

이러한 기업들은 수익성에 초점을 맞추기 때문에 AI의 개발과 배포에 상당한 영향력을 행사하는 기술 양극화 현상을 초래하고 있습니다.

AI에 대한 효과적인 글로벌 거버넌스와 규제의 부재는 일자리 대체, 경제적 불평등 악화 등의 위험을 초래할 수 있으므로 효과적인 거버넌스를 위해 정부가 민간 부문과 협력하는 것이 중요합니다.

글로벌 거버넌스에서 AI의 역할

유엔은 AI의 영향과 기회를 효과적으로 관리하기 위해서는 AI에 대한 공통된 이해가 필요하다고 강조합니다.

기후 변화와 인권과 같은 문제에 대한 글로벌 합의는 지속 가능한 개발 목표를 달성하기 위해 AI를 사용하기 위한 프레임워크를 제공합니다.

비즈니스 모델에 미치는 AI의 고유한 영향력은 글로벌 남부와 생물 다양성 문제 해결 등 공익을 위해 우선순위를 정할 수 있는 기회를 제공합니다.

AI 기술에 대한 공평한 접근 보장

AI 기업 리더들은 좋은 일을 하고자 하지만 어떻게 하면 가장 큰 영향을 미칠 수 있는지에 대한 공통된 이해가 부족합니다.

전 세계적으로 AI에 대한 공평한 접근이 강조되고 있으며, AI 역량 개발 네트워크와 AI를 위한 글로벌 기금 등의 방안이 제안되고 있습니다.

이러한 이니셔티브는 전문 지식, 컴퓨팅 및 훈련 데이터를 제공하는 동시에 기업의 시장 개척을 위한 인센티브를 조정하는 것을 목표로 합니다.

AI가 기업 권력과 소비자 권리에 미치는 영향

기업 차원에서 AI를 도입하면 효율성이 높아지고 경쟁 역학관계가 악화되어 소비자의 선택권이 줄어들 수 있습니다.

방대한 데이터 세트를 보유한 소수의 기업이 혁신을 주도하고 소비자에게 종속적인 환경을 조성할 수 있습니다.

AI 기반 경제에서 개인의 권리를 보호하고 힘의 불균형을 해소하기 위해서는 효과적인 거버넌스와 규제의 필요성이 더욱 중요해지고 있습니다.

AI 기술의 미래는 급진적인 개방성을 요구할 수 있으며, 이는 시스템적으로 중요하고 잠재적으로 위협이 될 수 있습니다.

금융안정위원회와 유사한 지질공학 안정성 위원회는 AI 시장에 대한 위협을 파악하고 회복력을 보장하는 데 도움이 될 수 있습니다.

기후 변화에 관한 정부 간 패널과 유사한 글로벌 기술 거버넌스는 AI 개발의 도전과 기회를 정의하고 지속 가능한 성장을 위한 노력을 조율하는 데 필요할 수 있습니다.

AI 거버넌스와 전쟁에서 정부의 역할

한 그룹의 국가들은 기회보다 도전이 더 크다는 믿음으로 AI 거버넌스의 기회와 도전을 정의하기 위해 노력하고 있습니다.

전쟁에서 AI의 사용을 규제하기 위한 노력이 유엔에서 진행 중이지만, 법 집행을 위해서는 소수의 강력한 정부가 필요할 수 있습니다.

연사는 AI가 경제 시스템을 조작할 수 있는 가능성과 인간 관계에 미치는 영향에 대해 우려를 표명하며, 알고리즘이 개인적인 상호작용보다 우선시되면서 인간성이 상실될 것을 우려합니다.

책임감 있는 AI 개발의 시급한 필요성

AGI의 잠재적 부상은 단 몇 년 안에 인간이 알고리즘화될 수 있는 단기적인 위협이 될 수 있습니다.

현재 기술을 통한 인간에 대한 실시간 실험은 재고되어야 하며, 소규모 테스트 환경의 중요성이 강조되고 있습니다.

AI라는 글로벌 과제를 해결하기 위해서는 소수의 거대 기술 기업에 권력이 집중되기보다는 개인과 국가에 권한을 부여하는 탈중앙화된 생태계를 지향하는 긴급한 협력과 공동의 행동이 필요합니다.