핀테크 규제 및 실무에서의 AI 안전

### 글로벌 규제 발전 속에서 핀테크 부문의 AI 안전에 대한 이해
* 🌐 유럽 의회는 3월에 **EU AI 법**을 승인했고, **영국과 미국 정부는 AI 안전에 관한 획기적인 합의**를 발표했습니다.
* 🔒 영국과 미국의 협력은 국가 안보를 넘어 **사이버 보안, 개인 보안, 시스템 안정성을 포괄하는 AI 안전**에 초점을 맞추고 있습니다.
* 🤖 AI 안전은 특히 빠르게 발전하는 **제너레이티브 AI에 따라 진화하는 위험**을 해결하기 위해 의도적으로 광범위하게 정의되었습니다.
* 🏦 규제가 엄격한 환경에서 운영되는 금융 기관은 **알고리즘과 한계를 이해하는 데 중점**을 두고 AI를 안전하게 배포해 왔습니다.
* 📜 영국은 EU의 사전 예방적 법적 접근 방식과 달리 **원칙에 기반하고 부문/규제 기관이 주도하는 AI 규제 접근 방식**을 채택하고 있습니다.
* 🇪🇺 EU AI 법은 위험도에 따라 AI 시스템을 분류하며, **고위험 시스템은 사람의 감독과 영향 평가**가 필요합니다.
* 📝 핀테크 업체는 **AI 인벤토리 작성, 정책 업데이트, 승인 프로세스 구현, 갭 평가 수행**을 통해 EU AI 법에 대비할 수 있습니다.
* 📊 개인정보 보호는 **데이터 주체에 대한 투명성을 확보**하고 다양한 개인정보 보호 함정을 해결해야 하는 등 제너레이티브 AI의 복잡한 문제입니다.

영국과 미국 정부는 AI 안전에 관한 획기적인 합의를 발표하여 공동의 목표를 위한 프로그램 개발을 위한 협업을 가능하게 했습니다.

welcome to fintech focus scadden's podcast for fintech industry professionals the global regulatory and legal updates you need start now. hello and welcome back to fintech focus. today you've got me again Joe Kamar. I'm an m&a lawyer in scad's fintech practice and with me is Nicola Kershaw who's one of our lead AI lawyers here in London. Nica, thanks for joining. thanks Jay, it's great to be here. so plan for today, we're going to unpack the term AI safety in the context of the fintech sector and we've obviously had some big headlines in the world of AI over the past few weeks. so first of all, we saw the European Parliament approve the EU AI act in the middle of March and then this month we had the UK and US governments announcing their Landmark agreement on AI safety which neatly ties into today's topic. and I guess when you look at some of the historic AI collaborations between the UK and US governments there's traditionally been a

heavy focus on National Security, and the examples I'm thinking of are the Partnerships between GCHQ and the NSA in the US. That said, when you look at the detail of the current partnership, and Nicha correct me if you disagree, this seems to be less focused specifically on National Security and actually much broader. Yeah, and I Joe, I entirely agree. There's a huge focus on security within this MOU, but security is indeed broader than just National Security. It isn't defined and doesn't appear to be limited to any particular type of AI or sector. So, we have loads of questions we can ask ourselves. Is the MOU seeking to capture cybersecurity and the risk of attacks fueled by AI, or is it looking at the security of individuals and the risk of harm caused to them by misuse of AI? Or are we looking at security of financial and other regulated systems and the potential threat to those systems and the stability of those systems caused by AI? And how is this collaboration going to work in practice?

there's a lot of questions we don't know a lot but what we do know is that this is a groundbreaking agreement and will enable the UK's new AI safety Institute to work collaboratively with its US counterpart. Together they're going to develop what they are calling an interoperable program of work and approach to Safety Research in order to achieve their standard shared objectives on AI safety. This is going to include developing strong methods to evaluate the safety of AI tools and their underlying systems along with information sharing with each other. So I think the definition of safety going back to your point has been left intentionally broad and is seeking to encapsulate many of the potential security concerns within AI. Generative AI is evolving at immense speed as we've seen recently, and we don't know many of the risks. In fact, in the UK's government's recent response to its AI regulation white paper, it specifically stated that it was waiting for any potential regulation of AI until its understanding of the risks has

matured. So, the Mau provides a framework to keep addressing this security as it evolves. And while those risks are maturing, I always find it interesting how the concept of AI safety can mean so many different things in so many different contexts. And I guess instinctively, I suspect lots of people, thanks to Hollywood, associate AI safety with the idea of robots taking over the world. But clearly in the context of financial services, that's hopefully less of a concern. Now, your background, Nicola, is an interesting one as you've only recently joined Scouten, having spent a number of years in the tech arm of a global financial institution. So, I'm curious to know from your previous life before Scouten, what were the sorts of areas and risks keeping you up at night from an AI safety perspective? So, my role and my focus was advising a broad range of technology and IP issues across a large financial institution and its group of companies. Technology did include AI, but at least in recent years, that was nothing novel or new. Financial institutions have been using forms of AI.

for years and for a wide range of purposes, Financial Services is also heavily regulated already. AI had therefore been developed within a sensible and controlled environment in a way which ensures continued compliance with financial regulation. So AI safety was, I and I think still is, a bit less of a concern for many banks. A key Focus Point surrounded understanding the algorithms and been able to explain them to any interested regulator. It also involved having proper control points to ensure safe testing and development. Understanding how AI can enhance services, for example, huge developments in detecting fraud and financial crime, but also understanding its limits, were absolutely key. It's also fair to note that a lot of use of AI within financial institutions has been machine learning and Bots, rather the new more difficult generative AI that has grabbed so many of the headlines this last year. So I guess we're saying financial institutions have, in some respects, already been Paving the way with examples of safely deploying AI given the highly regulated environments that're they're operating in. And I think

it's fair to say that we're not seeing a call to arms for fresh AI focus legislation in the UK. And my sense of that government's positioning seems seems to be that we have a principles-based approach that's adaptable and also sector and regulator-led. Now that's pretty distinct to the proactive and much broader statutory approach that we're seeing in Europe, for example, through the EU AI Act, which we'll come on to. Um, and when you then look at what the regulators actually saying, again my sentence, it seems there seems to be little appetite amongst likes of the Bank of England PFCA for a sector-specific statutory framework. And instead, the focus seems to be on understanding how the patchwork of existing regulatory frameworks AI technologies. So, Nica, what in your view are the most material touchpoints with the existing regulatory regimes? I mean, clearly for entities providing financial services in the UK, the FCA rules and CIS will continue to apply and are key touchpoints. This is nothing new, but these rules need to be

interpreted with a specific AI tool in mind. As with any technology within Financial Services, key Focus points will be on Market stability, maintaining due skilled care and diligence, and evidencing that operational resilience, consumer Duty, and treating customers fairly and transparency with the regulator. The senior managers and certification regime will continue to apply and needs to be considered in relation to the development of any internal or external AI tool. In addition to this, the use of personal data within an AI model or system is also a key touch point for financial institutions. For some banks, particularly institutional Banks, personal data is only incidental to the use case in hand, with the real Focus being on the corporate relationship. However, the UK gdpr still applies in full, and achieving transparency with data subjects in this context can be really quite challenging. Makes sense, and I guess my sense from recent discussions with Regulators that you also present at is that we can expect to see at the very least more guidance on how those regimes apply to AI. That said, you actually think it's

realistic that the UK continues to veer away from legislating for AI altogether. When you look at the industry feedback, the sense is that there's a push for a globally harmonized approach to AI regulation, and clearly that's not the case when you look at what's happening in the US and the UK vis-à-vis Europe. And a second question, if I can, are there any aspects of the existing regulatory regimes where you can see gaps forming as use cases for AI amongst FinTech and banks changes over time? These are such interesting questions, Joe, and clearly having global standardized laws is something we'd like across many different things, not just AI. I mean, I think there's strong use cases for AI within financial services, including things like AI-based creditworthiness assessments, fraud and financial crime detection, market research, drafting research papers, and risk assessments for insurance. The list goes on and on and on, and there's likely to be strong human input into most of these processes already to the extent that they use AI.

and so that along with the requirements of other financial regulation, it's arguably enough. However, it should be noted that some of these use cases, particularly those which involve individual consumers directly, for example, risk assessment for health insurance of an individual, will be considered high-risk AI under the new EU AI act and so we'll be subject to heightened requirements. We're expecting to see further development by the European standardization bodies here and whether in fact there will be additional requirements for those within financial services. I mean, for the most part, the training and control framework are already in place within financial services and this is likely to satisfy much of the EU AI act, so arguably there's no need in financial services, at least for additional regulation. Additional guidance though on how to evidence certain requirement or to comply with topics such as the GDPR would be very welcome. That all being said, for other industries, I think the use case against additional regulation is less clear, particularly with genitive AI and I think this is where government should focus both in terms of additional

regulation and guidance on how to comply with existing legislation. So on the topic of legislation, we've obviously both mentioned the EU AI Act. And now that's been formally approved by the EU Parliament within the past few weeks. It's obviously had various guises and iterations over a fairly protracted period. So perhaps you could give us a high-level overview if that's possible of where that piece of legislation has finally landed. I mean, that's a challenge. It's a vast piece of legislation, 270 pages I think. So a high-level overview. It's a bit of a challenge in the time, but here are some key takeaways. First of all, all AI systems need to be categorized on a risk-based approach. So there's minimal risk systems, which are already widely used, such as spam filters. They are, in fact, AI, and they are largely unregulated and will continue to be unregulated as normal. But then the other AI systems need to then be categorized according to their level of risk. There'll be minimal transparency and disclosure requirements to certain

AI systems such as chatbox, and the users need to be aware that they are interacting with an AI system. There are then high-risk systems. Example of these within Financial Services could include things like AI credit worth checks or risk assessments for health insurance where we'll need to uh Implement training appropriate human oversight and maintenance of technical documentation to ensure identifying risks are mitigated. There also be a need for fundamental right impact assessments prior to deployment, which is very similar to what we've seen under the gdpr. There are then AI systems which are prohibited as unacceptable risk, but it seems unlikely in my experience at least that Financial Service entities will be engaging with this type of AI. Other key headlines from the EU AI act include the need to respect intellectual property rights within the use of AI, so could impact research taken from the internet, and this could also actually impact Financial Services entities in needing to potentially share trade secrets on where they Source their research from. Entities will also be liable for breach of the ACT even if the underlying

technology is white-labeled or provided by a third party. Another key point to note is there'll be a new AI office, which will sit within the commission and will be tasked with overseeing the most advanced AI models. I could go on, but in the time, I think that's a good summary of key highlights. Fair enough. Well, more specifically then, so as I understand it, aspects of the ACT will have extraterritorial effect. So if you're a UK-based fintech, what are the sort of things you should be thinking about in practice, particularly for those fintechs looking to deploy AI either internally or as an integrated part of their client-facing products and services? I think there's a few things that all companies, including fintechs, can do now to prepare for the EU AI Act, but also to help ensure that they're complying with other regulation and legislation in other countries as well. Many fintechs will probably be doing a lot of these things already, but it's always good to have a checklist. Firstly, I'd suggest that companies create essential

inventory of AI systems and models that are deployed and being developed within the entity. This should include all systems and models, regardless of whether they are internal or external. The inventory should contain details such as the purpose of the AI, what data and IP is being processed, how the technology works, what are the use cases, and the intended purpose. So as much detail as possible. It should also reference any data privacy impact assessment conducted in relation to that AI if it uses personal data, and also note whether third-party technology is being used as part of it. Once the inventory is complete, companies can then look at that list and consider whether the AI Act would apply to each system and model, and to what extent, and then record that within the inventory. They might also like to categorize each model and system in relation to which risk category it would fall under as part of the EU AI Act. Next, the existing technology policies and procedures of the company should be updated to ensure they specifically cover AI. This should include statements prohibiting certain T

types of AI and specific action that should be taken when certain data or IP is going to be used. It should also include requirements to keep the inventory updated and what review process needs to be undertaken before a use case can firstly be explored and then deployed by the company. I mean to that end, I suggest fintex consider implementing an AI approval process, similar to how many financial institutions review and approve external vendors. This process would perhaps include an approval in committee, which could be virtual and in person, but would have representatives from risk, compliance, legal, the business, and the CESO team. And potentially, if personal data is involved, should include the DPO also. And that committee would review all AI proposals and approve them before they go live. Finally, the inventory should be then reviewed in light of the EU AI Act and the company's processes as a whole to determine what gaps, if any, there are and then form a remediation plan. We realize that companies have limited budget and time, so focus on the highest risk AI systems first and the biggest gaps.

consider what new internal processes need to be created as part of this third-party contract. Should be considered to ensure the appropriate due diligence has been conducted. There's ongoing right of audit, and there's proper contractual protections in place. We've obviously been looking at this from a regulatory and legislative perspective up to now, but as AI is increasingly deployed in client-facing platforms, it's interesting to see some of the contractual risks which are starting to emerge businesses the other month actually I had a friend complained to me about a customer service chatbot that he' introduced on his website and the chatbot had actually started offering out discounts to clients much to much to his despair and it was interesting particularly because it was around the time of the Air Canada ruling. Yeah I mean that's right the Air Canada en rolling was in February and in that one a tribunal ruled that Air Canada was obliged to honor a discount offered by a chatbot. It's a further example of the measures and safeguards that firms will need to have in place.

ensure that the outputs from its AI tools are accurate and not misleading. It's the same with chatbot used by Banks and other Financial Services. If a chatbot offers a price on something or conducts a trade, but there was a technology error, can that trade or contract be undone? Often with services such as electronic trading, the Electronic Services agreement in place will state that trades can be unhelmed at the bank's discretion if there is a technology error, but that potentially becomes a lot more complex when facing consumers, individuals rather than B2B. Also, there are questions like where did you put the terms and conditions of the bot, and how do they become binding? Where does someone click to accept them when they're dealing with a bot? Yeah, so sticking with the topic of safeguards but this time through a data lens, what are you hearing from privacy regulators in terms of safeguarding personal data and then the sorts of policies, procedures, measures that they're expecting to see in place? Privacy, in my view, is one of the most

complex topics within generative AI. There's a lot of questions and not a lot of answers. For example, we have extensive laws already around transparency of data processing and use of tracking technologies such as cookies. But how many people actually understand how their data is processed or what a cookie does? Most people, I think, click the X box to close the window and proceed to the website or click yes to state they've understood the privacy policy, even when they have not. But how do companies address this in the world of AI, where technology is even more complex than a website or a small piece of code dropped on a machine? And how do developers of AI achieve transparency with a data subject whose data they have used to train a model, particularly when they've not necessarily got a direct relationship with that individual? This is only one topic, and there are so many other potential privacy pitfalls, particularly around data minimalization, legitimate interest, consent, and many more. We've had some updated guidance from the I relating to AI back in 2023. However, following the government's response to

its consultation on AI regulation. We are expecting to see more, including their strategic approach to AI, and the deadline for this is the 30th of April. Although, just to note, we have heard through our network that the ICO will be increasing its own use of AI. Our bots will be checking your bots, so to speak, which is an interesting development. So I think this is an area to really watch. This space very good. Well, thanks very much, Nicholas. Sadly, that's all we've got time for, and really appreciate having you on the podcast today. Thanks, Joe. It's been an absolute pleasure. Lots of interesting developments here, and so much we could talk about. We haven't got on to the topic of AI in cyber attacks and what can be done to minimize legal risk here, so perhaps that's one for another time. Sounds right. And thanks again to everyone listening. See you next time. Thank you for joining us on Fintech Focus. If you enjoyed this conversation, be sure to subscribe in your favorite podcast app so you don't miss any future

Conversations. Additional information about Scadden can be found at scan.com. [Music]

What Does AI Safety Mean For Fintechs?

Skadden, Arps, Slate, Meagher & Flom LLP and Affiliates delivers the highest quality advice and novel solutions to legal challenges, enabling clients to achieve their business goals. We are known for the innovative and creative thinking we rely on to handle the most complex transactions, litigation/controversy issues, and regulatory matters, as well as the open, collaborative relationships we build with clients, including corporations; financial and governmental entities; small, entrepreneurial companies; and cultural, educational and charitable institutions. Our attorneys, who reflect diverse backgrounds and perspectives, collaborate seamlessly across our 21 offices in the world’s major financial centers.

Content may include attorney advertising.


Skadden, Arps, Slate, Meagher & Flom LLP

영국-미국 AI 안전 협정이 핀테크 부문에 미치는 영향

영국과 미국 정부는 최근 핀테크 분야와 그 밖의 보안에 중점을 둔 AI 안전에 관한 획기적인 협약을 체결했습니다.

이 협약은 AI 도구의 평가와 정보 공유를 포함하여 상호 운용 가능한 작업 프로그램과 안전 연구에 대한 접근 방식을 개발하는 것을 목표로 합니다.

이 협력은 진화하는 AI 보안 위험을 해결하기 위한 프레임워크를 제공하며, 전 세계 금융 서비스 산업에 영향을 미칠 수 있는 잠재력을 가지고 있습니다.

영국 금융 서비스에서의 AI 규제

영국 금융 기관의 AI 개발은 이미 엄격하게 규제되고 있으며 기존 금융 규정을 준수하는 데 중점을 두고 있습니다.

이러한 기관에서 AI를 사용하는 것은 주로 고급 생성 AI보다는 머신러닝과 봇을 중심으로 이루어지고 있습니다.

영국 정부와 규제 당국은 새로운 AI 중심 법안을 추진하는 것이 아니라 기존 규제 프레임워크가 AI 기술에 어떻게 적용되는지 파악하고자 합니다.

영국 금융 서비스에서 AI에 대한 기존 규제 체계와의 주요 접점에는 FCA 규칙, 시장 안정성, 실사, 운영 탄력성, 개인 데이터 사용, 고위 관리자 및 인증 제도가 포함됩니다.

향후 이러한 제도가 AI에 어떻게 적용되는지에 대한 더 많은 지침이 발표될 것으로 예상됩니다.

금융 산업에서의 AI 규제에 대한 글로벌 접근 방식

영국은 AI 법제화에서 멀어지고 있는 반면, 전 세계적으로 AI 규제의 조화가 추진되고 있습니다.

금융 산업에서 AI는 신용 평가, 사기 탐지 및 연구에 널리 사용되고 있으며, EU AI 법은 개별 소비자 사례에 사용되는 고위험 AI에 대해 강화된 요건을 부과하고 있습니다.

금융 서비스의 기존 교육 및 제어 프레임워크는 이미 EU AI 법의 많은 부분을 충족할 수 있지만, GDPR 준수에 대한 추가 지침이 있으면 감사하겠습니다.

EU AI 법은 금융 서비스의 AI 시스템에 대한 요건을 도입하고 있으며, 여기에는 인적 감독 및 영향 평가의 필요성이 포함됩니다.

또한 지적 재산권을 다루고 AI를 사용하는 기관에 책임을 부과합니다.

핀테크 기업은 AI 시스템 목록을 작성하고, 법에 따른 적용 가능성을 평가하고, AI를 포함하도록 기술 정책과 절차를 업데이트하여 AI 법에 대비할 수 있습니다.

It seems there was no text provided for me to work on.

Please share the sentence you'd like me to analyze, and I'll highlight the key phrase for you.

EU AI 법은 금융 서비스의 AI 시스템에 대한 감독, 영향 평가 및 법인에 대한 책임을 다루는 규정을 도입합니다.

핀테크 기업은 AI 시스템을 목록화하고, 이 법에 따른 적용 가능성을 평가하고, AI를 포함하도록 기술 정책을 업데이트하여 대비하는 것이 좋습니다.

AI 승인 프로세스 구현 및 규정 준수 보장

핀텍스는 리스크, 규정 준수, 법무, 비즈니스, CESO, 잠재적으로 제안서 검토를 위한 DPO의 대표로 구성된 위원회를 포함하는 AI 승인 프로세스를 구축해야 합니다.

위험도가 높은 AI 시스템과 가장 큰 격차가 있는 시스템을 우선순위에 두는 동시에 타사 계약에 대한 적절한 실사를 보장해야 합니다.

또한 핀텍스는 개인정보 보호 규정과 개인정보 규제 기관의 기대치를 준수하기 위해 특히 고객 대면 플랫폼에서 AI 결과물의 정확성을 보호하고 계약상의 위험을 해결하는 데 집중해야 합니다.

데이터 처리 및 추적 기술에 대한 이해 부족은 AI 기업에게 투명성 문제를 야기합니다.

데이터 주체에 대한 투명성 문제를 해결하고 개인정보 보호 규정을 탐색하는 것은 AI 개발자에게 중요한 과제입니다.

다가오는 정부 규제와 ICO의 AI 사용 증가는 이 문제에 대한 관심이 점점 더 커지고 있음을 나타냅니다.