마크 저커버그의 제너레이티브 AI 및 메타 AI 스튜디오에 대한 비전

### 마크 저커버그, 시그라프 2024에서 AI와 메타의 미래에 대해 논하다
* 🤖 저커버그는 즉석에서 콘텐츠를 만들거나 기존 정보를 합성하는 도구를 통해 **AI가 생성한 콘텐츠가 표준**이 되는 미래를 상상합니다.
* 🧠 그는 **다양한 유형의 콘텐츠를 포괄할 수 있는 보다 일반적인 추천 모델**을 구축하여 품질을 크게 개선하는 것이 중요하다고 강조합니다.
* 🎨 '크리에이터 AI'라는 개념이 도입되어 **크리에이터가 자신의 AI 버전을 구축**하여 잠재 고객과 상호 작용할 수 있게 됨으로써 콘텐츠 제작에 혁신을 가져올 수 있습니다.
* 🌐 저커버그는 오픈 소스 철학의 중요성을 강조하며, **AI에 대한 접근성을 높이고 Meta의 미래 기술이 폐쇄적인 플랫폼에 의해 제한되지 않도록 보장**합니다.
* 👓 또한 주커버그는 **10억 명 이상의 사용자가 스마트 글래스로 업그레이드**할 것으로 예상하면서 차세대 주요 컴퓨팅 플랫폼으로 여겨지는 스마트 글래스의 발전 방향에 대해서도 논의했습니다.
* 🏢 이 대담은 젠슨 황이 특히 **'무엇이든 분할하기'와 같은 컴퓨터 비전 애플리케이션에서 메타의 AI 기술이 미치는 영향**에 대해 칭찬하고 **레이 트레이싱을 넘어선 미래의 AI 프로젝트**에 대해 논의하는 것으로 마무리됩니다.

마크 저커버그가 AI의 진화, 오픈 소스 철학, AI 스튜디오와 스마트 글래스의 미래에 대해 이야기합니다.

Jensen hang sat down with Mark Zuckerberg at sigraph 2024 and discussed a range of artificial intelligence topics. I made a super cut of it so let's check it out. I guess my first question for you is, you know, how do you see how do you see the uh uh the advances of generative AI ad meta today and how do you apply it to either enhance your operations or introduce new capabilities that you're offering with generative AI. Um, I think we're going to quickly move into this Zone where not only is is the majority of the content that you see today on Instagram you know just recommended to you from kind of stuff that's out there in the world that matches your interests whether or not you follow the people. I think in the future a lot of the stuff is going to be created with these tools too. Some of that is going to be creators using the tools to create new content. Some of it, I think, eventually is going to be content that's either created on the Fly for you.

um, or, or, or kind of pull together and synthesize through different things that out there, so that that's just one example of how kind of the core part of what we're doing is just going to evolve, and it's been evolving for for 20 years already, but I think very few people realize that that, uh, one of the largest Computing systems the world has ever conceived of is a recommender system. Yeah, I mean it's this whole, yeah, it's this whole different path, right? It's, it's not quite the kind of gen hotness that people talk about, but I think it's, it's like, as I mean, it's all the Transformer architectures, and it's a similar thing of just building up more and more General models, embedding unstructured data into features, and yeah, I mean one of the big things that just drives quality improvements is, you know, it used to be that you'd have a different model for each type of content, right? So, a recent example is, you know, we had, you know, one model for ranking and

recommending reels and another model for ranking and recommending more long-form videos. And then, you know, take some product work to basically make it so that the system can display, you know, anything in line. But, you know, the more you kind of just create more general recommendations models that can span everything, it just gets better and better. So, I mean, part of it, I think, is just like economics and liquidity of content. And the broader of a pool that you can pull from, you're just not having these weird inefficiencies of pulling from different pools. But, yeah, I mean, as the models get bigger and more general, that gets better and better. So, I kind of dream of one day like you can almost imagine all of Facebook or Instagram being, you know, like a single AI model that has unified all these different content types and systems together that actually have objectives over different time frames, right? Because some of it is just showing you, you know, what's the interesting content that you're going to be that you want to see today. But some of it is helping you build out your

network over the long term. Right people you may know or accounts you might want to follow. And these multimodal models tend to be, yeah, tend to be much better at recognizing patterns, weak signals, and such. A lot of the Gen stuff is going to, on the one hand, it's, I think, going to just be this big upgrade for all of the workflows and products that we've had for a long time, but on the other hand, there's going to be all these completely new things that can now get created. So Meta AI, um, you know, the idea of having, you know, just an AI assistant that can help you with different tasks and, um, in our world, is going to be, you know, very creatively oriented like you're saying, but I mean they're very general, so I, you don't need to just constrain it to that. It'll be able to answer any question, um, over time. I think, you know, when we move from like the Llama 3 class of models to Llama 4 and beyond, it's going to, I think, feel

less like a chatbot, where it's like you give it a prompt and it just responds, then you give it a prompt and it responds, and it's just like back and forth. I think it's gonna pretty quickly evolve to you give it an intent and it actually can go away on multiple time frames. And I mean it probably should acknowledge that you gave it an intent upfront, but I mean, you know, some of the stuff I think will end up, you know, it'll spin up, you know, compute jobs that take, you know, weeks or months or something, and then just come back to you. And like something happens in the world, I think that that's going to be really powerful. So I mean, today's AI, as you know, is kind of term-based. You say something, it says something back to you. Um, but obviously when we think, when we're given a mission or we're giving a problem, you know, we'll contemplate multiple options or maybe we come up with a, you know, a tree of options, a decision tree.

and we walk down to the decision tree, simulating in our mind, you know, what are the different outcomes of each decision that we could potentially make. And so we were doing planning, and so in the future AIS will kind of do the same. One of the things that I was super excited about when you talked about your vision of Creator AI, I just think that's a home run idea, frankly. Tell everybody about the Creator AI and AI Studio that's going to enable you to do that. Yeah, so, so we actually, I mean this is something that we're, we're, you know, we've talked about it a bit, but we're rolling it out a lot wider today. You know, a lot of our vision is that I don't think that there's just going to be like one AI model, right. I mean this is something that some of the other companies in the industry, they're like, you know, it's like they're building like one Central agent and and yeah, we, we'll have the meta AI assistant that you can

use but a lot of our vision is that we want to empower all the people who use our products to basically create agents for themselves. So, whether that's, you know, all the many, many millions of creators that are on the platform or, you know, hundreds of millions of small businesses, um, we eventually want to just be able to pull in all your content and very quickly stand up a business agent and, um, be able to interact with your customers and, you know, do sales and customer support and all that. So, the one that we're, that just starting to roll out more now is, um, we call it AI studio, and it basically is, um, a set of tools that eventually is going to make it that every Creator can build sort of an AI version of themselves, um, as, as sort of an, an agent or an assistant that their Community can interact with, and know it's, it's going to be very clear that it's not engaging with the Creator themselves, but I think it'll be another interesting way just like how creators put out content on on these, um, social

systems to be able to have agents that do that. I think that there's going to be a thing where people basically create their own agents for all different kinds of uses. Some will be sort of customized utility things that they're trying to get done, that they want to kind of fine-tune and train an agent for. Some of them will be entertainment, and some of the things that people create are just funny, you know, and just kind of silly in different ways, or kind of have a funny attitude about things that, um, you know, we probably couldn't, we probably wouldn't build into meta AI as an assistant. But I think people, um, people are kind of pretty interested to see and interact with. And then one of the interesting use cases that we're seeing is people kind of using these agents for support. This was one thing that was a little bit surprising to me. One of the top use cases for meta AI already is people basically using it to roleplay difficult social situations that they're going to be in. So whether

it's a professional situation. It's like, All right, I want to ask my manager, like, how do I get a promotion or raise? Or I'm having this fight with my friend, or I'm having this difficult situation with my girlfriend. Like, how, like, how can this conversation go? And basically having a completely judgment-free zone where you can basically role-play that and see how the conversation will go and get feedback on it. Um, but a lot of people, they don't just want to interact with the same kind of, you know, agent, whether it's met AI or chat GPT or whatever it is that everyone else is using. They want to kind of create their own things. So that's roughly where we're going with AI Studio, but it's all part of this bigger, I guess, view that we have that there shouldn't just be one big AI that people interact with. We just think that the world will be better and more interesting if there's a diversity of these different things. I just think it's so cool that if

you're an artist, and you have a style. You could take your style, all of your body of work. You could fine-tune, yeah, one of your models. Yeah, and now this becomes an AI model that you can come, and you could prompt it. You could ask me to, uh, you know, create something along the lines of the art style that I have, and you might even give me a piece of art as a drawing, a sketch as an inspiration, and I can generate something for you. It could be, it could be a, uh, every single, uh, every single restaurant, every single website will probably in the future have these AIS, yeah. I mean, I, I kind of think that in the future, just like every business has, you know, an email address and a website and a social media account or several, I think in the future every business is going to have an AI agent that interfaces with their customer. So can I, can I use AI Studio to F tune with my images, my collection of images, yeah, you're, yeah, we're going to get there.

okay, and then I could, can I give it, load it all the things that I've written so that use it, use it as my rag, yeah, yeah, yeah, basically. Okay, and then every time I come back to it, it loads up the its memory again so it remembers where it left off last time, yep, and we carry on our conversation as if nothing ever happened. Tell me about your open source philosophy, where did I come from, and you know you open source py porch, yeah, and that it is now the framework by which AI is done, and now you've open source llama 3.1 or llama, there's a whole ecosystem built around it, and so I think it's terrific, but where did that all come from, yeah. So there's a bunch of history on a lot of this, I mean we've done a lot of Open Source work over time, um, I think part of it, you know, just bluntly is you know we got started after some of the other tech companies right in building.

out stuff like the distributed computing infrastructure and the data centers. And you know, because of that, by the time that we built that stuff, it wasn't a competitive advantage. So we're like, All right, we might as well make this open, and then we'll benefit from the ecosystem around that. So we had a bunch of projects like that. I think the biggest one was probably Open Compute, where we took our server designs, the network designs, and eventually the data center designs, and published all of that. And by having that become somewhat of an industry standard, all the supply chains basically got organized around it, which had this benefit of saving money for everyone. So by making it public and open, we basically saved billions of dollars from doing that work. Open Compute was also what made it possible for NVIDIA HGX. Is that we designed for one data center all of a sudden works in every data center. Awesome. So I'd say by the time that llama came around, we were sort of positively predisposed towards doing

this for for AI models specifically. I guess there's a few ways that I look at this. I mean, one is, you know, it's been really fun building stuff over the last 20 years at the company. Um, one of the things that has been sort of the most difficult has been kind of having to navigate the fact that we ship our apps through our competitors' mobile platforms. So, on the one hand, the mobile platforms have been this huge boon to the industry. That's been awesome. Um, on the other hand, having to deliver your products through your competitors is challenging. When you look at these generations of computing, there's this big recency bias where everyone just looks at mobile and thinks, Okay, because the closed ecosystem, because Apple basically won and set the terms of that. And like, yeah, I know that there's more Android phones out there technically, but like, Apple basically has the whole market. I think in general for the computing platforms that the whole industry is building on, there's a lot of value for that if the software

especially is open, so that's really shaped my philosophy on this. And um, both AI with Llama and with the work that we're doing in AR and VR, where we're basically making the Horizon OS that we're building for mixed reality, um, an open operating system in the sense of what Android or Windows was. And basically making it so that um, like we're going to be able to work with lots of different hardware companies to make all different kinds of devices. We basically just want to return the ecosystem to that level where that's going to be the open one. And I, I, I'm pretty optimistic that the next generation, the open ones are going to win for us specifically. Um, you know, I just want to make sure that we have access to. I mean, this is sort of selfish, but you, I mean, it's, you know, after building this company for a while, um, one of my things for the next 10 or 15 years is like I just want to make sure that we can build.

the fundamental technology that we're going to be building social experiences on because there have just been too many things that I've tried to build, and then have just been told, nah, you can't really build that by the platform provider. That at some level I'm just like, nah, that for the next generation. Um, like we're going to go build like all the way down, and make sure that we there goes our broadcast opportunity. Yeah, no, sorry, give me, give me talking about closed platforms, and I get angry. Um, so, um, you know, I do think there's this alignment where we're building it because we want the thing to exist, and we want to not get cut off from some closed model, right? And um, but it this isn't just like a piece of software that you can build. It's, you know, you need an ecosystem around it. And so it's almost like it kind of almost wouldn't even work that well if we didn't open source it, right? It's, it's

not we're not doing this because we're kind of altruistic people um even though I I I think that this is going to be helpful for the ecosystem. And we're doing it because we think that this is going to make the thing that we're building the best by by kind of having a robust ecos around. Look how many people contributed to PyTorch ecosystem. Yeah, totally. And I recognize an important thing. I recognize an important thing. And and I I think the Llama is genuinely important. We built this concept to call an AI Factory AI Foundry around it uh so that we can help everybody build take you know a lot of people they they they have a desire to um uh uh build AI. And it's very important for them to own the AI because once they put that into their their flywheel their data flywheel that's how their company's institutional knowledge is encoded and embedded into an an AI. So they can't afford to have that AI flywheel the data flywheel that experienced flywheel somewhere else so.

and and so open source allows them to do that. But they don't really know how to turn this whole thing into an AI. So we created this thing called an AI Foundry. We provide the tooling, we provide the expertise, uh llama, uh technology. We have the ability to help them, uh turn this whole thing, uh into an AI service. And then when, when we're done with that, they take it, they own it. We, the output of it, what we call a Nim, and this Nim, this, this neurom micro Nvidia inference microservice, uh they just download it, they take it and they run it anywhere they like, including on Prem. And we have a whole ecosystem of Partners, uh from oems that can run the NIMS to, uh gsis like Accenture that, that we've trained and work with to create llama based Nims and, and, uh and pipelines and, and now we're, we're off helping Enterprises all over the world do this. I mean it's really quite exciting thing. It's really all triggered off of, uh the Llama open

sourcing you know one of the things that I really love about the work that you guys do computer vision um uh one of the models that we use a lot internally uh is segment everything. The segment anything model that that you're talking about we're actually presenting I think the next version of that here at at sigraph segment anything to um and it is it now works it's faster it works with um oh here we go um it works in video now as well. Yeah so it's um a lot of fun effects will be able to be made with this and because it'll be open a lot of more serious applications across the industry too so I mean scientists use stuff to you know study um like coral reefs and natural habitats and um and kind of evolution of Landscapes and things like that but I mean it's uh being able to do this in video and having it be zero shot and be able to kind of interact with it and tell it what you want to track is um it's it's

it's pretty cool research. Now what else, what else are you guys going to work on Beyond uh Ray talk talk to me about. Yeah, so there's all the smart glasses, right? So I think when we think about the next Computing platform, you know, we kind of break it down into mixed reality, the headsets, and the smart glasses. And the smart glasses, I think it's easier for people to wrap their head around that and wearing it because it's, you know, pretty much everyone who's wearing a pair of glasses today will end up that'll get upgraded to smart glasses. And that's like more than a billion people in the world. So that's going to be a pretty big thing. The VMR headsets, I think some people find it interesting for gaming or different uses, some don't yet. My view is that they're going to be both in the world. I think the smart glasses are going to be sort of the mobile phone kind of always on version of the next Computing platform. And the mixed reality headsets are going to be

more like your workstation or your game console, where when you're sitting down for a more immersive session and you want access to more compute. I mean, look, I mean the glasses are just very small form factor. Um, there're going to be a lot of constraints on that, just like you can't do the same level computing on a phone. It came at exactly the time when all of these breakthroughs in generative AI happen, yeah. So we, we basically, for smart classes, we've been, we've been going at the problem from two different directions. On the one hand, we've been building what we think is sort of the technology that you need for the kind of ideal holographic AR glasses, and we're doing all the custom silicon work, all the custom display stack work, like all the stuff that you would need to do to make that work. And they're glasses, right? It's not a headset, it's not like a VR or MR headset. They look like glasses, but um, there's still quite a bit far off from the glasses that you're wearing now.

mean those are very thin, but even the Ray-Bans that we make, you couldn't quite fit all the tech that you need to into that yet for kind of full holographic AR, though we're getting close. The other angle that we've come at this is let's start with good-looking glasses. We have camera sensors, so you can take photos and videos. You can actually live stream to Instagram. You can take video calls on WhatsApp and stream to the other person. You know what you're seeing. I mean, it has a microphone and speaker. So, I mean, the speaker is actually really good. Like it's open ear, so a lot of people find it more comfortable than earbuds. You can listen to music, and it's just like this private experience that's pretty neat. People love that. You can take phone calls on it, but then it just turned out that that sensor package was exactly what you needed to be able to talk to AI too. It's great that we

have this but um but in the future, we're like not that many years away from being able to have a virtual meeting where like, you know, it's like I'm not here physically, it's just my hologram, and like, it just feels like we're there and we're physically present. We can work on something and collaborate on something together, but I think this is going to be especially important with application. I could live with a device that I'm not wearing all the time. Oh, yeah, but I think we're going to get to the point where it actually is, yeah. I could. It'll be, I mean, there's with within glasses, there's like thinner frames and there's thicker frames and there's like all these Styles, but um, so I don't I think we're we're a while away from having full holographic glasses in the form factor of your glasses, but I think having it in a pair of stylish, kind of chunkier framed glasses is not that far off. These sunglasses are face size these days. I could see that, yeah.

and and you know what that's that's a very helpful style TR exactly that's a very helpful St whoever you know it's like like I'm I'm trying to you know I'm trying to like make my way into becoming like a style influencer so I can like influence this before um you know before the glasses come to the market but you know well I can see you attempting it how's your style influencing work out for you you know it's early yeah but I don't know I feel like if if if a big part of the future of the business is going to be building um kind of stylish glasses that people wear um this is something I should probably start paying a little more attention to ladies and gentlemen Mark Zuckerberg thank you

Zuckerberg and Jensen Discuss The Future of AI

Artificial Intelligence (AI), Open Source, Generative Art, AI Art, Futurism, ChatGPT, Large Language Models (LLM), Machine Learning, Technology, Coding, Tutorials, AI News, and more

** Exclusive Pinecone Partner **

Matthew Berman

마크 저커버그의 시그그래프 2024에서의 제너레이티브 AI에 대한 비전

마크 저커버그가 제너레이티브 AI의 미래와 Instagram 콘텐츠 제작에 적용하는 방법에 대해 이야기합니다.

또한, 추천 시스템의 진화와 보다 일반적인 추천 모델을 통한 품질 개선에 대해 이야기합니다.

주커버그는 다양한 콘텐츠 유형과 시스템을 아우르는 Facebook과 Instagram의 통합 AI 모델을 구상하고 있습니다.

AI의 미래와 메타 AI 어시스턴트

메타 AI는 고급 멀티모달 모델을 통해 워크플로와 제품 제작에 혁신을 가져올 것으로 기대됩니다.

AI의 미래는 즉각적인 상호 작용에서 장기적인 계획과 다양한 결과 시뮬레이션으로 진화할 것입니다.

크리에이터 AI와 AI 스튜디오는 여러 AI 모델이 함께 작동하는 비전을 제시하며 널리 구현될 예정입니다.

메타의 AI 스튜디오와 사용자 제작 에이전트의 미래

메타의 AI 스튜디오는 수백만 명의 크리에이터와 소규모 비즈니스가 고객과의 상호작용을 위한 자체 AI 에이전트를 구축할 수 있도록 지원하는 것을 목표로 합니다.

이 스튜디오를 통해 사용자는 맞춤형 유틸리티 에이전트, 엔터테인먼트 에이전트, 심지어 사회적 상황을 연기하는 에이전트까지 만들 수 있습니다.

이러한 AI 에이전트의 놀라운 사용 사례 중 하나는 어려운 사회적 시나리오를 역할극으로 만들어 사용자가 연습하고 피드백을 받을 수 있는 판단력 없는 영역을 제공하는 것입니다.

오픈 소스 철학이 AI 개발에 미치는 영향

AI 개발의 오픈 소스 철학은 이전에 개발된 기술을 대중에게 공개하기로 한 결정에 그 뿌리를 두고 있습니다.

이러한 접근 방식은 작업을 공개하고 공개하면 주변 생태계의 혜택을 받고 수십억 달러를 절약할 수 있다는 깨달음에서 비롯되었습니다.

서버 설계와 네트워크 설계를 공개한 오픈 컴퓨트 프로젝트는 결국 업계 표준이 되어 모든 사람의 비용 절감으로 이어졌습니다.

모바일 플랫폼이 앱 전송에 미치는 영향과 개방형 시스템으로의 전환

연사는 경쟁사의 모바일 플랫폼이 업계에 가져다주는 이점에도 불구하고 경쟁사의 모바일 플랫폼을 통해 앱을 제공할 때의 어려움에 대해 설명합니다.

또한 현재의 폐쇄적인 생태계와 달리 AI, AR, VR 기술을 위한 개방형 운영체제를 만드는 것이 중요하다고 강조합니다.

폐쇄적인 모델에서 벗어나 다양한 디바이스를 개발할 수 있는 개방적인 기술 생태계를 구축하는 것이 목표입니다.

라마는 AI를 구축하고 기관 지식을 인코딩하는 데 매우 중요하며, AI 파운드리는 툴링과 전문 지식을 제공합니다.

이 프로세스의 결과물은 어디서나 실행할 수 있는 뉴로마이크로 Nvidia 추론 마이크로서비스인 Nim입니다.

라마의 오픈 소스 모델인 "Segment Anything"은 더 빠르게 작동하고 동영상과 함께 작동하도록 업그레이드되어 자연 서식지 및 경관 연구에 다양하게 응용될 수 있는 잠재력을 확장하고 있습니다.

컴퓨팅에서 스마트 글래스의 미래

스마트 글래스는 전 세계 10억 명 이상의 사용자에게 보급되어 주요 컴퓨팅 플랫폼으로 자리 잡을 것으로 예상됩니다.

이 안경은 작고 스타일리시한 폼 팩터에 라이브 스트리밍과 통화부터 AI와의 상호 작용까지 다양한 기능을 제공하도록 개발되고 있습니다.

이 연구는 이상적인 홀로그램 AR 안경을 위한 기술 발전에도 초점을 맞추고 있지만, 필요한 모든 기술을 얇고 스타일리시한 디자인에 적용하는 것은 아직 진행 중인 작업입니다.

가상 회의의 미래: 홀로그램 안경

가까운 미래에는 홀로그램 안경을 이용한 가상 회의가 현실화되어 실제로 현장에 가지 않고도 물리적 존재감을 느낄 수 있게 될 것입니다.

스타일리시한 홀로그램 안경의 개발로 사람들의 상호 작용과 협업 방식이 변화할 것으로 예상됩니다.

미래형 안경 디자인에 스타일을 접목하는 것이 중요하다는 인식이 확산되고 있으며, 이는 안경 업계가 보다 세련된 안경으로 변화할 가능성을 시사합니다.