IAN BALLANTYNE: Google just launched the latest in the Gemini family of multimodal large language models, Gemini 2.0 Flash Experimental.

Multimodal models like Gemini 1.5 take text, images, audio, and video as inputs. But Gemini 2.0 Flash also has the ability

to generate responses, including audio and images. With this ability, enhanced reasoning skills, and tools

such as Google Search and code execution, developers can now build applications with more advanced workflows; for example,

performing multi-step research, then visualizing the results with generated code. The exciting bit is communicating with Gemini 2.0

using the Multimodal Live API.  The Multimodal Live API is a bidirectional streaming interface, enabling users to have real-time voice

interactions and natural conversation. You can now interact with the model using real-time audio and video from a microphone,

The model can handle interruptions and spontaneous conversation flow, making the experience more interactive. Now that might be a bit tricky to imagine right now,

so let's look at some examples. This demo shows a user interacting with Gemini 2.0 Flash using the Multimodal Live API in Google AI Studio.

SPEAKER 1: Can you see my screen? SPEAKER 2: Yes, I see your screen.

It's showing a document with the title 'Demo.' It also contains bullet points and a Google Meet window

with a person in the video. Is there anything in particular you would like to talk about on the screen?

SPEAKER 1: Yeah, can you start the demo by just reading the text I highlighted?

SPEAKER 2: Sure, I can do that. The text you highlighted reads 'the Multimodal Live'

The API lets you build real-time, multimodal applications powered by Gemini 2.0 Flash.

It features tool use (Google Search and code execution), function calling, and more.

This demo shows a user having a conversation with Gemini 2.0 Flash. The app itself was built using the Multimodal Live API.

Make a bar graph comparing the runtime of "The Godfather" and "Oppenheimer."

Add the other two "Godfather" films to the graph.  Pick three random superhero movies and add them to the graph.

That's a quick demonstration of how you can have a live conversation with Gemini 2.0 Flash.

Notice that you can have natural conversations enabled by the low latency of responses.

The bidirectional streaming interface allows you to interrupt the model, letting you refine queries and iterate on discussions.

So how do you start developing with Gemini 2.0 Flash?  These examples show the code needed to access the interface.

This example is written in Python using WebSockets.

Hi. Can you list the planets in our solar system?

Sure, I can do that for you. The planets in our solar system, in order from the Sun, are Mercury, Venus, Earth, Mars, Jupiter, Saturn.

Actually, can you just list the gas planets?

No problem. The gas planets in our solar system are Jupiter, Saturn, Uranus, and Neptune.

of the new output modalities. Gemini 2.0 Flash now generates images and supports conversational multi-turn editing,

so you can build on previous outputs and refine them. It can output interleaved text and images,

making it useful for brainstorming design ideas or displaying visuals alongside step-by-step instructions.

Image output is available to early testers, with a wider rollout expected next year. Gemini 2.0 Flash features a new text-to-speech style

audio capabilities, making multilingual native audio output available for developers. What's different is that native audio has fine-grained control

over not just what the model says, but how it says it, with a choice of eight high-quality

voices and a range of languages and accents. Let's hear multilingual native audio output in action.

SPEAKER 5: What exactly is native audio? You're actually hearing it right now.

SPEAKER 6: Everything you hear in this video was generated with prompts like this actual prompt on your screen,

like right now. It was all generated by prompting the new experimental Gemini 2.0 Flash

model, like you see in this AI Studio screencap. SPEAKER 7: You can prompt it to just be like, "Dude, just

totally chill." SPEAKER 8: Or prompt it to speak with, oh, so very

many dramatic pauses. IAN BALLANTYNE: So you can see that when Gemini responds,

it generates not only the words, but the style, including emphasis, tone, and natural pauses.

It can also switch between languages in the same response. SPEAKER 9: So I'm starting out speaking English,

but then [SPEAKING FRENCH]
[SPEAKING KOREAN] IAN BALLANTYNE: Imagine how configuring audio output

in this way could make assistants more engaging. These new output modalities are available to early testers,

with a wider rollout expected next year. So these aren't the only features enabled with the launch of Gemini 2.0.

Gemini 2.0 is twice as fast as 1.5 Pro while achieving stronger performance.

The model also has improved spatial understanding, enabling more accurate bounding box generation on small objects

in cluttered images, and better object identification and captioning. Gemini 2.0 has also been trained to use tools.

Based on developer input, Gemini 2.0 can invoke and chain tools, iterate over actions,

and process results to enable more advanced workflows. It can be configured to use tools like Google Search, which

can run in parallel finding more relevant facts from multiple sources. It can then use code execution or function

calling to combine the results, leading to tailored responses with more accuracy. So if you're interested in trying out these developer

tools for building applications with Gemini's new capabilities, you can test and explore Gemini 2.0 via

the Gemini API, Google AI Studio, and Vertex AI during its experimental phase, with general availability coming

early next year. If you want to try some of the code examples shown in this video, follow the links to the docs

on Google Colab notebooks as part of the Gemini Cookbook. To see more interesting use cases of Gemini and Google AI,

follow the new Google AI for developers social channels. There are more demos and examples being announced in the coming weeks,

So subscribe and stay tuned. As always, we look forward to seeing what you'll build with Gemini.

Gemini 2.0 for developers

Subscribe to join a community of creative developers and learn the latest in Google technology — from AI and cloud, to mobile and web.

Explore more at developers.google.com



Google for Developers

Gemini 2.0 Flash Experimental 소개 및 주요 기능

텍스트, 이미지, 오디오, 비디오를 입력으로 받는 다중 모달 모델

오디오 및 이미지 생성 기능, 향상된 추론 능력, Google 검색 및 코드 실행 도구 지원

다단계 조사 및 생성된 코드를 사용한 결과 시각화 가능

Multimodal Live API를 통한 실시간 상호 작용 지원

마이크, 웹캠, 화면을 통한 실시간 오디오 및 비디오 상호 작용

Multimodal Live API 데모 및 사용 예시

화면 공유 및 문서 내용에 대한 실시간 대화

강조 표시된 텍스트 읽기 요청 및 응답

Multimodal Live API를 사용한 앱 개발 및 영화 런타임 비교 그래프 생성

자연스러운 대화, 낮은 지연 시간, 질문 수정 및 대화 반복 가능

Gemini 2.0 Flash 개발 시작 방법 및 코드 예시

Python과 WebSocket을 사용한 인터페이스 접근 코드 예시

태양계 행성 목록 요청 및 응답 (추가 질문 포함)

새로운 출력 방식: 이미지 생성 및 다국어 지원

이미지 생성, 대화형 다회차 편집 지원

새로운 텍스트 음성 변환 기능, 다국어 지원 및 다양한 음성 옵션 제공

다국어 네이티브 오디오 출력 데모

강조, 어조, 자연스러운 멈춤 등을 포함한 스타일 생성

Gemini 2.0의 추가 기능 및 성능 향상

Gemini 1.5 Pro 대비 2배 빠른 속도 및 향상된 성능

향상된 공간 이해 능력, 더 정확한 경계 상자 생성

도구 사용 기능, Google 검색, 코드 실행 및 함수 호출을 통한 고급 워크플로우 지원

Gemini API, Google AI Studio, Vertex AI를 통한 테스트 및 탐색

Google Colab 노트북의 코드 예시 및 Gemini Cookbook 링크

Google AI 개발자 소셜 채널 구독 및 추가 데모 확인