[MUSIC PLAYING] WEI WEI: Hello. Thank you for joining this technical session at Google I/O 2024. My name is Wei, and I'm a developer advocate at Google. Today, my colleague Shrestha and I will be talking to you about the exciting topic of AI-assisted coding. Generative AI is transforming many industries today. The software industry is no exception, and many practitioners are already benefiting from adopting AI. For example, in a typical software development life cycle, AI could help you analyze user requirements and write PRDs in the planning phase. While you're designing the architecture of your application, you could brainstorm with AI and come up with new, interesting solutions. When you start coding, AI could help you code faster and better. When you finally release your application to the world, you could use AI to monitor usage and analyze user feedback for future developments.

Let's tune in on the coding phase. AI could automate away
many tedious tasks, so that you have more time to
work on the more creative parts of your workflow. You may also use AI
as a coding buddy to review or critique
your code, and it will improve your code quality. Sometimes AI could even bring
more creativity to the table by suggesting alternative
or even better implementations for you. Ultimately, taking advantage
of AI coding assistance could lead to higher
productivity and more developer happiness, which is why when
Stack Overflow released their annual developer
survey last year, they reported that
70% of the respondents were already using or planning
to use AI in their development process. And 77% of the
respondents were in favor of AI tools for development. That's a lot of
developers in action. This year, the numbers
may be even higher. So if you still haven't
used AI for coding, now is a good time
to get started.

AI coding assistance covers a wide range of use cases in your development process, from code completion, code generation, and Code Chat, to code explanation and error debugging, and so many more. These use cases are well covered by a number of Google products since we have long been working on AI and developer tools. So let me demonstrate how to work. First, our coding workspaces or IDEs, such as Colab, project IDX, Vertex AI, Android Studio, and Chrome DevTools, have all incorporated coding AI. For example, here we are in Google Colab, which is a hosted Jupyter Notebook service that provides computing resources, including a usage tier available free of charge. You can use the AI-generated suggestions in Colab for code completion based on your current coding context. Let's start by creating a simple function to compute a circle's area. As I type the function name, the building AI

is able to guess what I want
and suggest implementation without me even asking. How convenient this is? I can simply hit Tab to
accept the suggestion or escape to ignore it. What if I want a class
instead of a function? I can type the
first few letters, and AI is again able to
figure out what I want and have the circle
class defined. I like this class, but
in the calculation, it uses only 3.14, which
is lower precision for pi. Let me change that. I know the math library has
a pi value defined in higher precision, so I import that. As I'm typing, AI
predicts I want to swap out 3.14 for [INAUDIBLE]
pi and all I need to do is to hit Tab. Next, let's run a simple
test on this new class. I define a list of circles based
on the AI-generated suggestions, but I don't like
the default values.

So I manually change them. I start writing a for loop, and immediately, AI predicts that I want to bring out the area value for each circle, and that's it. I accept this code snippet as well, and I run the last two code blocks to verify the results, and they look great. As you can see here, I have barely typed 30 characters, and AI is able to figure out my intentions and finish what I want to build. Code generation is another important use case. Instead of manually writing code, I can simply specify the functionality I want in natural language and let AI build it for me. This time, I'm using Google's Project IDX, a web-based workspace that enables full stack multi-platform app development workflow on the cloud. In IDX, I'm using command-I on my Mac to invoke the inline AI and to prompt it to write a function to find the non-common elements of two lists.

without using other data structures. IDX follows my instructions smoothly and generates the function I want. I can then accept or discard the code. To make sure my function here is solid, we need to write some unit tests, which sometimes feels like a chore to many developers. With AI, it doesn't have to be like that. I simply ask AI to add three unit tests, and voila, it is done automatically. Now, I think our function is more or less ready. So let me put a finishing touch on it by adding documentation. Writing documentation sometimes is tedious, but it is very important for readability and maintainability. So we got to do it. Similar to unit tests, all I need to do is to ask IDX AI to add docstring and it automatically populates Python documentation for our function without a hitch. Sometimes you inherit a code from someone else and you need to be able to understand the code.

Here we see an example of code explanation on Gemini Code Assist. Gemini Code Assist is available in multiple IDEs such as Visual Studio Code, GEDI prem IDEs, cloud workstations, Cloud Shell Editor, and supports more than 20 programming languages, including Go, Java, Javascript, Python, and SQL. And sometimes the code works, but you want to make it more optimal in some way, maybe more performance, maybe more secure, maybe more idiomatic, maybe just simpler. In this case, we see an example where the underlying code was translated from Java to Kotlin using the built-in IntelliJ functionality. However, if you want to make the code more idiomatic, we ask for that transformation in natural language and we can see from the resulting code that it follows the cross-link style better-- event syntax, string templates, smart casts, and so on. Have you ever had any experience when

the console just
throws up an arrow, and you have no clue about it? Like here in Chrome
DevTools, I have no idea what's causing the issue in my
Fibonacci Number implementation. But I get to ask the building
AI to explain this error, and I immediately get
insight to help me debug. It seems my recursive calls
are not properly terminated. So with that hint,
I can easily fix it. Now I'll pass it to
my colleague Shrestha to explore even more
use cases with you. Hi,. I'm Shrestha Basu Mallick. I'm a group product manager at
Google working on generative AI. Now I'll talk to you about
the next set of products. The first product
we'll talk about is Search Generative
Experience, also known as SGE. Historically, developers have
used Google search extensively to ask their
implementation questions, to find out when there are
errors in their code, what

to do, or to find documentation, and sometimes for more open-ended pursuits, such as researching and architecture decision, or learning a new skill. We have brought generative AI into Google Search through SGE. You can ask questions now and get an answer with code snippets, natural language explanations, and relevant code examples. In certain cases, the code is executed to show the results, as you can see here. The next product that we will be talking about is Gemini Chat. Developers can also ask their code-related questions here and get back code answers, use this to refactor their code, use this to explain code for them, and generally chat about anything code-related. Here in this example, you can see that Gemini Chat not only executes the code for you, but also lets you get in there, edit your code, and rerun that code. The next product we'll be talking about is AI on developers.google.com, which is Google's main developer

documentation website, which is visited by millions of developers. We have added AI capabilities to simplify the developers' journey so that the answers they get is a combination of generative AI as well as authoritative documentation from subject matter experts. Here on developers.google.com, you will have AI search which allow developers to quickly grasp key information from search results, Documentation Chat where developers can interact with the conversational AI assistant for focused guidance regarding particular developer products and pages, and AI code Explain, which provides a deeper understanding of code samples through AI explanations. Looking ahead, we plan to expand these AI features to other Google Developer properties. The products mentioned so far are all built on Google's cutting-edge foundation models. For developers who directly want to build applications on top of our code models, we have also released Code Gemma, which is our latest open code model, specifically targeted towards coding tasks. Code Gemma is a family of open code models based on Google

DeepMind's Gemma models. Continuing from the
base Gemma models, Code Gemma models were
further trained on up to 500 billion tokens of mostly code. They use the same architecture
as the Gemma model family. These models achieve
state-of-the-art performance in both code completion
and code generation tasks, while maintaining strong
understanding and reasoning skills. Code Gemma comes in two
different size variants, 2 billion and 7 billion. The smaller 2 billion model
is the pre-trained model with lightning-fast
speed, and it's perfect for the code
completion use case. The bigger 7 billion model has
two flavors, the base model and the instruction-tuned model. The 7 billion model requires
more computational resources, but you can still
use the 7 billion pretrained code variant
for code completion if you have a powerful machine. The instruction-tuned
7 billion model is better suited for
the Code Chat use case.

Here you see the example code
to use Keras to run CodeGemma. We first load the CodeGemma
instruct 7b model with the from_preset function, and
then call the generate function to get the prediction. This is similar to how you would
use the base Gemma instruct model, except that we are
passing code in the prompt, and we get a CodeGemma
prediction as inference. If you plan to use CodeGemma
to build core capabilities within your coding
environment, a simple inference will not be sufficient
in certain cases. You will need to
know about techniques such as Fill In the
Middle-- or FIM, and Retrieval Augmented
Generation, or RAG. Fill In the Middle
is a technique that uses language model to
impart infilling capabilities by packing both the left
context and the right context separated by special tokens. What we mean by this is when
you're using code completion

inside of an IDE, you
sometimes have code above where you're
doing the completion, and below where you're
doing the completion. The above is the
left context here; the below is the
right context here. And you pack both of
those into the prompt. Feel free to check out the
CodeGemma technical report to learn more about this. Retrieval Augmented
Generation-- or RAG-- is another technique
which is used to make the LLM answers either
more accurate or more relevant. It can also sometimes
help when you have to deal with the use case
where you need a large context, such as using or trying to get
an answer based on a large code base. For retrieval
augmentation, what you do is you take
the code base, you break it into chunks of code. Sometimes you can convert these
codes in code chunks of code into embeddings, and you
store it in a vector database. Then at inference time, you
retrieve the relevant snippets

of code, and you add it to your prompt, and you add it to your query, and you pass that as a prompt to the model. This ensures that the answers you get back are potentially more accurate or more relevant. In summary, Google is not only offering you code AI innovation across a lot of surfaces, we're also making our open modules available to you to build your apps on top of them. We have built all our code AI features to serve developers of all stripes, be they web developers, mobile developers, enterprise developers, or even developers of all experience levels, such as from beginners to experts. A lot goes into developing these features. We start with the right task. Sometimes a task may require building a model from scratch. Sometimes we may fine-tune on top of our base models. It all depends on the task. For example, for code completion, you may want to use a smaller model because you need

to have lightning-fast speed. On the other hand, if you're
doing code AI inside of Colab, you need to make sure that
the response tile adheres to the notebook coding format. Or, if you're building code
AI features in Android Studio, then you may want to make
sure languages that Android developers care about,
such as Java and Kotlin, are well-represented. When we start
building these models, the first thing we have to
think about is the data. We use high-quality,
permissively-licensed data, and we make sure that
we sanitize the data from all sensitive information. The next step is pre-training,
and specifically, if we are pre-training for
a specific domain, then we would do domain-specific
pre-training. In this case, we train models
on more tokens of code. The next step is
instruction tuning. Instruction tuning
is to make sure that the answer adheres to
a specific style or format.

For example, if you are using Code Chat, then you want your responses to be a little chatty. The next one is offline evaluations. Here we do a combination of human evaluations as well as automatic evaluations. And finally comes the integration step where we integrate the model with the other components of the LLM building blocks, such as serving configurations, monitoring, safety guardrails, and, of course, the UX. A lot of magic also happens in this post-processing step. For example, when you're outputting a code completion response, you need to know where to truncate the response. You may do it based on what's known as a stop token-- a specific token that you decide in advance-- or you do it based on, say, how much of the response overlaps with the suffix, or what we call the direct context. So a lot of decisions like this are made in the post-processing.

UX, you've already seen examples in Wei's section. You sometimes can interact with the models through chat, sometimes within your workflow inside of your coding environment. And finally, we need to talk about safety guardrails. A lot of attention is paid to safety. We make sure first when we are coming up with the data, we make sure the data is properly scrubbed and debiased. We also take proper anonymization into consideration. Then when we have safety filters in the API to safeguard against unsafe responses, and we also show citations. Whenever we have code that requires showing the source, we make sure to responsibly show that source. Enterprise users even can block out code that requires citations. As we look into the future, we are thinking about code agents. Here we show you an example of a data science agent that we are building. The things we want to highlight about this data science agent

is that it can build notebooks
from scratch, with detailed plans and complex reasoning. It supports a bunch of data
tasks, such as data cleaning, exploratory data
analysis, getting insights from data, statistical analysis,
and predictive modeling. In the spirit of experimenting
with the user experience, we will provide the user,
not just with the end result, but also provide an insight
into how the agent is doing reasoning and planning. And this will be provided
via a web app format, as well as in the final Colab
that you see on the screen here. This Colab that you see is
entirely generated by the agent. We want to make code innovations
like this available early to users through a
Developer Playground, so that we can test experiments
related to functionality, related to user experience,
and other aspects of code development. In this next phase
of AI, we believe generative AI will become
more mature and ubiquitous. The future of software
development, we believe,

will involve empowering
developers in AI to collaborate on creative
problem-solving and innovation. We are super excited
about this future and can't wait to build
it together with you. With that, thank
you for watching this technical session
at Google I/O 2024, and happy coding with AI. [MUSIC PLAYING]

How to elevate software development with AI-assisted coding

Subscribe to join a community of creative developers and learn the latest in Google technology — from AI and cloud, to mobile and web.

Explore more at developers.google.com

