[MUSIC PLAYING]

BRANDON DEROSIER: Hi. I'm Brandon. I'm a software engineer
working on Flutter Engine.

LEIGHA JARETT: And I'm Leigha,
a product manager for Flutter. If you tuned into
Flutter Forward or you've been reading
our release blogs, you've probably seen the
name Impeller thrown around quite a bit. But what actually is Impeller? And how does it
work under the hood? Well, today you're
going to find out. Impeller is a new renderer
within Flutter's Engine. Until now, Flutter has been
using something called Skia. The problem is that Skia
wasn't designed for Flutter. It has a ton of
rendering features built for a wide range
of devices, which means that it's not always
optimized for Flutter's needs. Enter Impeller,
Flutter's new renderer. We built Impeller to
specifically focus on the rendering needs
of Flutter applications.

And our main goal
was to eliminate jank or any stuttering that's
happening inside of your app. This way, it'll always look and
feel great for your end users. Now, for everyone who's watching
who's not a graphics engineer, you might be wondering,
what's a renderer. Well, a renderer
is software that helps you translate your
UI code into the pixels that you actually
see on the screen. So, say you have a simple
Flutter app like the one that I'm showing here. In the Flutter framework,
your tree of widgets is backed by a tree
of render objects. Render objects contain
the instructions for how to actually layout
and paint the widgets. These instructions are
given to the engine and stored in an ordered list
of simple commands called a display list. And now, this is where
things get interesting. The engine is
responsible for using one of the available renderers,
either Impeller or Skia.

to draw this display list
to a surface texture, or a grid of pixel values that
can be displayed to the screen. It leverages the
GPU by setting up a collection of things
called render pipelines that can be used to render
everything in the display list. So let's dig into the details. Before we can use
a render pipeline, we first need to take all the
paths drawn by the display list and tessellate them
into sets of triangles. Then each vertex or
point on the triangle is passed through something
called a vertex shader. Now a shader is just
a small piece of code that gets executed on
the graphics device. And in this case, the
shaders are internal, meaning that they're a part of
the Flutter engine's code base. But us Flutter developers
can author our own shaders too so more on that later. So the vertex shader
takes the vertices that make up the Flutter logo
and it moves them to the place

on the screen where we
actually want to draw them. And now the Flutter logo
is in the right place, and we begin iterating
through all the triangles. So let's zoom in a little more. For each triangle, we figure
out the specific pixels that are inside it. And this is called
rasterization. Then a few checks are run
for each of these squares to determine if we actually
need to compute a color for it. For Impeller, these
checks are really important, so we'll be coming
back to this in a bit. So now all those pixels
that are in the triangle are passed through
a fragment shader. A fragment shader is
another snippet of code, but this time it takes
the vertex shader's output and computes a color. And like I mentioned
before, Flutter developers can override this by
creating our own shaders to create really neat effects
using the fragment program API. Anyway, back to our
render pipeline.

Lastly, the output colors are blended with the color that's already been drawn. In this case, our blending was really simple, and we just replaced the color. And that's how things get rendered on the GPU. Now over to Brandon to talk through how these render pipelines are actually implemented in Impeller. BRANDON DEROSIER: Thanks, Leigha. Impeller has a layered architecture where each layer uses the one below it to accomplish its job. Before, Leigha showed an example of how the Flutter logo widget gets converted into a display list. In this example, the display list just contains a few operations to position it in the center of the screen along with four draw path ops, one for each of the quads in the logo. So let's take this display list and draw it using Impeller. First, all of the display list operations are dispatched to something called Aiks.

Aiks is the topmost layer of Impeller. And it mainly consists of a canvas drawing API. By the way, the name Aiks is just Skia in reverse. The job of Aiks is to take high-level commands like Draw Path and Draw Image from the display list and turn them into simpler, self-contained drawing operations called entities. And this is where the next layer comes in, the entities framework. In our example, the four draw path ops from the Flutter logo each end up getting an entity of their own. Each entity contains a bunch of properties that every single drawing operation needs, like a transformation matrix that encodes the position, rotation, and scale. Every entity also gets a content object assigned to it, which contains the actual GPU instructions needed to draw the entity. There's a contents for drawing solid colors, images, gradients, text clips, everything that Flutter apps can draw. Sometimes we even have multiple specialized contents

that draw the same thing,
using different algorithms with different performance
characteristics. That way Aiks can
pick and choose the most efficient
rendering algorithm depending on the situation. In this case, all four of
the Flutter logo's quads are just filled
with solid colors, and so each of their entities
get a solid color contents assigned to them. Now we need to spell out
these instructions in a way that the GPU can understand. Because Flutter apps can run on
a bunch of different platforms, Impeller needs some kind
of translation layer to communicate with the GPU. This is where the layer
below the entities framework comes in. I'll refer to this as the
hardware abstraction layer. This layer is just
a thin abstraction that talks to the
graphics driver through various
standard graphics APIs like Metal on iOS
and Vulkan on Android. Now we're ready to start telling
the GPU exactly what to do. Remember those render pipelines
Leigha mentioned before?

This is where those
come into play. Each content uses the
hardware abstraction layer to draw itself by instructing
the GPU to execute the render pipeline containing the shaders
we actually want to use. And really, that's it. Those commands are executed
using the graphics API, and then the resulting texture
gets displayed to the screen. But there's one huge detail
I haven't covered yet. Those shaders and
the render pipeline also need to get compiled
down to instructions that the GPU can execute. And this process
is very expensive. In Skia, this
compilation process happens at runtime
right on the frame. The pipeline actually needs to
be used to render something. This usually causes the
frame to go way over budget, resulting in a
noticeable stutter. We often refer to this problem
as shader compilation jank. Impeller vastly
improves on this issue by performing the most expensive
part of this compilation ahead of time.

And this is where the last
component of impeller comes in. When the Flutter
Engine is built, all of Impeller's
shaders get compiled into bundles using Impeller's
offline shader compiler called Impeller Scene. In walking through
Impeller's architecture, you've seen that Flutter really
has complete control over how it renders graphics. Flutter isn't subject to
the architectural or design limitations imposed by
platform-specific UI toolkits because Flutter
doesn't even use them. Instead, Flutter's
engine talks directly to the graphics driver. This is what allows you to
build rich, unique apps that behave exactly how you
want on any platform. This lack of dependencies also
makes Flutter a great solution for embedded use cases like
automotive dashboards, IoT, and digital signage. LEIGHA JARETT: Now that
you have an idea of what's happening under the
hood, let's highlight some of the key
architecture decisions that we made that makes Impeller
a great renderer for Flutter.

applications. So, like Brandon mentioned,
one huge benefit that Impeller offers
is eliminating jank from compiling those
programs, called shaders. To do so, Impeller
doesn't generate shaders in the same way that Skia does. Instead, Impeller has a set of
handwritten shaders compiled in advance. But you might be worried
that initializing a bunch of precompiled shaders
could mean slower startup times or bigger Flutter app sizes. To prevent this, Impeller
uses alternative rendering techniques that leverage
a much smaller, simpler set of shaders compared to
the many specialized shaders that Skia dynamically generates. Another reason why Impeller
offers great performance is the way we implement
anti-aliasing. Anti-aliasing is the
art of smoothing out jagged edges of drawn elements,
so that they look more natural. Impeller uses
something called MSAA to perform this work, which
is present across all devices supported by Flutter. It's pretty cheap to perform.

And it produces excellent quality results on modern mobile hardware. Another thing we focused on is how Impeller implements clipping. When rendering widgets, Flutter frequently uses clip masks to cut out shapes, so it's really important that this process is efficient. Impeller specifically takes an approach that leverages hardware features to ensure that clipping is super fast. Remember earlier when I mentioned those special checks? One is for something called a stencil buffer. Now we won't go into too much detail, but basically, as Impeller draws the pixels for the clip thing, it tells the GPU to check the stencil and filter away all the pixels that aren't part of the filled shape. This means that even if a clip is really complicated, like the ones used to create these kaleidoscope animations, it's still a pretty cheap operation, so we can see some really huge performance improvements. On the left-hand side, we see Flutter pre-Impeller.

And it's rendering at about
seven frames per second. And on the right
side, Impeller renders at about 60 frames per second.

BRANDON DEROSIER:
So we just talked about some of the design
choices in Impeller. For most Flutter
developers, this means you'll see more
predictable frame times when you use Impeller. But for those that
are interested in more advanced rendering use
cases, Impeller also aims to offer a launchpad
that allows you to solve for other graphical needs. For example, earlier this
year at Flutter Forward, we showed off a proof of concept
3D scene graph built directly on top of Impeller's
hardware abstraction layer called Impeller Scene. Impeller Scene has its own
contents in the entity's framework, and it
draws 3D objects using the same kinds
of render pipelines we talked about earlier. Now there are tons of
consequential decisions that go into building renderers.

And while we can certainly build
3D renderers that are useful and make smart tradeoffs,
no one renderer can solve every
use case perfectly. And so as a next
step for this, we're actively experimenting with
ways to surface more control over rendering in
the Flutter framework so that anyone in
the Flutter community can build their own
rendering packages and create custom
render pipelines that integrates seamlessly with the
rest of the widget ecosystem. Impeller's scene is
a proof of concept and a showcase for
the kinds of developer experiences we're seeking
to enable with this effort. With that in mind, let's take
a look at the demo widgets. Basically, everything
is centered around a special widget
called scene box. Scene box takes a node,
which can be created from a 3D model that's added
to your Flutter app's asset bundle. Nodes can be composed
together to form a tree just like normal Flutter widgets.

Here I'm wrapping the asset with another node that applies a rotation. You can add the same asset to the scene as many times as you want. And Flutter will know to only load the asset once and reuse it to inexpensively render copies, kind of like how image assets work. And you can even play animations built into the asset that were authored using 3D modeling software like Blender or Maya. And since Scene box is just a regular Flutter widget, any of these node or node animation properties can also be animated in exactly the same way that you would animate anything else in your build methods. And of course, hot reload just works. Simply export the file from your 3D modeling software and Flutter does the rest. With just these tools, it's possible to construct rich interactive scenes composed from many different assets. Again, this is the kind of fluttery experience we're seeking to enable as we surface.

more control over rendering thanks to Impeller. LEIGHA JARETT: So we just gave you an overview of Impeller. And we went under the hood to show just how Flutter renders your widgets to the device screen. But that's not all. Today, we're super excited to announce that Impeller is now the default renderer on iOS for the stable version of Flutter. So upgrade Flutter and try it out. And if you want to see Impeller in action, you can install G. Skinner's Wondrous app. Right now, Wondrous uses Impeller on iOS. But we're looking forward to the next version of Wondrous, which will include Android support for Impeller. We're really excited about where Impeller is headed and the new rendering capabilities that we can start bringing to Flutter, and we hope you are, too.

Introducing Impeller - Flutter's new rendering engine

Welcome to the official Flutter YouTube channel. Subscribe to stay up to date with best practices about the Flutter SDK. See real code examples, and watch engineers from around the world putting Flutter to work! 

