임펠러: 부드러운 그래픽을 위한 Flutter의 새로운 렌더러

Flutter의 새로운 렌더러인 Impeller는 끊김 현상을 없애고 Flutter 애플리케이션의 렌더링을 최적화합니다.

### 임펠러: 부드럽고 효율적인 그래픽을 위한 Flutter의 새로운 렌더러
* 🎨  Impeller는 Flutter 엔진의 새로운 렌더러로, 렌더링 요구 사항에 특별히 집중하여 Flutter 애플리케이션의 끊김 현상과 버벅거림을 제거하도록 설계되었습니다.
* 🔳  Flutter의 렌더러는 UI 코드를 화면에 표시되는 픽셀로 변환하는 데 도움을 주며, Impeller는 레이어 아키텍처와 사전 컴파일된 셰이더를 사용하여 성능을 향상시킵니다.
* 💡  임펠러는 대체 렌더링 기법을 사용하고 효율적인 앤티 앨리어싱 및 클리핑을 구현하여 뛰어난 성능을 제공하며, Flutter 개발자에게 고급 렌더링 사용 사례를 더 잘 제어할 수 있는 기능을 제공하는 것을 목표로 합니다.
* 🚀  임펠러의 하드웨어 추상화 레이어에 구축된 3D 씬 그래프인 임펠러 씬은 Flutter의 고급 렌더링 기능의 잠재력을 보여줍니다.
* 📱  임펠러는 이제 안정된 버전의 Flutter에서 iOS의 기본 렌더러로 사용되며, 안드로이드 지원도 계획되어 있어 새로운 렌더링 기능을 선보입니다.

[MUSIC PLAYING]

BRANDON DEROSIER: Hi. I'm Brandon. I'm a software engineer
working on Flutter Engine.

LEIGHA JARETT: And I'm Leigha,
a product manager for Flutter. If you tuned into
Flutter Forward or you've been reading
our release blogs, you've probably seen the
name Impeller thrown around quite a bit. But what actually is Impeller? And how does it
work under the hood? Well, today you're
going to find out. Impeller is a new renderer
within Flutter's Engine. Until now, Flutter has been
using something called Skia. The problem is that Skia
wasn't designed for Flutter. It has a ton of
rendering features built for a wide range
of devices, which means that it's not always
optimized for Flutter's needs. Enter Impeller,
Flutter's new renderer. We built Impeller to
specifically focus on the rendering needs
of Flutter applications.

And our main goal
was to eliminate jank or any stuttering that's
happening inside of your app. This way, it'll always look and
feel great for your end users. Now, for everyone who's watching
who's not a graphics engineer, you might be wondering,
what's a renderer. Well, a renderer
is software that helps you translate your
UI code into the pixels that you actually
see on the screen. So, say you have a simple
Flutter app like the one that I'm showing here. In the Flutter framework,
your tree of widgets is backed by a tree
of render objects. Render objects contain
the instructions for how to actually layout
and paint the widgets. These instructions are
given to the engine and stored in an ordered list
of simple commands called a display list. And now, this is where
things get interesting. The engine is
responsible for using one of the available renderers,
either Impeller or Skia.

to draw this display list
to a surface texture, or a grid of pixel values that
can be displayed to the screen. It leverages the
GPU by setting up a collection of things
called render pipelines that can be used to render
everything in the display list. So let's dig into the details. Before we can use
a render pipeline, we first need to take all the
paths drawn by the display list and tessellate them
into sets of triangles. Then each vertex or
point on the triangle is passed through something
called a vertex shader. Now a shader is just
a small piece of code that gets executed on
the graphics device. And in this case, the
shaders are internal, meaning that they're a part of
the Flutter engine's code base. But us Flutter developers
can author our own shaders too so more on that later. So the vertex shader
takes the vertices that make up the Flutter logo
and it moves them to the place

on the screen where we
actually want to draw them. And now the Flutter logo
is in the right place, and we begin iterating
through all the triangles. So let's zoom in a little more. For each triangle, we figure
out the specific pixels that are inside it. And this is called
rasterization. Then a few checks are run
for each of these squares to determine if we actually
need to compute a color for it. For Impeller, these
checks are really important, so we'll be coming
back to this in a bit. So now all those pixels
that are in the triangle are passed through
a fragment shader. A fragment shader is
another snippet of code, but this time it takes
the vertex shader's output and computes a color. And like I mentioned
before, Flutter developers can override this by
creating our own shaders to create really neat effects
using the fragment program API. Anyway, back to our
render pipeline.

Lastly, the output colors are blended with the color that's already been drawn. In this case, our blending was really simple, and we just replaced the color. And that's how things get rendered on the GPU. Now over to Brandon to talk through how these render pipelines are actually implemented in Impeller. BRANDON DEROSIER: Thanks, Leigha. Impeller has a layered architecture where each layer uses the one below it to accomplish its job. Before, Leigha showed an example of how the Flutter logo widget gets converted into a display list. In this example, the display list just contains a few operations to position it in the center of the screen along with four draw path ops, one for each of the quads in the logo. So let's take this display list and draw it using Impeller. First, all of the display list operations are dispatched to something called Aiks.

Aiks is the topmost layer of Impeller. And it mainly consists of a canvas drawing API. By the way, the name Aiks is just Skia in reverse. The job of Aiks is to take high-level commands like Draw Path and Draw Image from the display list and turn them into simpler, self-contained drawing operations called entities. And this is where the next layer comes in, the entities framework. In our example, the four draw path ops from the Flutter logo each end up getting an entity of their own. Each entity contains a bunch of properties that every single drawing operation needs, like a transformation matrix that encodes the position, rotation, and scale. Every entity also gets a content object assigned to it, which contains the actual GPU instructions needed to draw the entity. There's a contents for drawing solid colors, images, gradients, text clips, everything that Flutter apps can draw. Sometimes we even have multiple specialized contents

that draw the same thing,
using different algorithms with different performance
characteristics. That way Aiks can
pick and choose the most efficient
rendering algorithm depending on the situation. In this case, all four of
the Flutter logo's quads are just filled
with solid colors, and so each of their entities
get a solid color contents assigned to them. Now we need to spell out
these instructions in a way that the GPU can understand. Because Flutter apps can run on
a bunch of different platforms, Impeller needs some kind
of translation layer to communicate with the GPU. This is where the layer
below the entities framework comes in. I'll refer to this as the
hardware abstraction layer. This layer is just
a thin abstraction that talks to the
graphics driver through various
standard graphics APIs like Metal on iOS
and Vulkan on Android. Now we're ready to start telling
the GPU exactly what to do. Remember those render pipelines
Leigha mentioned before?

This is where those
come into play. Each content uses the
hardware abstraction layer to draw itself by instructing
the GPU to execute the render pipeline containing the shaders
we actually want to use. And really, that's it. Those commands are executed
using the graphics API, and then the resulting texture
gets displayed to the screen. But there's one huge detail
I haven't covered yet. Those shaders and
the render pipeline also need to get compiled
down to instructions that the GPU can execute. And this process
is very expensive. In Skia, this
compilation process happens at runtime
right on the frame. The pipeline actually needs to
be used to render something. This usually causes the
frame to go way over budget, resulting in a
noticeable stutter. We often refer to this problem
as shader compilation jank. Impeller vastly
improves on this issue by performing the most expensive
part of this compilation ahead of time.

And this is where the last
component of impeller comes in. When the Flutter
Engine is built, all of Impeller's
shaders get compiled into bundles using Impeller's
offline shader compiler called Impeller Scene. In walking through
Impeller's architecture, you've seen that Flutter really
has complete control over how it renders graphics. Flutter isn't subject to
the architectural or design limitations imposed by
platform-specific UI toolkits because Flutter
doesn't even use them. Instead, Flutter's
engine talks directly to the graphics driver. This is what allows you to
build rich, unique apps that behave exactly how you
want on any platform. This lack of dependencies also
makes Flutter a great solution for embedded use cases like
automotive dashboards, IoT, and digital signage. LEIGHA JARETT: Now that
you have an idea of what's happening under the
hood, let's highlight some of the key
architecture decisions that we made that makes Impeller
a great renderer for Flutter.

applications. So, like Brandon mentioned,
one huge benefit that Impeller offers
is eliminating jank from compiling those
programs, called shaders. To do so, Impeller
doesn't generate shaders in the same way that Skia does. Instead, Impeller has a set of
handwritten shaders compiled in advance. But you might be worried
that initializing a bunch of precompiled shaders
could mean slower startup times or bigger Flutter app sizes. To prevent this, Impeller
uses alternative rendering techniques that leverage
a much smaller, simpler set of shaders compared to
the many specialized shaders that Skia dynamically generates. Another reason why Impeller
offers great performance is the way we implement
anti-aliasing. Anti-aliasing is the
art of smoothing out jagged edges of drawn elements,
so that they look more natural. Impeller uses
something called MSAA to perform this work, which
is present across all devices supported by Flutter. It's pretty cheap to perform.

And it produces excellent quality results on modern mobile hardware. Another thing we focused on is how Impeller implements clipping. When rendering widgets, Flutter frequently uses clip masks to cut out shapes, so it's really important that this process is efficient. Impeller specifically takes an approach that leverages hardware features to ensure that clipping is super fast. Remember earlier when I mentioned those special checks? One is for something called a stencil buffer. Now we won't go into too much detail, but basically, as Impeller draws the pixels for the clip thing, it tells the GPU to check the stencil and filter away all the pixels that aren't part of the filled shape. This means that even if a clip is really complicated, like the ones used to create these kaleidoscope animations, it's still a pretty cheap operation, so we can see some really huge performance improvements. On the left-hand side, we see Flutter pre-Impeller.

And it's rendering at about
seven frames per second. And on the right
side, Impeller renders at about 60 frames per second.

BRANDON DEROSIER:
So we just talked about some of the design
choices in Impeller. For most Flutter
developers, this means you'll see more
predictable frame times when you use Impeller. But for those that
are interested in more advanced rendering use
cases, Impeller also aims to offer a launchpad
that allows you to solve for other graphical needs. For example, earlier this
year at Flutter Forward, we showed off a proof of concept
3D scene graph built directly on top of Impeller's
hardware abstraction layer called Impeller Scene. Impeller Scene has its own
contents in the entity's framework, and it
draws 3D objects using the same kinds
of render pipelines we talked about earlier. Now there are tons of
consequential decisions that go into building renderers.

And while we can certainly build
3D renderers that are useful and make smart tradeoffs,
no one renderer can solve every
use case perfectly. And so as a next
step for this, we're actively experimenting with
ways to surface more control over rendering in
the Flutter framework so that anyone in
the Flutter community can build their own
rendering packages and create custom
render pipelines that integrates seamlessly with the
rest of the widget ecosystem. Impeller's scene is
a proof of concept and a showcase for
the kinds of developer experiences we're seeking
to enable with this effort. With that in mind, let's take
a look at the demo widgets. Basically, everything
is centered around a special widget
called scene box. Scene box takes a node,
which can be created from a 3D model that's added
to your Flutter app's asset bundle. Nodes can be composed
together to form a tree just like normal Flutter widgets.

Here I'm wrapping the asset with another node that applies a rotation. You can add the same asset to the scene as many times as you want. And Flutter will know to only load the asset once and reuse it to inexpensively render copies, kind of like how image assets work. And you can even play animations built into the asset that were authored using 3D modeling software like Blender or Maya. And since Scene box is just a regular Flutter widget, any of these node or node animation properties can also be animated in exactly the same way that you would animate anything else in your build methods. And of course, hot reload just works. Simply export the file from your 3D modeling software and Flutter does the rest. With just these tools, it's possible to construct rich interactive scenes composed from many different assets. Again, this is the kind of fluttery experience we're seeking to enable as we surface.

more control over rendering thanks to Impeller. LEIGHA JARETT: So we just gave you an overview of Impeller. And we went under the hood to show just how Flutter renders your widgets to the device screen. But that's not all. Today, we're super excited to announce that Impeller is now the default renderer on iOS for the stable version of Flutter. So upgrade Flutter and try it out. And if you want to see Impeller in action, you can install G. Skinner's Wondrous app. Right now, Wondrous uses Impeller on iOS. But we're looking forward to the next version of Wondrous, which will include Android support for Impeller. We're really excited about where Impeller is headed and the new rendering capabilities that we can start bringing to Flutter, and we hope you are, too.

Introducing Impeller - Flutter's new rendering engine

Welcome to the official Flutter YouTube channel. Subscribe to stay up to date with best practices about the Flutter SDK. See real code examples, and watch engineers from around the world putting Flutter to work! 



Flutter

-  Impeller는 Flutter 엔진의 새로운 렌더러로, Flutter 애플리케이션의 렌더링 요구 사항에 특별히 초점을 맞추고 잠재적인 끊김 현상을 제거하도록 설계되었습니다.

-  소프트웨어의 렌더러는 UI 코드를 화면에 표시되는 픽셀로 변환하는 데 도움을 주며, 임펠러는 GPU를 사용하여 디스플레이 목록을 표면 텍스처로 그리는 작업을 담당합니다.

-  임펠러를 사용하면 경로를 삼각형으로 테셀레이션하고, 버텍스 셰이더를 통해 버텍스를 전달하고, 렌더 파이프라인을 설정하여 디스플레이 목록의 모든 항목을 렌더링함으로써 궁극적으로 최종 사용자에게 부드럽고 최적화된 시각적 경험을 제공할 수 있습니다.

Flutter에서 그래픽 렌더링 프로세스

Flutter의 렌더링 프로세스에는 래스터화를 통해 각 삼각형 내부의 픽셀을 결정한 다음 검사를 실행하여 각 픽셀의 색상을 계산하는 과정이 포함됩니다.

렌더 파이프라인에는 버텍스 셰이더, 프래그먼트 셰이더, 블렌딩이 포함되어 GPU에서 최종 출력 색상을 생성합니다.

임펠러는 렌더링 과정에서 상위 레벨 명령을 위한 Aiks, 드로잉 작업을 생성하는 엔티티 프레임워크, GPU 명령어를 위한 특수 콘텐츠를 사용하여 계층화된 아키텍처를 구현합니다.

Flutter의 그래픽 렌더링에서 임펠러의 역할

Impeller를 사용하면 상황에 따라 Flutter 로고에 다양한 렌더링 알고리즘을 사용할 수 있습니다.

GPU와 통신할 수 있는 번역 레이어를 제공하고 고가의 셰이더 컴파일을 미리 수행하여 눈에 띄는 끊김 현상을 방지합니다.

Flutter는 그래픽 드라이버와 직접 통신하여 그래픽 렌더링을 완벽하게 제어할 수 있으므로 다양한 플랫폼과 임베디드 사용 사례에 다양하게 활용할 수 있습니다.

플러터 렌더링을 위한 임펠러의 이점

임펠러는 사전 컴파일된 셰이더와 대체 렌더링 기술을 사용하여 성능을 개선하고 시작 시간과 앱 크기를 줄입니다.

또한 MSAA를 통한 고품질 안티앨리어싱과 하드웨어 기능을 사용한 효율적인 클리핑을 제공하여 성능이 크게 향상됩니다.

또한 임펠러는 3D 장면 그래프와 같은 고급 렌더링 사용 사례를 위한 런치패드를 제공하여 Flutter 개발자에게 더 많은 그래픽 기능을 제공하는 것을 목표로 합니다.

임펠러: Flutter에서 3D 렌더링 강화하기

임펠러는 커뮤니티가 자체 렌더링 패키지를 구축할 수 있도록 Flutter 프레임워크에서 렌더링에 대한 더 많은 제어 기능을 제공하는 실험을 진행하고 있습니다.

장면 상자 위젯을 사용하면 애니메이션과 핫 리로드를 지원하는 Flutter 앱의 에셋 번들에 있는 3D 모델을 사용하여 풍부한 대화형 3D 장면을 만들 수 있습니다.

임펠러는 이제 안정된 버전의 Flutter에서 iOS의 기본 렌더러로 사용되며, 곧 Android도 지원되어 Flutter 커뮤니티에 새로운 렌더링 기능을 제공할 예정입니다.