App Intents: Enhancing User Experience and App Features

App Intents elevate core app features to other parts of the device, enabling users to perform actions from Spotlight, Siri, widgets, and Control Center.

### Streamlining User Experience with App Intents
* 🌐 App Intents allow developers to **elevate core features** of their apps outside the app itself, enabling users to perform actions from the app directly from **Spotlight, Siri, widgets, and Control Center**.
* 🧠 The concept of "flow" is crucial for a delightful user experience, with flow defined as the **seamless transition between thought and action**. Friction, or any barriers that impede this flow, should be **minimized**.
* 🔍 By understanding the core features of an app, the system can present them **outside the app's "box,"** reducing the need to **switch between apps**.
* App Intents, as a common foundation for building features like Siri, Spotlight, and Shortcuts, allows developers to define **core actions and content** that the system can understand, and handles the **communication between the app and the presenter**.
* 🎯 App Shortcuts, which highlight important app functions, can be automatically offered in Spotlight and Siri without requiring the **app to be running**.
* Shortcuts actions provide flexibility for users to create **custom workflows**, while widgets and Control Center controls allow users to **personalize their app experience**.
* 🚀 By adopting App Intents, developers can **express their core concepts once and reuse them across multiple related features**, streamlining the user experience and making the app **more engaging**.

Hi! Welcome to Bring your app’s core features
to users with App Intents. I’m Christopher Nebel,
an engineer on the App Intents team. This is my app. There are many like it,
but this one is mine. Actually, it’s sample code;
it’s available from the developer site. It’s a catalog of trails; we’ll be seeing
it throughout this video. Just, work with me here. Anyway! This is my app, and it is awesome. People tell me that when they’re in it,
it’s a delight to use. What about when they’re not in it? It’s mostly invisible, it’s just a
box on their Home Screen? Not necessarily! With the right code, I can lift my app’s
features out of that box and elevate them to other places
in the device, so that people can perform actions
from my app directly from Spotlight, or by speaking to Siri. Or they can add information
in a widget to their Home Screen.

Or add actions and status to Control Center. So, my app will continue to delight people even when they aren’t in the app. This is what App Intents and the features that it powers delivers. How does that work? Well, you’re in the right place to find out. I’m going to talk about improving flow when using your app, and a family of system features that help you do that. What App Intents is, and how it fits in with those features, and how to get started building App Intents for your app. An important part of a delightful experience is flow. In psychology, that’s the melding of thought and action. Whatever you think to do, you do it effortlessly. On a device, that means the next thing you need is always nearby and easy to get to. If you have distracting extra steps in the way of your flow, that’s called friction. Metaphorically, anything that resists getting things done.

The same metaphor shows up
in lots of other expressions: Things with high friction are rough.
You get stuck. They’re a real drag, man. Things with low friction are fluid.
They’re like a well-oiled machine. They’re smooth. A little friction can be a good thing; don’t get me started
on completely frictionless surfaces, but generally, you want
to reduce friction wherever you can. Friction doesn’t have
to be a single big barrier. Often, it’s a small bump
that you hit over and over. An example when using your device
is switching apps. Your app defines a little world, its
own box, that you have carefully crafted, and within that app, it’s perfect. The catch is that people don’t use just
one app to get things done. They use many. What happens when they need
to move between boxes? Because each of them is in their own box. And by default, the only way to see
or do anything in a box is to go into it.

which means now you can’t see
or do anything in any of the other boxes. You have to leave your current box
and go into another one. That is, you have to switch apps. And switching apps isn’t really hard,
but it’s not free. It takes time and thought while you
navigate there and back. That’s friction. What if the device understood what’s
in your box, your core features? Then, it could present them higher up,
outside your box, and make them quicker and easier
to get to. There’s a whole family of system features
based on this idea, making the experience
of using your device flow. Let’s look at a few examples, all
of which I’m going to build later. These are based on my example app, which shows a catalog of trails
in the area, with various details about them. Some features help me get
to the right place in the app faster, such as searching in Spotlight.

Say I’m at the Home Screen,
and I want to look at my pinned trail. So I trigger Spotlight. Before I even type anything, Spotlight Suggestions guesses
that I want my example app, and offers the app shortcut that I need. Imagine that it hadn’t guessed that,
so I have to type: e, x, a. Now it’s sure I want my example app,
so there it is as a Top Hit, with the app shortcut
even more prominently placed. Or, from anywhere, I could have just said
what I wanted to Siri. Some features don’t just make it easier
to switch apps. They make it so I don’t need
to switch apps in the first place. Let’s say I want to keep an eye
on the conditions for my favorite trail. Instead of opening the app every time
I think of it, I can add that information
to my Home Screen with a widget. Now I can glance at it
as I’m doing other things.

I could add it to the Lock Screen, and then I wouldn’t even need
to unlock my phone. Or I can add a custom control
to Control Center, which I can reach while using
any other app. There are many other features
like these, and we keep adding them. Last year, we added the Action button. This year, we’re adding Camera capture,
Apple Pencil Pro: squeeze, and more. All of them elevate your app's features
out of their box, to become part of
the whole device experience, and streamline the flow between thinking
I want to do this and it’s done. There's an important distinction
among these features: Who chooses what to elevate? With features like Spotlight,
you, the developer, choose. You highlight important actions
in your app, and the device presents them
automatically in the right spots. Likely actions from your app
are always close by, no matter where people are on their
device, giving them that sense of flow.

With features like Widgets and Controls, people choose. You provide flexible components, and then people can choose precisely the ones that matter, and build the right workflow for them. In other words, they can personalize their experience of your app, making their device an extension of, and even an expression of, themselves. That’s good for them, and it’s good for you, the app developer. Because the more people can mold your app to fit themselves and their lives, the more likely they are to keep using it. At this point, you might be saying, this sounds great, I want my app to flow, what do I do? Glad you asked! The answer is, you adopt App Intents. And you say, wait, what? I thought we were talking about Siri, and Control Center, and all those other cool features. And I say, we are. Allow me to explain. App Intents is not a feature in itself. It’s a common foundation for building features.

All these features we’ve talked about, Siri, Spotlight, Shortcuts, and so on, have a common theme: They take the core features from inside your app, actions, and content that are meaningful to someone using your app, and present them outside your app. To do that, you need to expose your app’s core features in a way that the system can understand. App Intents is the framework to do that. It handles two big parts. First, it lets you define what your core actions and content are, so system features like Siri and Spotlight know what they can present. Second, actually presenting something means a bunch of back-and-forth communication between your app and the presenter. For example, if someone tells Siri to do a thing in your app, Siri has to send a message to your app to do the thing, and then your app has to send a message back saying that it did it, and here are the results.

App Intents handles that communication, so you can concentrate on implementing
the actual behavior of your features. And once you’ve done that,
to support one system feature? That same work is useful across
a range of system features, because they’re all based on App Intents. So you can share code between them, and even support several features
with the exact same code. There are three top-level things
you’ll write, and I’ll show how in the next section. Intents perform an action,
like Opening a view or Starting a hike. They’re the commands,
or verbs, in your app. Entities are objects, like a Trail
or a Collection. They’re the nouns. If intents are verbs,
and entities are nouns, then App Shortcuts are sentences. They package together a verb
and a noun or a blank to fill in a noun, and whatever other options
are needed into a single thing that describes an important function
of your app, something that you know anyone
who uses your app will want to do.

Features like Spotlight and the Action button will then offer that function, that sentence, so people can do it quickly and easily. Together, these define how your app appears across the device. Now that you have a general idea, let’s write some code. I’m going to add some features to my app to help it flow. Five of them, in fact: A simple Shortcuts action with no parameters. A Parameterized Shortcuts action. A Home Screen Widget. A Control Center control. And app shortcuts for Spotlight and Siri. That sounds like an awful lot, but they’re all based on the App Intents framework, and that’s going to help me share a lot of code between the features. Let’s get started. The Shortcuts app is a powerful tool where people can explore, combine, and remix actions to create their own custom shortcuts. For example, my teammate loves mountain biking with his daughter, so they used the actions from my app to build a custom shortcut.

that picks a random biking trail
in an area. As the developer, I might not prioritize
this feature. It’s kind of specific. But I can offer flexibility
in the form of Shortcuts actions, so they can build it for themselves,
and tune it to be exactly what they want. I recently added a feature
to Pin a trail to the top of the list. That way, when I’m planning a trip there,
I can open its details quickly, so I can answer questions like
how do I get there, or can my friend bring their dog? Works great, but I want to be able
to get to that trail quickly even if I’m not currently in the app. So let’s make a Shortcuts action that opens the pinned
trail details in the app. When I’m done, it’s going look like this
in the Shortcuts app. In the App Intents framework,
a Shortcuts action is an intent. All intents appear by default
as actions in the Shortcuts app.

An intent is a type that conforms
to the App Intent protocol. It has two required pieces: A localizable title, which will appear
as the name of the action, and a perform method,
which does the action. Intents should always be actions that are
meaningful to someone who uses the app. Here, OpenPinnedTrail
not internal implementation. Perform methods always have a result, but the result may be empty,
as it is here. For this intent, I want it to open
the app when it runs, because I’m showing a view in the app. App Intents will handle
that for me if I tell it to, which I do with openAppWhenRun
like this. An intent can also have extra parameters,
extra information needed to do its job. This intent doesn’t need any,
but I’ll come back to one that does. And that’s it! My new intent shows
as an action in the Shortcuts app library.

If I make a shortcut with this action,
I can run it in the Shortcuts app. I can take that one step further,
and add it to my Home Screen. If I tap it, it runs, and my app opens
with the pinned trail details, just like I wanted. That’s not bad, but I also want an action
that lets me pick the trail to open, so I can make a shortcut
that opens a particular trail, or that asks me for a trail when I run it. To do that, I need to make
an intent with a parameter. This is going to look very similar
to the first intent. In fact, I’m going to start
by cloning the first intent and changing its name. Intents that open their parameter
in the app have a special protocol, OpenIntent, so I’m going to add that. Open Intent implies openAppWhenRun,
so I can delete that. And I need to add the parameter
that the protocol defines.

A parameter is just a normal property
marked with @Parameter, so you can add extra metadata
like a localizable title. I said it should be a trail. Trails are a core content type in my app,
so I’ll make it a trail entity. Two things to remember here: First, entities should be things
that are meaningful to someone who uses the app. So this isn’t a database row,
or whatever my implementation is; it’s a trail. Second, a parameter that refers
to an entity should be an entity, not data describing one. So this isn’t a trail name, or UUID;
it’s a TrailEntity. I update the perform method
to navigate to the parameter. And I’m done, except for the part where
I have to define the TrailEntity. Fortunately, that’s not very hard. An entity is a type that conforms
to the AppEntity protocol. You can conform your model type directly. This works well if your model is small
enough to all fit into memory at once.

so all the necessary instances
already exist. Or, you can make your entity its own type
that merely refers to your implementation. This is useful if you only
create model instances as you need them, or if your model instances
have expensive properties that aren’t needed for your intents. In this example, I‘m going
with the second approach, so TrailEntity will refer
to my underlying model type Trail. An entity may have Properties here,
the trail name. It’s very similar to a parameter, but it’s on a content object,
not a command. An entity must have three things: A display representation, so the device
can do things like draw them in a menu; a persistent identifier; and a query. Right. What’s a query? Before I tell you what a query is,
I should tell you what a query does. A query turns questions asking
for entities into actual entities. And choosing a value
for the intent parameter is going to ask two questions:

First, what entities are there? So the device can show a list of options,
someone can pick from. When they pick one,
it’s going to save the ID for that entity. Then, when it’s time to run the intent,
it sends that saved ID, and the query needs to answer
the second question: What entity has this ID? Given that answer, it can fill
in the intent parameter, so your intent code gets an entity,
and not just an ID. What does that code look like? A query is a type that conforms
to EntityQuery. There are several different ways
to ask for entities, not just the two I already mentioned,
what options and what has this ID, but also searching for a string,
searching by predicate, and so on. So EntityQuery has several sub-protocols,
one for each kind of question. The what entities are there question can be handled by EnumerableEntityQuery, which defines a method allEntities,
like this.

This is the conceptually simplest
form of query. App Intents can derive the more
complicated ones from this if you build with the iOS 18 SDK. Of course, it only works if you can
actually return all your entities at once, so your whole model
has to fit into memory. If that’s not the case, or if you can do a better job than
the automatically derived queries, then you should use
one of the other query protocols. But this example is simple enough
for it to work. The what entity has this ID question
is answered by EntityQuery itself, with this method, entities(for:). Together, these handle everything
parameter configuration needs to work. And now my intent is ready to run. But there’s one more thing I should add,
which is a parameter summary. Without one, my intent looks like
this in the Shortcuts app. This works, but it’s hard
to read and understand because the trail that it’s going
to open is below the fold.

A parameter summary is a natural language sentence
that describes what the intent will do, including the values
of all the essential parameters. In this case, the trail to open. Now it’s easier
to see what this intent does, and it’s readable as I edit the parameter. Let’s see it in action. When I tap the Trail placeholder,
I get a sheet to pick a trail. The query provides the options,
and powers searching. When I pick one, it appears inline so I can immediately read
what this action will do when it runs. That’s the parameter summary at work. I actually don’t want
to choose a trail now. I want to choose one when it runs. So I’ll clear that parameter and save the shortcut to my Home Screen. Now, when I tap the shortcut, I get a
similar picker, again, powered by the query, and it works straight from my Home Screen
or anywhere I can trigger the shortcut. Shortcuts actions are the ultimate
in adding flexibility to your app.

People will build them into all kinds
of workflows that you never imagined. Let’s look at a more focused
form of flexibility, that lets you view information
at a glance: A widget. Most details about a trail
don’t change often, if ever. It’s not going to move to a new location. Probably. We do have earthquakes. Conditions, however,
can and do change routinely, so it would be handy
to keep an eye on them. That’s a perfect use for a widget. They’re
designed for glanceable information. I could make a widget that just shows
the details of my pinned trail, but I want to keep
an eye on a few different trails, so I can choose whichever one is nicest. To do that, I need to make my widget
configurable so I can set which trail to show. That means my widget is going to have
a trail parameter, much like my Open Trail action. In order to make a widget,
I need a widget definition.

This isn’t a WidgetKit video, so I’m skipping a number of details
like the view and timeline provider. If you want to know more about that,
refer to the WidgetKit documentation, and for more details about
how widgets work with App Intents, check out Explore enhancements
to App Intents from WWDC23. The relevant part here is the body, and especially the intent parameter where I set the configuration
intent type for this widget. The configuration intent is an intent that conforms
to WidgetConfigurationIntent. Funny how that works. Like all intents,
it needs to have a title. And I said I wanted this widget
to be configurable with a trail, so I’ll add a parameter. It’s supposed to be a trail, and I already
defined a trail entity for my open intent. Can I re-use that? Funny you should ask. Yes, you can. I’m making it optional so it can be empty when the widget is added
to the Home Screen.

but it’s otherwise the same type
as the parameter in the Open Trail intent. Now, I have a trail-conditions widget
that’s configurable for any trail, so I can add as many as I need. The trail entity and its query
that I already wrote handle picking a value
for the configuration, so I didn’t need to write that again. That was remarkably straightforward,
so let’s do another one: A Control Center control,
which have a new API in iOS 18. Like the widget, I want the control
to be configurable with a trail. When tapped, it should open
that trail’s details in the app. This is only going to cover
the App Intents-related parts of Controls. For the full Control details, see “Extend your app’s
controls across the system”. With that in mind, let’s get started. The basic outline of a control
is that it’s a special type of widget, ControlWidget. Which has a body that defines what
the control looks like and what it does.

A configurable control, which is what I’m aiming for, is configured using an App Intent. Much like a configurable widget, so I add an AppIntentControlConfiguration. I said it should act as a button, so I'll add a ControlWidgetButton. This is the basic shape for a configurable button control. I still need an intent type to hold the configuration so I can build the button contents, and an intent instance to handle when the button is tapped. Let’s think about this for a second. I said I wanted the configuration to have a trail parameter. That sounds kind of like the Open Trail intent I made earlier. Can I re-use that? You bet I can. The one thing I need to change about OpenTrail is to declare that it conforms to ControlConfigurationIntent. That’s an extension, and doesn’t require anything that the intent doesn’t already have, so the body is empty. Once that’s done, I can fill in the type.

and use the configured instance
of that type to build the image
and text for the button. And here’s the really clever bit. I still need an action, which is an intent instance,
and it should open the configured trail. Again, that sounds like OpenTrail. But, I don’t need to make
a new OpenTrail instance, because I already have one
as the configuration intent. I can simply pass that as the action, because it already has the Perform method
I need and is configured the way I want. Let’s see it work. This is the new configuration mode
for Control Center. I’ve already added the new control
at the lower left, here, and configured it to open
the Monterey Bay Coastal Trail. If I stop editing, the control is now live, so if I tap it, the intent runs, and my app opens
to the trail details I configured. Now, I’ll make one of my app’s
actions available automatically

in Spotlight and Siri, opening
the details of the pinned trail. To do that, I need to make
an App Shortcut. An App Shortcut is a wrapper
that you, the developer, create around an intent that highlights
it as an important function of your app. Various features will then offer
that App Shortcut, such as Spotlight, the Action button, and Apple Pencil Pro, without needing your app
to be running or even have run at all. Let’s look at the code for that. I define a single AppShortcutsProvider with a static member,
which is a list of app shortcuts. An AppShortcut wraps an intent. Here, the OpenPinnedTrail intent
I created earlier. Notice that this is an intent instance,
not a type. That means that I can pre-fill some
or all of the parameters here. So I could take a very general intent
with lots of parameters and wrap it as an AppShortcut
that does something very specific.

It will prompt for any unfilled required
ones when the shortcut runs, so it works either way. It depends on
what experience you want to provide. This intent doesn’t have any parameters,
so the parentheses are empty. Since App Shortcuts
are available from Siri, there’s a list of phrases to speak, each of which must contain the app name, and for places where it’s displayed
visually, such as Spotlight, a title and image. Notice one thing that’s not here:
Any sort of registration code. The App Intents framework automatically
detects the provider and handles registration, so my App Shortcuts are available
as soon as the app is installed. With that, the App Shortcut now
appears in Spotlight and can be spoken to Siri. Both Spotlight and Siri
are handled automatically; someone using their device doesn’t need
to do anything more than install the app. But, if they want to customize their
device, App Shortcuts are also available.

In the Action button and Apple Pencil Pro. So that’s four features
for one piece of code. And if you have existing App Shortcuts? They already work with Apple Pencil Pro, same as they automatically worked
with the Action button last year. This is looking good, but
the Siri interaction isn’t quite right. First, it’s going to show a view, which means I have
to at least be looking at my device. Second, it’s going to open the app, which is going to bounce me out
of whatever app I’m currently using. So let’s replace that intent
with a slightly different one that returns the information instead. I’ll make an intent, but
this one won’t open the pinned trail. It will show it in a snippet, and speak it
if the user isn't looking at their screen, or doesn’t have a screen,
like a HomePod or AirPods. I’ll start with a stub perform method. Step one is clearly
getting the pinned trail.

But what’s step two? How do I show or speak information
without using my app? I use the result. The result of a Perform method
isn’t just a value. It’s a rich combination of things,
including dialog that Siri can speak and a view snippet it can show. If I say that my result conforms
to ProvidesDialog and ShowsSnippetView, I can use a different method to create
the result that takes dialog and a view. I can supply just dialog,
or just a view if that makes sense, but here I’m going to do both. I’ll write what I want Siri to say, which may be different
from the text in the snippet. I’m using an interpolated
string here for convenience, but this parameter is actually
an Intent Dialog instance, which allows separate Full
and Supporting dialog if you need it. And I supply a SwiftUI view. I’ll spare you the exact details of my view and use
a separate function to get it.

but I could have put it inline, using a trailing closure
with a view builder expression. In either case, the view is archived like
a widget, so it can be relayed to Siri. So you can use any SwiftUI features
that widgets support. Now, when I ask Siri, it will show me
the pinned trail conditions in a snippet, if I'm looking at the screen. When I dismiss it,
I'm right where I left off, so I don't have to find my way back. If I'm not looking at the screen,
Siri will speak the dialog to me. If my phone is in my pocket because
I'm busy looking for my boots, I can ask my HomePod across
the room for the conditions and I don't even need
to take my phone out. Ok, wow. I just added
a redonkulous number of features to streamline how people can use my app,
and it wasn’t even that much code.

This is what App Intents does. It lets you express your
core concepts once, and then re-use them
across many related features. Wrapping up: You can improve flow when using your app. People will get things done faster,
they’ll enjoy it more, and they’ll be able to tune how they use
your app and build it into their lives. To do that, adopt features like
Siri, Shortcuts, widgets, and more, all of which let people use your app
even when they’re not in your app. The way you adopt those features
is to use App Intents, because it’s the foundation
for all of them. That means you need
to learn fewer different APIs, and you can share code between features. Want to learn more? Of course you do! Here are some great videos to watch next: If you’re still fairly new to App Intents
and honestly, even if you're not see Designing App Intents,
on what makes a great App Intent.

If you want to find out more about how App Intents work with Siri, see Bring your app to Siri. If you already have App Intents and want to find out what’s new since last year, see What’s new in App Intents. Thanks for watching, and we can’t wait to see what you do with App Intents!

WWDC24: Bring your app’s core features to users with App Intents | Apple

Hello and welcome to the official Apple Developer YouTube channel.

Apple Developer

Elevating Your App's Features with App Intents

App Intents allows users to access app features from various parts of the device, reducing friction and improving flow.

The goal is to make the app's core features easily accessible outside of the app's "box", through Spotlight, Siri, widgets, and Control Center.

This functionality is designed to enhance the user experience and streamline the process of switching between different apps.

Enhancing the User Experience with App Features

Spotlight Suggestions and app shortcuts make it easy to access the example app without switching apps.

Widgets and Control Center allow users to personalize their experience and access important information quickly.

App Intents provides a common foundation for building features like Siri and Spotlight, allowing developers to define their app's core actions and content for presentation outside the app.

The Role of App Intents in App Development

App Intents simplifies the communication process and allows for code sharing between different system features.

Intents, entities, and app shortcuts are the three key components involved in App Intents framework, with app shortcuts offering important functions for users.

The framework, based on App Intents, supports the creation of various features such as Shortcuts actions, Home Screen Widgets, and Control Center controls, allowing for code sharing and flexibility in customizing shortcuts.

Creating a shortcut in the Shortcuts app allows for easy trail navigation.

Adding the shortcut to the Home Screen provides even quicker access to a specific trail.

By incorporating an intent with a parameter of TrailEntity, users can select a trail to open, providing more flexibility in the shortcut.

Building Flexibility with App Intents and Widgets

App Intents can handle complicated queries and provide a parameter summary for better readability in the Shortcuts app.

Shortcuts actions offer ultimate flexibility and can be triggered from the Home Screen.

Widgets are designed for glanceable information and can be made configurable with parameters, similar to the Open Trail action.

The widget's configuration intent needs to conform to WidgetConfigurationIntent and can reuse the trail entity defined for the open intent, making it optional for added flexibility.

Configuring Trail-Related Widgets and Controls in iOS 18

Trail-related widgets can be easily configured for any trail, using the trail entity and its query to pick a value for the configuration.

A configurable control in iOS 18, such as a button, can also be configured using an App Intent, with the ability to open the configured trail when tapped.

Developers can make their app's actions available in Spotlight and Siri by creating an App Shortcut, which highlights specific functions of the app using an intent instance.

App Shortcuts can be accessed through Siri and Spotlight, requiring no additional registration code.

They offer four main features: accessibility through Siri, customization options, compatibility with the Action button and Apple Pencil Pro, and the ability to work with existing shortcuts.

The author is looking to improve the Siri interaction by returning information in a snippet and spoken form, even when the user is not actively using their device.

App Intents allows for the efficient use of core concepts across different app features, improving user flow and experience.

By adopting features like Siri, Shortcuts, and widgets, users can interact with the app even when they're not actively using it.

This approach not only makes it faster for people to accomplish tasks, but also integrates the app more seamlessly into their daily lives.