Android Studio

Create shortcuts.xml

Once you identify your in-app functionality and corresponding built-in intent (BII) to implement, declare the BIIs your functionality supports by defining a capability element in a shortcuts.xml resource file. Declaring a BII as a capability registers support for that semantic intent in your app, and enables voice query fulfillment of the intent using Google Assistant.

Assistant uses natural language processing to extract parameters from a user query. The built-in intents reference lists the fields that each BII is capable of extracting from an associated user query. For example, if a user invokes the [actions.intent.GET_FOOD_OBSERVATION][] capability in your app by saying, "Hey Google, Ask ExampleApp what did I eat for Lunch last Friday", Assistant extracts the following BII parameters from the user request:

Assistant passes BII parameters to the fulfillment intent defined in the capability. One or more intent elements can be defined in a capability to accommodate the different ways a user might invoke a BII. For instance, you could define a fulfillment intent that requires both BII parameters in the above example. You could then define a second intent that requires a single BII parameter, foodObservation.forMeal, that reports for all meals on a particular day, like "Hey Google, Ask ExampleApp what did I eat for Lunch."

Overview

You configure App Actions using a shortcuts.xml file placed in your app project's res/xml directory, and then creating a reference to shortcuts.xml in your app manifest. Add a reference to shortcuts.xml in your app manifest by following these steps:

  1. In your app's manifest file (AndroidManifest.xml), find an activity whose intent filters are set to the android.intent.action.MAIN action and the android.intent.category.LAUNCHER category.

  2. Add a reference to shortcuts.xml in AndroidManifest.xml using a <meta-data> tag in the Activity that has intent filters for both MAIN and LAUNCHER, as follows:

    <meta-data
   android:name="android.app.shortcuts"
   android:resource="@xml/shortcuts" />

The above example declares an XML resource for the xml/shortcuts.xml file in the APK. For more details on configuring shortcuts, see Create static shortcuts in the Android developer documentation.

The Jetpack library androidx.core:core:1.6.0 (or greater) is required in your Android project to avoid compilation errors when defining App Actions capabilities in shortcuts.xml. For details, see Getting started with Android Jetpack.

Static shortcuts

When defining your capability, you can declare static shortcut elements in shortcuts.xml to extend the functionality of the capability. Static shortcuts are ingested by Assistant when you upload a release to Google Play Console. Since static shortcuts can only be created and updated by creating new releases, they are most useful to highlight common activities and content in your app.

You can enable the following App Actions functionality with static shortcuts:

Capability Schema

The following table describes the App Actions schema for capability elements in shortcuts.xml. When including a tag, all of its attributes are required unless marked "optional".

Shortcuts.xml tag Contained in Attributes
<capability> <shortcuts>

android:name

app:queryPatterns (only applicable for custom intents)
<intent> <capability>

android:action (optional)

android:targetClass (optional)

android:targetPackage (optional)

android:data (optional)
<url-template> <intent>

android:value
<extra> <intent>

android:key

android:value

Only applicable for foreground app invocation
<parameter> <intent>

android:name

android:key

android:mimeType (only applicable for custom intents)

android:required (optional)

app:shortcutMatchRequired (optional)
<shortcut-fulfillment> <capability> Only applicable for inline inventory
<parameter> <shortcut-fulfillment> android:name
<slice> <capability>

Only applicable for Android Slices

Capability schema description

This section describes the capability schema elements.

<capability>

A capability that defines the App Action intent your app supports. Each <capability> element in your shortcuts.xml file must provide at least one <intent> to handle fulfillment of the action.

Attributes:

<intent>

Android intent element defining how a user query should be fulfilled using in-app functionality. Developers may provide multiple <intent> tags in a capability. Assistant attempts to fulfill a user query using the first <intent> in a capability for which all required parameters are provided.

Attributes:

<url-template>

Template for constructing a deep link URI to be opened on the device. The template may be expanded with built-in intent parameters if all required parameters for the template are available. For examples of the HTTP URL template, see the Wikipedia article on URL templates. The template format follows the RFC6570 URI template specification.

The following are some examples of URL template values:

Template Values Expanded value https://example.com/test{?foo,bar} "foo": "123" "bar": "456" https://example.com/test?foo=123&bar=456 https://example.com/test?utm_campaign=appactions{&foo,bar} "foo": "123"

"bar": "456" https://example.com/test?utm_campaign=appactions&foo=123&bar=456 https://example.com/test?utm_campaign=appactions{#foo} "foo": "123" https://example.com/test?utm_campaign=appactions#foo=123 myapp://example/{foo} "foo": "123" myapp://example/123

For more about configuring URL templates, see URL templates in fulfillment.

<extra>

Defines extra data for an intent. For App Actions, this field is only used to enable [foreground app invocation][] for a capability.

<parameter>

Maps a BII parameter to intent parameter values. For more information, see Parameter data and matching.

Attributes:

<shortcut-fulfillment>

Specifies that an intent defined in an inline inventory shortcut for a specified parameter be used for fulfillment. For details, see Fulfillment using shortcut intents.

<parameter> (for <shortcut-fulfillment>)

Optional attribute that maps a single BII parameter to inline inventory shortcut fulfillment. For details, see Fulfillment using shortcut intents.

Attribute:

<slice>

Enables Assistant to embed the result of a query matching this capability as an Android Slice. For details, see Integrate App Actions with Android Slices.

Shortcut schema

The following table describes attributes of shortcut elements that are used to enable App Actions functionality. When including a tag, all of its attributes are required unless marked "optional".

Shortcuts.xml tag Contained in Attributes
<shortcut> <shortcuts>

android:shortcutId

android:shortcutShortLabel

android:shortcutLongLabel (optional)

android:icon (optional)
<intent> <shortcut>

android:action

android:targetClass (optional)

android:targetPackage (optional)

android:data (optional)
<capability-binding> <shortcut>

android:key
<parameter-binding> <capability-binding>

android:key (optional)

android:value
<extra> <shortcut>

android:name (optional)

android:value

Only applicable for Enum parameter matching.

Shortcut schema description

This section describes the shortcut schema elements.

<shortcut>

An Android <shortcut> defined in shortcuts.xml with certain attributes that are relevant for App Actions. String values for the shortcutShortLabel and shortcutLongLabel fields are referenced via the APK's string resources.

Attributes:

<intent>

Android intent associated to this shortcut. This intent is executed when a user launches this shortcut using voice or touch.

shortcut intent attributes are identical to capability intent attributes.

<capability-binding>

Associates a shortcut to an App Actions capability. Adding this element to a shortcut enables it for voice fulfillment using Assistant.

Attributes:

<parameter-binding>

Optional attribute that associates a shortcut to a single parameter of an App Actions capability. If a parameter-binding is defined for a shortcut, the shortcut can be used to provide an inline inventory entity to a BII parameter. For more details, see Inline inventory for App Actions.

Attributes:

<extra>

The extra bundle data for the shortcut. sameAs is the only data relevant to App Actions shortcut elements. The sameAs URL refers to a reference web page that unambiguously identifies the entity. Used to specify an enum value if and only if the intent parameter type is a subtype of schema.org/Enumeration. It is required for parameter fields whose types are subtypes of schema.org/Enumeration (for example: MealTypeBreakfast).

Attributes:

For more details, see Matching enumerated parameter values.

Intent fulfillment options

You define intent elements within a <capability> to declare how Assistant responds to, or fulfills, user voice commands that match that capability. There are several ways to configure how an intent launches a fulfillment destination in your app, depending on how your app navigation is structured.

The following fulfillment options are available:

Parameter data and matching

By default, Assistant sends BII parameters extracted from the user query to your app as extra data of the Android intent defined in the capability.

Alternately, you can declare a <url-template> tag in the capability that contains placeholders for dynamic parameters. This template maps to one of your Android activities, using an App Links URL, a custom scheme, or an Intent-based URL.

Using intent Extras

The following example demonstrates an explicit intent defined for a capability fulfillment:

<capability android:name="actions.intent.START_EXERCISE">
  <intent
    android:targetPackage="com.example.myapp"
    android:targetClass="com.example.myapp.ExerciseActivity">
    <parameter android:name="exercise.name" android:key="exercise" />
  </intent>
</capability>

Given the above sample, for a user query like, "Hey Google, start my run," the app receives an intent that invokes the component: targetPackage, targetClass. The component receives an Extra with key = "exercise", value = "Running".

If your app is already able to handle app-linked URLs, with dynamic parameters, you can define a <url-template> in the intent to generate Android deep links for fulfillment. The following sample defines a <url-template>:

<capability android:name="actions.intent.START_EXERCISE">
  <intent>
    <url-template android:value="myapp://start{?exercise}" />
    <parameter android:name="exercise.name" android:key="exercise" />
  </intent>
</capability>

Given the above sample, for a user query like, “Hey Google, start my run," the app receives the generated URL: "myapp://start?exercise=Running".

To map the BII parameter to a position in your URL, you use the android:name attribute of the <parameter> tag. This attribute corresponds to the android:key value in the URL template that you want to substitute with information from the user. The android:key value must be present in your <url-template> and enclosed by curly braces ({}).

Match enumerated parameter values

Some BII parameters provide enumerated values to your fulfillment intent, for example, the supported text values of the RECORD_FOOD_OBSERVATION BII. For these parameters, Assistant matches the user's query ("Breakfast") to an entity whose sameAs value matches the enum schema URL (https://schema.googleapis.com/MealTypeBreakfast). To associate enum values for a supported entity, you declare a sameAs association in your shortcut. The following sample demonstrates a sameAs association for an inline entity shortcut:

<shortcut android:shortcutId="meal_breakfast" >
    <capability-binding android:key="actions.intent.RECORD_FOOD_OBSERVATION">
        <parameter-binding android:key="foodObservation.forMeal" />
    </capability-binding>
    <extra
        android:key="sameAs"
        android:value="http://schema.googleapis.com/MealTypeBreakfast" />
</shortcut>

<capability android:name="actions.intent.RECORD_FOOD_OBSERVATION">
  <intent targetPackage="com.example.app" targetClass="com.example.app.Class">
    <parameter android:name="foodObservation.forMeal" android:key="for_meal" />
  </intent>
</capability>

In the above example, if the RECORD_FOOD_OBSERVATION capability triggers a match for the "breakfast" meal type, the following Extra is sent with the fulfillment intent:

Features

The following App Actions features are available in shortcuts.xml.

Inline inventory for App Actions

For some BII parameters, shortcuts can be used to guide entity extraction to a set of supported entities specified in shortcuts.xml, known as inline inventory. For details, see Inline inventory.

Custom intents

Custom intents can be declared in shortcuts.xml to voice enable features in your app that don't match available BIIs. While similar in functionality to a BII definition, custom intents require two additional attributes in shortcuts.xml:

For more details, see Custom intents.