# Integrate your app with the Android intelligence system - Speakers: Sarah Shamsie (Product Manager at Google), Dan Galpin (Developer Relations Engineer) - Track: Intelligent Apps - Video: https://www.youtube.com/watch?v=UsqDEfyOH2E - Page: https://developer.android.com/events/dev-summit/talks/integrate-your-app-with-the-android-intelligence-system ## About the talk Android is evolving into an intelligence system where agents like Gemini orchestrate tasks across apps. To unlock new user experiences and participate in these agentic workflows, your app must expose its core capabilities. This session is a practical guide to integrating your app with Android's intelligence layer. We'll survey key integration paths - including on-device Android MCP, server-side MCP, and Task Automation - and provide a decision framework for choosing the right path for your use case. We'll also explore proactive experiences with Personal Context. You will learn how to: - Select the optimal integration path for your app's needs. - Implement both reactive and proactive intelligence features. - Balance performance and security in agentic integrations. ## Summary - Android MCP is a unified on-device tool calling framework powered by the Jetpack AppFunctions library. It maps MCP primitives to Kotlin constructs and runs on Android Binder IPC. - On-device tools excel when accessing encrypted data or device sensors. Alternatively, platform-driven Task Automation lets pre-loaded agents simulate touch events without code changes. - Your app acts as an Android MCP server while agents discover and call tools. Declare tools with `AppFunctionDeclaration`, provide KDoc descriptions, and avoid relying on Kotlin default values. - The Connected Apps Platform, currently in preview, lets you test your AppFunctions with Gemini. Set up your connected app in Google Connected Console, then upload your bundle in Play Console. ## Resources - [Jetpacker travel sample app](https://goo.gle/jetpacker) - [Sample agent source code](https://github.com/android/appfunctions) - [Connected Apps documentation and preview signup](https://developers.google.com/connected-apps) - [Android AppFunctions documentation](https://developer.android.com/ai/appfunctions) ## Topics android-mcp, appfunctions, gemini, connected-apps, model-context-protocol, on-device-ai ## Transcript ### Introduction to the intelligence era [00:05](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=5s) All right. [00:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=6s) Hello, everyone. [00:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=6s) I'm Dan Galpin from Developer Relations. [00:09](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=9s) And I'm Sarah Shamsie, a product manager working on Android's developer intelligence frameworks. [00:14](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=14s) We're super excited to welcome you to today's session. [00:18](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=18s) We're witnessing a paradigm shift in how users are interacting with their devices. [00:23](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=23s) In the Web Era, users typically started their journeys by navigating to the web browser and then using the search function. [00:30](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=30s) And then in the Mobile Era, we saw user behavior shift to using more specialized, native apps to complete one-off actions. [00:37](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=37s) Here's an example. [00:38](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=38s) A message comes in from my friend inviting me to dinner. [00:41](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=41s) In order to respond to this, I need to check my calendar to see if I'm free, add the event to my calendar, book a ride if I need a way to get there, and then message my friend back that I can attend. [00:51](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=51s) Previously, this required something called app-hopping, where I would basically have to toggle between different apps on my device to actually complete the actions and fulfill my journey. [01:00](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=60s) But now we're in the Intelligence Era. [01:03](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=63s) In the Intelligence Era, users are starting their journeys by expressing high-level intents and goals rather than specific actions they need done. [01:11](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=71s) There's a fundamental shift where they're viewing their device as something that actually proactively anticipates their needs and helps them fulfill their goals rather than something that just enables actions. [01:20](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=80s) And we noticed that they're expecting this intelligence framework to work not just on their mobile devices, but across a rich form factor ecosystem, including watch, auto, TV, laptops, and XR and glasses. [01:35](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=95s) In the Intelligence Era, a single cross-app prompt can actually facilitate the dinner invitation example from before. [01:42](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=102s) Just with one prompt, an agent on my device can help enable all the actions that I need from checking my calendar, adding the event, booking the ride, and sending a message all in one go. [01:52](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=112s) Today, we're going to share more about Android's plans in this space and then how you as developer developers can build first-class experiences. [02:00](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=120s) Dan. [02:01](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=121s) Thank you, Sarah. [02:03](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=123s) So, Android is continuing to add intelligence at its core, and that's offering new ways for your apps to, you know, equip for this Intelligence Era. [02:11](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=131s) And the first of these two developer intelligence pillars is in-app intelligence, and that's about obviously making your apps smarter, bringing intelligence into your app, and you can use on-device models like Gemini Nano 4 through ML Kit to summarize message threads, and rewrite replies, and understand images, or use natural language to call your app's functions. [02:30](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=150s) You can even use Gemini models in the cloud directly in your app through Firebase AI Logic. [02:35](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=155s) But our focus today is on that second pillar, making sure your app's capabilities are available within the intelligence system. [02:42](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=162s) And so, what's an intelligence system? [02:46](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=166s) So, in addition to, you know, talking and hosting about those prompted experiences that Sarah talked about earlier, it is a system that enables deep integrations to improve experiences across the app ecosystem. [02:58](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=178s) So, devices designed for Gemini intelligence, for example, leverage these integration points to provide things like suggestions, summarized notifications, improved voice input, and automated filling out of forms. [03:10](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=190s) Now, suggestions are part of a feature called proactive assistance. [03:15](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=195s) And up until now, we've been talking about prompted assistance. [03:17](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=197s) We're all used to that, where a user tells an agent to accomplish one or more tasks on their behalf. [03:22](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=202s) But proactive assistance is actually about anticipating the next action or actions a user might take. [03:28](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=208s) So, going back to our dinner example, the agent might proactively prompt the user with suggestions in response to the message that could either add the invite to their calendar or book a rideshare. [03:39](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=219s) But how does it actually prompt the user? [03:42](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=222s) Well, proactive assistance can surface actions in places such as the lock screen, notifications, virtual keyboards, and even within selected apps. [03:51](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=231s) So, what is the way you can make your app actually appear in these proactive ch- chips to drive traffic your way? [03:57](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=237s) Well, you're already doing things like building against the latest SDK version and building with our most modern tools, but the best thing you can do now is to make sure that your app is also ready for prompted assistance, which Sarah will talk about next. ### Prompted assistance and Android MCP [04:12](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=252s) Thanks, Dan. [04:13](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=253s) Okay, so prompted assistance involves users going directly to an agent and asking it to initiate and orchestrate tasks, even across apps. [04:21](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=261s) This can automate previously manual and time-consuming workflows. [04:26](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=266s) For our intelligence system on Android, we're primarily focused on system agents like Gemini that can tap into apps across the user's device. [04:36](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=276s) There's a few different ways your app can show up in this experience and in- interact with the system agent. [04:41](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=281s) There's our platform-driven Task Automation solution, and then there's direct API integrations into the system agent. [04:47](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=287s) We always recommend building with direct API integrations wherever possible. [04:51](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=291s) These are developer-controlled, so you can control you can decide and control how your app actually shows up and how your brand shows up. [04:58](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=298s) They're also faster and more reliable, offering a more premium experience to your users. [05:03](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=303s) There's two API paths you can consider when wanting to integrate with the agent. [05:07](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=307s) Model Context Protocol, or MCP for short, is the industry best practice for server-side function calling. [05:14](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=314s) But for on-device tool calling, we're formally announcing Android MCP, the unified function calling framework on Android. [05:22](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=322s) Android MCP is powered by our Jetpack AppFunctions library, which aligns closely, both technically and conceptually, to MCP, while supporting a familiar and idiomatic Kotlin implementation that many Android developers are familiar with. [05:35](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=335s) This makes implementing the API relatively intuitive and easy to use, and you can leverage any learnings you might have from an already-stood-up server-side MCP integration. [05:46](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=346s) Many MCP primitives map one-to-one to a Jetpack AppFunctions construct. [05:51](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=351s) Let's walk through some examples to illustrate this. [05:54](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=354s) An MCP tool end point maps nicely to a Kotlin Kotlin function annotated with `AppFunctionDeclaration`. [06:02](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=362s) Your pydantic models map to Kotlin data classes annotated with `AppFunctionSerializable`. [06:08](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=368s) But instead of streamable HTTP and or standard I/O transport, Android MCP runs on the optimized Android Binder IPC. [06:17](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=377s) Today, we're super excited to share that the Jetpack library, AppFunctions, is now in beta. ### Key use cases for Android MCP [06:24](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=384s) Okay, so I'm sure you're probably wondering when it makes sense to use one API path, and when it makes sense to use Android MCP, specifically. [06:31](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=391s) Well, as an overarching principle, Android MCP is especially useful when you want to leverage on-device or local context to fulfill a user journey. [06:39](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=399s) Let's dive into some examples where Android MCP really shines. [06:43](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=403s) Specifically, you should consider Android MCP when you need to access local, encrypted data. [06:48](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=408s) This is especially important for messaging and communications apps, where message history needs to stay on-device due to end-to-end encryption. [06:56](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=416s) Android MCP is also useful for purely on-device and offline actions, where agents do support this. [07:03](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=423s) Our Wear OS partner, Samsung Health, launched a function to start a run and start exercises, that are orchestrated entirely on-device. [07:12](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=432s) Users often want these types of actions to be done very quickly, high reliability, low latency, and sometimes, and even even in low-connectivity environments, making sure that the entire tool calling and routing and query routing can happen on-device ensures this. [07:28](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=448s) Android MCP is also the preferred API path if you want to access device sensors or interact directly with the device hardware. [07:35](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=455s) For example, many automotive apps want to allow users to unlock their car doors instantaneously through UWB connectivity and often hands-free. [07:43](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=463s) So, Android MCP can help with that. [07:46](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=466s) And finally, it's your solution if there's no available cloud endpoint for your apps or API. [07:52](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=472s) Many of our device manufacturer's first-party apps are client-side only, and so they're using Android MCP for this reason. [07:59](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=479s) For example, Gemini can be used to search and retrieve photos from gallery apps that are client-side only, like Xiaomi Gallery. [08:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=486s) Here, I can pull up Gemini, and ask, "Find the photos of the Android bot I took last week on Shaomi Xiaomi Gallery." [08:13](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=493s) Xiaomi has exposed an app function to search for photos, which Gemini then calls, and the relevant photos are returned and surfaced directly in the Gemini app UI, so the user never has to leave. [08:24](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=504s) And we're relying on the same exact architecture for on-device actions in the Android OS. [08:29](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=509s) Android MCP powers the Device Help feature that recently launched, which is a direct Android OS integration into the Gemini app. [08:37](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=517s) This capability has already launched on Pixel and Lenovo devices, and it's currently scaling out across the broader Android OEM ecosystem. [08:45](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=525s) Users can use Device Help by going to Gemini and asking how they can optimize their settings, their device settings, and system settings to improve their device experience. [08:55](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=535s) For example, instead of hunting through various menus, a colorblind user may ask Gemini, "I'm colorblind. [09:01](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=541s) How can I make this phone work better for me?" [09:04](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=544s) Gemini will call and invoke local app functions, and intelligently reason through the request, and suggest actual ways that the user can change their system settings, and specific ways they can change their device settings to improve the experience. [09:16](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=556s) Then, in-app controls are surfaced directly in the Gemini app UI, so users can take immediate control. [09:22](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=562s) Like, for example, in fulfilling this colorblind, optimization request, the Gemini has actually surfaced this color-correction toggle, so users can make the change on the fly. [09:34](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=574s) Android MCP experiences are already available on select devices, and we're working with multiple multiple partners to bring even more use cases to the Android ecosystem. [09:45](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=585s) While explicit Android MCP tool declarations are our gold standard, we know there's going to be instances where users will ask agents to complete complex or less-common use cases, and this is where our platform-driven Task Automation feature comes in. [09:58](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=598s) Under this model, the OEM pre-loaded agent can use Android's computer-control APIs to simulate touch events on an app surface. [10:07](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=607s) This requires no code changes from developers, and it requires the agent to tap through the UI like a user would, by analyzing the app screen. [10:15](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=615s) To ensure user control and safety, it's critical that agents pause at potentially sensitive actions like payment flows and request user confirmation to proceed. [10:25](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=625s) Let's take a look at an example in action. [10:28](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=628s) Here, a user has taken a photo of a coffee and chocolate tour, and they want to ask Expedia, or they want to ask Gemini to look into Expedia and see if they can find a similar tour, but adjust the number of people to six attendees. [10:40](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=640s) Without needing a direct API integration, Gemini navigates Expedia's UI in the background the same way that a user would. [10:48](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=648s) It then surfaces relevant options that it found and seamlessly hands off to the user when it's time for final confirmation and booking. [10:56](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=656s) It's important to realize that these three different paths are not mutually exclusive, and they're actually better together. [11:03](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=663s) Users and developers should expect agents to be able to coordinate tools across server implementations and then also the local Android app to offer the most com-- comprehensive and holistic experience to users. ### Technical deep dive into Android MCP [11:15](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=675s) All right. [11:16](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=676s) Thank you, Sarah. [11:16](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=676s) We're going to take a little deeper dive into Android MCP. [11:20](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=680s) And when possible, I want you to know I'm pulling the MCP code snippets from Jetpacker, which is our travel sample app, which is actually built with the help of Antigravity. [11:28](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=688s) You've seen this in the keynote. [11:29](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=689s) It's loaded with all sorts of cool intelligent features, including on-device integrations with the ML Kit Gen AI and ML Kit speech recognition APIs. [11:37](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=697s) Now Android MCP and cloud-based MCP have a similar structure. [11:40](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=700s) Both have clients and servers. [11:42](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=702s) Now in the world of MCP, servers provide tools, and clients can discover and call these tools. [11:48](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=708s) Now with Android MCP, your app is the server, and the agent is the client. [11:53](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=713s) The Android Platform plays the part of the tool registry. [11:56](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=716s) Now apps begin by registering a service to provide the server, and we'll begin by adding the AppFunctions Jetpack library dependencies in our sample here. ```kts implementation(libs.androidx.appfunctions) ksp(libs.androidx.appfunctions.compiler) ``` [12:04](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=724s) And then we'll use KSP annotations to help create an `AppFunctionService`. [12:08](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=728s) Now this service means that your app function can functions can actually properly respect Android lifecycles. [12:14](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=734s) We can use dependency injection with Hilt to inject data repositories that our AppFunctions can use. ```kotlin @RequiresApi(36) @AndroidEntryPoint @AppFunctionServiceEntryPoint( serviceName = "JetPackerAppFunctionService", appFunctionXmlFileName = "jetpacker_app_function_service" ) abstract class BaseJetPackerAppFunctionService : AppFunctionService() { @Inject internal lateinit var tripDao: TripDao @Inject internal lateinit var expenseDao: ExpenseDao ... ``` [12:18](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=738s) Sorry. [12:19](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=739s) Back to our diagram. [12:20](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=740s) After getting the service set up for our Android MCP server, the app registers tools in the form of AppFunctions, along with any associated data models used by them. [12:29](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=749s) Now you can register a data model using the `AppFunctionSerializable` annotation, and this KDoc here is pretty cool. ```kotlin /** Represents an event in a trip itinerary. */ @AppFunctionSerializable(isDescribedByKDoc = true) data class TimelineEventSerializable( /** The event's unique identifier. */ val id: String, /** The trip ID this event belongs to. */ val tripId: String, /** The type of event (e.g., TRANSPORTATION, ACCOMMODATION, FOOD_AND_DRINK). */ val type: String, /** The timestamp of the event. */ val timestamp: Long, /** The title of the event. */ val title: String, /** The location of the event. */ val location: String, /** A description of the event. */ val description: String?, /** The title of the trip this event belongs to. */ val tripTitle: String? = null, /** Optional title of a matched itinerary event when added as a reminder. */ val matchedEventTitle: String? = null, /** Execution status of the tool call (e.g., SUCCESS). */ val status: String? = "SUCCESS", /** Human-readable completion message instructing the agent not to repeat the call. */ val message: String? = null, ) ``` [12:34](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=754s) It's just going to become metadata that's used by the agent to understand how to use the model. [12:38](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=758s) You can use the `AppFunctionDeclaration` annotation to register your function as a tool. ```kotlin @AppFunctionDeclaration(isDescribedByKDoc = true) suspend fun getItinerary(tripId: String? = null): List { return withContext(Dispatchers.IO) { val trip = resolveTrip(tripId = tripId) val targetTripId = trip?.id ?: ActiveTripSession.activeTripId ?: tripId?.takeIf { it.isNotBlank() } ?: "2026-1" val dbEvents = eventDao.getEventsForTrip(targetTripId).first() val events = if (dbEvents.isNotEmpty()) { dbEvents } else { val fallback = DummyData.events.filter { it.tripId == targetTripId } if (fallback.isNotEmpty()) { eventDao.insertEvents(fallback) } fallback } events.map { it.toSerializable(tripTitle = trip?.title) } } } ``` [12:44](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=764s) And with `isDescribedByKDoc` set to true, KDoc provides the agent with a description to understand what your function does, and it also helps humans understand it as well. ```kotlin /** * Retrieves the itinerary events (with their eventId, title, type, and location) for a specific * trip. Note: getItinerary returns event summaries only; call getEventDetails(eventId) with a * specific event's ID to retrieve ticket prices, hotel nightly rates, baggage allowance, or * visitor tips. * * @param tripId Optional ID of the trip (e.g., '2026-1'). Defaults to the active trip if omitted. * @return A list of events in the itinerary. */ @AppFunctionDeclaration(isDescribedByKDoc = true) suspend fun getItinerary(tripId: String? = null): List { return withContext(Dispatchers.IO) { val trip = resolveTrip(tripId = tripId) val targetTripId = trip?.id ?: ActiveTripSession.activeTripId ?: tripId?.takeIf { it.isNotBlank() } ?: "2026-1" val dbEvents = eventDao.getEventsForTrip(targetTripId).first() ... ``` [12:52](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=772s) And here are some best practices for parameter design. ```kotlin /** * @param tripId Optional ID of the trip (e.g., '2026-1'). * Defaults to the active trip if omitted. * @return A list of events in the itinerary. */ @AppFunctionDeclaration(isDescribedByKDoc = true) suspend fun getItinerary(tripId: String? = null): List { return withContext(Dispatchers.IO) { // resolveTrip defaults to the active trip if tripId == null val trip = resolveTrip(tripId = tripId) ``` [12:56](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=776s) First of all, you want to focus on the minimal subset of parameters with clear, distinct names. [13:01](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=781s) Make non-essential parameters nullable and resolve their defaults in the function body. [13:05](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=785s) This isn't just a best practice. [13:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=786s) It just really won't work, because Kotlin default values will be ignored because the AppFunctions library passes null for omitted nullable parameters. [13:14](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=794s) I'm not even going to show an example of this first one, but don't use catch-all parameters. [13:19](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=799s) Things like full context or conversation history, they're just bad. [13:23](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=803s) They can lead the agents to pass large amounts of data and lead to exfiltration, and it's just bad practice. [13:30](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=810s) Since the AppFunction executes by default on the UI thread, you want to make sure any long-running operation operation happens in the background by choosing the appropriate coroutine dispatcher. ```kotlin @AppFunctionDeclaration(isDescribedByKDoc = true) suspend fun getItinerary(tripId: String? = null): List { return withContext(Dispatchers.IO) { val trip = resolveTrip(tripId = tripId) val targetTripId = trip?.id ?: ActiveTripSession.activeTripId ?: tripId?.takeIf { it.isNotBlank() } ?: "2026-1" val dbEvents = eventDao.getEventsForTrip(targetTripId).first() val events = if (dbEvents.isNotEmpty()) { dbEvents } else { val fallback = DummyData.events.filter { it.tripId == targetTripId } ... ``` [13:40](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=820s) You want to throw AppFunction exceptions when necessary with detailed information so the agent can understand how to resolve an error. ```kotlin @AppFunctionDeclaration(isDescribedByKDoc = true) suspend fun send( message: Message ): Result { ... throw AppFunctionException( errorCode = AppFunctionException.ErrorCode.ERROR_INVALID_ARGUMENT, errorMessage = "To address has invalid syntax!" ) } ``` [13:47](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=827s) All right, so now we got an AppFunction, let's, here's back to our architecture diagram here. [13:51](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=831s) The agent client app registers with the platform to call AppFunctions. [13:54](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=834s) So, this is something like Gemini, for example. [13:58](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=838s) And to do so, they need to hold a privileged permission, and registered agents can then look up AppFunction metadata from the Android Platform and decide if they should execute one or more registered AppFunctions to get a user's request filled. [14:10](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=850s) Now speaking of one or more AppFunctions, how does the agent actually decide? [14:15](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=855s) Well, one last thing you do need to do is describe your app capabilities in an app metadata file. ```xml ``` [14:20](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=860s) And agents select tools by first reading this description within your `AppFunctionAppMetadata` and then re-- then actually reading the descriptions with the individual AppFunctions. [14:29](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=869s) So, you want to focus the beginning of your description with a brief explanation of the primary domain of your app or tools. [14:35](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=875s) And you can follow this by operational patterns to help the agent know the logical flow of operations, and this is good for outlining prerequisites or directing the agent towards optimal paths for task completion. [14:47](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=887s) And you can add a section on constraints that defines logical limits of input data types or reinforces operational patterns. [14:55](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=895s) So, those are the Android MCP basics, but there are a few more things you can do. [15:00](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=900s) One thing you can do, which is kind of cool, is you can dynamically enable and disable AppFunctions. ```kotlin viewModelScope.launch { val newState = if (enabled) { AppFunctionManager.APP_FUNCTION_STATE_ENABLED } else { AppFunctionManager.APP_FUNCTION_STATE_DISABLED } appFunctionManager?.setAppFunctionEnabled(id, newState) } ``` [15:05](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=905s) And this is important, for example, if you wanted AppFunction to only be available if the user is logged in. [15:12](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=912s) Another thing you can do is create AppFunctions that are scoped to a specific screen or activity in your app. ```kotlin @AppFunctionSignature( scope = AppFunctionMetadata.SCOPE_ACTIVITY, appFunctionXmlFileName = "cart_functions" ) fun interface AddCurrentItemToCart { /** Adds the item currently shown on screen to the cart. Returns the new cart size. */ suspend fun addToCart(quantity: Int): Int } ``` [15:17](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=917s) So instead of declaring the AppFunction, you declare `AppFunctionSignature`. [15:22](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=922s) And then you can implement a handler for the AppFunction in your UI. ```kotlin val addToCartAppFunctionAdapter = appFunctionManager.getHandleAppFunctionRequestAdapter() coroutineScope.launch { appFunctionManager.handleAppFunction( addToCartAppFunctionAdapter.adapt { quantity -> val newCartSize = cart.addItem(quantity) newCartSize } ) } ``` [15:27](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=927s) And the cool part about this is the lifecycle of the handlers is automatically tied to the lifecycle of the coroutine scope you use. [15:33](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=933s) Now you will want to cache this AppFunctions request adapter, sorry the handle AppFunctions request adapter, since the library does make use of reflection under the hood to instantiate it. [15:43](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=943s) And there's also functionality to create scoped AppFunctions whose registration isn't directly tied to the Android lifecycle. [15:50](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=950s) Now, one final thing is we have this thing called access level. ```kotlin @AppFunctionDeclaration(isDescribedByKDoc = true) @AppFunctionAccessLevel( level = AppFunctionMetadata.ACCESS_LEVEL_SELF, isCompatEnforcementEnabled = true, ) suspend fun getItinerary(tripId: String? = null): List { return withContext(Dispatchers.IO) { ... ``` [15:52](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=952s) And you can get and you can use access level of self or system, or trusted, and this indicates which functions are only for use for your own in-app agent, such as one built with ML Kit GenAI, which functions are available only to the system, and which functions are available to trusted system agents. [16:07](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=967s) And to help you build your own in-app agent, we announced already, you saw that ML Kit natively supports AppFunctions as part of its unified tool calling framework. [16:17](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=977s) You define them once and get coverage across both system-level agents and local in-app models which allows you to avoid like fragmented tool declarations. [16:25](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=985s) And to learn more, you can check out this talk, Unlock agentic capabilities in your app with ML Kit GenAI, or you can actually just go and see what the latest stuff in Jetpacker is. [16:35](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=995s) It's pretty cool. [16:37](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=997s) All right, back to Sarah. [16:40](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1000s) All right. [16:40](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1000s) So, Dan just showed you how Android MCP works under the hood. [16:44](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1004s) Let's put it into practice. [16:46](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1006s) Remember, your app is a server and agents are the client, so the goal should be to make sure that your app is a good server for them. [16:53](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1013s) We'll follow Jetpacker, our travel sample app, from the first tool declaration to an actual function call through Gemini. [17:00](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1020s) You can write your AppFunctions by hand or use a coding agent to help you get started. [17:05](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1025s) We've released our AppFunctions skill that can help you start, get started in writing your functions, which you can hook up to Android Studio through Android CLI or use with the coding agent of your choice. ```shell android skills add appfunctions ``` [17:15](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1035s) The skill works in three steps. [17:18](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1038s) First, it analyzes your codebase to identify core user journeys that would make useful tools, like playing media, creating a note, or sending a message. [17:27](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1047s) Then, it generates, it actually generates AppFunction declarations that expose those journeys complete with the implementation and platform support that they need. [17:36](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1056s) For example, Gradle and KSP, App metadata XML, and dependency injection. [17:43](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1063s) Finally, it'll refine your KDoc descriptions because this is exactly what the agent uses to decide when to call your function and how. [17:51](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1071s) Here's the type of result that we're looking for. [17:54](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1074s) This is an `addExpense` function for Jetpacker. ```kotlin /** * Adds an expense to a trip. * * @param tripId ID of the trip. If null, the active trip is used. * @param category Expense category. If null, it's inferred from the title. */ @AppFunctionDeclaration(isDescribedByKDoc = true) suspend fun addExpense( title: String, amount: Double, currency: String, tripId: String? = null, category: String? = null, ): ExpenseSerializable = withContext(Dispatchers.IO) { if (amount <= 0.0) { throw AppFunctionInvalidArgumentException("amount must be greater than zero") } val trip = resolveTrip(tripId) ?: throw AppFunctionElementNotFoundException("No trip with ID $tripId") val expense = Expense(trip.id, title, amount, currency, category = category ?: inferExpenseCategory(title)) expenseDao.insertExpense(expense) expense.toSerializable(remainingBudgetUsd(trip.id)) } ``` [17:59](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1079s) The description is a one line sentence that starts with a verb and a resource: adds an expense to a trip. [18:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1086s) The trip and category fields are nullable because users may not always specify this, but the function will resolve them in the body. [18:14](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1094s) And when a request cannot succeed, it throws an `AppFunctionException` so the agent knows what went wrong. ### Testing app functions and agents [18:21](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1101s) Okay, now let's look at testing. [18:23](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1103s) Your function is are now, your functions are now services that agents can use. [18:27](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1107s) So there's actually two parts that you'll need to test: the way your function behaves and the way the agent behaves. [18:33](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1113s) For your function, we'll start with the unit test that you've probably already written, but there's a couple of things to keep in mind when the agent is the one actually doing the function call. [18:42](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1122s) First, Kotlin default values don't apply here. [18:45](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1125s) An omitted parameter arrives as null, false, or 0. [18:48](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1128s) So, a limit that previously defaulted to 10 will actually be 0 here. [18:53](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1133s) The call also starts on the main thread and calling your function from a test may not always make that obvious, so you should run your standard IO on a background dispatcher. [19:02](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1142s) When something fails, throw an `AppFunctionException`. [19:05](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1145s) Anything else will reach your agent as a generic unknown error with your raw exception message attached. [19:12](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1152s) And your your inputs will come from a model, so you should definitely test for edge case scenarios like blanks, synonyms for dollars like USD, and out-of-range values. [19:24](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1164s) For the agent side, you'll want to know whether an agent picks your tool at the right time and that it executes the function correctly. [19:31](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1171s) Each agent behaves a little bit differently, so it's important to test your functions with any agent that you expect to consume them. [19:38](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1178s) You can use our open-source sample agent to test or you can use Gemini to test this integration. ### Connecting to Gemini with the Connected Apps Platform [19:44](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1184s) If you want to see how Gemini interacts with your AppFunctions or you want to launch an Android MCP on the Gemini app, you can do so with the Connected Apps Platform, which we launched earlier today, which and it's currently in preview. [19:57](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1197s) You can learn more by checking out our earlier talk about extending your app to Gemini and Search through the Connected Apps Platform. [20:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1206s) Once you're part of the preview program, the process to just test with Gemini has two steps. [20:12](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1212s) First, in the connected in the Google Connected Console, we'll create a connected app that Gemini can actually consume. [20:18](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1218s) And then, second, in the Play Console, we'll need to upload your app bundle, so that we can access the AppFunctions, and you can start on an internal testing track, so testers can actually validate the functions. [20:30](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1230s) So, let's start in the Google Connected Console. [20:32](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1232s) Here, you'll create a connected app using the top-right button, "Create an app". [20:38](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1238s) When you create the connected app, choose Android MCP on this screen. [20:42](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1242s) If you were building a server-side MCP implementation, this is where you would upload your server code. [20:49](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1249s) When you click "Set up", any Google Play apps that are associated with your developer account will actually show up here ready for you to link to your connected app. [20:57](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1257s) If you don't already have a Google Play app, you'll need to create one first in the Play Console. [21:02](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1262s) Here, we already have Jetpacker, so we'll go ahead and link that. [21:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1266s) Next, add in the details that users will see in the Gemini front end when they try to interact with your app. [21:12](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1272s) This includes things like an app icon, a display name, the capabilities your app your connected app has, and example prompts that users can use to actually interact with it. [21:21](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1281s) These should be descriptive to help best guide users. [21:26](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1286s) Okay, that concludes the connected app setup, but you might be wondering how the connected app actually gets access to the functions. [21:33](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1293s) Because the functions are contained within your app code, we'll now need to go to the Play Console and upload your app bundle and create a test track, so we can actually test it out with actual users. [21:43](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1303s) So, on this screen, we'll actually upload the app bundle, and then we'll see here that Play can detect the AppFunctions, and you can link them directly to your connected app. [21:54](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1314s) Now, we can see the Jetpack connected app complete with your functions integrated. [21:58](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1318s) Next, we'll need to add testers, so there's someone actually ready to validate your functions and make sure that they're working properly. [22:05](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1325s) So, click into your connected app versions here. [22:08](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1328s) On this screen, you can select "Manage testers" and configure a limited list of testers who will be able to access your connected app in Gemini once it's available. [22:17](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1337s) You can add testers to this testing track the same way that you would for any other release. [22:22](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1342s) Make sure you have the same list of testers for the connected app as you do for the for your Google Play app. ### Demo of Gemini app integration [22:29](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1349s) Okay, now let's test out the integration. [22:32](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1352s) Here's the Jetpacker app on a user's phone, already with a trip logged for this summit. [22:39](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1359s) I'll pull up Gemini to connect the app. [22:41](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1361s) I need to make sure that I have the same version of the app installed on my device that we uploaded in Play earlier, and we should be using an account for Gemini that was added to the tester list. [22:51](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1371s) So here, I'll mention Jetpacker and ask for the confirmation number of my hotel room in Mountain View. [22:59](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1379s) Gemini calls Jetpacker's AppFunctions on the device and answers, "RED11223 at the Redwood Grove Hotel." [23:08](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1388s) Next, I'll ask, "How much budget do I have left for ADS?" [23:14](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1394s) And the function call will return $1850. [23:19](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1399s) Now, I want to make a change, so I'll ask Gemini to add $45 for the airport Uber to my ADS trip. [23:26](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1406s) Gemini calls our `addExpense` function from earlier to add this directly to my itinerary. [23:32](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1412s) And when I open Jetpacker, the expense is already there, and the budget is down accordingly. [23:38](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1418s) If you want to start building connected apps for Gemini today, apply for access to the Connected Apps Platform, which is currently in preview. ### Wrap-up and future of mobile agents [23:48](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1428s) All right. [23:49](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1429s) So, let's wrap this up. [23:52](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1432s) So, we are at the beginning of a massive shift in mobile computing, and it's really time to make sure that your app is ready. [24:02](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1442s) So, start by identifying your app's core high-value user journeys. [24:06](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1446s) Expose your cloud's back end using server-side MCP, and use Android MCP to work with end-to-end encrypted data, local operations, device sensors, and other on-device things. [24:16](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1456s) Now, to prototype now, you can download Android Studio and Android CLI. [24:20](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1460s) You could add the AndroidX AppFunctions beta dependency to your build files, and test your first on-device function locally. [24:28](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1468s) To register for an early start with going to production with Gemini, you can sign up through this QR code here. [24:35](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1475s) And of course, make sure to check to catch these related talks about working with Connected Apps Platform, and also how you can use internal AppFunctions with ML Kit or with your own agent to drive your own in-app agent. [24:47](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1487s) And finally, thank you so much for your time. [24:50](https://www.youtube.com/watch?v=UsqDEfyOH2E&t=1490s) We can't wait to see all the great experiences you're building for Android.