This reference details patterns for registering and executing local tools with the ML Kit GenAI Agent API on Android (AICore Developer Preview).
1. Android AppFunctions (@AppFunctionDeclaration)
The primary declarative mechanism for defining on-device tools is Android
AppFunctions (androidx.appfunctions). AppFunctions provide a standardized
platform model for declaring typed in-app capabilities with automatic schema
generation.
1.1 Declaring an AppFunction
Annotate public methods with
@androidx.appfunctions.AppFunctionDeclaration(isDescribedByKDoc = true). Tool
parameter descriptions are extracted automatically from KDoc @param tags:
package com.example.ai.agent.tools
import androidx.appfunctions.AppFunctionDeclaration
import androidx.appfunctions.AppFunctionSerializable
@AppFunctionSerializable
data class CalendarEvents(
val date: String,
val events: List<String>,
)
class CalendarTools {
/**
* Fetches scheduled calendar events for a specific date in
* YYYY-MM-DD format.
*
* @param date Date string in YYYY-MM-DD format (for example, "2026-10-15")
*/
@AppFunctionDeclaration(isDescribedByKDoc = true)
fun fetch_calendar(
date: String
): CalendarEvents {
return CalendarEvents(
date = date,
events = listOf(
"Team Sync at 10:00 AM",
"Product Review at 2:00 PM"
)
)
}
}
[!NOTE] AppFunction Parameter and Return Rules: 1. Parameters: Tool methods take domain parameters directly (such as
date). You don't need a framework context parameter. 2. Return Types: AppFunctions must return primitive types (String,Int,Boolean, and other primitives) or data classes annotated with@AppFunctionSerializable. ArbitraryMap<String, Any?>dictionaries aren't supported by the AppFunctions compiler.
1.2 Declarative path: whole-app discovery and scoping
For apps where all or specific AppFunctions must execute automatically when the model calls them:
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
// 1. Configure the Agent with AppFunctions enabled
val agent = Agent.builder()
.setName("CalendarAssistant")
.setInstructions("You assist with calendar queries.")
// Automatically discovers AppFunctions in your app package
.enableAppFunctions(true)
// Optional: scope to specific function IDs or method names
.addAppFunctionIds("fetch_calendar")
.build()
// 2. Initialize the Runner
val runner = AgentRunner(agent)
- Scoping to Specific Functions: Use
.addAppFunctionId(id)or.addAppFunctionIds(listOf(...))to restrict the agent's available functions. Supply either the full identifier ("com.example.MyApp#createNote") or the short method name ("createNote"). - Context Resolution:
AgentRunner(agent)automatically resolves the application context usingMlKitContext.getInstance().applicationContext. An explicitAgentRunner(context, agent)overload is available for headless tests or custom context scopes.
1.3 Advanced path: programmatic discovery
When your app needs to inspect discovered AppFunctions, filter them, or decorate them (for example, adding Human-in-the-Loop confirmation dialogs) before registering them with the agent:
import com.google.mlkit.genai.agent.Agent
import com.google.mlkit.genai.agent.AgentRunner
import com.google.mlkit.genai.agent.AgentTool
import com.google.mlkit.genai.agent.AppFunctionAgentTools
// Suspending discovery scoped to calling app package on Dispatchers.IO:
// Overloads:
// - AppFunctionAgentTools.discover() (uses applicationContext)
// - AppFunctionAgentTools.discover(context)
// - AppFunctionAgentTools.discover(enabledAppFunctionIds)
// - AppFunctionAgentTools.discover(context, enabledAppFunctionIds)
val discoveredTools: List<AgentTool> = AppFunctionAgentTools.discover(
enabledAppFunctionIds = listOf("fetch_calendar")
)
val agent = Agent.builder()
.setName("CalendarAssistant")
.setInstructions("You assist with calendar queries.")
.addTools(discoveredTools)
.build()
val runner = AgentRunner(agent)
[!WARNING] Mutual exclusivity guard: Use either the declarative path (
enableAppFunctions(true)oraddAppFunctionIdsonAgent.Builder) or the advanced path (AppFunctionAgentTools.discoverpassed toaddTools), but not both. Registering both emits a runtime warning and exposes duplicate tool schemas to the model.
1.4 Caller scoping and access levels
AppFunctions do not have to be public to the system or third-party assistants. In Android, access control is handled through caller access levels rather than manual caller identity checks (see Android AppFunctions caller scoping documentation).
By default (ACCESS_LEVEL_ANDROID_TRUSTED), platform-certified on-device
assistants holding android.permission.EXECUTE_APP_FUNCTIONS discover and
execute an AppFunction. However, for functions intended exclusively for your
app's internal agent (for example, querying private local databases, modifying
user preferences, or invoking transactional APIs), restrict caller access
using ACCESS_LEVEL_SELF:
import androidx.appfunctions.AppFunctionAccessLevel
import androidx.appfunctions.AppFunctionDeclaration
import androidx.appfunctions.ExperimentalAppFunctionsApi
import androidx.appfunctions.metadata.AppFunctionMetadata
class SecureAccountTools {
/**
* Updates an internal account setting. Restricted strictly to this app's
* agent.
*/
@OptIn(ExperimentalAppFunctionsApi::class)
@AppFunctionDeclaration(isDescribedByKDoc = true)
@AppFunctionAccessLevel(
level = AppFunctionMetadata.ACCESS_LEVEL_SELF,
isCompatEnforcementEnabled = true,
)
fun update_preference(
key: String,
value: String
): Boolean {
// Only your app's in-process agent invokes this function
return true
}
}
Access level hierarchy
| Access level | Constant | Scope and permissions |
|---|---|---|
| Self Only | AppFunctionMetadata.ACCESS_LEVEL_SELF (100) |
Restricted strictly to the host app itself. External assistants, system surfaces, and other apps cannot discover or invoke the function over IPC. |
| System | AppFunctionMetadata.ACCESS_LEVEL_SYSTEM (200) |
Restricted to privileged system callers holding android.Manifest.permission#EXECUTE_APP_FUNCTIONS_SYSTEM (for example, Launcher, Recents) and self. |
| App Trusted | AppFunctionMetadata.ACCESS_LEVEL_APP_TRUSTED |
Restricted to explicit agent package names declared in your application's manifest. |
| Android Trusted (Default) | AppFunctionMetadata.ACCESS_LEVEL_ANDROID_TRUSTED (300) |
Open to platform-certified on-device assistants holding EXECUTE_APP_FUNCTIONS, system callers, and self. |
[!NOTE] HITL Interception and Platform IPC: When an AppFunction is invoked externally by a system assistant over IPC, the request routes directly to your app's
AppFunctionService, completely bypassing any in-processHitlConfirmationToolwrappers. To require sensitive operations to run through your confirmation UI, restrict the AppFunction toACCESS_LEVEL_SELFso that only your in-app agent invokes it.
2. Decorator pattern and human-in-the-loop (HITL)
Sensitive tools—such as financial transfers, data deletion, or outbound
communication—MUST NOT execute autonomously without user consent. The Agent
API provides built-in support for the Decorator pattern on AgentTool.
2.1 Tool delegation
All tools registered with Agent must originate from Android AppFunctions
(validated at construction time). AgentTool provides a public delegating
constructor AgentTool(delegate: AgentTool, name: String = delegate.name,
description: String = delegate.description) that wraps an underlying
AppFunction tool discovered using AppFunctionAgentTools.discover (or another
decorator wrapping one).
2.2 Implementing a confirmation wrapper
import com.google.mlkit.genai.agent.AgentTool
/**
* Decorator tool that intercepts execution to prompt for user confirmation.
*/
class HitlConfirmationTool(
delegate: AgentTool,
private val onConfirm: suspend (
toolName: String,
args: Map<String, Any>
) -> Boolean
) : AgentTool(delegate) {
override suspend fun execute(args: Map<String, Any>): Map<String, Any?> {
// 1. Request user confirmation before invoking the sensitive tool
val confirmed = onConfirm(name, args)
if (!confirmed) {
return mapOf(
"status" to "CANCELLED_BY_USER",
"message" to "The user declined to confirm this action."
)
}
// 2. Delegate to the underlying tool execution
return checkNotNull(delegate).execute(args)
}
}
2.3 Wrapping discovered AppFunctions
val allTools = AppFunctionAgentTools.discover(context)
val configuredTools = allTools.map { tool ->
if (tool.name == "deleteCalendarEvent" || tool.name == "sendInvites") {
HitlConfirmationTool(tool) { name, args ->
showUserConfirmationDialog(name, args)
}
} else {
tool
}
}
val agent = Agent.builder()
.setName("SafeAssistant")
.addTools(configuredTools)
.build()
val runner = AgentRunner(agent)
3. Automated orchestration loop
Unlike manual function calling where the app must parse raw function call
responses and manually craft follow-up turns, AgentRunner handles the complete
loop internally:
- The model analyzes user prompt context and decides to invoke a tool.
AgentRunneremitsAgentEvent.FunctionCall(name, args).AgentRunnerexecutes the corresponding AppFunction tool.AgentRunneremitsAgentEvent.FunctionResponse(name, response).AgentRunnerfeeds the response back into inference until the model emitsAgentEvent.TextMessage(text, isPartial)ormaxTurnsis reached.
import com.google.mlkit.genai.agent.AgentEvent
session.runAsync(content).collect { event ->
when (event) {
is AgentEvent.TextMessage -> appendText(event.text)
is AgentEvent.FunctionCall ->
showToolProgress("Running ${event.name}...")
is AgentEvent.FunctionResponse ->
showToolProgress("Completed ${event.name}")
else -> {} // Required: AgentEvent is non-exhaustive for forward compat
}
}
4. Deprecated and removed tool patterns
The annotations @Tool and @Param from package
com.google.mlkit.genai.schema.annotations.*, reflection-based registration
using AgentTool.fromAnnotatedObject and
Agent.Builder.addTools(annotatedObject: Any), and dynamic FunctionTool
have been deprecated or removed. All tools added to Agent must be backed by
Android AppFunctions.
Migration guide
Legacy pattern (@Tool) |
AppFunctions pattern (@AppFunctionDeclaration) |
|---|---|
@Tool(description = "...") |
@androidx.appfunctions.AppFunctionDeclaration(isDescribedByKDoc = true) |
@Param(description = "...") |
KDoc @param tags or AppFunctions metadata |
Agent.builder().addTools(MyTools()) |
Agent.builder().enableAppFunctions(true) |
AgentTool.fromAnnotatedObject(instance) |
AppFunctionAgentTools.discover(context) |
Migrate existing code to @androidx.appfunctions.AppFunctionDeclaration
with Agent.Builder.enableAppFunctions(true) and AgentRunner(agent)
for declarative registration, or AppFunctionAgentTools.discover for
custom filtering and decoration.