Excessive agency is a vulnerability that occurs when a large language model (LLM) is granted unnecessary or overly permissive abilities to interact with other systems. When an LLM can call external tools, plugins, or functions (its "agency"), this vulnerability allows it to perform actions that are unintended, unauthorized, and potentially harmful. An attacker can exploit this by using prompt injection or other manipulation techniques to trick the LLM into using its granted agency for malicious purposes. The core issue is not just that the LLM can take actions, but that the scope of those actions is too broad and poorly controlled.
Why Android developers should care
Granting an LLM excessive agency within your Android application can lead to severe security incidents:
- Unauthorized system access: If the device's file system and storage resources or the ability to perform network calls are exposed to the model through function calling, an attacker could use prompt injection to access, modify, or delete files on the device (for example, user documents, app data) or connected network resources.
- Data exfiltration: If an app uses a function calling to give an LLM access to local data (such as, Room databases, SharedPreferences, or internal APIs). A malicious prompt could trick the model into retrieving sensitive information and passing it to an external tool, such as an email or network request function.
- Compromise of other functions/systems: If the LLM has agency over other functions (for example, sending SMS, making calls, posting on social media using implicit intents, modifying system settings, making in-app purchases), an attacker could hijack these functions to send spam, spread disinformation, or perform unauthorized transactions, leading to direct financial loss or user harm.
- Denial of service: If an LLM is integrated with function calling that exposes database queries or network requests, a malicious prompt could trigger these actions repeatedly. This may lead to degraded system health, such as excessive battery drain, data overages, or local resource exhaustion.
Mitigations for Android app developers
Mitigation for excessive agency in Android apps focuses on applying the principle of least privilege to every tool and function the LLM can access or trigger.
Limit the AI's toolbox (granular versus open-ended functions):
- Provide minimal tools: The LLM should only have access to the specific tools (functions, APIs, intents) that it absolutely needs to do its job within your app. If it doesn't need to be able to browse the web or send an email, don't expose those capabilities to it.
- Use simple, single-purpose tools: Design tools with a limited and specific
scope. For example, provide a tool that only reads a specific type of user
setting rather than a generic tool that accepts open-ended parameters to
access multiple data sources. Avoid exposing powerful system-level APIs,
such as
Runtime.getRuntime().exec(), to an LLM by allowing the model to define the command or arguments.
Restrict the AI's power
- Fine-grained Android permissions: When an LLM-triggered function interacts with Android system resources or other apps, verify that your app only requests and holds the absolute minimum Android permissions required.
- Per-user permissions: When the LLM performs an action on behalf of the user, it should do so with that user's specific permissions and context. An action taken by an LLM must be a direct response to a specific user command.
Keep a human in charge (user consent for critical actions)
- Require user approval: For any important or risky actions that an LLM might suggest or attempt to perform (for example, deleting data, making in-app purchases, sending messages, changing critical settings), always require explicit human approval using a confirmation dialog in your UI. Think of it as needing a manager to sign off on a major decision.
Trust but verify (input/output validation and robust backends)
- Backend security: Don't just rely on the LLM to decide if an action is allowed. Any backend services or APIs that the LLM-triggered functions connect to should have their own robust authentication, authorization, and input validation to double-check every request and verify it's legitimate and within expected parameters.
- Clean up data: Just like with other vulnerabilities, it's critical to sanitize and validate both the input that goes into the LLM and the parameters generated by the LLM for function calls to catch any malicious instructions or unexpected outputs before any action is executed.
Summary
Excessive Agency is a critical vulnerability where an LLM has overly broad permissions to interact with other systems or functions, allowing it to be tricked into performing harmful actions. This can lead to unauthorized data access, system compromise, financial loss, or user harm in Android applications. Mitigation relies heavily on the principle of least privilege: strictly limit the tools and Android permissions available to the LLM, verify each tool has minimal and specific functionality, and require human approval for all high-impact operations.