On Android, you can deliver rich generative AI experiences without needing a network connection or sending data to the cloud. On-device AI is a great solution for use-cases where low latency, low cost, and privacy safeguards are your primary concerns.
For on-device use-cases, you can take advantage of Google's Gemini Nano foundation model. While it is smaller than other Gemini models running inference in the cloud, you can fine-tune Gemini Nano to perform specialized tasks as well as its larger counterparts. Gemini Nano runs in Android's AICore system service, which leverages device hardware to enable low inference latency and keeps the model up-to-date.
Gemini Nano and AICore are currently only available on Google Pixel 8 Pro and Samsung S24 Series devices, with support for more devices coming soon.
Access to Gemini Nano API and AICore is provided by the Google AI Edge SDK, a comprehensive suite of tools for on-device ML.
Architecture
As a system-level module, you access AICore through a series of APIs in order to run inference on-device. You can also provide a LoRA fine-tuning block to further optimize the performance of Gemini Nano for your application. In addition, AICore has several built-in safety features, ensuring a thorough evaluation against our safety filters. The following diagram outlines how an app accesses AICore to run Gemini Nano on-device.
Keeping user data private and secure
AICore adheres to the Private Compute Core principles, with the following key characteristics:
Restricted Package Binding: AICore is isolated from most other packages, with limited exceptions for specific system packages. Any modifications to this allowed list can only occur during a full Android OTA update.
Indirect Internet Access: AICore does not have direct internet access. All internet requests, including model downloads, are routed through the open-source Private Compute Services companion APK. APIs within Private Compute Services must explicitly demonstrate their privacy-centric nature.
Benefits of accessing AI foundation models with AICore
AICore enables the Android OS to provide and manage AI foundation models. This significantly reduces the cost of using these large models in your app, principally due to the following:
Ease of deployment: AICore manages the distribution of Gemini Nano and handles future updates. You don't need to worry about downloading or updating large models over the network, nor impact on your app's disk and runtime memory budget.
Accelerated inference: AICore leverages on-device hardware to accelerate inference. Your app gets the best performance on each device, and you don't need to worry about the underlying hardware interfaces.
Supported functionality
AICore supports the following devices and modalities:
- Supported Devices: AICore is currently available on Google Pixel 8 Pro and Samsung S24 Series devices.
- Supported Modalities: AICore currently supports text modality for Gemini Nano.
Additional device and modality support are areas of active investment.
Use cases
Some tasks you can accomplish with Gemini Nano through AICore are the following:
- AI-enhanced content consumption: text summarization, answering questions, and entity extraction.
- AI-assisted content generation: proofreading, grammar correction, writing assistance, and contextual smart replies.
- Classifying text: sentiment detection or mood analysis
- Privacy: unlock generative AI use-cases to privacy focus applications
Several Google products already use AICore on Pixel, including Pixel Voice Recorder and Gboard.
Pixel voice recorder
The Pixel Voice Recorder application uses Gemini Nano and AICore to power an on-device summarization feature.
Gboard smart reply
Gboard leverages on-device Gemini Nano with AICore to provide accurate smart replies.