# Agentic engineering with Claude Code and Android CLI - Speakers: Steven Jenkins (Product Manager, Android Studio), Ben Trengrove (Developer Relations Engineer) - Track: Tools & Performance - Video: https://www.youtube.com/watch?v=Mb9H43ia0kE - Page: https://developer.android.com/events/dev-summit/talks/agentic-engineering-with-claude-code-and-android-cli ## About the talk Stop copy-pasting into AI chat windows and start delegating to your agent. This practical, example-driven session shows you how to level up your agentic engineering using Claude Code and the Android CLI. We will walk through a complete workflow to automate complex development tasks and ship features faster. You will learn to: - Optimize the token efficiency of your harness using Android CLI, Skills, Hooks and MCP. - Offload and parallelize long-running tasks to subagents. - Set up continuous verification loops that stop early termination and run until tests pass. ## Summary - Android CLI gives coding agents shell access to Android tools such as emulators and Firebase Crashlytics. It sits in your shell space and works across agents. - Keep agent conversations to around 150,000 tokens to prevent context rot. Slice work into scoped issues and run Gradle in quiet mode to avoid long logs. - Keep `AGENTS.md` under 250 lines and reserve it for permanent rules such as architecture constraints. Letting an agent edit this file degrades performance and raises costs. - Deterministic hooks and context-specific skills manage agent actions better than sprawling files. Install only vetted skills or start with the base Android CLI skill. - Verification loops stop developers from acting as manual QA testers. Tier checks to run fast unit tests first and slow emulator runs last. ## Resources - [Android CLI installation](https://goo.gle/android-cli) - [Agent modification study](https://arxiv.org/pdf/2602.11988) - [Android Bench](https://developer.android.com/bench) ## Topics claude-code, android-cli, agentic-engineering, benchmarking, performance ## Transcript ### Introduction and environment setup [00:05](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=5s) Hello, I'm Ben, Android Developer Relations. [00:09](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=9s) I'm Steven, product manager. [00:11](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=11s) And today we're going to talk about making apps with Claude Code and Android CLI, agentically. [00:18](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=18s) So, I think it's fair to say our role as developers, it's changing a bit, and things are moving crazy fast, so fast, in fact, that, let's be honest, the content in this talk is probably already out of date, especially if you're watching online. [00:33](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=33s) So, we took a forward-leaning stance with what we wrote, with the info we're presenting, but the demos they run today, they're grounded in reality, so they should be reliable now. [00:44](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=44s) The other thing is we've used Claude Code for this talk, but the information, it's agent-agnostic. [00:49](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=49s) It's more of a mindset, so it should apply to whatever you prefer to use when you're building your apps. [00:56](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=56s) So, we want you to start thinking about a world where you're managing teams of autonomous agents and how you set them up for success. [01:05](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=65s) When they fail to implement a feature or something like that, instead of just giving up and going, "Oh, AI can't do this yet," we need to step back and think what piece of information was missing, and kind of treat it how we treat bugs today. [01:17](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=77s) And if you've read anything coming out of the teams that make all these agent harnesses, Claude Code, Codex, Antigravity, you'll see that this is how their teams actually develop those harnesses, and if we take them as kind of a glimpse into the future, that's probably where we're all heading. [01:33](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=93s) So, it's probably never been a more exciting time to be a builder, to be an Android developer, but if you're like me, you might find it hard to know exactly what to be focusing on. [01:46](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=106s) And we, as developers, we need tools and we need workflows that really scale with intelligence, so that as a new model extends the frontier, all of our hard work doesn't become redundant. [02:01](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=121s) We want each model improvement to lead to more intelligent workflows for our agents, which ultimately culminate into better outcomes for our users. [02:13](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=133s) Today, we'll cover four topics. [02:15](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=135s) First, we'll explain why it's so important to set your agents up in environments for them to succeed. [02:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=144s) Then, we'll provide an overview on context and how best to manage it. [02:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=149s) After that, we'll touch on some of the best tools that we have available for Android developers working inside of agents, and finally, we'll make a case for adding loops to your agentic workflows. [02:44](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=164s) All right, let's dive in. [02:45](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=165s) So, environment. [02:47](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=167s) But like, what does that actually mean? [02:50](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=170s) Well, it's like I was saying, we're trying to create a space where our agent can succeed, and can succeed without you having to sit there and constantly, like, re-prompt it and put it back on track, because when you do that, that's when you unlock being able to run parallel agents, and that's where, like, the real productivity gains come in, when you can have these teams of agents all doing your work for you. [03:12](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=192s) But that goes way beyond just having access to the code. [03:16](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=196s) They need access to accurate documentation, which includes project specs and rules for how to work in that project. [03:22](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=202s) That's kind of the obvious one. [03:23](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=203s) Some other things people are doing, people are hooking their agent up to their team chat space now, not just to kick off agents, but to actually provide the context of how decisions were made that led to the feature they've been tasked to do, because you as a developer, that's the kind of subconscious information you're using when you solve problems, and so it can be helpful for your agent, too. [03:45](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=225s) Also like previous decision history and pull requests. [03:50](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=230s) They should be able to see the comments you're getting on your pull requests, the kind of things that they get pinged for, things like that, and then you'll find they can work and fix that up. [03:58](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=238s) And also, you wouldn't expect to be able to write Android code without access to a device or an emulator, and so you shouldn't expect your agent to be able to do that either. [04:07](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=247s) I know they seem like they're amazing and can do anything, but an emulator really helps. [04:12](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=252s) Same with profilers. [04:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=254s) If you want them to work on performance, they should be able to profile, which involves having access to a real device. [04:19](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=259s) And any back-end services that your app uses, ideally your agent would be able to talk to them, too. [04:27](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=267s) And that's where Android CLI comes in. [04:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=269s) If you haven't heard of it was in the What's New in Tools talk as well, but Android CLI is our new CLI for interfacing, just with various Android tools. [04:39](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=279s) It's not an AI product in itself, it's just a CLI, a command-line interface. [04:43](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=283s) So, it provides you access to things like the Android SDK Manager, being able to start emulators, a whole bunch of things. [04:51](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=291s) And it also connects to tools and back ends like Crashlytics. [04:55](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=295s) It works across whatever agent you're using, it just sits in your shell space. [04:58](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=298s) So, yeah. [05:00](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=300s) You can install it at this link, but I think it's easier if we just see it in action. ### Demo: Fixing an ANR with Android CLI [05:10](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=310s) All right. [05:10](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=310s) So, rather than simply explaining why it's so important to set your agents up for success, we wanted to really apply this principle to a production application that we built called Sideline. [05:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=324s) And Sideline is an Android application for tennis players that provides them with real-time feedback on ball location, player location, court location. [05:36](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=336s) It provides offline coaching feedback through video replay, and it provides fitness data for things like fastest speed, distance traveled, and longest rallies. [05:47](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=347s) It's all powered through custom TF Lite models running directly on the Android device. [05:53](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=353s) And first off, I just want to say we obviously cannot solve problems that we don't know exist. [06:01](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=361s) So, using Android CLI, we added Firebase Crashlytics into our developer environment, which allowed for us to find some problems that we didn't even know were occurring. [06:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=374s) And the first one was that there was an application not responding error, or an ANR, happening on a Pixel 11 Pro Fold device, which didn't reproduce in the emulator. [06:28](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=388s) So, fortunately, Android CLI has support for Android Device Streaming. [06:33](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=393s) And what the agent did once we told it to go fix this problem is it created a connection to the device that the ANR was taking place on, which was particularly useful for me because at the time, I didn't have a Pixel 11 Pro Fold to even validate the fix on. [06:51](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=411s) So, by giving the agent the ability to go and get access to the device that it needed to validate the fix, we were, ultimately, able to expand the capability set of what's possible. [07:04](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=424s) And to save time, I ran all the experiments ahead of time and I've summarized them here. [07:10](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=430s) And without getting too into the details, the issue that we were having was an affinity call having to do with Android 17. [07:18](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=438s) It was a quick two-line fix, which looked good to me. ```cpp setenv("KMP_AFFINITY", "disabled", 1); setenv("KMP_INIT_AT_FORK", "false", 1); ``` [07:21](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=441s) So, I created the pull request, merged it, and updated with a new build submitted to Play. [07:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=449s) So, had we not set up the coding environment with the capabilities that it needed to succeed, we wouldn't have been able to identify a real problem, nor would we have been able to validate a fix. [07:43](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=463s) So, with the 1.0 builds, you can see that the problem has been resolved. [07:48](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=468s) However, new issues have arisen and we'll get to them later. ### Managing context and token limits [07:56](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=476s) All right. [07:57](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=477s) Next up: context. [07:58](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=478s) So, about like 1 and 1/2, 2 years ago, which is not that long a time, but in the AI space, that is a lifetime, there was this race to have larger and larger context windows, and the theory was, the more we could kind of throw at the model, the better results we would get, and we were doing crazy things like we tried to include the entire developer.android.com in the prompt, it didn't really work. [08:21](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=501s) But even though, context windows have largely plateaued, model capabilities are still rising. [08:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=509s) So, this is Android Bench 1.0. [08:31](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=511s) Android Bench is our benchmark that benchmarks, the ability of AI to write Android code. [08:36](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=516s) The tasks in it are realistic. [08:37](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=517s) They're the kind of things that you work on, and we can see context windows plateaued at that 1 million mark, but model capability continues to rise. [08:47](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=527s) But context, is still a fixed budget, and you want to save it for what's important. [08:54](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=534s) You don't want things in there that aren't helping that exact problem you're working on. [08:59](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=539s) And a lot of the agent harnesses aren't actually exposing that full 1 million context window to you because they find, like, it's pointless. [09:06](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=546s) The results just get worse when they do that. [09:09](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=549s) And there's actually a sweet spot, and it does change depending on the model, but if you can keep your conversations to about 150,000 tokens, that's where you're getting the full power of the model without any context rot. [09:22](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=562s) But context isn't just what you're typing in. ```json [ { "role": "system", "content": "You are an expert Android engineering agent..." }, { "role": "workspace_context", "content": { "project_root": "/workspace/android-app", "build_system": "Gradle (Kotlin DSL)", "target_sdk": 35 } }, { "role": "available_tools", "content": [ { "name": "read_file" }, { "name": "edit_file" }, { "name": "bash" }, { "name": "skill_android_cli" } ] }, { "role": "user", "content": "ProfileViewModelTest is failing when loading errors. Fix the ViewModel and verify with unit tests." }, { "role": "assistant", "tool_calls": [ { "name": "read_file", "arguments": { "path": "app/src/main/.../ProfileViewModel.kt" } } ] }, { "role": "tool", "content": "class ProfileViewModel(...) : ViewModel() { ... // full file contents + Gradle output appended here }" } ] ``` [09:25](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=565s) There's actually quite a lot to it behind the scenes that you're not seeing. [09:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=569s) It's built up of a number of parts. [09:31](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=571s) First is the system instruction. [09:32](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=572s) The system instruction is provided by your agent. [09:35](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=575s) It's kind of just like a prompt that tells it how to work with this agent, the kind of behavior it should do. [09:40](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=580s) But, also, workspace information, and like, like your current working directory, what SDKs you have, any installed skills, we'll talk about that in a second, and MCP tools, any tools provided by your agent as well. [09:51](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=591s) The model needs to know it can call them, so that takes up context. [09:55](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=595s) But the big one is the full message and tool call history is sent with every prompt, every new message you send, with your agent. [10:04](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=604s) And that, that can really add up, especially with Gradle. [10:09](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=609s) I think, as we all know, Gradle logs are really long, like really, really long, very verbose. [10:13](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=613s) Most of that is not required. [10:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=614s) So there's a quick tip to watch out for that. [10:16](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=616s) You can run Gradle in quiet mode. ```shell ./gradlew assembleDebug --quiet ``` [10:18](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=618s) Or the bigger agent harnesses do truncate this for you. [10:22](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=622s) So, Claude Code and Android Studio, we're not sending the full Gradle log up. [10:26](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=626s) But if you're using one of the minimal agents, because their whole philosophy is kind of send everything to the model, model knows best, so mini-SWE-agent, it's sending that full Gradle log, and that can get really expensive really fast. [10:41](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=641s) But how do we stay within this 150,000 magic window? [10:45](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=645s) Well, the key is to use a workflow. [10:47](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=647s) And there are heaps of workflows out there. [10:49](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=649s) The one I really like is by a guy called Matt Pocock, and he has a series of skills to help you kind of manage these sessions. [10:57](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=657s) So, it starts with the skill called "grill-me". [11:00](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=660s) What you do is you run this skill when you're first starting either a whole project or just a new feature inside the project, and it gets your agent to ask you a bunch of questions. [11:09](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=669s) So, you might just say, "grill-me. [11:11](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=671s) I want to build I don't know, a new time alarm manager in my app." [11:16](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=676s) And what it'll do is it'll start asking questions back to you about like what should we implement it in, where does it belong. [11:21](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=681s) And it gets all the ambiguity out of the problem and out of your head down into a pa- into an issue. [11:28](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=688s) We then run the next skill, which is kind of writes a big spec doc. [11:31](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=691s) There's another skill that then slices it into issues, and we file those in whatever issue manager you prefer to use. [11:38](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=698s) And then the idea is each one of those issues should fit roughly in that 150,000 window. [11:44](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=704s) And then from there, you pick one issue either you're working in the terminal or your IDE, whatever you prefer, build, verify loop, we clear the context at the end, no more conversation history, and we stayed within the window. [11:59](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=719s) All right. ### Best practices for AGENTS.md [11:59](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=719s) Next up, let's talk about `AGENTS.md`. [12:02](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=722s) `AGENTS.md` is where you can provide extra information for your agent that applies to whatever problem it's working on in this project or in your whole workspace, if that's what you choose to do. [12:12](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=732s) But because it is provided to every prompt, it's important to keep it to things that only apply ideally all the time. [12:19](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=739s) So, the kind of things that work are architecture and module rules. [12:23](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=743s) They're the kind of things that your agent couldn't have possibly known, especially if you work on a closed source app. [12:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=749s) But are really important to understand just to save it reasoning at the start about where does this feature belong. [12:34](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=754s) So, you can put things like UI components belong in the design system module, or another thing that works really well is common code review mistakes, or just common things you're picking up that you're having to type in all the time. ```markdown # AGENTS.md - UI components belong in :core:designsystem - Never use java.util.Date; always use kotlinx.datetime ## General guidance - Avoid changing public API signatures. Anything under an `internal` directory is not part of the public API and may change freely. - The SDK code is written for Java 8. ``` [12:46](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=766s) So, you know, in this project, we prefer to use `kotlinx.datetime`, things like that. [12:51](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=771s) And I really wanted to provide sort of best practices that always apply, but the reality is this changes with every model release. [12:58](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=778s) Different models have different behaviors when it comes to extra context. [13:02](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=782s) So, in an example, back in the Opus 4.6 days, it was really common to have big capital letters that said, "Never do this." [13:08](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=788s) And then when Opus 5 came out, Anthropic released a statement saying, "Actually, capitals don't work that well anymore. [13:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=794s) Just use lowercase." [13:16](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=796s) And so this is what I mean: you have to keep on top of it. [13:18](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=798s) But some things that have remained pretty universal is to keep it short. [13:22](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=802s) So, keep it under about 250 lines. [13:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=804s) And this one, this next one is a bit polarizing, cuz I know a lot of people love to do this, but a study came out in February this year that showed if you let your agent modify its own `AGENTS.md`, it degrades performance by about 2 to 3%, which, you know, it's not that big a deal. [13:40](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=820s) But the important part was it increased costs by 20%. [13:43](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=823s) So, we lost a tiny bit of performance, and we blew out our costs, all from letting the agent write its own `AGENTS.md`. [13:49](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=829s) A way around this I've seen, you can get your agent to write a retrospective at the end of each pull request, and then you as a team in your retrospectives can read them all together and decide what modifications you're going to make, because this file is kind of becoming the key piece of critical team infrastructure if you're working in big teams with agents. [14:12](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=852s) So, just real quick, here's the `AGENTS.md` for the Sideline application that we built. [14:17](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=857s) There are really two things that I want to call out here. [14:21](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=861s) The first is that having a clearly defined architecture will save you tokens and save you on context length. [14:30](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=870s) Basically, it allows for your agent to avoid having to constantly look into more of your codebase and make more and more inferences. [14:39](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=879s) It provides a very clearly defined guideline for it to follow. [14:45](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=885s) And the second that that I wanted to call out is particularly for the closed-source applications. [14:51](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=891s) Obviously, if you're closed source, none of the agents have been able LLMs have been able to train on your codebase. [14:57](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=897s) So, in these cases, it's particularly important to just spend that extra time, spend those cycles constantly cleaning up, reviewing, refining, improving your `AGENTS.md`, because it will benefit you in the long run. [15:16](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=916s) So if we shouldn't just be putting everything in `AGENTS.md`, how are we meant to change the agent or model behavior? [15:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=924s) Well, the first one is hooks, and hooks are actually really good for the kind of things that people put in `AGENTS.md` like say, always run the linter when you're finished writing. ```json // settings.json "hooks": { "Stop": [{ "command": "./gradlew lintDebug" }] } ``` [15:34](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=934s) That can be a hook. [15:35](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=935s) What a hook is your agent is exposing all these events that you can subscribe to, and there's ones for like pretty much anything you can think of, but the main ones are start session, stop session, read and write files. [15:46](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=946s) So, instead of writing in the bottom of your `AGENTS.md`, "always run the linter," or, "always run my test suite before saying you're done," you just have a stop hook instead that can run a deterministic script, and it helps rein in the slight randomness of AI. [16:00](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=960s) Another one, if you have files that you really don't want your agent to be able to read or write, so you've got some secrets file, you don't want that being sent up to a server, or `AGENTS.md`, you don't want your agent to be able to write to it, you can have a hook for that. [16:13](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=973s) So you have a read or write hook, you get passed the file it's trying to edit, and you can just return, "No, you can't do that." [16:20](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=980s) It's just a way to add determinism. ### Agent skills and architectures [16:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=984s) The next one is skills. [16:26](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=986s) And there's actually skills to turn your `AGENTS.md` into hooks now. [16:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=989s) That's a whole another thing. [16:31](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=991s) But skills are how to add context only when it's relevant. [16:35](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=995s) So `AGENTS.md` is for things that are always relevant. [16:39](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=999s) Skills are for things that are relevant in certain situations, mainly documentation. [16:44](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1004s) So skills are made up of three parts: they have a name, they have a description, and that's the important part, this description is limited to 1,024 characters, and then they have a body. ```markdown --- name: android-cli description: > Loaded into context (≤1024 chars) --- # Skill Body ``` [16:52](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1012s) And the body can be massive, the body can even include other scripts, which is pretty common. [16:57](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1017s) You can do quite a lot with skills. [16:59](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1019s) Let's have an look at an example. [17:01](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1021s) This is the Android CLI description. [17:03](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1023s) You don't have to read all that. [17:04](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1024s) Just know that it's like a short description that describes when Android CLI is relevant. [17:09](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1029s) And then this is the body. [17:10](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1030s) It's massive, so we don't want to load all this all the time, right? [17:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1034s) We only want to load this when we're trying to use Android CLI, and that's what a skill enables us to do. [17:20](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1040s) But it's important to be selective about the skills you install. [17:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1044s) We don't want to just be blindly installing those repos we see online that are hundreds of agentic engineering skills. [17:32](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1052s) One, that is a massive security risk. [17:34](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1054s) You have no idea what's in those skills, it could be telling your agent to do anything. [17:37](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1057s) And we do have a set of Android skills, I promise we vetted them, they don't say that. [17:42](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1062s) But two, 100 descriptions times 1,000 characters is 100,000 characters you've just added before you've even started coding. [17:50](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1070s) So you're already like creeping up on that window. [17:52](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1072s) And so it's that's why with the Android skills, we recommend just installing for everything the Android CLI base skill, and from there Android CLI can search the available skills and install the ones that make sense for your app. ```shell > android init ✓ Installed base skill: android-cli # Audit & add skills only when needed: > android skills find compose > android skills add edge-to-edge ``` [18:05](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1085s) Or if you do have a look, it's nice to know what skills are available, you can add the ones manually that makes sense for your project, say like the Compose skills, the edge-to-edge skills, those are pretty universal. [18:16](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1096s) There are skills to audit skills, of course. [18:19](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1099s) So there's `/skill-doctor` in Claude Code, and that will have a look and tell you like what skills you're including that are never activated, so you're just wasting that thousand character description. [18:28](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1108s) And if you're not in Claude Code, pretty much every agent harness has some form of stats screen or audit screen to do the same thing. [18:36](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1116s) So again, Android CLI, that's the link. [18:38](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1118s) But it has a skills list command, and then say you have Navigation 3 in your project. ```shell android skills list android skills add navigation-3 ``` [18:43](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1123s) Navigation 3 is quite a new library, so a lot of training hasn't happened on it yet, you can add the Navigation 3 skill. [18:52](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1132s) We often Oh, sorry. [18:53](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1133s) The next one is MCP. [18:55](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1135s) But we often get asked why did we make Android CLI a CLI and not an MCP server? [19:01](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1141s) Well, we've been building these CLI things like way before AI existed, way before the mouse even existed, we were building CLIs, so they're fast, they're composable, and because they run locally in your shell session, they're effectively free to run. [19:15](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1155s) And also, benchmarks like Terminal Bench have become the biggest benchmarks when it comes to software engineering, and so model makers have been aggressively climbing against terminal usage, so they're really good at using CLIs these days. [19:32](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1172s) But that doesn't mean MCP has no place. [19:35](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1175s) MCP servers are really great for the external services you're using already, especially if they're already hosting an MCP service. [19:43](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1183s) You can hook straight into it, and it's an install-free way to get extra context into your agent. [19:48](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1188s) Things like GitHub, Figma, Slack are exposing MCP servers now. [19:52](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1192s) And then things that we've always run in the shell can stay in the shell, so Android commands through Android CLI, but Perfetto and Gradle as well just make sense to have them local where they have access to everything you use in your development. [20:05](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1205s) And I used to work on performance. [20:07](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1207s) I used to always say, Perfetto, it's the one tool you need to solve any performance problem. [20:12](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1212s) But the learning curve was so steep, as actually we just saw in the last talk from Rahul. [20:17](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1217s) Let's have a look at how AI can help improve performance. ### Optimizing app performance with AI [20:22](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1222s) Thanks, Ben. [20:23](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1223s) So, in order for our application, Sideline, to work, it needs to run at 30 frames per second, and in real time. [20:33](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1233s) And as you can see here on this Pixel 10 Pro Fold, we're only achieving about 20 to 22 frames per second. [20:42](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1242s) These dropped frames are going to be a problem, because it means that the application is not going to work as advertised, so we're really going to need to fix this. [20:53](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1253s) With Android CLI, we have the Android profiler commands. [20:58](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1258s) And when we add it to our coding agent, and we instruct it to meet this 30 frame per second requirement, two interesting things take place. [21:10](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1270s) The first is that it finds a fix using the Android profiler command. [21:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1274s) It recorded system traces on the attached Pixel 10 Pro Fold device, and it found that its AI threads were running on the slowest cores on the device. [21:27](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1287s) And the second is that it used Android docs search, interestingly, without us explic-- explicitly asking it for us it to do so, and pulled the official guidance for the Performance Hint API. [21:43](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1303s) What the agent learned to do was to optimize the system schedule, and by doing so, it was able to meet the 30 frames per second requirements. [21:55](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1315s) So, essentially what happened is the fix followed the guidance. [21:59](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1319s) So, with Android CLI, the key is, you know, through hooking it up with your with the agent, it helps you find the right tool, even if you're not familiar with that tool, for whatever job it is you need to be able to accomplish. [22:18](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1338s) So, once we run it, you can see that Sideline meets the 30 frames per second requirement that users have, and it now works on a Pixel 10 Pro Fold. ### Implementing task verification loops [22:32](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1352s) All right. [22:33](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1353s) Now, we're going to bring it all together with loops. [22:35](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1355s) So, I'm sure we've all done this, you ask your agent to implement some big change, and it comes back and says it's done. [22:41](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1361s) And if you're installing it and sort of like manually clicking around on the app yourself, and you type back the bugs, that's kind of what we need to stop doing. [22:50](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1370s) We need to stop being our agent's QA tester and define up front what we consider done to be. [22:57](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1377s) So, here's an example. [22:58](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1378s) We want to migrate the whole project to Nav3, and then the agent goes away, it does take a while, but it confidently comes back and says, "Migration complete. [23:06](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1386s) I'm the best." [23:07](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1387s) But then, of course, you rotate the phone, the app crashes, it missed all the slide transitions, it didn't clean up its Gradle file. [23:13](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1393s) This is kind of like the world we're in right now. [23:15](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1395s) Ideally, we it wouldn't have to do this, but this is the current reality. [23:19](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1399s) So, we should have defined all of that up front first about what we expect, and that would have allowed the agent to loop until it had met all of those criteria. [23:28](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1408s) That's what a verification loop is. [23:32](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1412s) And we actually have really strong data on the effectiveness of this. [23:35](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1415s) So, this is Android Bench 2.0, it's the new version of our AI benchmark I mentioned back at the start of the talk. [23:42](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1422s) Pass rates are currently at about 40%. [23:44](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1424s) These tasks are really hard, and they're really big and realistic, but one of them is Nav2 to Nav3, they're the kind of things that you're doing in your projects right now. [23:53](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1433s) And if we take Fable 5.1, its pass rate is 22%, that's what it gets just with the natural prompt set up, like "migrate Nav2 to Nav3." [24:03](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1443s) But if we give it a failing test suite in advance, so this is not giving it the answers, we're not telling it how to migrate to Nav3, we're just giving it a test suite that is failing already and will pass when all of those things are complete, its pass rates on the whole benchmark jump to 90%. [24:19](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1459s) And what this shows us is these models are really capable now of implementing pretty much any Android task we can think of, they just don't always know what "done" means. [24:29](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1469s) And so, you don't have to like write a full failing test suite in advance. [24:34](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1474s) Verification loops can be as simple as prompt additions. [24:37](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1477s) So, say you're working in the UI, you can ask it to render the Compose preview and ensure it matches the Figma spec, and show me the results. [24:45](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1485s) It seems to really work when you ask it to show you the results and prove it was done, rather than just accepting the text response, "I am done." [24:53](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1493s) If you're working on business logic, you can ask it to run your unit tests, or, like we saw at the start, if you're working on a crash fix, you can tell it to validate the crash is fixed on the real device that it happened via remote test lab. [25:05](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1505s) There's actually a skill built into Claude Code to help you build verification loops now. [25:10](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1510s) So, definitely check that out. [25:13](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1513s) And if you're working in Android Studio with bring your own agent, Android Studio is exposing a whole bunch of tools that you could build loops around depending on what you're working on. [25:22](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1522s) So, like the linter, you could tell it to not stop until all the lint errors are fixed, or the one I really like is journeys, you can ask it to run your journeys and ensure they are still functional. [25:32](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1532s) Or if you've written a journey, cuz journeys are pretty easy to write, they're just prompts about the feature you're working on, you can tell it to run that journey and ensure that it passes before it declares itself done. [25:44](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1544s) But journeys do take a while to run. [25:46](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1546s) So, what you can also do is tier the levels of your verification loop. [25:50](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1550s) So, we can ask it to first make sure the unit tests and Compose Previews are all rendering correctly. [25:56](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1556s) And then when that passes, please run our connected tests and Android Journeys, and only when that's passing, run the full like remote device test lab, full test suite that's going to take hours kind of thing. [26:06](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1566s) And if you give that extra information, we find that it loops per tier rather than trying to do each step at each run. [26:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1574s) Cuz like models are really confident. [26:15](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1575s) So, they always think they are complete, and they always think they're going to pass all the tests first go. [26:21](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1581s) So, let's have a look at that in Sideline. ### Android XR and autonomous bug fixing [26:25](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1585s) So, let's apply this idea to our Sideline application, and let's do this by adding new a new feature, specifically for the XR glasses, which are coming soon. [26:38](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1598s) And since we don't have hardware yet, of course, we'll use the XR emulator and instruct our coding agent to extend Sideline for the new form factor. [26:50](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1610s) When we do that, we're going to specifically ask our agents to loop through JVM tests, Compose Preview tests, and emulator tests in that order. [27:02](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1622s) And each loop is told to build on the loop prior to it. [27:09](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1629s) And when we do this, the first thing we notice is that between loop one and two, it's the JVM tests that are used to define the six Compose Preview states. [27:19](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1639s) Of course, no emulator was called, so that runs very fast. [27:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1644s) And then similarly, between loop two and loop three, issues like long strings are surfaced and fixed even before the emulator has to run. [27:34](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1654s) So, finally, once the emulator loop runs, it finds additional bugs that the JVM test and the Compose Previews couldn't fix. [27:41](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1661s) But essentially, with each loop, we're further hardening our application, and we want to run the fastest but least powerful loops first, followed by the slowest but most powerful. [27:55](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1675s) And here's the outcome. [27:57](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1677s) It shows that the glasses now show the swing type and the speed of the ball coming off of the racket, so that you can be playing tennis and you can be getting feedback on your gameplay in real time without taking your eye off of the ball. [28:17](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1697s) Oh, and I mentioned earlier that we would get back to those two Crashlytics problems that had arisen before. [28:25](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1705s) So, since we've already been able to validate that that approach worked, we can use the /schedule command in Claude Code to loop through it, and we can set any cadence we want, so that now we have a self-healing application that fixes all the bugs and validates that they're fixed, sends them as pull requests to our engineers to take a look at and to merge. ### Key takeaways and resources [28:55](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1735s) Cool. [28:55](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1735s) So, that's pretty much it for today. [28:57](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1737s) If you take one key takeaway, it's to start thinking about this world where we're building an environment for our agents to succeed, and treating that kind of like how we treat software development and bugs today. [29:08](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1748s) That's how we unlock parallel agents. [29:11](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1751s) You're going to find a lot of advice on this topic online, and a lot of it's going to conflict with each other. [29:17](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1757s) And it's just the reality of this fast-moving environment. [29:20](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1760s) So, the best thing you can do is write your own evals for your project. [29:24](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1764s) And then when some new advice comes out or some new model, you can actually measure it and see does it help you. [29:30](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1770s) And of course, there's a skill for everything. [29:32](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1772s) There's a skill to help you build evals in Claude Code. ```text /claude-api build-eval ``` [29:35](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1775s) Definitely recommend checking that out. [29:38](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1778s) I talked about Android Bench today. [29:40](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1780s) Android Bench is our eval. [29:41](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1781s) It's how we measure to make sure things are getting better. [29:44](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1784s) If you want to hear the lessons we learned building that, so you don't make the same mistakes, check out the Android Bench talk. [29:49](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1789s) There's also two panels. [29:51](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1791s) There's one in the US, there's one in London. [29:52](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1792s) They're both going up online that talk about this kind of concept in practice in the community. [29:58](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1798s) So, we've got Android Dev community members coming onto the panel to talk about how they're putting this into practice at their jobs. [30:05](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1805s) We also, we really want your feedback. [30:07](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1807s) We really want to hear if we're building the things that are actually helping you. [30:10](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1810s) So, please fill out the survey. [30:11](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1811s) I'll leave that up on screen. [30:13](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1813s) And, yeah. [30:14](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1814s) Thank you for listening. [30:15](https://www.youtube.com/watch?v=Mb9H43ia0kE&t=1815s) Thank you.