# Agentic coding: A roundtable with industry leaders (London) - Speakers: Arnaud Giuliani (CPTO & Co-founder @ Kotzilla), Christos Gkekas (Software Engineer), Iury Souza (Senior Engineer, SumUp), Maia Grotepass (Staff Engineer Android, Qonto), Slobodan Manojlović (VP, Lead Software Engineer), Tristan Hamilton (Technical Lead, HubSpot), Jose Alcérreca (Developer Relations Engineer) - Track: Tools & Performance - Video: https://www.youtube.com/watch?v=iQ0R0EfOBiw - Page: https://developer.android.com/events/dev-summit/talks/agentic-coding-a-roundtable-with-industry-leaders-london ## About the talk As new tools rapidly reshape modern software engineering, this panel will explore how companies and developers are leveraging agentic coding, automated agents, and specialized developer tooling in their daily workflows. We aim to highlight practical applications, discuss measurable productivity improvements, and candidly address the challenges engineering teams face when integrating these tools at scale. ## Summary - Code generation shifts development bottlenecks across the pipeline. Once teams accelerate code reviews, new constraints emerge around live experiments and safe production releases. - Specialized agent teams can replace single general assistants across complex domains. Teams build distinct agents for areas such as performance or logging, coordinating them through routers or shared pools. - Autonomous pull requests require guardrails and clear human accountability. Some workflows auto-submit routine fixes for tech debt, but each PR points back to a designated owner and relies on strict automated checks. - Harness engineering needs centralized shared tooling alongside room for experimentation. Teams maintain core infrastructure for shared concerns such as logs and crashes while pruning inactive skills through automated scripts. - Local developer hardware is strained by concurrent agent-driven builds. Engineers encounter exhausted memory and slow machines when agents trigger multiple Gradle builds simultaneously. ## Topics agentic-coding, ai-tooling, code-review, harness-engineering, mobile-development, developer-productivity ## Transcript ### Introduction and panelist introductions [00:05](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=5s) Hi, everyone. [00:06](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=6s) Thanks for being here. [00:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=8s) My name is Jose, and I'll be moderating, or, well, reading questions, in today's panel on agentic coding. [00:17](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=17s) We brought together six amazing panelists. [00:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=20s) So, let's meet them in a quick round of introductions. [00:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=23s) So, we'll go alphabetically. [00:25](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=25s) So, Arnaud, why don't you go first? [00:26](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=26s) Arnaud Giuliani, co-founder and CPTO for Kotzilla. [00:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=31s) We are a French startup. [00:32](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=32s) We are making tools and AI infrastructure for mobile app development. [00:36](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=36s) And we are making in-between open source to observability tools. [00:41](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=41s) And I'm managing product and engineering, still coding a lot of course, for Android and Kotlin. [00:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=47s) Great. [00:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=47s) Christos? [00:48](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=48s) Hello. [00:49](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=49s) My name is Christos Gkekas. [00:50](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=50s) I'm working on the Android Google Search app. [00:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=53s) The app itself has around 1 billion monthly active users. [00:57](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=57s) And I'm working on the feature platform side of things. [01:00](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=60s) So, providing the platform, the infrastructure, the tooling for the other teams to come in and develop their beautiful UI for our users. [01:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=68s) All right. [01:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=68s) Awesome. [01:09](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=69s) Next, we have Iury. [01:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=71s) Hi, everyone. [01:12](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=72s) Yeah, I'm Iury. [01:13](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=73s) Currently working as a on the mobile platform at SumUp. [01:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=78s) And yeah, we have a ton of experimentation with agentic tooling, and, yeah, keeping the agents on track on the bigger platform scale. [01:28](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=88s) All right. [01:29](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=89s) This is going great, by the way. [01:30](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=90s) Maia. [01:33](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=93s) So, I'm Maia. [01:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=94s) I work at Qonto. [01:35](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=95s) It's a big European fintech, 70+ Android engineers, 70+ iOS engineers. [01:44](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=104s) And I'm looking at how to make these AI-coding Android engineers use their tools effectively and produce code and apps quickly. [01:57](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=117s) Awesome. [01:58](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=118s) Slobodan. [01:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=119s) Hi. [01:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=119s) I'm Slo. [02:00](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=120s) I work at JPMorgan in Glasgow. [02:03](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=123s) I'm an Android tech lead. [02:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=124s) I've been working for the last 5 years out there, but today, I'll be speaking my personal capacity. [02:09](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=129s) In my spare time, I'm an amateur chess player. [02:12](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=132s) I'm an amateur chess player. [02:14](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=134s) Love surfing, snowboarding, and similar kind of things. [02:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=138s) Yeah, I can speak to that. [02:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=140s) Tristan. [02:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=140s) Hey, everyone. [02:22](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=142s) My name is Tristan Hamilton. [02:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=143s) I am the mobile DevEx and infra tech lead at HubSpot. [02:27](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=147s) I've been there for about 5 years, started my career there as an Android developer working on more feature development, moved into foundational and infrastructure, and then got the opportunity to lead this team for like the last 3 years, and our main focus is on basically making the mobile agentic development experience as good as possible from skills, context, MCP servers both locally and remotely. ### Shipping 10x code and development bottlenecks [02:52](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=172s) All right. [02:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=173s) So, the first topic we want to talk about is shipping 10x or 100x. [02:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=179s) We generate 10 times as much code, but are we actually shipping 10 times as many features? [03:05](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=185s) So, the question is, how do you deal with low-effort, AI-generated PRs? [03:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=190s) Do you have internal AI policy to control this? [03:14](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=194s) And also, more changes mean more CI servers, more emula- emulators running, and even more manual QA. [03:22](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=202s) So, have you had to invest in your dev infra because of this? [03:26](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=206s) So, actually, let's start this one with Slobodan, and then we'll open the… [03:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=211s) Yeah, this an interesting question to kick off. [03:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=214s) If you think that you can build 10 times more features, and let's imagine it's free of charge, the whole SDLC, the software de- development life cycle, is not just shipping these features. [03:44](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=224s) You have to think, if even more if you work in a large institution, you have to think about the whole process. [03:49](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=229s) It's starts from product, from the idea, then shipping the code itself. [03:55](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=235s) That's, nowadays, maybe not the bottleneck, but the whole scaffolding around it. [04:00](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=240s) It's someone that makes it more trickier because you have to ship a feature, and then 3 years down the line, you still have to maintain it. [04:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=248s) So, you have to ask yourself the questions of scalability, maintainability. [04:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=251s) Whose responsibility is actually the this code base? [04:16](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=256s) Also, if you venture yourself outside of your SME bit, what is going to happen if you're not an expert in some Python scripts that you've written, and that can have an impact on the production code? [04:28](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=268s) So, I think as as an opener, this could be the questions we could ask ourselves. [04:32](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=272s) It's not just shipping code faster. [04:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=274s) It's the whole larger picture that's that we have to take into account. [04:38](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=278s) So, I would be interested in knowing like is there like internal AI policy? [04:42](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=282s) Maybe others can tip in to control this. [04:46](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=286s) Something that you have, you know, agreed upon yourselves to control this kind of stuff? [04:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=293s) On our side, we see the landscape changing quite dramatically, day after day. [04:57](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=297s) We see also the bottleneck of, you know, feature development being shifted from generating the code, crafting the code, to then kind of reviewing the changes. [05:06](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=306s) And then, once we put the tooling in place to help with reviews, the bottleneck would then shift on to, how do we do the live experiments? [05:12](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=312s) How do we kind of release the code safely into production? [05:15](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=315s) So it's a matter of thinking outside the box and ahead of the curve, trying to figure out how you're going to deal with the next bottleneck in the pipeline and how you can use agents there as well to help if possible. [05:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=324s) Arnaud? [05:27](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=327s) Yeah, on top of that, I would say it's really useless to try to monitor tokens, because it's like monitoring electricity for your laptop, but you know, our side, we tried really to reinforce all the software factory, quality, and impact analysis, understanding how much, if you are not strong on that before AI, then after that it will be nightmare. [05:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=353s) And I would say productivity is in between input and output, then in between the effort you put first, and then what you get out of that, and do you make just tons of PR to have the same kind of product or do you break your product more more fastly? [06:09](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=369s) Then this is really the priority for me, like okay, you can make more code, but at the end, how the product is? [06:17](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=377s) And do you maintain it over time, and what is the quality behind that? [06:22](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=382s) Cool. [06:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=383s) I think this one is very important, maybe we can spend a little bit more time on it. [06:27](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=387s) Maia, do you want to add anything to it? [06:30](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=390s) Yes. [06:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=391s) I agree with this bottleneck idea, that just as you've sorted one bottleneck out, then you get the next one. [06:37](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=397s) So maybe your CI builds were need to be sped up, and they are, and then you find out that now we need to look at local builds again, cause people are doing multiple working local builds, so then it that this the focus changes there. [06:51](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=411s) The other thing that's interesting for me as well is at one point on this AI journey, we would think that it's okay, we can let this slop in, say when we're writing scripts, "Don't worry about the scripts, just ship the scripts. [07:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=424s) It's okay, as long as the production code is not sloppy." [07:09](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=429s) But I've also seen places where people decide, "Okay, our scripts are also need better reviews and better structure, and put things in place, and automated things in place to make make the skills be more structured, make the scripts be more structured." [07:25](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=445s) And tests, maybe. [07:26](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=446s) Yeah. ### Agent personas and multi-layer architecture [07:28](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=448s) Okay, so let's move on to the second topic, which is about personas, and what what different agents have you deployed? [07:36](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=456s) Code reviewer, security scanner, test reviewers? [07:40](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=460s) So Christos, you want to start with this one? [07:42](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=462s) When we started on our AI journey in the app, what we tried to do is initially try to create all the skills necessary for the agent to be able to contribute in the development pipeline. [07:52](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=472s) And this meant codifying all the expertise and all the knowledge that every team had into like skill files, Markdown files, procedures, and things like that. [08:00](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=480s) But we found out quickly that as we kind of scaled up the amount of skills that we had that using like a single agent across so many different domains was not as effective. [08:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=491s) So we're now trying like a different approach, where instead of having one kind of general assistant agent, to split it up into having a team of agents, each one specializing in a specific field. [08:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=504s) Like, for example, you could imagine having an agent specializing on performance and then giving advice on performance or crafting CLs for performance, another agent for logging, a different agent for compliance. [08:36](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=516s) And we found out that as we develop those agents, you can plug these agents into other automated kind of multi-agentic workflows, where, for example, if you have like an agentic team developing a feature, they could delegate to the performance agent to give advice on whether there are any performance bottlenecks on that feature, and then use that advice to create that feature in a more performant way, similar with other agents as well. [08:58](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=538s) Tristan? [09:01](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=541s) I think I can add to that. [09:02](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=542s) Yeah, very similar vein that you know, we had exposed Crashlytics MCP server and we had initially built an agent for really essentially just observability metrics, but then we saw, "Oh, well, now we can use that to sort of plug into our Crashlytics errors that arise and we can reuse that agent for basically solving PR fixes as well, and investigating." [09:22](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=562s) So, yeah, definitely that sort of multilayer approach where you start with an agent and you can kind of scale it, is definitely something we've seen as well. [09:33](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=573s) Yeah. [09:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=574s) I think it's in the same direction, having like tested and creating agents specialized in one topic, and then kind of promoting these agents to like the this pool that can be like created on demand, so you have a certain problem to solve, a certain PR to review, and then you can have like dynamically creating these this these workflows pulling these agents in. [09:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=599s) It's also like super effective because then you get both, right? [10:02](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=602s) You get these specialized agents tested for a certain topic and then you also get it into this pool that you have like like your like this fleet of agents that you can select and create these dynamic workflows. [10:14](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=614s) They seem to work quite well too. [10:17](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=617s) I think just to add I Something we've seen inside HubSpot is like the debate between like super-agent that has all the skills and powers versus like the router agent where you delegate out. [10:27](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=627s) And for our mobile case, we kind of chose to go the route of, yeah, we let developers sort of build out the sort of subcase agents, and then we plug them into our like router super mobile agent. [10:40](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=640s) Whereas, actually for the general HubSpot we've they've chosen that the sort of general developer sidekick agent you can speak to is a super-agent that just knows how to do everything. [10:50](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=650s) But it's interesting to see things like these different approaches. [10:51](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=651s) Yeah. [10:52](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=652s) There's a dynamic there, yeah. [10:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=654s) Just to add to that, it's also interesting to have some hooks and guardrails. [10:57](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=657s) If you have some specialized specialized agents, you can actually say, okay, before merging or before pushing any code to the codebase or to any specific branch, let's review it by our standards, if you want security, accessibility, or any other type of specialized agent that you want to pull in. [11:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=678s) All right. [11:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=679s) Let's move on to the Yeah, sorry for not spending too much time on each question, but we have very limited time. [11:26](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=686s) So, let's move on to the next one. [11:29](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=689s) So, this might be a little bit shorter, but are you letting LLMs submit any code without any human intervention intervention at all? ### Merging code without developer review [11:36](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=696s) Yeah, so I'm going to take that. [11:41](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=701s) What we have is a review It's We've got a lot of remote agents that can run with specialized skills, and the one is these guardrails with reviews. [11:51](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=711s) So, typically what we will do is if anybody puts a PR up on our Android codebase, it automatically gets reviewed by the review agent. [12:00](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=720s) Okay? [12:00](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=720s) And so then at that point, whoever's taken that PR can see that the architecture is fine. [12:05](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=725s) So, what we also found was that we found that some of the agentic code was adding tech debt to our codebase. [12:14](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=734s) And so, we have playbooks, which can clean up tech debt. [12:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=739s) So, every day, it goes through, and it makes PRs that cleans up tech debt, okay? [12:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=744s) So, we have On the one side, we've got a specialized agent cleaning up tech debt, but we also have the review agent who can then And if the things are small, we have a situation where, potentially, the review agent approves the one that came with the tech debt. [12:38](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=758s) So now, you'd think that, "Okay, we're shipping code straight into production." [12:41](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=761s) So, I had a check, and I spoke to everybody at the security department around that. [12:45](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=765s) And so, what we need to do is we need to make it very clear inside the PR, so you know where does this happen, who caused this, where is it. [12:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=774s) Basically, if any of this goes rogue, that it's super easy to stop it. [12:58](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=778s) That's the first thing. [12:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=779s) The second thing is we have these PRs, they don't come in a flood, they are limited to a few, and they only happen on these specific playbooks. [13:06](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=786s) And so, those playbook reviews come through in a place where you can review them and see them. [13:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=790s) And also, we ship every single day. [13:13](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=793s) So, we can ship and roll back and stop and block, and we can check everything through that. [13:17](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=797s) So, there's there's extra processes. [13:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=799s) So yes, we do have agents making PRs, and we have other agents Notably, they are not the same one, so you have one that's fixing the tech debt and one that's doing the reviews, and then a lot of things around that. [13:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=811s) But do they submit? [13:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=814s) Like they approve, and they review, and they approve. [13:36](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=816s) But do they submit automatically? [13:38](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=818s) That's Yeah. [13:38](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=818s) Okay. [13:38](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=818s) So, if you set the PR to be approved when all the checks have been met, like all the tests have passed, all the UI tests have passed, all the tests, all of your guardrails are there… [13:50](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=830s) And you set it that it must submit when it goes, then it goes. [13:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=833s) Got it. [13:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=834s) Okay. [13:55](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=835s) But only for ones that have a playbook that we know, that are on. [13:58](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=838s) Okay. [13:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=839s) So, does that mean the playbook can essentially be that sort of source of responsibility? [14:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=844s) You know if someone Like, who owns the playbook, I guess, is the question I have. [14:07](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=847s) You can't only have the playbook. [14:09](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=849s) That was the thing. [14:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=850s) Every single one of those PRs has to have a pointer back to the process so you can stop it and check it, which typically also has a person associated. [14:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=860s) Right. [14:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=860s) Like a DRI or something. [14:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=861s) Right. [14:22](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=862s) Yeah. [14:22](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=862s) That's quite interesting because this point of responsibility is really something that we keep in the middle of our way of working at Kotzilla. [14:33](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=873s) And even if if we try if we and we have lots of PRs, the problem is that who who is responsible for that? [14:41](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=881s) Even if it's scripts, even if it's agent, even in the adoption on of a small company like us, we try to test lots of things, lots of models and stuff. [14:52](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=892s) And then that's a matter of trust at the end is do you are you trusting the agent that is making your review or not? [14:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=899s) And are you sure that, on the time, you are not into effect that you have either two converging agent like they are always okay? [15:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=908s) And then, that's buddy. [15:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=910s) Do you trust me to review your PR on a Friday afternoon? [15:15](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=915s) It depends. [15:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=918s) It really depends. [15:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=919s) So, yes, I mean for us we, for the automatic reviews, we let the person who puts the PR up decide if this is just a README file or a script file whatever to go. [15:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=931s) We still very clearly communicate that the engineer needs to decide do they need extra reviews and in fact we've got a bunch of scripting that will call people as reviewers in case it's needed. [15:44](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=944s) So yes it's a question of trust. [15:46](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=946s) Exactly. [15:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=947s) I think we've been also observing in the whole industry that now the human in the loop bit is super important. [15:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=953s) But with so many PRs coming it's tricky to have the attention span to pass through all of the code that's being written and to assess it the same way we would have done like three years ago. [16:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=964s) So I think that's a real bottleneck. [16:06](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=966s) This is one way to tackle it, but it has its obvious downsides. [16:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=970s) So the thing is you need to decide where do you want people to think. [16:14](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=974s) Now I don't want people to review for nits. [16:17](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=977s) I can get an agent to review for nits. [16:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=979s) I don't want that. [16:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=980s) So our the PR will say, this PR has been reviewed for nits. [16:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=984s) Okay? [16:25](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=985s) I want people to think, is the thing that we're putting in what we want to put in? [16:29](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=989s) Is it the Is the intention that's I want That's what I want people to think about. [16:32](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=992s) That makes sense. [16:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=994s) But, I think what our reviewers are now finding is that, I agree like, you know, the first few passes of the agent will catch a lot of the nits, but the problem is now the scope and maybe the domain could be so disparate from what they're currently working on that they go, "Well, I don't know how I'm going to review this now when I I'm just I'm not clued in at the moment." [16:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1014s) Yeah, I mean it's not That's That's what we're doing to try and fix the problem, but I'm seeing also more and more people working in domains that they wouldn't have worked. [17:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1024s) Like I was trying to get some other thing and some Python script to work, it wasn't working. [17:09](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1029s) And all I'd do is put a PR up on that repo saying, "Hey, fix it like this." [17:13](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1033s) I don't know if I'm making it worse. [17:16](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1036s) So, yeah. ### Changes in developer team structure [17:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1038s) Which is a great segue to the next topic, which is about team structure. [17:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1044s) Has it changed around you because of the way the code is being shipped now? [17:30](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1050s) No? [17:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1051s) Yep. [17:33](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1053s) As a small company, I couldn't say that. [17:35](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1055s) It's really It's helping a lot and since I would say we see that in every months, every weeks we see that all those new models are really helping us. [17:46](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1066s) And it's really changing teams in a in the way that finally we don't have just backend, just front-end, just product team. [17:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1074s) Everyone can really contribute to the company OKRs and to the delivery of the product. [18:00](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1080s) And that's also a challenge because we can contribute, and that's it's a good comment, we can contribute to many things, many domain, but that's it's not because I can contribute to backend, I am infrastructure expert that I can really anticipate decision. [18:15](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1095s) And that's a bit the limit where in my position, I need to help people grow in their position to have more their hands on other domains, like products, like design, like infrastructure and so on, but also helping having the right decision because at the end, we need to still decide and take the right decision. [18:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1114s) If we just say, "Okay," to our model, it's not the right decision, I guess. [18:42](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1122s) Anyone else has seen team structure changes? [18:48](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1128s) Yeah, so our teams became much smaller, Not because of AI, because of the way that they want the teams to be very, very focused and have domain knowledge. [18:57](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1137s) It might be because of AI trying to make concentrated domain knowledge in a team. [19:03](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1143s) I see very many people Android engineers doing some some backend stuff, backend engineers doing some web stuff. [19:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1151s) I don't see that many other disciplines doing Android, mainly because it's not so easy for them to get the whole stack installed. [19:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1160s) And so they that's they're they're more hesitant about that. [19:25](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1165s) What else am I seeing? [19:26](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1166s) I'm seeing product people trying things out, but again, not on the Android app. [19:29](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1169s) They'll do something in web and try that out. [19:33](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1173s) I'm seeing people put more guardrails in because of people not from their stack putting code into their codebases, which is not a bad thing. [19:41](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1181s) It's a good thing, yeah. [19:42](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1182s) It's a very good thing. [19:43](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1183s) So the more guardrails I have for the Android repo the more confident I am that someone else would could come in and do some small fixes. ### Sharing the harness and engineering conventions [19:52](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1192s) Yeah, absolutely. [19:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1194s) All right. [19:56](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1196s) Maybe another good segue, by the way, to the next one. [19:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1199s) So, this is about sharing the harness or the conventions used by your team. [20:05](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1205s) How do you organize that by project you know, how do you personally like to do this? [20:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1211s) I'm talking about you know skills agents.md, all that kind of stuff. [20:15](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1215s) So, maybe Iury? [20:16](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1216s) Yeah. [20:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1218s) So first of all, I think I like to break down this in like in two two scenarios. [20:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1223s) Like two to because the concept of harness engineering is it's kind of very convoluted. [20:28](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1228s) So I think in one side we want to enable people to try different harnesses. [20:32](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1232s) So for example, I think of Claude Code or OpenCode or Pi or anything like that or Gemini CLI, anything. [20:40](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1240s) It's just that So this is one layer and ideally, I think you should be some harness there are like multi-provider, you know. [20:48](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1248s) Like so you can try as many models as you want and that's not going to be a constraint. [20:52](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1252s) So that's one thing. [20:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1253s) We kind of approve what harnesses people can try just because as you know like a bunch of these harnesses they don't really play well with each other. [21:01](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1261s) So like they're not all all following the standards the same way. [21:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1264s) Yeah, so it's a very fast-pacing environment. [21:06](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1266s) So we want to do that. [21:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1268s) And on the other side, there's like what you can do to customize the harness to yeah, to towards your codebase. [21:14](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1274s) So then that's where we get like skills and MCPs and everything. [21:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1279s) So on that side, I think there's one side which is like centralizing. [21:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1283s) So we want to have like cross cross-cutting concerns, for example, you know, like accessing logs or the analytics or, I don't know, the crashes, everything, data like that. [21:33](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1293s) I think there should be like a centralized repo for sharing those those skills and kind of enforcing them on every every developer's machine. [21:42](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1302s) And then we also want, on the other side, to enable people to experiment because I think it's you know, like now that we moved to this, you know, like this everyone's, yeah, the whole output goes through the agent, right? [21:56](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1316s) So people are very particular about how they go about doing their tasks, you know, like even back in the day, like a few years ago. [22:05](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1325s) So we want to also enable that. [22:07](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1327s) So every team should probably want to have like a repo where they share the skills and they iterate and they share and eventually, one of these skills can be promoted to the main repo. [22:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1338s) So this was some of the ways I think it's good, right? [22:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1341s) You can do both. [22:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1343s) And because it's a very fast-pacing fast-pacing environment and you want to have the ability to innovate, but also not go crazy, right? [22:30](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1350s) Because yeah, then that's kind of what's the goal of harness engineering, I think, it's kind of to have some coherence there, but yeah, you need some open doors, I guess. [22:40](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1360s) If I may to add like I think in the last 12 months we went crazy, went wild with experimenting on every level. [22:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1367s) So, I mean the first skills came It was roughly like a year ago. [22:51](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1371s) So, we can look at it from a different perspective where you have your individual level, where you have a team level, then you have a department level. [23:01](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1381s) If you're working a large company, then at a company level, and then at an industry level. [23:05](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1385s) So, in the last 12 months, I think that the whole market has matured a little bit, and that you have this consolidation even within the industry. [23:13](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1393s) But you could have noticed it also within your firms, teams, projects or even on your personal level. [23:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1399s) And what's super important, I think in this whole process of maturing is having some specific metrics, telemetry, understanding what you've been using, what's the input, what's the output, not only the tokens, but trying to measure what's the final output that you've had. [23:34](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1414s) Was it better in this setup or in the other setup? [23:37](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1417s) Which skills were better? [23:38](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1418s) Which plugins were better for your team, for yourself, for your company? [23:42](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1422s) These kind of questions, I think have matured a little bit and the solutions that come with it. [23:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1427s) So, Slobo, what does your agent's skill folder look like? [23:51](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1431s) Is it a mess? [23:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1434s) Some I mean… [23:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1434s) Or is it tidy? [23:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1434s) Your personal one. [23:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1434s) My my I think it's so I don't even know what's in there because I haven't been it's like a magic box. [24:03](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1443s) It's a Pandora box, yes. [24:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1444s) And so, what I'm seeing is a bunch of engineers have put so many skills in. [24:09](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1449s) That's me. [24:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1450s) Yeah. [24:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1450s) And I've literally yesterday, I wrote a script that checks how many times a skill's been used. [24:17](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1457s) And if it's zero in the past 30 days, it gets a PR up to delete it. [24:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1461s) So… [24:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1461s) Yeah. [24:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1461s) That's just for you or for the team? [24:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1463s) No, that's for the team. [24:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1464s) Good. [24:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1464s) Okay. [24:25](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1465s) I think that's this consolidation. [24:27](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1467s) We had approaches that were coming bottom-up, like our, and we would create a myriad of skills, and then there is also this upside-down approach that probably you're mentioning that you're working on, where you're going to say, "Okay, let's clean up this whole mess, and let's make something out of it." [24:44](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1484s) I was going to say then, yeah, you'd hope that every harness starts to adopt these kind of things where it will just tell you, "Hey, you haven't used these for this amount of time." [24:53](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1493s) But yeah, thinking on the I find it it's confusing when we talk about harness engineering and we have like harnesses that let you provide different models, so like different model providers. [25:04](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1504s) But then you also have like harness orchestrators like I guess we use Conductor quite heavily inside of HubSpot, which is really kind of piggybacking on the other harnesses and calling those rather than just their models. [25:16](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1516s) But yeah, I find that that whole name spacing is very confusing. [25:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1519s) And the other thing I just wanted to mention was on the maturing state of things. [25:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1524s) Yeah, the fact that, like, you know, we had a lot of rules in our repos and stuff, but now that's sort of being just superseded by skills in general, and I know I think Claude has even sort of gotten rid of the or they don't really use that nomenclature anymore. [25:36](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1536s) But one thing I just thought I'd mention that's interesting that has just managed to stick around inside HubSpot is we have a concept of one-pagers. [25:42](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1542s) They're not really skills, like they're not necessarily going to be picked up by keywords, but we run it as part of pretty much any agent where if you end up mentioning some keywords that's related to some internal infrastructure, it's going to pick that up and grab a little one-pager to describe what that framework is, because we have so many, you know, sort of wrapped layers of open-source tooling. [26:02](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1562s) And I just thought I'd mention that's we found that is a really powerful way to sort of connect different parts of the company with your agent, so it knows how it works. [26:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1570s) Yeah, that that that's a pre- a pretty good one because it it's kind of like building a knowledge base, but also this domain language that the agents can easily pick up, because they're pretty good at that. [26:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1580s) So if you can yeah, there's like every company has, you know, like their own acronyms, you know, like a bunch of these words and they don't mean a lot to anyone to know in a- anyone else outside. [26:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1591s) But when you once you get to that, I think it helps a lot the agents to understand what the what this product thing means in this context, right? [26:40](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1600s) Do you do any formal evaluations of your harness, your skills, things like that inside your company? [26:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1607s) Yes, that Well, on the Android team, yes. [26:48](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1608s) So, that's part of this. [26:50](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1610s) I don't want to write all those eval, so that's why I'm beating the skills, right? [26:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1614s) So, we do have evals for skills. [26:58](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1618s) I'm in the middle of building it so that it will run when someone touches the skill, then it will run the eval for the skill. [27:03](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1623s) But I've definitely deleted a bunch of skills which were built with older models. [27:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1628s) And now, all of a sudden, the models know how to do the things, and you actually get better results and fewer tokens without the skill. [27:15](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1635s) Yeah. [27:16](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1636s) Nice. ### What's missing and closing remarks [27:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1639s) Okay, let's go to the last question. [27:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1641s) Yeah? [27:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1641s) All right. [27:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1643s) So, yeah. [27:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1644s) What's missing? [27:25](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1645s) We like, from the Android team, we've released Android CLI. [27:28](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1648s) We're adding a lot of stuff to it, and it every week we have more and more ideas, things to add to Android CLI. [27:37](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1657s) So what tools would you ask from us to help you with your agentic workflows? [27:45](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1665s) Tristan, this was for you. [27:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1667s) Yeah. [27:47](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1667s) I have loads of ideas on Android CLI, but I think I'll talk about one other thing that I think could be useful for everyone, which is, you know, we have the Build Analyzer tool inside of Android Studio to improve the throughput, so basically the speed of your builds. [28:03](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1683s) But I think we're now encroaching on a place where we need something that can analyze the footprint of that as well. [28:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1690s) I don't know if you've I guess the irony is that as we make these tools better and better, the devs are actually becoming too productive now. [28:16](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1696s) So, they're, especially from a local perspective, so I've in the last like couple of months, we've had multiple devs coming saying, "Hey, my machine is just slowing to a halt." [28:24](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1704s) Or, it's crashing, or the thing is thrashing. [28:27](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1707s) And when we go look at our Gradle Develocity builds, you can see some devs are running, sometimes, up to three to five concurrent Gradle builds on the one machine. [28:36](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1716s) And, you know, the problem is that even if it's just unit tests that they're running from these agents that they might not even know that the agent has stepped kicked off something like that, each Gradle JVM is going to reserve a certain amount of space. [28:48](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1728s) So if you're not looking at what heap size you've actually defined for your build, you could actually be reserving a bunch of space that you might not that agent might not even go back and touch it again. [28:59](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1739s) So, we actually found switching from parallel GC to G1 and having a periodic garbage collection, that has actually sort of freed up space on people's machines as they run, so. [29:10](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1750s) Yeah. [29:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1751s) Is Seb here? [29:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1751s) Seb has Sebastiano has a project that for like call something Traffic Control, ATC. [29:18](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1758s) What's the A for now? [29:19](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1759s) It's not It was Android. [29:20](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1760s) Is that [29:21](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1761s) Agent Traffic Control, yeah, which helps with that yeah. [29:25](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1765s) Nice. [29:26](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1766s) Yeah, I mean, I think another place it could be improved is like I think a lot of companies right now are trying to build like this sort of black box testing, you know, that the agent can drive the app, and see the screen, and whatnot. [29:43](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1783s) And I talk to some of these people, and they're they mention they have different approaches, like taking screenshots, or using the accessibility tree. [29:50](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1790s) Like the there are different you know, approaches for solving this problem. [29:54](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1794s) And I feel like when we have this way to dump the, you know, the accessibility tree with the ADB, but I think maybe if we would have because that that doesn't really solve the problem because there's overlap, so it it's not clear to the agent what's actually being rendered. [30:11](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1811s) So, I don't know, if we could probably have some structural output that could give that information, I think the agent would be able to drive the app more accurately and we wouldn't need this whole Yeah, could be make things better. [30:26](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1826s) I've I've You go. [30:28](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1828s) All right, I can just tell you where the pain points are: people's memory run out when they're building, which is what you're talking about, people's disk space run out, because they got multiple worktrees and they've got Gradle artifacts everywhere and everything in the people's IDE startup is slow, because maybe they want to see the code, they only want to navigate the code, but they can't get to the code, because Android Studio takes a long time to start up. [30:50](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1850s) A lot of work been done in QA, trying to figure out how can we run the QA, how can we combine the QA, the tests, how can we test and click and view, and which is what you're talking about, and also, how can we move this off. [31:01](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1861s) I don't want my machine to be hot and taking up and about to take off with the fans. [31:06](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1866s) I want it to be somewhere off in the cloud. [31:08](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1868s) I want to do the same thing, but I want to do it elsewhere and get the results. [31:13](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1873s) Awesome. [31:15](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1875s) Okay. [31:16](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1876s) So, we probably could be talking about all these things for days, but that's all the time we have for it today. [31:23](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1883s) The good news is that we're go- we're going to be or the panelists are going to be around in the conference, so feel free to ask them questions as well. [31:31](https://www.youtube.com/watch?v=iQ0R0EfOBiw&t=1891s) And yeah now, it's time for lunch, so there will be a break and yeah, thank you very much.