Cormac Brick (Google) presents on fine-tuning tiny LLMs for on-device agents. The talk covers motivations for on-device AI (latency, privacy, offline use) and the two main developer options: Gemini Nano via AI Core for common system-level tasks, and LiteRT-LM for shipping custom models inside an app. It walks through a live agent skill harness built on Gemma, demonstrating a restaurant roulette app running fully on-device, plus Eloquent, a production transcription app chaining two sub-few-hundred-million-parameter models. A central case study is Function Gemma (270M parameters, ~2,000 tokens/sec prefill on a Pixel 7), which improves from 46% out-of-box accuracy on app intents to 90%+ on eight of ten functions after fine-tuning on a synthetically generated dataset. Topics include building and publishing agent skills, the LiteRT-LM runtime, the export and inference workflow from PyTorch to deployment, fine-tuning with synthetic data, and Q&A on skill robustness, file formats, and CPU/TPU performance. # Channel Presentation Observations ## Explicit Self-Description - **Speaker role and affiliation**: Cormac Brick introduces himself as *"a software engineering kind of tech lead working on the Google AI edge stack"* at Google. - **Team scope claim**: The team covers **MediaPipe**, **LiteRT LLM** (*"a LLM harness that you can integrate with your app"*), and **LiteRT** (*"formerly known as TensorFlow Lite"*), running *"on CPU, GPU, or NPU"*. - **Claimed scale**: The LiteRT runtime is *"built into Android OS"* and *"support[s] over 2.7 billion devices."* - **Openness claim**: The team describes its app (Google AI Edge Gallery) as *"open source on Android"* with *"code available on GitHub,"* and states LiteRT LLM is *"open source."* ## Demonstrated Presentation Patterns - **Developer-focused audience**: Content is addressed to **app developers** — e.g., *"as an app developer, when you go to build a mobile app… there will often be some form of intelligence built into the system that you can leverage."* - **Tutorial/practitioner framing**: Walkthroughs of deploy workflows (*"starting transformers… ai-edge-torch… LiteRT… you can deploy"*), fine-tuning labs (*"search function Gemma fine-tuning lab… available as a hugging face space"*), and how-to instructions (*"full instructions there if you want to hand write it out"*). - **Cross-platform scope**: Demonstrated coverage of Android, iOS, and beyond — *"works far beyond Android as well,"* with a **Swift API and JavaScript API "coming soon."** - **Concrete metrics as evidence style**: Uses measured results, e.g., function calling success *"from 46%… to over 90% for eight of the 10 functions"* and Pixel 7 performance of *"almost 2,000 tokens per second prefill and 140 decode."* - **Transparency about limitations**: Acknowledges current constraints — multi-skill calls in a single answer *"works sometimes… we're still figuring out how to make that more robust"*; the Eloquent app is *"only available on iOS and not available in Europe yet."* - **Community engagement**: Invites contribution — *"let us know in our discussion on GitHub that you've created a skill and then other people can check out your skill,"* and notes community skills built since the SDK shipped *"last Thursday."* - **Event/talk context**: References other speakers (*"Ali already covered this," "building on Ollie's and Florina's talk"*) and a related talk *"tomorrow"* — consistent with a conference session presentation rather than standalone content. ## Intended Audience - **Mobile/ML developers** building apps with on-device AI, evidenced by repeated second-person developer guidance (*"if you want to use one in your app, how would you go about that"*) and audience Q&A on technical topics (file formats, TPU/CPU performance). # Delivery Observation: Cormac Brick — On-Device Tiny LLMs ## Tone & Register - **Conversational, informal**: heavy use of "yeah," "okay," "um," "kind of," and "like" as connective tissue. > "Um okay. So AI edge SLMs and TLMs. Okay, so I think Ali already covered this." - **Self-aware about time constraints**: frequently references the clock. > "cuz I know we're short of time" > "The last 10 minutes we are going to spend on TLMs um or probably more ideally maybe five or six of the minutes, so there's time for questions." > "I'm at zero seconds and it's flashing at me." ## Vocabulary & Recurring Phrases - **"Kind of" as a hedge** — used constantly to soften claims or descriptions: > "It's kind of formerly known as TensorFlow Lite, which is a kind of cross-framework runtime." > "you can kind of dive into the code and see, 'Hey, how does it all kind of hang together?'" - **"Hey" as a framing device** for introducing ideas or example prompts: > "hey, how do you do skills on device" > "Hey, can you show us the location of the Google office?" > "Hey, we have these app functions." - **"Right?" as a checking tag**: > "the tool responds, and then it uses the show JS tool... to show you the location um in the app as well" — and throughout: "It's It's typically the workflow... it's the other option that's available. Okay." (e.g., > "which is what you would do... for example, um but um you instead need to kind of synthetically create a data set, right?") - **Reassurance/ease markers**: "really easy," "pretty easy," "really well." > "really just a few line like pretty easy to do with a few lines of code" ## Teaching Habits & Structure - **Upfront agenda-setting**, twice: > "One is um hey, how do you do skills on device... The other idea I want to cover is hey, for tiny models, what can you actually do with those types of models today." - **Concrete numbers to anchor claims**: 2.7 billion devices, 270 million parameters, ~2000 tokens per second prefill, 140 decode, 46% → 90% success rate, 80 skills. - **Defers detail to external resources** rather than covering everything live: > "there is a whole YouTube video on this called Function Gemma" > "another talk tomorrow, so you can hear more about Gemma performance" > "check out our model card... if you search Gemma light or TLM model card" - **Trade-off framing**: presents options with honest cost assessment. > "Though, it's clearly a lot more work, but it's the other option that's available." > "Yes, it's it's a bit more work than just prompting a larger model, um, but it does allow you to kind of ship something robust in your app at scale." - **Live-demo improvisation when things go wrong**, narrated transparently: > "Okay. Okay, I don't know how that Ah, yeah. Okay. Uh Okay, I kind of got lost a little bit." > "Oh, sorry. Going the wrong direction." ## Communication Patterns - **Honest uncertainty and limitations** stated directly rather than oversold: > "that works sometimes, right? And we're still like that's something we're still figuring out how to make that more robust" > "Apple also has something going on with their intelligence on iOS that I probably know a lot less about." - **Self-corrections and repairs mid-sentence**: > "our team have done like about 80 skills" — following "code coach like our team have done..." > "It's really useful for lots of It's really useful for lots of simple use cases" - **Enthusiasm markers for the topic**: "neat" appears repeatedly as a positive descriptor. > "that's an example of something neat that you can build" > "one of the things that's neat about being in an app" > "it's both a neat way for you to try things out" > "actually works really really well and it's fun" ## Uncertainty Notes - The "restaurant roulette" demo was partly improvised and verbally recovered from a stumble; the coherence of that segment is thin in the transcript. - Frequent filler ("um," "uh") may be partly an artifact of live-demo conditions rather than a stable baseline style — repetition is sufficient to confirm it as observable, but its cause is not. # Observation Record: From 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents — Cormac Brick, Google ## Speaker Identification - **Primary speaker:** Cormac Brick, self-identified as "a software engineering kind of tech lead working on the Google AI edge stack." - Audience questions appear at the end (unattributed speakers). ## Core Framing: Two Tiers of On-Device GenAI Brick presents two options for app developers: - **System GenAI:** intelligence pre-installed in the OS, e.g., **Gemini Nano via AI Core**; base models are **Gemma (stated as "Gemma 4") E2B and E4B**. Advantages: highly optimized, preloaded, "your app doesn't get any bigger." - **App GenAI:** models loaded with your app or webpage via the LiteRT-LM runtime; offers "higher degree of customization and reach," but "clearly a lot more work." - He notes Apple has "something going on with their intelligence on iOS that I probably know a lot less about." ## Google AI Edge Stack - Components: **MediaPipe**, **LiteRT-LM** (LLM harness), **LiteRT** runtime (formerly TensorFlow Lite). - Runs on CPU, GPU, or NPU; "you as a developer get to choose." - Claims scale: a version of the LiteRT runtime built into Android OS, supporting "over 2.7 billion devices," with many daily invocations. ## Agent Skills on Device - A **skill harness built on top of AI Core**; demo app: **Google AI Edge Gallery** (Android and iOS; open source on Android, built with LiteRT-LM). - **Mechanism:** skill descriptions are placed in the prompt; full function details loaded on demand via a built-in **load-skill tool call**. Skills can invoke JavaScript rendered in the app UI (e.g., restaurant roulette demo). - **Skills to write skills:** Brick says the team used **Gemini CLI or "code coach"** to author ~80 skills; "this is an example of a prompt which works really reliably." Custom skills can be loaded from a URL and shared via GitHub discussions. - Community skills appeared "since last Thursday." **Q&A qualification on skill reliability:** > "Within a conversation... if you have a conversation that uses skill, skill, skill, that works really robustly. The thing we're still working on that's harder is... the app to know to call multiple skills as part of a single answer. That works sometimes... we're still figuring out how to make that more robust." He notes the 4B model "by default we enable about eight skills and it's able to choose between the eight skills reasonably well." ## Tiny LLMs (TLMs) and Fine-Tuning - Defined as models "smaller than a billion parameters." - **Key claim:** once down to ~100–200M parameters, the model "needs to have a very narrow and focused task"; team has had success **fine-tuning with synthetic data**. - **Deployment path:** start in transformers → export via `litert-torch` → try on desktop (`litert_lm run`) or in Gallery → deploy with LiteRT-LM. Pre-built fixed-function models also available. ### FunctionGemma case study (headline claim) - **FunctionGemma**, published with DeepMind in December, based on Gemma 3, **270M parameters**, "robust function calling when fine-tuned." - Speed on a Pixel 7: "almost 2,000 tokens per second prefill and 140 decode." - **Fine-tuning result:** out-of-box success rate on app-intents tasks "I think 46% or something like that" — note the tentative phrasing. After synthetic dataset generation (using Flash) and fine-tuning, results were **"over 90% for eight of the 10 functions"**, with two functions "a bit lower." - Trade-off stated explicitly: "it's a bit more work than just prompting a larger model, but it does allow you to ship something robust in your app at scale." - A **FunctionGemma fine-tuning lab** is available as a Hugging Face space. ### Eloquent transcription app - Uses two chained tiny models (ASR engine + "text polishing engine"), both Gemma 3-based, each "a few hundred million parameters"; supports personal keyword dictionaries and removes filler words. - Qualifications: **only on iOS, not available in Europe yet**, "not available widely" — presented as a "proof of life example." ## Q&A Points - **LiteRT-LM file format:** "effectively a replacement for .task file" for LLMs; bundles tokenizer but is LLM-only; .task still useful for other assets (e.g., face mesh). - **Performance data:** deferred to a colleague's talk the next day; model card ("Gemma LiteRT-LM model card") is continuously updated. ## Preserved Tensions / Qualifications - The 46% baseline figure is stated tentatively ("I think... or something like that"). - Multi-skill single-answer behavior is unresolved ("works sometimes"). - Success was uneven: 2 of 10 functions underperformed post-fine-tuning. - iOS Gallery code not yet open-sourced, pending Swift API completion. - Speaker's knowledge of Apple's on-device intelligence explicitly limited. # Speaker Attribution Record — Video 01a15ca9-4632-7abd-8979-6e7dcd18a8c7 ## Video Overview - **Title:** From 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents — Cormac Brick, Google - **Type:** Conference talk with demo and audience Q&A ## Identified Speakers ### Primary Speaker — Cormac Brick (Creator / Presenter) - **Self-identified role:** *"I'm a software engineering kind of tech lead working on the Google AI edge stack."* - Delivers the full talk: on-device agents, skill harness on AI Core, tiny LLMs (sub-1B parameters), LiteRT/LiteRT-LM runtime, Google AI Edge Gallery app, FunctionGemma fine-tuning, and the Eloquent transcription app. - Also answers audience questions at the end (skills robustness, LiteRT vs. `.task` file format, performance data). ### Referenced (Non-Speaking) Individuals - **"Ali" / "Ollie" (possibly same person, transcription ambiguity)** — prior speaker; covered on-device motivations and system-level GenAI. - **Florina** — co-presenter of a prior talk referenced by the speaker. - **Wei** — colleague present in audience, second row; co-presenting a performance talk "tomorrow." - **"Chinchin"** — attendee greeted by name near the end. - **Unidentified audience member(s)** — asked questions on skills reliability and file formats; voices not individually transcribed/attributionally resolved. ## Roles Summary Table | Speaker | Role | Evidence | |---|---|---| | Cormac Brick | Creator / presenter, Google AI Edge tech lead | Title + self-introduction in transcript | | Audience questioners | Guests (unidentified) | Q&A segment at end | | Ali/Ollie, Florina | Referenced prior presenters | Named by speaker, did not speak in this video | ## Promotional Segments - **Google AI Edge Gallery app** showcase (Android/iOS, open source on GitHub) — product promotion of Google's own tooling. - **FunctionGemma** and **FunctionGemma fine-tuning lab** (Hugging Face space, YouTube video) — promoted as resources. - **Eloquent transcription app** — Google app, explicitly framed as "proof of life example"; **availability limits stated:** iOS only, *not available in Europe yet*. - **Gemma 4 / Gemini Nano / AI Core / MediaPipe / LiteRT** — Google products referenced throughout. ## Provenance Facts & Referenced Sources - **FunctionGemma** — published with DeepMind team, December (prior year), based on Gemma 3 technology, 270M parameters; performance claim: Pixel 7, ~2,000 tokens/sec prefill, 140 decode. - **Fine-tuning result claim:** app-intents success rate 46% out-of-box → over 90% for 8 of 10 functions (speaker: "I think 46% or something like that" — expressed with some uncertainty). - **Apple FastVLM** — third-party referenced example; 500M parameters, running on Qualcomm NPU. - **Apple Intelligence on iOS** — third-party system mentioned: "Apple also has something going on with their intelligence on iOS that I probably know a lot less about" (speaker explicitly disclaims knowledge). - **Synthetic dataset generation** — workflow stated to use *"flash"* (Gemini Flash) to create fine-tuning data. - **Third-party models** — app supports "quan or phi" (likely Qwen/Phi; transcript spelling uncertain) and 3P models. - **Ongoing/uncertain items:** Swift API and iOS open-sourcing "coming soon"; multi-skill single-answer calls "works sometimes"; community skills only live "since last Thursday." ## Comment Availability - No comment availability information present in the supplied transcript or metadata. ## Uncertainty Notes - Transcriber spellings "lighter TLM/LiteRT," "quan," "Java 4" (likely Gemma), "Ali"/"Ollie," and "function journal" (likely FunctionGemma) are ambiguous; roles assigned only where explicitly stated. - No timestamps were provided in the transcript. Yeah, so while we wait for it to come up cuz I know we're short of time, I'm going to talk about agents on device. So I know whoever asked the question about skills and AI core, we have an answer to that. We've built a simple skill harness on top of AI core that you can build skills on. We'll be able to show that. Also going to talk about tiny LLMs which are we would call LLMs that are like smaller than a billion parameters. They're small enough to build into your app if you want to have more customization or you want to do something that isn't already available for you in AI core. So that's the gist. So quick overview of AI edge, well, how we think about small language models, tiny LLMs, and system gen AI. Then we're going to take a quick look at agent skills which is something we can build on top of of kind of system gen AI or the new models that are coming down the pipe. Um and then we're going to take a quick look at tiny models. So that's that. Okay, so cool. Yeah. Cool. Yeah, I'll Okay. Yeah, feel free. Um okay. So AI edge SLMs and TLMs. Okay, so I think Ali already covered this. We know it's great to do things on device, latency, privacy, offline use, reliability, or savings depending on thing These are all motivations to do things locally. Um me by way of intro, I didn't really do this. I kind of I'm a software engineering kind of tech lead working on the Google AI edge stack. So that's We have MediaPipe which is an asset some people may be familiar with. We have lighter TLM which is a LLM harness that you can integrate with your app. Um where you download the model and ship the model with your app. And then we also have kind of lighter T as a runtime that supports both lighter T L M and MediaPipe. Uh, it's kind of formerly known as TensorFlow Lite, which is a kind of cross-framework runtime for running um, models. And all of that can run on CPU, GPU, or NPU, uh, depending on the platform and depending what's best. And you as a developer get to choose. Uh, yeah, it's already trusted at scale. Um, like the lighter T runtime, there's a version of that built into Android OS. Uh, lots of Android apps already use it. So, so it does uh, support over 2.7 billion devices, like lots and lots of daily invocations, and lots and lots of Android apps uh, leverage this. Uh, but also works far beyond Android as well. So, we support all of these platforms. Um, and uh, for example, Gemma is available on many of these platforms. Our team like uh, is giving another talk tomorrow, so you can hear more about Gemma performance on all of these types of platforms and are able to do uh, really useful things with the latest Gemma 4 models. Um, but then building on Ollie's and Florina's talk, this is kind of key idea is that we have um, system-level GenAI, which is something that will be pre-installed in the system. So, there's Gemini Nano via AI Core. This is an example of the summarization API. Um, Apple also has something going on with their intelligence on iOS that I probably know a lot less about. Um, but as a concept, right? Um, as an app developer, when you go to build a mobile app, this is kind of one choice is there will often be an some form of intelligence built into the system that you can leverage. Um, which is uh, you know, highly optimized as kind of Ollie and Florina covered, and that's available for use with your app. Um, then, so this is kind of typically like small language models like for for Nano, it is the Gemma 4 E2B and E4B are the base models for for what we ship there. That's really capable, highly optimized, preloaded with device. If you can use those, it's great. Your app doesn't get any bigger. And if it meets your use case needs, it's a great place to start. If you want like more uh if you have a more specific task um that you want to do that's kind of highly customized or something really boutique, um you can use an App Gen AP Gen AI. So, that's with the lighter TLN runtime. Uh that can be loaded with your app or even your web page, right? Um and this offers kind of a higher degree of customization and reach. Like definitely more work. Um but uh yeah, you kind of access to uh smaller models that can run on lots of devices. Um and full customization. Though, it's clearly a lot more work, but it's the other option that's available. Okay. So, uh rest of the talk uh 15 minutes going to cover two key ideas. One is um hey, how do you do skills on device cuz this is something new that we can do with Gemma 4 that came out last week. We've a few examples of that. This is one key idea. The other idea I want to cover is hey, for tiny models, what can you actually do with those types of models today because we've actually made a lot of progress in this in the last 6 to 12 months. So, I kind of just want to share what's the state of the art with tiny LLMs. And if you want to use one in your app, how would you go about that? Okay. So, this is wow, there's a lot on the screen. This is um uh an app that our team have developed that works on both iOS and Android for running LLMs locally. Um and here we show both really tiny LLMs so you can see what they can do, but also cuz Gemma 4 just came out, we're also using this to showcase what how Gemma 4 can work um on Android and iOS as well. And this actually builds on AI Core. When AI Core is available on the device, it will use AI Core to kind of provide the Gemma model for the app. So, skills is the thing I want to kind of go into deeply today, but there's a bunch of other things in the app. like you can do AI chat, you can ask image, you can do audio scribe, and there's lots of example models, and the app also supports 3P models, like kind of quan or phi or these types of models. If you just want to load a model, get a feel for how it performs on device. And this app is also open source on Android, and it's built using lighter GLM. So, it's both a neat way for you to try things out, but also if you're keen, you can kind of dive into the code and see, "Hey, how does it all kind of hang together?" as an example for lighter GLM. All right, but we're going to dive into skills, cuz this is kind of a topic du jour. Okay, I'm not going to play this video cuz I don't have enough time. But yeah, this app is available Android, iOS, code available on GitHub as well. Um let me just take a picture. Okay. Um Okay, and the app is called Google AI Edge Gallery. So, this is the video you will watch cuz it's shorter and meets my time budget. And we don't Could we get sounds? \>\> one agent task. \>\> Sorry, I'll go Please reply to me in English. So, this uses a restaurant related skill. And we'll see how that's built in a moment. Let's have a look at the restaurants. Select one. Winner, right? So, that's an example of something neat that you can build like with a simple agent harness on top of Gemma 4 that's like really just a few line like pretty easy to do with a few lines of code or a few lines of the right vibe coding prompt as you'll see in a minute. Okay. Okay, I don't know how that Ah, yeah. Okay. Uh Okay, I kind of got lost a little bit. Okay. All right, sorry. Back back back to where we're was to be. So, what's actually happening under the hood? So, like I was saying, this is built on like um this is built just using a prompt, right? And here you can provide We have our own system prompts in our app. Uh then we also put the skill descriptions into the prompt. So, the um so the model is aware of the types of skills it can use, but it doesn't have to see all of the functions and details of the skill. That's only kind of loaded on demand. Um so, we actually have a a load uh a load skill tool call built into the model that then like selectively, so if you say, "Hey, can you show us the location of the Google office?" It will then know, "Wow, I should use the map skill." It then loads the skill for map navigation. Um the tool responds, and then it uses the show JS tool um to show you the location um in the app as well. So, one of the things that's neat about being in an app is you can put simple JavaScript into the skill that we then call as part of the skill. Uh so, this is how like I don't have the corresponding demo for this, but this would kind of pop up a nice um uh kind of like JavaScript UI of kind of Google Maps to kind of just show you uh in the app right there. Uh similar to the restaurant roulette, that was a custom JavaScript uh to do the rendering to do the uh roulette wheel piece. Okay, so you can create your own skill as well. Um the app supports this. Uh sorry. Oh, yeah. Instructions on GitHub. I don't know if I can get past this page too fast. Uh but also um I just create your own skill, there's full instructions there if you want to kind of hand write it out. Uh this works really well though. Uh so, you can use skills to write skills. So, we have um Gemini CLI or code coach like our team have done like about 80 skills and had a lot of fun with this. Um so, this is an example of a prompt which works really reliably. Um in Gemini CLI, we actually have an ADB skill as well um that that we our team uses a lot. So, you can even debug and test by saying, "Hey, you have access to a device um via ADB and you can um also ask to test uh that." So, this type of thing actually works really really well and it's fun and you can then create a skill and then in the app there's a dot dot dot button and you can go to load your own skill from a URL if you kind of publish it your custom to your own GitHub. And it's kind of really easy to do from within the app. Uh you can then also uh let us know in our discussion on GitHub um that you've created a skill and then other people can check out your skill and kind of use that as well. So, these are some things This has only been out like since last Thursday, but these are some example skills that the community have built. So, feel free to do it and tag it up here. Okay, so that's skills. So, 10 The last 10 minutes we are going to spend on TLMs um or probably more ideally maybe five or six of the minutes, so there's time for questions. Okay, so lighter TLM, this is the runtime that we have that um we use for running models. It runs models in lighter or TLM format, which is a single file that packages everything we need to know about the model in order to be able to run it. Uh it's open source, it's fast and it works on multiple platforms. Uh and there is a Swift API and a JavaScript API coming soon. At the moment, if you go to the GitHub, you can see the C++ and Java version. And when we publish the Swift version, we will also publish um we'll also open source the iOS app at that point in time. So, if you go to gallery for at the moment, you can only see the code for Android, but um hopefully in the next few weeks um where uh we can get the Swift work finished, have a really good API and then we'll be able to um open source that as well. So, yeah, and it supports Java 4 as well on all of these devices. Uh it also supports loads of other models, but understandably Java 4 is our favorite. Um so then to deploy uh tiny model, what do you do? So, typically starting transformers, you then have a package called lighter or T torch that can help you export the model and then lighter or TLM. There's actually a reference version of that that you can use on your desktop as well if you want to try out a model. You can either try it out on a desktop or you can load it into the gallery and see it perform there. And then you can deploy with lighter or TLM. It's worth noting for smaller models, you would either pick a fixed function model like visual language model or a transcription model or something like this. So, there are some pre-built models available on our on our on our transformers page that you can use. But, something we also see that's really common is people fine-tuning models um because certainly once you get down to like 200 or 100 million parameters, for that model to work, it needs to have a very narrow and focused task and um we've had a lot of success deploying those models internally and in an app a different app that you're going to see in a minute um by doing kind of fine-tuning using synthetic data. So, this is what the export and inference flow looks like. So, this is um this is showing okay, on the left-hand side it's showing us exporting a quant 0.6 model and then running that um on desktop with lighter or TLM run so you can just see how that behaves using a GPU for example. Uh the right-hand side is showing a different example which is Apple's fast VLM. Um this is a visual language model that's only 500 million parameters and this is optimized and running on like a um this is running on the Qualcomm NPU that's also available through our stack NPU optimization. So, this is an example of that happening end-to-end. And this is running really quick cuz it's using hardware acceleration. And this is model is just that particular model is 500 million parameters by way of example. Another example that we've spent a bunch of time with uh the DeepMind team on was publishing function journal. It's something we published last uh December. This was based on Gemma 3 technology. This is only 270 million parameters, but it's robust function calling when fine-tuned, typo there. And this is then it's small and it's really fast even on legacy devices. So, if you go all the way back to a Pixel 7, they still can process almost 2,000 tokens per second prefill and 140 decode. So, it's really useful for lots of It's really useful for lots of simple use cases, like you can do text to function calling or voice to function calling using this size model. And there is a whole YouTube video on this called Function Gemma. If you want to find out lots more details about how to do it. We also from um have a function Gemma fine-tuning lab. So, if you search function Gemma fine-tuning lab, I don't have here. This is available as a hugging face space. You can kind of import, you can define functions, upload your own data, and see how they kind of fine-tune function Gemma. And this is kind of recommended for really high for really robust function calling. So, we have an example in the app called like app intents where it'll do like this the thing you saw previously of like add calendar or add email. So, when we took function Gemma out of the box, our success rate in that was I think 40 6% or something like that. Then we put it through this fine-tuning flow where we're like, "Hey, we have these app functions." And instead of providing that via a system prompt, which is what you would do if you're using a larger model or if you're on a device with AI core, for example, um but um you instead need to kind of synthetically create a data set, right? It's It's typically the workflow. We use flash to synthetically create a data set. Um upload it to this type of tool or we obviously have our own internal tools, but that then got that 46% um to over like it was over 90% for eight of the 10 functions we were trying and two of the functions were a bit lower in their kind of ACs. Um, so you can get really robust and reliable function calling using this fine tuning workflow. Yes, it's it's a bit more work than just prompting a larger model, um, but it does allow you to kind of ship something robust in your app at scale. Oh, sorry, going the wrong direction. Oh, yeah. So then pre-built uh, tiny models are here. Um, yeah, okay. I'm going to pull stop for questions. Okay, so we have and I don't want to go to the details. We have another app I'll just speed run this for 1 minute. We also have another app called Eloquent, which is a transcription service. Um, but what's more interesting than the app was just an example of like how we built that. So it also supports things like personalization. So it does transcription with with your own favorite keywords. So if you use a lot of like tech jargon or a lot of people's names, transcription service don't always get that correct. Um, sadly this is only available on iOS and not available in Europe yet, right? So this will be increasingly available uh, soon. But the more interesting thing for the purpose of this conversation is under the hood, this is something we've built using tiny LLMs ourselves. So this uses a ASR engine that we have built based on Gemma 3 technology and then it also has like something we call like a text polishing engine that we've also built uh, with Gemma 3 technology. And both each of these models are only a few hundred million parameters, uh, but chained together they can create a really compelling offline like offline transcription service that is able to leverage a personal dictionary, right? And also like the polishing also removes ums and ahs and that sort of stuff, right? which is also a common gripe with kind of um, kind of offline transcription apps. But yeah, so not really available widely yet. It'll be available in but for the purpose of this conversation, it's more just like a proof of life example. So, like this does work in production. Once you put in the effort to kind of fine-tune a model and you can create pretty compelling things. Okay, so that's not available in iOS in Europe so that's out of the equation. So, yeah. Takeaways, system gen AI and app gen AI, that's the kind of the overall yeah, overall kind of message and wrap up. Happy to take questions. I have 3 whole minutes. Uh I think person there was first. Um yeah, we are still putting models on the clock there. Uh so, we've literally been playing with a model for about 2 to 3 weeks now. It's been in public for about 1 week. We we see like within a single conver So, we can provide like certainly for the 4 billion parameter model, like if you by default we enable about eight skills and it's able to choose between the eight skills reasonably well, right? Um within a conversation you're able to say, "Hey, like you know, like um like find me out a fact on using a Wikipedia scale. Then, oh wow, show me where that is on Google Maps." So, if you have a conversation that uses skill, skill, skill, that works really robustly. The thing we're still working on that's harder is through a single like um interaction with the app for the app to know to call multiple skills as part of a single answer. That's um that works sometimes, right? And we're still like that's something we're still figuring out how to make that more robust, right? Um but yeah, like it's all in a just our like our engine harness thing is really simple. So, I'm sure we'll figure that out, but we're still kind of discovering the limits of how far we can push the model. Uh yeah. Yeah. Yeah. Yeah. Yeah. Light or TLM file format for just for stock LLM is effectively a replacement for dot task file. That's a transition we made last year. Dot task files are still useful for things like like a task file creates more things than just an LLM model, right? So there is like a face mesh task and obviously that has a lot of other code as well. But for LLMs we want something dedicated, simpler that people can use with open developer tooling. Yeah. So yeah, so it bundles things like the tokenizer but it's just the LLM model. Okay. Yeah. On TPU? CPU? Um Yeah. So there is another talk tomorrow from some of my colleagues including Wei who's here in second row and that has lots of performance data on Gemma and various models on Yeah. You can also check out our model card in the meantime if you search Gemma light or TLM model card. We we keep it kind of running like we update that whenever we have new performance numbers on new platforms. So there's a lot of comprehensive data there as well. All right. I'm at zero seconds and it's flashing at me. Yeah. Yeah. I was at 2:30. Sorry. Chinchin's here. Sorry Chinchin. Didn't see you.