Talk by Chintan Parikh (Google DeepMind) on accelerating AI at the edge. Covers moving AI workloads onto devices with small capable models, including Gemma edge models and on-device agent skills, and the tradeoffs around latency, privacy, cost, and cross-platform deployment. Discusses LiteRT and the Google AI Edge stack for running models across Android, iOS, desktop, web, and IoT, with demos of local tool calling, structured output, reasoning, benchmarking, and hardware acceleration on CPUs, GPUs, and NPUs. Presented as a practical overview of the current edge AI stack for builders of on-device AI systems. # Channel Presentation Observations: "Accelerating AI on Edge" ## Speaker Identity & Explicit Self-Description - **Named presenters**: Chintan Parikh (transcribed as "Parekh"), who self-identifies as *"product manager for light RT, which is part of Google AI Edge,"* joined by a colleague, Weiyi Wang (per title), present *"Google DeepMind"* material (title attribution). - **Role framing**: The speaker positions himself as a product representative demonstrating an official Google toolchain, not an independent creator — evidenced by repeated phrases like *"the gallery app, which I will showcase"* and *"our Hugging Face page."* ## Claimed Scope - **On-device/edge AI deployment**: The talk covers model offerings (*"Gemma 4... 2B and the 4B Edge models"*), deployment framework (*"light RT essentially is Google's on-device framework... built on the TensorFlow framework"*), and cross-platform support (*"Android, iOS, macOS, Linux, Windows, web, and even IoT devices"*). - **Developer tooling ecosystem**: Gallery demo app (*"you are able to also fork this app"*), CLI tool, AI Edge benchmarking portal, Model Explorer, and open-source GitHub resources (*"there's a whole GitHub repo... users are posting their own skills"*). - **Scale claims**: *"100,000 plus apps, billions of active users"* for Lite RT; performance claims (*"up to 13x boost,"* *"up to like 35 times faster performance"* vs. Llama on mobile). ## Intended Audience (Observed) - **Edge/on-device developers**: Opening poll — *"how many of you are working on deploying on Edge or are keen on learning more"* — with some audience members *"already deploying."* - **Multi-platform app builders**: *"how many of you are looking to deploy on multiple Android or iOS devices?"* and emphasis on *"more than just... mobile platforms."* - **Attendees of a developer event**: References to an *"AI engineer event"* and a demo booth (*"we have been demoing down on the third floor"*). ## Observable Presentation Patterns - **Technical, product-led talk format**: Live demos (robot on Raspberry Pi — *"we just made it this morning"*), slides, and open Q&A (*"we'll go through... Q&A part of this session"*). - **Developer enablement focus**: Repeated pointers to actionable resources — Apache 2.0 licensing, Hugging Face downloads, QR codes, sample code (*"each of these capabilities has sample code as well"*). - **Openness to audience use cases**: *"leave it open to understand if you guys have any use cases... you'd like to discuss."* - **Honest caveats alongside claims**: The speaker tempers the robot demo — *"it's not super performing, but... we just made it this morning"* — distinguishing demonstrated capability from polished claims. ## Claims vs. Demonstrated Distinctions - **Creator claims** (not independently verified in-transcript): performance multipliers, deployment scale figures, NPU partner integrations (*"Qualcomm, MediaTek"*). - **Demonstrated on stage**: live on-device demos (sleep-tracker agent, photo-to-music, Raspberry Pi robot), audience Q&A handled by the presenters. # Delivery Observations — "Accelerating AI on Edge" (Chintan Parikh, Google DeepMind) ## Tone & Register - **Conversational, informal conference-session tone** throughout: "Uh quick show of hands, how many of you are working on deploying on Edge" [27]. - **Frequent filler words** — "uh," "um," "you know," "like" — appear consistently across the talk, e.g., "I'll try to, you know, also leave it open" [51], "So, if you're looking for like extremely small models" [96]. - **Self-deprecating honesty about demos:** "So we just made it this morning, so it's not super performing" [1043] and "yeah, we we do need to work on the performance because we just made it this morning" [1078]. ## Teaching Habits & Audience Interaction - **Repeated audience polls by show of hands:** - "quick show of hands, how many of you are working on deploying on Edge" [27] - "Is anyone familiar with TensorFlow Lite?" [688] - "Maybe a quick question, how many of you are looking to deploy on multiple Android or iOS devices?" [802] - **Explicit agenda-setting and signposting:** "here's going to be a quick set of agenda items" [51]; "the next part of the talk is I'm going to focus a little bit on deploying this now on edge devices" [654]; "I have a minute left, so yeah, this In summary" [1142]. - **Live demo narration:** he describes on-screen app behavior in real time — "So let's try this one. >> I'm sending a photo. Can you pair this vibe with some music?" [523–545]; the Raspberry Pi robot demo [1032–1056]. ## Recurring Phrases & Vocabulary - **"On device" / "on the edge"** repeated constantly: "all of this is happening on device" [372]; "it's able to create this all on device" [501]; "generate music also all on device" [523]. - **"So" as a sentence-starter** — extremely frequent: "So, uh, in terms of models" [186]; "So, let's dive into uh the Gemma 4 use cases" [415]; "So, we have options for that, too" [824]. - **Sales-y intensifiers:** "it's going to be a game-changer" appears twice [944, 966]; "certainly" is used often — "we certainly have a Hugging Face page" [107], "you can certainly start the inferencing on edge" [252]. - **Hedged qualification habit:** "again, it depends on your end use cases again" [197]; "which I would not go through the whole journey here" [789]. ## Communication Patterns - **Deferral to external resources** rather than in-depth answers: references to the Hugging Face page appear many times — "our Hugging Face page has these models" [317], "you will find it there" [1131], "I don't have examples here" [1311]. - **Invitations to try things yourself:** "you're also welcome to explore that or you're welcome to download any of these apps" [622]; "you can try it out downstairs" [1067]; QR codes offered twice [350, 654]. - **Q&A style:** brief, direct answers; comfortable admitting limits — "I don't know if I have Orin here. Uh not here but I think we have it on our hugging face" [1287]. - **Occasional verbal stumbles/restarts**, e.g., "the the the skill was set up and created and loaded" [599], "a a a this is the coverage right now" [1000] — consistent with live, unscripted delivery. ## Uncertainty Notes - The transcript shows only the primary speaker in detail; the colleague ("Weiyi Wang" per the title) contributes minimally in the visible Q&A, so his delivery style cannot be assessed. - Timestamps are coarse (transcript line markers, not seconds), so pacing observations (e.g., rushing the summary at "I have a minute left" [1142]) are based on the speaker's own statements, not measured timing. # Position Extraction: Accelerating AI on Edge — Chintan Parikh (Google DeepMind / Google AI Edge) ## Speakers - **Chintan Parikh** — Product Manager, Lite RT (part of Google AI Edge), identified explicitly in transcript. A colleague (Weiyi Wang, per title metadata) is present for Q&A but does not speak substantively in the transcript. --- ## Claims and Positions ### Gemma 4 Edge Models - Google DeepMind recently launched **Gemma 4**, with the talk focusing on the **2B and 4B Edge models** for on-device deployment. - The **Gemma 3 family** also exists in smaller sizes, "all the way down to 270 million parameters," available via Hugging Face for fine-tuning. - The "big evolution" with Gemma 4 is moving from **chatbot capabilities to autonomous agents** with reasoning capabilities. ### Edge Deployment Benefits (Parikh's stated reasoning) - **Latency**: "real-time latency is king" for camera use cases like filters, background replacement, and video calls. - **Privacy**: critical for sensitive use cases such as summarizing sensitive documentation. - **Offline use cases**: where connectivity is poor. - **Cost**: on-device offers a **hybrid approach** to offset edge vs. cloud compute, addressing token-cost complaints he says he heard at the AI Engineer event. ### Model Resource Requirements (as stated, qualified) - **Gemma 4 E2B**: roughly **1–2 GB RAM** usage — usable, but "it depends on your end use cases." Good for voice interfaces, summarization, low-latency local processing. - **Gemma 4 E4B**: "a little more heavy duty," higher RAM requirements, suited to laptops/IoT; figures given post-quantization. ### New Capabilities in Gemma 4 Edge Models - **Function/tool calling** built-in, with the ability to call external APIs while keeping core inference on-device. - **Native structured JSON output** — built into the model architecture rather than achieved via prompt engineering. - **Chain-of-thought "thinking mode"** — demonstrated via the gallery app. - **Hardware-native optimization** across multiple platforms. - Models are **Apache 2.0 licensed** on Hugging Face. ### Gallery App and Use Cases - Gallery app demonstrates **agent skills, audio scribe, ask image, and chat experiences — all on device**. - Purpose stated as a **playground** to inspire developers; sample code provided; app is **open source and forkable** on GitHub, with a community skills repo. - Demonstrated examples (some via recorded demo audio): - Wikipedia-query skill for encyclopedia questions. - **Sleep/mood tracking agent** creating journal entries, summarizing, and analyzing 7-day mood trends on device. - **Photo-to-music pairing** with image understanding and on-device sound/music generation. - Animal vocal-calls app navigating multiple apps in a more complex workflow. - Users can **write skills within the app itself** and choose CPU/GPU (and "hopefully NPU support soon") as accelerator. ### Lite RT Framework - Lite RT is **Google's on-device framework, built on TensorFlow Lite**, using the same TFLite model format. - Adoption claims: **100,000+ apps, billions of active users**, many daily interpreter invocations — presented as a "trusted foundation." - Rebranding motivation: to show it accepts **PyTorch and JAX models**, not just TensorFlow Lite, convertible to the TFLite format. - **Cross-platform**: Android, iOS, macOS, Linux, Windows, web, IoT. - Tooling described: - **Lite RT Torch**: conversion path for bring-your-own models, with optional quantization; LLMs go through the Lite RT LLM path. - **Model Explorer**: study the graph and choose mixed-precision/quantization strategy. - **AI Edge Portal**: cloud-based benchmarking for broad Android fleet deployment decisions (e.g., ahead-of-time vs. just-in-time compilation). - **NPU acceleration**: integrations completed with **Qualcomm and MediaTek**; more partners in progress. Claimed **3–10x performance improvement** and a "game-changer" for energy use in AR/TTS/real-time camera use cases — framed as expectation, not measured result in this talk. ### Performance Claims (as presented, sourced to Hugging Face) - Up to **13x boost** on new accelerators/NPUs "in some cases." - iOS: roughly **56 tokens per second** (figures said to be listed on the Hugging Face page). - Versus **Llama**: up to **35x faster on mobile**, at par on desktop, **3x on IoT**. - Live demo: a robot built "this morning" on **Raspberry Pi, CPU, Lite RT LLM**, responding to a sign by wiggling Sharpie antennas. Parikh qualified: "it's not super performing... we do need to work on the performance." ### Q&A Positions - **On-device face unlock**: face unlock on phones (Pixels/iPhones) already runs locally; Google devices use this framework, Apple uses Core ML. - **Home camera use case**: differs from phone deployment; would need authenticity-checking algorithms, but can run on device — Raspberry Pi "could do it." - **Multi-agent distribution**: no examples on hand, but states Google has groups working on speaker/thinking-agent architectures with orchestration deciding what runs locally vs. elsewhere. - **Open-weight models beyond Gemma**: Lite RT can support other open-weight models if converted to the right format; Hugging Face page includes some, and users can convert others themselves. When asked about Jetson Orin support: "not here but I think we have it on our hugging face." --- ## Qualifications and Tensions Preserved - RAM figures for E2B ("roughly," "depends on your end use cases") and E4B are approximate and post-quantization. - NPU 3–10x claim is framed prospectively ("is going to give you") and "game-changer" is rhetorical emphasis. - The Raspberry Pi robot demo is explicitly acknowledged as underperforming since it was built that morning. - Performance comparisons vs. Llama are limited to "at least on mobile" per Parikh's own wording; "up to" and "in some cases" qualifiers retained. - Parikh repeatedly defers detail to external sources (Hugging Face, website, GitHub) rather than presenting data in the talk itself. # Speaker Attribution Record ## Video **Title:** Accelerating AI on Edge — Chintan Parikh and Weiyi Wang, Google DeepMind **Video ID:** `01a15ca9-7a8b-7332-b986-925ce7a587a9` ## Identified Speakers | Speaker | Role | Evidence | |---|---|---| | **Chintan Parikh** (transcribed as "Chintan Parekh") | **Creator / primary presenter** — Product Manager, Lite RT, Google AI Edge | Self-introduction: *"my name is Chintan Parekh, product manager for light RT, which is part of Google AI Edge"* | | **Weiyi Wang** (per title; unnamed in transcript) | **Co-presenter / Q&A participant** | *"I also have my colleague here, where he also will be joining us also for the Q&A part of this session"* — colleague is present but not individually identified in transcript | | **Audience members** | Unidentified questioners (Q&A) | Q&A exchange at end; audience questions not individually transcribed, only presenter responses | | **Device/app voice demonstrations** | Recorded demo audio, not live speakers | Journal-entry demo, mood-trend query, photo/music pairing prompts within the gallery app demos | ## Referenced Sources & Products - **Google DeepMind Gemma 4** (E2B, E4B Edge models); Gemma 3 family mentioned (down to 270M parameters) - **Lite RT** (formerly TensorFlow Lite / TFLite) — Google's on-device framework - **Hugging Face page** — model downloads, Apache 2.0 licensed; performance numbers sourced from it - **GitHub repo** — open-source sample app; community-posted skills - **AI Edge portal** — cloud-based benchmarking tool - **Model Explorer tool** — graph inspection/quantization - **Third-party claims cited:** Qualcomm and MediaTek (NPU integrations); Apple Core ML (face unlock, per audience Q&A); Llama (performance comparison: "up to 35x faster on mobile") - **Platforms named:** Android, iOS, macOS, Linux, Windows, web, Raspberry Pi, IoT ## Promotional Segments - Repeated promotion of the **gallery app** and **third-floor demo booth** ("we have been demoing down on the third floor... demo stand session after this") - **QR codes** directing to skill-building instructions and downloads - Distribution calls-to-action for **Hugging Face models** and **GitHub sample app** ("you're welcome to take it and fork it") - Performance claims framing (e.g., "3 to 10x improvement," "game-changer") are presenter assertions, unverified in-source ## Provenance Notes & Uncertainty - **No curated metadata supplied** — speaker identities rest solely on transcript self-identification and video title. - Speaker name discrepancy: transcript renders "Parekh"; title says "Parikh." Recorded as-is. - "Gemini 4" and "Gemma 4" are used interchangeably in the transcript; both transcribed verbatim. - Timestamps absent from transcript. - **Comment availability:** not indicated in source material. - Presenter explicitly defers some claims: performance numbers *"just pulling from [Hugging Face]"*; robot demo built "this morning... not super performing"; no examples available for speaker/thinking-agent architectures. Good afternoon everyone. We'll get started. Uh my name is Chintan Parekh, product manager for light RT, which is part of Google AI Edge. And I also have my uh my colleague here, where he also will be joining us also for the Q&A part of this session. Uh quick show of hands, how many of you are working on deploying on Edge or are keen on learning more about like what it's going to be the big benefits? Okay, and some of you already deploying. So, I'm going to go through the slides. I'll try to, you know, also leave it open to understand if you guys have any use cases or any sort of things you're working on that, you know, you'd like to discuss. So, we can kind of keep it open in that sense. Great. So, here's going to be a quick set of agenda items, and I'm going to go through some of the new models that are coming up. Also, some Edge use cases that would be relevant. I'm going to show you some of the new capabilities in the gallery app, and then you'll go through our stack for deploying AI on Edge devices. In addition, we also want to emphasize the cross-platform support we offer, cuz I think a lot of you are looking to deploy on just more than just on mobile platforms, but also other platforms. So, we'll kind of do some of that. Um Google DeepMind recently launched Gemma 4, and this talk is going to be focusing a little bit more on like the 2B and the 4B Edge models that are are focused on deploying on device. Google certainly has a big suite of models, and even the Gemma 3 family, which are also in the smaller size, like all the way down to 270 million parameters. So, if you're looking for extremely small models that you're able to fine-tune, we certainly have a Hugging Face page, which I'll go over, which has more of these models. Um so, yeah. So, the big evolution with Gemini 4 is going to be really moving from like chatbot type capabilities to more autonomous agents that also support reasoning capabilities and more sophisticated features. And uh, a lot of it is also uh, it's going to be great how you can deploy these on different devices. So, running on edge has many benefits. I think uh, my colleague went over some of these uh, yesterday and I'll just, you know, run through this uh, really quick also. So, certainly like latency is important for those who are keen um, any of these like real-time camera use cases like filters where you're looking to replace backgrounds or like video calls like real-time latency is king over there. So, that's, you know, on device can help with that. Privacy is also critical for any use cases where, you know, you have sensitive details, you're doing some kind of summarizing uh, documentation which is sensitive, offline use cases where you have poor connectivity, and costs like, you know, I've seen so many presentations uh, this uh, uh, AI engineer event where uh, people are complaining about the number of tokens they're getting used and this and that, but I think uh, on device always offers this hybrid approach where if you're looking for like how do I really offset, you know, uh, running things on the edge versus on cloud and where uh, can I get the best balance? I think there's a there is something here. Cool. So, uh, in terms of models, there's two models and I think like a lot of the use cases I'll show are going to be built on these. So, the Gemini 4 E2B roughly from a from a amount of RAM usage is roughly anywhere from 1 to 2 GB of RAM usage. So, it could be usable, but like yeah, it depends on your end use cases again. Uh, certainly good for any uh, voice interfaces or uh, use cases for uh, summarization or any kind of low latency low local processing. Uh, 4 4B is going to be a little more heavy duty if you're looking to run on bigger platforms like laptops or IoT devices. It'll have like a higher RAM RAM requirement. Again, this is once it's been quantized uh, to your desired size. So, uh, I want to do a quick deep dive on what's going to be new this time on on Gemma 4E2B and E4B capabilities in terms of the agent capabilities. So, I'm going to show you what's already there in the next couple of slides in terms of use cases and here touch a little bit on exactly like what are the new capabilities are there. So, function calling built-in support for tool calling and also essentially models to interact with other local APIs. So, you can certainly start the inferencing on edge, but you have the path to essentially, you know, call other APIs outside and that I'll show that in some of my examples, but the core of the inferencing will be on the edge. Uh structured JSON output. So, for native support for any structured JSON output is supported here, which was built into the model architecture rather than achieving through some kind of specific prompt engineering. You can kind of do this too and I'll show examples for the same. Uh chain of thought is new. So, there is a thinking mode uh which we would which can be demonstrated by app where thinking mode will help you understand like the thought process that the model is going through. Our gallery app, which I will showcase, will will support that. And finally, what's going to be great is these models are optimized for hardware native support means you can run seamlessly across multiple platforms and multiple hardwares and really we want to give you the flexibility of deploying into a various platforms. So, just for starters, if anyone's like thinking, "Okay, so where can I get these models?" Like we have these models already ready to go. So, our Hugging Face page has these models. So, if you're looking, you can uh you know, they all Apache 2.0 licensed models. So, you are able to download these and then, you know, start building with it and we'll give you some paths of how you may want to build with it. All right. So, So, yeah, let's talk about some of the use cases now. So, in terms of these are just some of the many possible use cases. Uh These are use cases that are possible today. Uh this is our gallery app, which we have been demoing down on the third floor and we also have a demo stand session after this for anyone who's keen on learning more. I do have QR codes for this, too, so in the next couple of slides. Um at present, you know, what's new is the demo the the gallery app allows you to demonstrate the agent skill capabilities. Um and also it has the audio scribe capabilities or ask image, and also other features that were uh related to uh chat experiences. Um all of this is happening on device. The purpose of this app that is created by Google is to help you get a playground to get a feel for what these models are capable of. Uh each of these capabilities has sample code as well that anyone can go and grab, and then you are able to also fork this app to build your own experiences, but this is essentially to inspire and motivate you to build your own experiences. So, some of the next couple of slides I'll be focusing on is going to be related to some of the Gemma 4 Edge use cases. Uh these are going to emphasize on like the voice agent capabilities of the local agent capabilities, and also a lot of these are going to be privacy-focused as well. So, there are three key pillars that are going to be new here uh versus what was already supported in the previous slides. So, let's dive into uh the Gemma 4 use cases. So, let's see if this plays. So, here's one use case where you're augmenting knowledge, and you know, here for example, you can build a skill to query the Wikipedia allowing the agent to query uh and respond to any encyclopedia question. So, this is uh this is a skill that's already available in our app that's uh that's there. So, if you're looking to build something like this, this is this is this is possible. Uh these are new skills in addition to like basic on-device skills like summarization or ask image and things of that nature. Um this is another category where \>\> essentially, let's see if you can journal entry with score nine and comment. Same here. \>\> 8 hours of sleep and I'm looking forward to hanging out with Amy today. So here is you're going to see creating and summarizes and display trends of hours of sleep. So you're going to build a sleep in a quick agent that helps you track your mood. \>\> Analyze the trend in my mood over the last 7 days. So you can kind of see that it's able to create this all on device. It's taking your input, it's feeding it, it's able to understand. So the reasoning and the thinking capability are new in the new model and that the the on-device capabilities have gotten a lot more powerful. Another similar example is going to be on expanding like the core capabilities. So for instance, you can also pair photos and it's going to read the photo and have image understanding and generate music also all on device. So let's try this one. \>\> I'm sending a photo. Can you pair this vibe with some music? Taking seconds and then So a lot of these skills can be written by yourself on the app itself. So you don't even need to leave the app. The app has instructions on how to do this. Again, like the whole sole focus of this is to help you understand the capabilities of a model and build something by yourself that you know you really like. So that's useful. One last one is to really want to do sort of a working app that describes, let's say, vocal calls of animals. Like now here we are trying to navigate multiple apps and users and you can manage a more complex workflow here in this case. So, this is another example. And you can also change the CPU, GPU, and hopefully you have NPU support soon so you can decide the accelerator you want to use when you decide. So, this was sound generation here from purely the prompt by the person. Again, the the the skill was set up and created and loaded and the skill is running on device and this is for something similar. Right. So, there's a lot you can do with it. There is a whole GitHub repo that we have where where actually there's like users are posting their own skills on the GitHub website and and they're able to basically share that with others with the community. So, you're also welcome to explore that or you're welcome to download any of these apps. Another option is the sample app is actually open source as well on GitHub so you're welcome to take it and fork it and you can also make changes to that app if you like. So, the GitHub link is also here and it's an opportunity to do that. And creating your own skill is also here. So, these are just an examples of what folks have have done. This QR code is instructions on how to build a skill. So, if you're looking to essentially get guidance on okay, how do I do this you know, the uh you can kind of get started. All right. Um the next part of the talk is I'm going to focus a little bit on deploying this now on edge devices. So, we've talked about what's possible with the Gemma models. Uh the framework that is used here, right? So, so Google has an offering with light RT for bring your own models essentially. So, light RT essentially is Google's on-device framework. It's built and built on the TensorFlow framework. TensorFlow Lite, if you guys are familiar with that. Is anyone familiar with TensorFlow Lite? Some of you guys, okay. Awesome, okay. So, that's what this is built on. Uh it's really meant to also be built on using the same TensorFlow Lite model format, and that's what we are focusing on at this point. So, here the the point is that the underlying framework for the app that was running all these experiences is Lite RT, and then we are going to show you that this is basically it's been one of the most widely deployed framework so far. It has uh 100,000 plus apps, billions of active users, and also lots of daily interpreter invocations. It's essentially number of inferences that are happening every day. Uh why this is interesting is to show that we are building on a trusted foundation, and when you do this, like we also have our TFLite file format. And this is going to be very important because if you were a developer that was building with TensorFlow Lite, your models are still going to run on Lite RT, and these models, the same model format is cross-platform. So, it's not just Android, but you can run it on iOS, macOS, Linux, Windows uh Windows, web, and even IoT devices, and I'll also show some examples on IoT that we were actually just building this morning, actually, so I'll show you that. Um Cool. So, from our development side of flow standpoint, and uh essentially, you have your model files. So, we are able to accept the reason it's got branded to Lite RT. One of the bigger motivation for the rebranding was to also demonstrate that it's not just TensorFlow Lite models that we accept, but also PyTorch models and JAX models. So, if you are working with PyTorch models, you can take one of those models, convert it to the TFLite file format, and then go through this journey, and deploy. So, we have a lot of sample apps and things like that, but I would not go through the whole journey here, but simply put, like you have portability of the model, and you have multi-framework support for the models. All right. So, the complete This is a complete solution that provides one unified cross-platform architecture. Maybe a quick question, how many of you are looking to deploy on multiple Android or iOS devices? Like you want something that you build that you can test easily on many devices. Cuz if you are trying to build apps that you're like, "Okay, I made it, but how do I know if it's going to work on like 5-year-old phones and 6-year-old phones and all of that stuff?" So, we have options for that, too. So, I'll walk you through the stack. So, Light RT torch is going to be your conversion path. If you basically it's your bring your own model. If you found a model, you like it, you want to run it, you convert it to TensorFlow TFLite format. You can quantize it if needed. Uh if you do if it's a LLM, you go through the Light RT LLM path uh essentially or and then if it's not, you can go through Light RT. What's interesting here is that we also have the model explorer tool which can help you explore the graph and decide which aspects of the graph you want to change and quantize so you can actually study the graph and decide how to best do mix precision or or basically complete convert the quantize the model as you like. The AI Edge portal is a benchmarking tool which could be of interest for those who are looking to deploy broadly on Android. So, this is a cloud-based benchmarking service called AI Edge portal. Um this is available basically to help a lot of our uh third-party app developers and even like internal developments use this tool essentially to get a good pulse check that, hey, you know, if I have a model that so many parameters, do I need to use ahead-of-time compilation or just-in-time compilation? What is going to be my right recipe to ensure it's actually deployable across a broad fleet of devices and be reliable in that manner? Um yeah, sorry. One more thing on this slide is uh the next part is acceleration. So, CPU and GPU are pretty universal right now. So, these are are essentially libraries that will help you run CPU and GPU. I also want to emphasize back like with this sort of uh framework, you are able to deploy on multiple platforms uh with the CPU and GPU running. Uh NPU acceleration is also another focus. So, we have completed uh integration with Qualcomm, MediaTek, and we are also focusing on additional integrations with other uh partners in the and on on different platforms. Uh we also offer if you are looking, I think this is going to be a game-changer for a lot of your apps who are or a lot of your products that you are trying to run lot of AR, TTS, so any of these applications that are going to be you're acquiring real-time capability or or you're setting up some kind of AR/VR application where you want to be able to have real-time improvements or updates to uh the camera feeds, I think the NPU is going to give you at least like 3 to 10x improvement in performance and it's going to be a game-changer in terms of the amount of energy you're going to use uh and the amount of performance you're looking for to unlock those use cases. So, the flexibility part is we have the head of time or on-device and then for ease of use also there are many options here in terms of how we simplify and there's a lot more uh documentation and support on that. All right. So, the next part is I'm going to go over some performance numbers on how we are performing. So, so far we've given you an example of what you can do at the app level, uh what's possible at the framework level, and how you can scale it. Uh in terms of our coverage, right? So, just with the Gemma models, like this is a this is the coverage right now. So, a lot of times they come to the booth our our demo booth and ask, "Hey, okay, so, you know, can you just do this on Android?" But, no, we've been we have tested this on all these platforms right now. So, the models that you have on Hugging Face, you can actually test it on on many of these platforms. So, on Android, iOS, uh Linux, Raspberry Pi as well. Uh and and here's like uh just a quick uh demo we just did this morning. It's going to take a while, but this is our uh little uh robot that's uh sitting down in our demo booth. So, we just made it this morning, so it's not super performing, but what we did here is that we showed a robot uh a sign to say move your antenna. So, now the two lights are blinking, so it's doing inferencing uh and then, you know, you'll see it's running a Raspberry Pi, it's running on a CPU, running on our uh light RTLM. And you'll see in a few seconds that it's going to wiggle the Sharpies that are its antennas. And uh so, there it is. So, so there are other questions like will you marry me and things like that? If you're interested, you uh you can try it out downstairs, it's there. Uh Cool. So, uh yeah, that that was amusing, but also yeah, we we do need to work on the performance because we just made it this morning. Uh We also have a CLI tool, so for those who are looking to deploy and you want to have a easier way of doing it, so there is a new CLI tool that's been developed, so that is also available on our website in terms of uh with Python binding support and many of that. So, so I'll just uh it's on our website, so if anyone's interested, you'll find it. Um in terms of performance, so I have some quick performance numbers like I just want to emphasize like running on some of these uh new accelerators and PUs can give you a big benefit advantage. You can get up to 13x boost in some cases. Um and also, you have uh iOS performance as well, so we are supporting all of this here. So, like roughly 56 tokens per second and so on so forth. Uh desktop is here, too, so a lot of these numbers are can be found uh on our hugging uh hugging face page, so the quant is already listed there. So, I'm just pulling from there, so you will find it there. So, when you download our models, you will get performance details on all these platforms and IoT. Uh our run time is also very performant versus Llama, at least on like mobile, we've seen up to like 35 times faster performance. On desktop, it's at par, and then IoT also, we have like 3x performance. All right. So, I have a minute left, so yeah, this In summary, yeah, light RT supports all these frameworks. If you're looking to bring models from different frameworks, you can. We have the models on Hugging Hugging Face, and then we also have a gallery app, which is a nice playground for you to try this out. Uh and these are there, yeah. So, there's any questions, happy to take them. Yeah, thank you. Yes. Yes. Yes. Right. Yeah, so on your phones, like on let's say even iPhones or Pixels, when you do face unlock, it's typically running locally already. So, and that is using this the same framework in Google devices as well. Uh Apple has Apple's face unlock is also on device on on Core ML. So, so that is possible, yes. Typically, yeah, you have you have the camera on and you can stream it, but like for for your home camera, I think it'll be a little different story, I think, yeah. So, you would have to maybe do some sort of a algorithm to check the authenticity and and kind of do that. So, it'll be a little different from how it's deployed on the phone, but yes, you can totally run that on device. Then then then message your phone, right? Right. Right. Raspberry Pi could do it, right? Yeah. Do you have a question? Yeah. \>\> Do you want to answer that? Uh let me see. I don't know if I have Orin here. Uh not here but I think we have it on our hugging face. I don't think I have it here. Yeah. Mhm. I think we have like we have some groups that are working on let's say like speaker and a thinking agent type of architectures. Uh I don't um have any examples to share but like that is one way of distributing like what and there's orchestration to decide what should run locally or not and not uh and should run elsewhere. So uh I don't have examples here but yes we there are like uh uh different type of you can think of like a health agent or coaching agent of sorts. Yes. Thank you. Yeah. Right. Right. Right. Right. So if we can if you have open open weight models that you prefer like uh we are able to support any of them at I mean hopefully we just need to make sure we get them in the right file format and you know if the sizes are so we should be able to if you know which models you are thinking about. Uh Uh for application. Not these at least they're in 4B Here here we have just focused on the Gemma because of the deep mind track but essentially our hugging face page has other open weight models and you are you can so some of them we provide for easy to use others you are welcome to like convert and use yourself as well. But yeah if there is a model that you've identified yeah then we can certainly discuss. Thank you. Yeah, thanks.