--- schema: 1 type: video title: 'Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google' description: >- The constraint on edge AI is not compute, it is RAM, and it is getting worse: phone makers are shipping less of it this year, and a 6GB Raspberry Pi costs 2.5 times what it did at launch. So Cormac Brick's team at Google AI Edge spends its effort making models small enough to fit. A 2 billion parameter Gemma, quantized to 2.9 bits per weight, runs on a Raspberry Pi at about 8 tokens per second and on a Qualcomm NPU fast enough for a few frames of vision a second. Below that sit tiny models, from 500 million parameters down to 50, that reach the older laptops and cheap devices where even a small model will not fit. They usually need fine tuning rather than prompting, but the payoff is real: a fine tuned Gemma turns free text into the right function call across ten actions at over 86% reliability, and putting a speech model in front gives you voice to function calling. One shipped example is an offline voice dictation app with no subscription, built on two sub billion Gemma models that also strip your ums and ahs. Speaker info: - https://x.com/cormacb - https://www.linkedin.com/in/cbrick/ - https://github.com/google-ai-edge/gallery Timestamps: 0:00 - Why intelligence at scale needs tiny models 1:17 - The Google AI Edge team and its open source stack 2:35 - Why run on the edge at all 3:25 - The real constraint: DRAM cost 4:40 - Small models: 1 to 4 billion parameters 6:08 - Shrinking Gemma to 2.9 bits per weight 7:36 - Decode speeds across Raspberry Pi, Jetson, and NPUs 9:30 - Try it yourself: AI Edge Gallery and a hobby robot 12:07 - When small is still too big: tiny models 13:24 - Off the shelf tiny models: ASR, vision, embeddings 14:28 - Fine tuning for voice to function calling 17:50 - In production: offline voice dictation 19:30 - Takeaways and Q&A --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/hacEQHHhu2Q-why-large-tiny-lms-agents-on-edge-robotics Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/why-large-tiny-lms-agents-on-edge-robotics-cormac-b-569f75ddc99b markdown: >- https://liquidgold.cc/channel/aidotengineer/video/why-large-tiny-lms-agents-on-edge-robotics-cormac-b-569f75ddc99b.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T06:35:47.023Z' language: null children: [] videoId: hacEQHHhu2Q durationSeconds: 1305 publishedAt: '2026-07-25T17:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 158 endSeconds: 169 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=158' heldReason: rights - ordinal: 1 text: null startSeconds: 212 endSeconds: 224 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=212' heldReason: rights - ordinal: 2 text: null startSeconds: 289 endSeconds: 302 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=289' heldReason: rights - ordinal: 3 text: null startSeconds: 378 endSeconds: 466 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=378' heldReason: rights - ordinal: 4 text: null startSeconds: 466 endSeconds: 476 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=466' heldReason: rights - ordinal: 5 text: null startSeconds: 476 endSeconds: 575 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=476' heldReason: rights - ordinal: 6 text: null startSeconds: 575 endSeconds: 586 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=575' heldReason: rights - ordinal: 7 text: null startSeconds: 748 endSeconds: 804 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=748' heldReason: rights - ordinal: 8 text: null startSeconds: 804 endSeconds: 814 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=804' heldReason: rights - ordinal: 9 text: null startSeconds: 879 endSeconds: 1073 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=879' heldReason: rights - ordinal: 10 text: null startSeconds: 879 endSeconds: 1073 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=879' heldReason: rights - ordinal: 11 text: null startSeconds: 1073 endSeconds: 1083 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=1073' heldReason: rights - ordinal: 12 text: null startSeconds: 1083 endSeconds: 1177 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=1083' heldReason: rights - ordinal: 13 text: null startSeconds: 1177 endSeconds: 1305 watch: 'https://liquidgold.cc/watch/hacEQHHhu2Q?t=1177' heldReason: rights --- # Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google The constraint on edge AI is not compute, it is RAM, and it is getting worse: phone makers are shipping less of it this year, and a 6GB Raspberry Pi costs 2.5 times what it did at launch. So Cormac Brick's team at Google AI Edge spends its effort making models small enough to fit. A 2 billion parameter Gemma, quantized to 2.9 bits per weight, runs on a Raspberry Pi at about 8 tokens per second and on a Qualcomm NPU fast enough for a few frames of vision a second. Below that sit tiny models, from 500 million parameters down to 50, that reach the older laptops and cheap devices where even a small model will not fit. They usually need fine tuning rather than prompting, but the payoff is real: a fine tuned Gemma turns free text into the right function call across ten actions at over 86% reliability, and putting a speech model in front gives you voice to function calling. One shipped example is an offline voice dictation app with no subscription, built on two sub billion Gemma models that also strip your ums and ahs. Speaker info: - https://x.com/cormacb - https://www.linkedin.com/in/cbrick/ - https://github.com/google-ai-edge/gallery Timestamps: 0:00 - Why intelligence at scale needs tiny models 1:17 - The Google AI Edge team and its open source stack 2:35 - Why run on the edge at all 3:25 - The real constraint: DRAM cost 4:40 - Small models: 1 to 4 billion parameters 6:08 - Shrinking Gemma to 2.9 bits per weight 7:36 - Decode speeds across Raspberry Pi, Jetson, and NPUs 9:30 - Try it yourself: AI Edge Gallery and a hobby robot 12:07 - When small is still too big: tiny models 13:24 - Off the shelf tiny models: ASR, vision, embeddings 14:28 - Fine tuning for voice to function calling 17:50 - In production: offline voice dictation 19:30 - Takeaways and Q\&A --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/hacEQHHhu2Q-why-large-tiny-lms-agents-on-edge-robotics Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/why-large-tiny-lms-agents-on-edge-robotics-cormac-b-569f75ddc99b · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.