Sheilah Kirui (developer advocate at Akamai) explains speculative decoding in LLM inference: how a small draft model proposes tokens that a larger target model verifies in one forward pass, and when the technique is worth enabling. She covers prefill vs decode phases, the memory cost of hosting a second model and its KV cache, how to choose a draft model (size, shared tokenizer, cost, accuracy), and demos both setups side by side with vLLM on a single NVIDIA Blackwell GPU — structured output ran 1.6x faster with a high acceptance rate, while creative writing saw far lower acceptance. She also notes why long-context and high-concurrency workloads gain less. Recorded at the AI Engineer World's Fair 2026 in San Francisco. Topics: prefill and decode phases; choosing a draft model; demo of acceptance rates on structured vs creative tasks; when to use it (spare VRAM, small batches, structured output, short context). # Channel Presentation Observations: Akamai Speculative Decoding Talk > **Scope note:** This document covers one video. Observations describe what this video shows or states, not conclusions about the channel as a whole. No curated metadata was supplied, so the analysis relies on the title and transcript only. --- ## 1. Explicit Self-Description (Creator Claims) - **Speaker identity and affiliation:** The speaker introduces herself as a DA at Akamai. > "My name is Sheila. I'm a DA at Akami and we're here today..." The transcript spells the company "Akami." The title spells it "Akamai," so the spelling is most likely a transcription error. - **Speaker's area of focus:** The speaker describes her work as inference optimization, and says other colleagues cover other areas. > "I'm focusing on in inference optimizations. Uh we have other people that are focusing on building AI agents..." - **Channel or content-organization claims:** The speaker points viewers to Akamai resources. > "...you can find it on our website, our GitHub, GitHub. Akami developers." > "...we have a Discord channel that we're trying to build up now." - **Stated future plans:** The speaker says she may create more content, but this is a personal intention, not a confirmed channel commitment. > "I'm also trying to maybe like make more content around this, like maybe like videos... I would definitely love to maybe like publish a blog or make snippet videos." --- ## 2. Intended Audience (Demonstrated) The audience is inferred from the content and the way it is delivered. - **Technical practitioners deploying LLMs:** The talk assumes familiarity with terms such as KV cache, forward passes, tokenizers, VRAM, and concurrency. > "...the model would read your prompt, process all the input tokens and then build what is known as a KV cache..." - **Engineers deciding whether to adopt a技术 optimization:** The talk frames the material as an evaluation framework. > "...it's worth considering but uh we'll talk about how to evaluate whether it's worth..." - **Conference or meetup attendees:** The format includes a live demo, audience questions, and a closing request to talk "on the side." > "...if anyone has questions, I'll take one or two questions now." --- ## 3. Claimed and Demonstrated Scope ### Topics Covered - **Speculative decoding mechanics:** Draft and target model roles, draft token counts (described as "typically... between three to five"), and verification in a single forward pass. - **Cost and resource tradeoffs:** Extra memory for a second model and its KV cache, and the need for spare GPU capacity. - **Draft model selection criteria:** Size ("typically 10 to 50 times smaller"), shared tokenizer, same model family, and the cost/accuracy balance. - **Workload suitability:** Structured tasks (JSON, SQL, coding) versus creative tasks (poetry, brainstorming), and long-context or RAG workloads. ### Tools and Setup - **Serving engine:** The speaker names vLLM as the engine she used. - **Hardware:** A single Blackwell GPU, shared between a baseline server and a speculative-decoding server. - **Model sizes (as stated):** Baseline weights of "about 16 gigabytes" and a draft model of "2.5" (units implied as GB). The transcript does not name the specific models in the demo. ### Demonstrated Results (as presented) - **Structured-output demo:** The speaker reports the speculative setup was "1.6 times faster" with a high acceptance rate. - **Creative-output demo:** The speaker reports a low acceptance rate, which she attributes to high temperature and a creative task. > The speaker's performance numbers come from a single live demo. The transcript does not show methodology, repeat runs, or the full data. --- ## 4. Evidence Quality and Limits | Observation | Evidence Type | Strength | |---|---|---| | Speaker is an Akamai DA | Direct self-statement | Strong | | Focus on inference optimization | Direct self-statement | Strong | | Audience is LLM practitioners | Inferred from terminology and framing | Moderate | | Demo speedup of 1.6x | Live demo, speaker-reported | Limited (single run, no methodology) | | Plans for more videos or blogs | Stated intention | Weak (aspirational) | --- ## 5. Observations Not Supported by This Video - No evidence on publishing frequency, series structure, or recurring formats across the channel. - No evidence on the channel's broader audience size or reach. - No evidence on whether this talk represents Akamai's official positioning or only the speaker's view. # Delivery Observation Report — "Is Speculative Decoding Worth It? Profiling vLLM on NVIDIA Blackwell — Akamai" ## Tone and Overall Manner - **Informal, conversational presentation style.** The speaker addresses the audience directly and casually rather than formally: *"Okay, I'm a little anxious."* - **Frequent self-acknowledged nervousness.** She openly names her nervousness at the start and again near the end: *"I'm a little all over the place."* - **Spontaneous, live feel.** Delivery includes live troubleshooting and unscripted asides, e.g., *"Technical difficulties. I'm not sure why my demo is not running. Um, let's see."* and *"Whoa. Oh god, I'm just waiting."* ## Filler and Hesitation Patterns - **Very high density of "um" and "uh"** throughout, appearing in nearly every transcript segment (e.g., *"the model's working um memory um for the rest of the request"*). - **Recurring filler phrase "like"**, used both as hesitation and as an example-introducer: > "typically like this would require like even up to hundreds like forward passes" - **"Yeah" used as a transition/reset word** between points: *"Yeah, so typically when you're deciding whether it's worth enabling..."* and *"Yeah. So, um, this process it's great."* ## Teaching Habits and Structure - **Question-driven framing.** She repeatedly poses rhetorical questions to structure content: > "what if you didn't have to use your bigger model to handle the token generation. What if you use a smaller model to um do some of that guesswork?" And later: *"can the draft model predict as accurate as the target model? And then the other question is whether um it's as fast."* - **Concrete anchoring via analogies and plain-language definitions.** She glosses technical terms in accessible terms: the KV cache is *"essentially the model's working um memory"*; a small model does *"guesswork to speculate on a couple tokens."* - **Use of personal/demo framing.** She grounds abstract criteria in her own setup: *"For my situation, um I will explain why I chose these two models."* - **Explicit summarizing takeaways.** She closes sections with condensed decision questions: *"is it highly structured? Do you have enough VRAM space to even consider it? Are you doing more creative scenarios? Are you running a lot of concurrency um or not?"* ## Recurring Phrases | Phrase | Usage | |---|---| | **"like I said"** | Anaphoric callback device: *"like I said, it has to be smaller"*; *"And like I said, this is because..."* | | **"I'll just leave this here"** | Habitual slide-pause while audience reads: *"I'll just leave this here for a second."*; *"I'm just going to leave this here."* | | **"typically"** | Frequent hedged generalization marker: *"typically when you ask LLM a question..."*; *"typically you can set it between three to five"*; *"typically 10 to 50 times smaller"* | | **"as expected"** | Used to preface demo results: *"as expected with a speculative on the right side... it was able to generate much much faster"* | | **"the main point here is"** | Section-summarizing marker: *"I think the main point here is just how to think through like what is your application doing?"* | ## Self-Corrections and Live Recovery - **On-the-fly recall struggles, handled transparently:** > "Um, what's the other? Yeah, those are the main things. I think I'm forgetting one. Um, oh yeah, and like I said, the cost and accuracy." Also: *"by Google and what's the other company?"* - **Retrospective self-corrections:** *"oops oh yeah this is what I was I should have put this up here"*; *"Right. I should have covered that."*; *"I should have put a QR."* - **Repetition self-repair:** *"the number of tokens that were accepted over the total generated by the draft model"* — mid-sentence restart during metric definition. ## Demo Handling - **Live-demonstration-centric delivery.** The middle of the talk depends on running code in real time, with visible stalls: *"Looks like I need to restart the VLM servers and my setup. Give me just a few minutes."* - **Hedged expectations set beforehand:** *"this is a quick application. I'm not even sure if it's going to work."* - **Metric-by-metric narration of results** — she names acceptance rate, tokens-per-second, and throughput one at a time while pointing at outputs. ## Uncertainty Notes - Patterns such as **"um" frequency and "like I said" callbacks** are well-evidenced by many instances across the transcript. - The **anxiety/nervousness observations** rest on only two explicit statements (opening and closing), so the overall degree of nervousness beyond those moments is uncertain — hesitation markers may reflect live-demo context rather than sustained anxiety. - This appears to be a **conference or meetup talk with audience Q&A** (evidenced by *"if anyone has questions, I'll take one or two questions now"* and repeated unprompted answers to unheard questions), though the format itself is inferred from interaction markers, not stated. # Observation Record: "Is Speculative Decoding Worth It? Profiling vLLM on NVIDIA Blackwell" (Akamai) **Source:** Video `01a15ca6-e8d8-75a4-ac00-27560dd3f939` **Speaker:** Sheila, introducing herself as a DA at Akamai (self-identified in transcript) **Scope:** Positions expressed in the presentation and Q&A. Claims are the speaker's own and were not independently verified. --- ## 1. Core Mechanism ### 1.1 Two-phase inference - **Speaker's account:** When an LLM receives a prompt, it first reads and processes all input tokens, building a **KV cache** ("the model's working memory"). This happens once per request. - **Decoding phase:** The model writes the answer, typically **one token at a time**, because each token depends on previous tokens. ### 1.2 Speculative decoding as a response to decoding cost - **Stated problem:** Serving a large model such as Llama 70B can require "even up to hundreds" of forward passes, which is expensive and adds user-facing latency. - **Stated idea:** A **smaller draft model** speculates several tokens ahead. The **target model** verifies them in **one forward pass**. - **Stated mechanism per cycle:** - The draft model typically proposes **3 to 5 tokens** per cycle. - The target model approves or rejects each one. - Rejected tokens are **recomputed by the target model**. - **Stated guarantee:** "The output is still the same," because the target model verifies every token. --- ## 2. Costs and Resource Trade-offs - **Memory overhead:** Hosting two models requires memory for both. - The draft model adds relatively little weight memory ("not that much since it's a smaller model"). - Extra space is also needed for **KV cache for both models**. - **Conditions where it is worth considering:** - Available GPU headroom exists. - Workload is **low concurrency / smaller batch sizes**. - **Conditions where it likely is not worth it:** - High-concurrency workloads, where the speaker says "your GPUs are probably already busy." > **Qualification:** The speaker frames these as considerations, not universal thresholds. She says "we'll talk about how to evaluate whether it's worth," and the evaluation is presented as workload-specific. --- ## 3. Draft Model Selection The speaker identifies two primary decisions to balance: 1. **Accuracy:** Can the draft model predict nearly as accurately as the target? 2. **Speed:** Is the draft model fast enough, i.e., "a fraction of the cost"? Additional criteria she listed: | Criterion | Speaker's Statement | |---|---| | Size | Typically **10 to 50 times smaller** than the target | | Tokenizer | Must match; ideally from the **same model family** to avoid manual tokenizer translation | | Cost and accuracy | Mentioned, though she said she was "forgetting one" | --- ## 4. Workload Suitability The speaker argues the benefit depends on how predictable the output is. - **Likely to benefit: highly structured outputs** - Examples given: **coding**, **JSON**, **SQL**. - Rationale: the draft model can more often match the target's next tokens. - **Likely to benefit less: creative outputs** - Examples given: **poetry**, **brainstorming**. - Rationale: more variability in possible next tokens, leading to lower acceptance. ### 4.1 Context length - **Stated point:** Speculative decoding accelerates the **generation** phase, not the **prompt-reading (prefill)** phase. - **Implication:** Workloads with very large input context relative to output length (e.g., **RAG**, document analysis) may spend most of their time building the KV cache, limiting the benefit. --- ## 5. Demo Observations **Setup, as stated:** - Single **NVIDIA Blackwell** GPU. - Baseline and speculative configurations run **on the same GPU**, which required a small enough model. - Reported memory: baseline model weights about **16 GB**; draft model about **2.5 GB**. The speaker said this leaves substantial room for KV cache. - Implementation served via **vLLM**. ### 5.1 Structured task demo - **Observed result:** Speculative configuration generated **about 1.6 times faster**. - **Key metric highlighted:** **Acceptance rate**, defined as accepted tokens divided by total tokens generated by the draft model. The speaker described it as "how high" it was but did not state a numeric value in the transcript. - **Additional claim:** Higher throughput in tokens per second with speculation enabled. ### 5.2 Creative task demo - **Observed result:** Low acceptance rate, as expected. - **Speaker's explanation:** Greater variety in next-token possibilities, and the **temperature was set high** for a creative use case. - **Note:** No speed figure was reported for this case in the transcript. > **Caveat:** The speaker stated uncertainty before the first demo ("I'm not even sure if it's going to work") and experienced technical difficulties requiring a server restart. Results are from a single live run as described, with no reported repetitions or variance. --- ## 6. Q&A Points ### 6.1 Tools and resources - **Recommended serving engine:** vLLM, which she says has good getting-started documentation. - **Method complexity:** The draft-model approach is "the simpler one of the simple versions." - **Other methods mentioned:** Speaker named **"engram," "medusa," and "eagle"** as more advanced architectures. *(Transcript rendering is uncertain for some names.)* - **Reading suggestions:** - A blog that appears first when searching "speculative decoding," described as covering the topic in detail. *(Transcript rendering of the source name is unclear.)* - **Two research papers** published by Google and "what's the other company?", which the speaker could not name in the transcript. - **Stated scarcity:** The speaker said there is "not much content out there on this." ### 6.2 Speaker's stated plans - She would "love to maybe" publish a blog or make snippet videos. This is an aspiration, not a commitment. ### 6.3 Akamai resources (self-promotional context) - Pointed to Akamai's website and GitHub (under "Akamai developers") for replicating the setup. - Mentioned other Akamai teams on AI agents, managed Kubernetes, and Akamai Functions. - Invited viewers to a Discord channel "we're trying to build up now." --- ## 7. Unresolved Tensions and Open Points - **Acceptance rate values:** The key metric was emphasized but no numeric acceptance rate for either demo was given in the transcript. - **Generalizability:** Results are from one model pair (16 GB target, 2.5 GB draft) on one GPU. The speaker did not claim the 1.6x figure generalizes. - **Concurrency threshold:** "High concurrency" and "smaller batch sizes" are not quantified. - **Draft model specifics:** The 2.5 GB draft model is not named in the transcript. - **Tradeoff not quantified:** The speaker says speculation "is not free" and requires extra memory, but no figures are given for KV cache overhead on the second model. - **Speaker uncertainty:** The speaker is self-described as "a little all over the place" and expressed uncertainty about the demo, the tool names, and the research sources. - **Transcript artifacts:** Several terms (for example, "Akami," "recomputee," "engram," "general compute") appear as likely transcription errors and should be verified against primary sources before citation. # Speaker Attribution and Provenance Record **Video ID:** `01a15ca6-e8d8-75a4-ac00-27560dd3f939` **Title (as supplied):** *Is Speculative Decoding Worth It? Profiling vLLM on NVIDIA Blackwell — Akamai* --- ## 1. Speakers and Roles ### 1.1 Primary Speaker - **Name (as stated in transcript):** Sheila - **Stated affiliation:** "a DA at Akami" (transcript spelling; the title reads "Akamai") - **Role:** Presenter. Delivers the talk, runs the live demo, and answers audience questions. - **Identity basis:** Self-introduction in the opening line. No other curated metadata supplied. ### 1.2 Audience Questioner - **Identity:** Not identified in the transcript. - **Role:** Audience member who asked one question about recommended tools for speculative decoding. The question is transcribed only as relayed by the presenter ("the question was..."). ### 1.3 Unattributed Audio - **Content:** Brief inaudible or unclear exchange ("I can't hear you") during Q&A, plus a music cue at the end. - **Attribution:** Unknown. Not assignable to a named speaker. --- ## 2. Promotional Segments > **Flag: Promotional content present.** The following segments promote Akamai products, resources, and community channels. - **Closing promotion (near end of transcript):** - Directs viewers to the **Akamai website**. - Directs viewers to the **Akamai Developers GitHub**. - Mentions other Akamai teams building **AI agents**. - Mentions Akamai's **managed Kubernetes service**. - Mentions **Akamai Functions**. - Invites viewers to a **Discord channel** the presenter describes as being "built up now." - **Self-referential promotion:** The presenter describes her own intent to "publish a blog or make snippet videos," which is a stated future content plan, not a current offering. --- ## 3. Referenced Sources and Third-Party Claims ### 3.1 Tools and Software - **vLLM:** Serving engine used for the demo. The presenter states it has "really good documentation." - **Other methods named (not used in demo):** Ngram, Medusa, Eagle. Described as more architecturally advanced than the simple draft-model approach used. ### 3.2 Models Referenced - **Llama 70B:** Cited as an example of a large target model. - **Unnamed draft and target models:** The demo uses a smaller draft model and a larger target model. Specific model names are not stated in the transcript. ### 3.3 Written Sources - **Blog:** The presenter says a blog by "general compute" (as transcribed) is among the first results for "speculative decoding." Attribution spelling is uncertain due to transcription. - **Research papers:** Two papers are referenced as "published by Google" and by "what's the other company," which the presenter did not name. The second source is **unattributed**. ### 3.4 Third-Party Claims Relayed - Speculative decoding can yield "a lot" of savings in inference generation (presenter's assertion). - The presenter states that for a 70B-scale model, generation "can require up to hundreds" of forward passes (presenter's assertion, no cited source). - Guidance that the draft model is typically "10 to 50 times smaller" than the target model is stated without an attributed source. --- ## 4. First-Party Demo Provenance > **Note:** All demo results are the presenter's own live measurements on a single NVIDIA Blackwell GPU. They are not sourced from an external dataset or independently replicated in this transcript. - **Hardware:** One Blackwell GPU, hosting both baseline and speculative-decoding servers concurrently. - **Memory figures stated:** Baseline weights about 16 GB; draft model about 2.5 GB. - **Reported result:** A speedup described as "1.6 times faster" for the structured-output task, with acceptance rate and tokens per second noted on screen. - **Second task:** Creative (high temperature) task, with the presenter pointing to a low acceptance rate. - **Interruption:** The demo experienced "technical difficulties." The presenter restarted the vLLM servers and reran the demo. The rerun is the one presented. --- ## 5. Availability Limits - **Comments:** No comment data was supplied with this record. Comment availability **cannot be determined** from the provided material. - **Curated metadata:** Only the title was supplied. No speaker bio, description, chapter markers, or cited-source list accompanies the transcript. - **Slides:** The presenter refers to slides ("I'll just leave this here," "this slide here") that are not included in the transcript. Any visual claims on slides are **unavailable for verification**. - **Speaker name spelling:** "Akami" in the transcript versus "Akamai" in the title. The record uses the title spelling for the organization. --- ## 6. Summary of Provenance Status | Channel | Status | |---|---| | Speaker identity | Stated in transcript (first name only) | | Speaker affiliation | Stated in transcript; spelling variance noted | | Audience questioner | Unidentified | | Promotional segments | Flagged (closing Akamai resources and Discord) | | Named sources | vLLM, Llama 70B, Ngram, Medusa, Eagle, one blog, two papers (one unnamed) | | Demo results | Presenter first-party, live, single run after restart | | Comments | Unavailable | | Slides | Referenced, not provided | My name is Sheila. I'm a DA at Akami and we're here today to discuss um the topic speculative decoding and how you can um figure out whether it's worth enabling for your workloads. Okay, I'm a little anxious. Okay. So, typically when you ask LLM a question, there are usually two phases that happen under the hood. The first is the model would read your prompt, process all the input tokens and then build what is known as a KV cache um which is essentially the model's working um memory um for the rest of the request. And this happens only once. Um and then you have the decoding phase which is uh when the model starts to actually write the answer. Um so because every token depends on the previous tokens um this process usually happens one at a time. Um and so imagine you're deploying an application that uses a big model like llama 70 billion model. Um typically like this would require like even up to hundreds like forward passes on the model and this is a very expensive process right um and it can cause a lot of latency for your user applications which um brings the question of like what if you didn't have to use your bigger model to handle the token generation. What if you use a smaller model to um do some of that guesswork to speculate on a couple tokens? Um which is still an auto reggressive process, but the whole point is because it's smaller, there's it's a faster process, so it would be a cheaper process. Um the output is still the same because um the target model, the main model that has the accuracy still has to verify the tokens and it does this in one forward pass. Um so that's the idea behind speculative decoding. Um I'll just leave this here for a second. So as I said you have a small model um which generates token typically you can set it between three to five for each cycle. The target model would handle the predictions um like be able to scale like basically approve or reject it. Um and then for all the for the tokens that are rejected the target model would um recomputee the the token for that. Okay. So, um this process it's great. It can save you a lot in terms of inference generation, but it's not free. Um because you're hosting now two models, you have to account for the memory required to host that second model. Not that much since it's a smaller model, but you also have to allocate extra space for um the KV cache for both models. Um yeah so typically you would consider running speculative decoding when you have um extra space in GPU um for example if your workload is high concurrency your your GPUs are probably already busy trying to run all the requests so it wouldn't make sense to implement speculative decoding but if you have extra GPU space it's worth considering but uh we'll talk about how to evaluate whether it's worth um and the first question is um choosing the draft model. Um yeah, so typically when you're deciding whether it's worth enabling, um can the draft model predict as accurate as the target model? And then the other question is whether um it's as fast. So you would typically choose a model that's definitely like a fraction of the cost like um yeah, so these are the two main um decisions that you have to balance, right? Um, I'm just going to leave this here. Um, so for my situation, um, I will explain why I chose these two models. For my demo setup, when I was running this, I have access to one GPU. I was running on a Blackwell GPU and I wanted to run both the baseline and the speculative decoding setup in one. Um so to be able to do this I had to choose a model that was small enough to be able to fit because I have to split the GPU utilization between the both servers. Um so this is what that looked like for instance you can the baseline alone um takes about 16 gigabytes of weight and then the draft model is 2.5. So that leaves a huge amount and space for KB caching. Um, and then when you're thinking about choosing the model, some of the things that you would usually consider is um, first of all, like I said, it has to be smaller, typically 10 to 50 times smaller than your target model. Um, it has to have the same tokenizer. Um, ideally from the same model family, otherwise you know, you would have to like manually translate between the different tokenizers. Um, what's the other? Yeah, those are the main things. I think I'm forgetting one. Um, oh yeah, and like I said, the cost and accuracy. Uh, okay. So, oops. So when I was running this, there's three questions. Um, speculative decoding, there's some work loads that would benefit from it versus others that wouldn't. Um, typically because the goal here is to get a smaller model that can predict almost as accurately as the target model would. It helps with some use cases when you have a workload that is more like highly structured like coding use cases. Um, what's another one? Like writing JSON or SQL prompts. Um, it would probably like benefit more versus if you were using it for a highly creative use case like writing poetry or brainstorming or anything like that. Yeah. So, I think I have this is a quick application. I'm not even sure if it's going to work, but um so here I do have for the first demo, I just have um two tabs here. Um like I said, this is just to focus on looking at how the acceptance rate performs depending on the type of task that you have. Um so the first tab here is a structured output. Um when I click this, it will run both at the same time. Whoa. Oh god, I'm just waiting. Technical difficulties. I'm not sure why my demo is not running. Um, let's see. Oh, do I have to reconnect? Um, just give me one second. I think I might have to restart the Looks like I need to restart the VLM servers and my setup. Give me just a few minutes. Is this on? LSU I Oh. Um, should I Yeah, I think I need Oh, it's showing up. Okay, there we go. Um, so yeah, I was saying this is a demo. So when I run this, I'll redo it. So you can see um as expected with a speculative on the right side um it was able to generate much much faster um 1.6 times faster. Um the key thing to note here is how high the acceptance rate is which is the rate of the tokens that were accepted by the the number of tokens that were accepted over the total generated by the draft model. Um another metric here um you can see with the speculative enabled um you we are able to generate a lot more tokens per second um which improves the throughput. And then if I run it for the other case. So here it's another use case. Um as expected same thing. Um but in this case I guess the main metric to see here is the low acceptance rate. And like I said, this is because this is a more um there's more variety in how the next tokens that would be generated because first of all, the temperature is set high. It's a creative use case. Um yeah. Um what else? I think that's pretty much for this um case. And then the other situation where oops oh yeah this is what I was I should have put this up here um just to explain when it would benefit and then the other use case would be context length. Um so as I mentioned earlier there's two parts of generating tokens. Um with speculative decoding it's supposed to accelerate the generating part not the part where the model reads the prompt. So typically if you have a use case where you need to provide the model with a lot of context like for rag applications or um analyzing lots of documents um with those situations um the model will spend a lot more time with the portion of like building the KV caching um especially if you're using it to do like a quick where the the generated tokens are much less than the context like the input um Yeah. So ba I think the main point here is just how to think through like what is your application doing? Um is it highly structured? Do you have enough VRAM space to even consider it? Are you doing more creative scenarios? Are you running a lot of concurrency um or not? Like smaller batch sizes would benefit from this. Um yeah. Yeah, I'm gonna leave this slide here. This just basically summarizes uh what I just went through. And if anyone has questions, I'll take one or two questions now. \>\> Oh, I can't hear you. \>\> Oh, yeah. Um the question was are there particular tools that I recommend for speculative decoding? So I implemented this with VLM um the serving engine. Um they have really good documentation on how to get started with it. This is the simpler one of the simple versions of speculative decoding. There's a couple others like the engram medusa eagle that's more like the architecture beneath is like more advanced. Um, but for getting started, yeah, I would say you can't go wrong with just reading up I when I was doing this, there's there's not much content out there on this, but I did find a few like blogs. Uh, when you re when you type in circulative decoding, um, one of the first blogs that will come up by general compute talks about more in detail. Um, there's also like two research papers that were published um, by Google and what's the other company? Yeah, it but that's more like goes like more in depth. Yeah. Um, but I'm also trying to maybe like make more content around this, like maybe like videos. It was kind of hard to go through this. I I'm a little all over the place, but I would definitely love to maybe like publish a blog or make snippet videos. Um, yeah, on like YouTube or somewhere, but you can definitely find the resources. \>\> Okay. Right. I should have covered that. Um, but I don't have the I should have put a QR. Oh, um, as I said, I I'm from Akamay. And if you're interested in like learning how to replicate this, you can find it on our website, our GitHub, GitHub. Um, Akami developers. Yeah. Um, yeah. So we have a a lot of like different resources on here. Um I'm focusing on in inference optimizations. Uh we have other people that are focusing on building AI agents um and how to leverage our like manage Kubernetes service, how to use Akami functions. So if you have time, please go check out our GitHub pages and also we have a Discord channel that we're trying to build up now. Um so it would be great to chat with some of you on the side. Thank you. \>\> \[music\]