Talk from AI Engineer World's Fair 2026 by Jetashree Ravi (Tech Lead Manager, Applied ML, Fireworks AI) on moving from closed AI models to open ones through continuous training and deployment. She argues open models are production-ready, that customization closes remaining quality gaps, and that the AI stack is shifting toward a continuous train-and-serve loop. Topics include: the FireAttention inference engine with custom CUDA kernels; speculative decoding with custom draft models; three training approaches (training agent, managed training, and a training API covering SFT, RFT, and DPO); and a data flywheel with one-click deploys, monitoring, hot-loading new weights without downtime, and matching numerics between training and serving. Includes customer examples (Cursor, Vercel, Genspark) and demos of the training agent and managed training. # Video Identity Observations: Fireworks AI — "Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI" ## Source - **Title:** *Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI — Fireworks AI* - **Presenter:** Self-identified as "Jatashri... feel free to call me Jet," who "work[s] at fireworks" and "lead[s] part of the applied machine learning team." - **Format:** A conference-style talk with live product demos, ending with Q&A time ("happy to take them in the next minute or so"). ## Intended Audience **AI practitioners considering a move from closed-source to open-source models.** Evidence: - Opens with a live audience poll: *"How many of you actually today use a closed source model like chat GPD or Anthropic?"* followed by *"How many of you feel like it's getting really expensive really quickly?"* - Frames the talk around *"users... switching from closed source to open source models for cost reasons, performance reasons."* **Audience is segmented by technical level, mapped to product tiers:** - **Training Agent** — described as *"very popular among our product managers and builders that got into this AI inferencing and training field very recently."* - **Managed Training** — described as *"one of our most popular flavors among our customer and user base,"* accessible via API or UI. - **Training API** — *"very popular among some of the ML engineers within... our customer base,"* for those who want to *"define their own training loop and loss functions."* ## Claimed Scope **Vendor positioning:** This is a product presentation for Fireworks AI, explicitly positioned as the platform solving the closed-to-open migration: > *"That's where fireworks kind of comes in. It makes the whole process... of training these open source models as well as hosting these open source models at scale much more easier and also much more cost effective."* **Claimed scale and traction (creator claims):** - *"We serve about 30 trillion tokens"* daily in production, estimated to have grown *"by about at least 5 trillion tokens a day"* post-GLM 5.2 release. - *"We support more than 10,000 companies globally and process about 200k requests a particular second."* - Funding/team: *"we have raised around 327 million"* and *"are about 200 people working at fireworks."* **Named customer claims:** - **Cursor:** Composer 2 *"trained and also hosted on our platform,"* achieving *"13x faster inference"* via the "Fire Attention" engine. - **Vercel:** fine-tuned and hosted, *"40x latency improvement."* - **GenSpark:** RFT engagement yielding *"12% quality improvement."* **Claimed differentiation:** - Custom **CUDA kernels** / Fire Attention engine tuned per customer workload. - **Hot loading:** swapping new checkpoints into live deployments *"without any downtime."* - **Numerical fidelity:** *"We offer training and inferencing on the same platform"* to avoid quality issues from cross-platform train/host splits. - An end-to-end **"data flywheel"** of train → deploy → monitor → collect feedback → retrain. ## Observable Presentation Patterns (demonstrated, not claimed) - **Live product demos** of the training agent and managed training UI, showing dataset upload, model selection (e.g., *"quen 38B"*), hyperparameter handling, loss curves, cloning training runs, and one-click deployment templates (*"throughput, fast, minimal"*). - **Honest limitation statements**, which lend credibility: *"Unfortunately the training agent itself does not support RFT,"* and no training API demo was available because *"the setup itself differs from customer to customer."* - **Interactive, practitioner-oriented tone:** polls, hands-on walkthroughs, references to Weights & Biases integration and cookbooks on the website. ## Claim vs. Demonstration Distinction - **Claims (creator-stated):** all scale figures, customer outcomes (13x, 40x, 12%), and funding/team numbers are presenter assertions, not independently verified in the material. - **Demonstrated:** the UI walkthroughs concretely show the training agent's plan-approve-execute flow and the managed training's create-and-deploy workflow, substantiating the claimed ease-of-use for at least the first two product tiers. # Delivery Observation Report **Video:** *Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI — Fireworks AI* **Speaker:** Introduces herself as "Jatashri... but feel free to call me Jet" — leads part of the applied ML team at Fireworks. > **Scope note:** Evidence is drawn from a single ~19-minute talk. Patterns flagged below as stable-looking are those repeated across the transcript; isolated moments are labeled as such. --- ## 1. Tone - **Conversational and audience-involved from the start.** The talk opens with live audience polls rather than a formal preamble: > "How many of you actually today use a closed source model like chat GPD or Anthropic?" > "How many of you feel like it's getting really expensive really quickly?" - **Peer-framing ("just like all of you").** She positions herself alongside the audience's journey: > "just like all of you might be thinking of switching from closed source model to open source models... other customers and users that are going through the same journey." - **Casual, lightly hedged delivery.** Frequent fillers and softeners ("uh," "I just wanted to," "kind of") appear throughout, e.g. "I just wanted to show you a quick demo," "the way this flywheel works is kind of very simple." - **Warm, appreciative closes.** Ends with: > "Thank you very much. And if you have any questions, I'm happy to take them in the next minute or so that we have." ## 2. Vocabulary and Recurring Phrases - **"Customer and user base"** — the most persistent collocation, appearing repeatedly: > "among our customers and user base" ([161]), "among our customer and user base" ([225], [494]), "among all our other customers and user base" ([300]) - **"Closed source" / "open source" as the talk's central axis** — invoked in nearly every section: the opening polls, the "three fundamental truths," the quality-gap framing, and the Gen Spark case study. - **Self-flagging structure:** "That brings me to the next portion of it," "before we get into that," "I'll get into how we actually do the training... towards the end of the presentation." - **Simplicity claims as a persuasion device:** "That's how simple we make it," "the way we do it is quite simple at a at a higher level," "you're just one click away from deploying." - **Numbers-heavy delivery:** "30 trillion tokens," "10,000 companies," "200k requests a particular second," "13x faster inference," "40x latency improvement," "12% quality improvement," "327 million," "about 200 people." ## 3. Teaching Habits - **Enumerated frameworks.** She explicitly numbers key points: > "the three fundamental truths that are true uh are one open source models for the most part are production ready... Two, if open source models don't... beat or match the quality... And three, the AI stack itself..." Training products are likewise presented as "three different flavors" (training agent, managed training, training API). - **Delayed-detail promises.** She repeatedly defers explanations to manage scope: "I'll speak a little bit about that in the next slide as well," "I'll get into how we actually do the training itself... towards the end." - **Concrete customer stories as evidence.** Cursor, Vercel, and Gen Spark each anchor an abstract claim with a quantified outcome (13x, 40x, 12%). - **Step-by-step demo narration.** The demo walkthrough is procedural and inclusive of audience prior knowledge: > "I know people have the train and eval or a test data set separately." - **Anticipating objections/limits.** She volunteers product limitations unprompted: > "Unfortunately the training agent itself does not support RF [RFT]." And: "I unfortunately don't have an entire demo for our training API because the setup itself differs from customer to customer." - **Empathy for practitioner pain points:** "I know one challenge a lot of people have is observability," "it's not always easy to get the right uh hyperparameters." ## 4. Communication Patterns - **Structured loop-backs.** She explicitly reconnects later material to earlier slides: > "I know in the first slide I was talking to you about the realities of AI today..." > "customers like Genpark that I was talking to you about in the previous slide." - **Persuasion via scale, then via simplification.** Each product section follows the pattern: large number → named customer → "it's simple" mechanism explanation. - **Sponsor-context transparency.** She discloses company funding and headcount mid-talk: > "we have raised around 327 million... and we are about 200 people working at fireworks." ## 5. Isolated Moments (low-confidence as patterns) - **The opening poll technique** — occurs only in the first ~30 seconds; whether interactive polling is a habitual style cannot be confirmed from one talk. - **The nickname offer** ("feel free to call me Jet") — a single moment; not enough evidence to characterize as a standard rapport-building habit. - **Self-corrections mid-sentence** (e.g., "RFD" vs. "RFT," "genpark lovables") appear several times but are likely transcript/speech artifacts rather than a deliberate delivery choice. --- **Uncertainty statement:** All observations are based on one presentation. Recurrence counts reflect within-talk repetition only; cross-video stability of these patterns is unverified. # Video Observation Record ## Video Metadata - **Title:** Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI — Fireworks AI - **Speaker:** Jatashri (prefers "Jet"), who states she works at Fireworks and "lead[s] part of the applied machine learning team." Speaker identification is based solely on her self-introduction in the transcript. --- ## Speaker's Framing and Audience Polls The talk opens with two live audience questions, presented as a framing device: - How many attendees use **closed source models** (she cites "chat GPD or Anthropic"). - How many feel it's "getting really expensive really quickly?" — she characterizes the response as "a good percentage of you all." She frames the talk as addressing users switching from closed to open source models **"for cost reasons, performance reasons,"** with the stated goal of explaining "where these open source models shine and how to make the performance or quality of these open source models match the closed source models itself." --- ## Claim 1: Three "Fundamental Truths" of the Current AI Ecosystem > "the three fundamental truths that are true uh are one open source models for the most part are production ready" - **Truth 1:** Open source models "for the most part are production ready." She adds a qualification: *"If any of you have used GLM 5.2... you know that that is true especially for the coding use cases"* — i.e., the claim is illustrated specifically by coding use cases with GLM 5.2. - **Truth 2:** If open source models don't "beat or match" closed source quality, the differentiator is "how you customize, train and improve these open source models." - **Truth 3:** The AI stack is shifting "from let's just serve a model once and that's done to a more continuous loop" of training-to-inference improvement on the same open source model. She then positions Fireworks as making "the continuous journey of training these open source models as well as hosting these open source models at scale much more easier and also much more cost effective." --- ## Claim 2: Fireworks Inference Scale and Customer Results These are presented as company/platform figures and customer outcomes as stated by the speaker: | Claim | Details / Qualifications | |---|---| | ~30 trillion tokens served | "in production at scale"; she notes this figure "was actually preGLM 5.2 release" | | Estimated growth | "I'm sure this has pretty much kind of increased by about at least 5 trillion tokens a day" — this is her **speculative estimate**, marked by "I'm sure" and "kind of" | | 10,000+ companies globally, ~200k requests/second | Stated without further qualification | | Cursor (Composer 2) | "trained and also hosted on our platform"; **13x faster inference** attributed to the FireAttention engine | | Vercel | Fine-tuned and hosted on the platform; **40x latency improvement** | | Gen Spark | Quality was "a big bottleneck or was a big bottleneck" switching from closed to open source; Fireworks helped with fine-tuning, RFT, dataset cleaning, and evaluator function definition, yielding a **12% quality improvement** "at the end of the RFT training job itself" | | Company funding/size | "raised around 327 million CDC uh a while ago" and "about 200 people" — note: "327 million CDC" appears to be a transcription artifact; the currency/unit is stated as-is in the transcript | --- ## Claim 3: FireAttention Inference Engine - Fireworks builds **its own CUDA kernels**, described as "a translation layer" between model and GPUs. - Workflow claim: customers come with "a workload and a performance target" (e.g., x input tokens, y output tokens, z prompt-cache hit rate, latency targets), and Fireworks optimizes the whole inference stack — including the CUDA kernel — "so that we can serve this model uh at a cheaper price but also at a that can meet their performance target and for their specific workload." - Implemented techniques: **speculative decoding, disaggregation, quantization**. - Customers can also **train their own draft models** on the platform (dataset upload similar to fine-tuning), which she says helps the main model "perform much faster uh meet latency requirements also gives you better throughput **if** the draft model is trained uh very well" — a stated conditional. --- ## Claim 4: Three Flavors of Training Product Introduced with the framing: *"when customers are switching from closed source to open source models the biggest bottleneck or the gap is quality."* ### 1. Training Agent - "very popular among our product managers and builders that got into this AI inferencing and training field very recently." - Accessed via UI; user links a dataset and a task description; agent selects model, cleans data, tunes hyperparameters. ### 2. Managed Training - "one of our most popular flavors"; "what most of our customers use today." - Via API or UI: user inputs hyperparameters, chooses model, uploads dataset, "clicking a button." - Observability: UI dashboard showing per-epoch learning and loss; optional export to **Weights & Biases** via API key. ### 3. Training API - "very popular among some uh of the ML engineers." - Lets ML engineers "define their own training loop and loss functions" — described as going "well above beyond the managed training service." **Cross-cutting qualifications:** - All three flavors support **supervised fine-tuning (SFT), reinforcement fine-tuning (RFT), and DPO**. - She observes "RFT has become extremely extremely popular." - Explicit limitation: *"the training agent itself does not support RF [RFT]"* — it does not clean data or tune hyperparameters for RFT, though she offers manual help "if you have a big use case in mind." --- ## Claim 5: The "Data Flywheel" Described loop: train → one-click deploy → monitor (tokens, cache hit rate, speculator performance) → gather feedback → retrain from a prior checkpoint to a new one on the same platform. **Enabling capabilities as stated:** - **GPU capacity** via partnerships "with different cloud and neocloud providers." - **Hot loading** — swapping new checkpoint weights into a live deployment "without any downtime." She notes this was "a feature we implemented specifically for cursor and then we thought would be useful for general public." - **Numerical fidelity** — the claim that using platform A for training and platform B for hosting causes quality issues because "humix [hardware micro-architecture, per transcript] is not always compatible"; Fireworks offers training + inference on the same platform to "ensure that uh you won't find any numerical differences." - **One-click deployment** resulting from training and inference co-location. --- ## Demo Walkthroughs (as narrated) ### Training Agent Demo - User provides training dataset name, eval/test dataset, and a fine-tuning request; agent then: - Details an SFT plan and **why it selected the base model** (in the demo: **Qwen 3 8B** — "quen 38B" in transcript). - Presents hyperparameters and a high-level training process outline. - After user approval, the agent executes steps with progress updates, then delivers a final summary (config, base model, hyperparameter impacts), **accuracy and F1 metrics**, and "recommended next step." ### Managed Training Demo - Upload dataset → choose model → optionally set evaluator dataset and fine-grained hyperparameters → click **"create"** → training job starts. - Completed job view shows dataset used, hyperparameters, loss curve; jobs can be **cloned and reproduced** with different settings. - Deployment options: custom configurations, or templates labeled **throughput, fast, minimal** ("a template that works well for generic workloads... optimized based on throughput latency uh time to first token and other such metrics"). ### Training API - She explicitly acknowledges: *"I unfortunately don't have an entire demo for our training API because the setup itself differs from customer to customer."* She points instead to cookbooks on the company website. --- ## Notable Tensions and Qualifications Preserved - The **token-volume growth figure** ("at least 5 trillion tokens a day") is the speaker's own inference, not a reported metric. - The **speculative decoding throughput benefit is conditional** on the draft model being trained well. - The **RFT limitation of the Training Agent** is stated plainly as an unsupported capability. - The **funding figure** ("327 million CDC") contains a transcription artifact and is recorded as expressed. - Gen Spark's quality bottleneck is described in both present and past tense by the speaker ("is a big bottleneck or was a big bottleneck"). # Video Observation Record ## Video Identification - **Title:** Stop Renting Intelligence: The Train-to-Deploy Loop for Specialized AI — Fireworks AI - **Video ID:** 01a15ca6-a044-73f4-8f58-efe57276cbe8 ## Speakers & Roles | Speaker | Role | Basis | |---|---|---| | **Jatashri ("Jet")** | Creator/presenter; leads part of the applied machine learning team at **Fireworks AI** | Stated in transcript: "I work at fireworks. I lead part of the applied machine learning team. My name is Jatashri but feel free to call me Jet" | - Single-speaker talk with a live audience (she polls the audience at the start and offers Q&A at the end: "if you have any questions, I'm happy to take them in the next minute or so"). - No guests, narrators, or other identified speakers appear. ## Content Type - **Promotional/product presentation.** The entire talk is a vendor presentation about Fireworks AI's inference and training products. This is explicitly a company-presenter format, not independent analysis — flag as **promotional segment (full video)**. ## Referenced Sources & Third-Party Claims ### Company claims (self-reported by presenter, not independently verified) - Fireworks serves ~**30 trillion tokens** in production (pre-GLM 5.2 release), with an estimated increase of at least 5 trillion tokens/day after the release. - Supports **10,000+ companies globally**; processes ~**200k requests/second**. - Raised ~**$327M**; ~**200 employees**. ### Named customers and attributed outcomes - **Cursor:** Composer 2 model trained and hosted on Fireworks; claimed **13x faster inference** via the "Fire Attention" engine. Hot-loading feature was built for Cursor. - **Vercel:** Fine-tuned and hosted on Fireworks; claimed **40x latency improvement**. - **GenSpark:** RFT support including dataset cleaning, evaluator function definition, hyperparameter selection; claimed **12% quality improvement**. ### Third-party tools referenced - Closed-source models: ChatGPT, Anthropic. - Open-source models: **GLM 5.2**, **Qwen 3 8B** (used in demo). - **Weights & Biases** — optional observability export via API key. - Cloud and neocloud providers as compute partners (unnamed). - Techniques cited: speculative decoding, disaggregation, quantization, DPO, SFT, RFT. ## Availability & Access Facts - Training API demo not shown; presenter cites **cookbooks on the company website** for custom training loops/loss functions. - Training agent product limitation stated: does not support RFT (dataset cleaning/hyperparameter tuning for RFT not automated). - Live audience Q&A offered at the end of the session. ## Speaker Attribution Notes - All claims above are attributed to **Jatashri (Jet)** speaking on behalf of Fireworks AI; customer outcome figures are her statements, not sourced documents. ## Evidence Gaps - No curated metadata supplied beyond the title; speaker identity rests solely on self-introduction in the transcript. - No timestamps, external links, or cited documents provided in the transcript. - Comment availability: not determinable from the source material. Hi everyone. Uh before we get started, I actually have a quick question for all of you. How many of you actually today use a closed source model like chat GPD or Anthropic? Okay. How many of you feel like it's getting really expensive really quickly? Okay, that's that's a good percentage of you all. And uh just like all of you might be thinking of switching from closed source model to open source models for cost reasons, performance reasons, there uh other customers and users that are going through the same journey. And today I'm going to talk about where these open source models shine and how to make the performance or quality of these open source models match the closed source models itself. And before I get into that quick introduction of myself uh I work at fireworks. I lead part of the applied machine learning team. My name is Jatashri but feel free to call me Jet if that's easier. Um let's talk about the whole train to deploy loop that that the open source models are taking today when users are switching from closed source to open source models. But even before we get into that let's talk about some fundamental truths that are true today in the AI ecosystem. So the three fundamental truths that are true uh are one open source models for the most part are production ready. If any of you have used GLM 5.2 two, you know that that is true especially for the coding use cases. Two, if open source models don't like beat or match the quality of close source models, the differentiating factor uh comes from how you customize, train and improve these open source models to match the performance and quality of closed source models. And three, the AI stack itself for most of our customers and users is shifting from let's just serve a model once once and that's done to a more continuous loop that is improving from training to inferencing uh the same open source model itself. So that's where fireworks kind of comes in. It makes the whole process of you having to go through the continuous journey of training these open source models as well as hosting these open source models at scale much more easier and also much more cost effective as well. And I'll get into how we actually do the training itself and different flavors of training that we offer towards the end of the presentation. But I did also want to quickly talk about inference before we go to the training portion of it which has become really popular among our customers and user base. And before we also talk about inference and training, I just wanted to give you a quick overview of how uh inference looks like especially at scales and what are the kind of production impact that we have seen among some of our customers and user base itself. So today at in production at scale uh we serve about 30 trillion tokens and it's increasing. Um this number was actually preGLM 5.2 release. And I'm sure this has pretty much kind of increased by about at least 5 trillion tokens a day. Now with the GLM 5.2 release, we support more than 10,000 companies globally and process about 200k requests a particular second itself. And the kind of production impact that we are seeing within our customer and user base is very significant. So cursor uh if for those of you who already use cursor and cursor models u the composer 2 model for cursor was trained and also hosted on our platform today and um it is able to achieve 13x faster inference because of the fire attention engine that we have and I'll speak a little bit about that in the next slide as well. Uh Versell is also another big customers uh of ours. We help them actually fine-tune and also host the model on our platform and they see a 40x latency improvement. And another big use case study is actually genpark. Um genpark lovables are in very similar field where they actually let you design systems with just prompts itself. And uh quality is a big bottleneck or was a big bottleneck for them when they were trying to switch from closed source to open source models. and we helped them fine-tune. We did uh RFD for them. We helped them clean the data set, define evaluator functions and at the end of the RFT training job itself, we saw that there was 12% quality improvement for the Gen Spark use case. Um also we are a trusted platform among all our other customers and user base as well and we have raised around 327 million CDC uh a while ago and we are about 200 people working at fireworks. That brings me to the next portion of it. I know in the previous slide I did mention that we made latency faster for a couple of our customers such as versell and how we do it is quite simple at a at a higher level. Um at fireworks we have our own inference engine called fire attention. So you have a model you have GPUs and the way the model sits on top of the GPUs is through a translation layer called CUDA kernels and as part of fire attention we build our own CUDA kernel. So when a customer comes to us uh with a workload and a performance target in mind then we can optimize this particular inference engine and the whole AI stack including that CUDA kernel for that specific use case. So most of the time customer like versel in the previous slide come to us and tell us uh they have x input tokens y output tokens uh they have zprompt to uh prompt cache hit rate and they want to meet so and so latency targets and we help them optimize this whole inference stack including that cuda kernel so that we can serve this model uh at a cheaper price but also at a that can meet their performance target and for their specific workload itself. uh as part of fire attention and inference optimization, we do also implement the very well-known techniques in the AI field which is things like speculative decoding, disagregation, quantization. In terms of speculative decoding itself, we also allow customers to train their draft models on our platform. Uh so they can even upload the data set very similar to how they do for fine-tuning for uh training a draft model. uh which helps them uh make the model the main model uh perform much faster uh meet latency requirements also gives you better throughput if the draft model is trained uh very well and um so once people actually people actually get to this stage mainly once they're happy with model quality but when customers are switching from closed source to open source models the biggest bottleneck or the gap is quality and the The way we we try to solve for it is through our training product and our training product itself comes in three different flavors for different segments uh within our customer and user base. One being the training agent. This is very popular among our product managers and builders that get that got into this AI inferencing and training field very recently. So the agent can be accessed through our UI and uh you can you you just need to link it the data set that you have uploaded uh on the fireworks platform as well as description on what you're trying to do as part of this training run itself. Uh it will even let you choose the model. It will clean the data set for you. It will also help you do the hyperparameter uh fine-tuning itself because it's not always easy to get the right uh hyperparameters itself. um when you are doing training for your specific workload. The next flavor also one of our most popular flavors among our customer and user base is our managed training and this is what most of our customers use today. They can use our managed training platform one through API and second through the UI itself. It is as simple as you inputting the hyperparameters that you think will be good for your model. choosing a model, uploading the data set itself and then clicking a button. That's how simple we make it on uh fireworks for the manage training itself. And uh I know one challenge a lot of people have is observability once the training run itself kicks in. So we also have a good UI dashboard that shows you how the training uh is happening, how well your model is uh learning at each epoch, what does the loss function look like. Additionally, you can also plug in u your weights and biases API key into the manage training service itself uh and export some of the observability metrics to platforms like weights and biases if you're already using that. And the third and the kind of the most advanced flavor of training that we offer is the training API itself. And the training API is very popular among some uh of the ML engineers within uh different organizations or within our customer base. And in this in this particular this particular offering of the product lets uh ML engineers define their own training loop and loss functions. And this becomes important when you want to customize your training job according to your needs. that is that goes well above beyond the managed training service that we offer itself and these these different uh flavors or the products within the training ecosystem that we offered are offered for supervised finetuning reinforcement fine-tuning and even DPO as well. Um nowadays I think RFT has become extremely extremely popular and if you ever need help in helping define a good evaluator function then we have worked with customers like Genpark that I was talking to you about in the previous slide to help them define an evaluator function clean the data set um and also choose the right hyperparameters especially for RFD. Unfortunately the training agent itself does not support RF. So it does not uh help you clean the data set or even do the hyperparameter tweaking but we are more than happy to help you do it if you have a big use case in mind. So now that we know all about the different um training offerings that we have I also wanted to show you a quick demo of each of these the training agent manage training and the training API itself. Uh but before we get into that I just wanted to close off by saying how the data flywheel looks like today on our platform. I know in the first slide I was talking to you about the realities of AI today where most people prefer doing a continuous loop where they train host train host as they get new and new data and feedback from their users and that is what we are trying to make more seamless on our platform and that's also why we have different training products different offerings as well as making it easier uh for you to host and make it also cheaper and faster for you to host those models at scale itself. And the way this flywheel works is kind of very simple. Once these uh models are trained, I will show you uh a demo in the next slide where the models get trained. It's it's you just are one click away from deploying these models into production. And once you've deployed, you can uh monitor. We have u monitoring metrics for the number of tokens, cash it rate, how your speculator is working, if you've trained a speculator. uh we help you collect gather uh feedback from all the all the metrics that you have collected and if you choose to also retrain on the same platform from the end from the checkpoint that you were using to a new checkpoint itself. So some of some capabilities on the platform that makes these data flywheel possible is one definitely having a host of uh GPUs. We do have we do partner with different cloud and neocloud providers which makes it easier for us to provide compute and also host these models at scale. Another thing is we offer is hotloading and this has become really popular especially when we were training models for cursor and this was actually a feature we implemented specifically for cursor and then we thought would be useful for general public while they were training models as well. uh what hot loading essentially means is when you have a new checkpoint uh you you can directly go to live deployment without any downtime itself. So if you have a deployment and you've gone through this continuous loop and trained a new checkpoint then you can just click a button and swap out the weights itself that you were using uh uh previously with the new endpoints. Another thing that makes it uh easier to train on our platform is the numerical fidelity. So a lot of the time customers use platform A for training and host on platform B. But what happens when you do that is you'll have issues in quality of the model. And the reason you have those issues is humix is not always compatible when you train on a different platform and host on a different platform. So that's why we offer training and inferencing on the same platform and ensure that uh you won't find any numerical differences after you finish training and once you decide to host this particular model on the platform itself. And um one other thing that that makes it more seamless is because training and inference is on the same platform. You're just one click away from uh deploying this model at production scale. Now that we have learned all about uh the inference and training, I just wanted to show you quick demo of different training products and how they work on our platform. The first portion of the demo is showing you how you can use the training agent itself. Over here you can uh input where your training data set lives within the fireworks what it's called. Uh you can also uh give it the eval data set itself. uh I know people have the train and eval or a test data set separately. So you give the mo you give the agent both the data sets and then you ask it uh to fine-tune and even select the hyperparameters and the model for you. The the agent then goes on and tells you the step that it will it will take to accomplish the task that you have given it. So it it details the SFT plan. It tells you why it selected the base model. In this case, it's selected quen 38B. And it'll also give you some information on the hyperparameters and what the whole training process can look like at a high level. And once you're done uh once you are uh you feel okay about the output that the agent has given, you can approve whatever the agent has uh the a the agent has outlined. And once you approve it, it will actually go through different steps and keep you posted on if it finishes step two, what does uh it look like? If it successfully finishes step three, what does that look like now? And at the end of the whole training run, it will also give you a summary of uh the config, the base model um and uh different hyperparameters and how that impacts it as well. And once it's once the whole model training is done, it actually gives you metrics such as accuracy and F1 score. Uh and it also summarizes kind of resummarizes and gives you an overview of everything that uh that it did. And one cool feature is it also gives you recommended next step as part of u the training run itself. Now the next portion of uh this demo actually talks about how we can use our uh manage training platform. I did mention that you just have to select the model data set and the hyperparameters and then you're just a button click away from actually training a model. Say you've uploaded uh the data set, chosen a model. Uh all you'll have to do is then uh go to the go to the u evaluator settings, choose the evaluator data set and also you get to choose the hyperparameter as part of optional settings if you decide to have a more fine grained hyperparameters for your uh training run itself. And once you're happy with all the configurations including the model and the data set itself, you just need to create uh you just need to click on the button that says create. Um and as soon as you click on the button that says create, a training job then kick kicks out. And you can see that uh the UI looks similar to this when a training job actually gets completed. And if you click on that particular training job itself, it'll give you information on the data set it used, the hyperparameters it used, what the model loss itself looks like when the model was training. Um, and then you can even clone these training runs and reproduce it with different settings and configurations itself. And once you're happy with the fine-tuned model, you can go to the deployment section, choose the custom model that just got uh trained on our platform, and then click on deploy once you're happy with the deployment configurations. Or you can choose a set of deployment configurations that we have such as throughput, fast, minimal. uh and this is uh kind of a template that that works well for generic workloads that we have optimized based on throughput latency uh time to first token and other such metrics and uh I unfortunately don't have an entire demo for our training API because the setup itself differs from customer to customer but we do have a lot of cookbooks on our websites that you can see and follow especially if you're interested in customizing your own training loops and loss functions itself. Uh and that's all I have for you for today. I just wanted to give you a brief overview of how the whole training to inferencing loop looks like. Thank you very much. And if you have any questions, I'm happy to take them in the next minute or so that we have.