Audry Hsu of RunPod demonstrates deploying an LLM inference endpoint in under five minutes: selecting a model from the Hub, configuring a context window, and deploying a serverless endpoint on H100 GPUs. The demo shows a cold start of about 41 seconds while the container initializes and the model downloads, with subsequent requests completing in roughly 1.5 seconds; billing occurs only while a worker handles a request. The talk also recounts RunPod's origin: founders repurposed GPUs from failed crypto mining rigs offered free via Reddit for feedback, growing to a reported $120M ARR and 500,000 developers (promotional/creator-claimed figures). # Video Identity Extraction — RunPod Serverless Demo Talk ## Video Overview A live, in-person conference-style introductory talk by **Audry Hsu** of **RunPod**, demonstrating deploying an LLM inference endpoint on RunPod's serverless platform in under five minutes. Includes live console demo and audience Q&A close. ## Intended Audience - **Developers**, explicitly framed throughout: - > "we make it easy for developers to deploy models" - > "we as software developers, we bring the value through the applications that we build" - Audience is addressed as **newcomers to the product**: - > "Can I just get a quick hands to see how many people have already heard of RunPod...? Okay, newbies for everybody." - Assumes technical familiarity (Docker, Hugging Face, vLLM, environment variables) but not with RunPod itself. ## Claimed Scope of the Channel/Company - **Company self-description:** > "RunPod, we are a cloud AI infrastructure company. So, we have the hardware, we have the GPUs" - **Scope of offerings demonstrated:** four product areas described: - **Pods** — "our sandbox virtual environment... spin up a container for you" - **Serverless** — "our auto-scaling product" for "bursty workloads or batch workloads" - **Clusters** — "multi-node clusters with high-speed networking" for training - **Hub** — "our central repository for AI repos... preconfigured, pre-vetted" - **Business claims (creator-stated, not independently verifiable from video):** - > "over 500,000 developers on our platforms" - > "30-plus data centers across the world, including Europe and the EU" - > "120 million in annual recurring revenue" - **Founding story claim:** founders started with "a couple of GPU rigs in their basement in 2022" after "failed crypto mining," growing from a Reddit free-GPU offer. ## Explicit Self-Description vs. Demonstrated Patterns ### Explicit creator claims - Community engagement: > "we are definitely very engaged with our users on Reddit, on Discord" - Customer motivation: customers come "because they need flexible and reliable GPU infrastructure" - Developer-first ethos: > "You bring your code and we'll bring the rest." - Honest-limitation framing: > "I won't promise that we'll be perfect" ### Observable presentation patterns - **Hands-on demo style:** full live console walkthrough — hub listing selection, deploy click, worker initialization, telemetry check, and a live API request (first request ~41s queue, ~1.5s execution). - **Marketing-backed technical talk:** mixes founding narrative and revenue milestones with concrete configuration detail (max workers, always-on workers, spending caps, H100/A100 defaults, per-second pricing). - **Part of a session series:** references > "another session later today at 4:00... focused on our Python SDK... completely via the terminal" — indicates a multi-talk event presence. - **Interactive delivery:** audience hand-raise poll, crowd-sourced prompt choice ("Say one"), and closing Q&A. ## Audience-Addressed Value Proposition - **Problem framing** (creator-claimed): infrastructure is hard, "GPU access is slow and opaque" (with toilet-paper/COVID analogy), and builders should focus on building. - **Speed claim, then demonstrated:** title promises under 5 minutes; speaker confirms post-demo: > "that was probably less than 5 minutes to get started and get something deployed... from a hub listing." # Delivery Observations — "Under 5 minutes to a deployed LLM endpoint" (Audry Hsu, RunPod) ## Overall Tone & Register - **Casual, conversational, demo-driven** delivery. The speaker addresses the room directly and keeps things light: she opens with an audience poll — *"Can I just get a quick hands to see how many people have already heard of RunPod?"* — and jokes about the result: *"Okay, newbies for everybody. Great."* - She uses **first-person plural "we"** consistently when describing the company ("we have the GPUs," "we want to build"), and **direct address "y'all"/"you guys"** when narrating the demo (*"I'm going to show you via the console"*). - She sets expectations of modesty rather than hype: *"I won't promise that we'll be perfect."* ## Vocal & Verbal Fillers - **Very frequent "um"** throughout, appearing in nearly every paragraph of the transcript — e.g., *"Um So, RunPod, we are a cloud AI infrastructure company"* and *"So, um I don't know if you if anybody has tried to buy a GPU recently."* This is a **stable, pervasive pattern**, not isolated. - Occasional **restarts and self-corrections**: *"it's even it's even more abstracted"*, *"These are These are our founders"*, *"we bring bring the value"*, *"I'll I'll I'll switch to in a second."* Repetition of a word mid-sentence recurs multiple times. - Filler phrases like *"just really quickly"*, *"just at a glance"*, *"really quick"* signal a habit of **flagging brevity** before explanatory sections. ## Teaching Habits & Structure - **Problem → solution framing**: she explicitly poses a rhetorical question — *"what problems does RunPod solve? Like, why are we even here today?"* — then enumerates three problems ("infrastructure can be hard," "GPU access is slow and opaque," "builder primary focus should be building"). - **Analogies to familiar experience**: the COVID/toilet-paper comparison for the GPU supply crunch — *"it's a bit like in COVID when everybody went to the store and bought all the toilet paper."* Also a pre-cloud comparison: *"back in the day before we had AWS, um Google Cloud."* Two analogies in one short talk suggests a habit, though the sample is small. - **Storytelling for credibility**: a founder-origin anecdote ("a couple of GPU rigs in their basement in 2022, um failed crypto mining... They posted on Reddit"), followed by explicit signposting of the story's point: *"the reason why I want to tell this story is um not because it's it's very like bootstrappy, but because..."* - **Live demo as the centerpiece**: she announces it beforehand (*"I'm going to switch over and just show y'all really quick how easy it is"*) and narrates every click in real time, including waiting states: *"This might take a minute since or two."* - **Self-deprecating audience interaction mid-demo**: *"Let's Which which should we ask the LLM today? Say one. Okay. How did I'm I'm American, so how did Bid Then get its name? I don't know."* — she solicits and then lightly riffs on audience input. ## Recurring Vocabulary & Phrases - **"Um we" / "so we" sentence starters** appear repeatedly at section openings. - The phrase **"just"** as a minimizer recurs: *"I'll just open the underlying repository"*, *"I'm just going to go ahead and click deploy"*, *"it's literally just a GitHub um repo."* - **"Literally"** appears at least twice: *"that is literally how our company has started"* and *"you could guys can see what it is literally just a GitHub repo."* Two instances suggest a mild verbal tic, but repetition is thin — stated with uncertainty. - Domain jargon used fluently and unprompted: *Dockerfiles, vLLM serve, cold start, H100s, A100s, max loras, workers, Hugging Face*. ## Communication Patterns - **Closing loops / bookending**: she returns to the talk's title premise at the end — *"that was probably less than 5 minutes to get started and get something deployed"* — explicitly tying the demo back to the promised outcome. - **Forward-referencing other sessions**: *"we have another session later today at 4:00... focused on our Python um SDK"* — a habit of orienting the audience to the event schedule. - **Transparency about demo mechanics**: she explains why latency occurred (*"that's going to be a little bit longer than all of the subsequent requests because of some of the cold start time"*) rather than glossing over it — evidence of a **teach-through-the-hiccup** approach. - Human-vs-agent aside: *"everything that's ready for your agent so you don't have to read our documents, but since we're all humans here today, I'm going to show you via the console"* — a humorous framing device. ## Uncertainty Notes - Audience interaction is observable mainly at the opening and once mid-demo; whether it's a deep habit or situational to a live crowd cannot be confirmed from this single transcript. - Transcript artifacts (e.g., "Bid Then") suggest possible transcription errors of spoken content; quoted fidelity beyond the transcript cannot be verified. # Observation Document: RunPod Intro & Serverless Deployment Demo **Speaker:** Audrey Hsu (RunPod) — sole presenter in transcript. ## Company Positioning - RunPod is a **cloud AI infrastructure company** providing GPUs and hardware to make model deployment easy for developers. - Deploys **private models or open-source models from Hugging Face** — "You bring your code and we'll bring the rest." ## Problems RunPod Claims to Solve - **Infrastructure management burden**: Comparable to pre-AWS on-prem server maintenance; something developers "happily have moved away from" and delegated to DevOps. - **Slow, opaque GPU access**: Global supply crunch likened to COVID toilet-paper panic buying; Hsu expects **market recovery** as compute demand estimation improves (a prediction, qualified as expectation). - **Builder focus**: Software developers create value through applications, not infrastructure management. ## Founding Story - Founders **Zenon and Pardeep** had GPU rigs in their basement in **2022** after "failed crypto mining." - Prototyped RunPod's foundations, offered free GPU access on **Reddit** for feedback; company has been **revenue-generating ever since**. - Stated relevance: origin in community feedback persists via Reddit and Discord engagement. - Explicit qualification: "I won't promise that we'll be perfect." ## Scale Claims - **500,000+ developers** on the platform - **30+ data centers** worldwide, including Europe/EU - **$120M annual recurring revenue** milestone - AI cloud-native companies among customers; motivation cited: **flexible, reliable GPU infrastructure** ## Product Line Overview - **Pods**: sandbox virtual environments; container spun up with allocated GPUs; user brings Dockerfiles and code. - **Serverless**: auto-scaling for **bursty/batch workloads**; workers spin down when idle, no idle charges. - **Clusters**: multi-node with high-speed networking for "heavy-duty training." - **Hub**: preconfigured, pre-vetted central repository of AI repos; community contributions; fork, watch, star, deploy. ## Serverless Focus - Best for **real-time inference**; no need to pre-provision compute. - Configurable: **max workers**, **spending caps**, **always-on workers** (pre-loaded models for immediate responses). - Claimed as **fastest path to a production-ready API** for many teams. ## Live Demo Claims - Console used for visibility; **CLI support and agent-ready skills** also exist. - Hub listing is "literally just a GitHub repo" with a preconfigured Dockerfile; environment variables configurable; options passed as **vLLM serve flags**. - Demo deployment: default **H100s with A100 backup**; pricing "a fraction of a cent per second"; charged only while workers handle requests. - Observability: telemetry for requests, execution time, delay time. - Measured results: first request queued **~41 seconds** (attributed to cold-start: model download, container initialization); **execution time ~1.5 seconds**. - Claim: **"probably less than 5 minutes"** from hub listing to deployed endpoint ("probably" is the speaker's own hedging). ## Closing - Follow-up session announced at 4:00 focused on the **Python SDK**, terminal-based, covering deploying code as remote functions to GPU and creating production-ready endpoints. # Video Attribution & Provenance Record **Video ID:** `01a15ca9-1005-791d-b451-fd517b6916e5` **Title:** Under 5 minutes to a deployed LLM endpoint — Audry Hsu, RunPod ## Speakers & Roles - **Audrey Hsu (spelled "Audry" in title, "Audrey" in self-introduction)** — **Speaker / presenter**, identified as representing **RunPod** ("I am from RunPod. Um this is an intro to RunPod"). - **Live audience** — unattributed interactive participants; hands raised in a poll, one audibly suggested an LLM prompt ("Say one"), applause at the end. No narrator, guest, host, or other distinct speaker roles appear in the transcript. ## Promotional Segments (Flagged) The entire talk is a **company promotional session** delivered by a RunPod representative: - Intro to RunPod as a **cloud AI infrastructure company**. - Company stats: **500,000+ developers**, **30+ data centers**, **$120M annual recurring revenue**. - Founders' origin story (Zenon and Pardeep; GPU rigs in 2022, Reddit launch). - Product tour: **Pods, Serverless, Clusters, Hub**. - Live console **demo deploying an LLM endpoint** from a Hub listing. - Promotion of a **follow-up session** the same day at 4:00 on the Python SDK. ## Referenced Sources & Third-Party Claims - **AWS and Google Cloud** — cited as historical comparison to on-prem servers. - **Hugging Face** — cited as source for open-source models and the model download in the demo. - **Reddit and Discord** — cited as RunPod community/feedback channels; founders' original launch post on Reddit. - **GitHub** — Hub listings underpinned by GitHub repos with Dockerfiles. - **vLLM** — configuration flags said to be passed to `vLLM serve`. - Third-party customer names were referenced on a slide ("these are just a few of our customers") but **not named in the transcript**. - Demo query asked "How did Bid… get its name?" (response content not quoted in transcript). ## Provenance Notes & Unavailable Evidence - No curated metadata was supplied; all facts above come from the transcript alone. - Audience-submitted question content, demo LLM output, and slide-visible customer names are **not available** in the source. - **Comment availability:** unknown / not indicated. - Speaker name spelling discrepancy ("Audry" vs. "Audrey") noted; no authoritative spelling available in source. \>\> Audrey, um I am from RunPod. Um this is an intro to RunPod. Can I just get a quick hands to see how many people have already heard of RunPod or maybe even used RunPod before? Okay, newbies for everybody. Great. Um So, RunPod, we are a cloud AI infrastructure company. So, we have the hardware, we have the GPUs, and we make it easy for developers to deploy um models. And that can be your own private model, it can be an open-source model from Hugging Face. Doesn't matter to us. You bring your code and we'll bring the rest. Um just really quickly, what problems does RunPod solve? Like, why are we even here today? Um infrastructure can be hard, managing it. I think about back in the day before we had AWS, um Google Cloud, when everybody would have to have on-prem servers and manage those, maintain those. That \[clears throat\] is something that we don't want to have to do as developers. Those are things that we happily um have moved away from and given off to DevOps, and now it's even it's even more abstracted for us. GPU access is slow and opaque. So, um I don't know if you if anybody has tried to buy a GPU recently, we're in a global supply crunch. Um it's a bit like in COVID when everybody went to the store and bought all the toilet paper because we didn't know how how long they would need to be at home for. We're a little bit in that right now. Um but we expect the market will recover um as customers, companies, people figure out a little bit better, uh get a little bit better at um estimating what kind of compute they need. Um and then last, builder primary focus should be building. So, again, um we want to build we as software developers, uh we bring bring the value through the applications that we build, um not from managing the infrastructure. And I think RunPod has a pretty unique story. These are These are our founders, Zenon and Pardeep. Um so, they had a couple of GPU rigs in their basement in 2022, um failed crypto mining, and then so they're like, "What are we going to do with our GPUs now?" Um so, they prototyped what is now the foundations of RunPod. They posted on Reddit and said, "Hey, anyone want to use these GPUs for free? Just give us feedback on it." And that is literally how our company has started, and we have been um revenue generating ever since. Um and the reason why I want to tell this story is um not because it's it's very like bootstrappy, but because um the origin origin story of RunPod has always started with uh builders and getting feedback from the community, and that is still true today. So, I won't promise that we'll be perfect, but um we are definitely very engaged with um our users on on Reddit, um on Reddit, on Discord. So, um we're always trying to stay engaged with y'all. Um just at a glance to give you an idea of RunPod, we have over 500,000 developers on our platforms, 30-plus data centers across the world, including um Europe and the EU. Um and we've just passed a significant revenue milestone for us, 120 million in annual recurring revenue. Uh these are just a few of our customers. Um you might be surprised to see some of the AI cloud-native companies on here, too, but um they come to us for the same reasons that most of our customers come to us. It's um because they need flexible and reliable GPU infrastructure. This is a really high-over high-level overview of um different ways you can build on RunPod. So, I would say our core at our core, um pods, it's our sandbox virtual environment. We spin up a container for you, um allocate GPUs to it, and we manage the rest. You just bring your um Dockerfiles, you bring your code. Serverless, um it's our auto-scaling product. So, when you're thinking more about like bursty workloads or batch workloads, um serverless is really great because um instead of being always on like a container is, serverless um your workers spin down, and when they're idle you don't pay for anything. Clusters, um if you were doing some heavy-duty training, there's a place for you as well on RunPod. Um multi-node clusters with high-speed networking. And then the hub, which I'll I'll I'll switch to in a second, um it's kind of like our central repository for AI repos. Um these are already preconfigured, pre-vetted. Um we have a couple of examples of listings by RunPod for popular models, but also our community um contributes to them as well. So, there's repos that you can fork, you can watch, and then um you can star and deploy on RunPod. Um so, today we're going to be talking mostly about serverless. Um so, serverless is best for real-time inference. I talked about the auto-scaling that comes with it. Um why teams use it is mostly because they don't need to um preempt and figure out how much compute they need ahead of time. Um You can set you can configure the number of max workers that you want to scale up to. You can set limits for caps, for spending caps. And you can also configure workers that are always on, so they're um already have your models downloaded, and they can respond to requests um immediately. For a lot of teams, serverless is the fastest way um if you want to start deploying a production-ready API. And now I'm going to switch over and just show y'all really quick how easy it is to get started and deploy something. Okay. Where are we? Okay. So, right now I'm um going to do everything via the console so that it's nice and pretty for you guys to see, but we also have um CLI support. We have skills um to help work with RunPod, everything that's ready for your agent so you don't have to read our documents, but since we're all humans here today, I'm going to show you via the console. Um we'll start in the hub, which is if you're just trying to explore and see what's out there, what is something that you can get up and running right now, the hub is a great place to start. So, like I mentioned, these are already vetted open-source listings for um AI repos, and I am going to pick the LLM, um and I'll just open the underlying repository as well so you could guys can see what it is literally just a GitHub um repo. It tells you how to get set up for it. Um we can see there's already the Dockerfile here. It's already preconfigured for you. It's got some defaults for you um depending on the listing. You can um pass in different environmental variables um to configure it how you wish. But I'm just going to go ahead and click deploy. And I have a model that I wanted I Let me see. I was going to just pick one. Looks well. This is going to download it from Hugging Face. I'm just expand the advanced options and look for the max model length, and I'm going to bump this up for the context window, and leave everything else as the defaults, but there's settings for max loras. Um all of these configuration options get passed as um as flags to the uh vLLM serve. And I'm going to spin it up as an endpoint here. This might take a minute since or two since this is the very I just created it. I've got to initialize my workers. Let's check out. So, the default configuration here is it's going to deploy on some H100s, and A100s are the backup here. I have my pricing. This is fraction of a cent per second. Um like I mentioned before that this is only going to be charged for a while the worker is actually running and handling a request. Max workers is where I can bump this up if I want to have my workload scale up up to 15 workers at a time and I can set um some active workers, ones that I want always to be on that I don't want um the container to ever spin down. And I can save that. Okay, so how do I how does one interact with the serverless endpoint? Um this is just an API uh HTTP endpoint right here. Um we provision this endpoint for you. You can send requests to this. Your customers can send requests to this. If I just hit run and I'm going to add a few Let's Which which should we ask the LLM today? Say one. Okay. How did I'm I'm American, so how did Bid Then get its name? I don't know. \>\> \[clears throat\] \>\> Um okay, well these requests are queued. Let me check on our workers. Okay. We have a handful that are in initializing. Um this is the container's being created. That's the model being downloaded. Um getting ready and the ones that are running, they've already finished. These are probably going to be the ones who are going to pick up those requests that we just added. I've got telemetry about um it's blank right now, but the number of requests, execution time, delay time, so you have observability into how your endpoints are operating. \>\> \[snorts\] \>\> And let's see. Okay, it's already done. \>\> \[snorts\] \>\> I got a request back and it sat in the queue for about 41 seconds. Um that's going to be a little bit longer than all of the subsequent requests because of some of the cold start time that I talked about, like downloading the model, um initializing the first container, but um execution time only about 1 and 1/2 seconds, so yeah, that was probably less than 5 minutes to get started and get something deployed um on serverless from a hub listing. Does anyone have any questions? This is is a very short and sweet intro. Um we have another session later today at 4:00 um and that one is going to be focused on our Python um SDK and that one is going to be completely via the terminal. Um and I'm going to walk you through how I can spin up uh and deploy my code on my code as a remote remote function onto a GPU um and uh deploy in the end and make it like a production-ready endpoint here as well. Okay. But that's all I got for today, so yeah, thanks thanks for coming. \>\> \[applause\]