Conference talk by Ahmad Osman (Founder, Osmantic) on running frontier-class open models locally on desktop hardware, demonstrated on a DGX Station. Topics include efficiency trends over raw model size (e.g., "impact per parameter" and the Densing Law), shifting workloads from server-grade to consumer-grade GPUs, a retrospective on open-weight models from Mistral to Qwen, reasoning model evolution (DeepSeek R1), agentic tool calling, sovereign AI / owning your own compute stack, and whether GPU hardware appreciates in value as models become more efficient. # Observation Document — "The Desktop Frontier" ## Video Overview - **Format**: Live conference/stage presentation ("we are about to start this presentation") with audience interaction (hand-raising, show-of-hands questions, laughter). - **Presenter**: Ahmad Osman of **Osmantic** (per title). He refers to himself as "Osmantic" implicitly via title only; no explicit channel self-description appears in the transcript itself. - **Topic**: The trajectory of **local and open-source AI models** — "where we started and how far we've come with local and open source models." ## Creator Self-Description & Claims - **Public audience on X/Twitter**: Presenter explicitly claims a following there: > "how many of you here follow me on X? ... I'm amazed. Love you all." > "there is a reason that I'm not selling any of my RTX3090s **if you follow me**." This demonstrates he maintains a public presence on X where he shares predictions about local AI hardware/models. - **Track record claim**: "Earlier this year in December, I had a very viral post that I predicted that we're going to have the quality of OBS 4.5 running locally ... on a single RTX Pro 6000. That happened by March." - **Personal hardware stake** (stated, not inferred): "I have a lot of hardware, guys" — including "8 RTX3090s at home." ## Claimed Scope / Subject Matter - **Core thesis**: Open-source/local model efficiency is compounding — framed as the "**densing law**" (attributed to *Nature Machine Intelligence*): "every three and a half months we are having 50% fewer parameters." - **Key framing concept**: "**Impact per parameter**" — comparing hardware footprint year-over-year for equivalent capability. - **Prediction made in-video**: "within roughly 18 months we are going to have the equivalent of GLM 5.2 class intelligence running on a single RTX 5090 with 32 GB of VRAM" — described as "conservative." - **Advocacy position**: Promotes **sovereign AI** / hardware ownership over cloud subscription: > "Why wouldn't you as a consumer, as an individual, as a small-size business, mid-size business, enterprise ... want to be in control of the models that you're on?" - **Community framing**: "we need enterprises for open source AI to win" — positions open-source AI advocacy as a community cause. ## Intended Audience (Demonstrated) - **Local AI enthusiasts and hardware owners** — assumes familiarity with consumer GPU models (RTX 3090, RTX 5090, DGX Station, RTX Pro 6000). - **Open-source model users** — rapid-fire references to Llama 2/3, Mistral 7B, Qwen 2.5/3.5, DeepSeek R1, GLM 4.5/5.2, GPT-OSS 12B without explanation, expecting recognition ("Anyone remembers that one from last summer?"). - **Developers running local agents** — references running models "within Claude Code" / "Open Code" and "15 parallel agents." - **Potential buyers of hardware** — closing rhetorical question: "Should you buy a GPU?" (with disclaimer: "This is not financial advice"). ## Observable Presentation Patterns - **Data-driven but informal**: mixes benchmark comparisons, historical model timelines, and casual jokes ("If you didn't raise your hand, we know you lie about other things as well"). - **Hardware cost/value focus**: recurring theme that purchased GPUs appreciate in capability as models shrink ("a hardware purchase today does it get more valuable as models become more efficient?"). - **No channel-level metadata supplied** — identity observations rest entirely on the title and in-transcript statements. ## Notes on Evidence Limits - The channel's broader identity (Osmantic's product/service scope, "ODS for consumers" — likely a product reference — is mentioned once without explanation) cannot be characterized beyond what appears here. - Motivations are taken only as stated (e.g., keeping GPUs to "see what I could do with them in a year or two"); no private intent inferred. # Delivery Observations — "The Desktop Frontier" ## Tone & Overall Style - **Conversational, casual talk-track**: frequent filler and self-corrections throughout — "Uh," "um," "you know," "so yeah." > "Uh it's called the desktop frontier [snorts] and um [clears throat]" > "So um yeah, as I was saying, similar capabilities are moving into smaller hardware footprint." - **Enthusiastic, rhetorical, audience-engaging** — frequently poses questions to the room rather than lecturing. ## Recurring Linguistic Habits - **"You know" as a connective filler** — appears constantly: > "So, you know, right now, where we're at, it's uh GLM 5.2. That's uh our, you know, biggest player" > "So, you know, again, Lama 2, uh, that was a 70 billion parameter model." - **"As I was saying / as I was saying earlier"** — repeated callback phrase: > "as I was saying earlier I used to run lama 2 on an RTX 3090" > "That's the impact per parameter. As I was saying I want everybody to live here thinking about this term" - **"Um, so yeah" / "Uh so yeah"** as segment transitions: > "So um yeah, as I was saying…" > "So yeah, the forwardl lookinging question is basically…" - **"That's massive"** as an enthusiasm marker, repeated: > "That's that's a massive jump in terms of performance gains." > "That's massive amount of performance gains and a very small amount of time" > "That that's massive. That thing require data centers to serve." ## Teaching / Persuasion Patterns - **Rhetorical questions directed at the audience**, often chained: > "So is this trend just random or is there more to it? That's a question that everyone should ask." > "Why wouldn't you invest, you know, in sovereign AI? Why wouldn't you as a consumer… Why wouldn't you want to make sure that nothing gets taken away from you?" > "Should you buy a GPU?" - **Audience interaction** — polls and playful banter: > "how many of you here follow me on X? >> I'm I'm amazing. Love you all. Love you all." > "raise your hand if you have a phone. If you didn't raise your hand, we know you lie about other things as well. So, come on, guys." > "Anyone remembers that one from last summer? Yeah. Nobody here used it. Come on, guys. I I need some help here. [laughter]" - **Anchoring arguments in specific numbers and hardware names** — model sizes, parameter counts, GPU models, and dates are the core evidence style: > "a 400 billion 400 billion plus parameters model that you beat with a 27 billion parameter model a year and a half after" > "that's summer 2024 to March 2026 that's uh that's about 21 months" - **Coining/emphasizing a takeaway term** — explicitly flags key concepts for retention: > "the term that I like to think about is impact per parameter" > "I want everybody to live here thinking about this term" - **Citing external backing**: > "the literature I back this up with uh uh nature machine intelligence uh calls this pattern densing law" ## Recurring Narrative Devices - **Then-vs-now comparisons** as the main rhetorical structure: > "a year ago this time a year ago we didn't have any local models that were able to successfully run within clo code" > "I used to run lama 2 on an RTX 3090 it's now running qu 3.5 3.6 6 27 billion parameter." - **Citing his own past predictions as credibility**: > "I had a a very viral post that I predicted that we're going to have the quality of OBS 4.5 running locally at home on a single RTX uh Pro 6000. That happened by March." - **Memorable aphorism-style line**, self-consciously highlighted: > "And I think I like this line. It's not that small models are beating big models. It's that newer, more efficient models are beating older, less efficient ones." - **Hedged disclaimer moment** — a single instance: > "This is not a financial advice by the way. Like let me make this very clear." ## Verbal & Physical Mannerisms - Audible **snorts** appear multiple times in the transcript ("[snorts]" at [12], [459], [759], [1012]) — a recurring physical habit, though the frequency may be partly a transcription artifact (uncertainty noted). - **Repeated words/stutters under pace**: > "million parame million uh tokens locally" > "it's it's it was it was it was one of the first open source models" - **Slide-dependent pacing** with a mid-talk disruption: > "I think I yeah sorry that was a misclick um [clears throat]" ## Uncertainty Notes - The "[snorts]" and "[laughter]" markers depend on the transcript's annotations; their frequency as a genuine habit cannot be fully confirmed. - One-off moments (the "misclick," the financial-advice disclaimer) are isolated, not established patterns. # The Desktop Frontier — Ahmad Osman, Osmantic ## Speaker **Ahmad Osman** (identified from video title/curated metadata) — sole presenter. ## Core Prediction - **Claim:** Within roughly **18 months** (late 2027, described as "conservative, we might get there faster"), the equivalent of **GLM 5.2-class intelligence** will run on a single **RTX 5090 with 32 GB VRAM**. - Earlier prediction cited as precedent: in **December**, he predicted **GPT-OSS 4.5-quality** models would run locally on a single RTX Pro 6000 — which he says **happened by March**. ## Central Thesis: **Impact per Parameter** - The dominant story of "bigger models" is shifting to **efficiency gains**. - Qualification preserved: **"There will always be a gap"** between frontier intelligence and open source models — but that gap will shrink and efficiency will improve exponentially. - Key framing: **"It's not that small models are beating big models. It's that newer, more efficient models are beating older, less efficient ones."** - Supporting evidence cited from **Nature Machine Intelligence**, which he says calls the pattern the **"Densing Law"**: every **3.5 months**, roughly **50% fewer parameters** (dense or activated) for equivalent capability. ## Evidence of Efficiency Gains (Speaker's Claims) - Ran **Llama 2** on an RTX 3090 previously; now runs **Qwen 3.5/3.6 27B**, which he claims beats **Llama 405B** — a 400B+ parameter model — ~18–21 months later. - **GLM 4.5 Air** (late July) required 4× RTX 3090s or an RTX Pro 6000; now a single consumer GPU runs something more capable. - **GLM 5.2**: 744B total parameters, 40B activated, 1M context; runs in **NVFP4** on a **DGX Station** or 8× RTX Pro 6000; on one benchmark **beats GPT 5.5 Extra High**. - **Nemotron 3 Ultra**: proved NVFP4 training is feasible — implying cheaper fine-tuning and specialized small models. - **Llama 2 70B** used to require 8× RTX 3090s; those same cards now run **~15 parallel agents with Qwen 3.5 27B**. - **GPT-OSS 12B**: first open-source model to do tool calling successfully; ~1/5 the size of DeepSeek R1 (671B) with comparable or better agentic performance. - **Qwen 3.5** (397B MoE) vs **Qwen 3.6 dense 27B**: the dense model wins across benchmarks despite being ~15× smaller, with only 40% more activated parameters (27B vs 17B activated) — trend occurred in **2–3 months**. - **GPT-4o-quality** models can now run on an iPhone, something that previously required data centers. - Context length: from models that couldn't sustain 4,000 tokens to **1M-token contexts locally**. ## Historical Progression (as narrated) Llama 2 → **Mistral 7B** ("one of my favorite models," wouldn't work in Claude Code/OpenCode today) → **Mixtral 8x7B** (MoE) → **Llama 3 8B** ("unique identity") → Llama 3 70B → **Llama 3 405B** → **Gemma 27V** ("next big thing in my opinion") → **Qwen 2.5** ("the moment I was like okay we actually are making progress"; enabled fine-tuning, papers, community work) → **DeepSeek R1** (reasoning at home, ~700B MoE; May release showed post-training gains on same checkpoints) → **GPT-OSS 12B** (tool calling) → **Qwen 3.5 / 3.6**. ## Argument for Sovereign / Local AI - Reasoning: cloud subsidies will eventually go away; models can be taken away, limited, or refuse requests; owning hardware means control, per-use-case optimization, and long-run savings. - **Community claim:** "We need enterprises for open source AI to win" — enterprises should own the full stack end-to-end to sustain incentives, new licenses, and continued open-source model releases. ## Hardware Value Argument - Forward question: **what will a DGX Station run in 3–36 months?** - RTX 3090 (Ampere, 2020) still sells above MSRP and remains useful — evidence hardware appreciates in capability as models shrink. - Personal position: he owns substantial hardware and won't sell his RTX 3090s — interested in future utility over present resale value. **Explicit disclaimer: "This is not financial advice."** - Closing question to audience: **"Should you buy a GPU?"** ## Qualifications & Caveats Preserved - The frontier/open-source **gap will always exist** (speaker's own concession). - Speaker explicitly frames his 18-month prediction as **conservative math**. - Densing-law figure applies to "dense or activated" parameters — "that's a different story," he notes. - Benchmark comparisons are speaker's claims from single/unnamed benchmarks ("on one benchmark"). # Video Attribution Record — "The Desktop Frontier — Ahmad Osman, Osmantic" ## Speakers & Roles - **Ahmad Osman (Osmantic)** — **Creator / Presenter**. Sole identified speaker throughout the presentation; delivers predictions, hardware comparisons, and open-source model history. - **Audience (unidentified)** — present via interaction (raising hands, implied laughter); no speaking lines attributed. - No **narrator**, **guest**, or other identified roles appear. ## Promotional / Advocacy Segments (Flagged) - Promotion of **sovereign AI / hardware ownership**: presenter argues consumers and enterprises should buy their own GPUs rather than fund cloud data centers ("Why wouldn't you invest in sovereign AI?", "Should you buy a GPU?"). - Promotion of **Osmantic ODS**: "ODS for consumers it's basically the way that we support individuals but enterprises also." - Self-promotion of the presenter's X account and prior viral prediction post. - Explicit disclaimer: "This is not financial advice by the way." ## Referenced Sources & Third-Party Claims - **Nature Machine Intelligence** — cited as literature backing the "densing law" pattern (~50% fewer parameters every 3.5 months). - **Model claims made by the presenter** (not independently verified in the video): - GLM 5.2 (744B total / 40B activated, 1M context; claimed to beat "GBT 5.5 extra high" on one benchmark) - Qwen 3.5 / 3.6 variants, Llama 2/3 family, Mistral 7B, Mixtral 8x7B, DeepSeek R1 (671B), Nemotron 3 Ultra (NVFP4 training), GPT-OSS 12B, Hermes/Telegram-run 9B models - Prior prediction (December): OBS 4.5-quality locally on one RTX Pro 6000 — claimed fulfilled by March ## Provenance Boundaries & Uncertainty - All performance and benchmark claims are **presenter assertions**; no cited benchmark documents shown/linked in transcript. - Future predictions (GLM 5.2-class on RTX 5090 within ~18 months / late 2027) are explicitly presenter opinion, labeled "conservative." - Hardware pricing claim (RTX 3090 selling above MSRP) is presenter statement without citation. ## Metadata Availability - **Curated metadata:** not supplied beyond title and video ID. - **Comment availability:** not indicated. - **Timestamps:** none available in transcript. Hey everyone, we are about to start this presentation. Uh it's called the desktop frontier \[snorts\] and um \[clears throat\] it's basically about where we started and how far we've come with local and open source models. Um how many like just a quick question how many of you here follow me on X? \>\> I'm I'm amazing. Love you all. Love you all. Uh so you know I sometimes every now and then I would say a prediction uh here is a new one within roughly 18 months we are going to have the equivalent of GLM 5.2 class intelligence running on a single RTX 5090 with 32 GB of VRAM. Um that's basically late 2027. Um this this is conservative. We might actually get there faster. So uh you know for a long time uh the story has been bigger models bigger models bigger models how can we get to the next 5 trillion how can we get to the 20 trillion and I'm not saying that there won't ever be like a gap between frontier intelligence um and u you know open source models there will always be a gap but that gap um will shrink and the efficiency of the models will get exponentially better. So the the term that I like to think about is impact per parameter. Um you know what capability are we talking about? What could a model do? Um what footprint like hardware footprint did it have last year in comparison to now? and uh what hardware does that use and what hardware does it need to use a year ago and uh you know are we moving down for the same kind of quality on the on that hardware again as I was saying earlier I used to run lama 2 on an RTX 3090 it's now running qu 3.5 3.6 6 27 billion parameter. That's better than Lamas 405. That's a 400 billion 400 billion plus parameters model that you beat with a 27 billion parameter model a year and a half after. So um yeah, as I was saying, similar capabilities are moving into smaller hardware footprint. Um benchmark scores are one thing but also you know a year ago this time a year ago we didn't have any local models that were able to successfully run within clo code right it wasn't until GLM 4.5 that came out in late July and GLM 4.5 air required at least four RTX uh 3090s or an RTX Pro 6000. Now that footprint for hardware is not needed anymore. All that you need is a single RTX 1390 1590 and you have something much more capable, much more intelligent. So is this trend just random or is there more to it? That's a question that everyone should ask. Um, is it just by by random chance that we've gotten this far from models that were weren't able to sustain more than 4,000 um tokens in terms of context lengths and now we have things that are million parame million uh tokens locally on your hardware that you own. It's not by chance, you know, it's not just a coincidence that we got here. Um, there is research being done. There is efficiency gains to be made. There are architecture hacks that compound and they will continue to compound. And I think I like this line. It's not that small models are beating big models. It's that newer, more efficient models are beating older, less efficient ones. Uh so yeah capability density uh is you know the literature I back this up with uh uh nature machine intelligence uh calls this pattern densing law and uh basically um you know every three and a half months we are having 50% fear parameters whether that's in dense or activated that's a different story but we're getting way more intelligence out of the models that we're running. So, you know, right now where we're at, it's uh GLM 5.2. That's uh our, you know, biggest player and um it's 744 billion parameters total with only 40 billion parameter activated. And that supports up to 1 million contexts. You can run this in MVFP4 on a machine uh on a GGX station or on a server with eight RTX Pro 6000. That's something that you like a GX station is something that you can sit under your desk and it's running this kind of frontier intelligence. Whether you know it it's on one benchmark it actually beats GBT 5.5 extra high. Doesn't that mean that we're getting somewhere with local and open source models that we can compete with the frontier that we're not that far off from the best that you can get from the cloud? We also have Neatron 3 ultra which proved that NVFB4 training more efficient training can be done on uh on hardware right that's that's very important that means that the footprint even for training these models for fine-tuning them for making small and specialized models as I was talking earlier could be more efficient could be done cheaper and could be you know could deliver you value in terms of economics way sooner or you know for much less money than you used Yeah. So, you know, again, Lama 2, uh, that was a 70 billion parameter model. If you try to run that right now, you're you'd laugh at it, right? That used to take eight RTX 1390s to load up and it those same eight RTX3090s could run something like 15 parallel agents right now with Quen 3.5 27. That's that's a massive jump in terms of performance gains. Um so the densing law basically means that we have similar or better um capabilities with significantly fewer parameters. That's the impact per parameter. As I was saying I want everybody to live here thinking about this term and you know thinking where are we going to get a year from today as I was saying earlier everyone here has a phone. I'm assuming raise your hand if you have a phone. If you didn't raise your hand, we know you lie about other things as well. So, come on, guys. So, \[clears throat\] you know, you can run you can now run GBT40 quality on your iPhone. That that's massive. That thing require data centers to serve. So, why wouldn't you invest, you know, in sovereign \[snorts\] AI? Why wouldn't you as a consumer, as an individual, as a smalls size business, middlesiz business, enterprise, why wouldn't you want to be in control of the models that you're on? Why wouldn't you want to make sure that nothing gets taken away from you? That every little thing can be optimized for you later on, that the performance gains can be made specially and specifically for your use cases, and that you can save more money that way in the long run. and you know ODS for consumers it's basically the way that we support individuals but enterprises also and I think that there is something that we like as a community we need to think about deeply we need enterprises for open source AI to win we need these people that are using the cloud right now that are basically supporting data centers being built for cloud providers to come on this side to own their own hardware to own the stack fully end to end so that we can keep delivering open source models so that there is an incentive for open source providers to actually come up with models so that we can come up with new licenses that allows open source to thrive and so again um open weight and the frontier I think I yeah sorry that was a misclick um \[clears throat\] you know so smaller models started bunching above the weight after lama 2 with mistral 7B one of my favorite models if you try to build that model right now and um clo code or oven code it's not going to work but it used to take so much in terms of hardware right that you would now get from a 9b mill model that I can run with telegram with or with hermes for example and uh do a lot of stuff with so we've come a long Okay. Uh we had that we had mixed trial 8 by 7B which you know everybody knows is an MOE. Then the progression went uh from that to Lamas 3 you know Lamas 3 8B was one of my favorites still is um it had unique identity in my opinion. Uh then we had like the 70 billion which was like the the thing that I would run basically on my 8 RTX3090s at home. Then there was like the 405 the 400 billion plus parameter lamas which again required a lot of hardware and if you put it now against 3.5 the 27 billion parameter would lose against it that's in the span of what two years two years and some no I think I think less than two years that's summer 2024 to March 2026 that's uh that's about 21 months and the next big thing in my opinion gamma 27V and then we had the Quinn 2.5 and that that was the moment that I was like okay we actually are making progress and the gap was shrinking between open source models and uh the frontier uh really lamas 3 saved like you know it really helped us a lot and then um Gwen 2.5 delivered a massive improvement and there was a lot of fine-tuning and experiments that could be done on that one there was amazing papers and um they helped the community immensely in my opinion. Then the next big thing was Deepseek R1 in my opinion and uh reasoning becoming something that you can run at home. That was a massive MOE almost 700 billion parameters. Um you know you had to have like a very beefy server to actually get it up and running. Um and then you know the improvements that came from just more training on that one and Deepseek R1 that was released in May last year made massive jump again. So it showed that post training could deliver more improvements on the same on the same checkpoints. Then GBT open source like GBT OSS 12B. Anyone remembers that one from last summer? Yeah. Nobody here used it. Come on, guys. I I need some help here. \[laughter\] Uh it was it was it was one of the first open source models that were able to successfully do tool calling. Um and uh it was a step forward. It showed us that we can do more with uh with the hardware that we have at running at home. Uh that was a footprint shift right from like you know that massive 700 billion parameters deepseek uh R1 that was yeah 671 billion parameters to something that was 1/5 of its size in GBTSS with comparable maybe better more agentic performance. Um \[snorts\] then the moment of uh quinc 3.5 that's 397 that's three that's 397 billion parameters uh that's uh that's a BVOE and uh you know what's funny is that about uh it's about 15 times the size of uh the Quinn 3.6 and I'm here I'm comparing 3.5 to 3.6 6 of the dense 27 billion parameter model and that dense model beats it and that dense model has 40% higher number of activated parameters. So it's not that far off. That's massive amount of performance gains and a very small amount of time with massively different footprint in terms of hardware uh requirements. And that trend happened in like what two three months. So you know how far could we go from here? Um how far before we get to you know uh a recent model that there was some news about you know that is uh finally relaunch the game. How far before open source delivers something of that quality that you could run on your own hardware and you can control and will not be taken away from you and will not refuse a request from you. So again these are just some benchmarks where you can see that an iteration on the 27 billion parameter model a little bit more post training proved it across all benchmarks and made it one against a model that is almost 15 size for 15 times its size. And again, remember this is 27 billion parameters activated versus 17 billion parameter activated. It's still massively the same amount like you know it's it's only 40% less in terms of the amount of time it would take to process things but it's 15 times smaller. That's a lot. So again um how long until the prediction I made earlier becomes plausible when I said that we're going to have the equivalent of GLM 5.2 2 running on an RTX uh 1590. This is the mass 17 months and this is a conservative math. Uh earlier this year in December, I had a a very viral post that I predicted that we're going to have the quality of OBS 4.5 running locally at home on a single RTX uh Pro 6000. That happened by March. So a question um hardware purchase today does it get more valuable as models become more efficient and smaller in size? That's a good question. So why are you funding other people to build data centers so that you can subscribe to them and pay subsidized tokens and then later on get those subsidies are going to go away and you're not going to be able to run those models and they will have so many limitations. So might as well ask yourself why not own the hardware yourself and be in control. Um, so yeah, the forwardl lookinging question is basically what will a DGX station be able to run in three, six, 12, 18 months from now? That's something that there is a reason that I'm not selling any of my RTX3090s if you follow me. And I have a lot of hardware, guys. U but I'm interested in seeing what I could do with them in a year or two from now more than in the amount of money I would get for them today. This is not a financial advice by the way. Like let me make that very clear. Um so yeah um the disk side frontier potential um you know an Nvidia GX station could run a lot of uh today it could run GLM 5.2 to what will it be able to run tomorrow 6 months 18 months two years from today we know that you know RTX3090 \[snorts\] is the amber uh architecture from 2020 sells at higher value than MSRP today and it's still being utilized for a lot of use cases so what well at GGX station the actively developed blackwell architecture will be able to run in a few months a couple of years that's a Good question. So the question you have to ask yourself um if an RTX30590 with 32 GB of VRAM runs in the equivalent of a GLM 5.2 and 18 months and this is the question that everybody should be asking themselves and I want you all to be looking at the screen taking this very seriously. Okay. Should you buy a GPU? \>\> Thank you.