--- schema: 1 type: video title: >- Self-Training Agents: Hermes Agent, HF Traces, Skills, MCP & Finetuning — Merve Noyan, Hugging Face description: >- Open-source models have caught up. GLM 5.1 is leading the Artificial Analysis intelligence index over closed models, and the gap is closing fast with each release cycle. The practical upside beyond benchmarks: full weight access means you can quantize, fine-tune, and deploy to edge devices or browsers without data leaving your infrastructure. @MerveNoyan walks through the Hugging Face ecosystem built around this: inference providers that route to the fastest or cheapest option per model, benchmark datasets for filtering by SWE-bench or AIME scores directly on Hub, a traces repository type for storing and exploring agent sessions, and skills that plug into coding agents. The closer is a live demo where she asks Claude Code to fine-tune a vision-language model on a dataset by name. The agent calculates VRAM requirements, selects an instance, and kicks off the job. What used to be a day of napkin math is now a prompt. Speaker info: - https://x.com/mervenoyann - https://www.linkedin.com/in/merve-noyan-28b1a113a/ - https://github.com/merveenoyan Timestamps 0:00 Introduction to Open Agent Ecosystem 0:39 Importance of Open Source in Machine Learning 2:36 Hugging Face Hub overview 3:06 Agentic models and Vision-LMs 4:24 Benchmark datasets and model filtering 5:16 Inference providers and model routing 6:50 Local coding agents and tools 7:46 Hermes agents for memory management 9:20 Traces repository for agent sessions 10:22 Tips for finding and serving local models 12:07 Supercharging agents with Hugging Face skills 13:41 Live demonstration of agent-driven fine-tuning 14:41 Training vision models (object detection/segmentation) 15:00 Using Model Context Protocol (MCP) for agents 16:30 Case study: OCR processing for AI papers --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/OV56RddyFuU-self-training-agents-hermes-agent-hf-traces-skills-mcp-finetuning-merve-noyan-hugging-face Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/self-training-agents-hermes-agent-hf-traces-skills-85fd451ea941 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/self-training-agents-hermes-agent-hf-traces-skills-85fd451ea941.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T14:11:14.922Z' language: null children: [] videoId: OV56RddyFuU durationSeconds: 1151 publishedAt: '2026-05-13T17:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 39 endSeconds: 52 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=39' heldReason: rights - ordinal: 1 text: null startSeconds: 52 endSeconds: 162 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=52' heldReason: rights - ordinal: 2 text: null startSeconds: 52 endSeconds: 162 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=52' heldReason: rights - ordinal: 3 text: null startSeconds: 186 endSeconds: 198 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=186' heldReason: rights - ordinal: 4 text: null startSeconds: 198 endSeconds: 266 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=198' heldReason: rights - ordinal: 5 text: null startSeconds: 278 endSeconds: 326 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=278' heldReason: rights - ordinal: 6 text: null startSeconds: 326 endSeconds: 337 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=326' heldReason: rights - ordinal: 7 text: null startSeconds: 337 endSeconds: 417 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=337' heldReason: rights - ordinal: 8 text: null startSeconds: 417 endSeconds: 428 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=417' heldReason: rights - ordinal: 9 text: null startSeconds: 475 endSeconds: 487 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=475' heldReason: rights - ordinal: 10 text: null startSeconds: 569 endSeconds: 582 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=569' heldReason: rights - ordinal: 11 text: null startSeconds: 582 endSeconds: 630 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=582' heldReason: rights - ordinal: 12 text: null startSeconds: 641 endSeconds: 730 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=641' heldReason: rights - ordinal: 13 text: null startSeconds: 730 endSeconds: 742 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=730' heldReason: rights - ordinal: 14 text: null startSeconds: 742 endSeconds: 824 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=742' heldReason: rights - ordinal: 15 text: null startSeconds: 824 endSeconds: 834 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=824' heldReason: rights - ordinal: 16 text: null startSeconds: 891 endSeconds: 903 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=891' heldReason: rights - ordinal: 17 text: null startSeconds: 903 endSeconds: 916 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=903' heldReason: rights - ordinal: 18 text: null startSeconds: 916 endSeconds: 997 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=916' heldReason: rights - ordinal: 19 text: null startSeconds: 1009 endSeconds: 1151 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=1009' heldReason: rights - ordinal: 20 text: null startSeconds: 1009 endSeconds: 1151 watch: 'https://liquidgold.cc/watch/OV56RddyFuU?t=1009' heldReason: rights --- # Self-Training Agents: Hermes Agent, HF Traces, Skills, MCP & Finetuning — Merve Noyan, Hugging Face Open-source models have caught up. GLM 5.1 is leading the Artificial Analysis intelligence index over closed models, and the gap is closing fast with each release cycle. The practical upside beyond benchmarks: full weight access means you can quantize, fine-tune, and deploy to edge devices or browsers without data leaving your infrastructure. @MerveNoyan walks through the Hugging Face ecosystem built around this: inference providers that route to the fastest or cheapest option per model, benchmark datasets for filtering by SWE-bench or AIME scores directly on Hub, a traces repository type for storing and exploring agent sessions, and skills that plug into coding agents. The closer is a live demo where she asks Claude Code to fine-tune a vision-language model on a dataset by name. The agent calculates VRAM requirements, selects an instance, and kicks off the job. What used to be a day of napkin math is now a prompt. Speaker info: - https://x.com/mervenoyann - https://www.linkedin.com/in/merve-noyan-28b1a113a/ - https://github.com/merveenoyan Timestamps 0:00 Introduction to Open Agent Ecosystem 0:39 Importance of Open Source in Machine Learning 2:36 Hugging Face Hub overview 3:06 Agentic models and Vision-LMs 4:24 Benchmark datasets and model filtering 5:16 Inference providers and model routing 6:50 Local coding agents and tools 7:46 Hermes agents for memory management 9:20 Traces repository for agent sessions 10:22 Tips for finding and serving local models 12:07 Supercharging agents with Hugging Face skills 13:41 Live demonstration of agent-driven fine-tuning 14:41 Training vision models (object detection/segmentation) 15:00 Using Model Context Protocol (MCP) for agents 16:30 Case study: OCR processing for AI papers --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/OV56RddyFuU-self-training-agents-hermes-agent-hf-traces-skills-mcp-finetuning-merve-noyan-hugging-face Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/self-training-agents-hermes-agent-hf-traces-skills-85fd451ea941 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:15 to 0:52) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (0:52 to 2:42) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (2:42 to 3:06) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (3:06 to 4:26) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (4:26 to 5:26) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (5:26 to 6:57) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (6:57 to 7:55) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (7:55 to 9:29) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (9:29 to 10:30) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (10:30 to 12:10) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 10: Document excerpt (12:10 to 13:44) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 11: Document excerpt (13:44 to 14:51) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 12: Document excerpt (14:51 to 15:03) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 13: Document excerpt (15:03 to 16:37) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 14: Document excerpt (16:37 to 16:49) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 15: Document excerpt (16:49 to 19:11) > Held: transcript rights do not permit this chunk's text.