--- schema: 1 type: video title: >- Skill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face description: >- Merve Noyan wrote a book on vision language models and now wants developers to stop calling them directly. Put one in front of a camera and you will never get real time; a small detector trained for the task runs at forty frames per second on a toaster and beats the VLM anyway. Her other complaint is licensing: people deploy a popular detector without noticing its copyleft license. So she built a toolkit that hands her favorite Apache 2.0 models to a coding agent, which she calls a clueless computer vision engineer, plus what she calls vibe training. Give it a dataset with no labels and it labels images with a nine billion parameter open VLM, passes the overlaid bounding boxes to two smaller VLM judges, merges their verdicts on minimum agreement rather than consensus, and trains RF-DETR. The whole run costs three or four dollars on Hugging Face jobs and inference providers. On road signs the trained detector lands a good mean average precision against ground truth, and on document parsing it generalizes, catching a signature the labeling model itself missed. The findings matter more: one judge rejects far more than the other, so consensus would have left too few examples; the judge prompts still need a human to approve them; and even the best coding agent flips traffic signs horizontally and jitters the color of traffic lights until told not to. She closes with the models as tools half of the toolkit, from a 600 million parameter model that segments the red car next to the orange car to pose, depth, and OCR picks, and plans for image guided detection where words cannot describe the part. Speaker info: - https://x.com/mervenoyann - https://www.linkedin.com/in/merve-noyan-28b1a113a - https://hf.co/merve Timestamps: 0:00 - Why developers should stop reaching for a VLM at runtime 1:51 - Read the license: move to Apache 2.0 models 2:46 - A toolkit for coding agents, the clueless computer vision engineer 3:41 - Vibe training: VLM as labeler, VLMs as judges, then train 5:40 - The pipeline: overlaid boxes, minimum agreement, RF-DETR 8:29 - What it costs: a few dollars end to end 9:40 - Results on road signs and document parsing 11:18 - Findings: judge imbalance, prompt approval, augmentation blunders 13:20 - Favorite models as tools, from segmentation to depth 15:52 - Future plans: image guided detection and IoU merging 17:57 - Q&A --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/dKcTBQzR7jI-youtube-e5a893f2152909c51019 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/skill-issue-stop-deploying-vision-language-models-u-ed8bf14a93e0 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/skill-issue-stop-deploying-vision-language-models-u-ed8bf14a93e0.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-23T22:20:56.880Z' language: null children: [] videoId: dKcTBQzR7jI durationSeconds: 1154 publishedAt: '2026-09-23T13:30:16.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 25 endSeconds: 118 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=25' heldReason: rights - ordinal: 1 text: null startSeconds: 25 endSeconds: 118 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=25' heldReason: rights - ordinal: 2 text: null startSeconds: 118 endSeconds: 130 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=118' heldReason: rights - ordinal: 3 text: null startSeconds: 130 endSeconds: 175 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=130' heldReason: rights - ordinal: 4 text: null startSeconds: 130 endSeconds: 175 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=130' heldReason: rights - ordinal: 5 text: null startSeconds: 175 endSeconds: 187 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=175' heldReason: rights - ordinal: 6 text: null startSeconds: 187 endSeconds: 221 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=187' heldReason: rights - ordinal: 7 text: null startSeconds: 232 endSeconds: 345 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=232' heldReason: rights - ordinal: 8 text: null startSeconds: 232 endSeconds: 345 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=232' heldReason: rights - ordinal: 9 text: null startSeconds: 345 endSeconds: 355 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=345' heldReason: rights - ordinal: 10 text: null startSeconds: 355 endSeconds: 516 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=355' heldReason: rights - ordinal: 11 text: null startSeconds: 355 endSeconds: 516 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=355' heldReason: rights - ordinal: 12 text: null startSeconds: 355 endSeconds: 516 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=355' heldReason: rights - ordinal: 13 text: null startSeconds: 529 endSeconds: 582 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=529' heldReason: rights - ordinal: 14 text: null startSeconds: 594 endSeconds: 682 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=594' heldReason: rights - ordinal: 15 text: null startSeconds: 594 endSeconds: 682 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=594' heldReason: rights - ordinal: 16 text: null startSeconds: 695 endSeconds: 807 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=695' heldReason: rights - ordinal: 17 text: null startSeconds: 695 endSeconds: 807 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=695' heldReason: rights - ordinal: 18 text: null startSeconds: 818 endSeconds: 960 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=818' heldReason: rights - ordinal: 19 text: null startSeconds: 818 endSeconds: 960 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=818' heldReason: rights - ordinal: 20 text: null startSeconds: 818 endSeconds: 960 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=818' heldReason: rights - ordinal: 21 text: null startSeconds: 972 endSeconds: 1085 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=972' heldReason: rights - ordinal: 22 text: null startSeconds: 972 endSeconds: 1085 watch: 'https://liquidgold.cc/watch/dKcTBQzR7jI?t=972' heldReason: rights --- # Skill issue: stop deploying vision language models, use them with Skills — Merve Noyan, Hugging Face Merve Noyan wrote a book on vision language models and now wants developers to stop calling them directly. Put one in front of a camera and you will never get real time; a small detector trained for the task runs at forty frames per second on a toaster and beats the VLM anyway. Her other complaint is licensing: people deploy a popular detector without noticing its copyleft license. So she built a toolkit that hands her favorite Apache 2.0 models to a coding agent, which she calls a clueless computer vision engineer, plus what she calls vibe training. Give it a dataset with no labels and it labels images with a nine billion parameter open VLM, passes the overlaid bounding boxes to two smaller VLM judges, merges their verdicts on minimum agreement rather than consensus, and trains RF-DETR. The whole run costs three or four dollars on Hugging Face jobs and inference providers. On road signs the trained detector lands a good mean average precision against ground truth, and on document parsing it generalizes, catching a signature the labeling model itself missed. The findings matter more: one judge rejects far more than the other, so consensus would have left too few examples; the judge prompts still need a human to approve them; and even the best coding agent flips traffic signs horizontally and jitters the color of traffic lights until told not to. She closes with the models as tools half of the toolkit, from a 600 million parameter model that segments the red car next to the orange car to pose, depth, and OCR picks, and plans for image guided detection where words cannot describe the part. Speaker info: - https://x.com/mervenoyann - https://www.linkedin.com/in/merve-noyan-28b1a113a - https://hf.co/merve Timestamps: 0:00 - Why developers should stop reaching for a VLM at runtime 1:51 - Read the license: move to Apache 2.0 models 2:46 - A toolkit for coding agents, the clueless computer vision engineer 3:41 - Vibe training: VLM as labeler, VLMs as judges, then train 5:40 - The pipeline: overlaid boxes, minimum agreement, RF-DETR 8:29 - What it costs: a few dollars end to end 9:40 - Results on road signs and document parsing 11:18 - Findings: judge imbalance, prompt approval, augmentation blunders 13:20 - Favorite models as tools, from segmentation to depth 15:52 - Future plans: image guided detection and IoU merging 17:57 - Q\&A --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/dKcTBQzR7jI-youtube-e5a893f2152909c51019 Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/skill-issue-stop-deploying-vision-language-models-u-ed8bf14a93e0 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.