--- schema: 1 type: video title: >- How I deleted 95% of my agent skills and got better results — Nick Nisi, WorkOS description: >- Claude would fake running tests by touching the expected output file. Nick Nisi, DX engineer at WorkOS, fixed it by SHA-256 hashing the actual test output and verifying it cryptographically. His principle: make it easier to do the real work than to lie about it, and enforce that through code and state machines, not prompts. The same discipline reversed an opposite problem. He generated 10,000 lines of skills from WorkOS documentation, measured with evals, and found one skill was dropping a task from 97% correct to 77% correct. He deleted 95% of it, rewrote 553 lines of handwritten gotchas, and eval time dropped from 68 minutes to 6. The model already knew how to code. It just needed to know where the landmines were. Speaker info: - https://x.com/nicknisi - https://linkedin.com/in/nicknisi - https://github.com/nicknisi Timestamps 0:00 Introduction 1:22 The challenge of context switching with agents 2:33 Introducing Case: A harness for agentic workflows 3:33 Rebuilding with a TypeScript state machine 4:45 The critical importance of evidence-based verification 5:59 Applying agentic principles to the WorkOS CLI 7:44 Lessons in documentation: Generating skills from docs 8:52 Why more data (10,000 lines) led to worse performance 9:36 The impact of using evals to measure accuracy 10:40 Key takeaway: Enforce with code, not just prompts 12:41 Treating failures as bugs in the harness system 14:39 Advice for building agentic-ready products 16:01 Final summary: Replacing trust with evidence --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/vy7o1g2iHY8-pruning-agent-skills Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/how-i-deleted-95-of-my-agent-skills-and-got-better-1af3b7d6c4fb markdown: >- https://liquidgold.cc/channel/aidotengineer/video/how-i-deleted-95-of-my-agent-skills-and-got-better-1af3b7d6c4fb.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T10:07:43.859Z' language: null children: [] videoId: vy7o1g2iHY8 durationSeconds: 1063 publishedAt: '2026-05-30T18:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 27 endSeconds: 91 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=27' heldReason: rights - ordinal: 1 text: null startSeconds: 101 endSeconds: 157 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=101' heldReason: rights - ordinal: 2 text: null startSeconds: 157 endSeconds: 169 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=157' heldReason: rights - ordinal: 3 text: null startSeconds: 169 endSeconds: 213 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=169' heldReason: rights - ordinal: 4 text: null startSeconds: 213 endSeconds: 223 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=213' heldReason: rights - ordinal: 5 text: null startSeconds: 223 endSeconds: 293 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=223' heldReason: rights - ordinal: 6 text: null startSeconds: 304 endSeconds: 363 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=304' heldReason: rights - ordinal: 7 text: null startSeconds: 375 endSeconds: 464 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=375' heldReason: rights - ordinal: 8 text: null startSeconds: 375 endSeconds: 464 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=375' heldReason: rights - ordinal: 9 text: null startSeconds: 476 endSeconds: 541 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=476' heldReason: rights - ordinal: 10 text: null startSeconds: 552 endSeconds: 586 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=552' heldReason: rights - ordinal: 11 text: null startSeconds: 586 endSeconds: 598 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=586' heldReason: rights - ordinal: 12 text: null startSeconds: 645 endSeconds: 655 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=645' heldReason: rights - ordinal: 13 text: null startSeconds: 655 endSeconds: 765 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=655' heldReason: rights - ordinal: 14 text: null startSeconds: 655 endSeconds: 765 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=655' heldReason: rights - ordinal: 15 text: null startSeconds: 777 endSeconds: 884 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=777' heldReason: rights - ordinal: 16 text: null startSeconds: 777 endSeconds: 884 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=777' heldReason: rights - ordinal: 17 text: null startSeconds: 884 endSeconds: 895 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=884' heldReason: rights - ordinal: 18 text: null startSeconds: 895 endSeconds: 961 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=895' heldReason: rights - ordinal: 19 text: null startSeconds: 961 endSeconds: 971 watch: 'https://liquidgold.cc/watch/vy7o1g2iHY8?t=961' heldReason: rights --- # How I deleted 95% of my agent skills and got better results — Nick Nisi, WorkOS Claude would fake running tests by touching the expected output file. Nick Nisi, DX engineer at WorkOS, fixed it by SHA-256 hashing the actual test output and verifying it cryptographically. His principle: make it easier to do the real work than to lie about it, and enforce that through code and state machines, not prompts. The same discipline reversed an opposite problem. He generated 10,000 lines of skills from WorkOS documentation, measured with evals, and found one skill was dropping a task from 97% correct to 77% correct. He deleted 95% of it, rewrote 553 lines of handwritten gotchas, and eval time dropped from 68 minutes to 6. The model already knew how to code. It just needed to know where the landmines were. Speaker info: - https://x.com/nicknisi - https://linkedin.com/in/nicknisi - https://github.com/nicknisi Timestamps 0:00 Introduction 1:22 The challenge of context switching with agents 2:33 Introducing Case: A harness for agentic workflows 3:33 Rebuilding with a TypeScript state machine 4:45 The critical importance of evidence-based verification 5:59 Applying agentic principles to the WorkOS CLI 7:44 Lessons in documentation: Generating skills from docs 8:52 Why more data (10,000 lines) led to worse performance 9:36 The impact of using evals to measure accuracy 10:40 Key takeaway: Enforce with code, not just prompts 12:41 Treating failures as bugs in the harness system 14:39 Advice for building agentic-ready products 16:01 Final summary: Replacing trust with evidence --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/vy7o1g2iHY8-pruning-agent-skills Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/how-i-deleted-95-of-my-agent-skills-and-got-better-1af3b7d6c4fb · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.