--- schema: 1 type: video title: 'Stop Making Models Bigger, Make Them Behave — Kobie Crawford, Snorkel' description: >- Qwen 3 235B was asked for YouTube's year over year ad revenue growth from 2023 to 2024. It queried a table that didn't exist, tried again, got nothing back both times, and hallucinated an answer. The 4B model Snorkel finetuned with RL called `get_table_name` first, inspected the schema, ran a query, hit a column error, self-corrected, and got the right answer. The training run cost under $500. Kobe Crawford covers why tool discipline matters more than reasoning depth for this class of tasks, how single table training transferred cleanly to harder multi table problems (13.9% to 26.6% on the FinQA reasoning benchmark), and why breaking evals into rubrics helps identify which specific behavior to fix before writing any training data. Speaker info: - https://www.linkedin.com/in/kobie-crawford - https://snorkel.ai/author/kobie-crawford/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/TNwJ1LMiENk-stop-making-models-bigger-make-them-behave-kobie-crawford-snorkel Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: >- https://liquidgold.cc/channel/aidotengineer/explore/video/stop-making-models-bigger-make-them-behave-kobie-cr-9e00a245b431 markdown: >- https://liquidgold.cc/channel/aidotengineer/video/stop-making-models-bigger-make-them-behave-kobie-cr-9e00a245b431.md llms: null parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T09:26:40.693Z' language: null children: [] videoId: TNwJ1LMiENk durationSeconds: 1256 publishedAt: '2026-06-10T17:00:25.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 139 endSeconds: 228 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=139' heldReason: rights - ordinal: 1 text: null startSeconds: 139 endSeconds: 228 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=139' heldReason: rights - ordinal: 2 text: null startSeconds: 240 endSeconds: 348 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=240' heldReason: rights - ordinal: 3 text: null startSeconds: 240 endSeconds: 348 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=240' heldReason: rights - ordinal: 4 text: null startSeconds: 358 endSeconds: 449 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=358' heldReason: rights - ordinal: 5 text: null startSeconds: 358 endSeconds: 449 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=358' heldReason: rights - ordinal: 6 text: null startSeconds: 460 endSeconds: 541 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=460' heldReason: rights - ordinal: 7 text: null startSeconds: 553 endSeconds: 638 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=553' heldReason: rights - ordinal: 8 text: null startSeconds: 649 endSeconds: 736 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=649' heldReason: rights - ordinal: 9 text: null startSeconds: 649 endSeconds: 736 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=649' heldReason: rights - ordinal: 10 text: null startSeconds: 746 endSeconds: 835 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=746' heldReason: rights - ordinal: 11 text: null startSeconds: 746 endSeconds: 835 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=746' heldReason: rights - ordinal: 12 text: null startSeconds: 845 endSeconds: 934 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=845' heldReason: rights - ordinal: 13 text: null startSeconds: 934 endSeconds: 946 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=934' heldReason: rights - ordinal: 14 text: null startSeconds: 946 endSeconds: 1035 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=946' heldReason: rights - ordinal: 15 text: null startSeconds: 1046 endSeconds: 1154 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=1046' heldReason: rights - ordinal: 16 text: null startSeconds: 1046 endSeconds: 1154 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=1046' heldReason: rights - ordinal: 17 text: null startSeconds: 1046 endSeconds: 1154 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=1046' heldReason: rights - ordinal: 18 text: null startSeconds: 1165 endSeconds: 1256 watch: 'https://liquidgold.cc/watch/TNwJ1LMiENk?t=1165' heldReason: rights --- # Stop Making Models Bigger, Make Them Behave — Kobie Crawford, Snorkel Qwen 3 235B was asked for YouTube's year over year ad revenue growth from 2023 to 2024. It queried a table that didn't exist, tried again, got nothing back both times, and hallucinated an answer. The 4B model Snorkel finetuned with RL called \`get_table_name\` first, inspected the schema, ran a query, hit a column error, self-corrected, and got the right answer. The training run cost under $500. Kobe Crawford covers why tool discipline matters more than reasoning depth for this class of tasks, how single table training transferred cleanly to harder multi table problems (13.9% to 26.6% on the FinQA reasoning benchmark), and why breaking evals into rubrics helps identify which specific behavior to fix before writing any training data. Speaker info: - https://www.linkedin.com/in/kobie-crawford - https://snorkel.ai/author/kobie-crawford/ --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/TNwJ1LMiENk-stop-making-models-bigger-make-them-behave-kobie-crawford-snorkel Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/channel/aidotengineer/explore/video/stop-making-models-bigger-make-them-behave-kobie-cr-9e00a245b431 · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.