--- schema: 1 type: video title: >- Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber description: >- This talk covers how Uber designed evals for its food enhancement agent, which edits food photography to better present dishes for smaller, independent Uber Eats merchants, along with the pitfalls and lessons learned along the way. The problem is uniquely hard: the system must stay faithful to the original dish, preserve each merchant's brand and packaging, and avoid homogenizing the marketplace, all without an existing playbook for multimodal evals in a narrow domain. Soumya Gupta and Jai Chopra explain how they navigated reward hacking, built a closed feedback loop combining offline and online signals, and balanced creativity against rigid safety guardrails at scale. ML and applied AI practitioners working on multimodal systems, agentic pipelines, or eval design will take away practical strategies for narrow-domain multimodal evaluations, countering reward hacking, and production feedback loops. Speakers: Soumya Gupta — ML Engineer, Uber Soumya is a Tech Lead and Applied AI Engineer who architects and scales production-grade generative AI and computer vision systems at Uber. X/Twitter: https://x.com/guptasoumya12 LinkedIn: https://www.linkedin.com/in/guptasoumya12/ Jai Chopra — Product Manager, Uber Jai is a Product Lead on Uber's Applied AI team and previously worked at Cruise and several startups. X/Twitter: https://x.com/jai_chopra LinkedIn: https://linkedin.com/in/jaichopra --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/31GUkCBD-Uc-building-closed-loop-evals-for-a-multimodal-agent-at-uber-scale Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/31GUkCBD-Uc' markdown: 'https://liquidgold.cc/watch/31GUkCBD-Uc.md' llms: 'https://liquidgold.cc/watch/31GUkCBD-Uc/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T06:35:52.570Z' language: null children: [] videoId: 31GUkCBD-Uc durationSeconds: 1299 publishedAt: '2026-07-24T22:00:25.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 27 endSeconds: 157 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=27' heldReason: rights - ordinal: 1 text: null startSeconds: 27 endSeconds: 157 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=27' heldReason: rights - ordinal: 2 text: null startSeconds: 157 endSeconds: 169 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=157' heldReason: rights - ordinal: 3 text: null startSeconds: 169 endSeconds: 247 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=169' heldReason: rights - ordinal: 4 text: null startSeconds: 258 endSeconds: 347 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=258' heldReason: rights - ordinal: 5 text: null startSeconds: 258 endSeconds: 347 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=258' heldReason: rights - ordinal: 6 text: null startSeconds: 347 endSeconds: 358 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=347' heldReason: rights - ordinal: 7 text: null startSeconds: 358 endSeconds: 445 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=358' heldReason: rights - ordinal: 8 text: null startSeconds: 445 endSeconds: 457 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=445' heldReason: rights - ordinal: 9 text: null startSeconds: 457 endSeconds: 584 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=457' heldReason: rights - ordinal: 10 text: null startSeconds: 584 endSeconds: 595 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=584' heldReason: rights - ordinal: 11 text: null startSeconds: 595 endSeconds: 716 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=595' heldReason: rights - ordinal: 12 text: null startSeconds: 716 endSeconds: 728 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=716' heldReason: rights - ordinal: 13 text: null startSeconds: 728 endSeconds: 851 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=728' heldReason: rights - ordinal: 14 text: null startSeconds: 728 endSeconds: 851 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=728' heldReason: rights - ordinal: 15 text: null startSeconds: 863 endSeconds: 945 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=863' heldReason: rights - ordinal: 16 text: null startSeconds: 863 endSeconds: 945 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=863' heldReason: rights - ordinal: 17 text: null startSeconds: 956 endSeconds: 1067 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=956' heldReason: rights - ordinal: 18 text: null startSeconds: 956 endSeconds: 1067 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=956' heldReason: rights - ordinal: 19 text: null startSeconds: 1067 endSeconds: 1078 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=1067' heldReason: rights - ordinal: 20 text: null startSeconds: 1078 endSeconds: 1211 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=1078' heldReason: rights - ordinal: 21 text: null startSeconds: 1222 endSeconds: 1299 watch: 'https://liquidgold.cc/watch/31GUkCBD-Uc?t=1222' heldReason: rights --- # Building Closed-Loop Evals for a Multimodal Agent at Scale — Soumya Gupta & Jai Chopra, Uber This talk covers how Uber designed evals for its food enhancement agent, which edits food photography to better present dishes for smaller, independent Uber Eats merchants, along with the pitfalls and lessons learned along the way. The problem is uniquely hard: the system must stay faithful to the original dish, preserve each merchant's brand and packaging, and avoid homogenizing the marketplace, all without an existing playbook for multimodal evals in a narrow domain. Soumya Gupta and Jai Chopra explain how they navigated reward hacking, built a closed feedback loop combining offline and online signals, and balanced creativity against rigid safety guardrails at scale. ML and applied AI practitioners working on multimodal systems, agentic pipelines, or eval design will take away practical strategies for narrow-domain multimodal evaluations, countering reward hacking, and production feedback loops. Speakers: Soumya Gupta — ML Engineer, Uber Soumya is a Tech Lead and Applied AI Engineer who architects and scales production-grade generative AI and computer vision systems at Uber. X/Twitter: https://x.com/guptasoumya12 LinkedIn: https://www.linkedin.com/in/guptasoumya12/ Jai Chopra — Product Manager, Uber Jai is a Product Lead on Uber's Applied AI team and previously worked at Cruise and several startups. X/Twitter: https://x.com/jai_chopra LinkedIn: https://linkedin.com/in/jaichopra --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/31GUkCBD-Uc-building-closed-loop-evals-for-a-multimodal-agent-at-uber-scale Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/31GUkCBD-Uc · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.