--- schema: 1 type: video title: >- Beyond Code Coverage: Functionality Testing with Playwright MCP — Marlene Mhangami, Microsoft description: >- When an LLM writes your tests, it tends to write tests that confirm what the code does rather than tests that verify what the user experiences. Your test suite goes green. The app still breaks in ways none of those tests would catch. Marlene Mhangami from Microsoft makes the case for flipping the order: get the agent to write failing Playwright tests against the expected behavior first, then generate code to pass them. The demo runs this live with GitHub Copilot and the Playwright MCP server on a toy store search feature, with the browser open so you can watch the agent click through filters and validate results in real time. Speaker info: - https://x.com/marlene_zw - https://www.linkedin.com/in/marlenemhangami/ - https://github.com/marlenemhangami Timestamps: 0:00 Introduction to GitHub Octoverse stats and 2025/2026 growth 2:13 Does AI actually increase developer productivity? 3:52 Importance of maintaining a clean codebase 4:36 Test-Driven Development (TDD) and the Red-Green-Refactor cycle 6:07 Common criticisms of TDD and unit testing 7:43 The problem with AI-generated self-affirming tests 8:09 Introduction to Playwright for functional testing 9:18 Integrating AI agents with Playwright for faster TDD 10:54 Live Demo: Adding search and filter features to a toy store app 12:25 Using GitHub Copilot CLI and Work IQ for feature requests 13:50 Generating and running Playwright tests live 16:10 Best practices for using AI with Playwright 17:30 Q&A: Handling state management and testing across different screen sizes --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/FWEInOtngmM-beyond-code-coverage-functionality-testing-with-playwright Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/FWEInOtngmM' markdown: 'https://liquidgold.cc/watch/FWEInOtngmM.md' llms: 'https://liquidgold.cc/watch/FWEInOtngmM/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T14:11:04.660Z' language: null children: [] videoId: FWEInOtngmM durationSeconds: 1185 publishedAt: '2026-05-16T18:00:06.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 25 endSeconds: 136 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=25' heldReason: rights - ordinal: 1 text: null startSeconds: 25 endSeconds: 136 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=25' heldReason: rights - ordinal: 2 text: null startSeconds: 148 endSeconds: 237 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=148' heldReason: rights - ordinal: 3 text: null startSeconds: 148 endSeconds: 237 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=148' heldReason: rights - ordinal: 4 text: null startSeconds: 237 endSeconds: 248 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=237' heldReason: rights - ordinal: 5 text: null startSeconds: 294 endSeconds: 372 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=294' heldReason: rights - ordinal: 6 text: null startSeconds: 294 endSeconds: 372 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=294' heldReason: rights - ordinal: 7 text: null startSeconds: 384 endSeconds: 473 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=384' heldReason: rights - ordinal: 8 text: null startSeconds: 384 endSeconds: 473 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=384' heldReason: rights - ordinal: 9 text: null startSeconds: 384 endSeconds: 473 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=384' heldReason: rights - ordinal: 10 text: null startSeconds: 473 endSeconds: 484 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=473' heldReason: rights - ordinal: 11 text: null startSeconds: 494 endSeconds: 507 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=494' heldReason: rights - ordinal: 12 text: null startSeconds: 507 endSeconds: 564 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=507' heldReason: rights - ordinal: 13 text: null startSeconds: 507 endSeconds: 564 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=507' heldReason: rights - ordinal: 14 text: null startSeconds: 564 endSeconds: 575 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=564' heldReason: rights - ordinal: 15 text: null startSeconds: 575 endSeconds: 657 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=575' heldReason: rights - ordinal: 16 text: null startSeconds: 575 endSeconds: 657 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=575' heldReason: rights - ordinal: 17 text: null startSeconds: 667 endSeconds: 750 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=667' heldReason: rights - ordinal: 18 text: null startSeconds: 761 endSeconds: 830 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=761' heldReason: rights - ordinal: 19 text: null startSeconds: 761 endSeconds: 830 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=761' heldReason: rights - ordinal: 20 text: null startSeconds: 840 endSeconds: 978 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=840' heldReason: rights - ordinal: 21 text: null startSeconds: 978 endSeconds: 989 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=978' heldReason: rights - ordinal: 22 text: null startSeconds: 978 endSeconds: 989 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=978' heldReason: rights - ordinal: 23 text: null startSeconds: 978 endSeconds: 989 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=978' heldReason: rights - ordinal: 24 text: null startSeconds: 1081 endSeconds: 1185 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=1081' heldReason: rights - ordinal: 25 text: null startSeconds: 1081 endSeconds: 1185 watch: 'https://liquidgold.cc/watch/FWEInOtngmM?t=1081' heldReason: rights --- # Beyond Code Coverage: Functionality Testing with Playwright MCP — Marlene Mhangami, Microsoft When an LLM writes your tests, it tends to write tests that confirm what the code does rather than tests that verify what the user experiences. Your test suite goes green. The app still breaks in ways none of those tests would catch. Marlene Mhangami from Microsoft makes the case for flipping the order: get the agent to write failing Playwright tests against the expected behavior first, then generate code to pass them. The demo runs this live with GitHub Copilot and the Playwright MCP server on a toy store search feature, with the browser open so you can watch the agent click through filters and validate results in real time. Speaker info: - https://x.com/marlene_zw - https://www.linkedin.com/in/marlenemhangami/ - https://github.com/marlenemhangami Timestamps: 0:00 Introduction to GitHub Octoverse stats and 2025/2026 growth 2:13 Does AI actually increase developer productivity? 3:52 Importance of maintaining a clean codebase 4:36 Test-Driven Development (TDD) and the Red-Green-Refactor cycle 6:07 Common criticisms of TDD and unit testing 7:43 The problem with AI-generated self-affirming tests 8:09 Introduction to Playwright for functional testing 9:18 Integrating AI agents with Playwright for faster TDD 10:54 Live Demo: Adding search and filter features to a toy store app 12:25 Using GitHub Copilot CLI and Work IQ for feature requests 13:50 Generating and running Playwright tests live 16:10 Best practices for using AI with Playwright 17:30 Q\&A: Handling state management and testing across different screen sizes --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/FWEInOtngmM-beyond-code-coverage-functionality-testing-with-playwright Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/FWEInOtngmM · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks > Missing: no stored document chunks are available.