--- schema: 1 type: video title: >- Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI description: >- An agent hands a doctor a clean, confident fact: the patient has a penicillin allergy. But that fact was synthesized from three sources, an EHR record, a lab report, and something the patient typed into an intake chatbot, and by the time it reaches the doctor, which one it came from is gone. You cannot just stamp a source ID on it, because the LLM merged entities and later data invalidated earlier facts, so the store keeps shifting under your pointer. Daniel Chalef's argument is that provenance for a knowledge graph an LLM builds has to be a graph itself. In Graphiti, the open source framework behind Zep, sources become episodes and every derived fact links back to them, so tracing a fact to its origin is just a graph walk. Tag a source once and the tag follows every node and edge derived from it, which lets an agent keep only facts from verified clinical sources. Deletion is the same walk in reverse: a GDPR erasure removes a source, and a fact survives only if another source still supports it. Compliance gets an audit trail, and engineers get agents they can debug instead of black boxes. Speaker info: - https://x.com/danielchalef - https://www.linkedin.com/in/danielchalef/ - https://github.com/getzep/graphiti Timestamps: 0:00 - Why LLM synthesis destroys the paper trail 1:10 - Graphiti, Zep, and the provenance problem 1:47 - The failure mode: a penicillin allergy from three sources 2:53 - Why a source ID does not survive an LLM pipeline 4:20 - Provenance as a graph: tracing a fact is a walk 5:09 - Keeping lineage correct through merges and invalidation 6:06 - Metadata projection: tag a source once 7:25 - Mixed trust parents: allergy flags versus consent 8:57 - Deletion: GDPR erasure through the same edges 10:26 - Benefits: compliance, veracity, and debuggability 11:31 - Q&A: cost, dedup, and why not just markdown --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/H7puB0RwJMM-citation-needed-provenance-for-llm-built-knowledge-graphs Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- canonical: 'https://liquidgold.cc/watch/H7puB0RwJMM' markdown: 'https://liquidgold.cc/watch/H7puB0RwJMM.md' llms: 'https://liquidgold.cc/watch/H7puB0RwJMM/llms.txt' parent: 'https://liquidgold.cc/channel/aidotengineer.md' channel: aidotengineer updated: '2026-10-24T06:48:56.698Z' language: null children: [] videoId: H7puB0RwJMM durationSeconds: 1254 publishedAt: '2026-07-23T04:00:02.000Z' searchChunks: - ordinal: 0 text: null startSeconds: 23 endSeconds: 70 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=23' heldReason: rights - ordinal: 1 text: null startSeconds: 107 endSeconds: 119 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=107' heldReason: rights - ordinal: 2 text: null startSeconds: 178 endSeconds: 189 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=178' heldReason: rights - ordinal: 3 text: null startSeconds: 189 endSeconds: 267 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=189' heldReason: rights - ordinal: 4 text: null startSeconds: 279 endSeconds: 313 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=279' heldReason: rights - ordinal: 5 text: null startSeconds: 323 endSeconds: 368 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=323' heldReason: rights - ordinal: 6 text: null startSeconds: 379 endSeconds: 449 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=379' heldReason: rights - ordinal: 7 text: null startSeconds: 461 endSeconds: 541 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=461' heldReason: rights - ordinal: 8 text: null startSeconds: 541 endSeconds: 552 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=541' heldReason: rights - ordinal: 9 text: null startSeconds: 552 endSeconds: 631 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=552' heldReason: rights - ordinal: 10 text: null startSeconds: 697 endSeconds: 708 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=697' heldReason: rights - ordinal: 11 text: null startSeconds: 823 endSeconds: 916 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=823' heldReason: rights - ordinal: 12 text: null startSeconds: 927 endSeconds: 1019 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=927' heldReason: rights - ordinal: 13 text: null startSeconds: 1019 endSeconds: 1031 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=1019' heldReason: rights - ordinal: 14 text: null startSeconds: 1031 endSeconds: 1110 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=1031' heldReason: rights - ordinal: 15 text: null startSeconds: 1121 endSeconds: 1254 watch: 'https://liquidgold.cc/watch/H7puB0RwJMM?t=1121' heldReason: rights --- # Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI An agent hands a doctor a clean, confident fact: the patient has a penicillin allergy. But that fact was synthesized from three sources, an EHR record, a lab report, and something the patient typed into an intake chatbot, and by the time it reaches the doctor, which one it came from is gone. You cannot just stamp a source ID on it, because the LLM merged entities and later data invalidated earlier facts, so the store keeps shifting under your pointer. Daniel Chalef's argument is that provenance for a knowledge graph an LLM builds has to be a graph itself. In Graphiti, the open source framework behind Zep, sources become episodes and every derived fact links back to them, so tracing a fact to its origin is just a graph walk. Tag a source once and the tag follows every node and edge derived from it, which lets an agent keep only facts from verified clinical sources. Deletion is the same walk in reverse: a GDPR erasure removes a source, and a fact survives only if another source still supports it. Compliance gets an audit trail, and engineers get agents they can debug instead of black boxes. Speaker info: - https://x.com/danielchalef - https://www.linkedin.com/in/danielchalef/ - https://github.com/getzep/graphiti Timestamps: 0:00 - Why LLM synthesis destroys the paper trail 1:10 - Graphiti, Zep, and the provenance problem 1:47 - The failure mode: a penicillin allergy from three sources 2:53 - Why a source ID does not survive an LLM pipeline 4:20 - Provenance as a graph: tracing a fact is a walk 5:09 - Keeping lineage correct through merges and invalidation 6:06 - Metadata projection: tag a source once 7:25 - Mixed trust parents: allergy flags versus consent 8:57 - Deletion: GDPR erasure through the same edges 10:26 - Benefits: compliance, veracity, and debuggability 11:31 - Q\&A: cost, dedup, and why not just markdown --- Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/H7puB0RwJMM-citation-needed-provenance-for-llm-built-knowledge-graphs Join our upcoming AI Engineer conferences (see https://ai.engineer landing page): - AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney --- Canonical: https://liquidgold.cc/watch/H7puB0RwJMM · Channel: [AI Engineer](https://liquidgold.cc/channel/aidotengineer.md) > Search chunk text is held until public-source provenance is verified. ## Document chunks --- #### Chunk 0: Document excerpt (0:12 to 1:10) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 1: Document excerpt (1:10 to 1:47) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 2: Document excerpt (1:47 to 2:58) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 3: Document excerpt (2:58 to 4:27) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 4: Document excerpt (4:27 to 5:13) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 5: Document excerpt (5:13 to 6:08) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 6: Document excerpt (6:08 to 7:29) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 7: Document excerpt (7:29 to 9:01) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 8: Document excerpt (9:01 to 10:31) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 9: Document excerpt (10:31 to 11:37) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 10: Document excerpt (11:37 to 13:32) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 11: Document excerpt (13:32 to 15:16) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 12: Document excerpt (15:16 to 16:59) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 13: Document excerpt (16:59 to 18:30) > Held: transcript rights do not permit this chunk's text. --- #### Chunk 14: Document excerpt (18:30 to 20:54) > Held: transcript rights do not permit this chunk's text.