Talk by Joshua Mo, a developer relations engineer at Venice AI, on how private data and secrets leak through AI agent systems even when the underlying model is private. It covers why privacy is about blast radius, how logs and secrets leak through a typical agent, and patterns such as hashed IDs, split-key custody, revocable handles, narrow agent scope, prompt-injection screening, keeping credentials in the execution layer rather than the LLM, sandboxes with permission monitoring, and confidential compute with attestations. It closes on the open problem of confidential tool calls. The Venice AI practices described are the speaker's account and are not independently verified in this record. # Channel Presentation Observations **Source:** "How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI" **Video ID:** 01a15ca6-a62e-7443-a467-4495ec1d564f **Metadata supplied:** None beyond title and transcript --- ## 1. Explicit Self-Description ### Speaker Identity (Stated in Video) - **Name:** Joshua Mo - **Current role:** "I'm currently the lead developer relations engineer at Venice AI." He lists responsibilities as partnerships, documentation, developer engagement, and "some engineering." - **Prior role:** "I was actually the lead maintainer for rig which is the leading AI framework in Rust." > "So currently I handle partnerships, documentation, talking to developers, um some engineering and other things other things as well." ### Platform Self-Description (Creator Claims) The speaker describes Venice as: - **"a private and unrestricted AI platform"** - **"100% open AI compatible"** - Hosting "the latest open-source models" - Offering a web app for conversations with its models - Offering a **studio** product for "content creators," covering video, audio, image generation, and transcription > "We don't store any prompts or conversations and we also have a web app that you can use to talk to our models as well." **Note:** These are creator claims about the platform. The talk does not demonstrate them, and no independent verification appears in the material. --- ## 2. Intended Audience Evidence suggests the talk targets **developers and engineering teams building agentic AI systems**. - **Technical depth:** Discusses observability (metrics, logs, traces), sandboxing with Firecracker and AWS EC2/VPC, hardware-level trusted execution environments (TEEs), and authorization using JSON web tokens and session keys. - **Tooling references:** Names specific developer tools, including "open claw," "cursor," "claude code," and "1Password"/"HashiCorp" vaults, which signals a practitioner audience. - **Direct address to builders:** > "So my personal suggestion is building your own agentic runtime." - **Explicit framing for engineering teams:** > "...engineering your prompts with privacy in mind can be uh very helpful for engineering teams who are using coding harnesses..." The greeting ("Hi everyone... welcome to my talk") indicates a live presentation format, likely a conference talk. --- ## 3. Claimed Scope The speaker frames the topic as **privacy in AI agent systems**, specifically how secrets and sensitive data leak and how to reduce that exposure. ### Topics Covered - **Business case for privacy** - Reputational risk from data breaches - Enterprise customer concentration risk - Reduced "blast radius" through less stored user data - **Agent architecture patterns** - Separating the reasoning layer from the execution layer - Keeping secrets out of the model - Narrow agent scope and tool minimization - **Data protection techniques** - Hashed identifiers instead of raw user data - Split-key custody encryption - Revocable handles for resource access - **Infrastructure** - Sandboxing and VM isolation - TEEs and attestation - **Open problems (self-acknowledged)** > "...there's definitely some work to be done." ### Scope Limitations Acknowledged by Speaker - Sandboxing "doesn't act as a cure all for your agentic systems." - Confidential tool calls remain "an ongoing challenge." --- ## 4. Presentation Patterns Observed ### Claims vs. Demonstrations | Category | Type | Example | |---|---|---| | Platform privacy (no prompt storage) | Creator claim | "We don't store any prompts or conversations" | | TEE attestation verifiability | Creator claim | "you can verifiably prove that your inference was run in a totally private process" | | Split-key custody protecting stored results | Creator claim | "attackers can't access your data and neither can the team should their accounts be compromised" | | General architecture patterns (hashing, handles, scoping) | Generic technical advice | Presented as applicable beyond Venice | | Logging risks | Generic technical observation | "you can also accidentally store prompts and secrets by default" | ### Venice-Specific Promotion The talk includes product references throughout, such as: > "...when you use Venice, you don't need to worry about any of that." The speaker also ties general principles back to Venice's practices (for example, "one thing that we've used at Venice is storing hashes"). This makes the talk partly **product-oriented** while still containing general engineering guidance. ### Delivery Characteristics - Conversational, informal tone with filler words and self-corrections - Acknowledges a pause mid-talk ("just give me a sec") - Ends with a brief closing ("Thank you for listening") --- ## 5. Limits of This Observation - This document covers **only this single video**. It does not establish channel-wide patterns. - The source material contains **no curated metadata**, so no additional channel identity context is available. - Platform privacy and security claims are **reported, not verified** by the material. # Delivery Observation — Joshua Mo, "How Secrets Leak Through AI Agents" ## Tone & Register - **Conversational, engineer-to-engineer framing** — the talk repeatedly appeals to shared professional knowledge: > "as we know data breaches are increasingly becoming an eventuality" > "So logging, as we all know, is actually pretty great for debugging" > "you guys know how it goes" - **Low-key, matter-of-fact delivery** with frequent discourse fillers: *"uh," "um," "so," "of course"* appear throughout, e.g.: > "Uh so what is Venice? So Venice is a private and unrestricted AI platform." - **Occasional informal asides and self-corrections**, including a visible stumble near the end: > "However, um [laughter] sorry, just give me a sec." > "Um, oh my god. So the reasoning layer will then make an API request..." ## Vocabulary & Terminology Habits - Dense, consistent security/infra vocabulary: **split key custody, trusted execution environments, blast radius, attestation, enclave, non-deterministic, exfiltrated, sandboxing**. - Recurring adverbs of degree that soften or intensify claims: > "significantly upgrade the security of your agent" > "performance can actually degrade uh considerably" > "This is still kind of an ongoing challenge." - Repeated hedge-and-certify pattern — claims paired with qualifiers: > "it's much more reasonable to say that it's private and secure rather than feeding your password... directly into the reasoning layer" - "So" as a pervasive section-opener — it introduces nearly every slide transition: > "So why does privacy matter?" / "So how does Venice do privacy today?" / "So let's talk about..." / "So let's look at keeping secrets out of the model." ## Recurring Phrases & Structural Markers - **"As well as that"** is the dominant additive transition, used at least four times: > "As well as that, storing less use uh user data directly reduces the blast radius..." > "As well as that, we encrypt data and store it in an inaccessible enclave..." - **"For example" / "So for example"** drives nearly all explanations — the speaker almost never asserts a pattern without an immediately following scenario: > "So for example, let's say you're you're using open claw." > "So for example, let's say you have a database pool using a handle." - **"Let's say"** is the standard hypothetical construction, appearing in nearly every threat-model walkthrough. - **"Let's talk about" / "let's look at"** as explicit roadmap language between sections. - Repeated closure frames: "there's still some work to be done" (used twice for confidential tool calls). ## Teaching Habits - **Define-then-apply pattern**: each concept is first defined in plain terms, then applied: > "So a handle is simply an object that grants you access to something. So for example, let's say you have a database pool..." > "So traces are simply a span of work uh done over a unit of time." - **Enumerated breakdowns of abstract concepts**, e.g. decomposing observability into "metrics, logs and traces" and privacy into "predefined tiers." - **Concretizes risk with vivid, specific stakes**: > "maybe you lose the enterprise customer who's been fueling almost 90% of your annual revenue" - **Naming coinage**: introduces a personal term for a concept — > "there's something I like to call confidential compute" - **Honest limitation disclosure**: explicitly flags immature areas rather than overselling: > "in terms of oh confidential tool calls there is still some work to be done" ## Communication Patterns - **Alternating "problem → Venice solution → general pattern" rhythm**: threat descriptions frequently pivot to product claims ("However, when you use Venice, you don't need to worry about any of that"), then broaden to general engineering guidance. - **Second-person address throughout** ("you," "your agents," "your context window"), keeping the audience in the position of the system builder. - **Rhetorical-question transitions**: > "So the real question is how can we optimize for this kind of problem." ## Uncertainty Notes - The transcript appears auto-generated with model-name garbling ("GLM 5.2 and Kimmy 2.7," "clawed code," "NCP server," "quadrant lance db"), so exact tool/product names should not be treated as verbatim. - "As well as that" and "So for example" repetition is high enough to be called stable; single-occurrence quirks (e.g., the mid-talk stumble at ~[870]) are isolated moments, not patterns. - Speaker identity and company affiliation are taken from the transcript itself and not independently verified. # Observation: Secrets Leakage and Privacy in AI Agent Systems ## Source Overview - **Title:** "How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI" - **Speaker:** Joshua Mo, identified in the transcript as "lead developer relations engineer at Venice AI," and previously "lead maintainer for rig, which is the leading AI framework in Rust" - **Format:** Conference-style talk, with a short Q&A-free closing - **Speaker affiliation:** Venice AI, which the speaker presents as the platform being discussed. Claims about Venice are therefore self-reported by a company representative. --- ## Central Thesis The speaker argues that: > "So the real question is how can we optimize for this kind of problem." The problem is that **agentic systems are more prone to data leakage and breaches than models alone**, because a complete agent must be built as a whole process around the LLM. The talk presents several architectural and hardware-level approaches to reducing leakage, and frames the problem as partly unsolved. --- ## Claims About Venice AI (Self-Reported) ### Platform Description - Venice is described as a "private and unrestricted AI platform." - It hosts "the latest open-source models." - It is described as "100% open AI compatible." - It states it does not store prompts or conversations. - It offers a web app and a studio product for content creators (video, audio, image generation, and transcription). ### Privacy Tiers and Mechanisms The speaker describes several mechanisms: 1. **No prompt storage:** Conversations are not saved on Venice's side. For open-weight models such as the ones named (transcribed as "GLM 5.2" and "Kimmy 2.7"), the speaker says conversations "will be all stored locally." 2. **"Private by default" tier:** For third-party providers such as OpenAI and Claude, the speaker says the above measures "apply," and that user footprint is anonymized so users "can't identify you from your simply from your prompt or your request." 3. **Trusted execution environments (TEEs):** Some models run in TEEs, which the speaker says lets users "verifiably prove that your inference was run in a totally private process." 4. **Encrypted media payloads:** For asynchronous endpoints such as video generation, results are stored in an encrypted payload using **split key custody**, so that "attackers can't access your data and neither can the team should their accounts be compromised." > The speaker's own framing: "not all systems are made equal… Some companies may claim their systems are private, but they're not really." No specific competitor is named for this comparison. --- ## Why Privacy Matters: Reasoning Given The speaker gives two main lines of reasoning: - **Reputational risk:** "You only need one data breach before your users stop trusting you and they move on to another provider." - **Enterprise revenue risk:** Losing one enterprise customer "who's been fueling almost 90% of your annual revenue" is presented as a plausible consequence. The 90% figure is stated as an example and is not sourced. - **Blast radius:** "Storing less user data directly reduces the blast radius of a security incident." --- ## Observability and Logging **Reasoning:** - Observability comprises metrics, logs, and traces. - Logging is useful for debugging, "especially in production or dev environments." - Logging can capture prompts, secrets, and personal information "by default if you're not careful." **Claim about Venice:** "When you use Venice, you don't need to worry about any of that." This is an assertion about the platform's own logging practices and is not independently verified in the talk. --- ## How Secrets Flow Through an Agent The speaker describes a typical path, using OpenClaw as an example: 1. A user request reaches an **agent runtime**. 2. The runtime fetches a secret from a password vault (1Password or HashiCorp Vault, transcribed as "hashc vault"). 3. The secret is passed to the **LLM reasoning layer**. 4. The reasoning layer makes an API request through a tool call. 5. The answer returns to the user. **Stated risk:** Because of this architecture, "the secrets get fed directly to the LLM, which means it then may get used in training data." > Note: The speaker hedges here with "may." The talk does not establish that any specific provider trains on such inputs. --- ## Privacy-Preserving Patterns Proposed ### 1. Hashed Identifiers - Store hashes representing needed IDs instead of raw personal data. - Example: rate limiting can identify a user without using their direct footprint. ### 2. Split Key Custody - Encryption keys are split into a **read half** and a **write half**. - Per the speaker, an attacker "cannot access the data due to needing both halves." - Data is stored in an "inaccessible enclave." ### 3. Revocable Handles - A handle is "an object that grants you access to something." - Example: a database pool is accessible only when the handle is held. - Described as especially relevant for JWTs or session keys. ### 4. Privacy-Aware Prompt Engineering - Relevant for teams using coding harnesses such as Cursor and Claude Code. - Focus: **narrow agent scope** to reduce risk. ### 5. Narrow Tool Scope to Limit Exfiltration - Example: an MCP server (transcribed as "NCP") may expose 15–20 tools, but only one is needed. - If an API key is pulled from a vault and passed into the agent context, it "can" be exfiltrated through a tool call to a malicious web server. - The speaker presents this as a reason to limit both tool count and the secrets exposed to the agent. ### 6. Classification Layer for Prompt Injection - Placed in front of the agent. - Options given: - An LLM-based classifier - Semantic search, e.g., vector search via Qdrant or LanceDB - An in-memory process - The speaker does not evaluate the trade-offs among these options. ### 7. Context Hygiene (Performance Side Benefit) - Narrow scope produces "tight context loops," which the speaker says can improve performance. - Reasoning: performance "can actually degrade considerably" as the context window fills, so "keeping an empty or near empty context window" helps. - This performance claim is asserted, not measured in the talk. --- ## Keeping Secrets Out of the Model **Core proposal:** Tie the password vault to the **agent runtime**, not the **reasoning layer**. **Reasoning given:** - The reasoning layer produces "non-deterministic inputs." - The runtime becomes an **execution layer**, which the speaker calls deterministic. - Deterministic execution permits controls that "you normally wouldn't be able to do for privacy and security reasons at the LLM layer." **Example:** For a browser tool that must log into a site, the speaker argues it is "much more reasonable to say that it's private and secure" to keep credentials out of the reasoning layer. --- ## Sandboxing and Its Limits **Claim:** Sandboxes and VMs (e.g., Firecracker, AWS EC2 in a VPC) "can significantly upgrade the security of your agent." **Qualification:** This is "not a cure all." For example, if an agent has internet access, it can still exfiltrate data through external APIs, so **monitoring permissions is essential**. **Personal suggestion (the speaker's own view):** Build your own agentic runtime, with custom steps, observability, and tools, so that you "own the execution layer and reasoning layer." --- ## Hardware-Level Privacy (Confidential Compute) **Claim:** A trusted execution environment (TEE) enables inference to run privately at the hardware level. **Venice's implementation, as described:** - On each request, Venice returns an **attestation**. - The user can verify that attestation with Venice's attestation verifier. - This allows the reasoning layer to be "isolated… cryptographically and prove[n]." --- ## Unresolved Problems and Acknowledged Limits The speaker repeatedly signals incomplete work: | Area | Speaker's Statement | |---|---| | Confidential tool calls | "there is still some work to be done" | | TEEs for tools | "still great for tool calls" (affirmed), but not sufficient alone | | End-to-end encryption | Work remains, due to the need for an ergonomic API while still allowing fully encrypted tool calls | | Overall | "This is still kind of an ongoing challenge." | **Possible direction (stated tentatively):** "a very reliable solution would be to allow providers to own the compute layer at which your tool calls being run." This is presented as a possibility, not an implemented result. **Closing:** "In terms of confidential tool calls… there's definitely some work to be done." --- ## Tensions and Qualifications to Retain - **Self-interest:** The speaker works for Venice, and the talk promotes Venice's approach. Product claims ("you don't need to worry about any of that") are vendor assertions. - **Hedged vs. firm claims:** Training-data risk is hedged ("may"). Venice's privacy guarantees are stated firmly. The talk does not reconcile these differences. - **Partial solutions:** Sandboxing is described as helpful but insufficient. TEEs are described as strong for inference but incomplete for tool calls. - **Unverified figures:** The 90% revenue example and the performance-degradation claim are not sourced in the talk. - **Competitor criticism:** The speaker says some companies overclaim privacy, without naming any. --- ## Transcript Notes - Model names and some technical terms appear to be transcription errors: "Kimmy 2.7" (possibly Kimi), "GLM 5.2," "NCP server" (possibly MCP), and "hashc vault" (possibly HashiCorp Vault). They are recorded here as transcribed, except where the speaker's intended referent is clear from context. - Audio artifacts include "Oh my god" and an audible laughter tag during a pause in the TEE section. These are not substantive claims. # Speaker Attribution and Provenance Record **Video ID:** 01a15ca6-a62e-7443-a467-4495ec1d564f **Title:** How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI **Format:** Single-speaker presentation (talk) --- ## 1. Speakers and Roles ### 1.1 Joshua Mo (Presenter) - **Self-identified role:** Lead developer relations engineer at Venice AI - **Stated responsibilities:** Partnerships, documentation, developer communication, and some engineering - **Prior role (self-reported):** Lead maintainer of **rig**, described by the speaker as "the leading AI framework in Rust" - **Speaking role:** Presenter and sole speaker. Introduces himself at the start and gives the closing remarks. - **Affiliation note:** The speaker is affiliated with Venice AI. Statements about Venice's products, practices, and comparisons are made by this affiliated speaker. No other speakers are identified in the transcript. No guests, moderators, or narrators are named. --- ## 2. Promotional Segments The following segments promote Venice AI or its offerings. They are flagged here as promotional based on the speaker's affiliation and the nature of the content. > ⚠️ **Promotion flag: Venice AI platform description** > Describes Venice as "a private and unrestricted AI platform," states it hosts open-source models, is "100% open AI compatible," offers a web app, and has a "studio offering" for content creators. > ⚠️ **Promotion flag: Venice privacy practices** > Claims include: > - Venice does not store prompts or conversations. > - Privacy tiers exist, including "private by default." > - Models run in trusted execution environments, with claimed attestation that inference ran in a private process. > - Encrypted storage of generated results using "split key custody." > - Statement that when using Venice, "you don't need to worry about" logging-related leakage. > ⚠️ **Promotion flag: Comparative statements** > The speaker states that "some companies may claim their systems are private, but they're not really," without naming any company. --- ## 3. Referenced Sources and Named Third Parties The transcript names the following external products and services. These are referenced only. No claims about their behavior are attributed to a published source. **AI models and providers** - GLM 5.2 (transcribed as "GLM 5.2") - Kimi 2.7 (transcribed as "Kimmy 2.7") - OpenAI (referred to as "open AI") - Claude (referred to as "claude") **Agent tools and frameworks** - OpenClaw ("open claw"), used as an example agent - rig (the speaker's former project) - Cursor and Claude Code, referred to as "coding harnesses" - An MCP server (transcribed as "NCP server"), described as offering 15 to 20 tools **Secrets management** - 1Password (referred to as "one password vault") - HashiCorp Vault (transcribed as "hashc vault") **Vector and storage tools** - Qdrant ("quadrant") - LanceDB ("lance db") **Infrastructure** - Firecracker (microVM) - AWS EC2 and VPC **Attestation** - Venice's own attestation verifier, referenced but not named externally --- ## 4. Third-Party and Unattributed Claims Each claim below is recorded as stated. None is sourced in the transcript. - **Enterprise revenue concentration:** An enterprise customer is described as "fueling almost 90% of your annual revenue." The speaker does not say whose revenue this is. It is a hypothetical framed as "maybe you lose the enterprise customer," so it is not an attributed statistic. - **Breach frequency:** "Data breaches are increasingly becoming an eventuality." No source given. - **Context window performance:** "It's well known that… performance can actually degrade considerably" when the context window is near its limit. No source given. - **Sandboxing effectiveness:** "It's a very well-known fact" that sandboxes and VMs significantly upgrade agent security. No source given. - **Comparative claims about other providers:** Statements that OpenAI and Claude are covered by Venice's "private by default" measures. These describe Venice's handling of those providers, not the providers' own practices. --- ## 5. Evidence Channels and Availability Limits - **Transcript:** Available in full. It contains auto-transcription artifacts, including mistranscribed product names (for example, "hashc vault," "NCP server," "Kimmy 2.7," "affforementioned"). Some passages are garbled or uncertain. - **Transcript interruption:** The transcript contains a bracketed note, "\[laughter\]," and an unscripted pause ("just give me a sec") near the end. Some speech may be lost at this point. - **Slides, demonstrations, or screen content:** Not present in the transcript. Any visual references, such as diagrams described as "here we have," cannot be verified against visuals. - **Curated metadata:** Only the title and video ID were supplied. No description, chapter list, speaker list, or citation list is available. - **Comments:** Availability is not indicated in the source material. Comment content is not available for review. - **Upload date, event context, and publisher verification:** Not supplied. --- ## 6. Provenance Summary | Category | Status | |---|---| | Named speakers | 1 (Joshua Mo) | | Speaker affiliation | Venice AI (self-stated) | | Promotional segments | Flagged (platform, privacy practices, comparisons) | | Named external sources | Listed in Section 3; none cited with links or documents | | Unattributed claims | Listed in Section 4 | | Transcript completeness | Partially uncertain (artifacts and a passage with laughter or pause) | | Visual evidence | Not available | | Metadata beyond title | Not supplied | | Comment availability | Not indicated | Hi everyone. Uh, welcome to my talk. I'm Joshua Mo and today we're going to be talking about some of the challenges of building a privacy respecting AI system. So just for an introduction as to who I am, I'm currently the lead developer relations engineer at Venice AI. So currently I handle partnerships, documentation, talking to developers, um some engineering and other things other things as well. Uh so prior to this I was actually the lead maintainer for rig which is the leading AI framework in Rust. Um so what is Venice? So Venice is a private and unrestricted AI platform. Uh so we host all the latest open-source models. We're 100% open AI compatible. We don't store any prompts or conversations and we also have a web app that you can use to talk to our models as well. We also have a studio offering that is um a offering for our content creators that includes video, audio, image generation and transcription. So why does privacy matter? So let's d deep dive into this. So as we know data breaches are increasingly becoming an eventuality which means that essentially reputational risk matters. You only need one data breach before your users stop trusting you and they move on to another provider. Or if you're enterprise, maybe you lose the enterprise customer who's been fueling almost 90% of your annual revenue. So in this case, sensitive data can severely increase the impact of any security incident. As well as that, storing less use uh user data directly reduces the blast radius of a security security incident that happens. So, how does Venice do privacy today? Well, there's a few ways. The first way is that we don't store any prompt conversations, of course, as we mentioned previously, which means that any digital footprint incurred in your side is is essentially reduced to zero. So, when you use models like GLM 5.2 and Kimmy 2.7, you can guarantee that none of your conversations will be saved on our side. It will be all stored locally. We also have predefined tiers of privacy. So private by default means that the affforementioned uh measures apply for providers like open AI and claude. Uh we anonymize your user footprint so that none of well they can't identify you from your simply from your prompt or your request. We also have models hosted in trusted execution environments, meaning that you can verifiably prove that your inference was run in a totally private process. As well as that, we also use a privacy first software architecture. So for endpoints like video uh generation where you have to pull the endpoints, we store the results in a totally encrypted payload using split key custody, meaning that attackers can't access your data and neither can the team should their accounts be compromised. So I just want to introduce you to the guys the uh the idea that not all systems are made equal. So, not all systems are made private. Some companies may claim their systems are private, but they're not really. Uh, you guys know how it goes. So, while models are a very important part of an aentic system, in fact, they're a vital part of the system. Um, they are still only one part. And in fact to create a complete system to accompany an LLM in doing a task you have to build an entire process around it. However, these systems are also much more prone to data leakage and breaches. So the real question is how can we optimize for this kind of problem. So one example I just wanted to bring up was observability logging. So observability as many of you know is a core part of any serious production system today. So just to break it down quickly, observability can be broken down into metrics, logs and traces. So traces are simply a span of work uh done over a unit of time. Metrics as are simply numbers like counters, gauges for instance and logs are the actual messages. So logging, as we all know, is actually pretty great for debugging, especially in production or dem dev environments. But you can also accidentally store prompts and secrets by default if you're not careful, which can also hold personal information. However, when you use Venice, you don't need to worry about any of that. So let's talk about how secrets can flow through a regular agent. So for example, let's say you're you're using open claw. You send a user request that gets sent to an agent runtime. The runtime might need a secret that it will fetch from a one password vault for instance or hashc vault. The secret will then get sent to the LLM reasoning layer. Um, oh my god. So the reasoning layer will then make an API request through a tool call which will then return the answer back to the user. However, due to this the way that it's architected, the secrets get fed directly to the LLM, which means it then may get used in training data. So in terms of privacy preserving patterns that work, one thing that we've used at Venice is storing hashes that rep represent ids that we need as opposed to direct data. So for example, if you need to identify a user for rate limiting purposes, you might not want to use their direct user footprint directly. You might want to, you know, store it as a hash for instance, just so you can identify them without directly using any of their personal data. As well as that, we encrypt data and store it in an inaccessible enclave using split key custody. So for context, encryption keys are stored in two halves, a read half and a write half. By splitting these, the attacker cannot access the data due to needing both halves for the decryption to complete. Additionally, you can use design patterns like handles that can be easily revoked. So a handle is simply an object that grants you access to something. So for example, let's say you have a database pool using a handle. So when you have the handle, you can access the database pool. When you don't, then you can't access it. This is very important especially when you're using authorization like JSON web tokens or session keys that may be uh a core part of your authorization system. As well as that, of course, engineering your prompts with privacy in mind can be uh very helpful for engineering teams who are using coding harnesses like cursor and clawed code in terms of protecting agent handle data. So essentially ensuring narrow agent scope to to decrease risk. So for example, let's say you pull tools from an NCP server that may have 15 20 different tools. you don't really want to use all of them. You actually only really need one, but you also need an API key to handle that uh API call. So if you pull that API key in from you know your your password vault and then you send it as part of the agent um what can happen is that it gets exfiltrated through through the agent uh on a tool hall to whatever you know web server is being hosted by a bad actor for instance. As well as that um we also have a classification layer in front of an agent to check for prompt injection for example. So there's a couple of ways you can do this. So you can either use an LLM based uh layer to do this or you can alternatively use a semantic um search layer for example using vector search uh via something like quadrant lance db or additionally you can use an in-memory process. It should also be noted that tight context loops caused by narrow agent scope can also improve context window performance. So for example, because you don't have as many tools, uh your agents may be able to perform much better. It's well known that when you fill your context window, by doing so, when you reach near the limit of your context window, performance can actually degrade uh considerably. By keeping an empty or near empty context window, we can make sure that the agent is performing at its best. So let's look at keeping secrets out of the model. So here we have a user request that gets sent to the reasoning layer. This reasoning layer is actually then uh formulating a series of non-deterministic inputs for an agentic runtime. So by tying the password vault to the agentic runtime instead of the reasoning layer, what happens then is that the run the agent runtime is essentially turned into an execution layer which then we can you know we can consider it deterministic. We can do all kinds of things that you normally wouldn't be able to do for privacy and security reasons at the LLN layer. Uh so for example, let's say you want to use a uh browser user tool and you need to log into a into a website that requires your credentials. With this uh software architecture, it's much more reasonable to say that it's private and secure rather than feeding your password see uh directly into the reasoning layer as well as you know calling external APIs. So in terms of sandboxing your agents um it it's a very well-known fact that using sandboxes and VMs for example firecracker um and you know AWS EC2 in a VPC can significantly upgrade the security of your agent. However this doesn't act as a cure all for your agentic systems. Uh so for example, if you have internet access enabled for your agent, it can still accidentally exfiltrate data through external API access if you're not careful and therefore monitoring permissions is essential. So my personal suggestion is building your own agentic runtime. So essentially building your own coding harnesses where you can add your own steps, your own observability, uh your own tools, things of that nature. Essentially owning the execution layer and reasoning layer. So let's talk about privacy at the hardware level very quickly. So in terms of this um there's something I like to call confidential compute where a trusted execution environment essentially uh at the hardware level enables the inference process to be run privately. So with Venice, how this works is that when you send a request, uh you get a response and then we actually return an attestation that you can then um verify with our atestation verifier. So in terms this what this actually means is that you can fully isolate an agent at the reasoning layer cryptographically and prove it in terms of oh confidential tool calls there is still some work to be done um tees are still great for tool calls however we also offer end to end encryption um but there's also you know some work to be done there because of uh the need to provide an ergonomic API I experience while still allowing users to return entirely encrypted tool calls. This is still kind of an ongoing challenge. Um in terms of this, a very reliable solution would be to allow providers to own the computes layer at which your tool calls being run. However, um \[laughter\] sorry, just give me a sec. So in terms of confidential tool calls um yeah there's definitely some work to be done. Um yeah that's it. Thank you for listening.