Skip to content

prompt injection

How this channel uses the term Prompt injection appears in this channel's material as the security problem Alex Komoroske predicts will become widely known in the industry within 6–12 months — and he argues it may be the era's defining security issue [^759f5666-c356-59d0-8a71-9b53e5735bf7@3699]. His framing of why it is so serious contrasts it with an older problem: Why prompt injection is harder than what came before - SQL injection is child's play by comparison: SQL is highly regular, so escaping inputs largely obviates it. - LLMs are imminently gullible and make all text effectively executable. Any text an agent reads can act as instructions. - The combination is explosive: bring in untrusted text (emails, web pages) alongside tool use with irreversible side effects — and any network request can be an irreversible side effect. [^759f5666-c356-59d0-8a71-9b53e5735bf7@3699] The threat model the material emphasizes A concrete scenario from the source: a spammer's email that did not hit the spam folder can instruct an agent to send financial data to an attacker's site. The key point is that the threat model is not badly behaved MCP integrations — it is any context that flows in through them. Even if every MCP integration you installed is trustworthy, the text moving through the system is the attack surface. The proposed remedy and current state Komoroske argues that fixing prompt injection requires a different architecture at the security and privacy layer, and that MCP punts on the problem with a few preemptive dialogs. Large-scale attacks have not hit yet, in his view, only because we are still in the tinkering phase rather than wide deployment. [moment: 3699] The related ChatGPT operator anecdote shows the stakes even without malice: a reporter asked for cheap eggs, logged into Instacart, and walked away — 30 minutes later very expensive eggs arrived because the agent acted too eagerly. Agents taking irreversible actions is the precondition that makes injected instructions dangerous. - Prompt injection exploits gullible LLMs that treat any incoming text as executable instructions. - It becomes dangerous when untrusted text combines with tool use with irreversible side effects. - The threat is any context flowing through integrations, not just malicious integrations themselves. - Komoroske believes a fix requires a new security/privacy architecture; MCP currently punts on it. - Large-scale attacks haven't hit yet because we're still in the tinkering phase.

glossary term

Ready

Loading…

2 min read