Skip to content

smuggled infinities

Where the term comes from Smuggled infinities is a frame Alex Komoroske attributes to his friend Ben Mathysse: the moment an infinity — something like perfect — enters an argument, everything downstream of it becomes absurd. The trap is easy to fall into when reasoning about AI agents: imagine the agent is perfect and it will never make a decision you dislike. But real capability is asymptotic, not infinite. At 99.99% reliability, the one $5,000 mistake is game over. [^bb2c42aa@4009] Why the trap matters for agents Many agent value propositions look like a logarithmic value curve crossing an exponential cost curve: amazing at first, underwater soon after. Arguments built on an assumed perfect agent hide the cost side of that curve — so evaluating agentic systems means asking where the infinity was smuggled in, and what the picture looks like at realistic, less-than-perfect reliability. [^bb2c42aa@4009] The mirror image: smuggled intelligence Komoroske coins a companion term, smuggled intelligence: when evaluating how smart an LLM is, beware smuggling your own intelligence into the test — who prompted it, what they gave it, how it was set up. The most important input to an LLM's output is the user; prompting skill is a real, distinct intuition with little to do with technical savviness. And unless you actively seek disconfirming evidence, the model will happily hand you confirming evidence for anything. [^bb2c42aa@4086] - Smuggled infinities: once an infinity like perfect enters an argument, everything downstream is absurd. [^bb2c42aa@4009]- Agent capability is asymptotic — at 99.99%, the one costly mistake is game over. [^bb2c42aa@4009]- Smuggled intelligence is the mirror trap: your own setup and prompting hidden inside an LLM evaluation. [^bb2c42aa@4086]- Seek disconfirming evidence; the model will not volunteer it.

glossary term

Ready

Loading…

2 min read