smuggled intelligence
Where the term comes from
Smuggled intelligence is Alex Komoroske's coinage — the mirror image of his friend Ben Mathysse's frame of smuggled infinities [^bb2c42aa-478f-5c9a-ac26-e83a54c63127@4009]. Where a smuggled infinity breaks an argument by importing something impossible (like an agent that is perfect), a smuggled intelligence corrupts an evaluation by importing something unaccounted for: the evaluator's own intelligence.
How it is used in this channel
In the interview, the term names a trap to avoid when judging how smart an LLM is: beware smuggling your own intelligence into the test — who prompted it, what they gave it, and how it was set up all shape the output [^bb2c42aa-478f-5c9a-ac26-e83a54c63127@4086]. Komoroske emphasizes that the most important input to an LLM's output is the user, and that prompting skill is a real, distinct intuition with little to do with technical savviness. The practical upshot he draws: if you are not actively seeking disconfirming evidence, the model will happily hand you confirming evidence for anything.
- Smuggled intelligence = your own intelligence leaking into an LLM evaluation via prompting, setup, and inputs.- It pairs with smuggled infinities as a two-part guard against over-crediting agent capabilities.- Prompting skill is real and distinct from technical savviness — it is part of what you are measuring.- Seek disconfirming evidence, because the model will not volunteer it.