---
schema: 1
type: term
title: '**Static evaluations**'
description: >-
  **Static evaluations** are fixed, benchmark-style tests of a model's
  capabilities — the kind of evaluation that, unlike dynamic head-to-head games,
  is easily saturated and can be _taught to the test_, so it can miss
  capabilities that gameplay-based evaluation exposes.
canonical: >-
  https://liquidgold.cc/channel/everyinc/explore/term/static-evaluations-0559fb476e00
markdown: >-
  https://liquidgold.cc/channel/everyinc/explore/term/static-evaluations-0559fb476e00.md
llms: null
parent: 'https://liquidgold.cc/channel/everyinc/llms.txt'
channel: everyinc
updated: '2026-10-23T22:12:35.346Z'
language: null
children: []
---


# \*\*Static evaluations\*\*

Canonical: https://liquidgold.cc/channel/everyinc/explore/term/static-evaluations-0559fb476e00

Parent: [Channel index](https://liquidgold.cc/channel/everyinc/llms.txt)

## Definition

**Static evaluations** are fixed, benchmark-style tests of a model's capabilities — the kind of evaluation that, unlike dynamic head-to-head games, is easily saturated and can be _taught to the test_, so it can miss capabilities that gameplay-based evaluation exposes.

In this channel's material, **static evaluations** come up when the hosts explain why they moved beyond benchmarks and into *games* as testing arenas. The argument: modern models are evaluated hands-on, and **games are dynamic** — they pit agents against each other head-to-head and *do not saturate the way benchmarks do*. Benchmarks like SWE-bench can end up being **taught to the test**, so scores stop telling you what a model can actually do. Dynamic game settings, by contrast, expose capabilities that static evaluations miss — for example, structured behaviors like map understanding and issuing orders correctly can be observed directly in play, and reading the resulting data reveals different play styles and model behavior.