---
schema: 1
type: term
title: _evals_
description: >-
  _evals_ are an organized system of tests and measurements used to check how
  well a model, tool, or product performs across different clients and models —
  as with the evals system Rattray builds across MCP clients (Cursor, Claude
  Code) and underlying models, or the evals Val Town eventually built for Townie
  so customer complaints could be checked against measured behavior rather than
  anecdote.
canonical: 'https://liquidgold.cc/channel/everyinc/explore/term/evals-891b54c98dcc'
markdown: 'https://liquidgold.cc/channel/everyinc/explore/term/evals-891b54c98dcc.md'
llms: null
parent: 'https://liquidgold.cc/channel/everyinc/llms.txt'
channel: everyinc
updated: '2026-10-23T22:12:35.346Z'
language: null
children: []
---


# \_evals\_

Canonical: https://liquidgold.cc/channel/everyinc/explore/term/evals-891b54c98dcc

Parent: [Channel index](https://liquidgold.cc/channel/everyinc/llms.txt)

## Definition

_evals_ are an organized system of tests and measurements used to check how well a model, tool, or product performs across different clients and models — as with the evals system Rattray builds across MCP clients (Cursor, Claude Code) and underlying models, or the evals Val Town eventually built for Townie so customer complaints could be checked against measured behavior rather than anecdote.

## What _evals_ are **_Evals_** are an organized system of tests and measurements used to check how well a model, tool, or product is actually performing. In this channel's material, the term comes up in two concrete settings: designing **MCP servers** and shipping an AI coding product like **Townie**.## Evals in MCP server design Alex Rattray argues that building an MCP server well takes real **product management** — talking to customers, watching how they work — plus serious engineering and an **evals system** that covers the whole matrix of clients (Cursor, Claude Code, and so on) and underlying models. Because the same server behaves differently across clients and models, the evals system is what lets you explore that matrix systematically rather than guessing.## Evals in a shipped product Steve Krouse describes the cost of skipping them: for the first three or four months of Townie, Val Town had **no real evals** — they would _YOLO_ changes to production, test manually, and rely on anecdotal reports. Users would report the model got _stupider_ over a weekend, and Krouse would take it to Twitter to ask if anyone else saw it. Once evals existed, he could check customer complaints against them — and if nothing showed up, confidently say _try again_. The catch he names honestly: you start only taking seriously what you can measure, and when a newer Sonnet shipped, their evals showed **no difference** — because none of their evals were hard enough.- **_Evals_ = an organized system for measuring how a model, tool, or product performs** — not a single test, but infrastructure for checking behavior.- Rattray ties evals to MCP server quality: an evals system must span the **matrix of clients and models**.- Krouse shows the downside of skipping them: anecdotal panic over perceived model degradation, and no way to check complaints — and the upside: once evals exist, complaints can be checked and answered with confidence.- Evals have limits: if they aren't hard enough, they can show **no difference** even when a new model ships.