Skip to content

Pelican Bench

A deliberately absurd image-model probe Pelican Bench is a benchmark for evaluating image generation models, built around a deliberately strange prompt: render a pelican riding a bicycle and then inspect the details. Its creator, Simon, describes it as the strangest and most ridiculous reference point — and yet one that offers real value for exploring AI progress. How it shows up in the DevDay 2026 discussion Simon joined the DevDay 2026 live stream with the bench in hand, and it got a workout against the new Sol 6.1 model:- The rendered pelican came out well enough, but the bicycle was, in Simon's words, “garbage” — a surprisingly deformed frame.- Comparing across versions suggested a pattern: Sol 5.6 produced the right all-around bike shape, and quality dropped below that. But as Simon noted, you “roll the dice every time” — one bad render doesn't settle the comparison.- The test had run at the default thinking level, which hadn't been checked beforehand — a reminder that benchmark-style probes carry confounds worth exploring.The pelican test in miniature: ask a model to render a pelican on a bicycle, then look for legs on both sides of the frame, a neck that isn't too thick, and a coherent bicycle. Small, absurd, and surprisingly discriminating. - Pelican Bench is an image-model benchmark built on the absurd pelican riding a bicycle prompt.- It probes fine details — leg placement, neck thickness, bicycle frame coherence — that reveal quality differences between models.- Results are stochastic: every render is a fresh roll of the dice, so a single bad output doesn't prove a model is worse.

glossary term

Ready

Loading…

1 min read