---
title: "Evals: do coding agents build better sites with void?"
description: "How void is evaluated: the same model and brief with and without void, scored by void audit, lint and a blind design judge. Results and method."
canonical: https://void-design.vercel.app/evals
lastModified: 2026-09-17
---

# Evals

The same model gets the same brief twice, once without void and once with it. Both sites are audited by the CLI and judged blind on screenshots.

Results generated 2026-09-17T21:26:21.019Z.

## Tasks

- **Conference site** (40 min, ≤ 200 turns): Northbound 2026, a two-day web-infrastructure conference in Edinburgh. Two-day schedule, speakers, tickets, venue and travel, FAQ; image-light.
- **Designer-engineer portfolio** (40 min, ≤ 200 turns): Mara Okafor, a Lisbon studio for climate and science interfaces. Work, writing with an article page, about, contact, tasteful motion.
- **SaaS landing page** (40 min, ≤ 200 turns): Tidewater, a Postgres schema-drift and safe-migration tool. Hero, how it works with a CLI example, three pricing tiers, FAQ, footer, SEO metadata.

## Conditions

- **baseline**: Claude Code headless (`claude -p`) in an empty directory with the brief.
- **void**: identical flags, prompt and limits, plus `--plugin-dir <void snapshot>`.

## Composite score (0–100)

| Component | Weight | Measured by |
| --- | --- | --- |
| Design judge | 30% | Blind LLM judge on desktop and mobile screenshots, 7 anchored dimensions, median of 3 samples |
| Performance | 20% | void perf, mobile and desktop, median of 3 runs |
| Accessibility | 15% | void a11y: axe WCAG 2.2 AA, focus, target size, reflow |
| Smoothness | 10% | void smooth: scripted scroll and interactions |
| Lint | 10% | void lint on the generated source |
| Design heuristics | 5% | void design: type, colour, spacing and slop tells from computed styles |
| SEO | 5% | void seo |
| AI search | 5% | void geo |

A failed build scores 0. The composite is multiplied by (0.5 + 0.5 × the judge's required-checklist rate).

## Judge dimensions

- `hierarchy_typography`: How well the type guides the eye and reads.
- `layout_rhythm`: Grid, spacing, alignment and pacing down the page.
- `color_cohesion`: Palette, contrast and the consistency of the visual language.
- `originality`: Does it look designed for this client, or like a generic AI or template output?
- `responsiveness`: Judge from the mobile screenshots.
- `polish_detail`: The small things.
- `task_fulfilment`: How completely and faithfully the site covers the brief and the required checklist.

## Limitations

- Samples are small: two to five trials per cell. Single-cell differences are anecdotes; look at pairwise agreement and the paired difference across tasks.
- An LLM judge is not a human panel, and may favour busier designs. Screenshots are committed for spot checks.
- The void arm is told to run void audit, the same instrument that scores performance, accessibility and SEO. That is the intended workflow, but it favours void on those categories. The design judge and the pairwise comparison don’t share that coupling.
- Screenshots are static, so motion quality is only partly judged.

User settings, hooks, skills, MCP servers and memory are excluded; every trial asserts its isolation from the session’s init event and is discarded if it fails. The plugin is a frozen snapshot per run, recorded with its git SHA. The judge runs isolated too, and sees neutrally named screenshots, the brief and the rubric: never the transcript or the condition.

| Task | Condition | n | Composite | Build | Cost |
| --- | --- | --- | --- | --- | --- |
| SaaS landing page | baseline | 2 | 68 | 100% | $1.11 |
| SaaS landing page | void | 2 | 92 | 100% | $3.46 |
| Designer-engineer portfolio | baseline | 2 | 54 | 100% | $3.22 |
| Designer-engineer portfolio | void | 2 | 80 | 100% | $3.92 |
| Conference site | baseline | 2 | 70 | 100% | $1.35 |
| Conference site | void | 2 | 80 | 100% | $2.82 |

## Notes from the run

- Agents: claude-sonnet-5 via Claude Code 2.1.274 (Claude Code) (headless, identical flags; only --plugin-dir differs). Judge: claude-opus-5 (absolute median of 3 + pairwise in both orders).
- Plugin snapshot: git 1cb7931, skills hash aa364d43f0a9da5c.
- Composite = 0.3·judge_design + 0.05·design_heuristic + 0.2·perf + 0.1·smooth + 0.15·a11y + 0.05·seo + 0.05·geo + 0.1·lint; failed build = 0; × (0.5 + 0.5 × required-checklist rate).
- 2 trial(s) per cell; 95% CIs are percentile bootstrap over trials and are wide at this n.
- Isolation: identical non-void skill sets in both arms; no MCP servers; no user-level skills.
- void arm: 6/6 runs invoked at least one void skill.

Harness and methodology: https://github.com/ParthKapoor-dev/design/tree/main/evals