chaffchaff

static analysis · zero network

95.3% of this context file reloads on every single turn.

Static analysis for CLAUDE.md, AGENTS.md and tool definitions. Every enforced rule ships with the eval that measured its effect on behavior.

Analyze a file in your browserRead the rule catalog

✓ runs in a Web Worker✓ nothing uploaded✓ MIT

chaff analyze · case-study fixturebuild-time run

A real profile, not a mockup

6,197 ref tokens across 3 file(s), o200k_base

resident5,904 · 95.3%

2 file(s) · loads into context on every single turn

on-demand184 · 3%

1 file(s) · loads only when a path-scoped rule matches

dropped109 · 1.8%

past a documented read limit · silently cut every session, never loads

Most teams have never separated these three piles. Computed from the committed case-study fixture, the same way the full case study is.

95.3%reloaded every turnunscoped, on the fixture set
109tokens silently cutpast a documented read limit
6ship enforcedstructural, no eval needed
0measured so farwe would rather say that

the three piles

Resident vs. on-demand vs. dropped

Your CLAUDE.md loads on every turn. A skill body loads only when it routes. A .claude/rules/*.md file with paths: loads only for matching files. Content past MEMORY.md’s documented read limit does not load at all.

drawn to scale from the same analysisregenerated and diffed by CI — scripts/diagram.mjs
Resident, on-demand, and dropped tokens: a real chaff profile A horizontal bar chart with three bands, drawn to scale from a real chaff analysis of the committed fixtures/case-study/before context set (o200k_base tokenizer, --today 2026-08-09). Resident: 5,904 ref tokens across 2 file(s); this content loads into the model's context on every turn, filled solid. On-demand: 184 ref tokens across 1 file(s); this content loads only when a path-scoped rule matches the file being worked on, drawn as an outline. Dropped: 109 ref tokens; content past MEMORY.md's documented 200-line read limit that is silently cut and never reaches the model at all, filled amber, the one amber element in this diagram. Bars are linearly proportional to token count against a shared 0-6,000 scale; the two smaller bands use a minimum visible width for legibility, and the exact count is always printed as a label regardless of bar width. Reproduce with chaff analyze --today 2026-08-09 against fixtures/case-study/before. CHAFF'S OWN CASE-STUDY FIXTURE, BEFORE THE FIX resident vs. on-demand vs. dropped — o200k_base ref tokens, one real file set RESIDENT loads on every turn 5,904 tok · 2 files ON-DEMAND loads only when a rule path matches 184 tok · 1 file DROPPED past the read limit, never reaches the model 109 tok 0 1,000 2,000 3,000 4,000 5,000 6,000 ref tok (o200k_base)

what is allowed to fail a build

Two tiers gate a build. Everything else is labelled.

enforced · fails CI

Either a structural fact — a broken import, or content past a documented read limit, both deterministic and needing no eval — or a measured behavioral claim backed by a committed, reproducible eval.

6structural
0measured

The honest count, today

17 rules in the v1 catalog: 6 structural facts enforced today, 8 heuristics labelled unmeasured in every line of output and excluded from the chaff score, and 3 planned measured-rule candidates — specified in full (three-arm design, ≥12-item battery, seeded bootstrap admission test) but not yet run, because running them costs real API spend and needs explicit approval.

0 of 17 rules are measured so far. We would rather say that than pad the enforced tier. The rule catalog keeps a graveyard for rules that do not survive their eval.

the real thing, recorded

Not a screencast.

A scripted run against the live site: paste a file with a broken @-import into /analyze, press Analyze, watch the finding appear.

The corner badge is injected only for the recording and counts real network requests live. After the first analysis loads this site’s own worker code (same-origin GET, expected), the upload/cross-origin count stays at zero for the rest of the run — which is the actual privacy claim (worker source). Recorded with scripts/record-demo.mjs.

chaff's /analyze page, before the recorded run starts

Your system requests reduced motion, so autoplay is off. Watch the recording (webm, plays on click, still muted and silent).

chaff, on chaff

It holds itself to the bar it enforces.

The root chaff.config.json declares this repo’s own working agreements (CLAUDE.md) as the set to analyze, and evals/dogfood.eval.test.ts asserts --fail-on enforced exits 0 on every CI run.

chaff analyze --today 2026-08-08

resident

666

this file, o200k_base ref tok

findings

0

enforced or heuristic

chaff %

0.0%

clean

pre-release

Install

bash
git clone https://github.com/jamessuuu/chaff
pnpm install && pnpm --filter chaff build
node packages/chaff/dist/cli/main.js analyze CLAUDE.md
# npx chaff@latest analyze (once the package publishes at M7)

status

The CLI is not on npm yet — the workspace-local build above is the real path today. See the README for current status, or the docs for a five-minute quickstart.