Product

One comparison. Every failure kept visible.

StructTrace joins baseline and candidate outputs by case identity, evaluates both against the same contract and correctness rules, and explains exactly what changed.

Try the workflow Read the guide

The evaluation path.

Recorded files are the Stable candidate. Command and Python adapters are Beta. Direct OpenAI-compatible execution remains Experimental.

Cases + expected→Baseline + candidate→Strict evaluation→Paired evidence→Gate + replay

Watch the real product path.

The 71-second capture uses the release binary and embedded local UI. It contains no composited product screen or hosted evaluator.

The actual local workflow.

These images are deterministic Playwright captures from the capability-protected Rust server and embedded production UI.

StructTrace recorded-output source setup screen — **Source setup**Golden, baseline, candidate, and unchanged external schema remain visibly distinct.

StructTrace deterministic correctness rule builder — **Correctness builder**Rules are selected against fields discovered from the full bounded sources and schema.

StructTrace paired case explorer filtered to regressions — **Paired cases**Search and filters lead from aggregate evidence to the exact regression rows.

StructTrace case detail showing baseline and candidate evidence — **Case evidence**JSON-aware differences, evaluator facts, and evidence integrity are retained together.

What the engine measures.

No single validity number substitutes for the rest.

Strict parsing

The entire trimmed output must be one JSON value. Duplicate keys are rejected at every untrusted JSON boundary.

External validity

The unchanged caller-facing JSON Schema is validated with remote retrieval disabled.

Task correctness

Deterministic rules score exact fields, normalized text, dates, numbers, keyed arrays, tool arguments, and invariants.

Deployment success

Operational, structural, and semantic failures stay in the full denominator.

Paired transitions

Every case becomes both-pass, regression, improvement, or both-fail over explicit independent evidence units.

Release authority

Evidence sufficiency, quality rules, and runtime errors remain distinct. Only a configured Release gate can authorize deployment.

Evidence that survives the dashboard.

A completed run is an immutable local record, not a mutable chart.

MANIFEST

Source-bound

Configuration, datasets, schemas, implementation sources, model-facing schemas, and output artifacts are hash-bound.

REPLAY

Independently recomputed

Built-in evaluator facts and summaries are reconstructed from retained inputs. Tampering fails closed.

CI

Reviewable automation

The visual setup exports a project-specific CI starter with stable gate states and exit codes.