Structured-output release evidence

Your schema passed.Did the answer?

Catch valid-but-wrong regressions before switching models, prompts, providers, decoders, or implementations.

Try the invoice demo Download StructTrace View on GitHub

Local-firstNo accountNo telemetry

invoice-extraction / local evidence

StructTrace local results screen showing an insufficient-evidence decision and paired regression metrics

One Rust pathEvery adapter enters the same parsing, scoring, statistics, gate, artifact, and replay engine.

Complete denominatorMissing outputs, invalid JSON, evaluator errors, and timeouts stay visible as failures.

Replayable evidenceImmutable local runs and BLAKE3-bound artifacts can be verified without rerunning a model.

Data stays localThe browser UI is served from your machine. The public site never receives evaluation data.

From output files to a release decision.

StructTrace keeps the first workflow intentionally narrow: compare one baseline and candidate against the same external contract and the same cases.

01 / SOURCE

Drop outputs

Import JSONL, JSON, or CSV. Inspect field mappings before StructTrace creates the reproducible project.

02 / EVALUATE

Define correctness

Use deterministic rules for identity, dates, exact numbers, tolerances, arrays, tool arguments, and invariants.

03 / DECIDE

See regressions

Inspect every paired transition, evidence limit, failure hotspot, and exact case before changing production.

Valid JSON can carry the wrong answer.

A parser and JSON Schema can confirm shape. They cannot establish whether an extracted total, selected tool, or business outcome is correct.

Baseline / invoice-011

Correct extraction

SCHEMA VALID

{
  "line_items": [
    {"description": "Masks", "amount": "40.00"},
    {"description": "Sanitizer", "amount": "36.00"}
  ],
  "total": "82.08"
}

Candidate / invoice-011

Valid, but wrong

SCHEMA VALID

{
  "line_items": [
    {"description": "Masks", "amount": "40.00"}
  ],
  "total": "82.08"
}

Regression: /line_items is missing the Sanitizer entry. The object satisfies the schema, but the extraction outcome fails.

A local product, not a data-collection service.

The public website explains and demonstrates the workflow. Full evaluations run through the actual Rust engine on your machine.

Your data→Local browser UI→Rust engine→Immutable evidence→CI gate

Strict boundaries

Duplicate keys fail. Remote schema retrieval is disabled. Dataset labels never enter live application prompts.

Honest uncertainty

Small, duplicated, conflicted, or incompletely evaluated datasets cannot silently authorize a release.

Portable proof

Reports, manifests, source hashes, outcome rows, and replay receipts remain inspectable outside the UI.

Built because structural success hid semantic loss.

Constrained-decoding experiments motivated the product. Cross-family replication also showed why StructTrace measures instead of prescribing one universal optimization.

Accepted studies remain separate. Effects are not pooled.
Gate	Cases	Control	Treatment	Paired effect
Corrected Qwen representation	49	18/49	24/49	+12.2 pp
Canonical Llama replication	150	92/150	82/150	-6.7 pp
Executable tool-call pilot	30	26/30	24/30	-6.7 pp

The estimates use different models and tasks. They are displayed as a research chain, not one combined effect.

Read the evidence and claim boundaries

Start with the real local workflow.

Use the bundled invoice comparison, inspect the three regressions and three improvements, then bring one of your own stable-contract migrations.

Get StructTrace Run your first comparison