Attest Fleet

Kill switch engaged. Every mutating tool call is blocked fleet-wide; workers report blocked, never done.

Silent failure rate

–

Runs the fleet reported done that the system of record says were not. Invisible to the agents' own scoreboard.

Reported–

Verified–

The overstatement: — claimed successes were false.

Computing escalation policy…

Recent runs

Agent platforms score themselves on what they say. Attest scores them on what actually changed.

Built on my own research on agent reliability and silent failure. A paper I have under review is titled Silent Failure in Automated Model Adaptation, and Attest Fleet is that thesis made runnable.

The problem

An agent reports refund issued. The payment gateway quietly left it pending. The run counts as a success, the board stays green, and the customer never gets their money. Research finds this silent failure is up to 48% of failures in single-control tasks and 75.8% on AppWorld. It is the failure nobody sees.

Why today's tools miss it

Observability tools grade the agent with another model as judge. That LLM judge catches silent failure at only ~0.65 AUROC, barely better than a coin flip, because a confident wrong claim reads as success to a model too. Attest does not ask a model. It reads the system of record and compares, deterministically.

Who it benefits

The night-shift support lead who inherits a fleet she did not build and cannot read the code of. At 2am the board says every ticket closed green. She is the one a customer calls when it did not. Attest gives her the one thing she never had: a list of the runs that lied, before the complaints arrive.
The solo operator at a small lender or clinic who bought agents from a vendor, has no ML team, and is personally accountable for every refund and record change.
Customers who would otherwise be silently wronged: refunds lost, tickets closed unfixed.
Auditors and risk teams who need evidence a claimed outcome actually happened, not a log saying an agent said so.

The vision: agent fleets you can trust in production, because every claimed outcome is verified against reality, risky actions pause for a human, and the whole run is auditable evidence instead of faith in a self-report.

Roadmap tier — not part of the verification path

The result below is a controlled demonstration on one small open model. Its controls are uninformative by construction, so it is reported here as a research direction and never as evidence about production agents. State verification against the system of record remains the only ground truth Attest acts on.

Interpretability probe: the representation–action gap

method + reproduction →

On a task built so that surface text and stated confidence carry no signal by construction, the label is still linearly recoverable from the model's internal activations. The information is present internally even when it is absent from the output. A controlled demonstration on one small open model — not a measurement of production agents, whose calibration is the 40-ticket sweep above.

Probe on activations1.00

TF-IDF surface text0.53

Stated confidence0.54

Shuffled-label null0.47

AUROC, 5-fold CV · Qwen-2.5-1.5B · label = does the read-back equal the request. A controlled result: the contrast is the point, not the 1.00. State verification stays ground truth; the probe is the white-box roadmap tier.