Ticket System

Auditable, controlled execution for AI agents

Node.jsFastifyAI AgentsSystems Design
Role
Independent full-stack and systems developer
Context
Agent-run work needs explicit authority, evidence, verification, and operator review
Implemented
Ticket and run lifecycles, evidence ledger, triage, replay, CLI operations, and eval harnesses
Constraint
No off-ledger effects and no completion without independent verification
Outcome
A seeded reference system with inspectable failure paths and regression-tested invariants
Status
Working reference implementation with public source and operational documentation

Every unit of work is a durable ticket, every execution runs under explicit authority, every external effect flows through a target provider.

The screenshots below are from the seeded demo environment. Six tickets are engineered to show the full lifecycle, including the failure paths.

01

Every ticket and its state, at a glance

The ticket list shows the whole story at a glance: completed-and-verified work, a verification failure routed to run triage, a ticket blocked at ticket-level triage, and a manual rerun ceiling (maxAttempts 2). States are explicit and auditable. There is no "the agent is doing something" limbo.

Ticket list showing completed, failed, and blocked tickets with execution status lines
02

Finished doesn't mean verified

A finished run doesn’t get a pass by default. This ticket completed, but the system flags it "Needs review" and says exactly why: the full objective was not independently verified, and objective path coverage was not scored. Rerun, test-gate, and plan-simulation controls sit alongside. Simulation tests the agent’s plan without creating a run or touching the workspace.

Ticket detail with a Needs Review banner explaining unverified objectives, plus rerun and simulation controls
03

Triage tells the operator what they may and may not do

The triage inbox is read-only and decision-oriented. A blocked ticket needing a scope change lists allowed actions (review, edit ticket) and prohibited ones (start a run without the scope change); a failed verification lists review and rerun-from-start, and explicitly forbids marking complete without verification. The system encodes its own guardrails.

Triage inbox with ticket-level and run-level entries showing allowed and prohibited actions
04

A live snapshot of the whole system

The ops page is a read-only snapshot computed live from the underlying stores. It creates no new source of truth and offers no mutation controls. Warnings surface unresolved triage, blocked tickets, and failed runs; the summary counts every primitive in the substrate, from work contexts to model routing policies.

Operational Transparency page with warnings and substrate-wide summary counts

It studies its own failures and hardens against them

The interesting part isn’t that the system runs agent work; it’s that it treats every failure as material to improve from. Agent runs execute against realistic business fixtures — vendor compliance, legal intake, customer support, and shared-drive cleanup — and a verifier scores each result against an explicit contract. A wrong answer isn’t discarded. It becomes a case.

From there the repo runs a loop. A failure-classification workflow sorts each failure into a frozen S1–S5 taxonomy, and recurring cases are grouped by root cause in a failure-cluster report. The classifier itself is agent work: it builds a dossier per failed run, drafts a classification on a cheap model tier, and escalates to a stronger one only when confidence is low — the same governed execution the rest of the system uses, pointed at itself.

None of this needs a human at the console. Every action in the web UI has a headless equivalent in oquery, a CLI an agent drives under the same authority and evidence rules: create a ticket and wait for its run as JSON, inspect a timeline, replay a run, resolve triage, even operate the bounded workspace. It’s the seam that lets the system act on itself — the failure classifier files its own tickets through this very interface.

When a fix is proven by direct causal evidence, it graduates through a documented promotion process — observation, analysis, experiment, validated improvement, candidate principle, invariant — and a regression test locks it in so the same failure can’t return. The architecture invariants even forbid the tempting shortcut: don’t widen a limit or special-case the runtime to make a benchmark pass. Improve the guidance instead.

The whole thing leaves receipts, and the receipts are built to be trusted rather than just kept. The append-only event log is the source of truth; the tickets and runs you see are projections rebuilt from it, and an integrity audit proves the derived state still matches. Replay goes beyond reproduce — a tamper test corrupts snapshots ten ways and asserts the verifier catches every one, so history can’t be quietly rewritten and still pass.

Collaboration is held to the same standard. A handoff between agents isn’t a private channel or a chat transcript; it’s an ordinary ticket transition with receipts, so whoever picks up the work inherits the full trail. Agent-produced changes don’t merge on trust either — a verification gate runs build, workflow, postcondition, endurance, and regression checks before one is accepted.

The result reads less like source and more like an operated system’s logbook — the reasoning behind each principle written down next to the evidence that earned it.

  • OPERATOR_CONTRACT.mdThe authority model: what a run may and may not do, and why nothing executes off-ledger.
  • docs/EXECUTION_MODEL.mdTicket → run → authority → target provider → evidence, defined against committed milestone work rather than aspiration.
  • scripts/oquery.jsThe agent-facing interface: a headless CLI mirroring every UI control, so an agent can query and drive the system as JSON under the same authority rules.
View 8 more implementation artifacts
  • A single-process Fastify server with EJS views and JSON-file stores. Keeps infrastructure simple so the execution model stands out.
  • Sessions with argon2 password hashing; a seeded demo mode; and an eval harness that scores agent behavior against fixed rubrics, including workflow-draft quality, repair ability, and operational endurance. It also includes regression suites for recovery and truncation edge cases.
  • Documented like an operated system, not a repo: operator contract, state surfaces, failure-cluster reports, and a release-candidate audit.

Like what you see? I'm available for full-stack and automation work.

Get in touch