Architecture

Three layers, clean boundaries. The engine doesn’t know about parallel execution. The LLM client doesn’t know about pipelines.

System layers

Each box: layer name · source path · what it owns.

graph TB CLI["CLI
cmd/tracker
flags · name resolution · TUI · embedded workflows"] Engine["Pipeline Engine
pipeline/
run loop · edge selection · checkpoints · budget guard · steering channel"] Handlers["Handlers
pipeline/handlers
codergen · tool · human · parallel · fan_in · conditional · subgraph · manager_loop"] Backends["Agent Backends
pipeline/backend.go
native · claude-code · acp"] Agent["Agent Session
agent/
turn loop · tool registry · context compaction · episodic memory · verify-after-edit"] LLM["LLM Client
llm/
Anthropic · OpenAI · Gemini · OpenAI-compat · retry · cost estimation"] CLI --> Engine --> Handlers --> Backends --> Agent --> LLM

The engine run loop

Every node is treated the same: execute the handler, evaluate edges, advance. Parallel, subgraphs, human gates — handlers encapsulate it all.

graph LR A["Clear routing
context"] --> B["Execute
handler"] B --> C["Set outcome
in context"] C --> D["Select
outgoing edge"] D --> E["Checkpoint"] E --> F["Advance to
next node"] F --> A

Edge selection priority

  1. Suggested next nodes

    If the handler set suggested_next_nodes (e.g., parallel pointing at a fan-in), take that edge.

  2. Conditional match

    Evaluate conditions in declaration order; first match wins. Operators: =, !=, contains, not contains, startswith, endswith, in, plus logical and/or/not.

  3. Label match

    Match a human gate’s preferred_label against edge labels.

  4. Default edge

    First unconditional edge. Safety net — and the source of infinite loops if pointed at a loop target.

Handler types

Each node resolves to a handler. The engine calls Execute() and reads the outcome. New handlers register without touching the run loop.

HandlerNode kindWhat it does
codergenagentRuns an LLM session via the selected backend. Emits an agent.Event stream.
tooltoolShell command with timeout, output cap, env scoping, denylist/allowlist, comment stripping for LLM-written lists.
wait.humanhumanChoice / freeform / hybrid / yes-no / interview gates. Pluggable Interviewer for autopilot, auto-approve, webhook routing.
parallelparallelDispatches branches from parallel_targets as goroutines. Honors max_concurrency and branch_timeout.
parallel.fan_infan_inJoins parallel branches; aggregates branch SessionStats into its own trace entry.
conditionalconditionalPure routing — evaluates conditions without an LLM call.
subgraphsubgraphRuns a referenced child pipeline; events propagate to the parent TUI.
stack.manager_loopmanager_loopAsync child-pipeline supervisor. Polls, evaluates stop and steer conditions, injects context mid-execution. Native .dip syntax via ir.NodeManagerLoop.

The dippin adapter

pipeline/dippin_adapter.go bridges dippin-lang’s IR to tracker’s Graph model. Every naming mismatch lives here.

Dippin IRTracker GraphNotes
modelllm_modelAttribute rename
providerllm_providerAttribute rename
googlegeminiProvider name normalization
ctx.outcomeoutcomeNamespace prefix stripped at evaluation time
ParallelConfig.TargetsImplicit edgesSynthesized from parallel targets
FanInConfig.SourcesImplicit edgesSynthesized from fan-in sources
ir.NodeManagerLoophandler=stack.manager_loopFlattens the IR config into six unprefixed attrs (subgraph_ref, poll_interval, max_cycles, stop_condition, steer_condition, steer_context). steer_context uses canonical k=v,k=v with percent-encoding. Steered values land under ${ctx.steer.<key>} on the child to avoid colliding with the tool_command safe-key allowlist.

Agent backends

AgentBackend is a one-method contract: execute a node, return an agent.Event stream. The engine doesn’t know which backend ran. Mix per node; per-node attr beats --backend.

graph TB CH["CodergenHandler"] --> SB["selectBackend()"] SB -->|"default"| NB["Native\nagent.Session\nturn loop + tool registry"] SB -->|"backend: claude-code"| CC["Claude Code\nclaude CLI subprocess\nNDJSON stdout"] SB -->|"backend: acp"| AC["ACP\nJSON-RPC over stdio\nclaude-agent-acp / codex-acp / gemini --acp"] NB --> Events["agent.Event stream"] CC --> Events AC --> Events

Native

Wraps agent.Session — turn loop, tool registry, context compaction, optional plan-before-execute, episodic memory, structured reflection on tool failure. Anthropic, OpenAI, Gemini, OpenAI-compat.

Claude Code

Spawns claude as a subprocess; parses NDJSON. File editing, terminal, MCP servers come for free. API keys stripped by default so the CLI uses subscription auth (override with TRACKER_PASS_API_KEYS=1).

ACP

Spawns any Agent Client Protocol agent over stdio. Provider-based defaults: anthropic → claude-agent-acp, openai → codex-acp, gemini → gemini --acp. Override per node via acp_agent.

Lazy LLM client. buildLLMClient() failure is non-fatal when the global --backend isn’t native. A pipeline can run with no API keys at all if every node uses claude-code (subscription auth) or acp (bridge native auth).

Agent-runtime tools

The native backend ships an extensible tool registry. Architect-tier tools register on demand via env vars — no env var, no tool. Lives in agent/tools/; registered from pipeline/handlers/backend_native.go.

TerminalTool — primitive

An interface a tool can implement to flag itself terminal. The runtime ends the session loop the moment it succeeds — before the next LLM call — so dispatch tools don’t waste a post-dispatch turn.

dispatch_sprints

Reads a {path, description} JSONL plan and runs the per-sprint author + audit pipeline once per line. Bounded retry+backoff for transient provider errors; non-retryable errors bubble out.

write_enriched_sprint

Mid-tier LLM (Sonnet by default) with a 4-strategy SEARCH/REPLACE matcher (exact → indent-preserving → whitespace-insensitive → fuzzy). Partial-apply semantics distinguish PATCHED-PARTIAL from clean PATCHED. Override via TRACKER_SPRINT_WRITER_MODEL.

generate_code

Cheap/fast model expands a contract into one or more files. Default gpt-4o-mini; override via TRACKER_CODEGEN_MODEL. Companion to write_enriched_sprint.

State, budgets & resume

The PipelineContext is the typed key-value store shared across nodes. Variable expansion resolves ${ctx.*}, ${params.*}, ${graph.*}, and ${ctx.node.<id>.*} in prompts, tool commands, and edge conditions — single-pass, never re-scanning resolved values.

Per-node scoping

After each node, every key it wrote is also copied into node.<nodeID>.<key>. Downstream reads ${ctx.node.ReadSpec.last_response} reference a specific upstream output instead of the latest.

Declarative writes: / reads:

Nodes declare the keys they produce and consume. Declared writes are validated into the context after the handler runs — missing required fields fail the node. Self-healing JSON cascade catches the common “LLM returned prose” failure modes.

Budget guard

Enforces max_total_tokens / max_cost_cents / max_wall_time between every pair of nodes. Breach halts the run with OutcomeBudgetExceeded and a full CostSnapshot. Subgraph and manager_loop children inherit the parent guard so ceilings bind through nested execution.

Cost-accounting fidelity

Per-call usage carries an Estimated flag that propagates through SessionStatsProviderUsageCostSnapshot. Estimated rollups render as ~$X.XXXX (estimated). Cache reads priced at 10% of input, cache writes at 25%.

Git artifacts

With WithGitArtifacts(true), the run dir is a git repo. Every terminal node creates a commit (node(<id>): <handler> outcome=<status>) carrying duration, edge, token, and cost metadata. git log is a human-readable audit trail.

Resume & bundle

tracker.Config.ResumeRunID resolves a run ID to its checkpoint.json path. ExportBundle wraps git bundle create --all into a single self-contained file you can git clone anywhere — no network needed.

.dipx bundle identity v0.26.0

Content-addressed .dipx bundles (from dippin pack) load via pipeline.LoadDipxBundle: SHA-256 verifies every file in manifest.json, uses the pre-parsed *ir.Workflow directly (no re-parse), bypasses the filesystem subgraph walker. The sha256:<hex> identity is stamped on every activity.jsonl line, persisted into checkpoint.json, surfaced in tracker list/audit, and exposed on tracker.Result.BundleIdentity & RunSummary.BundleIdentity. Resume strictly verifies identity match; --force-bundle-mismatch is the escape hatch.

Steering channel. pipeline.WithSteeringChan(chan map[string]string) lets a supervisor inject context updates between node executions via MergeWithoutDirty — writes the global namespace without leaking into per-node scope. Used by stack.manager_loop to steer a child pipeline mid-run. Steered values are namespaced under steer. on the child.

Run terminal statuses

EngineResult.Status carries one of four values when a run ends. The enum is open — future minor releases may add values, so consumers should classify via TerminalStatus.IsSuccess() (or the status_class JSON companion field) rather than switching on the raw string.

ValueMeaningIsSuccess()
successRun reached the success exit; every validation along the way passed.true
validation_overridden v0.35.0Run reached the success exit, but a human, autopilot, or webhook accepted a failed validation along the way via a wait.human edge marked override: true. Routing is unchanged from the success path — the marker is an audit signal recorded on EngineResult.ValidationOverrides.true
budget_exceededThe BudgetGuard halted the run after a token, cost, or wall-time ceiling tripped. CostSnapshot on the result names the dimension.false
failRun halted via failure (strict failure edge, provider error, etc.).false

The tracker run CLI maps fail/budget_exceeded to exit 1 and (with --fail-on-override) maps validation_overridden to exit 2. Without the flag, override runs exit 0 — the default treats an operator-accepted override as audit-positive. See the CLI exit-codes table.

LLM client middleware

Provider-agnostic. Middleware wraps every request for cross-cutting concerns.

graph LR Req["Request"] --> Retry["Retry
502, 503, 429"] Retry --> Token["Token
tracker"] Token --> Provider["Provider
adapter"] Provider --> API["Provider API"] API --> SSE["SSE Stream"]

Infra retry handles transient errors transparently: max 3 retries, 2s base delay, exponential backoff. Pipeline-level retries are reserved for actual node logic failures. SSE error handling matters because OpenAI’s Responses API returns HTTP 200 and sends error / response.failed as SSE events — the adapter parses these out; they are NOT reflected in the HTTP status.

TUI model

Built on bubbletea. The pipeline runs in a background goroutine; the TUI receives events via tea.Msg. No polling.

Library API

Tracker is also a Go library. The CLI is a thin printer over the same APIs — and one transport among peers. The TUI, the Slack bot, and any future web or mobile app plug into the same tracker.Config → Engine boundary.

Structured reports

tracker.Diagnose, DiagnoseMostRecent, Audit, Doctor, Simulate return typed reports. Typed enums for CheckStatus and SuggestionKind — no stdout scraping.

NDJSON streaming

tracker.NewNDJSONWriter(io.Writer) produces the same wire format as --json. Plug it into Config.EventHandler, Config.AgentEvents, and the LLM trace hook. Per-writer write-failure recovery.

Context-first

Every entry point takes context.Context as its first argument, honored by provider probes and binary version lookups. Cancellation is real.

Typed NodeConfig

*pipeline.Node exposes AgentConfig / ToolConfig / HumanConfig / ParallelConfig / RetryConfig. Graph-default-then-node-override merge happens once inside the accessor; handlers consume typed structs, not raw Node.Attrs reads.