Technical assessment · Milestr v1.3.2 · 2026-10-26

Why a task hierarchy with derived progress is exactly what an agent needs to run a project, not a prompt

An evidence-first assessment of the structural substrate AI agents require to execute high-level functions across long, multi-session, multi-agent projects — mapped onto what Milestr actually implements, and where it still falls short of the tools it must beat.

Evidence 30+ primary sources, all read Method literature + code audit + npm telemetry Code audited 2,994 LOC src · 3,241 LOC tests · 152 tests green Subject milestr@1.3.2 · Dr-Agentic/milestr Author Milie · for Morsy Cheikhrouhou
Sections Verdict· The 6 failures· Measured shape· 7 primitives· Session loop· Rollup· Contract· Accountability· The gap· Monday· Sources

01 The verdict in one page

TLDR Milestr is not a task tracker that happens to be agent-friendly — it is the smallest honest implementation of the one substrate the literature converges on: a structured, machine-checkable, attributed project state living outside the context window. The evidence supports every component. It does not yet support the combined claim, and Milestr is distribution-dead (0 GitHub stars, median 1 npm download/day) against a competitor with 28,211 stars. The thesis is right; the adoption is not.

Every serious lab that ran agents across more than one context window converged on the same substrate, and none of them called it a "project tracker". They called it a feature list (Anthropic), an ExecPlan (OpenAI Codex), a todo provider (Microsoft), a memory stream (Generative Agents), a task network (HTN planning), or Task JSON (the practitioner literature). The common shape is always the same five properties:

  1. It is a file, not a conversation. It survives compaction, a crash, a new session, a new agent.
  2. It is structured enough that only one field per item should change. Progress is flipped or derived — never rewritten in prose.
  3. It is hierarchical. A goal rolls up from milestones, milestones from initiatives, initiatives from tasks — so "how far along is the goal?" is a number, not an opinion.
  4. It is attributable. Every change carries a timestamp and the identity of the agent that made it.
  5. It ends in something a human or a dashboard can read. Otherwise the human becomes the verification loop.

Milestr implements all five, and — unusually — it implements them as a deterministic CLI contract with a self-healing state engine, not as a prompt convention. That is the substance of the fit. The rest of this page walks the evidence, the mapping, and the honest gaps.

Supported

Component claims

Every individual mechanism Milestr uses has independent published support: context rot, derived rollup, field-flip discipline, attribution, KPI steering, durable state.

§3–§8, 30+ primary sources
Not proven

The combination

No published ablation isolates "hierarchical rollup DAG" against "flat todo list" on a long-horizon agent benchmark. Say "the literature supports each component; the combination is our design."

Claim discipline — do not overstate
Critical gap

Adoption

0 GitHub stars. 6,413 lifetime npm downloads, median 1/day, 43 zero-days out of 120. Beads — a direct comparator — has 28,211 stars and 32,270 monthly npm downloads.

§9 — this is the binding constraint
Sequenced

Missing surface

No MCP server. Milestr is CLI-only — the correct side of the CLI-vs-MCP axis for local single-user work, but invisible to every MCP-native agent runtime.

§7 · §10 step 2

02 The six documented failures agents actually hit

These are not opinions. Each is named, described and instrumented in a primary source — and each one is a state failure, not a reasoning failure.

Failure 1

One-shotting

The agent tries to do everything at once, runs out of context mid-implementation, and leaves the next session with a feature half-built and undocumented. The next agent then has to guess what happened.

Anthropic, Effective harnesses for long-running agents, Nov 26 2025
Failure 2

Premature victory declaration

A later agent instance "would look around, see that progress had been made, and declare the job done." Anthropic's internal name for it: "Claude declares victory on the entire project too early."

Anthropic, same source
Failure 3

Undocumented progress → dirty handoff

"Claude leaves the environment in a state with bugs or undocumented progress." The next agent re-reads files, re-asks answered questions, re-derives decisions — and sometimes re-attempts fixes that already failed.

Anthropic + PROJECTMEM (arXiv:2606.12329)
Failure 4

Premature feature completion

"Claude marks features as done prematurely" — the agent changes code, runs unit tests, and never checks the feature works end-to-end. The state lies, so every downstream decision is built on a false premise.

Anthropic, same source
Failure 5

Unverifiable completion claims

"Claude stops when the work looks done. Without a check it can run, 'looks done' is the only signal available, and you become the verification loop."

Claude Code best practices, Anthropic
Failure 6

Drift from the objective

Over extended interactions, agent behaviour degrades in three named ways: semantic drift (progressive deviation from the original intent), coordination drift (breakdown in multi-agent consensus), behavioural drift (unintended strategies emerge). Left unchecked: lower task-completion accuracy, more human intervention.

Agent Drift, arXiv:2601.04170
The industry-wide admission The PM-tool market already conceded the gap. "An AI session is not a shell. It's a conversation with a finite memory window that empties on Stop… The ticket exists; the conversation that produced it does not." — Sprintra. And from the Scrum.org analysis of Jira: "Jira stores workflow state and fragmented context. It does not store a coherent, versioned knowledge model optimized for reasoning across time." Every attempted "Claude Code + Jira" and "Cursor + Linear" workflow hits the same wall, and the gap is not fixable with an integration tab.

03 The problem is measured, not anecdotal

Three independent quantitative results define the constraint any long-horizon agent substrate has to beat.

METR · Mar 2025
<10%success on tasks taking humans more than ~4 hours

Models are near-100% on tasks under 4 human-minutes. The 50%-success task length has been doubling roughly every 7 months — but the current ceiling is a few minutes of human work.

metr.org · arXiv:2503.14499
arXiv:2609.01660 · Aug 2026
16agentic steps before collapse, every model tested

9 models, 4 task families, 5 horizons, 3 context regimes, 10,664 trajectories. Task success follows a geometric law; per-step reliability saturates well below 1. Projected success falls from 0.42 at GAIA horizons to 0.24 at hundred-step production horizons.

arxiv.org · degradation driven by step count, not context length
Context rot · 2025–26
13.9–85%accuracy loss with retrieval held perfect

Chroma tested 18 LLMs with input length as the only variable: performance degrades on every model, non-uniformly, long before the window fills. Anthropic calls context "a finite resource with diminishing marginal returns" and an "attention budget".

trychroma.com · Anthropic
The result that changes the design Degradation is driven by step count, not context length — and truncating the context window makes it worse (logit slope −0.69 vs −0.44, p = 3×10⁻⁶). The paper's operational corollary: "step-level verification, wherever an oracle or a cheap consistency check can be inserted, is likely to catch failures far earlier and far more cheaply than waiting for end-to-end task failure to surface." That is the entire argument for making progress a checkable number at every node rather than a narrative at the end.

Measured consequence of not having durable state

PROJECTMEM (arXiv:2606.12329, Jun 2026) ran a six-month self-study on 27 projects: 86 of the 427 distinct issues closed by a recorded fix had at least one failed attempt before the first fix in event-log order. That is the cost of a next agent that cannot see what was already tried. Their engineering answer — an append-only typed event log projected deterministically into compact summaries — took the median analysis of a 1,500-event log from 42.0 seconds to 79.3 milliseconds.

04 Seven primitives, each with its own literature — and its line in Milestr

Read left to right, this table is the whole argument: a failure mode, the primitive that fixes it, the primary source that prescribes it, and the exact implementation in the Milestr codebase.

PrimitiveFixesPrescribed byMilestr implementation
Durable state outside the window One-shotting, context rot, dirty handoff Anthropic harnesses: init.sh + claude-progress.txt + git history; OpenAI: "anything it can't access in-context while running effectively doesn't exist" data.json on disk, one file per project; every command re-reads it — src/data/store.ts:13
Structured, field-level mutation Undocumented progress, accidental rewrites Anthropic: "the model is less likely to inappropriately change or overwrite JSON files compared to Markdown files"; "edit this file only by changing the status of a passes field" Typed CLI actions with a Zod-validated schema (src/data/schema.ts); SKILL.md §"NEVER edit data.json by hand" (line 207)
Derived rollup progress Premature victory, drift ST-WebAgentBench (arXiv:2410.06703): Completion Rate vs Partial Completion Rate; Anthropic: agent may flip one boolean, not restate progress cascadeUpdate() src/actions/utils.ts:87 — parent progress = mean of children; parent done ⇒ progress 100
Self-healing state Drift from out-of-band edits OpenAI harness engineering: "recurring cleanup process… functions like garbage collection"; linters + CI validate the knowledge base is current healParentRollups() src/actions/utils.ts:155, wired into every loadData(); logs a HEAL: line (store.ts:67)
Attributed append-only log Coordination drift, "who did what", cost attribution OpenTelemetry GenAI conventions: gen_ai.agent.id, gen_ai.memory.record.id; Anthropic: evaluate final state via discrete checkpoints dashboard.log — every mutation writes CALLED: / CHANGE: with agent + action; --agent is mandatory (no silent agent=unknown writes)
KPI as steering truth Drift from the objective, token-chasing StateLens (ACM TURC '26): progress-as-scored-signal took success 0.333→0.667 and cut repeat rate 0.285→0.083 with no human in the loop First-class KPI objects (value, unit, trend, source) + the four-role agent mesh where the KPI tracker is the sole writer of values
Human-readable projection Verification loop, invisible progress Claude Code: "you become the verification loop"; LangChain: "A single number like '67% complete' is immediately understandable" Single-file HTML dashboard regenerated on every write, published to Cloudflare Pages — 5 views, plus an export-time parse guard that refuses to publish a broken script
Why the last row is not a nice-to-have Milestr v1.3.2 shipped the proof that the projection is load-bearing. v1.3.1's dashboard had a single-quote escaping bug in its inline <script> block, so the browser discarded the whole script: tab switcher, tree zoom, detail drawer and theme toggle were all dead — silently, on every published dashboard, for a month. v1.3.2 fixed it, moved the script into its own module, added assertDashboardScriptParses() at export time, and added 30 jsdom tests that execute the dashboard and assert behaviour. A projection that lies is worse than no projection — that is the same reason Milestr self-heals its rollups instead of trusting them.

05 What a session actually looks like

Anthropic's fix for one-shotting was a fixed orientation routine. Every session starts the same way, reads the same three artifacts, and ends by writing back. This is the loop Milestr makes mechanical — the state is queryable instead of reconstructed from a transcript.

SESSION N — ANY AGENT, ANY RUNTIME 1 · ORIENT pwd milestr list --json read goal → milestone → task 2 · SELECT read KPI deltas read activity log pick highest-value node 3 · EXECUTE + VERIFY run the real work milestr status <id> done only after a check passes 4 · PUBLISH backup → write rollup → log dashboard → Pages URL PERSISTENCE BAND — OUTSIDE THE CONTEXT WINDOW data.json graph · KPIs · activity backups/ (10-deep) pre-write snapshot dashboard.log append-only · attributed dashboard.html 5 views · Pages every write next session reads this — never the transcript GUARDS — WHAT MAKES THE STATE TRUSTWORTHY PID file lock (no concurrent writes) · Zod schema validation · versioned data migrations with a CI-enforced registry · self-heal of stale rollups on every load · out-of-band-edit detector · export-time JS parse guard
The Milestr session loop. The two dashed arrows are the whole point: the next session reads state, never the transcript.

Anthropic's routine

1. pwd. 2. Read the git logs and progress files. 3. Read the feature list and choose the highest-priority feature that's not yet done.

anthropic.com/engineering/effective-harnesses-for-long-running-agents

OpenAI's ExecPlan

Same shape, richer schema: mandatory Progress, Surprises & Discoveries, Decision Log, Outcomes sections. "It should always be possible to restart from only the ExecPlan."

cookbook.openai.com/articles/codex_exec_plans

Microsoft's todo provider

todos_add / todos_complete (requires a completion reason) / todos_get_remaining; persisted via TodoFileStore; injected before each run so the agent can resume outstanding work.

learn.microsoft.com — Agent Framework, updated Sept 19 2026

06 Why hierarchy beats a flat list

A flat list answers "what is left?". It cannot answer "how far along is the goal?" without re-deriving that from the items — by a reader (human or agent) that may mis-weigh, mis-count, or simply be optimistic. Anthropic's failure #2 is exactly that: progress becomes an impression. With rollup it becomes a number.

FLAT TODO LIST — THE PROBLEM [ ] ship v1.3.2 [x] fix rollup drift [ ] write migration [x] open issue [ ] … 196 more Progress = ? Re-derived by a reader, every time, with no single source of truth. No dependency. No owner. No scope. "looks 90% done" → declares victory MILESTR GRAPH — THE FIX GOAL · 1M downloads progress 74% (derived) M1 · release progress 82% M2 · adoption progress 61% M3 · surface progress 78% I1.1 · docs done · 100% I1.2 · ci ongoing · 64% ROLLUP RULE parent.progress = mean(children.progress) all children done ⇒ parent.status = done rollup
Rollup makes progress derived. There is no second copy of "the goal is 74% done" stored anywhere — so it cannot drift.
"We prompt coding agents to edit this file only by changing the status of a passes field… we landed on using JSON for this, as the model is less likely to inappropriately change or overwrite JSON files compared to Markdown files." — Anthropic, Effective harnesses for long-running agents, Nov 26 2025
"Faster agents do not reduce the need for source of truth. They increase the penalty for not having one. Without a stable task contract, speed turns into drift… 'Done' should not be a feeling at the end of a persuasive transcript. It should be a state the workflow can inspect." — Alex Sandruk, Task JSON as source of truth for agent work, sandruk.org

Formal backing for the hierarchy itself

The goal → milestone → initiative → task shape is Hierarchical Task Network planning, formalized decades before LLM agents: a task network "represents a hierarchy of tasks each of which can be executed," separating what we want to do from how it is implemented (Ghallab et al., arXiv:1403.7426). Complexity work (Alford et al., arXiv:2401.14174, 2024) establishes that HTN-level structure is what makes plan verification and state reachability tractable — hierarchy is the structure that makes state checking cheap. That is precisely what a parent rollup needs, and precisely what a flat list cannot do.

07 A deterministic contract for a nondeterministic agent

Every failure above is a reliability problem, and reliability comes from making the agent's interface boring: typed commands, validated state, machine-checkable output. Milestr's surface is deliberately narrow.

# every mutation names its author — no silent agent=unknown writes
$ milestr --agent runner status M1.2 done "rollup fixed"

# machine-readable state for the next agent or a script
$ milestr list --json | jq '.[] | select(.status=="blocked") | .id'

# derived numbers, never narrated
$ milestr metrics --json
{ "total": 7, "byStatus": { "ongoing": 3, "done": 2 },
  "byType": { "goal": 3, "milestone": 4 }, "kpis": 4 }
Design constraint

One field per mutation

The agent's job is to flip a status or a progress number and attach a reason — never to rewrite the document. The command surface makes that the only path: there is no "edit the JSON" verb.

SKILL.md:207 · src/actions/handlers.ts
Design constraint

Failure is loud, not silent

Concurrency → LockError with the holder's PID. Schema drift → Zod error. Stale version → explicit migration error. Out-of-band edit → MANUAL_EDIT: line in the log. Broken dashboard → export refuses to publish.

src/data/lock.ts · src/data/schema.ts · store.ts:42 · dashboardHtml.ts:549

CLI vs MCP — where Milestr sits

A 2026 comparative analysis across 14 dimensions (arXiv:2610.30157, Jul 29 2026) is blunt about the trade-off: CLI is superior for single-user, local-only workflows (2–5× latency advantage), while MCP is what CLI fundamentally cannot satisfy — fine-grained access control, credential isolation, structured audit, multi-tenant governance — at a cost of up to 55K tokens for naive MCP loading. Its recommendation is a hybrid.

DimensionCLI (Milestr today)MCP serverVerdict for Milestr
Latency, local single-userwins2–5× slowerCorrect side of the axis
Zero setup for an agent runtimepartial — agent must shell outwins — native tool surfaceReal gap. Every MCP-native agent sees nothing without a server
Token cost of loading the toolwins — 0 tokensup to 55K naiveMilestr's surface is tiny; an MCP wrapper would be small too
Structured audit / multi-tenant RBACpartial — attributed log + file lockwinsNeeded before selling to teams
Human oversight surfacewins — published dashboardagent-onlyKeep the dashboard as the human's window
The distribution consequence The MCP ecosystem is where the agent traffic is. The official MCP Registry held 47,267 active servers on Oct 21 2026 (Tanod's daily index: 50,078 distinct servers on Oct 25); the TypeScript SDK alone pulled 231.9M npm downloads in September 2026 versus 31.3M a year earlier. A CLI that no agent runtime can call is invisible to that traffic. This is the single highest-leverage missing surface.

08 Accountability is a design constraint, not a feature

Anthropic's multi-agent research system reported that multi-agent setups beat single-agent by 90.2% on their internal eval — while burning ~15× the tokens of chat, with token usage explaining 80% of the performance variance. At that burn rate and agent count, "which agent produced this change" stops being a nicety: it is the only way to do root-cause analysis, cost attribution and quality control.

Agent roleSole writer ofMay never touchWhy
Runnertask.status, task.progress, task.activityLogkpi.*, root.*Execution truth
Plannernew tasks, parent.plan_versionkpi.*, task.statusStructure only
KPI trackerkpi.value, kpi.trendtask.*Measured truth
Orchestratorroot.activityLog, task.needs_replankpi.value, task.statusControl plane

The rule is two agents must never write the same field. Violations are bugs. The single-writer discipline mirrors the multi-agent literature: the MDPI survey of LLM-based multi-agent systems warns that without a control plane "even highly capable agents risk duplication of effort, logical inconsistency, or unbounded autonomy that diverges from the system's objectives."

Identity is a standard, not an invention

OpenTelemetry's GenAI conventions define gen_ai.agent.id, gen_ai.agent.name, gen_ai.conversation.id, gen_ai.memory.record.id. The existence of a vendor-neutral agent identity is the signal that "which agent wrote which record" is a first-class concept.

opentelemetry.io — semantic-conventions-genai

Attribution is what makes it debuggable

Milestr's dashboard.log carries CALLED: <agent> | action=… | args=… on every invocation, plus CHANGE:, HEAL:, MANUAL_EDIT:, MIGRATION: lines. Root-causing a bad state change is a grep, not an archaeology exercise.

src/data/logger.ts · src/data/store.ts

KPI beats token-count

The orchestrator's metric is goal progress against KPI trajectory, not tokens. That choice is deliberate: it makes the loop steer toward an outcome rather than toward efficiency theatre.

agents/orchestrator/PROMPT.md

09 The honest gap — where Milestr actually stands

A positioning page that hides this would be worthless. Here is the state of play, verified on 2026-10-26.

Adoption — the binding constraint

GitHub stars0
npm lifetime downloads6,413 (since 2026-06-28)
npm last 30 days5,847
npm median / day1
zero-download days43 of 120
best single day5,568 (2026-10-02)
npm versions5
last releasev1.3.2 · 2026-10-23

The 5,568-download spike is 87% of lifetime volume. Strip the spikes and the tool averages 2.6 downloads/day. Real usage is a handful of internal agents, not a market.

The competitive set (npm downloads / 30d, GitHub stars)

@beads/bd32,270 dl · 28,211 ★
task-master-ai63,352 dl
jira-cli523 dl
gstack861 dl
todoist-cli35 dl
taskwarrior26 dl · 6,138 ★
milestr5,847 dl · 0 ★

beads is the direct comparator and it is not close: a dependency-aware graph issue tracker for agents, 28k stars, 32k monthly downloads, with Claude Code / Codex / Cursor hooks already installed. What it does not have is a goal/milestone/initiative hierarchy, cascade rollup, a KPI scorecard, or an auto-published dashboard — that is the defensible wedge.

Milestr's daily npm downloads, 2026-06-28 → 2026-10-25

Full scale — one spike dominates everything
5,568 0 2026-06-28 2026-10-25
spike ≥100 downloads 10–99 1–9 (the baseline) 0 downloads
Same 120 days, spikes capped at 20 — this is the baseline everything else lives on
20+ 0 2026-06-28 2026-10-25
120 days. 43 days at zero. One 5,568-download day (2026-10-02) and two other spikes above 100. Everything else is a thin baseline of 1–5 downloads.
Gap A — distribution 0 stars against 28k. The engine is credible (152 tests green, 51 commits, 5 releases, CI-enforced migrations). The problem is that nobody has found it. A correct thesis with no audience is a private tool, and a private tool cannot reach a North Star of 1,000,000 downloads.
Gap B — surface No MCP server. Milestr ships a CLI and a SKILL.md, but the majority of agent traffic now enters through MCP tool surfaces. Until Milestr exposes its state as MCP tools, it is invisible to MCP-native runtimes and to the registries those runtimes query.
Gap C — claims No published ablation isolates the rollup DAG from a flat todo list on a long-horizon agent benchmark. The evidence supports each component individually and the combination is a sound engineering inference — but any copy that says "benchmarks prove Milestr's hierarchy wins" is overclaiming and will not survive a competent technical reader. Say: "the literature supports every component; the combination is our design."

10 What to do on Monday

The assessment's conclusion is that the thesis is sound and the bottleneck is reach, not correctness. Three moves, in order.

  1. Ship the MCP surface first. Expose list, view, status, progress, create, list-kpis as MCP tools over the same CLI code path — no second implementation, so the state engine stays single-source. This is the only move that puts Milestr in front of the 47k-server registry and the 231M-download SDK audience. Cost: one small server module. Highest leverage in the whole plan.
  2. Turn the proof into a public artifact. Publish a dashboard for a real multi-agent project (a Milestr instance running its own release cycle), with the published URL in the README. The dashboard-publish loop is the distribution mechanic — an agent that ends every task with a public URL has something to share; a CLI that prints a table has nothing. Cost: one project. This is what the Cloudflare publish feature is actually for.
  3. Fix the one-clause gap and claim it cleanly. Beat beads on the axis it does not cover — goal hierarchy, cascade rollup, KPI-as-truth, self-healing state — and say so explicitly, because "a memory upgrade for your coding agent" is a stronger market position than "another issue tracker". Then ship the open install-skill so an agent can self-install the contract in one command. Cost: one issue (#22) and one README rewrite.
Why this order Correctness is not the constraint. 152 tests green, self-healing rollups, a parse guard that refuses to publish a broken dashboard, CI-enforced migrations — the engine is in better shape than the funnel. Every hour spent hardening the engine is an hour not spent being found. Fix reach first; the engine is ready for the traffic.

11 Sources

Every source below was read in full during this assessment (2026-10-26). Statistics quoted in the body are taken from these texts, not from summaries of summaries.

Primary — vendor engineering

Primary — papers

Primary — practitioner + ecosystem

First-party — the subject itself