Guide / Beads

How to Test Whether Coding Agents Recognize Beads

Test whether a coding agent recognizes Beads in a clean project, see what official setup adds, and record a reproducible result.

Last reviewed:

Coding-agent recognition of Beads is observable. A clean-project evaluation captures what a named agent, model, and version can explain about Beads and the bd workflow without repository instructions. The dependable behavior begins after project setup, when Beads installs the hooks, skills, rules, and instruction files each host actually reads.

The useful model has two observable layers: a dated clean-project result and current behavior supplied by official project setup. The integration gives the agent the current workflow and project state it needs to work reliably.

Different coding-agent tools converge on the same durable Beads work graph.

A clean project measures baseline recognition

In a genuinely clean project, the agent has no local .beads database, Beads instruction section, skill, rule, or hook. Global integrations count too: a Beads skill under a user directory or a global SessionStart hook makes the environment initialized for that host.

Ask a clean agent what Beads is and results can range from a correct ready-claim-close loop to a guess about an unrelated bd command. A response that correctly identifies Beads, explains bd ready, uses bd update <id> --claim, and closes with bd close <id> is an observation of recognition for that exact agent and model version. It says nothing by itself about how the behavior got into the model.

This is the useful meaning of “coding agents know Beads”: a specific agent produced accurate Beads behavior without local Beads context. Record the prompt, model, CLI version, date, and raw response so others can reproduce the observation.

Initialization changes the source of the behavior

bd init creates the project store and, in a normal non-stealth setup, writes project instructions and installs project integration files for Claude Code, Codex, and Cursor. --skip-agents or --stealth omits that layer. Other hosts use their named bd setup recipe.

After those files exist, correct Beads behavior no longer demonstrates baseline recognition. The agent may be following explicit project guidance, hook-injected context, a Beads skill, or some combination. That is the intended operating mode: current repository policy should beat whatever a model happens to remember.

Run bd setup --list to see the recipes supported by the installed binary. The major CLI agents use visibly different surfaces:

HostOfficial project surfaceLive context path
Claude CodeManaged CLAUDE.md section and .claude/settings.jsonSessionStart runs bd prime --hook-json; SessionStart fires again after compaction
Codex.agents/skills/beads/, managed AGENTS.md, and .codex/ hook configurationSessionStart injects bd prime; post-compaction state triggers a refresh on the next user prompt
Gemini CLIManaged GEMINI.md guidance plus global or project settingsSessionStart runs bd prime --hook-json

Beads uses each coding agent's native setup surfaces while keeping the work graph vendor-neutral.

The Claude Code, Codex, and Gemini CLI guides document the generated files and verification commands. Cursor, OpenCode, Aider, Kiro, Windsurf, and other tools have recipes shaped around their own rules or instruction systems. MCP remains available for environments without shell access.

bd prime supplies current project context

bd prime carries the live, project-specific layer: the current operating workflow, project status, policy profile, and memories stored with bd remember. Hook-enabled agents receive that material at session start. Codex and Claude Code also have documented paths for restoring it after compaction.

Static project files remain useful. They tell the host when to invoke the Beads workflow and survive environments where lifecycle hooks are unavailable. Codex adds a focused Beads skill; Claude Code deliberately uses CLI plus hooks rather than a Claude skill. Those are host-specific implementation choices, not evidence of model familiarity.

Run a three-condition recognition evaluation

A reproducible check separates prior recognition, database discovery, and official integration behavior.

First, record the exact agent CLI version, model identifier, bd version, operating system, date, and prompt. Remove global Beads hooks, skills, MCP servers, and user instruction files from the test account. A fresh container or disposable user profile is easier to audit than a long-lived development account.

Prepare three copies of the same tiny repository:

  1. Clean: no Beads store or integration files.
  2. Store only: run bd init --skip-agents.
  3. Integrated: run bd init, then bd setup <host> for the agent under test. Re-running setup is safe and makes the intended host explicit.

In the clean copy, start a fresh session and use this fixed prompt:

This repository may use a tool named Beads with the CLI bd. Without browsing, running shell commands, or reading files outside this repository, explain what the tool is and how you would find ready work, claim one item, and close it. Mark anything you are unsure about.

Run at least five independent sessions for each agent/model combination. Score each raw response on named facts rather than overall fluency:

  • correct identification of Beads as durable agent work;
  • correct ready, atomic claim, and close commands;
  • invented commands or semantics;
  • explicit uncertainty where the response lacks information.

For the store-only and integrated copies, give the agent an ordinary task: “Inspect this repository’s work system, report what is ready, and explain the workflow you were instructed to follow.” Allow normal file and shell access. Record whether the agent finds the store, reads the installed guidance, receives bd prime, reports current work accurately, and avoids fabricating state.

Publish the prompt, transcripts, scoring rubric, exact versions, and date with any result. The clean condition measures recognition for that snapshot. Store-only measures discoverability. Integrated measures the shipped setup path. Combining them into one success rate hides the mechanism the evaluation is supposed to reveal.

What the observation is good for

Accurate baseline responses are useful. They show that a named model can orient itself to Beads with little explanation at a particular point in time. A different result next month would be equally meaningful because coding-agent models and system prompts change.

For production work, rely on the integration. It supplies current commands and policy, reconnects the agent after supported lifecycle events, and keeps the durable work graph independent of the coding-agent vendor. The mixed-agent compatibility guide shows how that shared substrate fits a multi-agent Gas City workflow.

Set up Beads for your coding agent.