Verification & Testing System

TL;DR. make check validates the system (deps, schemas, symlinks, hooks, provisions); make test runs the suites; make ci runs both — what GitHub Actions runs. One command to verify everything.

Bob's infrastructure-as-code verification layer. One command to validate the entire system.

make check    # verify everything
make test     # run all tests
make ci       # both (what GitHub Actions runs)

Quick Start

bash

cd ~/projects/bigbrain    # or wherever BOB_SOURCE is
make help                 # see all targets
make check-deps           # am I set up correctly?
make check                # is everything healthy?
make test                 # do all tests pass?

Targets

Verification (`make check`)

make check runs all five checks in order. Stops at first failure.

Target	What It Checks	Speed
`check-deps`	node >= 18, jq, git, make installed	instant
`check-schemas`	All JSON configs valid against schemas	instant
`check-symlinks`	Symlinks in the current repo resolve (`--all` / `make check-fleet` for every project)	~1s
`check-hooks`	All hooks in settings.json exist + executable	instant
`check-provisions`	The current repo's symlinks match its manifest (`--all` for the fleet)	~1s
`check-fleet`	`check-symlinks` + `check-provisions` across all registered projects (#294)	~2s

Testing (`make test`)

Target	What It Tests	Count
`test-schemas`	Schema validator unit tests (type, enum, pattern, etc.)	34
`test-warp-drive`	Warp-drive state machine transitions	99
`test-checks`	Integration tests for all check scripts	13

CI (`make ci`)

Runs check then test. This is the target GitHub Actions calls on every push and PR to master.

Check Details

check-deps

Verifies the four required tools are installed:

  PASS  node 25.5.0 (>= 18)
  PASS  jq 1.7.1
  PASS  git 2.48.1
  PASS  make (GNU Make 3.81)

If something fails, the output includes a fix command:

  FAIL  node 16.20.0 (need >= 18)
        Fix: Install Node.js 18+ via nvm or brew

check-schemas

Validates every JSON configuration file against its JSON Schema definition:

File	Schema
`projects.json`	`schemas/projects.schema.json`
`settings.json`	`schemas/settings.schema.json`
`provisions/*.json` (17 files)	`schemas/manifest.schema.json`

If something fails, the output shows the exact path and error:

  FAIL  provisions/bodmail.json
        $._meta.project: expected type string, got number
        $.skills[3]: duplicate item "webapp-testing"

To fix: Edit the JSON file to match the schema. The error path ($._meta.project) tells you exactly where the problem is.

check-symlinks

Scans every project listed in projects.json, finds all symlinks in each project's .claude/ directory, and verifies each one resolves to a real file.

  FAIL  /Users/you/Sites/myproject/.claude/commands/sprint.md -> /Users/you/.claude/registry/commands/sprint.md (dangling)

To fix dangling symlinks:

bash

# Option 1: Re-provision the project (recommended)
cdprov --refresh /path/to/project

# Option 2: Remove the stale symlink manually
rm /path/to/.claude/commands/sprint.md

Projects that don't exist on disk or lack a .claude/ directory are silently skipped.

check-hooks

Reads settings.json, extracts every hook command path (from SessionStart, PreToolUse, PostToolUse, Stop), and verifies:

The script file exists
The script is executable (chmod +x)

Also checks the statusLine.command if configured.

  PASS  /Users/you/.claude/hooks/temporal-context.sh
  FAIL  /Users/you/.claude/hooks/missing-hook.sh
        Fix: File not found — remove from settings.json or create the script

To fix:

bash

# If the hook should exist:
chmod +x ~/.claude/hooks/the-hook.sh

# If the hook was removed:
# Edit settings.json and remove the hook entry

check-provisions

For each project in projects.json, loads its manifest (provisions/<project>.json or provisions/_default.json) and verifies that every declared skill, command, and agent is:

Present in the registry (registry/skills/, registry/commands/, registry/agents/)
Symlinked into the project's .claude/ directory
Not dangling

  PASS  bodmail (bodmail.json)
  FAIL  myproject: commands/sprint declared in _default.json but not symlinked
  FAIL  myproject: agents/code-expert symlink is dangling

Failure types and fixes:

Failure	Meaning	Fix
`declared but missing from registry`	Manifest references something that doesn't exist	Remove from manifest or add to registry
`declared but not symlinked`	Item exists in registry but wasn't linked to project	Run `cdprov --refresh /path/to/project`
`symlink is dangling`	Symlink exists but target was moved/deleted	`cdprov --refresh` if the target still exists; `cdprov --prune` if it's gone/cross-machine (see below)

Repairing drift (prune & repair-fleet)

check-fleet reports drift; these repair it (#303).

`cdprov --prune` — per-project

Removes dangling symlinks under a project's .claude/ that point into BoB (*/.claude/*, $BOB_SOURCE, $BOB_HOME). It is dry-run by default; pass --yes (or --apply) to delete. Each candidate is classified:

removed-tooling — target under this machine's BoB home/source but gone (e.g. the retired PM-file / sprint / requirements / risk items).
cross-machine — an absolute target under a different home's .claude (e.g. a repo provisioned on another box: /home/farm/.claude/...).
other — any other dangling BoB symlink.

Safety guarantees (covered by make test-prune): prune never removes a resolving symlink, a real (non-symlink) file, or a foreign symlink (one whose target is not a BoB path). It is idempotent.

bash

cdprov --prune            # dry run — preview what would be removed, by class
cdprov --prune --yes      # apply

`make repair-fleet` — every project

Walks projects.json and, per live project, runs cdprov --prune then (on apply) cdprov --refresh to re-link currently-declared items. Dry-run unless ARGS=--yes, and best-effort — a failure in one project warns and the run continues.

bash

make repair-fleet            # dry run across the whole fleet (prune preview)
make repair-fleet ARGS=--yes # apply: prune + refresh every registered project
make check-fleet             # confirm remaining drift afterwards

The prune→refresh order matters: prune clears stale/cross-machine links first, then refresh recreates the items the manifest still declares.

JSON Schemas

Located in schemas/. Each defines the exact structure a config file must follow.

manifest.schema.json

For provisions/*.json files:

json

{
  "_meta": {
    "project": "string (required)",
    "path": "string | null",
    "stack": ["string"]
  },
  "skills": ["string (unique)"],
  "commands": ["string (unique)"],
  "agents": ["string (unique)"]
}

projects.schema.json

For projects.json:

json

{
  "_meta": {
    "description": "string (required)",
    "updated": "YYYY-MM-DD"
  },
  "projects": [{
    "name": "string (required)",
    "path": "string (required)",
    "group": "string (required)",
    "active": "boolean (required)"
  }]
}

settings.schema.json

For settings.json:

permissions.allow / permissions.deny — string arrays
hooks — keyed by event name (SessionStart, PreToolUse, PostToolUse, Stop, Notification, SubagentStop)
Each hook entry has type: "command", command: string, optional timeout: integer
statusLine — type + command
enabledPlugins — map of string: boolean

File Layout

~/.claude/
├── Makefile                          # Entry point — run `make help`
├── schemas/
│   ├── manifest.schema.json          # Provisions manifest schema
│   ├── projects.schema.json          # Project registry schema
│   └── settings.schema.json          # Settings schema
├── scripts/checks/
│   ├── check-deps.sh                 # Dependency verification
│   ├── check-hooks.sh                # Hook script verification
│   ├── check-provisions.sh           # Manifest-vs-reality verification
│   ├── check-schemas.js              # JSON Schema validation runner
│   └── schema-validator.js           # Shared validator module (zero deps)
├── tests/
│   ├── test-checks.sh                # Integration tests (13 tests)
│   └── test-schemas.js               # Schema validator unit tests (34 tests)
└── .github/workflows/
    └── ci.yml                        # GitHub Actions — runs on push/PR

CI

GitHub Actions runs on every push and PR to master:

Checkout repo
Setup Node.js 18
Install jq
make check-deps check-schemas check-hooks (skip symlinks/provisions — no projects on CI runner)
make test

CI skips check-symlinks and check-provisions because they validate local provisioning state — the cdi-created symlinks in a working copy — which doesn't exist in a bare CI checkout (.claude/ symlinks are gitignored). Since #294 these checks default to the current repo (so make check in any working copy is no longer failed by drift in other registered projects); the cross-project sweep moved behind --all / make check-fleet. They remain local/dev gates by design.

Common Workflows

After cloning Bob on a new machine

bash

cd ~/projects/bigbrain    # BOB_SOURCE
make check-deps           # verify tools are installed
make check-schemas        # verify configs are valid
make check-hooks          # verify hooks are wired up

After changing settings.json or a hook

bash

make check-hooks check-schemas

After changing a provision manifest

bash

make check-schemas check-provisions

After adding a new project

bash

# Add to projects.json, then:
make check-schemas      # validates projects.json structure
make check-symlinks     # checks new project's symlinks
make check-provisions   # checks new project matches manifest

Before pushing changes to Bob

bash

make ci     # runs everything

Investigating a project that feels broken

bash

make check-symlinks     # find dangling symlinks
make check-provisions   # find manifest mismatches
cdprov --refresh /path/to/project   # fix it

Design Principles

Zero dependencies — only node, jq, git, make (all pre-installed or one brew install)
Every check exits 0 (pass) or 1 (fail) — no ambiguity
Actionable output — every failure includes a fix instruction
Idempotent — safe to run any target as many times as you want
Fast — full make check completes in under 5 seconds
Composable — run individual targets or combine them: make check-deps check-hooks

Verification & Testing System ​

Quick Start ​

Targets ​

Verification (make check) ​

Testing (make test) ​

CI (make ci) ​

Check Details ​

check-deps ​

check-schemas ​

check-symlinks ​

check-hooks ​

check-provisions ​

Repairing drift (prune & repair-fleet) ​

cdprov --prune — per-project ​

make repair-fleet — every project ​

JSON Schemas ​

manifest.schema.json ​

projects.schema.json ​

settings.schema.json ​

File Layout ​

CI ​

Common Workflows ​

After cloning Bob on a new machine ​

After changing settings.json or a hook ​

After changing a provision manifest ​

After adding a new project ​

Before pushing changes to Bob ​

Investigating a project that feels broken ​

Design Principles ​

Verification & Testing System

Quick Start

Targets

Verification (`make check`)

Testing (`make test`)

CI (`make ci`)

Check Details

check-deps

check-schemas

check-symlinks

check-hooks

check-provisions

Repairing drift (prune & repair-fleet)

`cdprov --prune` — per-project

`make repair-fleet` — every project

JSON Schemas

manifest.schema.json

projects.schema.json

settings.schema.json

File Layout

CI

Common Workflows

After cloning Bob on a new machine

After changing settings.json or a hook

After changing a provision manifest

After adding a new project

Before pushing changes to Bob

Investigating a project that feels broken

Design Principles