verification-system

Verification & Testing System

Bob’s infrastructure-as-code verification layer. One command to validate the entire system.

make check    # verify everything
make test     # run all tests
make ci       # both (what GitHub Actions runs)

Quick Start

cd ~/projects/bigbrain    # or wherever BOB_SOURCE is
make help                 # see all targets
make check-deps           # am I set up correctly?
make check                # is everything healthy?
make test                 # do all tests pass?

Targets

Verification (make check)

make check runs all five checks in order. Stops at first failure.

Target	What It Checks	Speed
check-deps	node >= 18, jq, git, make installed	instant
check-schemas	All JSON configs valid against schemas	instant
check-symlinks	All symlinks across all registered projects resolve	~2s
check-hooks	All hooks in settings.json exist + executable	instant
check-provisions	Each project’s symlinks match its manifest	~2s

Testing (make test)

Target	What It Tests	Count
test-schemas	Schema validator unit tests (type, enum, pattern, etc.)	34
test-warp-drive	Warp-drive state machine transitions	99
test-checks	Integration tests for all check scripts	13

CI (make ci)

Runs check then test. This is the target GitHub Actions calls on every push and PR to master.

---

Check Details

check-deps

Verifies the four required tools are installed:

  PASS  node 25.5.0 (>= 18)
  PASS  jq 1.7.1
  PASS  git 2.48.1
  PASS  make (GNU Make 3.81)

If something fails, the output includes a fix command:

  FAIL  node 16.20.0 (need >= 18)
        Fix: Install Node.js 18+ via nvm or brew

check-schemas

Validates every JSON configuration file against its JSON Schema definition:

File	Schema
projects.json	schemas/projects.schema.json
settings.json	schemas/settings.schema.json
provisions/*.json (17 files)	schemas/manifest.schema.json

If something fails, the output shows the exact path and error:

  FAIL  provisions/bodmail.json
        $._meta.project: expected type string, got number
        $.skills[3]: duplicate item "webapp-testing"

To fix: Edit the JSON file to match the schema. The error path ($._meta.project) tells you exactly where the problem is.

check-symlinks

Scans every project listed in projects.json, finds all symlinks in each project’s .claude/ directory, and verifies each one resolves to a real file.

  FAIL  /Users/you/Sites/myproject/.claude/commands/sprint.md -> /Users/you/.claude/registry/commands/sprint.md (dangling)

To fix dangling symlinks:

# Option 1: Re-provision the project (recommended)
cdprov --refresh /path/to/project
 
# Option 2: Remove the stale symlink manually
rm /path/to/.claude/commands/sprint.md

Projects that don’t exist on disk or lack a .claude/ directory are silently skipped.

check-hooks

Reads settings.json, extracts every hook command path (from SessionStart, PreToolUse, PostToolUse, Stop), and verifies:

1. The script file exists
2. The script is executable (chmod +x)

Also checks the statusLine.command if configured.

  PASS  /Users/you/.claude/hooks/temporal-context.sh
  FAIL  /Users/you/.claude/hooks/missing-hook.sh
        Fix: File not found — remove from settings.json or create the script

To fix:

# If the hook should exist:
chmod +x ~/.claude/hooks/the-hook.sh
 
# If the hook was removed:
# Edit settings.json and remove the hook entry

check-provisions

For each project in projects.json, loads its manifest (provisions/<project>.json or provisions/_default.json) and verifies that every declared skill, command, and agent is:

1. Present in the registry (registry/skills/, registry/commands/, registry/agents/)
2. Symlinked into the project’s .claude/ directory
3. Not dangling

  PASS  bodmail (bodmail.json)
  FAIL  myproject: commands/sprint declared in _default.json but not symlinked
  FAIL  myproject: agents/code-expert symlink is dangling

Failure types and fixes:

Failure	Meaning	Fix
declared but missing from registry	Manifest references something that doesn’t exist	Remove from manifest or add to registry
declared but not symlinked	Item exists in registry but wasn’t linked to project	Run cdprov --refresh /path/to/project
symlink is dangling	Symlink exists but target was moved/deleted	Run cdprov --refresh /path/to/project

---

JSON Schemas

Located in schemas/. Each defines the exact structure a config file must follow.

manifest.schema.json

For provisions/*.json files:

{
  "_meta": {
    "project": "string (required)",
    "path": "string | null",
    "stack": ["string"]
  },
  "skills": ["string (unique)"],
  "commands": ["string (unique)"],
  "agents": ["string (unique)"]
}

projects.schema.json

For projects.json:

{
  "_meta": {
    "description": "string (required)",
    "updated": "YYYY-MM-DD"
  },
  "projects": [{
    "name": "string (required)",
    "path": "string (required)",
    "group": "string (required)",
    "active": "boolean (required)"
  }]
}

settings.schema.json

For settings.json:

• permissions.allow / permissions.deny — string arrays
• hooks — keyed by event name (SessionStart, PreToolUse, PostToolUse, Stop, Notification, SubagentStop)
• Each hook entry has type: "command", command: string, optional timeout: integer
• statusLine — type + command
• enabledPlugins — map of string: boolean

---

File Layout

~/.claude/
├── Makefile                          # Entry point — run `make help`
├── schemas/
│   ├── manifest.schema.json          # Provisions manifest schema
│   ├── projects.schema.json          # Project registry schema
│   └── settings.schema.json          # Settings schema
├── scripts/checks/
│   ├── check-deps.sh                 # Dependency verification
│   ├── check-hooks.sh                # Hook script verification
│   ├── check-provisions.sh           # Manifest-vs-reality verification
│   ├── check-schemas.js              # JSON Schema validation runner
│   └── schema-validator.js           # Shared validator module (zero deps)
├── tests/
│   ├── test-checks.sh                # Integration tests (13 tests)
│   └── test-schemas.js               # Schema validator unit tests (34 tests)
└── .github/workflows/
    └── ci.yml                        # GitHub Actions — runs on push/PR

---

CI

GitHub Actions runs on every push and PR to master:

1. Checkout repo
2. Setup Node.js 18
3. Install jq
4. make check-deps check-schemas check-hooks (skip symlinks/provisions — no projects on CI runner)
5. make test

CI skips check-symlinks and check-provisions because those scan local project directories that don’t exist on the CI runner.

---

Common Workflows

After cloning Bob on a new machine

cd ~/projects/bigbrain    # BOB_SOURCE
make check-deps           # verify tools are installed
make check-schemas        # verify configs are valid
make check-hooks          # verify hooks are wired up

After changing settings.json or a hook

make check-hooks check-schemas

After changing a provision manifest

make check-schemas check-provisions

After adding a new project

# Add to projects.json, then:
make check-schemas      # validates projects.json structure
make check-symlinks     # checks new project's symlinks
make check-provisions   # checks new project matches manifest

Before pushing changes to Bob

make ci     # runs everything

Investigating a project that feels broken

make check-symlinks     # find dangling symlinks
make check-provisions   # find manifest mismatches
cdprov --refresh /path/to/project   # fix it

---

Design Principles

• Zero dependencies — only node, jq, git, make (all pre-installed or one brew install)
• Every check exits 0 (pass) or 1 (fail) — no ambiguity
• Actionable output — every failure includes a fix instruction
• Idempotent — safe to run any target as many times as you want
• Fast — full make check completes in under 5 seconds
• Composable — run individual targets or combine them: make check-deps check-hooks