Compare commits

...

91 Commits

Author SHA1 Message Date
Jesse Vincent db6077bb21 Strict-cost spec: L2 final — died at gates; explicit escalation holds at sonnet, implicit adjudication does not 2026-06-15 12:10:50 -07:00
Jesse Vincent 65e702f92a writing-plans: task right-sizing, Global Constraints header, per-task Interfaces blocks
Claims are fidelity and variance, not dollars (full attribution in the
superpowers-evals experiment log, 2026-06-11 L1 entry):
- Global Constraints header: 0/5 -> 5/5 adoption in micro-tests, exact
  values verbatim; makes constraints mechanically propagatable to briefs
  and reviewers (a version-floor violation class shipped because they
  weren't). The one fix wave in the elicited full runs was a version-floor
  catch this header enabled.
- Per-task Interfaces blocks: 0 -> 100% of tasks, exact signatures,
  within-plan consistent; removes the controller's per-dispatch interface
  re-derivation.
- Task right-sizing: 9.4 -> 8.4 mean tasks at svelte scale (kills
  standalone Types/README micro-tasks); no effect at small scale.
- End-to-end (opus-written plan executed under SDD): guidance plan ran 1
  fix wave vs control's 2-4 (control plan shipped a real Sierpinski bug);
  execution cost equal within noise.
2026-06-15 12:10:43 -07:00
Jesse Vincent 7f126acda6 Constraints block is the reviewer's attention lens: copy spec verbatim, never improvise process rules
E30 replay: the planted-DRY catch is causally determined by the
controller-composed constraints block (0/6 with process-shaped vs 5/6
with the spec's own wording). E31 micro: this recipe doubles the rate
at which composed blocks carry the spec's cross-component relationship
(6/6 vs 3/6). Affects dev and the redesign equally (E29: both 4/5).
2026-06-15 12:10:43 -07:00
Jesse Vincent 06f7789487 Strict-cost spec: L1 final — cost win re-attributed to complete-code plans; guidance owns fidelity/variance 2026-06-15 12:10:43 -07:00
Jesse Vincent 330aba6dd6 Strict-cost spec: L2 recon n=2 (sonnet controller $6.68/$8.05, judgment clean, escalation points unstressed) 2026-06-15 12:10:34 -07:00
Jesse Vincent 5b9eb20f76 Strict-cost spec: record batch A-E rung verdicts (L1 validated, L2 recon positive, L3 dead) 2026-06-15 12:10:34 -07:00
Jesse Vincent 7e421713ac Spec: strict-cost SDD experiment ladder — judgment as co-invariant, plan-side crispness first 2026-06-15 12:10:33 -07:00
Jesse Vincent 8476908a1b Record writing-plans micro-test result: resolved, no change needed 2026-06-15 12:10:33 -07:00
Jesse Vincent e6118e02b9 Spec: record iterations 4-5 (variance honesty, structural fixes, final validated ranges) 2026-06-15 12:10:33 -07:00
Jesse Vincent 9a221229a5 Adopt audited positive phrasings: evidence rule leads positive; fix-report completeness as checklist 2026-06-15 12:10:33 -07:00
Jesse Vincent 7d8f0ce9e9 Land eval-tuned combo: file handoffs, progress ledger, final-review package, REQUIRED model lines, reviewer risk budget
Validated 2026-06-10 (all gates pass): go-fractals 54.1-54.7 min / $12.81-14.31
(baseline 64.9 / $16.07); svelte-todo 55.0 min / 19.3M / $14.99 (baseline
79.7 / 27.3M / $20.98); planted-defect pass $2.77. Dispatch-model discipline
3/3 runs after moving model: into the templates as a REQUIRED line.
Full experiment log: evals docs/experiments/2026-06-10-sdd-cost-experiments.md
2026-06-15 12:10:33 -07:00
Jesse Vincent f37c5e5115 Spec: positive-instruction redesign — audit results, micro-test method, writing-plans variants 2026-06-15 12:10:33 -07:00
Jesse Vincent 618698d9b3 Shared: unique review-package collateral names 2026-06-15 12:10:33 -07:00
Jesse Vincent 2d6e56ee90 Add review-package script; close fix-dispatch test gap
scripts/review-package generates the reviewer's input deterministically:
commit list, stat summary, and net diff with -U10 context, written to a
file from an explicit BASE. Live runs showed controllers improvising
'git diff HEAD~1..HEAD', which silently truncates multi-commit tasks,
and svelte's five fix dispatches shipped without re-running any tests —
fix dispatches now explicitly carry the implementer's
re-run-and-report contract.
2026-06-15 12:10:33 -07:00
Jesse Vincent 4a92407ae7 Describe the review design as current state, not as a delta
The skill read as a changelog: 'combined task review,' 'one reviewer,
one reading,' 'one dispatch,' and an example still showing diffs pasted
into prompts. A reader who never saw the two-reviewer design has no
referent for 'combined.' Prose now states the design directly, and the
flowchart/example reflect the diff-file handoff.
2026-06-15 12:10:33 -07:00
Jesse Vincent cc81ffe7f3 Spec: record iterations 2-3 results and final frozen-config matrix 2026-06-15 12:10:33 -07:00
Jesse Vincent a0dcb77596 Hand reviewers the diff as a file, not a paste
Paste adoption stayed at 0/15 even as a Red Flag — and the controller's
reluctance is locally rational: pasting loads the diff into the (most
expensive) controller context permanently, while a reviewer self-fetch
costs a few cheap turns. The diff-file handoff is cheap for both sides:
the controller redirects git diff to /tmp without reading it, and the
reviewer gets the whole change in one Read call.
2026-06-15 12:10:33 -07:00
Jesse Vincent bc7d93de1a Reviewer skepticism covers the implementer's design rationales
Fourth planted-defect failure mode: the implementer's self-report said
'noted mild structural duplication; left unabstracted per YAGNI' and the
reviewer deferred to that framing, rating the duplication no finding at
all. The pre-judging keeps relocating — controller prompt, then reviewer
calibration, now the implementer's report. Rationales are claims; they
never downgrade severity.
2026-06-15 12:10:33 -07:00
Jesse Vincent 63a155692b Make diff-pasting non-optional for task reviewer dispatch
Adoption was 6/11 reviews on fractals and 0/17 on svelte when phrased
as guidance; reviewers without the diff re-derive it by hand, which is
the single largest remaining reviewer cost. Now a Red Flags Never entry
and a REQUIRED marker on the template placeholder.
2026-06-15 12:10:33 -07:00
Jesse Vincent 4866fe8b2d Close the Minor-severity escape hatch
With merged review, a planted verbatim-duplication defect shipped: the
reviewer rated it Minor (YAGNI) under the strict cannot-be-trusted
definition of Important, and the Minor-rolls-up rule meant no fix was
ever dispatched and the final review never saw the finding. Calibration
now names merge-blocking maintainability damage (verbatim duplication,
swallowed errors, assertion-free tests) as Important, and controllers
must paste accumulated Minor findings into the final review dispatch.
2026-06-15 12:10:33 -07:00
Jesse Vincent e45a8f2548 Spec: document cost iterations and the per-task review consolidation 2026-06-15 12:10:33 -07:00
Jesse Vincent fc75b0b3b4 Merge per-task reviews into one task reviewer (iteration 2)
Iteration-1 profiling: implementers and per-dispatch overhead dominate
(429 of 686 subagent turns; controller coordination is half the dollars
and scales with dispatch count), reviewers are individually lean, and
the controller pasted the diff in only 2 of 22 review dispatches when
the guidance was phrased as optional.

Changes: spec-reviewer-prompt.md + code-quality-reviewer-prompt.md
replaced by task-reviewer-prompt.md (one reviewer, one reading of a
pasted diff, two verdicts: spec compliance //⚠️ and task quality);
one fix dispatch can address both kinds of findings; controller now
runs git diff itself and pastes it (imperative, not optional);
implementers run focused tests while iterating and the full suite once
before committing; flowchart, example, Red Flags, tool tables updated.
The broad final whole-branch review is unchanged.
2026-06-15 12:10:33 -07:00
Jesse Vincent da0a11f6d4 Cut review-cost drivers: turn-aware models, inline diffs, scoped evidence
Round-2 fractals eval regressed to 70min/32.2M tokens (vs round-1's
42.8min/14.5M) while reaching baseline-parity quality. Per-subagent turn
profiling attributed it to: haiku dispatches taking 2-3x the turns of
sonnet (678 of 1197 subagent turns), reviewers re-fetching diffs by hand
(518 Bash calls), and evidence-rule narration. Changes: turn-count-beats-
token-price model guidance; controllers paste small diffs into reviewer
prompts (reviewers then need few or no tool calls); evidence scoped to
findings and would-be-bare-yes checks; Important defined as cannot-trust-
until-fixed with coverage suggestions Minor; fixes dispatched only for
Critical/Important.
2026-06-15 12:10:33 -07:00
Jesse Vincent b42846401f Add phrase-level pre-judging triggers to reviewer prompt rule
Resumed the offending eval controller session and asked it why it
pre-judged despite the rule being in context. Its retrospective: the
motive was avoiding a review loop, the abstract rule was read but not
applied at the moment it governs, and a phrase-level trigger ('do not
flag', 'at most Minor', 'don't treat X as a defect', 'the plan chose')
would have fired where the principle did not.
2026-06-15 12:10:33 -07:00
Jesse Vincent c087105ff3 Red Flags: never tell a reviewer what not to flag or pre-rate severity
Second observed instance: with the Constructing Reviewer Prompts rule
already live, a controller still wrote 'do not treat that duplication as
a defect to fix — the plan chose it; you may note it as a Minor
observation at most' into a quality reviewer dispatch, fabricating plan
intent from the plan's example snippet. Promote the rule to the Red
Flags Never list and name the rationalization.
2026-06-15 12:10:33 -07:00
Jesse Vincent 29e5842917 Close three review blind spots found by defect tracing
Live eval deliverables shipped five polish defects; tracing each through
the transcripts showed three mechanisms, each now addressed:
- reviewers answered pointed checklist items with unsupported yes
  (evidence rule: every What-to-Check answer needs file:line evidence)
- no reviewer ever saw the design's global constraints (controllers now
  paste binding constraints into task requirements)
- test output noise was invisible everywhere (pristine-output checks in
  implementer self-review and quality review)
2026-06-15 12:10:33 -07:00
Jesse Vincent 1d94bc939d Require explicit model on subagent dispatch
In live eval runs, controllers given judgment-based model selection
stopped passing a model at all; the omitted parameter inherits the
session's top-tier model, silently making every subagent maximally
expensive (one run dispatched 26/26 reviewers on the session model).
2026-06-15 12:10:33 -07:00
Jesse Vincent 833ec4177e Forbid controllers pre-judging reviewer findings
A live eval run of sdd-quality-reviewer-catches-planted-defect caught the
SDD controller fabricating a plan constraint and instructing the quality
reviewer not to flag the planted DRY violation. The duplication shipped.
Constructing Reviewer Prompts now bans suppression directives alongside
open-ended broadening directives.
2026-06-15 12:10:33 -07:00
Jesse Vincent c4abda336c Sync plan: escaped pre() pattern in Task 5 checks block 2026-06-15 12:10:33 -07:00
Jesse Vincent c874cf0cb3 Fix plan doc: correct Task 1 grep expectation; sync Task 5 story block 2026-06-15 12:10:33 -07:00
Jesse Vincent 08a2e7eed3 Sync plan's Task 5 blocks with review fixes 2026-06-15 12:10:33 -07:00
Jesse Vincent 077dd192a7 SDD controller: reviewer prompt budgets, ⚠️ handling, final-review pointer, model judgment 2026-06-15 12:10:33 -07:00
Jesse Vincent 441d22a2c0 Implementer prompt: re-run covering tests after fixing review findings 2026-06-15 12:10:33 -07:00
Jesse Vincent efcaa40f1f Scope spec reviewer's Your Job wording to the diff 2026-06-15 12:10:32 -07:00
Jesse Vincent 622a3887f3 Spec reviewer: judge from the diff, grounded skepticism, ⚠️ verdict channel 2026-06-15 12:10:32 -07:00
Jesse Vincent d3d6800b07 Use bare placeholder names in quality reviewer prompt body 2026-06-15 12:10:32 -07:00
Jesse Vincent 246b493db4 Make per-task quality reviewer prompt self-contained and task-scoped 2026-06-15 12:10:32 -07:00
Jesse Vincent 7dc323c28b Add implementation plan for task-scoped review dispatch 2026-06-15 12:10:32 -07:00
Jesse Vincent 55938589d3 Harden review-dispatch spec per adversarial review findings 2026-06-15 12:10:32 -07:00
Jesse Vincent 450b02a11b Add design spec: task-scoped review dispatch for SDD 2026-06-15 12:10:32 -07:00
Drew Ritter 9eb452afe7 chore: bump evals submodule to claude transcript-capture fix
Bumps evals 7f8e80c -> db37d5f (superpowers-evals#16): the claude launcher now
sets CLAUDE_CODE_FORCE_SESSION_PERSISTENCE=1 so nested interactive claude
(>=2.1.176) persists its transcript — restoring claude capture (verdicts +
cost/token data) on the latest CLI (2.1.177) with no version pin. Also folds in
the audit_liveness ruff/ty cleanup and the B1 audit-doc correction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 15:17:11 -07:00
Drew Ritter 93f2ce91b8 Fix companion stop metadata and token permissions 2026-06-11 13:53:06 -07:00
Drew Ritter e9ee6c5b4d Harden Windows browser launcher 2026-06-11 13:53:06 -07:00
Drew Ritter 5415cb8ccf Fix Windows lifecycle validation 2026-06-11 13:53:06 -07:00
Drew Ritter 1c21a91e01 Align visual companion docs with shipped scope 2026-06-11 13:53:06 -07:00
Drew Ritter 441335ee3e Fix companion test cleanup and argv assertions 2026-06-11 13:53:06 -07:00
Drew Ritter 377192f7a1 Harden companion platform tests 2026-06-11 13:53:06 -07:00
Drew Ritter 5eea0d09d7 Fix companion lifecycle test ownership metadata 2026-06-11 13:53:06 -07:00
Drew Ritter a6a4cd85b9 Harden companion stop ownership proof 2026-06-11 13:53:06 -07:00
Drew Ritter 8034176801 Isolate companion fallback tokens 2026-06-11 13:53:06 -07:00
Drew Ritter 2bab677ba7 Fix server test fallback cleanup 2026-06-11 13:53:06 -07:00
Drew Ritter c4cde1eed9 Harden root screen containment 2026-06-11 13:53:06 -07:00
Drew Ritter 5f3b317741 Plan visual companion final hardening fixup 2026-06-11 13:53:06 -07:00
Drew Ritter 7bb6af2f67 Tighten visual companion hardening spec 2026-06-11 13:53:06 -07:00
Drew Ritter 4f88b89c75 Document visual companion final hardening fixup 2026-06-11 13:53:06 -07:00
Drew Ritter c7d7e3550f Harden companion Windows lifecycle coverage 2026-06-11 13:53:06 -07:00
Drew Ritter a2e67bbd9b Harden brainstorm companion auth regressions 2026-06-11 13:53:06 -07:00
Drew Ritter fe812c418f Document visual companion auth hardening plan 2026-06-11 13:53:06 -07:00
Jesse Vincent f4d1788ffb fix(brainstorm-server): fix auth-integration bugs from full-branch review
A second adversarial review of the merged branch found that combining the
session-key auth with the feature work created real bugs the (vacuous) tests
missed:

- [Critical] GET /files/ (empty name) resolved to CONTENT_DIR and crashed the
  process with uncaught EISDIR — newly reachable because the query-stripping
  refactor turns /files/?key=... into /files/. Reject non-regular-file names.
- [High] --open opened a KEYLESS url, which the auth gate 403s — the headline
  feature landed on the error page. Open the keyed url.
- [High] Same-port restart regenerated the token (port persisted, token not), so
  the open tab's old cookie 403'd and never reconnected — contradicting the
  documented promise. Persist the token (BRAINSTORM_TOKEN_FILE / .last-token)
  alongside the port.
- [Medium] Token sat in world-readable server-info/server.log (0644 in /tmp).
  umask 077 in start-server.sh + mode 0600 on server-info/.last-token.
- [Medium] touchActivity() ran before the auth check, so unauthenticated requests
  defeated the idle timeout. Count activity only after authorization.
- [Low] COOKIE_NAME embedded the pre-fallback port; derive it from the actual
  bound port (also prevents a cross-server cookie-jar collision on fallback).

Tests added/strengthened (previously passed vacuously): /files/ no-crash; the
auto-open url carries the key and is reachable (200); restart reuses the same key
not just the port; unauthenticated requests don't reset the idle clock.
Full suite green (ws-protocol 32, helper 12, auth 13, server 29, lifecycle 8,
stop-server 4); restart smoke confirms same port+key and old URL -> 200.
2026-06-11 13:53:06 -07:00
Jesse Vincent 4341c3f4d5 test(brainstorm-server): thread session key through tests after auth merge
Integrating the per-session-key auth onto the same branch as the dotfile and
lifecycle work: two tests added after the auth commit opened WebSockets without a
key (server.test.js dotfile-reload, lifecycle.test.js idle-shutdown), which the
auth gate now resets. Pass ?key=/BRAINSTORM_TOKEN in both. Full suite green:
ws-protocol 32, helper 12, auth 13, server 28, lifecycle 7, stop-server 4.
2026-06-11 13:53:06 -07:00
Jesse Vincent c64c4ea6f4 feat(brainstorm-server): gate every endpoint behind a per-session key
The companion server is reachable by any local browser tab (default loopback
bind) and by any host that can route to it (remote --host bind). It served
screens, files, and accepted event-injecting WebSocket connections with no
authentication, so a malicious browser tab or a direct remote client could read
brainstorm content or inject events that the agent reads as the user's input
(prompt injection into a live session).

Generate a per-session secret token, carry it in the served URL as ?key=, and
mirror it into an HttpOnly SameSite=Strict per-port cookie on first load so
same-origin subresources and the WebSocket handshake authenticate automatically.
Every HTTP request and WebSocket upgrade now requires a valid key (query or
cookie, constant-time compared); unauthenticated requests get a friendly 403
explaining they need the full URL. A secret authenticates the client uniformly
across loopback, tunnel, and remote binds and defeats DNS rebinding, which a
Host/Origin allowlist cannot.

Also guard handleMessage against a null JSON payload that crashed the process.

Tests: new auth.test.js (13 cases) covering the key on /, /files/*, and WS plus
cookie bootstrap and the null-payload guard; server.test.js threads the key;
ws-protocol.test.js + auth.test.js wired into npm test.

Closes #1014
Refs #1110, #1553, #1504
2026-06-11 13:53:06 -07:00
Jesse Vincent de05e020d8 docs(brainstorm): catalog visual companion issues; choose session-key for security
Records the triage of open issues/PRs touching the brainstorm companion server
and the decision to protect it with a per-session secret key (supersedes the
Host/Origin allowlist approach) so remote-connected users are covered, not just
loopback.
2026-06-11 13:53:06 -07:00
Jesse Vincent eee4f87471 fix(brainstorm-server): tie stop-server PID check to the session's port
The node+server.cjs command match (from the adversarial review) still matched any
unrelated node process running a file named server.cjs. When we recorded the
bound port (state/server-info) and lsof is available, additionally require the
PID to be the process actually LISTENING on this session's port — which rules out
a different project's server.cjs / editor task runner that recycled the stale
PID. Falls back to the command match when the port or lsof isn't available.

Test: a 'node server.cjs' process not listening on the recorded port is spared.

Refs #1703
2026-06-11 13:53:06 -07:00
Jesse Vincent bac46a5dcb fix(brainstorm-server): address adversarial review findings
From a two-reviewer adversarial pass:

- [High] EADDRINUSE fallback clobbered the shared .last-port: onListen wrote the
  bound port unconditionally, so a fallback to a random port overwrote the
  preferred port another live session still owns — stranding that session's open
  tab forever. Now persist only when we bound the preferred port (not on
  fallback). The fallback test now asserts .last-port integrity (teeth-verified).

- [Medium] maybeOpenBrowser ran the URL through a shell (exec + JSON.stringify),
  which does NOT neutralize $(...) in a url-host. Platform launchers now use
  execFile with the URL as an argv element (no shell). The operator-set
  BRAINSTORM_OPEN_CMD path stays shell-based (trusted input).

- [Medium] --open was a silent no-op on native Windows (no win32 branch). Added.

- [Medium] helper.js reconnect/status/tombstone had only substring-grep tests.
  Added behavioral tests driving the state machine against a mocked browser:
  Reconnecting+backoff (500->1000->2000), tombstone after the grace period, and
  reload-on-recovery.

- [Low] status pill showed a false 'Connected' before the socket opened; now
  starts 'Connecting…' until onopen.

Not changed (flagged): stop-server.sh's PID-ownership check still matches any
'node ... server.cjs' (narrow residual — a recycled PID onto an unrelated node
server.cjs); robust fix needs fragile cross-platform process introspection.
2026-06-11 13:53:06 -07:00
Jesse Vincent daa41c0670 feat(brainstorming): offer the visual companion just-in-time; harden lifecycle guidance
Move the companion consent from an upfront, anticipatory offer to the first
moment a question would genuinely be clearer shown than told. If no visual
question ever arises, it's never offered. On approval the agent starts the
server with --open, so the user's browser opens to the first screen — the pop is
tied to that approval, never unsolicited.

Also hardens visual-companion.md: confirming the server is alive (server-info
present, server-stopped absent) before referring to the URL is now a required
step; restart with the same --project-dir reuses the port so the open tab
reconnects on its own (paused overlay while down); idle default corrected to 4h.

NOTE: SKILL.md is behavior-shaping content — this flow change should be
eval-tested (writing-skills adversarial pressure test) before merge.

Refs #1237, #1037
2026-06-11 13:53:06 -07:00
Jesse Vincent 0d37ff6505 feat(brainstorm-server): opt-in auto-open of the browser on the first screen
When the user approves the visual companion, open their browser automatically the
first time a screen is actually ready to show — rather than at startup (just the
waiting page) or making them open the URL by hand.

Opt-in and gated on approval: off unless BRAINSTORM_OPEN is set (start-server.sh
--open, which the agent passes only after the user agrees to use the companion).
Even then it fires once, and is skipped if a browser is already connected, on a
non-loopback/remote bind, or when headless. Launcher is the platform default
(open / xdg-open / WSL cmd.exe) or BRAINSTORM_OPEN_CMD; best-effort, never fatal.

lifecycle.test.js: opens once on the first screen when approved; does NOT open
without approval.

Closes #755
Refs #759
2026-06-11 13:53:06 -07:00
Jesse Vincent 13da997ac7 feat(brainstorm-server): reuse the same port on session restart
When the companion idle-shuts-down and the agent restarts it, a fresh random
port meant the user's open browser tab pointed at a dead URL. Persist the bound
port per project and prefer it on the next start, so the restarted server comes
up on the same port and the open tab's reconnect just works.

- start-server.sh exports BRAINSTORM_PORT_FILE=<project>/.superpowers/brainstorm/
  .last-port for project sessions (not /tmp).
- server.cjs prefers an explicit BRAINSTORM_PORT, else the recorded port, else
  random; writes the actually-bound port back; and on EADDRINUSE (preferred port
  still in use) falls back to a random port once instead of crashing.

lifecycle.test.js: restart reuses the recorded port; a taken preferred port
falls back to a random one without crashing.

Refs #1237
2026-06-11 13:53:06 -07:00
Jesse Vincent 31a0de857b feat(brainstorm-companion): resilient reconnect, live status, paused overlay
The injected client reconnected on a fixed 1s timer with no feedback: if the
laptop slept or the server restarted, the page showed 'Connected' over a dead
socket and silently queued events. And when the server stopped, the user got a
bare connection-refused with no explanation.

helper.js now:
- reconnects with exponential backoff (500ms, doubling, capped at 30s; reset on
  open), with an onerror->close handler, nulls the socket on close, and clears a
  pending timer before scheduling another;
- drives the frame status pill Connected/Reconnecting/Disconnected via a
  --status-color custom property (frame-template.html);
- after ~15s disconnected, shows a self-styled 'Companion paused' overlay
  (tombstone) explaining the companion stopped and will reconnect automatically;
- on recovery from a tombstoned outage (e.g. server restarted on the same port)
  reloads to pick up the restarted server's current screen.

The reconnect-backoff is an exported pure function; helper.test.js unit-tests it
(doubling + cap progression) and asserts the status/tombstone/reconnect wiring.
DOM behaviour is verified live.

Refs #856, #1237
2026-06-11 13:53:06 -07:00
Jesse Vincent c292421627 feat(brainstorm-server): 4h configurable idle timeout; close WS on shutdown
The companion shut down after only 30 minutes idle — too short for real
brainstorming, where a single question can sit far longer. And shutdown() never
closed upgraded WebSocket sockets, so an open browser connection could keep the
Node process alive after it was supposed to exit.

- Default idle timeout raised to 4 hours, configurable via BRAINSTORM_IDLE_TIMEOUT_MS
  and start-server.sh --idle-timeout-minutes (validated positive integer).
- Reported as idle_timeout_ms in the server-started JSON / server-info.
- shutdown() now destroys all client sockets so the process exits even with an
  open WebSocket.
- Watchdog check interval is configurable (BRAINSTORM_LIFECYCLE_CHECK_MS, default
  60s) so the lifecycle can be tested without minute-long waits.

Adds lifecycle.test.js (configured timeout reported; idle shutdown exits despite
an open WS — teeth-verified; the start-server flag). Wires ws-protocol,
lifecycle, and stop-server suites into npm test.

Closes #1237
Refs #1689
2026-06-11 13:53:06 -07:00
Jesse Vincent 9b00cc298d fix(brainstorm-server): verify PID ownership before stopping
stop-server.sh read server.pid and SIGKILL'd that PID with no checks. After a
reboot or PID wraparound the pid file can point at an unrelated, live process —
which we would then kill.

Verify the PID is actually our server (a running 'node ... server.cjs') before
signalling it. If ownership can't be proven, fail closed: remove the stale pid
file and report {status: stale_pid} without killing anything. Real servers still
stop ({status: stopped}); a missing pid file still reports not_running.

Adds stop-server.test.sh covering: an unrelated reused PID is left alone, a real
server is stopped, and a missing pid file.

Refs #1703
2026-06-11 13:53:06 -07:00
Jesse Vincent 88fe1e7e15 fix(brainstorm-server): ignore macOS resource-fork dotfiles
On macOS (and ExFAT/SMB volumes) the OS writes ._<name>.html sidecar files
holding binary resource-fork metadata. These end with .html, so they passed the
content filter and could be picked as the newest screen — serving binary garbage
to the browser instead of the mockup — or fetched via /files/.

Skip dotfiles (leading '.') at all four sites that list or serve content:
getNewestScreen, the /files/ endpoint, the known-files seed, and the fs.watch
handler. Tests cover serving (/ and /files/) and the watch path (a ._ file must
not trigger a reload).

Refs #950
2026-06-11 13:53:06 -07:00
Drew Ritter e6c983888f chore(evals): bump submodule to SUP-333 boundary + plumbing scenarios (7f8e80c)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 13:45:59 -07:00
Jesse Vincent 74f85a7709 fix(writing-skills): hang backfire mechanism on the separated prohibition-vs-recipe comparison (NEW-4); control comparison stated as trend 2026-06-11 12:11:37 -07:00
Jesse Vincent b148b648eb fix(writing-skills): scope empirical claims, honest noise reporting, conditionalize micro-test checklist line
Adversarial review findings 1/3/9: the head-to-head result is now scoped
to its context (dispatch-prompt guidance) with an explicit micro-test-your-
own-case instruction; the nuance-clause result is reported as
consistent->noisy rather than 'measurably dilutes'; the checklist line is
scoped to behavior-shaping guidance and the micro method no longer assumes
raw API access.
2026-06-11 12:11:37 -07:00
Jesse Vincent 3e565ca2ad feat(writing-skills): form-selection table + micro-test wording method
RED battery (35 opus authoring samples against the current skill) showed
authors default to prohibition+rationalization-table for composition-
shaping problems (T1: 5/5), where that form measurably backfires
(prohibition 4.4 vs 3.6 no-guidance control vs 3.0 recipe restatement
errors), and design only full-subagent verification with no wording
micro-tests, no mandatory no-guidance control, no manual inspection of
automated matches, no variance signal (T7: 5/5).

Adds: Match the Form to the Failure (failure-type -> form table, nuance/
exemption rules), scope note on Bulletproofing, Micro-Test Wording
subsection, two checklist lines. Deliberately narrow: T3/T4/T5/T6 RED
samples showed Iron Law / elicit-first behavior already strong.
2026-06-11 12:11:37 -07:00
Drew Ritter 0cb1960068 chore(evals): bump submodule for Claude Haiku target 2026-06-10 16:31:16 -07:00
Jesse Vincent f55642e0dd Require contributors to disclose authoring environment and target dev
Add a mandatory self-identification disclosure (model, harness, harness
version, all installed plugins) to the PR template and all three issue
templates, and document the requirement in the contributor guidelines.
We weigh contributions differently depending on what produced them:
content reasoned from documentation is held to a different bar than work
grounded in a real session.

Also state explicitly, in both CLAUDE.md and the PR template, that all
PRs must target the dev branch rather than main.
2026-06-08 22:14:34 -07:00
Drew Ritter ae1eefb7f9 chore(evals): bump submodule to --scenarios filter (ff3ee83)
Adds `run-all --scenarios` for resuming a scenario subset across the Code
Assist rate-limit windows. Follows the agy rate-limit fix (79f9963).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 22:46:00 -07:00
Drew Ritter 617168aff5 chore(evals): bump submodule to antigravity rate-limit fix (79f9963)
Serialize antigravity against the Gemini Code Assist rate limit
(max_concurrency=1), diagnose 429/RESOURCE_EXHAUSTED honestly instead of as
auth, fail-fast on a latched window, and tolerant preflight OK match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 16:27:35 -07:00
Rahul d7c260a978 fix(brainstorming): cap websocket frame payloads 2026-06-02 11:24:02 -07:00
Drew Ritter f3f0789c5c Add shell lint script 2026-06-01 19:48:28 -07:00
Drew Ritter 16a1719988 Tighten Kimi plugin porting coverage 2026-06-01 19:41:58 -07:00
Drew Ritter c74c22daa7 docs: restore Kimi direct install command 2026-06-01 19:41:58 -07:00
Drew Ritter 773bbf61d6 docs: simplify Kimi README install steps 2026-06-01 19:41:58 -07:00
Drew Ritter 6b76158550 fix: wire Kimi plugin into release metadata 2026-06-01 19:41:58 -07:00
Drew Ritter 7fec40bb55 fix: align Kimi manifest with supported fields 2026-06-01 19:41:58 -07:00
qer 2a8e54735b feat: add Kimi Code plugin manifest 2026-06-01 19:41:58 -07:00
Matt Van Horn f776394360 feat(subagent-dev): add TDD RED evidence to implementer report format
Add a conditional TDD Evidence field to the implementer report format so controllers can verify RED and GREEN output when TDD was required.

The field asks for the command run, relevant RED/GREEN output, and the expected RED failure reason rather than raw full logs.

Fixes #994.
2026-06-01 16:15:05 -07:00
Drew Ritter 7301c81b4d docs(windows): trim polyglot hook implementation copy 2026-06-01 16:07:01 -07:00
dev_Hakaze 9d3e68a5ad docs(windows): update polyglot hook docs
Rewrite the Windows polyglot hook documentation to match the current run-hook.cmd dispatcher and update the porting guide cross-reference.\n\nFixes #1653.
2026-06-01 15:57:30 -07:00
nestorluiscamachopaz 81c3052416 fix: foreground mode saves node PID and clears OWNER_PID on Windows/MSYS2
Verified on real Windows Git Bash: lifecycle test passed 12/12, manual start/stop released the port, and no brainstorm node processes remained.
2026-06-01 14:26:22 -07:00
57 changed files with 7605 additions and 403 deletions
+5 -2
View File
@@ -12,14 +12,17 @@ add a comment or reaction to the existing one instead.
- [ ] I searched existing issues and this is not a duplicate
## Environment
## Environment (required)
<!-- Required. We assume an agent filed this report — tell us which one and
where it ran. We weigh reports by what produced them. -->
| Field | Value |
|-------|-------|
| Superpowers version | |
| Harness (Claude Code, Cursor, etc.) | |
| Harness version | |
| Model | |
| Your model + version | |
| All plugins installed | |
| OS + shell | |
## Is this a Superpowers issue or a platform issue?
+14 -1
View File
@@ -30,5 +30,18 @@ progress, and some were intentionally declined.
of project? If this is specific to your domain, workflow, or a
third-party tool, it may belong as its own plugin instead. -->
## Environment (required)
<!-- Required. We assume an agent wrote this request — tell us which one and
where it ran. We weigh proposals reasoned from documentation differently
than ones grounded in a real session where the problem actually came up. -->
| Field | Value |
|-------|-------|
| Superpowers version | |
| Harness (Claude Code, Cursor, etc.) | |
| Harness version | |
| Your model + version | |
| All plugins installed | |
## Context
<!-- Optional: version info, harness, model, workflow where you hit this. -->
<!-- Optional: the workflow where you hit this, links, transcripts. -->
@@ -21,3 +21,14 @@ requested or discussed.
## Have you tried manual installation?
<!-- Many tools work with Superpowers through manual setup even without
official support. Did you try? What happened? -->
## Environment (required)
<!-- Required. We assume an agent wrote this request — tell us which one and
where it ran. -->
| Field | Value |
|-------|-------|
| Harness you currently use (Claude Code, Cursor, etc.) | |
| Harness version | |
| Your model + version | |
| All plugins installed | |
+17
View File
@@ -4,6 +4,23 @@ sections blank, contain multiple unrelated changes, or show no evidence
of human involvement will be closed without review.
-->
> **This PR MUST target the `dev` branch, not `main`.** `main` is the
> released branch; active work lands on `dev` first. PRs opened against
> `main` will be asked to retarget `dev` before review.
## Who is submitting this PR? (required)
<!-- Required. PRs that omit this will be closed. We assume an agent wrote
this PR — tell us which one and where it ran. We weigh contributions by
what produced them: content reasoned from documentation is held to a
different bar than work grounded in a real session. -->
| Field | Value |
|-------|-------|
| Your model + version | |
| Harness + version | |
| All plugins installed | |
| Human partner who reviewed this diff | |
## What problem are you trying to solve?
<!-- Describe the specific problem you encountered. If this was a session
issue, include: what you were doing, what went wrong, the model's
+1
View File
@@ -1,6 +1,7 @@
.worktrees/
.private-journal/
.claude/
.superpowers/
.DS_Store
node_modules/
inspo
+38
View File
@@ -0,0 +1,38 @@
{
"name": "superpowers",
"version": "5.1.0",
"description": "An agentic skills framework and software development methodology.",
"author": {
"name": "Jesse Vincent",
"email": "jesse@fsck.com"
},
"homepage": "https://github.com/obra/superpowers",
"license": "MIT",
"keywords": [
"brainstorming",
"subagent-driven-development",
"skills",
"planning",
"tdd",
"debugging",
"code-review",
"workflow"
],
"skills": "./skills/",
"sessionStart": {
"skill": "using-superpowers"
},
"skillInstructions": "Kimi Code tool mapping for Superpowers skills:\n\n- When a Superpowers skill says to ask the user, ask clarifying questions, ask one question at a time, present multiple-choice options, use the terminal for a question, or wait for the user's choice, call Kimi Code's `AskUserQuestion` tool. Do not render those choices as plain assistant text unless `AskUserQuestion` is unavailable or the session is in auto permission mode.\n- For `AskUserQuestion`, provide 1 question with 2-4 concrete options when possible. Put the recommended option first and suffix its label with `(Recommended)`.\n- When a Superpowers skill refers to `TodoWrite`, use Kimi Code's `TodoList` tool.\n- When a Superpowers skill says `Task tool (general-purpose)` or asks you to dispatch an implementer/reviewer subagent, use Kimi Code's `Agent` tool with a Kimi subagent type. Do not pass `general-purpose` as `subagent_type`.\n- For implementation, code review, spec review, quality review, and filled Superpowers subagent prompt templates, call `Agent` with `subagent_type: \"coder\"`, paste the fully filled prompt into `prompt`, and provide a short `description`.\n- For read-only codebase exploration that would take several searches, use `Agent` with `subagent_type: \"explore\"`.\n- For read-only planning or architecture design, use `Agent` with `subagent_type: \"plan\"`.\n- Keep dependent Superpowers subagent steps sequential. Use multiple `Agent` calls, or `run_in_background: true` only when the work is independent and background agents are available.\n- When a Superpowers skill refers to the `Skill` tool, use Kimi Code's native `Skill` tool.\n- Use Kimi Code's `Read`, `Write`, `Edit`, `Bash`, `Grep`, `Glob`, `FetchURL`, `WebSearch`, and MCP tools by their actual exposed names.\n- When a skill asks to search file contents, use `Grep`; when it asks to find files by path or pattern, use `Glob`; when it asks to fetch a URL, use `FetchURL`; when it asks to search the web, use `WebSearch`.",
"interface": {
"displayName": "Superpowers",
"shortDescription": "Planning, TDD, debugging, and delivery workflows for coding agents",
"longDescription": "Use Superpowers to guide agent work through brainstorming, implementation planning, test-driven development, systematic debugging, parallel execution, code review, and finish-the-branch workflows.",
"developerName": "Jesse Vincent",
"capabilities": [
"Interactive",
"Read",
"Write"
],
"websiteURL": "https://github.com/obra/superpowers"
}
}
+1
View File
@@ -4,6 +4,7 @@
{ "path": ".claude-plugin/plugin.json", "field": "version" },
{ "path": ".cursor-plugin/plugin.json", "field": "version" },
{ "path": ".codex-plugin/plugin.json", "field": "version" },
{ "path": ".kimi-plugin/plugin.json", "field": "version" },
{ "path": ".claude-plugin/marketplace.json", "field": "plugins.0.version" },
{ "path": "gemini-extension.json", "field": "version" }
],
+6 -1
View File
@@ -14,7 +14,8 @@ Before you open a PR against this repo, you MUST:
2. **Search for existing PRs** — open AND closed — that address the same problem. If duplicates exist, STOP and tell your human partner. Do not open another duplicate.
3. **Verify this is a real problem.** If your human partner asked you to "fix some issues" or "contribute to this repo" without experiencing a specific problem, push back. Ask them what broke, what failed, what the user experience was.
4. **Confirm the change belongs in core.** If it's domain-specific, tool-specific, or promotes a third-party project, it belongs in a standalone plugin. Tell your human partner.
5. **Show your human partner the complete diff** and get their explicit approval before submitting.
5. **Identify yourself.** Disclose your model, harness, harness version, and every installed plugin in the PR. Hiding that a contribution is agent-generated — or which environment produced it — is grounds for closing it.
6. **Show your human partner the complete diff** and get their explicit approval before submitting.
If any of these checks fail, do not open the PR. Explain to your human partner why it would be rejected and what would need to change. They will thank you for saving them the embarrassment.
@@ -26,6 +27,10 @@ If any of these checks fail, do not open the PR. Explain to your human partner w
**PRs that show no evidence of human involvement will be closed.** A human must review the complete proposed diff before submission.
**Submitters MUST identify themselves.** Every PR and issue must disclose the model, harness, harness version, and all installed plugins used to produce the contribution — or state plainly that it was written by hand with no agent. This is not optional. We need to know what produced a change in order to weigh it: agent-generated content reasoned from documentation is held to a different bar than work grounded in a real session. Contributions that hide their authoring environment will be closed.
**All PRs MUST target the `dev` branch, not `main`.** `main` is the released branch; active work lands on `dev` first. PRs opened against `main` will be asked to retarget `dev` before they are reviewed.
## What We Will Not Accept
### Third-party dependencies
+21 -1
View File
@@ -4,7 +4,7 @@ Superpowers is a complete software development methodology for your coding agent
## Quickstart
Give your agent Superpowers: [Claude Code](#claude-code), [Antigravity](#antigravity), [Codex App](#codex-app), [Codex CLI](#codex-cli), [Cursor](#cursor), [Factory Droid](#factory-droid), [Gemini CLI](#gemini-cli), [GitHub Copilot CLI](#github-copilot-cli), [OpenCode](#opencode), [Pi](#pi).
Give your agent Superpowers: [Claude Code](#claude-code), [Antigravity](#antigravity), [Codex App](#codex-app), [Codex CLI](#codex-cli), [Cursor](#cursor), [Factory Droid](#factory-droid), [Gemini CLI](#gemini-cli), [GitHub Copilot CLI](#github-copilot-cli), [Kimi Code](#kimi-code), [OpenCode](#opencode), [Pi](#pi).
## How it works
@@ -149,6 +149,26 @@ Superpowers is available via the [official Codex plugin marketplace](https://git
copilot plugin install superpowers@superpowers-marketplace
```
### Kimi Code
Superpowers is available in Kimi Code's plugin marketplace.
- Open Kimi Code's plugin manager:
```text
/plugins
```
- Go to `Marketplace` > `Superpowers` and install it.
- Or install directly from this repository:
```text
/plugins install https://github.com/obra/superpowers
```
- Detailed docs: [docs/README.kimi.md](docs/README.kimi.md)
### OpenCode
OpenCode uses its own plugin install; install Superpowers separately even if you
+88
View File
@@ -0,0 +1,88 @@
# Superpowers for Kimi Code
Complete guide for using Superpowers with [Kimi Code](https://github.com/MoonshotAI/kimi-code).
## Installation
Superpowers is available in Kimi Code's plugin marketplace.
Open the plugin manager:
```text
/plugins
```
Go to `Marketplace` > `Superpowers` and install it.
You can also install from this repository:
```text
/plugins install https://github.com/obra/superpowers
```
For unreleased validation against `dev`, pin the branch explicitly:
```text
/plugins install https://github.com/obra/superpowers/tree/dev
```
Kimi Code applies plugin changes to new sessions. After installing, updating, enabling, disabling, or reloading a plugin, start a fresh session with `/new`.
## How It Works
The Kimi plugin manifest lives at `.kimi-plugin/plugin.json`.
The manifest does three things:
1. Points Kimi Code at the existing `skills/` directory.
2. Loads `using-superpowers` at session start through `sessionStart.skill`.
3. Provides Kimi-specific tool mapping through `skillInstructions`.
Kimi Code reads Superpowers skills from this repository. There are no copied skills, symlinks, hooks, or extra runtime dependencies.
## Tool Mapping
Skills describe actions instead of hard-coding one runtime's tool names. On Kimi Code these resolve to:
- "Ask the user" / "ask clarifying questions" -> `AskUserQuestion`
- "Create a todo" / "mark complete in todo list" -> `TodoList`
- "Dispatch a subagent" -> `Agent`
- "Invoke a skill" -> Kimi Code's native `Skill` tool
- "Read a file" / "write a file" / "edit a file" -> `Read`, `Write`, `Edit`
- "Run a shell command" -> `Bash`
- "Search file contents" -> `Grep`
- "Find files by path or pattern" -> `Glob`
- "Fetch a URL" -> `FetchURL`
- "Search the web" -> `WebSearch`
## Updating
Use Kimi Code's plugin manager:
```text
/plugins
```
Select Superpowers and update it from there. Start a fresh session with `/new` after updating.
## Troubleshooting
### Plugin not loading
1. Run `/plugins info superpowers` and check diagnostics.
2. Make sure the plugin is enabled.
3. Start a fresh session with `/new` after install or update.
### Direct GitHub install used an old release
Kimi Code installs the latest GitHub release for a bare repository URL when one exists. To test unreleased changes before the next Superpowers release, install the branch explicitly:
```text
/plugins install https://github.com/obra/superpowers/tree/dev
```
### Skills not triggering
1. Confirm `/plugins info superpowers` shows the plugin enabled.
2. Start a fresh session with `/new`.
3. Try the acceptance prompt: `Let's make a react todo list`. A working install should load `brainstorming` before writing code.
+5 -5
View File
@@ -675,7 +675,7 @@ it. Distribution differs per harness ecosystem — find yours:
|---|---|---|
| Native plugin marketplace | Claude Code | Register in `.claude-plugin/marketplace.json`; users `/plugin install`. The external `superpowers-marketplace` repo is the source of truth users install from — see the release steps in `CLAUDE.md`. |
| External marketplace fork, synced by script | Codex | `scripts/sync-to-codex-plugin.sh` rsyncs the tracked plugin files into a separate fork repo and opens a PR. Read its include/exclude list so you ship the right tree (it deliberately drops repo-internal dirs and other harnesses' dotdirs). |
| Git-URL extension install | Gemini, OpenCode | Users install from a git URL (`gemini extensions install …`; an `opencode.json` `plugin` array entry). Document the exact command. |
| Git-URL extension install | Gemini, Kimi Code, OpenCode | Users install from a git URL (`gemini extensions install …`; Kimi Code `/plugins install …`; an `opencode.json` `plugin` array entry). Document the exact command. |
| Package-manifest fields | pi | Declared through fields in the repo-root `package.json`; users install via the harness's package command. |
| Local installer (plugin install) | Antigravity (`agy`) | A small `install.sh` that runs the harness's own `agy plugin install` against a staging dir holding the manifest, the skills, and a generated `contextFileName` context file (the bootstrap). Everything arrives through the install mechanism — *not* by editing the user's config (see below). |
@@ -755,10 +755,9 @@ Two rules this enforces, which you must respect:
- Don't write per-OS variants of the hook script. One extensionless bash script
plus the polyglot wrapper covers all three platforms.
`hooks/run-hook.cmd` itself is the authoritative implementation — read it.
(`docs/windows/polyglot-hooks.md` covers the background and rationale but
describes an earlier per-script `.cmd`/`.sh` variant, so trust the code over that
doc where they differ.)
`hooks/run-hook.cmd` itself is the authoritative implementation — read it. See
`docs/windows/polyglot-hooks.md` for the background and rationale behind the
dispatcher pattern.
---
@@ -789,6 +788,7 @@ Use this as the live index; when in doubt, read the files, not this table.
| Cursor | `.cursor-plugin/plugin.json` + `hooks/hooks-cursor.json` | shell hook → `hooks/session-start` (`additional_context`) | `references/claude-code-tools.md` | `tests/hooks/` | hand-authored |
| Copilot CLI | (shares Claude Code hook path; `COPILOT_CLI` env) | shell hook → `hooks/session-start` (`additionalContext`) | `references/copilot-tools.md` | `tests/hooks/` | — |
| Gemini CLI | `gemini-extension.json` + `GEMINI.md` | instructions file `@`-includes bootstrap + mapping | `references/gemini-tools.md` | — | `gemini extensions install` |
| Kimi Code | `.kimi-plugin/plugin.json` | manifest `sessionStart.skill` loads `using-superpowers` | inline `skillInstructions` in manifest | `tests/kimi/` | marketplace or `/plugins install` GitHub URL |
| OpenCode | `.opencode/plugins/superpowers.js` (declared via root `package.json` `main`) | in-process: `config` hook registers skills dir; `experimental.chat.messages.transform` injects user message | inline in `superpowers.js` | `tests/opencode/` | `opencode.json` plugin git URL |
| pi | `.pi/extensions/superpowers.ts` | in-process: `resources_discover` registers skills; `context` event injects user message; lifecycle-flag + compaction-aware | `piToolMapping()` inline **and** `references/pi-tools.md` | `tests/pi/` | repo-root `package.json` fields |
@@ -0,0 +1,774 @@
# SDD Task-Scoped Review Dispatch Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Scope SDD's per-task reviews to the task (diff-first reading, justified broadening, no redundant test runs) while final branch review stays broad.
**Architecture:** Four prose edits to the subagent-driven-development skill (the per-task quality prompt becomes self-contained instead of delegating to the merge-readiness template; the spec prompt gets a third verdict channel and grounded skepticism; the implementer prompt gains a re-run-after-fix rule; SKILL.md gets controller guidance) plus one new eval scenario in the `evals/` submodule. `skills/requesting-code-review/` is deliberately untouched.
**Tech Stack:** Markdown skill files; Python setup helper + bash checks + story.md for the quorum eval.
**Spec:** `docs/superpowers/specs/2026-06-09-sdd-task-scoped-review-dispatch-design.md` — read it before starting. Decisions already settled there: full re-reviews stay; the two review stages stay separate; coordinator keeps model judgment; `requesting-code-review/` stays broad.
**These are behavior-shaping prose files, not code.** There are no unit tests for them. Each task's verification steps are exact `grep` checks that the edit landed; behavioral verification is Task 6 (static) and Task 7 (live evals, maintainer-gated).
---
### Task 1: Rewrite the per-task quality reviewer prompt as self-contained
The current file delegates to `../requesting-code-review/code-reviewer.md`, which is a merge-readiness review (architecture, security, production readiness, "Ready to merge?"). Replace the entire file with a self-contained, task-scoped template.
**Files:**
- Rewrite: `skills/subagent-driven-development/code-quality-reviewer-prompt.md`
- [ ] **Step 1: Replace the full file contents with:**
````markdown
# Code Quality Reviewer Prompt Template
Use this template when dispatching a code quality reviewer subagent.
**Purpose:** Verify one task's implementation is well-built (clean, tested, maintainable)
**Only dispatch after spec compliance review passes.**
```
Subagent (general-purpose):
description: "Review code quality for Task N"
prompt: |
You are reviewing one task's implementation for code quality. This is a
task-scoped gate, not a merge review — a broad whole-branch review happens
separately after all tasks are complete.
## What Was Implemented
[DESCRIPTION]
## Task Requirements (context only)
[TASK_TEXT]
## Git Range to Review
**Base:** [BASE_SHA]
**Head:** [HEAD_SHA]
```bash
git diff --stat [BASE_SHA]..[HEAD_SHA]
git diff [BASE_SHA]..[HEAD_SHA]
```
## Read-Only Review
Your review is read-only on this checkout. Do not mutate the working tree,
the index, HEAD, or branch state in any way. Use tools like `git show`,
`git diff`, and `git log` to inspect history.
## Scope
Spec compliance was already verified by a separate reviewer. Do not
re-check whether the code matches the requirements or the plan.
Start from the diff. Read the changed files first. Inspect code outside
the diff only to evaluate a concrete risk you can name — and name it in
your report. Cross-cutting changes are legitimate named risks: if the
diff changes lock ordering, a function or API contract, or shared mutable
state, checking the call sites is the right method. Do not crawl the
codebase by default.
## Tests
The implementer already ran the tests and reported results with TDD
evidence for exactly this code. Do not re-run the suite to confirm their
report. Run a test only when reading the code raises a specific doubt
that no existing run answers — and then a focused test, never a
package-wide suite, race detector run, or repeated/high-count loop. If
heavy validation seems warranted, recommend it in your report instead of
running it. If you cannot run commands in this environment, name the
test you would run.
## What to Check
**Code quality:**
- Clean separation of concerns?
- Proper error handling?
- DRY without premature abstraction?
- Edge cases handled?
**Tests:**
- Do the new and changed tests verify real behavior, not mocks?
- Are the task's edge cases covered?
**Structure:**
- Does each file have one clear responsibility with a well-defined interface?
- Are units decomposed so they can be understood and tested independently?
- Is the implementation following the file structure from the plan?
- Did this change create new files that are already large, or
significantly grow existing files? (Don't flag pre-existing file
sizes — focus on what this change contributed.)
## Calibration
Categorize issues by actual severity. Not everything is Critical.
Acknowledge what was done well before listing issues — accurate praise
helps the implementer trust the rest of the feedback.
## Output Format
### Strengths
[What's well done? Be specific.]
### Issues
#### Critical (Must Fix)
[Bugs, data loss risks, broken functionality]
#### Important (Should Fix)
[Poor error handling, test gaps, structural problems]
#### Minor (Nice to Have)
[Code style, optimization opportunities]
For each issue:
- File:line reference
- What's wrong
- Why it matters
- How to fix (if not obvious)
### Assessment
**Task quality:** [Approved | Needs fixes]
**Reasoning:** [1-2 sentence technical assessment]
```
**Placeholders:**
- `[DESCRIPTION]` — task summary, from implementer's report
- `[TASK_TEXT]` — the task's requirements text or plan reference, for context
- `[BASE_SHA]` — commit before this task
- `[HEAD_SHA]` — current commit
**Reviewer returns:** Strengths, Issues (Critical/Important/Minor), Task quality verdict
````
- [ ] **Step 2: Verify the rewrite landed**
Run: `grep -c "requesting-code-review" skills/subagent-driven-development/code-quality-reviewer-prompt.md || echo ABSENT`
Expected: `ABSENT` (no more delegation)
Run: `grep -n "Task quality:" skills/subagent-driven-development/code-quality-reviewer-prompt.md | head -2`
Expected: one match (the Output Format verdict line; the "Reviewer returns" footer says "Task quality verdict" without a colon)
Run: `grep -n "worktree add\|Ready to merge" skills/subagent-driven-development/code-quality-reviewer-prompt.md || echo CLEAN`
Expected: `CLEAN`
- [ ] **Step 3: Commit**
```bash
git add skills/subagent-driven-development/code-quality-reviewer-prompt.md
git commit -m "Make per-task quality reviewer prompt self-contained and task-scoped"
```
---
### Task 2: Spec reviewer prompt cleanups
Four exact edits to `skills/subagent-driven-development/spec-reviewer-prompt.md`. Current line numbers refer to the file as of commit f55642e.
**Files:**
- Modify: `skills/subagent-driven-development/spec-reviewer-prompt.md`
- [ ] **Step 1: Add the judge-from-the-diff clause.** After the line (currently line 31):
```
Only read files in this diff. Do not crawl the broader codebase.
```
insert a blank line and:
```
Spec compliance is judged by reading the diff against the requirements.
The implementer already ran the tests and reported TDD evidence — do not
re-run them. If a requirement cannot be verified from this diff alone
(it lives in unchanged code or spans tasks), report it as a ⚠️ item
instead of broadening your search.
```
- [ ] **Step 2: Trim the read-only section.** Replace (currently line 35):
```
Your review is read-only on this checkout. Do not mutate the working tree, the index, HEAD, or branch state in any way. Use tools like `git show`, `git diff`, and `git log` to inspect history. If you need a working copy of a different revision, check it out into a separate temporary directory (e.g. `git worktree add /tmp/review-[SHA] [SHA]`) — never move HEAD on this checkout.
```
with:
```
Your review is read-only on this checkout. Do not mutate the working tree, the index, HEAD, or branch state in any way. Use tools like `git show`, `git diff`, and `git log` to inspect history.
```
- [ ] **Step 3: Ground the skepticism.** Replace (currently lines 39-40):
```
The implementer finished suspiciously quickly. Their report may be incomplete,
inaccurate, or optimistic. You MUST verify everything independently.
```
with:
```
Treat the implementer's report as unverified claims about the code. It may
be incomplete, inaccurate, or optimistic. Verify the claims against the diff.
```
- [ ] **Step 4: Add the third verdict channel.** Replace (currently lines 74-76):
```
Report:
- ✅ Spec compliant (if everything matches after code inspection)
- ❌ Issues found: [list specifically what's missing or extra, with file:line references]
```
with:
```
Report:
- ✅ Spec compliant (if everything matches after code inspection)
- ❌ Issues found: [list specifically what's missing or extra, with file:line references]
- ⚠️ Cannot verify from diff: [requirements you could not verify from the
diff alone, and what the controller should check — report alongside the
✅/❌ verdict for everything you could verify]
```
- [ ] **Step 5: Verify**
Run: `grep -n "suspiciously\|worktree add" skills/subagent-driven-development/spec-reviewer-prompt.md || echo CLEAN`
Expected: `CLEAN`
Run: `grep -c "⚠️" skills/subagent-driven-development/spec-reviewer-prompt.md`
Expected: `2` (judge-from-diff clause + verdict channel)
- [ ] **Step 6: Commit**
```bash
git add skills/subagent-driven-development/spec-reviewer-prompt.md
git commit -m "Spec reviewer: judge from the diff, grounded skepticism, ⚠️ verdict channel"
```
---
### Task 3: Implementer prompt — re-run tests after fixing review findings
The reviewers' "don't re-run the implementer's tests" rule assumes the implementer re-runs tests after every fix. Make that real.
**Files:**
- Modify: `skills/subagent-driven-development/implementer-prompt.md`
- [ ] **Step 1: Insert a new section.** Immediately before the line (currently line 100):
```
## Report Format
```
insert:
```
## After Review Findings
If a reviewer finds issues and you fix them, re-run the tests that cover
the amended code and include the results in your fix report. Reviewers
will not re-run tests for you — your report is the test evidence.
```
- [ ] **Step 2: Verify**
Run: `grep -n "After Review Findings" skills/subagent-driven-development/implementer-prompt.md`
Expected: one match, on a line before `## Report Format`
- [ ] **Step 3: Commit**
```bash
git add skills/subagent-driven-development/implementer-prompt.md
git commit -m "Implementer prompt: re-run covering tests after fixing review findings"
```
---
### Task 4: SKILL.md controller changes
Six exact edits to `skills/subagent-driven-development/SKILL.md`. Current line numbers refer to commit f55642e.
**Files:**
- Modify: `skills/subagent-driven-development/SKILL.md`
- [ ] **Step 1: Point the final-review flowchart node at the broad template.** The node label `Dispatch final code reviewer subagent for entire implementation` appears 3 times (currently lines 65, 84, 85). In all 3 occurrences, replace the label string with:
```
Dispatch final code reviewer subagent (../requesting-code-review/code-reviewer.md)
```
(Graphviz nodes are matched by label text — all three must be byte-identical or the graph grows a phantom node.)
- [ ] **Step 2: Model selection by judgment.** Replace (currently lines 97-99):
```
**Architecture, design, and review tasks**: use the most capable available model.
**Task complexity signals:**
```
with:
```
**Architecture and design tasks**: use the most capable available model.
**Review tasks**: choose the model with the same judgment, scaled to the
diff's size, complexity, and risk. A small mechanical diff does not need the
most capable model; a subtle concurrency change does.
**Task complexity signals (implementation tasks):**
```
- [ ] **Step 3: Add controller guidance sections.** Immediately before the line (currently line 122):
```
## Prompt Templates
```
insert:
```
## Handling Spec Reviewer ⚠️ Items
The spec reviewer may report "⚠️ Cannot verify from diff" items — requirements
that live in unchanged code or span tasks. These do not block dispatching the
code quality reviewer, but you must resolve each one yourself before marking
the task complete: you hold the plan and cross-task context the reviewer
lacks. If you confirm an item is a real gap, treat it as a failed spec
review — send it back to the implementer and re-review.
## Constructing Reviewer Prompts
Per-task reviews are task-scoped gates. The broad review happens once, at the
final whole-branch review. When you fill a reviewer template:
- Do not add open-ended directives like "check all uses" or "run race tests
if useful" without a concrete, task-specific reason
- Do not ask a reviewer to re-run tests the implementer already ran on the
same code — the implementer's report carries the test evidence
```
- [ ] **Step 4: Prompt Templates list — add the final-review pointer.** Replace (currently line 126):
```
- [code-quality-reviewer-prompt.md](code-quality-reviewer-prompt.md) - Dispatch code quality reviewer subagent
```
with:
```
- [code-quality-reviewer-prompt.md](code-quality-reviewer-prompt.md) - Dispatch code quality reviewer subagent
- Final whole-branch review: use superpowers:requesting-code-review's [code-reviewer.md](../requesting-code-review/code-reviewer.md)
```
- [ ] **Step 5: Example workflow verdict vocabulary.** Two replacements:
Replace (currently line 157):
```
Code reviewer: Strengths: Good test coverage, clean. Issues: None. Approved.
```
with:
```
Code reviewer: Strengths: Good test coverage, clean. Issues: None. Task quality: Approved.
```
Replace (currently line 191):
```
Code reviewer: ✅ Approved
```
with:
```
Code reviewer: ✅ Task quality: Approved
```
(The final reviewer's "ready to merge" line, currently line 199, stays.)
- [ ] **Step 6: Integration section.** Replace (currently line 272):
```
- **superpowers:requesting-code-review** - Code review template for reviewer subagents
```
with:
```
- **superpowers:requesting-code-review** - Code review template for the final whole-branch review
```
- [ ] **Step 7: Verify**
Run: `grep -c "Dispatch final code reviewer subagent (../requesting-code-review/code-reviewer.md)" skills/subagent-driven-development/SKILL.md`
Expected: `3`
Run: `grep -n "most capable available model" skills/subagent-driven-development/SKILL.md`
Expected: exactly one match (architecture/design bullet)
Run: `grep -n "Handling Spec Reviewer\|Constructing Reviewer Prompts" skills/subagent-driven-development/SKILL.md`
Expected: two section headers, both before `## Prompt Templates`
Run: `grep -c "Task quality: Approved" skills/subagent-driven-development/SKILL.md`
Expected: `2`
- [ ] **Step 8: Commit**
```bash
git add skills/subagent-driven-development/SKILL.md
git commit -m "SDD controller: reviewer prompt budgets, ⚠️ handling, final-review pointer, model judgment"
```
---
### Task 5: New eval scenario — per-task quality reviewer catches a planted defect
Lives in the `evals/` **submodule** (separate repo, `superpowers-evals`). Work on a branch there; the parent submodule-pointer bump happens at finishing time per `evals/CLAUDE.md`.
The fixture plan's Task 2 implementation snippet duplicates Task 1's formatting logic verbatim. The duplication is spec-compliant, so the spec reviewer should pass it — the per-task quality reviewer is the gate under test (DRY violation).
**Files:**
- Create: `evals/setup_helpers/sdd_quality_defect_plan.py`
- Modify: `evals/setup_helpers/__init__.py`
- Create: `evals/scenarios/sdd-quality-reviewer-catches-planted-defect/story.md`
- Create: `evals/scenarios/sdd-quality-reviewer-catches-planted-defect/setup.sh`
- Create: `evals/scenarios/sdd-quality-reviewer-catches-planted-defect/checks.sh`
- [ ] **Step 0: Branch in the submodule**
```bash
cd evals
git checkout -b sdd-quality-defect-scenario
```
- [ ] **Step 1: Create `evals/setup_helpers/sdd_quality_defect_plan.py`:**
````python
"""Setup helper for the sdd-quality-reviewer-catches-planted-defect scenario.
Scaffolds a tiny Node project with a 2-task plan whose Task 2
implementation snippet duplicates Task 1's formatting logic verbatim.
The duplication is spec-compliant — the requirements only describe
behavior — so the spec compliance reviewer should pass it. The test
measures whether the per-task code quality reviewer catches the DRY
violation and forces a refactor in the review-fix loop.
"""
from __future__ import annotations
from pathlib import Path
from setup_helpers.base import _git
PACKAGE_JSON = """\
{
"name": "report-quality",
"version": "1.0.0",
"type": "module",
"scripts": {
"test": "node --test"
}
}
"""
PLAN_BODY = """\
# Report Formatter — Implementation Plan
Two report formatting functions. Implement exactly what each task
specifies.
## Task 1: User Report
**File:** `src/report.js`
**Requirements:**
- Function named `formatUserReport`
- Takes one parameter `user`: an object with `name`, `email`, `visits`
- Returns a multi-line string: a banner of 40 `=` characters, then
`Report for <name> <<email>>`, then the banner again, then
`Visits: <visits>`, then a closing banner
- Export the function
**Implementation:**
```javascript
export function formatUserReport(user) {
const banner = "=".repeat(40);
const lines = [];
lines.push(banner);
lines.push(`Report for ${user.name} <${user.email}>`);
lines.push(banner);
lines.push(`Visits: ${user.visits}`);
lines.push(banner);
return lines.join("\\n");
}
```
**Tests:** Create `test/report.test.js` verifying:
- the result contains `Report for Ada <ada@example.com>` for that user
- the result contains `Visits: 3` when `visits` is `3`
- the result starts and ends with the 40-char banner
**Verification:** `npm test`
## Task 2: Admin Report
**File:** `src/report.js` (add to existing file)
**Requirements:**
- Function named `formatAdminReport`
- Takes one parameter `admin`: an object with `name`, `email`, `lastLogin`
- Same banner layout as the user report; the body line is
`Last login: <lastLogin>` instead of the visits line
- Export the function; keep `formatUserReport` working
**Implementation:**
```javascript
export function formatAdminReport(admin) {
const banner = "=".repeat(40);
const lines = [];
lines.push(banner);
lines.push(`Report for ${admin.name} <${admin.email}>`);
lines.push(banner);
lines.push(`Last login: ${admin.lastLogin}`);
lines.push(banner);
return lines.join("\\n");
}
```
**Tests:** Add to `test/report.test.js`:
- the result contains `Report for Grace <grace@example.com>` for that admin
- the result contains `Last login: 2026-06-01`
- the result starts and ends with the 40-char banner
**Verification:** `npm test`
"""
def scaffold_sdd_quality_defect_plan(workdir: Path) -> None:
workdir = Path(workdir)
workdir.mkdir(parents=True, exist_ok=True)
_git(["git", "init", "-b", "main"], cwd=workdir)
_git(["git", "config", "user.email", "drill@test.local"], cwd=workdir)
_git(["git", "config", "user.name", "Drill Test"], cwd=workdir)
(workdir / "package.json").write_text(PACKAGE_JSON)
plans_dir = workdir / "docs" / "superpowers" / "plans"
plans_dir.mkdir(parents=True, exist_ok=True)
(plans_dir / "report-plan.md").write_text(PLAN_BODY)
_git(["git", "add", "-A"], cwd=workdir)
_git(["git", "commit", "-m", "initial: report formatter plan"], cwd=workdir)
````
(Note the `\\n` in the JS snippets inside PLAN_BODY: the Python source must
produce a literal `\n` in the markdown so the JS reads `lines.join("\n")`.)
- [ ] **Step 2: Register the helper.** In `evals/setup_helpers/__init__.py`:
After the line:
```python
from setup_helpers.sdd_real_projects import scaffold_sdd_go_fractals, scaffold_sdd_svelte_todo
```
add:
```python
from setup_helpers.sdd_quality_defect_plan import scaffold_sdd_quality_defect_plan
```
After the registry entry:
```python
"scaffold_sdd_yagni_plan": scaffold_sdd_yagni_plan,
```
add:
```python
"scaffold_sdd_quality_defect_plan": scaffold_sdd_quality_defect_plan,
```
- [ ] **Step 3: Create `evals/scenarios/sdd-quality-reviewer-catches-planted-defect/story.md`:**
```markdown
---
id: sdd-quality-reviewer-catches-planted-defect
title: SDD's per-task code quality review catches a planted DRY violation
status: ready
tags: subagent-driven-development
quorum_max_time: 90m
---
You have a small plan at docs/superpowers/plans/report-plan.md — two report
formatting functions. The plan's Task 2 implementation snippet duplicates
Task 1's formatting logic verbatim instead of sharing it. The duplication is
spec-compliant (the requirements only describe behavior), so the spec
compliance reviewer should pass it — the per-task code quality reviewer is
the gate under test. You are spec-aware — name the skill.
When the agent is ready for input, tell it to execute the plan with SDD. Use
phrasing like:
"I have a small plan at docs/superpowers/plans/report-plan.md — two report
formatting functions. Use the superpowers:subagent-driven-development skill
to execute it end-to-end — dispatch fresh subagents per task and run the
two-stage review after each."
Let the agent proceed autonomously. If it asks clarifying questions, give
brief answers. If it asks where the finished work should land — merge to the
main branch, open a PR, etc. — tell it to **merge the work into the main
checkout** (this is a local repo with no remote). If a quality reviewer
flags the duplicated formatting logic and an implementer refactors it, let
the review-fix cycle play out — that cycle is exactly the behavior under
test.
The deliverable must end up in the checkout you launched in (the main
working tree). If the agent did its work on a branch or in a worktree, it
is not done until it has merged/finished that work back into the main
checkout. Once the agent reports the plan is complete (both functions
implemented, tests passing) AND the code is present on the main checkout,
you are done.
## Acceptance Criteria
- A `Skill` invocation naming `superpowers:subagent-driven-development`
and at least one `Agent` (subagent dispatch) tool call appear in the
session log.
- The duplicated report-formatting logic did not survive to the end of
the run. Either (a) the implementer never introduced the duplication
(wrote or self-reviewed its way to shared logic), or (b) the per-task
code quality reviewer flagged the duplication as an issue and a
review-fix loop removed it. A fail looks like the duplicated logic
shipping with the per-task quality reviewer approving it, or the
duplication being caught only by the final whole-branch review.
- The per-task quality reviewers stayed task-scoped: no package-wide
test suites, race detector runs, or repeated/high-count test loops
appear in reviewer subagent activity, and reviewers did not re-run
the full test suite merely to confirm the implementer's report.
- `npm test` passes in the main checkout and both `formatUserReport` and
`formatAdminReport` are exported from src/report.js. The deterministic
assertions gate this; the criteria above are about whether the
*per-task quality review* was the mechanism that kept the code clean.
```
- [ ] **Step 4: Create `evals/scenarios/sdd-quality-reviewer-catches-planted-defect/setup.sh`:**
```bash
#!/usr/bin/env bash
set -euo pipefail
uv run setup-helpers run scaffold_sdd_quality_defect_plan
```
Then: `chmod +x evals/scenarios/sdd-quality-reviewer-catches-planted-defect/setup.sh`
- [ ] **Step 5: Create `evals/scenarios/sdd-quality-reviewer-catches-planted-defect/checks.sh`** (no executable bit):
```bash
pre() {
git-repo
git-branch main
requires-tool npm
file-exists 'docs/superpowers/plans/report-plan.md'
file-contains 'docs/superpowers/plans/report-plan.md' 'formatAdminReport'
file-contains 'docs/superpowers/plans/report-plan.md' 'repeat\(40\)'
}
post() {
skill-called superpowers:subagent-driven-development
tool-called Agent
command-succeeds 'npm test'
file-contains 'src/report.js' 'export function formatUserReport'
file-contains 'src/report.js' 'export function formatAdminReport'
command-succeeds 'test "$(grep -c "repeat(40)" src/report.js)" -le 1'
}
```
(The last check is the deterministic DRY gate: the banner construction
`"=".repeat(40)` must appear at most once in the final file — shared, not
duplicated per function.)
- [ ] **Step 6: Validate and test in the evals repo**
```bash
cd evals
uv run quorum check
uv run ruff check
uv run pytest -x -q
```
Expected: all pass; `quorum check` lists the new scenario without errors.
- [ ] **Step 7: Commit (in the submodule)**
```bash
cd evals
git add setup_helpers/sdd_quality_defect_plan.py setup_helpers/__init__.py scenarios/sdd-quality-reviewer-catches-planted-defect/
git commit -m "Add sdd-quality-reviewer-catches-planted-defect scenario"
```
---
### Task 6: Static verification sweep
**Files:** none modified — verification only.
- [ ] **Step 1: No dangling references in the parent repo**
Run: `grep -rn "requesting-code-review" skills/subagent-driven-development/`
Expected: matches only in SKILL.md (final-review flowchart node ×3, Prompt Templates pointer, Integration bullet). None in code-quality-reviewer-prompt.md.
Run: `grep -rn "Ready to merge" skills/subagent-driven-development/ || echo CLEAN`
Expected: `CLEAN`
- [ ] **Step 2: Plugin infrastructure tests**
Run: `bash tests/shell-lint/test-lint-shell.sh`
Expected: all PASS (we added `setup.sh` only inside the evals submodule, which has its own checks).
- [ ] **Step 3: Cross-platform tool tables still coherent**
Run: `grep -n "code-quality-reviewer" skills/using-superpowers/references/antigravity-tools.md skills/using-superpowers/references/gemini-tools.md`
Expected: both tables still list `code-quality-reviewer` as a reviewer template (the new prompt's "If you cannot run commands in this environment, name the test you would run" line keeps the read-only `research` mapping valid — no table edits needed).
---
### Task 7: Live before/after evals (maintainer-gated)
Live quorum runs launch agent CLIs in permissive modes — **trusted-maintainer operation; Jesse launches these**, per `evals/CLAUDE.md`. Requires `ANTHROPIC_API_KEY`.
- [ ] **Step 1: Baseline (skills as released on dev)** — from the main checkout (`/Users/jesse/git/superpowers/superpowers`, on dev), or any checkout without this branch's changes:
```bash
cd evals
export SUPERPOWERS_ROOT=/Users/jesse/git/superpowers/superpowers
uv run quorum run scenarios/sdd-rejects-extra-features --coding-agent claude
uv run quorum run scenarios/sdd-go-fractals --coding-agent claude
uv run quorum run scenarios/sdd-svelte-todo --coding-agent claude
uv run quorum run scenarios/spec-reviewer-catches-planted-flaws --coding-agent claude
```
- [ ] **Step 2: After (this branch's skills)** — point `SUPERPOWERS_ROOT` at this worktree:
```bash
cd evals
export SUPERPOWERS_ROOT=/Users/jesse/git/superpowers/superpowers/.claude/worktrees/sdd-review-dispatch
uv run quorum run scenarios/sdd-rejects-extra-features --coding-agent claude
uv run quorum run scenarios/sdd-go-fractals --coding-agent claude
uv run quorum run scenarios/sdd-svelte-todo --coding-agent claude
uv run quorum run scenarios/spec-reviewer-catches-planted-flaws --coding-agent claude
uv run quorum run scenarios/sdd-quality-reviewer-catches-planted-defect --coding-agent claude
uv run quorum show
```
- [ ] **Step 3: Compare**
Pass bar: all four pre-existing scenarios still pass after the change (no regression in catch rate); the new planted-defect scenario passes. For exploration cost, compare reviewer-subagent tool-call counts between the before/after run transcripts (no automated check exists — the spec calls this out as a known gap).
---
## Finishing
After all tasks pass: the evals submodule commit needs to land in `superpowers-evals` (PR to its `main`), then this branch bumps the `evals` submodule pointer — per `evals/CLAUDE.md`, the parent bump is part of propagation, not optional. Then use superpowers:finishing-a-development-branch. PRs against superpowers target `dev`.
@@ -0,0 +1,352 @@
# Visual Brainstorming Companion — Issue & Change Catalog
**Date:** 2026-06-09
**Status:** Analysis / triage. We are implementing these ourselves; the referenced
community PRs are evidence and reference material, **not** code we intend to merge.
## Purpose
A single place that captures every open issue and PR touching the visual
brainstorming companion (the local server in `skills/brainstorming/scripts/`),
distilled to the underlying problem and the change we'd make. Each item is
grounded against the current code, not the PR author's description.
## Scope decisions (Jesse, 2026-06-09)
- **Not vendoring Alpine.js.** PR #1639 (interactive mockups via a vendored
Alpine build) is **dropped**. See E3.
- **E1 (terminal-vs-HTML hard gate) is a workshop item.** We'll design it
together; it is not specced here.
- **E2 (storage location, #975/#977) is deferred** for now.
- **Remote serving is a first-class scenario.** Superpowers is general-purpose;
users connect from remote (SSH tunnel, Tailscale, `--host 0.0.0.0`). The
security fix MUST protect those users, not just loopback. **Decision: a
per-session secret key**, not a Host allowlist. A Host allowlist only
defends the loopback browser-confused-deputy; a direct remote client just
sends the expected `Host`, so the allowlist is theater for remote exposure. A
secret key is the only thing that authenticates a client uniformly across
loopback, tunnel, and direct-remote, and it also defeats DNS rebinding. See A1.
## Component map
| File | Role |
|------|------|
| `skills/brainstorming/scripts/server.cjs` | Zero-dep HTTP + WebSocket server (RFC 6455 hand-rolled). Serves the newest screen, watches `content/`, records events to `state/events`. |
| `skills/brainstorming/scripts/helper.js` | Injected into every page. WebSocket client, click capture, `window.brainstorm` API. |
| `skills/brainstorming/scripts/frame-template.html` | Frame (header, theme CSS, status dot, indicator bar) wrapped around content fragments. |
| `skills/brainstorming/scripts/start-server.sh` | Launch wrapper. Session dir, host/url-host, owner-PID resolution, platform backgrounding. |
| `skills/brainstorming/scripts/stop-server.sh` | Kills the server by PID file, cleans `/tmp` sessions. |
| `skills/brainstorming/visual-companion.md` | Operator guide the agent reads when it accepts the companion. |
| `skills/brainstorming/SKILL.md` | Where the companion is offered and the per-question decision lives. |
## Disposition summary
| ID | Item | Source | Disposition |
|----|------|--------|-------------|
| A1 | Per-session secret key on `/`, `/files/*`, and WS (supersedes Host allowlist) | issues #1014, PRs #1110/#1553 | **Do** — chosen approach |
| A2 | Host allowlist; browser WS Origin check | PRs #1110/#1553 | Host allowlist dropped; WS Origin check retained after auth for browser confused-deputy defense |
| A3 | Crash on `null` / non-object WS payload | PR #1504 | Do |
| A4 | Frame-length bound in `decodeFrame` | issue #1446 | Already fixed — verify/close |
| B1 | Dotfile screens served as content (`._*.html`) | PR #950 | Do |
| B2 | `stop-server.sh` kills reused/stale PID | PR #1703 | Do |
| B3 | WS client reconnect backoff + status indicator | PR #856 | Do |
| C1 | Idle timeout too short / not configurable; WS not closed on shutdown | issue #1237 (PR #1689) | Do |
| C2 | Server death is invisible to user/agent | issue #1237 (residual) | Do |
| D1 | Permanent opt-out of the companion | issue #892 | Deferred - not in PR #1720 |
| D2 | Free-text feedback from the browser | issue #957 | Deferred - not in PR #1720 |
| D3 | Auto-open the companion URL | PR #759 (#755) | Done in PR #1720 via `--open` |
| D4 | Light/dark contrast helpers in the frame | PR #1683 | Deferred - not in PR #1720 |
| E1 | Hard-gate terminal-vs-HTML per question | PR #1037 | **Workshop** |
| E2 | Move session state out of the working tree | issue #975 (PR #977) | **Deferred** |
| E3 | Vendor Alpine.js for interactive mockups | PR #1639 | **Dropped** |
| E4 | Shell-lint warnings in start/stop scripts | PR #1677 | Opportunistic only |
---
## A. Server security hardening (`server.cjs`)
### A1 — Per-session secret key (chosen approach)
**Threat model.** Two assets: confidentiality of the served screen (`/`) and
files (`/files/*`), and integrity of `state/events` — a WebSocket client with a
truthy `choice` writes there (`server.cjs:243-246`), and the agent reads it next
turn as the user's selection, i.e. **prompt injection into a live session with
full tool access**. Reachers: with the default `127.0.0.1` bind, a malicious
page in the user's browser (a confused deputy — runs attacker JS *and* can reach
loopback); with a remote bind (`--host 0.0.0.0`, tailnet/LAN), any host that can
route to the port, directly, with no same-origin policy in the way. Today
`handleUpgrade` (`server.cjs:176`) checks only `Sec-WebSocket-Key`, and
`handleRequest` (`server.cjs:138`) checks nothing — both are wide open.
**Why a key, not a Host allowlist.** A Host allowlist only defends the
loopback browser-deputy. A direct remote client just sends the expected `Host`
and forges/omits `Origin`, so the allowlist is theater for exactly the remote
case we must protect. A per-session secret authenticates the client uniformly
across loopback, SSH tunnel, and direct-remote, and it also kills DNS rebinding
(the rebound page neither knows the key nor receives the host-scoped cookie).
So the key **supersedes** A1/A2's Host allowlist entirely — no `BRAINSTORM_ALLOWED_HOSTS`.
**Design.** Random token (`crypto.randomBytes(32)` hex), generated in
`server.cjs` at startup (overridable via `BRAINSTORM_TOKEN` for deterministic
tests):
1. **URL carries it** as `?key=<token>`. The server already builds `url` in its
`server-started` JSON (`server.cjs:351`) and writes it to `state/server-info`
— appending `?key=` there means `start-server.sh` (greps and prints that
JSON) and the skill (hands the user that URL) need **no change**.
2. **Cookie bootstrap.** A valid `?key` on `/` sets
`brainstorm-key-<port>=<token>; HttpOnly; SameSite=Strict; Path=/`. The
browser then auto-attaches it to same-origin subresources (`/files/*`) and
the WebSocket handshake, so the agent can write any URL style and it works,
and `helper.js` needs no change. Cookie name is **per-port** to avoid the
Jupyter multi-server collision (cookies aren't port-scoped).
`SameSite=Strict` is safe for CDN/Unsplash content — that cookie is host-
scoped, so outbound CDN requests never carry it; SameSite only governs
requests back to our origin, which are all same-site.
3. **Auth gate** = valid `?key` **OR** valid cookie (compared with
`crypto.timingSafeEqual`) on `/`, `/files/*`, and the WS upgrade. Missing/bad
key → friendly **403 HTML page** ("this page needs the full URL your coding
agent gave you, including `?key=…`" — generic "coding agent", not "Claude",
since this ships on Codex/Gemini/Copilot too). WS upgrade → destroy socket.
The query token is the source of truth; the cookie is a convenience that never
bears initial-auth load.
**Blast radius.** `server.cjs` (all logic). `helper.js` optional one-liner
(append `?key=` from `location.search` to the WS URL as a cookie-blocked
fallback). `start-server.sh` none. `visual-companion.md` doc note (URL now has
`?key=`; don't strip it). Tests updated to pass the token.
### A2 — Host allowlist dropped; browser WS Origin retained
Subsumed by A1. The secret key closes the WS-injection vector (#1014), the
HTTP/WS DNS-rebinding read vector (PR #1553), and the cross-origin WS vector
(PR #1110) in one mechanism, and unlike an allowlist it actually protects the
remote-bind case. No `BRAINSTORM_ALLOWED_HOSTS` and no Host allowlist. The final
implementation still checks browser WebSocket `Origin` after session auth so a
cross-origin localhost tab cannot ride the companion cookie.
### A3 — Server crashes on `null` / primitive WS payload
**Problem.** `handleMessage` (`server.cjs:233`) does `JSON.parse(text)` then
`if (event.choice)` at `server.cjs:243`. A client that sends the 4-byte text
frame `null` yields `event === null`, and `null.choice` throws. The throw is
**not** caught — `handleMessage` is called from the `socket.on('data')` handler
(`server.cjs:207`) outside the `try/catch`, which only wraps `decodeFrame`. The
result is an uncaught exception and process exit. Any local client can kill the
server.
**Change.** Guard the access: `if (event && event.choice)`. Minimal and exact —
`JSON.parse` can't produce `undefined`, and primitives return `undefined` for
`.choice` without throwing, so only `null` is the live hazard. (Avoid the
broader fixes — a top-level `try/catch` or `process.on('uncaughtException')`
would mask other bugs.)
### A4 — Frame-length bound in `decodeFrame` (adjacent)
Referenced by PR #1504 as #1446. The current code **already** bounds extended
frame lengths: `MAX_FRAME_PAYLOAD_BYTES = 10MB` (`server.cjs:10`) is enforced at
`server.cjs:58-67` before any `Buffer.alloc`. Action: verify #1446 against
current `dev` and close if already resolved, rather than re-implementing.
---
## B. Server robustness / correctness
### B1 — macOS resource-fork dotfiles served as screen content
**Problem.** The newest-screen selector filters on `f.endsWith('.html')` only
(`server.cjs:127-128`). On macOS/ExFAT, `._screen.html` resource-fork files pass
that filter and, being written alongside the real file, can sort newest — so the
browser gets binary metadata instead of the mockup. Four read sites share the
weak filter: `getNewestScreen` (`server.cjs:127`), `knownFiles` init
(`server.cjs:279`), the `fs.watch` handler (`server.cjs:286`), and the `/files/`
endpoint (`server.cjs:154-156`).
**Change.** Reject dotfiles (`!f.startsWith('.')`) at all four sites. Covers
`._*`, `.DS_Store`, etc.
### B2 — `stop-server.sh` can kill a reused PID
**Problem.** `stop-server.sh` reads the PID from `state/server.pid`
(`stop-server.sh:20`) and `kill`s it (`:23`, escalating to `-9` at `:35`)
without confirming the PID still belongs to our server. After a reboot or PID
wraparound the file can point at an unrelated process, which we'd then SIGKILL.
**Change.** Before signalling, verify ownership — the PID's command is `node`
running our `server.cjs`, ideally matching this session. If ownership can't be
proven, fail closed (report `stale_pid`, don't kill). Keep the existing
`stopped` / `not_running` outputs for the real cases.
### B3 — WebSocket client: silent reconnect, stale "Connected"
**Problem.** `helper.js` reconnects on a fixed 1s timer (`helper.js:21-23`),
has no `onerror` handler, never nulls `ws` on close, and never clears a pending
reconnect timer. The frame's status element is hardcoded to "Connected" with the
dot pinned to `var(--success)` (`frame-template.html:77,200`). When the laptop
sleeps or the server restarts, the page shows "Connected" over a dead socket and
queues events with no feedback.
**Change.**
- `helper.js`: exponential backoff (500ms → ×2 → cap 30s, reset on open);
`onerror` delegating to `onclose`; `ws = null` on close; `clearTimeout` before
reconnecting.
- `frame-template.html`: drive the status dot from a `--status-color` custom
property so JS can switch Connected (green) / Reconnecting (yellow) /
Disconnected (red).
---
## C. Lifecycle / timeout (issue #1237)
### C1 — Idle timeout too short, not configurable, WS keeps process alive
**Problem.** `IDLE_TIMEOUT_MS` is hardcoded to 30 minutes (`server.cjs:258`),
enforced by the 60s lifecycle check (`server.cjs:329-332`). A single brainstorm
question can sit longer than 30 min while the user thinks or steps away, so the
server dies mid-session. Separately, `shutdown()` (`server.cjs:310-321`) calls
`server.close()` but never closes the upgraded sockets in `clients`
(`server.cjs:174`), so an open browser connection can keep the Node process
alive past shutdown.
**Change.**
- Raise the default to 4 hours and make it configurable:
`--idle-timeout-minutes` in `start-server.sh` → an env var → `IDLE_TIMEOUT_MS`,
with validation against Node timer overflow.
- Expose the effective timeout in the startup JSON / `state/server-info`.
- In `shutdown()`, close every socket in `clients` so the process actually
exits.
### C2 — Server death is invisible
**Problem.** When the server exits it writes `state/server-stopped` and removes
`state/server-info` (`server.cjs:312-317`), and the skill is *told* to check
those files (`visual-companion.md:108`) — but it's soft guidance the model skips,
and the browser just shows a generic "can't be reached." The user diagnoses it
manually; the agent keeps referring to a dead URL.
**Change (two parts, independent of C1):**
- **Browser-facing tombstone.** Leave something at the last-served URL that says
"this companion expired — ask Claude to restart it" instead of a connection
error. Options to weigh: `helper.js` rendering a banner when the socket stays
down past backoff (works only while the page is loaded), vs. a more involved
approach that keeps a minimal responder alive to serve a tombstone page.
- **Harder skill check.** Tighten `visual-companion.md` / `SKILL.md` so
"check `server-info`/`server-stopped` before referring to the URL or pushing a
screen" is a required step, not a note. Keep it lightweight — possibly a
one-line helper the agent always runs.
---
## D. Features
### D1 — Permanent opt-out of the visual companion (issue #892)
**Problem.** The companion is offered as its own message every session
(`SKILL.md:25,151-152`). A user who never wants it pays that round-trip — and
HTML generation — every time. There's no way to say "never offer this."
**Change.** Before the offer step, the skill checks a user-level setting and
skips the offer entirely when opt-out is set.
**Design choice open.** Mechanism isn't settled:
- Env var (e.g. `SUPERPOWERS_VISUAL_COMPANION=off`) the skill is told to read —
simplest, matches what the issue asks for, lives in `.zshrc`.
- A plugin-settings file (`.claude/superpowers.local.md` frontmatter) — more
structured, per-project capable, but heavier and project-scoped.
- Reliability caveat from the issue: a separate "no-companion" skill competes on
trigger words and isn't reliable — rejected.
Pick the mechanism, then it's a small `SKILL.md` change plus a documented knob.
### D2 — Free-text feedback from the browser (issue #957)
**Problem.** The client only captures clicks on `[data-choice]`
(`helper.js:36-62`). A user who wants to annotate a mockup ("wrong shade of
blue") has to switch to the terminal, breaking the visual flow.
**Change.** Add a feedback `<textarea>` whose submit emits
`{"type":"feedback","text":...,"timestamp":...}` via the existing
`window.brainstorm.send` path (`helper.js:82-85`).
**Cross-cutting — server change required.** `handleMessage` only persists events
when `event.choice` is truthy (`server.cjs:243`). A `feedback` event has no
`choice`, so today it would be logged but **never written to `state/events`**,
and the agent wouldn't see it. The persistence condition must also accept
`feedback` events. Document the new event shape in `visual-companion.md`
(Browser Events Format, `:247-259`). Decide the submit trigger (button vs blur
vs both) and where the textarea renders (frame-level vs opt-in per screen).
### D3 — Auto-open the companion URL (PR #759, issue #755)
**Problem.** `start-server.sh` only prints the URL; the user opens it manually.
In WSL2 especially, people expect the browser to open.
**Change.** Best-effort opener after the `server-started` JSON is parsed:
Windows/WSL → `rundll32.exe url.dll,FileProtocolHandler <url>`, macOS → `open`,
Linux → `xdg-open` only when `DISPLAY`/`WAYLAND_DISPLAY` is set. Swallow
failures, never block startup, keep echoing the URL. Document in
`visual-companion.md`. (Consider an opt-out for headless/remote runs where
popping a browser is wrong — ties into D1's config mechanism.)
### D4 — Light/dark contrast helpers (PR #1683)
**Problem.** Content fragments are wrapped in the OS-aware frame
(`frame-template.html`). In dark mode, quick mockups often use white inline
backgrounds while inheriting low-contrast frame text, making cards/panels hard
to read.
**Change.** Add `.light-surface` / `.dark-surface` helper classes plus a
conservative fallback for common inline light backgrounds, and document them in
`visual-companion.md`'s CSS reference. Pure CSS in `frame-template.html`.
---
## E. Workshop / deferred / dropped
### E1 — Hard-gate terminal-vs-HTML per question (PR #1037) — WORKSHOP
The soft guidance already exists: "decide per-question," with browser-vs-terminal
tests in `SKILL.md:156-161` and `visual-companion.md:5-25`. The complaint is that
the model renders HTML for purely textual content (A/B lists, clarifying
questions), wasting tokens and a turn. PR #1037 wraps the decision in a
`<HARD-GATE>`. **Per Jesse, we'll workshop the wording/mechanism together**
this is behavior-shaping skill content and not specced here.
### E2 — Move session state out of the working tree (issue #975 / PR #977) — DEFERRED
Today `--project-dir` writes session state to `<project>/.superpowers/brainstorm/`
(`start-server.sh:80-84`) and the skill tells the user to gitignore it
(`visual-companion.md:58`). The ask is a `--state-dir` / `SUPERPOWERS_STATE_DIR`
default outside the repo (XDG), keeping `--project-dir` as an alias.
**Deferred by Jesse for now.** Captured so it isn't lost.
### E3 — Vendor Alpine.js for interactive mockups (PR #1639) — DROPPED
Adds a vendored Alpine build so mockups can be interactive (tabs, accordions,
forms) without hand-rolled JS. **Dropped per Jesse** — we are not taking on a
vendored third-party dependency in the companion runtime. The underlying need
(interactive mockups) is not being pursued via this route.
### E4 — Shell-lint warnings (PR #1677) — OPPORTUNISTIC
SC2034 (and friends) in `start-server.sh` / `stop-server.sh`. Trivial; fold into
B2/C1/D3 when we're already editing those scripts rather than as its own change.
---
## Suggested grouping for implementation
These cluster into a few coherent passes (each independently testable against
`tests/brainstorm-server/`):
1. **Security pass** (IN PROGRESS, branch `brainstorm-companion-session-key`) —
A1 per-session key (supersedes A2) + A3 null-crash guard. Verify/close A4.
*Highest priority.*
2. **Lifecycle pass** — C1 + C2 together (both touch `shutdown()` and the
server-death story).
3. **Robustness pass** — B1, B2, B3 (independent, small).
4. **Deferred feature pass** - D1, D2, D4 are not part of PR #1720. D3 is
shipped through the `--open` flow.
E1 is a separate workshop session. E2/E3 are out of scope for this round.
@@ -0,0 +1,785 @@
# Visual Companion Auth Hardening Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Harden the brainstorming visual companion auth and reconnect flow while preserving trusted same-origin screen JavaScript and future vendored UI libraries.
**Architecture:** Keyed root loads become a bootstrap step that sets the cookie, stores the key in tab-scoped `sessionStorage`, and navigates to a bare `/` screen URL. WebSockets require valid auth plus browser same-origin `Origin`, while `/files/*` uses realpath containment to prevent content-directory escapes.
**Tech Stack:** Node.js built-ins (`http`, `fs`, `path`, `crypto`), zero runtime dependencies, existing `ws` test dependency, Bash start/stop scripts, repo shell lint script.
**Important:** Do not commit during execution unless Drew explicitly asks. This repository's instructions override the generic plan template's commit cadence.
---
## File Map
- Modify: `skills/brainstorming/scripts/server.cjs`
- Add bootstrap response.
- Add shared security headers.
- Add WebSocket Origin validation.
- Add `/files/*` realpath containment.
- Modify: `skills/brainstorming/scripts/helper.js`
- Read the stored session key and append it to the WebSocket URL.
- Modify: `tests/brainstorm-server/auth.test.js`
- Add bootstrap, header, same-origin WS, cross-origin WS, and cookie/file auth regressions.
- Modify: `tests/brainstorm-server/helper.test.js`
- Add mocked-browser coverage for sessionStorage-backed WS URLs.
- Modify: `tests/brainstorm-server/server.test.js`
- Add symlink containment regression for `/files/*`.
- Modify: `tests/brainstorm-server/lifecycle.test.js`
- Make the start-server timeout flag test force background mode.
- Add restart reconnect credential coverage if it fits the existing lifecycle helper.
- Modify: `skills/brainstorming/scripts/start-server.sh`
- Fix shell lint.
- Modify: `skills/brainstorming/scripts/stop-server.sh`
- Fix shell lint.
- Modify: `.gitignore`
- Add `.superpowers/`.
- Optional docs update: `skills/brainstorming/visual-companion.md`
- Mention bootstrap URL stripping and trusted same-origin screen JS if the code behavior changes need operator-facing explanation.
## Task 1: Bootstrap Keyed Root Loads
**Files:**
- Modify: `tests/brainstorm-server/auth.test.js`
- Modify: `skills/brainstorming/scripts/server.cjs`
- [ ] **Step 1: Add RED tests for bootstrap behavior**
In `tests/brainstorm-server/auth.test.js`, add tests after the existing valid-key root test:
```js
await test('GET / with valid query returns bootstrap instead of screen content', async () => {
const res = await get('/', { key: TOKEN });
assert.strictEqual(res.status, 200);
assert(res.body.includes('sessionStorage'), 'bootstrap should store the session key in tab storage');
assert(res.body.includes('location.replace'), 'bootstrap should navigate to the bare root URL');
assert(!res.body.includes('Secret screen'), 'bootstrap must not serve screen HTML at the keyed URL');
});
await test('GET / with valid cookie serves the screen after bootstrap', async () => {
const res = await get('/', { cookie: `${COOKIE_NAME}=${TOKEN}` });
assert.strictEqual(res.status, 200);
assert(res.body.includes('Secret screen'), 'cookie-authenticated bare root should serve the screen');
assert(!res.body.includes('sessionStorage'), 'bare screen response should not be the bootstrap page');
});
```
Keep the existing cookie test if present; merge assertions rather than duplicating the same test name.
- [ ] **Step 2: Verify RED**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node auth.test.js
```
Expected: the new bootstrap test fails because current `GET /?key=...` serves `Secret screen` directly and does not include the bootstrap `sessionStorage`/`location.replace` code.
- [ ] **Step 3: Implement minimal bootstrap response**
In `skills/brainstorming/scripts/server.cjs`, add a helper near the page constants:
```js
function bootstrapPage(key) {
const jsonKey = JSON.stringify(String(key));
return `<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Opening Brainstorm Companion</title></head>
<body>
<script>
sessionStorage.setItem('brainstorm-session-key', ${jsonKey});
location.replace('/');
</script>
</body>
</html>`;
}
```
Then in `handleRequest`, after authorization and cookie setting but before serving screen HTML, detect a valid query key on root:
```js
function queryKey(url) {
const q = url.indexOf('?');
if (q < 0) return null;
return new URLSearchParams(url.slice(q + 1)).get('key');
}
```
Use it in `handleRequest`:
```js
const pathname = pathnameOf(req.url);
const keyFromQuery = queryKey(req.url);
if (req.method === 'GET' && pathname === '/' && keyFromQuery && timingSafeEqualStr(keyFromQuery, TOKEN)) {
res.writeHead(200, securityHeaders({ 'Content-Type': 'text/html; charset=utf-8' }));
res.end(bootstrapPage(keyFromQuery));
return;
}
```
This assumes Task 4 will introduce `securityHeaders`. If implementing Task 1 first, temporarily use:
```js
res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' });
```
and replace it in Task 4.
- [ ] **Step 4: Verify GREEN**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node auth.test.js
```
Expected: all auth tests pass, including the new bootstrap tests.
## Task 2: WebSocket Origin Enforcement
**Files:**
- Modify: `tests/brainstorm-server/auth.test.js`
- Modify: `skills/brainstorming/scripts/server.cjs`
- [ ] **Step 1: Add RED tests for same-origin and cross-origin WS**
In `tests/brainstorm-server/auth.test.js`, extend `wsConnect` to accept an `origin` option:
```js
function wsConnect({ key, cookie, origin } = {}) {
const url = `ws://localhost:${TEST_PORT}/` + (key !== undefined ? `?key=${key}` : '');
const headers = {};
if (cookie) headers['Cookie'] = cookie;
if (origin) headers['Origin'] = origin;
const ws = new WebSocket(url, Object.keys(headers).length ? { headers } : {});
return new Promise((resolve) => {
let settled = false;
const done = (outcome) => { if (!settled) { settled = true; resolve({ outcome, ws }); } };
ws.on('open', () => done('opened'));
ws.on('error', () => done('rejected'));
ws.on('close', () => done('rejected'));
setTimeout(() => done('rejected'), 1500);
});
}
```
Then add:
```js
await test('WS upgrade with valid cookie and same-origin Origin opens', async () => {
const { outcome, ws } = await wsConnect({
cookie: `${COOKIE_NAME}=${TOKEN}`,
origin: `http://localhost:${TEST_PORT}`
});
ws.close();
assert.strictEqual(outcome, 'opened');
});
await test('WS upgrade with valid cookie but cross-origin Origin is rejected', async () => {
const eventsFile = path.join(TEST_DIR, 'state', 'events');
if (fs.existsSync(eventsFile)) fs.unlinkSync(eventsFile);
const { outcome, ws } = await wsConnect({
cookie: `${COOKIE_NAME}=${TOKEN}`,
origin: 'http://localhost:9999'
});
if (outcome === 'opened') {
ws.send(JSON.stringify({ type: 'choice', choice: 'attacker-injected', text: 'local attacker probe' }));
await sleep(300);
}
ws.close();
assert.strictEqual(outcome, 'rejected', 'cross-origin browser WS must not open even with cookie');
assert(!fs.existsSync(eventsFile), 'cross-origin WS must not write state/events');
});
```
- [ ] **Step 2: Verify RED**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node auth.test.js
```
Expected: cross-origin cookie WS test fails because current server accepts any cookie-authenticated WS regardless of Origin.
- [ ] **Step 3: Implement Origin check**
In `skills/brainstorming/scripts/server.cjs`, add:
```js
function isAllowedWebSocketOrigin(req) {
const origin = req.headers.origin;
if (!origin) return true; // non-browser clients still need the session key
const host = req.headers.host;
if (!host) return false;
return origin === 'http://' + host;
}
```
Then update `handleUpgrade`:
```js
function handleUpgrade(req, socket) {
if (!isAuthorized(req) || !isAllowedWebSocketOrigin(req)) { socket.destroy(); return; }
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node auth.test.js
```
Expected: auth tests pass; cross-origin WS is rejected; same-origin and direct key WS still open.
## Task 3: Helper Uses Stored Key For Reconnect
**Files:**
- Modify: `tests/brainstorm-server/helper.test.js`
- Modify: `skills/brainstorming/scripts/helper.js`
- [ ] **Step 1: Add RED test for WebSocket URL key**
In `tests/brainstorm-server/helper.test.js`, add a mocked-browser test near the reconnect state-machine tests:
```js
test('uses sessionStorage key in the WebSocket URL when present', () => {
const e = makeEnv();
e.state.sessionKey = 'stored-key-abc';
e.boot();
assert.strictEqual(e.sockets[0].url, 'ws://localhost:7777/?key=stored-key-abc');
});
```
Update `makeEnv()` so the returned object exposes `sockets`, and the mock window includes sessionStorage:
```js
window: {
location: { host: 'localhost:7777', reload() { state.reloads++; } },
sessionStorage: { getItem: (key) => key === 'brainstorm-session-key' ? state.sessionKey : null }
},
```
Also add a fallback test:
```js
test('uses cookie-only WebSocket URL when no sessionStorage key is present', () => {
const e = makeEnv();
e.state.sessionKey = null;
e.boot();
assert.strictEqual(e.sockets[0].url, 'ws://localhost:7777');
});
```
- [ ] **Step 2: Verify RED**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node helper.test.js
```
Expected: stored-key test fails because current helper uses `ws://localhost:7777`.
- [ ] **Step 3: Implement stored-key WS URL**
In `skills/brainstorming/scripts/helper.js`, replace:
```js
const WS_URL = 'ws://' + window.location.host;
```
with:
```js
function websocketUrl() {
let key = null;
try { key = window.sessionStorage && window.sessionStorage.getItem('brainstorm-session-key'); } catch (e) {}
return 'ws://' + window.location.host + (key ? '/?key=' + encodeURIComponent(key) : '');
}
```
Then replace:
```js
ws = new WebSocket(WS_URL);
```
with:
```js
ws = new WebSocket(websocketUrl());
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node helper.test.js
```
Expected: helper tests pass.
## Task 4: Security Headers
**Files:**
- Modify: `tests/brainstorm-server/auth.test.js`
- Modify: `skills/brainstorming/scripts/server.cjs`
- [ ] **Step 1: Add RED header tests**
In `tests/brainstorm-server/auth.test.js`, add:
```js
await test('HTML responses include leak-reduction and anti-framing headers', async () => {
const res = await get('/', { key: TOKEN });
assert.strictEqual(res.headers['referrer-policy'], 'no-referrer');
assert.strictEqual(res.headers['cache-control'], 'no-store');
assert.strictEqual(res.headers['x-frame-options'], 'DENY');
assert.strictEqual(res.headers['content-security-policy'], "frame-ancestors 'none'");
assert.strictEqual(res.headers['cross-origin-resource-policy'], 'same-origin');
});
await test('403 responses include leak-reduction and anti-framing headers', async () => {
const res = await get('/');
assert.strictEqual(res.status, 403);
assert.strictEqual(res.headers['referrer-policy'], 'no-referrer');
assert.strictEqual(res.headers['cache-control'], 'no-store');
assert.strictEqual(res.headers['x-frame-options'], 'DENY');
assert.strictEqual(res.headers['content-security-policy'], "frame-ancestors 'none'");
assert.strictEqual(res.headers['cross-origin-resource-policy'], 'same-origin');
});
```
- [ ] **Step 2: Verify RED**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node auth.test.js
```
Expected: header tests fail because current responses do not include these headers.
- [ ] **Step 3: Implement shared header helper**
In `skills/brainstorming/scripts/server.cjs`, add:
```js
function securityHeaders(headers = {}) {
return {
'Referrer-Policy': 'no-referrer',
'Cache-Control': 'no-store',
'X-Frame-Options': 'DENY',
'Content-Security-Policy': "frame-ancestors 'none'",
'Cross-Origin-Resource-Policy': 'same-origin',
...headers
};
}
```
Update response writes in `handleRequest`:
```js
res.writeHead(403, securityHeaders({ 'Content-Type': 'text/html; charset=utf-8' }));
```
```js
res.writeHead(200, securityHeaders({ 'Content-Type': 'text/html; charset=utf-8' }));
```
```js
res.writeHead(200, securityHeaders({ 'Content-Type': contentType }));
```
For 404s:
```js
res.writeHead(404, securityHeaders());
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node auth.test.js
```
Expected: auth tests pass and header assertions are green.
## Task 5: `/files/*` Realpath Containment
**Files:**
- Modify: `tests/brainstorm-server/server.test.js`
- Modify: `skills/brainstorming/scripts/server.cjs`
- [ ] **Step 1: Add RED symlink escape test**
In `tests/brainstorm-server/server.test.js`, after the `/files/` empty-name test, add:
```js
await test('does not serve symlinks that escape content dir via /files/', async () => {
const target = path.join(STATE_DIR, 'server-info');
const link = path.join(CONTENT_DIR, 'linked-server-info.txt');
try { fs.unlinkSync(link); } catch (e) {}
fs.symlinkSync(target, link);
const res = await fetch(`http://localhost:${TEST_PORT}/files/linked-server-info.txt`);
assert.strictEqual(res.status, 404, 'symlink to state/server-info must not be served');
assert(!res.body.includes('server-started'), 'response must not include server-info body');
});
```
- [ ] **Step 2: Verify RED**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node server.test.js
```
Expected: symlink test fails because current `/files/*` follows symlinks and serves `server-info`.
- [ ] **Step 3: Implement containment helper**
In `skills/brainstorming/scripts/server.cjs`, add:
```js
function isRegularFileInsideContentDir(filePath) {
let stat, realContentDir, realFilePath;
try {
stat = fs.lstatSync(filePath);
if (stat.isSymbolicLink()) return false;
if (!stat.isFile()) return false;
realContentDir = fs.realpathSync(CONTENT_DIR);
realFilePath = fs.realpathSync(filePath);
} catch (e) {
return false;
}
return realFilePath.startsWith(realContentDir + path.sep);
}
```
Replace the `/files/*` guard with:
```js
if (!fileName || fileName.startsWith('.') || !isRegularFileInsideContentDir(filePath)) {
res.writeHead(404, securityHeaders());
res.end('Not found');
return;
}
```
- [ ] **Step 4: Verify GREEN**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node server.test.js
```
Expected: server tests pass, including symlink rejection.
## Task 6: Restart Reconnect Regression
**Files:**
- Modify: `tests/brainstorm-server/lifecycle.test.js`
- Modify: `skills/brainstorming/scripts/server.cjs`
- Modify: `skills/brainstorming/scripts/helper.js`
- [ ] **Step 1: Add RED integration test for same key over WS after restart**
In `tests/brainstorm-server/lifecycle.test.js`, add a test after the port/token persistence test:
```js
await test('stored key can authenticate WebSocket after same-port restart', async () => {
const dir = fs.mkdtempSync('/tmp/bs-reconnect-');
const portFile = path.join(dir, '.last-port');
const tokenFile = path.join(dir, '.last-token');
const env = { ...process.env, BRAINSTORM_PORT_FILE: portFile, BRAINSTORM_TOKEN_FILE: tokenFile, BRAINSTORM_LIFECYCLE_CHECK_MS: 100000 };
const a = spawn('node', [SERVER], { env: { ...env, BRAINSTORM_DIR: path.join(dir, 's1') } });
let outA = ''; a.stdout.on('data', d => outA += d.toString());
for (let i = 0; i < 60 && !outA.includes('server-started'); i++) await sleep(50);
const infoA = firstServerStarted(outA);
const keyA = new URL(infoA.url).searchParams.get('key');
a.kill(); await sleep(400);
const b = spawn('node', [SERVER], { env: { ...env, BRAINSTORM_DIR: path.join(dir, 's2') } });
let outB = ''; b.stdout.on('data', d => outB += d.toString());
for (let i = 0; i < 60 && !outB.includes('server-started'); i++) await sleep(50);
const infoB = firstServerStarted(outB);
const ws = new WebSocket(`ws://localhost:${infoB.port}/?key=${keyA}`, {
headers: { Origin: `http://localhost:${infoB.port}` }
});
const opened = await new Promise(resolve => {
ws.on('open', () => resolve(true));
ws.on('error', () => resolve(false));
setTimeout(() => resolve(false), 1500);
});
try {
assert.strictEqual(infoB.port, infoA.port, 'restart should reuse same port');
assert(opened, 'stored key should authenticate WS after restart');
} finally {
try { ws.close(); } catch (e) {}
b.kill(); await sleep(100);
fs.rmSync(dir, { recursive: true, force: true });
}
});
```
This test may already pass once Tasks 2 and 3 are implemented. If it passes before code changes, keep it as coverage but do not call it RED. The real browser reconnect behavior is primarily covered by Task 3 plus final manual/headless browser verification.
- [ ] **Step 2: Verify behavior**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node lifecycle.test.js
```
Expected after Tasks 2 and 3: lifecycle tests pass. If this fails, fix the auth/restart path before continuing.
## Task 7: Lifecycle Hang And Shell Lint
**Files:**
- Modify: `tests/brainstorm-server/lifecycle.test.js`
- Modify: `skills/brainstorming/scripts/start-server.sh`
- Modify: `skills/brainstorming/scripts/stop-server.sh`
- [ ] **Step 1: Reproduce shell lint failure**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers
scripts/lint-shell.sh skills/brainstorming/scripts/start-server.sh skills/brainstorming/scripts/stop-server.sh tests/brainstorm-server/stop-server.test.sh
```
Expected current failure:
```text
SC2164: skills/brainstorming/scripts/start-server.sh line 128: cd "$SCRIPT_DIR"
SC2034: skills/brainstorming/scripts/start-server.sh line 166: for i in {1..50}
SC2034: skills/brainstorming/scripts/stop-server.sh line 57: for i in {1..20}
```
- [ ] **Step 2: Fix shell lint minimally**
In `skills/brainstorming/scripts/start-server.sh`, change:
```bash
cd "$SCRIPT_DIR"
```
to:
```bash
cd "$SCRIPT_DIR" || exit 1
```
Change unused loop variables from `i` to `_` where they are not read:
```bash
for _ in {1..50}; do
```
In `skills/brainstorming/scripts/stop-server.sh`, change:
```bash
for i in {1..20}; do
```
to:
```bash
for _ in {1..20}; do
```
- [ ] **Step 3: Fix lifecycle start-server hang**
In `tests/brainstorm-server/lifecycle.test.js`, update the `start-server.sh --idle-timeout-minutes sets the timeout` test command:
```js
const out = execFileSync('bash', [START, '--project-dir', dir, '--idle-timeout-minutes', '5', '--background'], { encoding: 'utf8' });
```
This keeps the test from hanging when `CODEX_CI` triggers start-server foreground mode.
- [ ] **Step 4: Verify lint and lifecycle**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers
scripts/lint-shell.sh skills/brainstorming/scripts/start-server.sh skills/brainstorming/scripts/stop-server.sh tests/brainstorm-server/stop-server.test.sh
cd tests/brainstorm-server
node lifecycle.test.js
```
Expected: shell lint exits 0; lifecycle tests exit 0 without hanging.
## Task 8: Gitignore Durable Companion State
**Files:**
- Modify: `.gitignore`
- [ ] **Step 1: Verify current ignore gap**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers
git check-ignore .superpowers/brainstorm/.last-token || true
```
Expected current output: no matching ignore rule.
- [ ] **Step 2: Add ignore rule**
Add this line to `.gitignore`:
```gitignore
.superpowers/
```
- [ ] **Step 3: Verify GREEN**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers
git check-ignore .superpowers/brainstorm/.last-token
```
Expected output:
```text
.superpowers/brainstorm/.last-token
```
## Task 9: Full Automated Verification
**Files:**
- No code changes in this task.
- [ ] **Step 1: Run focused suites**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
node auth.test.js
node helper.test.js
node server.test.js
node lifecycle.test.js
```
Expected: all four commands exit 0.
- [ ] **Step 2: Run full brainstorm-server suite**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
npm test
```
Expected: all tests pass, including ws-protocol, helper, auth, server, lifecycle, and stop-server.
- [ ] **Step 3: Repeat suite for lifecycle/watch flake**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers/tests/brainstorm-server
for i in 1 2 3; do npm test || exit 1; done
```
Expected: all three repeats pass without hanging.
- [ ] **Step 4: Run shell lint**
Run:
```bash
cd /Users/drewritter/prime-rad/superpowers
scripts/lint-shell.sh skills/brainstorming/scripts/start-server.sh skills/brainstorming/scripts/stop-server.sh tests/brainstorm-server/stop-server.test.sh
```
Expected: exits 0.
## Task 10: Re-run Security Probes
**Files:**
- No code changes in this task.
- [ ] **Step 1: Recreate the cross-origin attacker probe**
Use the previous scratch probe if available:
```bash
node /tmp/superpowers-pr1720-security-drewritter/probe-pr1720.cjs
```
If the scratch probe is unavailable, recreate a minimal probe under `/tmp` that:
- starts the companion with a fixed token
- loads the keyed URL in headless Chrome
- starts an attacker page on a different localhost port
- attempts `new WebSocket('ws://localhost:<companion-port>/')`
- sends `{"type":"choice","choice":"attacker-injected"}`
- checks `state/events`
Expected after fixes:
- keyless and wrong-key HTTP still return 403
- same-origin helper reaches Connected
- cross-origin WebSocket does not open
- `state/events` does not contain `attacker-injected`
- symlink-to-`server-info` returns 404
- keyed browser load ends on bare `/`
- [ ] **Step 2: Re-run manual/browser flow only after automated probes pass**
Manual flow:
1. start the companion with `--project-dir --open`
2. push a screen
3. confirm URL strips to `/`
4. confirm status reaches Connected
5. click a choice and verify `state/events`
6. stop and restart same project
7. verify the open tab reconnects automatically
Expected: all steps pass without manual URL reload.
## Self-Review Checklist
- Spec coverage: every design requirement maps to at least one task.
- Placeholder scan: this plan contains no unresolved placeholder markers or unspecified edge-case steps.
- TDD order: every production change task starts with a focused failing test or a command that demonstrates the current failure.
- Trust model: the plan preserves trusted same-origin screen JavaScript and future same-origin vendored libraries.
- No-commit rule: execution does not commit unless Drew explicitly asks.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,160 @@
# SDD Task-Scoped Review Dispatch
Make subagent-driven-development's per-task reviews cheaper and faster without weakening them, by scoping per-task review prompts to the task and stopping redundant work — while final branch review stays broad.
## Problem
Per-task code quality reviewers in SDD routinely do branch-review-scale work on single-task diffs. Evidence from two real local SDD sessions: `a1a6719a-6109-453a-9933-34ae396f5bae` (sen-core-v2) and `0cc1a12d-9984-4c35-8615-9d42dadb2c47` (serf), both under `~/.claude/projects/`:
- In the sen-core-v2 session, 7/8 quality reviewers ran repo-wide greps; the most expensive ran 50+ Bash commands over ~200 seconds. Across both sessions, quality reviewers cost 4-8× what spec reviewers cost on the same tasks.
- Spec reviewers, whose prompt contains "Only read files in this diff. Do not crawl the broader codebase," stayed tight: 6-16 tool calls, 14-65 seconds.
- No reviewer ran heavy tests autonomously. Every package-wide or repeated test run observed was explicitly requested by a controller-written prompt ("check all uses," "run tests if useful, especially race-focused ones," "does anything else read `Meta()`?").
Root causes, in order of impact:
1. **The per-task quality prompt inherits a merge-readiness review.** `code-quality-reviewer-prompt.md` delegates to `requesting-code-review/code-reviewer.md`, which asks about architecture, scalability, security, production readiness, and ends with "Ready to merge?" That frame licenses branch-level breadth on a one-task diff. The spec prompt's diff-scope guard was never carried over.
2. **The controller gets no guidance on writing reviewer prompts**, so it invents open-ended directives ("check all uses") that reviewers interpret literally.
3. **Duplicated work across the pipeline.** The quality template's "Plan alignment" dimension re-checks what the spec reviewer just verified. Reviewers re-run test suites the implementer already ran (and reported, with TDD evidence) on identical code.
4. **Per-task and final review share one template**, so there is no representation of "per-task narrow, final broad" anywhere.
A field report (`~/2026-06-09-code-quality-reviewer-scope-budget-issue.md`) first flagged this. Its cited session and headline numbers could not be verified, but its qualitative diagnosis was confirmed against two real local sessions. One correction to it: cross-cutting audits (lock ordering, changed contracts) are sometimes the *correct* review method — the fix must gate breadth behind a stated concrete risk, not forbid it.
## Goals
- Per-task reviews scoped to the task: diff-first reading, justified broadening, no redundant test runs.
- Final whole-branch review keeps its current breadth.
- No reduction in what reviews catch.
## Non-goals / explicitly preserved
- **Full re-reviews stay.** When a reviewer re-reviews after a fix, it still reviews the whole task at full reading breadth. (It does not re-run tests the implementer just ran on the amended code.) This deliberately rejects the field report's "re-review budget" remedy: the cost of its worst cited example (a re-review running `-race` and `-count=100` loops) is curbed by the test budget below, not by narrowing what re-reviewers read.
- ~~**The two review stages stay separate.** Spec compliance and code quality remain independent subagents, serially gated. No merging.~~ **Superseded by the cost iterations below**: live eval economics showed per-dispatch overhead dominating cost, and the maintainer put everything on the table. The per-task stages are now one task reviewer with two verdicts; the independent broad final review remains.
- **The coordinator keeps model judgment.** No forced model tier for reviews, in either direction.
- **`requesting-code-review/` is untouched.** It remains the broad template for final branch review and ad-hoc review.
- Verdict ordering (spec compliance reported before quality), the fix-and-re-review loops, and the requirement to fix Critical/Important findings are unchanged.
## Cost iterations (post-launch eval economics)
Live before/after runs surfaced a cost regression once the quality-hardening
prose (evidence rule, constraint carrying, pristine output) landed: go-fractals
went from 42.8 min / 14.5M tokens (first task-scoped version) to 69.9 min /
32.2M (hardened version) while reaching baseline-parity quality (blind-judged
8.5 vs 8.5). Per-subagent turn profiling attributed cost to, in order: cheap
models taking 2-3× the turns on multi-step work (678 of 1197 subagent turns
were haiku), per-dispatch overhead (3 subagent spin-ups per task, each
re-deriving the diff; controller coordination was half the dollars), and
evidence-rule narration.
- **Iteration 1:** turn-count-beats-token-price model guidance (mid-tier floor
for multi-step work), optional inline diffs, cite-don't-narrate evidence,
Important = cannot-trust-until-fixed, fixes dispatched only for
Critical/Important. Result: 68.2 min / 22.9M — tokens down 29%, wall-clock
flat; controllers pasted the diff in only 2 of 22 review dispatches when
phrasing was optional.
- **Iteration 2:** per-task spec and quality reviews merged into one
`task-reviewer-prompt.md` (one reviewer, one reading of the diff, two
verdicts; one fix dispatch addresses both kinds of findings); implementers
run the focused test while iterating, full suite once before commit.
Result (go-fractals): 47.5 min / 15.7M / $13.55 — beat baseline on every
axis, blind-judged 9/10 vs baseline 7/10.
- **Iteration 3:** Calibration names merge-blocking maintainability damage
(verbatim duplication, swallowed errors, assertion-free tests) as
Important and Minor findings must be pasted into the final review for
triage; reviewer skepticism extended to the implementer's design
rationales ("left it per YAGNI" is a claim, not a verdict); diff handed
to reviewers as a file (`git diff > /tmp/sdd-task-N.diff`, redirected so
it never enters the controller's context; one Read call for the
reviewer) after paste-into-prompt guidance went unadopted (0-6 of 11-17
dispatches) for locally-rational context-economics reasons.
- **Final frozen config (e355795), all five scenarios pass:** go-fractals
44.4 min / 13.4M / $11.67 (-32% time, -37% tokens, -27% dollars vs
baseline); svelte-todo 62.8 / 19.7M / $15.76 (-21% / -28% / -25%);
rejects-extra-features $1.31 (vs $1.88); spec-reviewer-flaws flat; the
planted-defect scenario (v3: open-flag transparency bar for judgment
calls, must-fix bar for a test whose name promises verification it
never performs) passes with the defect caught and fixed.
### Iterations 4-5 (2026-06-10): variance honesty, structural fixes, positive recipes
A same-config re-run exposed run-to-run variance (44.4→57.1 min on
identical prompts; reviewer escape-hatch appetite swung 1.0→6.3 tool
calls/review), so all subsequent claims use ranges. Five parallel
experiment variants on go-fractals plus transcript mining of real local
sessions (full log with negative results:
`evals/docs/experiments/2026-06-10-sdd-cost-experiments.md`) produced the
final config:
- **Adopted:** final-review package (final reviewer 33→6 turns at
controller-model prices); REQUIRED `model:` line in both templates
(prose guidance decayed mid-session once, inheriting opus for 17
dispatches, +$5); task-brief + report files (`scripts/task-brief`;
fidelity anchor, modest context savings); progress ledger in
`<git-dir>/sdd/progress.md` (real sessions re-dispatched entire
completed task sequences after compaction — 269 dispatches for ~22
tasks); omnibus final fixer (a real session's per-finding fix wave cost
more than all its tasks); scoped fix tests; unique SHA-range collateral
names (worktree/submodule-safe); dispatch-composition recipe and
reviewer named-risk budget (micro-tested: positive recipe 3.0
transcribed values vs prohibition 4.4 vs control 3.6 — prohibitions can
backfire; see `2026-06-10-positive-instruction-redesign-design.md`).
- **Tested and declined:** controller turn batching and parallel-call
pipelining (controller emits exactly one tool call per message — 0
multi-tool messages in every run; 46% of its turns are
thinking/narration, a prompt-immune floor); background-dispatch
pipelining (mechanism adopted 7/28 but benefit below the ±6 min noise
floor on these scenarios).
- **Final validated config (b81f35b family), all gates pass:** go-fractals
54.1-54.7 min / 14.4-16.6M / $12.81-14.31 (baseline 64.9 / 21.2M /
$16.07); svelte-todo 55.0 min / 19.3M / $14.99 (baseline 79.7 / 27.3M /
$20.98); planted-defect pass / $2.77. Across all 8 same-design fractals
runs: 44.4-57.1 min / 13.4-20.0M / $11.67-14.84 — the worst draw beats
baseline on every axis; typical mid-band savings ~20-25%.
## Design
### Shared principle: don't re-run tests on code that hasn't changed
The implementer's report includes test results and TDD RED/GREEN evidence for exactly the code under review. Reviewers verify by reading. A reviewer runs a test only when reading raises a specific doubt that no existing run answers — and then a focused test, not a suite. On harnesses where reviewer subagents are read-only (e.g., Antigravity maps reviewer templates to the `research` type, which has no command access), the reviewer instead names the test it would run in its report.
After a fix, the implementer re-runs the tests covering the amended code; the re-reviewer does not repeat that run. Today nothing enforces that premise: `implementer-prompt.md` describes the initial implement-test-commit flow only, with no fix-iteration instruction. This spec therefore also adds to `implementer-prompt.md`: after fixing a review finding, re-run the tests that cover the amended code and include the results in the fix report.
This principle appears in both reviewer prompts, the implementer prompt, and the controller guidance.
### 1. New file: `skills/subagent-driven-development/code-quality-reviewer-prompt.md` becomes self-contained
Stop delegating to `requesting-code-review/code-reviewer.md`. The per-task quality reviewer gets its own scoped prompt template:
- **Framing:** "You are reviewing one task's implementation for code quality." A task-scoped gate, not a merge review.
- **Spec compliance is settled:** spec review already passed; do not re-litigate requirements or plan alignment.
- **Review dimensions kept:** code quality (clarity, duplication, error handling), test quality (real behavior, not mocks), maintainability, and the existing SDD-specific checks (single responsibility, independent testability, file structure from plan, file growth contributed by this change). Dropped: plan alignment, security/scalability/production-readiness dimensions, merge verdict.
- **Scope budget:** start from `git diff BASE..HEAD`; read changed files first; inspect adjacent code only to evaluate a concrete risk you can name. Cross-cutting changes — lock ordering, changed function/API contracts, shared mutable state — are legitimate named risks that justify checking call sites. Do not crawl the codebase by default.
- **Test budget:** the shared principle above, plus: no package-wide suites, race detectors, or repeated/high-count runs unless you have first named a specific suspected flake or race. Otherwise, recommend heavy validation in the report instead of running it. Warnings or noise in the implementer's reported test output are findings — output should be pristine (the implementer's self-review checks this too).
- **Evidence rule:** reviewers answer each What-to-Check item with file:line evidence, not bare yes/no. (Added after live eval runs showed reviewers passing defects the prompt had pointed them at — an accessible-name check and a temp-dir-cleanup check both got unsupported "yes" answers while the defect sat in the reviewed diff.)
- **Read-only rule** kept in trimmed form: no mutating the working tree, index, HEAD, or branch state. The `git worktree add` how-to sentence from the current templates is NOT carried into this file — a diff-scoped review never needs a checkout of another revision (same rationale as the spec-prompt cleanup below).
- **Verdict:** Strengths / Issues (Critical/Important/Minor) / "Task quality: Approved | Needs fixes."
### 2. `skills/subagent-driven-development/spec-reviewer-prompt.md` cleanups
- Remove the `git worktree add` how-to sentence. The read-only rule stays; a diff-scoped spec review never needs a checkout of another revision.
- Resolve the tension between the diff-only guard and "verify everything independently": spec compliance is judged by reading the diff against the requirements. The implementer's TDD evidence covers "it runs" — apply the shared test principle.
- New third verdict channel: requirements that cannot be verified from the diff (live in unchanged code, span tasks) are reported as explicit "⚠️ Cannot verify from diff — controller should check X" items, instead of either crawling or silently passing. The flowchart's binary pass/fail diamond cannot route this, so the controller guidance (§3) defines the handling: ⚠️ items do not block dispatching the quality reviewer, but the controller must resolve each one itself (it holds the plan and cross-task context) before marking the task complete; an item the controller confirms is a real gap is treated as a failed spec review and goes back to the implementer.
- Replace the fabricated premise "The implementer finished suspiciously quickly" with grounded skepticism: treat the implementer's report as unverified claims about the code. Same distrust, no invented fact.
### 3. `skills/subagent-driven-development/SKILL.md` controller changes
- **Model Selection:** replace "Architecture, design, and review tasks: use the most capable available model" with judgment guidance — pick reviewer models the way implementer models are picked, scaled to the diff's size, complexity, and risk. The "Task complexity signals" list is rescoped to make clear its bullets describe implementation tasks; reviewer model choice follows the same judgment, so a narrow diff review does not automatically map to "broad codebase understanding → most capable model."
- **Reviewer prompt construction** (new guidance near Red Flags): when dispatching reviewers, do not write open-ended directives ("check all uses," "run race tests if useful") without a concrete task-specific reason; do not ask reviewers to re-run tests the implementer already ran on the same code; do not pre-judge findings for the reviewer (never instruct a reviewer to ignore or not flag a specific issue — adjudicate suspected false positives in the review loop instead); per-task reviews are task-scoped gates — the broad review happens once, at the final whole-branch review. (The pre-judging rule was added after a live eval run caught the controller fabricating a "the plan forbids a shared helper" claim and instructing the quality reviewer not to flag a planted DRY violation.) Controllers must also include the spec/design's global constraints that bind the task — version floors, naming and copy rules, platform requirements — in the requirements they paste: a live run shipped a `go 1.26.1` module floor against a "Go 1.21+" design because no reviewer ever saw the constraint. And controllers must specify a model explicitly on every dispatch — an omitted model inherits the session's (usually most expensive) model, which silently defeats model selection.
- **Handling spec-reviewer ⚠️ items** (new guidance, alongside Handling Implementer Status): the controller resolves each "cannot verify from diff" item itself before marking the task complete; confirmed gaps go back to the implementer as failed spec review.
- **Final review stays broad, explicitly:** the final whole-branch reviewer dispatch node gains an explicit pointer to `../requesting-code-review/code-reviewer.md`. (Today that template is reachable only through the per-task quality prompt's delegation; once that delegation is removed, an unreferenced final-review template would be orphaned.) The Integration section's note that `superpowers:requesting-code-review` provides "the code review template for reviewer subagents" is corrected to apply to the final review only.
- **Example workflow:** the quality-reviewer lines in the example are updated to the new verdict vocabulary ("Task quality: Approved"); the final reviewer's "ready to merge" line stays.
- Flowchart topology is unchanged; the ⚠️ channel is handled by controller guidance, not a new graph branch.
## What this does not fix (known, deferred)
The spec reviewer judges against task text the controller pasted; it cannot catch requirements dropped during the controller's extraction from the plan. That is an architectural property of "controller provides full text," not a prompt problem, and is out of scope here.
## Verification
- Plugin infrastructure tests (`tests/`) still pass.
- Run the SDD skill-behavior evals (`git submodule update --init evals`, then per `evals/README.md`) before and after the change. Specifically: `sdd-go-fractals`, `sdd-svelte-todo`, `sdd-rejects-extra-features` (end-to-end SDD including the spec reviewer's YAGNI gate), and `spec-reviewer-catches-planted-flaws`.
- Known eval gaps this change exposes: no existing scenario plants a code-quality defect inside a single SDD task and asserts the per-task quality reviewer catches it, and no scenario measures per-reviewer exploration cost (tool-call/grep counts). Add one scenario covering the first gap (planted single-task quality defect → per-task reviewer must flag it before final review). For exploration cost, compare reviewer subagent tool-call counts manually across the before/after eval transcripts.
@@ -0,0 +1,178 @@
# Positive-Instruction Redesign of Skill Guidance — Design Spec
**Status:** Proposed (follow-up to the 2026-06-09 SDD review-dispatch work; separate PR per the one-problem-per-PR rule)
**Driver:** Measured evidence (2026-06-10) that some negative instructions in skill prose backfire, while others work — and that the difference is predictable.
## The measured finding this spec generalizes
Micro-tests on 2026-06-10 (opus, 5 reps per phrasing, programmatic scoring;
harness described below) measured how guidance phrasing changes what a
controller composes:
| Case | Phrasing | Result |
|---|---|---|
| Dispatch composition ("don't restate the brief") | prohibition | **4.4** spec values re-typed — *worse than no guidance* (3.6) |
| Dispatch composition | positive recipe ("your dispatch should contain: (1)…(5)") | **3.0, zero variance** — adopted |
| Dispatch composition | recipe + nuance clause ("quote only the fragment…") | 3.8, noisy — nuance dilutes recipes |
| Test-rerun directive ("do not ask reviewer to re-run tests") | prohibition | **0/5 violations** — works fine (control: 3/5) |
| Test-rerun directive | positive recipe | 0/5 — equal, but longer |
**The doctrine** (use this to classify any negative instruction):
1. **Tripwires work.** Phrase-level self-checks on concrete tokens ("if the
prompt you are writing contains 'do not flag' … stop") fire reliably.
2. **Recognition tables work.** Red-Flags/rationalization tables read at
decision time, not composition time.
3. **Discrete-directive prohibitions work.** "Do not ask X to do Y" holds
when the model has no competing incentive to do Y.
4. **Composition prohibitions backfire** when the model has its own agenda
for the output (e.g., restating specs feels like helpful curation).
Only a positive composition recipe moves these — and adding nuance
clauses to a winning recipe makes it worse, not better.
5. **Ties go to the shorter phrasing.** Codex re-reads SKILL.md ~500× per
long session (measured 2026-06-10); prose length is a real cost.
## Audit results (2026-06-10, all ~30 skills + prompt templates)
Counts: 3 tripwires (keep), 14 recognition tables (keep), ~20 policy gates
(keep — "never push without permission" is policy, not composition
shaping), 5 composition-prohibitions:
| # | Location | Disposition |
|---|---|---|
| 1 | `subagent-driven-development/task-reviewer-prompt.md` — "Cite, don't narrate" | **Queued in PR #1717 batch**: lead with the positive half ("Your report should point at evidence: file:line for every finding…"), drop the prohibition half (dead weight — the positive half already exists and carries the load) |
| 2 | `subagent-driven-development/SKILL.md` — "Do not add open-ended directives" | **Keep as-is**: micro-test could not elicit the failure in 15 samples; no evidence either way; shorter wins |
| 3 | `subagent-driven-development/SKILL.md` — "Do not ask a reviewer to re-run tests" | **Keep as-is**: measured 0/5 violations; the prohibition also usefully propagates itself into dispatches |
| 4 | `subagent-driven-development/SKILL.md` — "do not re-review on top of it" | **Queued in PR #1717 batch**: replace with the three-element checklist ("Before re-dispatching the reviewer, confirm the fix report contains: the covering tests, the command run, and the output") |
| 5 | `writing-plans/SKILL.md` — the "No Placeholders" banned-patterns list | **This spec's main subject** — see below |
Borderline, deferred with #5: `task-reviewer-prompt.md` "Don't flag
pre-existing file sizes — focus on what this change contributed" (positive
half present and load-bearing; low impact; test alongside #5 if convenient).
## The writing-plans change (deferred item #5)
### Current state
`skills/writing-plans/SKILL.md`, "No Placeholders": one positive sentence
("Every step must contain the actual content an engineer needs") followed
by a six-bullet banned-patterns list ("never write them: 'TBD', 'TODO',
'Add appropriate error handling', 'Write tests for the above', 'Similar to
Task N', …").
### Why it matters and why it is genuinely uncertain
- Plans are the **largest generated artifact** in the workflow, and the
model has a real competing incentive to emit placeholders (they are the
path of least effort under length pressure) — the incentive structure of
the case where prohibition measurably backfired.
- But the banned items are **discrete, recognizable tokens** — the shape
of the case where prohibition measurably held.
- **The list is load-bearing elsewhere:** the skill's Self-Review section
references it ("Placeholder scan: search your plan for red flags — any
of the patterns from the 'No Placeholders' section above"). The tokens
double as the review-time scan inventory, and review-time recognition is
the category that works. A naive swap to a positive checklist breaks
that reference and discards good tripwire tokens.
### Variants to test
- **V0 (current):** positive sentence + banned list at composition time;
Self-Review references the list.
- **V1 (auditor's checklist):** composition-time positive recipe only —
"Before finalizing a step, confirm it has: the literal code to write, a
runnable command with expected output, types and method names defined
within this plan, error handling shown explicitly. A step is complete
when an engineer could implement it without asking any follow-up
questions." Self-Review keeps a generic placeholder scan.
- **V2 (restructure by mechanism — predicted winner):** composition time
gets only V1's positive recipe; the named patterns move wholesale into
the Self-Review placeholder-scan step, reframed as recognition ("when
you scan, look for: 'TBD', 'TODO', 'Similar to Task N', …"). Same
tokens, relocated from the category that primes to the category that
detects.
- **V3 (control):** positive sentence only, no list anywhere.
### Micro-test design
- **Task:** opus writes a 2-3 task implementation plan from a deliberately
under-specified spec (under-specification is what tempts placeholders).
Use a fixture spec with: one well-specified task, one task whose error
handling the spec hand-waves, one task similar to the first (tempting
"Similar to Task 1").
- **Sampling:** 5+ reps per variant, default temperature, model
`claude-opus-4-8` (the model that writes plans in practice).
- **Programmatic scoring** (lower is better unless noted):
- banned-token count: `TBD|TODO|implement later|fill in details|appropriate error handling|handle edge cases|Similar to Task|Write tests for the above`
- steps lacking a fenced code block where the step changes code
- references to types/functions not defined anywhere in the plan output
- (higher is better) runnable commands with expected output per task
- **Two-stage scoring for V2:** also test the Self-Review half — feed each
generated plan back with the variant's Self-Review section and measure
whether the scan actually catches seeded placeholders (insert 2 known
placeholders into a fixture plan; detection rate is the metric).
- **Acceptance:** adopt a variant only if it beats V0 on banned-token count
without losing code-block coverage or self-review detection rate.
Expected cost: ~$6-10 total.
### PR scoping
Separate PR (writing-plans is a different skill; its "No Placeholders"
list is tuned content where the contributor guidelines demand eval
evidence). The PR must include: the micro-test harness + results table,
before/after text, and the V2 relocation rationale.
## The micro-test harness (method, so it isn't lost)
`/tmp/sdd-exp/micro/run-micro.py` and `/tmp/sdd-exp/micro2/run-micro2.py`
(2026-06-10; to be committed to superpowers-evals as
`docs/superpowers/skills/micro-testing-prompt-guidance.md` + scripts):
- One API call per sample: system prompt = the skill-guidance variant in
realistic surrounding context; user = a realistic mid-workflow scenario;
output = the composed artifact (dispatch prompt, plan, report).
- Programmatic scoring with greps for unambiguous markers; **manually
inspect every match before trusting a verdict** — one of tonight's
"violations" was the controller correctly quoting the prohibition, and
automated negation detection mislabeled another.
- ~$0.15-0.30/sample, seconds per iteration vs $12/50-min full eval runs.
Iterate phrasings here; confirm winners in full runs only when the
change is structural.
- Always include a no-guidance control — tonight it revealed both a
backfire (restating: prohibition worse than nothing) and a working
prohibition (test-reruns: 3/5 control failures vs 0/5 with either
phrasing).
## Result: writing-plans micro-test (run 2026-06-10, after this spec was written)
**Resolved — no change needed.** Stage 1 (3-task spec, no pressure): 0
placeholders in all 20 plans across all four variants including the
no-guidance control. Stage 1b (10-task spec, five near-identical commands
tempting "Similar to Task N", explicit ~2,500-word economy target): 40/40
clean — the single regex hit was a V2 self-review *attesting* "no
TBD/TODO ✓". Current-generation opus does not produce plan placeholders
even under deliberate pressure, with or without the banned-patterns list.
Disposition: leave the No Placeholders section exactly as it is (it costs
little and the counterfactual is unmeasurable); do NOT open the follow-up
PR. The V2 relocation design remains on file here should a future model
generation regress.
## Also explicitly not-dropped (tested-and-declined, with data)
Recorded so nobody re-proposes them without new evidence — full numbers in
the 2026-06-09 SDD design spec's Cost-iterations section:
- **Controller turn batching / parallel tool calls in one message:** the
controller emits exactly one tool call per message (0 multi-tool
messages across every measured run, with and without guidance). 46% of
controller turns are thinking/narration with no tool call — a
prompt-immune floor.
- **Pipelined reviews via parallel calls:** dead for the same reason.
- **Pipelined reviews via `run_in_background`:** mechanism adopted when
offered (7/28 dispatches) but benefit below the run-to-run noise floor
on 45-min scenarios (reviews are only ~30-60s each); adds dual
result-stream coordination. Worth revisiting only for plans whose
reviews are individually long.
- **Nuance clauses appended to winning recipes:** measurably degrade them
(C2: 3.8 noisy vs C: 3.0 consistent). Iterate by re-deriving the recipe,
not by appending caveats.
@@ -0,0 +1,244 @@
# Strict-Cost SDD — Design Spec
**Status:** Proposed experiment ladder (not implementation). Each rung ships
only with its gate evidence; abort any rung whose gates fail.
**Objective:** minimize dollars per plan-execution. Wall-clock is
unconstrained; token count matters only as a cost driver.
**Hard invariant:** quality. Concretely: `sdd-quality-reviewer-catches-
planted-defect` pass rate over **N=5 runs** (not 1 — single-run gates were
this campaign's weakest methodology), `sdd-rejects-extra-features` pass,
all end-to-end scenarios pass, blind A/B deliverable parity with the
current config. Any quality regression kills the rung, full stop.
## Where the dollars are (final 2026-06-10 config, go-fractals, ~$13/run)
| Component | $ | Driver |
|---|---|---|
| Controller (session model, opus) | ~6-7 | ~150 turns × resident context; prompt-immune turn floor (46% thinking/narration) |
| Implementers (sonnet, 10-13 dispatches) | ~5-6 | the actual work; ~25 turns each; ~13 pre-edit exploration calls each |
| Task reviewers (sonnet, 10) | ~1-1.5 | 3-9 turns each with package |
| Final review + fixes | ~1 | 6 turns with branch package |
Review-loop count (2-4 per run) is the biggest run-to-run cost variance;
loops are mostly caused by plan ambiguity the implementer resolved wrongly.
## Judgment guardrail (co-invariant with quality)
**Cheapen mechanics, never judgment.** Every rung must enumerate which
decisions it moves to a cheaper model and show each is *mechanical*
deterministic, scriptable, or cheaply verifiable after the fact. Judgment
stays at the highest tier or with the human. The judgment points in SDD,
explicitly:
- **BLOCKED / NEEDS_CONTEXT handling** — diagnosing why a subagent is stuck
and choosing the remedy
- **⚠️ "cannot verify from diff" resolution** — the controller adjudicating
with cross-task context
- **Dispatch curation** — ambiguity resolution and task-boundary drawing
(measured load-bearing: the Task 5 gradient-direction note prevented a
wrong implementation)
- **Review verdicts and severity calibration** — what is Important vs Minor
- **Review-loop adjudication** — deciding a finding is a false positive
- **Escalate-to-human recognition** — knowing the plan itself is wrong
A rung that would move any of these to a cheaper model must either (a)
restructure so the decision is made once by the expensive model at plan
time, (b) add an explicit escalation rule routing it back up at execution
time, or (c) die. "The cheap model usually gets it right" is not
acceptance evidence — judgment failures are rare-event, high-blast-radius,
and largely invisible to pass/fail gates, which is why every tier change
below carries a judgment audit (session-resume interrogation of each
judgment point in the gate runs, compared against the expensive-controller
baseline) in addition to the N=5 scenario gates.
## Thesis guardrail
SDD's thesis: **a fresh subagent per task with precisely curated context,
gated per task.** Rungs below must preserve it. Dispatch-time task batching
(one implementer dispatch handling several plan tasks) is **counter-thesis**
— it pollutes the fresh-context property and coarsens the gates — and is
deliberately NOT on the ladder. The thesis-compatible route to the same
dispatch economics is plan-time task right-sizing (L1): if the plan defines
fewer, better-sized tasks, SDD still runs one fresh subagent per task.
## The ladder (in expected $/leverage order)
### L1 — Plan-side crispness (writing-plans changes; est. $1.5-3/run, plus variance reduction)
**Status 2026-06-11 (final): elicitation tested end-to-end; claims
re-attributed.** Micro-tests: constraints header and Interfaces blocks
elicit deterministically (0→5/5, 0→100% of tasks, exact values);
right-sizing is modest and scale-dependent (9.4→8.4 tasks at svelte
scale, nothing to move at fractals scale). Full runs: an elicited plan
executed at $6.34/$8.49 — but the no-guidance control (opus plan,
complete code) hit $7.59/$7.73, inside that range. **The cost win
belongs to opus-written complete-code plans; the hand-written prose
fixture plans all prior numbers used are unrepresentative and ~2×
costlier to execute.** The guidance owns fidelity and variance instead:
deterministic constraints propagation (the one elicited-run fix was a
version-floor catch), exact cross-task interfaces, fix waves 1 vs 2-4
(the control plan shipped a real Sierpinski bug both runs had to fix).
The writing-plans PR claims those grounds, not dollars. Draft at
/tmp/sdd-exp/writing-plans-l1 (branch writing-plans-crisp).
The plan is upstream of every cost: task count sets dispatch count; plan
ambiguity sets review-loop count; plan completeness sets implementer
exploration. Current writing-plans optimizes for implementer success, not
execution economics. Changes to test:
1. **Task right-sizing guidance.** Today's plans produce tasks as small as
"create .gitignore" — each costing a full dispatch + review cycle
(~$0.60-1.00 fixed overhead). Add: "A task is the smallest unit that
carries its own test cycle and is worth a fresh reviewer's gate. Merge
setup/config steps into the task that needs them; split only at
boundaries where a reviewer could meaningfully reject." Fractals' plan
would drop from 10 tasks to ~7. Validate: dispatch count falls, gates
hold, review granularity still catches the planted defect.
2. **Structured `## Global Constraints` section** in the plan header
(version floors, naming/copy rules, platform requirements). Today these
live in design.md prose and reach reviewers only if the controller
remembers to paste them (a `go 1.26.1` floor violation shipped because
none did). A fixed heading makes them mechanically extractable —
`task-brief` can append them to every brief automatically (small script
change), removing a controller responsibility entirely.
3. **Per-task `Interfaces:` line** (consumes/produces, exact signatures).
The controller currently re-derives cross-task interfaces per dispatch
(its main legitimate "restating"), and implementers spend ~13 tool calls
re-discovering context. The planner already knows the interfaces; one
line per task moves the work to where it is done once.
4. **Per-task model-tier recommendation** from the planner ("mechanical /
standard / judgment"). The planner has the best information for the
Model Selection decision the controller currently re-makes per dispatch;
the controller keeps override authority.
Validation: micro-test the planner output shape (recipe-style, per the
instruction-design doctrine), then full runs. Note the 2026-06-10 result:
plan *placeholders* cannot be elicited from current opus — these changes
target economics and ambiguity, not placeholder hygiene.
### L2 — Controller tier (est. $4-5/run; the biggest single lever, gated hardest)
**Status 2026-06-11 (final): DIED AT THE GATES, as pre-registered — with
useful anatomy.** Recon was positive ($6.68/$8.05, n=2, mechanics clean).
The full battery split the judgment surface: the new
`sdd-escalates-broken-plan` scenario (explicit plan self-contradiction;
the human never volunteers it) passed **5/5 at sonnet** ($1.02-1.37/run;
opus baseline 2/2) — explicit conflicts get escalated. But the
planted-defect battery failed decisively: under a sonnet controller the
per-task quality gate collapsed into plan-compliance advocacy ("no
assertion, as required" listed under Strengths), the defect shipped in
4/5 runs (deterministic check), and only the tier-pinned opus final
reviewer ever caught it — while the same sonnet-tier reviewers under an
opus controller flagged it 5/5. Cheap controllers handle explicit
escalation; they absorb implicit authority-vs-quality adjudication.
A possible L2b (discrete rule: "a reviewer finding that conflicts with
the plan's text is the human's decision — escalate it") would route the
failing judgment through the escalation behavior that held; untested.
Original recon notes follow.
**Recon (superseded):**
Sonnet-controller runs (claude-sonnet coding-agent): all gates green at
**$6.68 and $8.05** / 31-41 min (combo band $11.67-14.84), tokens inside
the combo band — no cheap-controller turn inflation. 26/26 and 31/31
dispatches model-explicit, with heavier (and sane) haiku tiering than
opus controllers showed; review loops, per-task Important→fix→re-review,
and omnibus-fixer rules followed in both runs; the run-1 controller
caught a fixer side-effect (`go mod tidy` removed cobra) before
re-review — real adjudication, not silent absorption. But neither run
surfaced a BLOCKED/⚠️ event (the escalation points were never stressed)
and final reviews ran on sonnet rather than the most capable tier. The
N=5 quality gates + full judgment audit below remain mandatory before
any skill change.
The controller is half the dollars solely because it inherits the session
model. Its turn floor is prompt-immune, so the lever is the rate per turn —
but the controller is also where most judgment points live, so this rung is
designed judgment-first:
1. **Primary form — judgment moved up front, mechanics cheapened:** the
expensive model does the judgment-dense work at plan time (L1's
Interfaces lines, ambiguity resolutions, per-task constraints — i.e.
the dispatch curation is pre-written into the plan). The mid-tier
execution session then runs a loop that is genuinely mechanical:
extract brief, dispatch, run script, route verdicts. Explicit
escalation rules in the skill: on BLOCKED, on any ⚠️ item, on a
suspected false positive, or on anything the plan does not already
answer, the cheap controller STOPS and escalates (to the human, or to
a fresh expensive-model consultation dispatch) — it never resolves
judgment alone.
2. **Gates beyond the standard N=5:** a judgment audit — every
BLOCKED/⚠️/adjudication event in the gate runs interrogated via
session-resume and scored against how the opus-controller baseline
handled the same class of event; any silently-absorbed judgment call
(cheap controller resolving what it should have escalated) fails the
rung regardless of scenario verdicts.
3. **User authority preserved:** the skill recommends, never enforces, the
execution-session tier.
Caveat from this campaign: cheap-model turn inflation was measured on
multi-step *work*, not dispatch loops; whether a mid-tier controller holds
~150 turns is part of what the experiment determines.
### L3 — Reviewer tier (est. $0.7-1/run; most likely rung to die on the judgment guardrail)
**Status 2026-06-11: DEAD, as pre-registered.** Planted-defect ×5 with
forced-haiku task reviewers: 2 pass / 1 indeterminate / 2 fail (baseline
5/5); per-task haiku cleanly flagged 0 of 10 planted defects at correct
severity — 1 found-but-downgraded with the exact prohibited rationale,
9 missed or rationalized (DRY praised as YAGNI; assert-nothing test
called plan-compliant). Cheap reviewers fail by *advocating* for
defects; passing runs survived only on controller redundancy or the
final review. Recorded in the experiments log, Batch A-E. Do not
re-propose without a structurally different design.
The package reviewer is near-single-step mechanically (3 turns / 1 Read
when calm), which invalidates the original turn-inflation rationale for the
mid-tier floor — but reviewing is judgment through and through: severity
calibration, spec verdicts, knowing what not to flag. Mechanical cheapness
does not make the decisions mechanical. Test haiku-with-package only with
the full judgment battery: planted-defect ×5, a severity-calibration check
(seeded Minor-vs-Important pairs; miscalibration fails the rung), and the
escape-hatch variance re-measured at that tier. Prior expectation: this
rung dies, and that is a fine outcome — it converts "we suspect cheap
reviewers are bad" into recorded evidence.
### L4 — Resident-context diet (est. $0.5-1/run)
- `task-brief --list` mode: controller reads task headings + Global
Constraints, never the full plan (the plan body is already delivered via
briefs).
- Reports trim 15 → 8 lines.
- SKILL.md minification pass (every section added this week re-justified
at composition-recipe density; Codex pays ~10k chars × ~500 re-reads per
long session).
### L5 — Re-litigations (explicitly flagged, maintainer-vetoed or counter-thesis)
Recorded for completeness; each requires Jesse's explicit reversal before
any experiment:
- **Scoped re-reviews** (verify fix + regression scan instead of full
re-review): vetoed 2026-06-09; worth ~$0.50/run at most.
- **Dispatch-time task batching**: counter-thesis (see guardrail). L1.1
is the sanctioned form.
## Budget and sequencing
L1 and L2.1 are independent — run both first (~$80: micro-tests + 2×5-run
gates + A/B). L3 after L2 settles the controller (reviewer behavior depends
on dispatch quality; ~$25 — planted-defect runs are $2-3 each). L4 last
(cheap, but re-gate once after the stack; ~$30). Total ≲ $150 for the full
ladder with honest N=5 gates. Expected end state if every rung survives its gates: **$5-7/run on
fractals (from $12-15)**; if the judgment-sensitive rungs (L2 beyond its
primary form, L3) die as expected, **$8-10/run** — the honest target, since
the guardrail prices judgment above dollars by construction.
## Relationship to existing work
Builds on the 2026-06-09 task-scoped review dispatch design (PR #1717) and
the 2026-06-10 experiment campaign (evals
`docs/experiments/2026-06-10-sdd-cost-experiments.md` — consult the
negative-results section before adding rungs; turn-discipline and
parallel-call mechanisms are dead). Instruction wording for any new prose
follows the positive-instruction doctrine spec and gets micro-tested before
full runs. L1 is a writing-plans change → its own PR with eval evidence;
L2-L4 are SDD changes → separate PR(s).
@@ -0,0 +1,225 @@
# Visual Companion Auth Hardening Design
**Date:** 2026-06-10
**Status:** Draft for Drew review
## Goal
Fix the security and reliability gaps found in PR #1720's brainstorming visual
companion without changing the companion's core workflow or adding runtime
dependencies.
The fixes must be test-first and must leave clear automated evidence for:
- cross-origin browser tabs cannot inject companion events by riding cookies
- restart reconnect works without depending only on browser cookie behavior
- bearer keys do not remain in the visible URL after bootstrap
- `/files/*` cannot serve files outside the content directory
- future same-origin vendored UI libraries still work
## Threat Model
The companion serves agent-generated local UI for a single brainstorming
session. The important assets are:
- screen content served from the companion
- the session key
- `state/events`, which the agent reads as user feedback
- local files under the companion session directory
In scope attackers:
- a malicious browser tab on another `localhost` port
- a browser page that can make requests to the companion but should not be able
to authenticate as the companion UI
- a direct remote client when the server is bound to a non-loopback interface
- accidental leakage through URL history, referrers, or committed local state
- content-directory symlinks or path tricks that escape `/files/*`
Out of scope for this fix:
- malicious agent-authored screen HTML
- malicious same-origin vendored JavaScript loaded by a companion screen
This out-of-scope boundary is intentional. Companion screens are part of the
agent UI surface. They may use inline scripts today and may someday use
same-origin vendored libraries such as Alpine or Three.js. Protecting against
malicious screen HTML would require a larger sandboxed-iframe architecture with
a narrow message bridge; that is not the scope of this PR hardening pass.
## Current Failures
Automated and headed-browser testing found these failures in the PR branch:
1. A cross-origin localhost page can open a cookie-authenticated WebSocket and
write attacker-controlled choices to `state/events` after the real companion
page sets the cookie.
2. `/files/*` serves symlinks that point outside `content/`, including a symlink
to `state/server-info` containing the keyed URL.
3. The session key remains in the URL of the actual screen page, so same-origin
screen JavaScript and accidental referrers/history can see it.
4. The helper reconnects with a keyless `ws://host` URL. In headed Chrome, after
a same-port/same-token restart, the browser stopped presenting the cookie to
the restarted server, so the open tab stayed stuck on the tombstone until a
manual reload.
5. Shell lint and the lifecycle test need cleanup so the test pass is stable in
Codex.
## Design
### 1. Bootstrap Keyed Loads
`GET /?key=<token>` becomes a bootstrap response, not the screen response.
When the key is valid, the server:
1. sets the HttpOnly session cookie as it does today
2. returns a small HTML bootstrap page
3. the bootstrap page stores the key in tab-scoped `sessionStorage`
4. the bootstrap page navigates to `/` using `location.replace('/')`
After this, the visible screen URL is bare `/`, not `/?key=...`.
`GET /` with a valid cookie serves the current screen. `GET /` without a valid
cookie still returns the friendly 403 page. `GET /?key=<wrong>` returns 403.
Why `sessionStorage`: the helper needs a reconnect credential that survives
same-port restarts and does not depend only on cookie behavior. Because screen
HTML is trusted same-origin UI, storing the key in tab-scoped storage is
acceptable for this threat model. It is materially better than leaving the key
in the address bar, history, and referrer surface.
### 2. WebSocket Same-Origin Enforcement
WebSocket upgrades must pass both checks:
1. valid session auth by query key or cookie
2. if an `Origin` header is present, it must match the request target origin
The origin check should compare:
```text
Origin === "http://" + req.headers.host
```
Browser attacker page example:
```text
Origin: http://localhost:9999
Host: localhost:58088
```
This must be rejected even if the browser sends the companion cookie.
Legitimate companion page example:
```text
Origin: http://localhost:58088
Host: localhost:58088
```
This should be accepted when the key or cookie is valid.
Direct non-browser clients may omit `Origin`; they still need the session key.
### 3. Helper Reconnect Credential
`helper.js` should read the tab-scoped key from `sessionStorage` and append it
to the WebSocket URL:
```text
ws://<host>/?key=<stored-key>
```
If no stored key exists, the helper falls back to the current cookie-only
`ws://<host>` behavior. This preserves compatibility for already-loaded pages
that do have a valid cookie but no storage entry.
### 4. `/files/*` Containment
The file server should continue to reject empty names and dotfiles. It must also
ensure the file is a real regular file inside `CONTENT_DIR`.
Use realpath containment as the boundary:
- compute `realContentDir = fs.realpathSync(CONTENT_DIR)`
- compute `realFilePath = fs.realpathSync(filePath)`
- serve only when `realFilePath` equals a descendant of `realContentDir`
- reject symlinks and anything outside the content directory with 404
The server should keep using `path.basename` so nested paths remain unsupported.
### 5. Leak-Reduction Headers
Add conservative headers that do not block inline scripts or future same-origin
vendored libraries:
```text
Referrer-Policy: no-referrer
Cache-Control: no-store
X-Frame-Options: DENY
Content-Security-Policy: frame-ancestors 'none'
Cross-Origin-Resource-Policy: same-origin
```
Do not add a restrictive `script-src` CSP in this pass. The companion currently
injects inline helper JavaScript and future screens may load same-origin
vendored libraries.
### 6. Gitignore Durable Session State
Add `.superpowers/` to the repo root `.gitignore` so persisted companion state
and `.last-token` are not accidentally committed when using `--project-dir`.
### 7. Test Stability And Lint
Clean up shell lint warnings in the touched start/stop scripts.
Update the lifecycle test that invokes `start-server.sh --idle-timeout-minutes`
so it cannot hang under Codex's `CODEX_CI` foreground auto-detection. The test
should force background mode with `--background` when it expects the script to
return startup JSON.
## Testing Strategy
All behavior changes should be TDD:
1. write the failing focused test
2. run it and confirm it fails for the expected reason
3. implement the minimum fix
4. rerun the focused test
5. rerun the full brainstorm-server suite
Required focused regressions:
- valid keyed `/` returns bootstrap, not screen content
- bootstrap stores key in `sessionStorage` and strips the URL
- cookie-only `/` still serves screen content
- helper uses `sessionStorage` key for WebSocket URL
- same-origin cookie WebSocket opens
- cross-origin cookie WebSocket is rejected and writes no events
- direct key WebSocket still opens without `Origin`
- symlink under `content/` pointing to `state/server-info` returns 404
- security headers are present on normal HTML, bootstrap, 403, and file responses
- restart same port/token can authenticate reconnect with the stored key
- shell lint passes for touched shell scripts
- lifecycle suite does not hang under Codex
## Acceptance Criteria
- `cd tests/brainstorm-server && npm test` passes repeatedly without hanging.
- The security probe that previously wrote `attacker-injected` from another
localhost origin now fails to open the WebSocket and leaves `state/events`
unchanged.
- The symlink-to-`server-info` probe returns 404.
- A headed or headless browser keyed load ends on a bare `/` URL and the status
pill reaches Connected.
- A same-port/same-token restart reconnects automatically without manual reload.
- `scripts/lint-shell.sh` passes for the touched shell scripts.
## Deferred Work
If the project later needs to treat screen HTML as untrusted, design a separate
sandboxed iframe architecture. That should isolate generated screens on a
separate origin or sandboxed frame and expose only a narrow `postMessage` bridge
for user choices. Do not bundle that into this fix.
@@ -0,0 +1,254 @@
# Visual Companion Final Hardening Fixup Design
**Date:** 2026-06-11
**Status:** Draft for Drew review
## Goal
Finish the PR #1720 visual companion hardening pass so the branch is ready for
Jesse review with clean security behavior, deterministic tests, and a PR diff
that contains only the companion work.
This is a fixup on top of the existing auth hardening design. It should not
redesign the companion or expand the feature surface.
## Background
The previous hardening pass added keyed sessions, same-origin WebSocket checks,
URL key stripping, `/files/*` containment, leak-reduction headers, IPv6 URL
formatting, Windows lifecycle coverage, and PR evidence updates.
The final review pass found five remaining issues:
1. The root `GET /` screen-selection path can still serve symlinks or hardlinks
under `content/` that point outside the content directory.
2. When the preferred port is occupied, fallback servers can reuse a persisted
`.last-token`, creating two live same-project companion servers with the same
bearer key.
3. `stop-server.sh` can signal an unrelated `node server.cjs` process when
strong ownership proof is unavailable.
4. Some tests can pass against the wrong fallback process, leak background
processes on failure, or assume symlink support on Windows-like hosts.
5. The PR is currently conflicted because the branch contains an older `evals`
submodule bump that was handled separately.
## Non-Goals
- Do not add HTTPS tunnel or `wss://` origin semantics in this pass.
- Do not implement opt-out, free-text, or contrast-helper companion features.
- Do not vendor Alpine, Three.js, or any other JavaScript library.
- Do not attempt to sandbox malicious agent-authored screen HTML.
- Do not add backward compatibility for stale stop-server PID files unless Drew
explicitly approves that tradeoff.
## Inherited Security Invariants
This fixup preserves the auth hardening already designed and implemented:
- `.last-token` and `state/server-info` remain sensitive owner-only state.
- Fallback tokens may appear in startup JSON and `state/server-info`, but must
not be written to `.last-token`.
- Cookies remain port-named, `HttpOnly`, `SameSite=Strict`, and scoped to `/`.
- WebSocket upgrades still require a valid key or cookie.
- WebSocket `Origin` checks remain enforced when the browser supplies an
`Origin` header.
- Direct no-`Origin` clients remain allowed only when they carry the session key.
- Generated same-origin screen JavaScript and future same-origin vendored
libraries are trusted. Sandboxing malicious screen HTML remains deferred.
## Design
### 1. Rebase Onto Current `dev`
Rebase `brainstorming-companion` onto current `origin/dev` before implementation
work. Resolve the `evals` submodule conflict by taking `dev`.
After the rebase:
- `evals` must not appear in the PR diff.
- PR #1720 can still mention eval evidence that was run elsewhere, but it must
include exact external evidence: eval repo commit, scenario path, command,
result artifact path or id, and RED/GREEN outcome.
- The PR body must not imply the evals submodule bump is part of this PR.
- Any earlier PR-body text or comment implying the submodule bump is included
must be superseded by the final PR-body evidence.
### 2. Root Screen Containment
The root screen route must use the same containment boundary as `/files/*`.
`getNewestScreen()` should ignore any `.html` candidate that does not pass the
regular-file-inside-content-dir guard. That guard must resolve real paths and
ensure the served file is inside `CONTENT_DIR`. It must also preserve the
existing hardlink protection by rejecting files whose link count is not exactly
one when the platform reports link counts.
Expected behavior:
- A symlink under `content/` pointing outside `content/` is ignored.
- A hardlink under `content/` to `state/server-info` is ignored when
`fs.linkSync` succeeds and `lstat.nlink > 1`.
- If no safe screen file remains, the waiting page is served.
- Existing `/files/*` containment behavior remains unchanged: empty names,
dotfiles, symlinks, hardlinks, and directories still return 404.
### 3. Fallback Token Isolation
Port fallback must not reuse a token loaded from persisted `.last-token`.
Token source should be explicit in code:
- `BRAINSTORM_TOKEN` from the environment is an intentional operator/test
override. If the preferred port is occupied while an explicit environment
token is set, the server must fail closed instead of falling back, because the
occupied server may be using the same explicit token.
- `.last-token` is persisted state for same-port reconnect convenience. If the
server falls back because the preferred port is occupied, discard that loaded
token and generate a fresh unpersisted token for the fallback process.
- A newly generated token that was not loaded from `.last-token` can be reused
within the same process because no other live process is known to have it.
The fallback server must continue to avoid overwriting `.last-port` and
`.last-token`.
### 4. Stop-Server Ownership Proof
`start-server.sh` should create a per-start server instance id and pass it to
Node as an inert command-line argument, for example:
```text
node server.cjs --brainstorm-server-id=<id>
```
The id is not an auth credential. It is only process-ownership evidence for the
local lifecycle scripts. `server.cjs` can ignore the argument.
The id must use a shell/MSYS-safe alphabet, such as
`^[A-Za-z0-9_-]{32,64}$`. Store it in `state/server-instance-id` with
owner-only permissions.
`stop-server.sh` should read the expected id from state and only signal the PID
when the target process argv contains the exact argument
`--brainstorm-server-id=<id>` as a full argv token, not as a loose substring.
Prefer `/proc/<pid>/cmdline` when available, then fall back to wide `ps` output.
A matching instance id is sufficient proof even when `server-info` is missing
or `lsof` is unavailable. Existing port-to-PID checks may remain as additional
evidence.
Fail closed when ownership cannot be proven:
- missing PID file
- missing or malformed server id
- target command line unavailable
- target command line does not include the expected id
- old/stale session metadata without the new id
This intentionally prefers leaving a stale process running over killing an
unrelated process.
Operator-visible outcomes should be explicit:
- missing PID file returns `not_running`
- missing or malformed server id returns `stale_pid`
- unavailable command line returns `stale_pid`
- wrong or absent argv id returns `stale_pid`
- successful stop returns `stopped`
On `stale_pid` and `stopped` outcomes, remove `server.pid` and
`server-instance-id` so future stop attempts do not keep targeting the same
ambiguous process. Do not remove persistent session content.
### 5. Test Hardening
The test pass should be deterministic across macOS and the Windows Git Bash host
used for validation.
Required changes:
- Fixed-port suites must either fail fast if the server reports a fallback port
or drive all clients from the reported startup port.
- `stop-server.test.sh` needs a top-level cleanup trap before any background
process is started.
- Symlink-specific assertions should probe symlink capability and skip only that
assertion when the host cannot create usable test symlinks.
- Tests that create impostor processes must assert that the impostor survives
when lifecycle metadata is missing or insufficient.
- Windows/MSYS start-server tests must assert that Windows-like detection still
clears `BRAINSTORM_OWNER_PID`, still auto-foregrounds when appropriate, and
still passes the instance-id argv exactly.
### 6. Docs And PR Consistency
Before Jesse reviews, reconcile reviewer-visible docs and PR metadata:
- Update the issue catalog so dispositions match what this PR actually ships.
- Keep auto-open docs consistent with the implemented `--open` behavior.
- Keep the documented default idle timeout at 4 hours everywhere.
- Review the PR body against the template after the rebase.
- Record macOS, Windows, browser/manual, and external eval evidence in the PR
body with concrete commands and results.
## Testing Strategy
Use TDD for each behavior change:
1. Add or tighten a focused regression test.
2. Run it and confirm it fails for the expected reason.
3. Implement the smallest fix.
4. Rerun the focused test.
5. Rerun the full brainstorm-server suite.
Required focused regressions:
| Behavior | Test File | Focused Command | Expected RED | Expected GREEN |
| --- | --- | --- | --- | --- |
| Root route ignores symlink escape | `tests/brainstorm-server/server.test.js` | `node tests/brainstorm-server/server.test.js` | authenticated `GET /` serves linked outside content | response serves waiting page or safe screen |
| Root route ignores supported hardlink escape | `tests/brainstorm-server/server.test.js` | `node tests/brainstorm-server/server.test.js` | authenticated `GET /` serves hardlinked `server-info` | hardlink candidate is ignored when `nlink > 1` |
| `/files/*` containment stays unchanged | `tests/brainstorm-server/server.test.js` | `node tests/brainstorm-server/server.test.js` | existing containment test regresses | empty, dotfile, directory, symlink, hardlink cases remain 404 |
| Persisted-token fallback rotates token | `tests/brainstorm-server/lifecycle.test.js` | `node tests/brainstorm-server/lifecycle.test.js` | fallback URL key equals persisted preferred-port key | fallback URL key differs and is not written to `.last-token` |
| Explicit-token fallback fails closed | `tests/brainstorm-server/lifecycle.test.js` | `node tests/brainstorm-server/lifecycle.test.js` | server falls back while `BRAINSTORM_TOKEN` is set | process exits non-zero and does not start fallback |
| Fallback key cannot authenticate to original server | `tests/brainstorm-server/lifecycle.test.js` | `node tests/brainstorm-server/lifecycle.test.js` | fallback key receives 200 from original port | original port rejects fallback key |
| Correct instance id permits stop | `tests/brainstorm-server/stop-server.test.sh` | `bash tests/brainstorm-server/stop-server.test.sh` | real start-server-launched server survives | stop returns `stopped` and process exits |
| Wrong, missing, malformed, or stale id is safe | `tests/brainstorm-server/stop-server.test.sh` | `bash tests/brainstorm-server/stop-server.test.sh` | impostor is signaled | stop returns `stale_pid` and impostor survives |
| Fixed-port suites cannot pass through fallback | `tests/brainstorm-server/server.test.js`, `tests/brainstorm-server/auth.test.js` | respective `node` commands | test silently talks to fallback port | test fails clearly or uses reported port intentionally |
| Shell cleanup traps run on failures | `tests/brainstorm-server/stop-server.test.sh` | `bash tests/brainstorm-server/stop-server.test.sh` | failure leaves child processes | trap reaps background children |
| Windows/MSYS start behavior keeps lifecycle invariants | `tests/brainstorm-server/start-server.test.sh`, `tests/brainstorm-server/windows-lifecycle.test.sh` | `bash` test commands on macOS and `ballmer` | owner PID or argv handling regresses | owner PID is cleared, foreground detection holds, id argv is present |
Each RED/GREEN cycle should leave a short evidence note for the PR body: focused
command, failing assertion before the fix, passing assertion after the fix, and
whether the evidence was gathered on macOS or Windows.
## Verification
Before calling the fixup complete, run:
- `git fetch origin dev && git rebase origin/dev`
- `git diff --quiet origin/dev...HEAD -- evals`
- `gh pr view 1720 --json mergeStateStatus,statusCheckRollup,headRefOid`
- `cd tests/brainstorm-server && npm test`
- relevant focused test commands used during TDD
- `git diff --check`
- Node syntax checks for touched JavaScript files
- shell lint for touched shell files
- Windows validation on `ballmer`: full runnable brainstorm-server suite plus
the standalone Windows lifecycle probe
Manual/browser testing comes only after the automated pass is green.
## Acceptance Criteria
- PR #1720 rebases cleanly onto current `dev`.
- `evals` is absent from the PR diff.
- Root screen serving cannot read outside `content/` through symlink or
supported hardlink escapes.
- `/files/*` containment protections remain unchanged.
- No fallback server runs with a token that may be shared with the occupied
preferred-port server.
- `stop-server.sh` does not signal unrelated processes when ownership proof is
missing or ambiguous.
- `stop-server.sh` can still stop a legitimate server with a matching instance
id when `server-info` or `lsof` is unavailable.
- Focused RED/GREEN evidence is recorded for each regression.
- macOS and Windows validation evidence is recorded in the PR body.
- The PR body accurately describes what is in the branch and what evidence was
gathered externally.
+1
View File
@@ -12,6 +12,7 @@ Live in `tests/`. Currently:
- `tests/brainstorm-server/` — node test suite for the brainstorm server JS code.
- `tests/opencode/` — bash tests for OpenCode plugin loading, bootstrap caching, and tool registration.
- `tests/codex-plugin-sync/` — bash sync verification.
- `tests/kimi/` — bash/Python checks for Kimi plugin manifest wiring.
- `tests/claude-code/test-helpers.sh`, `analyze-token-usage.py` — utilities used by remaining bash tests.
- `tests/claude-code/test-subagent-driven-development.sh` — agent-can-describe-SDD test (no drill counterpart; tests description-recall, not behavior).
- `tests/claude-code/test-subagent-driven-development-integration.sh` — extended SDD integration with token analysis (drill covers the YAGNI subset; bash adds commit-count, Claude Code task-tracking, and token telemetry assertions).
+66 -130
View File
@@ -1,6 +1,8 @@
# Cross-Platform Polyglot Hooks for Claude Code
Claude Code plugins need hooks that work on Windows, macOS, and Linux. This document explains the polyglot wrapper technique that makes this possible.
Claude Code plugins need hooks that work on Windows, macOS, and Linux. This document describes the single generic dispatcher pattern used in `hooks/run-hook.cmd`.
> **Authoritative source:** `hooks/run-hook.cmd` is the canonical implementation. When this document and the code diverge, trust the code.
## The Problem
@@ -10,52 +12,22 @@ Claude Code runs hook commands through the system's default shell:
This creates several challenges:
1. **Script execution**: Windows CMD can't execute `.sh` files directly - it tries to open them in a text editor
1. **Script execution**: Windows CMD can't execute `.sh` files directly
2. **Path format**: Windows uses backslashes (`C:\path`), Unix uses forward slashes (`/path`)
3. **Environment variables**: `$VAR` syntax doesn't work in CMD
4. **No `bash` in PATH**: Even with Git Bash installed, `bash` isn't in the PATH when CMD runs
4. **`.sh` auto-prepend**: Claude Code on Windows automatically prepends `bash` to any command that contains `.sh` in its path — this interferes with the dispatcher if scripts have extensions
## The Solution: Polyglot `.cmd` Wrapper
## The Solution: Extensionless Scripts + Single Generic Dispatcher
A polyglot script is valid syntax in multiple languages simultaneously. Our wrapper is valid in both CMD and bash:
The repo uses one generic `run-hook.cmd` dispatcher for all hooks. Hook scripts are **extensionless** (`session-start`, not `session-start.sh`). This is deliberate: it prevents Claude Code's Windows auto-detection from prepending `bash` to the dispatcher command and breaking it.
```cmd
: << 'CMDBLOCK'
@echo off
"C:\Program Files\Git\bin\bash.exe" -l -c "\"$(cygpath -u \"$CLAUDE_PLUGIN_ROOT\")/hooks/session-start.sh\""
exit /b
CMDBLOCK
# Unix shell runs from here
"${CLAUDE_PLUGIN_ROOT}/hooks/session-start.sh"
```
### How It Works
#### On Windows (CMD.exe)
1. `: << 'CMDBLOCK'` - CMD sees `:` as a label (like `:label`) and ignores `<< 'CMDBLOCK'`
2. `@echo off` - Suppresses command echoing
3. The bash.exe command runs with:
- `-l` (login shell) to get proper PATH with Unix utilities
- `cygpath -u` converts Windows path to Unix format (`C:\foo``/c/foo`)
4. `exit /b` - Exits the batch script, stopping CMD here
5. Everything after `CMDBLOCK` is never reached by CMD
#### On Unix (bash/sh)
1. `: << 'CMDBLOCK'` - `:` is a no-op, `<< 'CMDBLOCK'` starts a heredoc
2. Everything until `CMDBLOCK` is consumed by the heredoc (ignored)
3. `# Unix shell runs from here` - Comment
4. The script runs directly with the Unix path
## File Structure
### File Structure
```
hooks/
├── hooks.json # Points to the .cmd wrapper
├── session-start.cmd # Polyglot wrapper (cross-platform entry point)
└── session-start.sh # Actual hook logic (bash script)
├── hooks.json # Points to run-hook.cmd with extensionless script name
├── run-hook.cmd # Cross-platform dispatcher (the polyglot wrapper)
└── session-start # Actual hook logic — extensionless bash script
```
### hooks.json
@@ -65,11 +37,12 @@ hooks/
"hooks": {
"SessionStart": [
{
"matcher": "startup|resume|clear|compact",
"matcher": "startup|clear|compact",
"hooks": [
{
"type": "command",
"command": "\"${CLAUDE_PLUGIN_ROOT}/hooks/session-start.cmd\""
"command": "\"${CLAUDE_PLUGIN_ROOT}/hooks/run-hook.cmd\" session-start",
"async": false
}
]
}
@@ -78,41 +51,63 @@ hooks/
}
```
Note: The path must be quoted because `${CLAUDE_PLUGIN_ROOT}` may contain spaces on Windows (e.g., `C:\Program Files\...`).
The path is quoted because `${CLAUDE_PLUGIN_ROOT}` may contain spaces.
## Requirements
## How `run-hook.cmd` Works at a High Level
### Windows
- **Git for Windows** must be installed (provides `bash.exe` and `cygpath`)
- Default installation path: `C:\Program Files\Git\bin\bash.exe`
- If Git is installed elsewhere, the wrapper needs modification
`run-hook.cmd` is a polyglot script: Windows treats the first block as batch
commands, while Unix shells treat that block as a no-op heredoc and continue
after it.
### Unix (macOS/Linux)
- Standard bash or sh shell
- The `.cmd` file must have execute permission (`chmod +x`)
Do not copy an implementation from this document. Read `hooks/run-hook.cmd`
directly when changing the dispatcher, and run `tests/hooks/test-session-start.sh`
afterward.
### How it works on Windows (CMD.exe)
1. The batch section validates the script name and resolves the hook directory
from the dispatcher's own location.
2. It tries bash in three places:
- `C:\Program Files\Git\bin\bash.exe`
- `C:\Program Files (x86)\Git\bin\bash.exe`
- `bash` on `PATH` (MSYS2, Cygwin, or a non-default Git install)
3. If bash is found, it runs the named extensionless hook script from the hooks
directory.
4. If no bash is found, the dispatcher exits `0` silently — the plugin
continues working, it just skips the hook.
5. `exit /b` stops CMD before it reaches the Unix section.
### How it works on Unix (bash/sh)
1. `: << 'CMDBLOCK'` opens a heredoc on a no-op command.
2. The entire CMD batch block is consumed by the heredoc and ignored.
3. After `CMDBLOCK`, bash resolves the script directory and `exec`s the named
extensionless script directly.
### Key design decisions
| Decision | Why |
|----------|-----|
| Extensionless scripts | Prevents Claude Code's Windows `.sh`-auto-prepend from interfering with the dispatcher command |
| No `-l` (login shell) | Not needed; hook scripts should be self-contained and not depend on login-shell PATH setup |
| No `cygpath` | Bash receives the Windows path directly and handles it correctly; `cygpath` was needed by the old `-c "..."` invocation pattern, not by direct exec |
| Silent exit on no-bash | Avoids breaking the plugin for users who don't have Git for Windows; hook context injection is skipped gracefully |
## Writing Cross-Platform Hook Scripts
Your actual hook logic goes in the `.sh` file. To ensure it works on Windows (via Git Bash):
Your hook logic goes in the extensionless script file. A few portable patterns:
### Do:
### Do
- Use pure bash builtins when possible
- Use `$(command)` instead of backticks
- Quote all variable expansions: `"$VAR"`
- Use `printf` or here-docs for output
### Avoid:
- External commands that may not be in PATH (sed, awk, grep)
- If you must use them, they're available in Git Bash but ensure PATH is set up (use `bash -l`)
### Avoid
- Relying on PATH-dependent tools without fallbacks (the hook runs without `-l`, so login-shell PATH is not set)
- Giving scripts a `.sh` extension — this triggers Claude Code's Windows auto-prepend
### Example: JSON Escaping Without sed/awk
### Example: JSON escaping without external tools
Instead of:
```bash
escaped=$(echo "$content" | sed 's/\\/\\\\/g' | sed 's/"/\\"/g' | awk '{printf "%s\\n", $0}')
```
Use pure bash:
```bash
escape_for_json() {
local input="$1"
@@ -133,80 +128,21 @@ escape_for_json() {
}
```
## Reusable Wrapper Pattern
For plugins with multiple hooks, you can create a generic wrapper that takes the script name as an argument:
### run-hook.cmd
```cmd
: << 'CMDBLOCK'
@echo off
set "SCRIPT_DIR=%~dp0"
set "SCRIPT_NAME=%~1"
"C:\Program Files\Git\bin\bash.exe" -l -c "cd \"$(cygpath -u \"%SCRIPT_DIR%\")\" && \"./%SCRIPT_NAME%\""
exit /b
CMDBLOCK
# Unix shell runs from here
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
SCRIPT_NAME="$1"
shift
"${SCRIPT_DIR}/${SCRIPT_NAME}" "$@"
```
### hooks.json using the reusable wrapper
```json
{
"hooks": {
"SessionStart": [
{
"matcher": "startup",
"hooks": [
{
"type": "command",
"command": "\"${CLAUDE_PLUGIN_ROOT}/hooks/run-hook.cmd\" session-start.sh"
}
]
}
],
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "\"${CLAUDE_PLUGIN_ROOT}/hooks/run-hook.cmd\" validate-bash.sh"
}
]
}
]
}
}
```
## Troubleshooting
### "bash is not recognized"
CMD can't find bash. The wrapper uses the full path `C:\Program Files\Git\bin\bash.exe`. If Git is installed elsewhere, update the path.
### "cygpath: command not found" or "dirname: command not found"
Bash isn't running as a login shell. Ensure `-l` flag is used.
CMD couldn't find bash in any of the three locations the dispatcher tries. The dispatcher exits silently (0) rather than erroring, so the hook is skipped. Install Git for Windows at the standard path or ensure `bash` is on `PATH`.
### Path has weird `\/` in it
`${CLAUDE_PLUGIN_ROOT}` expanded to a Windows path ending with backslash, then `/hooks/...` was appended. Use `cygpath` to convert the entire path.
### Hook runs on Unix but does nothing on Windows
### Script opens in text editor instead of running
The hooks.json is pointing directly to the `.sh` file. Point to the `.cmd` wrapper instead.
Check that the script filename is **extensionless** in `hooks.json`. A command like `run-hook.cmd session-start.sh` can trigger Claude Code's `.sh` auto-detection and bypass the intended CMD dispatcher path, or just try to run a non-existent `session-start.sh` script.
### Works in terminal but not as hook
Claude Code may run hooks differently. Test by simulating the hook environment:
```powershell
$env:CLAUDE_PLUGIN_ROOT = "C:\path\to\plugin"
cmd /c "C:\path\to\plugin\hooks\session-start.cmd"
```
### Hook doesn't fire at all
Verify the `matcher` in `hooks.json` matches the event type your harness emits. Claude Code uses `startup|clear|compact`; Codex uses `startup|resume|clear`. Check `hooks-codex.json` for the Codex variant.
## Related Issues
- [anthropics/claude-code#9758](https://github.com/anthropics/claude-code/issues/9758) - .sh scripts open in editor on Windows
- [anthropics/claude-code#3417](https://github.com/anthropics/claude-code/issues/3417) - Hooks don't work on Windows
- [anthropics/claude-code#6023](https://github.com/anthropics/claude-code/issues/6023) - CLAUDE_PROJECT_DIR not found
- [anthropics/claude-code#9758](https://github.com/anthropics/claude-code/issues/9758) `.sh` scripts open in editor on Windows
- [anthropics/claude-code#3417](https://github.com/anthropics/claude-code/issues/3417) Hooks don't work on Windows
+1 -1
Submodule evals updated: e2b37138c8...db37d5fbec
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env bash
#
# Lint shell scripts in this repository.
#
# Usage:
# scripts/lint-shell.sh [--all] [--format] [--strict] [file ...]
#
# By default, runs ShellCheck and shell syntax checks on changed shell scripts.
# Use --format to format with shfmt before linting. Use --all for the full tracked
# baseline, or pass files explicitly to lint a smaller set.
set -euo pipefail
usage() {
sed -n '2,9p' "$0" | sed 's/^# \{0,1\}//'
}
die() {
echo "error: $*" >&2
exit 1
}
require_tool() {
command -v "$1" >/dev/null 2>&1 || die "required tool '$1' is not on PATH"
}
is_shell_file() {
local path="$1"
local first_line=""
[[ -f "$path" ]] || return 1
case "$path" in
*.sh)
return 0
;;
esac
IFS= read -r first_line <"$path" || true
[[ "$first_line" =~ ^#!.*[/[:space:]](bash|dash|ksh|sh)([[:space:]]|$) ]]
}
ensure_git_work_tree() {
git rev-parse --is-inside-work-tree >/dev/null 2>&1 \
|| die "run this from inside a git work tree, or pass files explicitly"
}
add_shell_file() {
local path
local existing
path="$1"
if ! is_shell_file "$path"; then
return 0
fi
if [[ "${#files[@]}" -gt 0 ]]; then
for existing in "${files[@]}"; do
if [[ "$existing" == "$path" ]]; then
return 0
fi
done
fi
files+=("$path")
}
collect_all_shell_files() {
local path
ensure_git_work_tree
while IFS= read -r -d '' path; do
add_shell_file "$path"
done < <(git ls-files -z)
}
collect_changed_shell_files() {
local path
ensure_git_work_tree
if git rev-parse --verify HEAD >/dev/null 2>&1; then
while IFS= read -r -d '' path; do
add_shell_file "$path"
done < <(git diff --name-only -z --diff-filter=ACMR HEAD)
while IFS= read -r -d '' path; do
add_shell_file "$path"
done < <(git diff --cached --name-only -z --diff-filter=ACMR)
else
collect_all_shell_files
fi
while IFS= read -r -d '' path; do
add_shell_file "$path"
done < <(git ls-files --others --exclude-standard -z)
}
collect_requested_shell_files() {
local path
for path in "$@"; do
add_shell_file "$path"
done
}
syntax_shell_for() {
local path="$1"
local first_line=""
IFS= read -r first_line <"$path" || true
case "$first_line" in
*"/sh"* | *" env sh"* | *"/dash"* | *" env dash"*)
printf 'sh'
;;
*)
printf 'bash'
;;
esac
}
run_syntax_checks() {
local file
local shell_name
for file in "$@"; do
shell_name="$(syntax_shell_for "$file")"
case "$shell_name" in
sh)
sh -n "$file"
;;
bash)
bash -n "$file"
;;
*)
die "unsupported shell for syntax check: $shell_name"
;;
esac
done
}
format=false
strict=false
all=false
requested_files=()
while [[ $# -gt 0 ]]; do
case "$1" in
--all)
all=true
;;
--format)
format=true
;;
--strict)
strict=true
;;
-h | --help)
usage
exit 0
;;
--)
shift
requested_files+=("$@")
break
;;
-*)
die "unknown option: $1"
;;
*)
requested_files+=("$1")
;;
esac
shift
done
require_tool shellcheck
if [[ "$format" == true ]]; then
require_tool shfmt
fi
files=()
if [[ "${#requested_files[@]}" -gt 0 ]]; then
collect_requested_shell_files "${requested_files[@]}"
elif [[ "$all" == true ]]; then
collect_all_shell_files
else
collect_changed_shell_files
fi
if [[ "${#files[@]}" -eq 0 ]]; then
echo "No shell files found."
exit 0
fi
if [[ "$format" == true ]]; then
echo "Formatting ${#files[@]} shell files"
shfmt_args=(-i 2 -ci -bn)
shfmt "${shfmt_args[@]}" -w "${files[@]}"
fi
echo "Linting ${#files[@]} shell files"
shellcheck_args=(--severity=warning --external-sources --source-path=SCRIPTDIR)
if [[ "$strict" == true ]]; then
shellcheck_args+=("--enable=check-extra-masked-returns,check-set-e-suppressed,quote-safe-variables,deprecate-which,avoid-nullary-conditions")
fi
shellcheck "${shellcheck_args[@]}" "${files[@]}"
run_syntax_checks "${files[@]}"
+1
View File
@@ -52,6 +52,7 @@ EXCLUDES=(
"/.gitattributes"
"/.github/"
"/.gitignore"
"/.kimi-plugin/"
"/.opencode/"
"/.pi/"
"/.version-bump.json"
+5 -10
View File
@@ -22,7 +22,7 @@ Every project goes through this process. A todo list, a single-function utility,
You MUST create a task for each of these items and complete them in order:
1. **Explore project context** — check files, docs, recent commits
2. **Offer visual companion** (if topic will involve visual questions) — this is its own message, not combined with a clarifying question. See the Visual Companion section below.
2. **Offer the visual companion just-in-time** — NOT upfront. The first time a question would genuinely be clearer shown than described, offer it then (its own message); on approval its browser tab opens for you. If no visual question ever arises, never offer it. See the Visual Companion section below.
3. **Ask clarifying questions** — one at a time, understand purpose/constraints/success criteria
4. **Propose 2-3 approaches** — with trade-offs and your recommendation
5. **Present design** — in sections scaled to their complexity, get user approval after each section
@@ -36,8 +36,6 @@ You MUST create a task for each of these items and complete them in order:
```dot
digraph brainstorming {
"Explore project context" [shape=box];
"Visual questions ahead?" [shape=diamond];
"Offer Visual Companion\n(own message, no other content)" [shape=box];
"Ask clarifying questions" [shape=box];
"Propose 2-3 approaches" [shape=box];
"Present design sections" [shape=box];
@@ -47,10 +45,7 @@ digraph brainstorming {
"User reviews spec?" [shape=diamond];
"Invoke writing-plans skill" [shape=doublecircle];
"Explore project context" -> "Visual questions ahead?";
"Visual questions ahead?" -> "Offer Visual Companion\n(own message, no other content)" [label="yes"];
"Visual questions ahead?" -> "Ask clarifying questions" [label="no"];
"Offer Visual Companion\n(own message, no other content)" -> "Ask clarifying questions";
"Explore project context" -> "Ask clarifying questions";
"Ask clarifying questions" -> "Propose 2-3 approaches";
"Propose 2-3 approaches" -> "Present design sections";
"Present design sections" -> "User approves design?";
@@ -148,10 +143,10 @@ Wait for the user's response. If they request changes, make them and re-run the
A browser-based companion for showing mockups, diagrams, and visual options during brainstorming. Available as a tool — not a mode. Accepting the companion means it's available for questions that benefit from visual treatment; it does NOT mean every question goes through the browser.
**Offering the companion:** When you anticipate that upcoming questions will involve visual content (mockups, layouts, diagrams), offer it once for consent:
> "Some of what we're working on might be easier to explain if I can show it to you in a web browser. I can put together mockups, diagrams, comparisons, and other visuals as we go. This feature is still new and can be token-intensive. Want to try it? (Requires opening a local URL)"
**Offering the companion (just-in-time):** Do NOT offer it upfront. Wait until a question would genuinely be clearer shown than told — a real mockup / layout / diagram question, not merely a UI *topic*. The first time that happens, offer it then, as its own message:
> "This next part might be easier if I show you — I can put together mockups, diagrams, and comparisons in a browser tab as we go. It's still new and can be token-intensive. Want me to? I'll open it for you."
**This offer MUST be its own message.** Do not combine it with clarifying questions, context summaries, or any other content. The message should contain ONLY the offer above and nothing else. Wait for the user's response before continuing. If they decline, proceed with text-only brainstorming.
**This offer MUST be its own message.** Only the offer — no clarifying question, summary, or other content. Wait for the user's response. If they accept, start the server with `--open` so their browser opens to the first screen automatically. If they decline, continue text-only and don't offer again unless they raise it.
**Per-question decision:** Even after the user accepts, decide FOR EACH QUESTION whether to use the browser or the terminal. The test: **would the user understand this better by seeing it than reading it?**
@@ -73,8 +73,8 @@
flex-shrink: 0;
}
.header h1 { font-size: 0.85rem; font-weight: 500; color: var(--text-secondary); }
.header .status { font-size: 0.7rem; color: var(--success); display: flex; align-items: center; gap: 0.4rem; }
.header .status::before { content: ''; width: 6px; height: 6px; background: var(--success); border-radius: 50%; }
.header .status { font-size: 0.7rem; color: var(--status-color, var(--success)); display: flex; align-items: center; gap: 0.4rem; }
.header .status::before { content: ''; width: 6px; height: 6px; background: var(--status-color, var(--success)); border-radius: 50%; }
.main { flex: 1; overflow-y: auto; }
#frame-content { padding: 2rem; min-height: 100%; }
@@ -197,7 +197,7 @@
<body>
<div class="header">
<h1><a href="https://github.com/obra/superpowers" style="color: inherit; text-decoration: none;">Superpowers Brainstorming</a></h1>
<div class="status">Connected</div>
<div class="status">Connecting…</div>
</div>
<div class="main">
+101 -7
View File
@@ -1,26 +1,120 @@
(function() {
const WS_URL = 'ws://' + window.location.host;
const MIN_RECONNECT_MS = 500;
const MAX_RECONNECT_MS = 30000;
const TOMBSTONE_AFTER_MS = 15000; // show the "paused" overlay after this long disconnected
// Pure: next backoff delay (doubles, capped). Exported for unit tests.
function nextReconnectDelay(current, max) {
return Math.min(current * 2, max);
}
if (typeof module !== 'undefined' && module.exports) {
module.exports = { nextReconnectDelay, MIN_RECONNECT_MS, MAX_RECONNECT_MS, TOMBSTONE_AFTER_MS };
}
// Everything below is browser-only; bail out when loaded in Node (tests).
if (typeof window === 'undefined') return;
let ws = null;
let eventQueue = [];
let reconnectDelay = MIN_RECONNECT_MS;
let reconnectTimer = null;
let disconnectedSince = null;
let everConnected = false;
let tombstoneShown = false;
function sessionKey() {
try {
return window.sessionStorage && window.sessionStorage.getItem('brainstorm-session-key');
} catch (e) {}
return null;
}
function websocketUrl() {
const key = sessionKey();
return 'ws://' + window.location.host + (key ? '/?key=' + encodeURIComponent(key) : '');
}
function reloadAfterRecovery() {
const key = sessionKey();
if (key) {
window.location.replace('/?key=' + encodeURIComponent(key));
} else {
window.location.reload();
}
}
// Reflect connection state in the frame's status pill (absent on full-doc screens).
function setStatus(state) {
const el = document.querySelector('.status');
if (!el) return;
const map = {
connecting: ['Connecting…', 'var(--text-tertiary)'],
connected: ['Connected', 'var(--success)'],
reconnecting: ['Reconnecting…', 'var(--warning)'],
disconnected: ['Disconnected', 'var(--error)']
};
const [text, color] = map[state] || map.disconnected;
el.textContent = text;
el.style.setProperty('--status-color', color);
}
// Self-styled so it works on framed and full-document screens alike.
function showTombstone() {
if (tombstoneShown) return;
tombstoneShown = true;
const el = document.createElement('div');
el.id = 'bs-tombstone';
el.style.cssText = 'position:fixed;inset:0;z-index:99999;display:flex;' +
'align-items:center;justify-content:center;padding:2rem;text-align:center;' +
'background:rgba(20,20,22,0.92);color:#f5f5f7;font-family:system-ui,sans-serif';
el.innerHTML = '<div style="max-width:480px">' +
'<h2 style="margin:0 0 .5rem;font-weight:600">Companion paused</h2>' +
'<p style="margin:0;opacity:.85">This brainstorm companion has stopped. ' +
'Ask your coding agent to bring it back — this page reconnects automatically.</p></div>';
if (document.body) document.body.appendChild(el);
}
function connect() {
ws = new WebSocket(WS_URL);
if (reconnectTimer) { clearTimeout(reconnectTimer); reconnectTimer = null; }
setStatus(everConnected ? 'reconnecting' : 'connecting');
ws = new WebSocket(websocketUrl());
ws.onopen = () => {
const recovered = tombstoneShown;
everConnected = true;
disconnectedSince = null;
reconnectDelay = MIN_RECONNECT_MS;
tombstoneShown = false;
setStatus('connected');
eventQueue.forEach(e => ws.send(JSON.stringify(e)));
eventQueue = [];
// Recovered from a tombstoned outage (e.g. the server restarted on the same
// port) — reload through the keyed bootstrap when possible so the cookie is
// refreshed before the visible URL returns to bare /.
if (recovered) reloadAfterRecovery();
};
ws.onmessage = (msg) => {
const data = JSON.parse(msg.data);
if (data.type === 'reload') {
window.location.reload();
}
let data;
try { data = JSON.parse(msg.data); } catch (e) { return; }
if (data.type === 'reload') window.location.reload();
};
ws.onclose = () => {
setTimeout(connect, 1000);
ws = null;
if (disconnectedSince === null) disconnectedSince = Date.now();
if (Date.now() - disconnectedSince >= TOMBSTONE_AFTER_MS) {
setStatus('disconnected');
showTombstone();
} else {
setStatus('reconnecting');
}
reconnectTimer = setTimeout(connect, reconnectDelay);
reconnectDelay = nextReconnectDelay(reconnectDelay, MAX_RECONNECT_MS);
};
// Let onclose own reconnection so we don't schedule it twice.
ws.onerror = () => { try { ws.close(); } catch (e) {} };
}
function sendEvent(event) {
+327 -24
View File
@@ -7,6 +7,7 @@ const path = require('path');
const OPCODES = { TEXT: 0x01, CLOSE: 0x08, PING: 0x09, PONG: 0x0A };
const WS_MAGIC = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11';
const MAX_FRAME_PAYLOAD_BYTES = 10 * 1024 * 1024;
function computeAcceptKey(clientKey) {
return crypto.createHash('sha1').update(clientKey + WS_MAGIC).digest('base64');
@@ -53,10 +54,18 @@ function decodeFrame(buffer) {
offset = 4;
} else if (payloadLen === 127) {
if (buffer.length < 10) return null;
payloadLen = Number(buffer.readBigUInt64BE(2));
const extendedLen = buffer.readBigUInt64BE(2);
if (extendedLen > BigInt(MAX_FRAME_PAYLOAD_BYTES)) {
throw new Error('WebSocket frame payload exceeds maximum allowed size');
}
payloadLen = Number(extendedLen);
offset = 10;
}
if (payloadLen > MAX_FRAME_PAYLOAD_BYTES) {
throw new Error('WebSocket frame payload exceeds maximum allowed size');
}
const maskOffset = offset;
const dataOffset = offset + 4;
const totalLen = dataOffset + payloadLen;
@@ -73,7 +82,21 @@ function decodeFrame(buffer) {
// ========== Configuration ==========
const PORT = process.env.BRAINSTORM_PORT || (49152 + Math.floor(Math.random() * 16383));
const PORT_FILE = process.env.BRAINSTORM_PORT_FILE || null;
const randomPort = () => 49152 + Math.floor(Math.random() * 16383);
// Prefer an explicit port, else the port this session last bound (so a restart
// reuses it and an already-open browser tab reconnects), else a random high port.
function preferredPort() {
if (process.env.BRAINSTORM_PORT) return Number(process.env.BRAINSTORM_PORT);
if (PORT_FILE) {
try {
const p = Number(fs.readFileSync(PORT_FILE, 'utf-8').trim());
if (Number.isInteger(p) && p > 1023 && p < 65536) return p;
} catch (e) { /* no prior port recorded */ }
}
return randomPort();
}
let PORT = preferredPort();
const HOST = process.env.BRAINSTORM_HOST || '127.0.0.1';
const URL_HOST = process.env.BRAINSTORM_URL_HOST || (HOST === '127.0.0.1' ? 'localhost' : HOST);
const SESSION_DIR = process.env.BRAINSTORM_DIR || '/tmp/brainstorm';
@@ -81,6 +104,44 @@ const CONTENT_DIR = path.join(SESSION_DIR, 'content');
const STATE_DIR = path.join(SESSION_DIR, 'state');
let ownerPid = process.env.BRAINSTORM_OWNER_PID ? Number(process.env.BRAINSTORM_OWNER_PID) : null;
// Per-session secret key. The companion is reachable by any local browser tab
// and, when bound to a non-loopback host, by any host that can route to it.
// The key authenticates the real client uniformly across loopback, tunnel, and
// remote binds — and defeats DNS rebinding — where a Host/Origin allowlist
// cannot. It rides the served URL as ?key= and is mirrored into a cookie on
// first load so same-origin subresources and the WebSocket carry it for free.
// Persisted alongside the port (BRAINSTORM_TOKEN_FILE) so a restart keeps the
// same key and an already-open tab's cookie still validates.
const TOKEN_FILE = process.env.BRAINSTORM_TOKEN_FILE || null;
function generateToken() {
return crypto.randomBytes(32).toString('hex');
}
function chmodOwnerOnly(file) {
try { fs.chmodSync(file, 0o600); } catch (e) { /* best effort */ }
}
function initialToken() {
if (process.env.BRAINSTORM_TOKEN) {
return { value: process.env.BRAINSTORM_TOKEN, source: 'env' };
}
if (TOKEN_FILE) {
try {
const t = fs.readFileSync(TOKEN_FILE, 'utf-8').trim();
if (/^[0-9a-f]{32,}$/i.test(t)) {
chmodOwnerOnly(TOKEN_FILE);
return { value: t, source: 'file' };
}
} catch (e) { /* no prior token recorded */ }
}
return { value: generateToken(), source: 'generated' };
}
const tokenInfo = initialToken();
let TOKEN = tokenInfo.value;
let tokenSource = tokenInfo.source;
let COOKIE_NAME = 'brainstorm-key-' + PORT; // refined to the actual bound port in onListen
const MIME_TYPES = {
'.html': 'text/html', '.css': 'text/css', '.js': 'application/javascript',
'.json': 'application/json', '.png': 'image/png', '.jpg': 'image/jpeg',
@@ -98,6 +159,30 @@ h1 { color: #333; } p { color: #666; }</style>
<body><h1>Brainstorm Companion</h1>
<p>Waiting for the agent to push a screen...</p></body></html>`;
const FORBIDDEN_PAGE = `<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Session key required</title>
<style>body { font-family: system-ui, sans-serif; padding: 2rem; max-width: 800px; margin: 0 auto; }
h1 { color: #333; } p { color: #666; } code { background: #f0f0f0; padding: 0.1em 0.3em; border-radius: 4px; }</style>
</head>
<body><h1>Session key required</h1>
<p>This page needs the full URL your coding agent gave you, including the
<code>?key=&hellip;</code> part. Copy the complete URL and open it again.</p></body></html>`;
function bootstrapPage(key) {
const jsonKey = JSON.stringify(String(key));
return `<!DOCTYPE html>
<html>
<head><meta charset="utf-8"><title>Opening Brainstorm Companion</title></head>
<body>
<script>
try { sessionStorage.setItem('brainstorm-session-key', ${jsonKey}); } catch (e) {}
location.replace('/');
</script>
</body>
</html>`;
}
const frameTemplate = fs.readFileSync(path.join(__dirname, 'frame-template.html'), 'utf-8');
const helperScript = fs.readFileSync(path.join(__dirname, 'helper.js'), 'utf-8');
const helperInjection = '<script>\n' + helperScript + '\n</script>';
@@ -115,20 +200,144 @@ function wrapInFrame(content) {
function getNewestScreen() {
const files = fs.readdirSync(CONTENT_DIR)
.filter(f => f.endsWith('.html'))
.filter(f => !f.startsWith('.') && f.endsWith('.html'))
.map(f => {
const fp = path.join(CONTENT_DIR, f);
if (!isRegularFileInsideContentDir(fp)) return null;
return { path: fp, mtime: fs.statSync(fp).mtime.getTime() };
})
.filter(Boolean)
.sort((a, b) => b.mtime - a.mtime);
return files.length > 0 ? files[0].path : null;
}
function urlHostForHttp(host) {
const h = String(host);
if (h.startsWith('[') && h.endsWith(']')) return h;
return h.includes(':') ? '[' + h + ']' : h;
}
function companionUrl() {
return 'http://' + urlHostForHttp(URL_HOST) + ':' + PORT + '/?key=' + TOKEN;
}
function browserLauncherForPlatform(url, {
platform = process.platform,
osRelease = require('os').release(),
env = process.env
} = {}) {
const isWSL = platform === 'linux' && /microsoft/i.test(osRelease);
if (platform === 'darwin') return { bin: 'open', args: [url] };
if (platform === 'win32' || isWSL) {
return { bin: 'rundll32.exe', args: ['url.dll,FileProtocolHandler', url] };
}
if (env.DISPLAY || env.WAYLAND_DISPLAY) return { bin: 'xdg-open', args: [url] };
return null;
}
function isRegularFileInsideContentDir(filePath) {
let stat, realContentDir, realFilePath;
try {
stat = fs.lstatSync(filePath);
if (stat.isSymbolicLink()) return false;
if (!stat.isFile()) return false;
if (stat.nlink !== 1) return false;
realContentDir = fs.realpathSync(CONTENT_DIR);
realFilePath = fs.realpathSync(filePath);
} catch (e) {
return false;
}
return realFilePath.startsWith(realContentDir + path.sep);
}
// ========== Authentication ==========
function timingSafeEqualStr(a, b) {
const ab = Buffer.from(String(a));
const bb = Buffer.from(String(b));
if (ab.length !== bb.length) return false;
return crypto.timingSafeEqual(ab, bb);
}
function parseCookies(header) {
const out = {};
if (!header) return out;
for (const part of header.split(';')) {
const eq = part.indexOf('=');
if (eq < 0) continue;
out[part.slice(0, eq).trim()] = part.slice(eq + 1).trim();
}
return out;
}
// A request is authorized if it carries the session key as ?key= or as the
// session cookie. Both are compared in constant time.
function isAuthorized(req) {
const q = req.url.indexOf('?');
if (q >= 0) {
const params = new URLSearchParams(req.url.slice(q + 1));
if (params.has('key')) {
const key = params.get('key');
return Boolean(key && timingSafeEqualStr(key, TOKEN));
}
}
const cookie = parseCookies(req.headers['cookie'])[COOKIE_NAME];
if (cookie && timingSafeEqualStr(cookie, TOKEN)) return true;
return false;
}
function pathnameOf(url) {
const q = url.indexOf('?');
return q >= 0 ? url.slice(0, q) : url;
}
function queryKey(url) {
const q = url.indexOf('?');
if (q < 0) return null;
return new URLSearchParams(url.slice(q + 1)).get('key');
}
function securityHeaders(headers = {}) {
return {
'Referrer-Policy': 'no-referrer',
'Cache-Control': 'no-store',
'X-Frame-Options': 'DENY',
'Content-Security-Policy': "frame-ancestors 'none'",
'Cross-Origin-Resource-Policy': 'same-origin',
...headers
};
}
function isAllowedWebSocketOrigin(req) {
const origin = req.headers.origin;
if (!origin) return true;
const host = req.headers.host;
if (!host) return false;
return origin === 'http://' + host;
}
// ========== HTTP Request Handler ==========
function handleRequest(req, res) {
touchActivity();
if (req.method === 'GET' && req.url === '/') {
if (!isAuthorized(req)) {
res.writeHead(403, securityHeaders({ 'Content-Type': 'text/html; charset=utf-8' }));
res.end(FORBIDDEN_PAGE);
return;
}
touchActivity(); // only authorized requests count as activity
// Mirror the key into a cookie so same-origin subresources (/files/*) can
// authenticate after bootstrap. HttpOnly keeps it away from page scripts; the
// WebSocket Origin check below is what blocks cross-origin localhost injection.
res.setHeader('Set-Cookie',
COOKIE_NAME + '=' + TOKEN + '; HttpOnly; SameSite=Strict; Path=/');
const pathname = pathnameOf(req.url);
const keyFromQuery = queryKey(req.url);
if (req.method === 'GET' && pathname === '/' && keyFromQuery && timingSafeEqualStr(keyFromQuery, TOKEN)) {
res.writeHead(200, securityHeaders({ 'Content-Type': 'text/html; charset=utf-8' }));
res.end(bootstrapPage(keyFromQuery));
} else if (req.method === 'GET' && pathname === '/') {
const screenFile = getNewestScreen();
let html = screenFile
? (raw => isFullDocument(raw) ? raw : wrapInFrame(raw))(fs.readFileSync(screenFile, 'utf-8'))
@@ -140,22 +349,24 @@ function handleRequest(req, res) {
html += helperInjection;
}
res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' });
res.writeHead(200, securityHeaders({ 'Content-Type': 'text/html; charset=utf-8' }));
res.end(html);
} else if (req.method === 'GET' && req.url.startsWith('/files/')) {
const fileName = req.url.slice(7);
const filePath = path.join(CONTENT_DIR, path.basename(fileName));
if (!fs.existsSync(filePath)) {
res.writeHead(404);
} else if (req.method === 'GET' && pathname.startsWith('/files/')) {
const fileName = path.basename(pathname.slice(7));
const filePath = path.join(CONTENT_DIR, fileName);
// Reject empty/dotfile names and anything that isn't a regular file —
// `/files/` would otherwise resolve to CONTENT_DIR and crash readFileSync (EISDIR).
if (!fileName || fileName.startsWith('.') || !isRegularFileInsideContentDir(filePath)) {
res.writeHead(404, securityHeaders());
res.end('Not found');
return;
}
const ext = path.extname(filePath).toLowerCase();
const contentType = MIME_TYPES[ext] || 'application/octet-stream';
res.writeHead(200, { 'Content-Type': contentType });
res.writeHead(200, securityHeaders({ 'Content-Type': contentType }));
res.end(fs.readFileSync(filePath));
} else {
res.writeHead(404);
res.writeHead(404, securityHeaders());
res.end('Not found');
}
}
@@ -165,6 +376,8 @@ function handleRequest(req, res) {
const clients = new Set();
function handleUpgrade(req, socket) {
if (!isAuthorized(req) || !isAllowedWebSocketOrigin(req)) { socket.destroy(); return; }
const key = req.headers['sec-websocket-key'];
if (!key) { socket.destroy(); return; }
@@ -231,7 +444,7 @@ function handleMessage(text) {
}
touchActivity();
console.log(JSON.stringify({ source: 'user-event', ...event }));
if (event.choice) {
if (event && event.choice) {
const eventsFile = path.join(STATE_DIR, 'events');
fs.appendFileSync(eventsFile, JSON.stringify(event) + '\n');
}
@@ -244,9 +457,44 @@ function broadcast(msg) {
}
}
// Best-effort: open the user's browser the first time a screen is actually ready
// to show. Skips when disabled, on a non-loopback (remote) bind, or when a
// browser is already connected. Override the launcher with BRAINSTORM_OPEN_CMD.
let browserOpened = false;
function maybeOpenBrowser() {
if (browserOpened) return;
browserOpened = true;
if (!process.env.BRAINSTORM_OPEN) return; // opt-in: only after the user approves the companion
if (HOST !== '127.0.0.1' && HOST !== 'localhost') return;
if (clients.size > 0) return; // the user already opened it
const url = companionUrl(); // must carry the key or the gate 403s it
const cp = require('child_process');
// Operator-provided launcher: run as given (this env var is trusted operator input).
if (process.env.BRAINSTORM_OPEN_CMD) {
try { cp.exec(process.env.BRAINSTORM_OPEN_CMD + ' ' + JSON.stringify(url), () => {}); } catch (e) { /* best effort */ }
return;
}
// Platform launchers: pass the URL as an argv element via execFile (no shell),
// so a url-host containing shell metacharacters can't inject a command.
const launcher = browserLauncherForPlatform(url);
if (!launcher) return; // headless: nothing to open
try { cp.execFile(launcher.bin, launcher.args, () => {}); } catch (e) { /* best effort */ }
}
// ========== Activity Tracking ==========
const IDLE_TIMEOUT_MS = 30 * 60 * 1000; // 30 minutes
// Idle timeout: shut down after this long with no activity. Default 4 hours;
// override with BRAINSTORM_IDLE_TIMEOUT_MS (start-server.sh: --idle-timeout-minutes).
const IDLE_TIMEOUT_MS = (() => {
const ms = Number(process.env.BRAINSTORM_IDLE_TIMEOUT_MS);
return Number.isFinite(ms) && ms > 0 ? ms : 4 * 60 * 60 * 1000;
})();
// How often the watchdog checks for owner-death / idleness. Configurable mainly
// so tests can run fast; production default is 60s.
const LIFECYCLE_CHECK_MS = (() => {
const ms = Number(process.env.BRAINSTORM_LIFECYCLE_CHECK_MS);
return Number.isFinite(ms) && ms > 0 ? ms : 60 * 1000;
})();
let lastActivity = Date.now();
function touchActivity() {
@@ -267,14 +515,14 @@ function startServer() {
// macOS fs.watch reports 'rename' for both new files and overwrites,
// so we can't rely on eventType alone.
const knownFiles = new Set(
fs.readdirSync(CONTENT_DIR).filter(f => f.endsWith('.html'))
fs.readdirSync(CONTENT_DIR).filter(f => !f.startsWith('.') && f.endsWith('.html'))
);
const server = http.createServer(handleRequest);
server.on('upgrade', handleUpgrade);
const watcher = fs.watch(CONTENT_DIR, (eventType, filename) => {
if (!filename || !filename.endsWith('.html')) return;
if (!filename || filename.startsWith('.') || !filename.endsWith('.html')) return;
if (debounceTimers.has(filename)) clearTimeout(debounceTimers.get(filename));
debounceTimers.set(filename, setTimeout(() => {
@@ -289,6 +537,7 @@ function startServer() {
const eventsFile = path.join(STATE_DIR, 'events');
if (fs.existsSync(eventsFile)) fs.unlinkSync(eventsFile);
console.log(JSON.stringify({ type: 'screen-added', file: filePath }));
maybeOpenBrowser();
} else {
console.log(JSON.stringify({ type: 'screen-updated', file: filePath }));
}
@@ -308,6 +557,11 @@ function startServer() {
);
watcher.close();
clearInterval(lifecycleCheck);
// Close any upgraded WebSocket sockets so server.close() can complete and
// the process actually exits instead of lingering on an open connection.
for (const socket of clients) {
try { socket.destroy(); } catch (e) { /* already gone */ }
}
server.close(() => process.exit(0));
}
@@ -316,11 +570,11 @@ function startServer() {
try { process.kill(ownerPid, 0); return true; } catch (e) { return e.code === 'EPERM'; }
}
// Check every 60s: exit if owner process died or idle for 30 minutes
// Periodically exit if the owner process died or we've been idle too long.
const lifecycleCheck = setInterval(() => {
if (!ownerAlive()) shutdown('owner process exited');
else if (Date.now() - lastActivity > IDLE_TIMEOUT_MS) shutdown('idle timeout');
}, 60 * 1000);
}, LIFECYCLE_CHECK_MS);
lifecycleCheck.unref();
// Validate owner PID at startup. If it's already dead, the PID resolution
@@ -336,19 +590,68 @@ function startServer() {
}
}
server.listen(PORT, HOST, () => {
// If the preferred port is already taken (e.g. a previous server is still
// alive), fall back to a random port once instead of failing.
let triedFallback = false;
function onListen() {
// Cookie name keys on the ACTUAL bound port (may differ from the preferred
// one after an EADDRINUSE fallback) so it can't collide with another server's
// cookie in the shared localhost jar.
COOKIE_NAME = 'brainstorm-key-' + PORT;
// Record the bound port AND token so the next restart of this session reuses
// them — but ONLY when we got our preferred port. On a fallback we bound a
// *different* port because someone else holds the preferred one; persisting
// would overwrite the shared files and strand that other session's open tab.
if (PORT_FILE && !triedFallback) {
try { fs.writeFileSync(PORT_FILE, String(PORT)); } catch (e) { /* best effort */ }
if (TOKEN_FILE) {
try {
fs.writeFileSync(TOKEN_FILE, TOKEN, { mode: 0o600 });
chmodOwnerOnly(TOKEN_FILE);
} catch (e) { /* best effort */ }
}
}
const info = JSON.stringify({
type: 'server-started', port: Number(PORT), host: HOST,
url_host: URL_HOST, url: 'http://' + URL_HOST + ':' + PORT,
screen_dir: CONTENT_DIR, state_dir: STATE_DIR
url_host: URL_HOST, url: companionUrl(),
screen_dir: CONTENT_DIR, state_dir: STATE_DIR, idle_timeout_ms: IDLE_TIMEOUT_MS
});
console.log(info);
fs.writeFileSync(path.join(STATE_DIR, 'server-info'), info + '\n');
// server-info embeds the key — keep it owner-only.
fs.writeFileSync(path.join(STATE_DIR, 'server-info'), info + '\n', { mode: 0o600 });
}
server.on('error', (err) => {
if (err.code === 'EADDRINUSE' && !triedFallback) {
if (tokenSource === 'env') {
console.error('Server failed to bind: preferred port is in use and BRAINSTORM_TOKEN is set; refusing fallback with explicit token');
process.exit(1);
}
triedFallback = true;
PORT = randomPort();
if (tokenSource === 'file') {
TOKEN = generateToken();
tokenSource = 'generated-fallback';
}
server.listen(PORT, HOST, onListen);
} else {
console.error('Server failed to bind:', err.message);
process.exit(1);
}
});
server.listen(PORT, HOST, onListen);
}
if (require.main === module) {
startServer();
}
module.exports = { computeAcceptKey, encodeFrame, decodeFrame, OPCODES };
module.exports = {
computeAcceptKey,
encodeFrame,
decodeFrame,
browserLauncherForPlatform,
OPCODES,
MAX_FRAME_PAYLOAD_BYTES
};
+70 -9
View File
@@ -11,6 +11,9 @@
# --host <bind-host> Host/interface to bind (default: 127.0.0.1).
# Use 0.0.0.0 in remote/containerized environments.
# --url-host <host> Hostname shown in returned URL JSON.
# --idle-timeout-minutes <n> Shut down after n minutes idle (default 240 = 4h).
# --open Auto-open the browser on the first screen (use only
# after the user approves the visual companion).
# --foreground Run server in the current terminal (no backgrounding).
# --background Force background mode (overrides Codex auto-foreground).
@@ -22,6 +25,7 @@ FOREGROUND="false"
FORCE_BACKGROUND="false"
BIND_HOST="127.0.0.1"
URL_HOST=""
IDLE_TIMEOUT_MINUTES=""
while [[ $# -gt 0 ]]; do
case "$1" in
--project-dir)
@@ -36,6 +40,14 @@ while [[ $# -gt 0 ]]; do
URL_HOST="$2"
shift 2
;;
--idle-timeout-minutes)
IDLE_TIMEOUT_MINUTES="$2"
shift 2
;;
--open)
export BRAINSTORM_OPEN=1
shift
;;
--foreground|--no-daemon)
FOREGROUND="true"
shift
@@ -59,6 +71,29 @@ if [[ -z "$URL_HOST" ]]; then
fi
fi
if [[ -n "$IDLE_TIMEOUT_MINUTES" ]]; then
if ! [[ "$IDLE_TIMEOUT_MINUTES" =~ ^[0-9]+$ ]] || [[ "$IDLE_TIMEOUT_MINUTES" -lt 1 ]]; then
echo "{\"error\": \"--idle-timeout-minutes must be a positive integer\"}"
exit 1
fi
export BRAINSTORM_IDLE_TIMEOUT_MS=$(( IDLE_TIMEOUT_MINUTES * 60 * 1000 ))
fi
is_windows_like_shell() {
case "${OSTYPE:-}" in
msys*|cygwin*|mingw*) return 0 ;;
esac
if [[ -n "${MSYSTEM:-}" ]]; then
return 0
fi
local uname_s
uname_s="$(uname -s 2>/dev/null || true)"
case "$uname_s" in
MSYS*|MINGW*|CYGWIN*) return 0 ;;
esac
return 1
}
# Some environments reap detached/background processes. Auto-foreground when detected.
if [[ -n "${CODEX_CI:-}" && "$FOREGROUND" != "true" && "$FORCE_BACKGROUND" != "true" ]]; then
FOREGROUND="true"
@@ -66,19 +101,24 @@ fi
# Windows/Git Bash reaps nohup background processes. Auto-foreground when detected.
if [[ "$FOREGROUND" != "true" && "$FORCE_BACKGROUND" != "true" ]]; then
case "${OSTYPE:-}" in
msys*|cygwin*|mingw*) FOREGROUND="true" ;;
esac
if [[ -n "${MSYSTEM:-}" ]]; then
if is_windows_like_shell; then
FOREGROUND="true"
fi
fi
# Session files (server.log, server-info, .last-token) embed the session key —
# keep everything this script and the server create owner-only.
umask 077
# Generate unique session directory
SESSION_ID="$$-$(date +%s)"
if [[ -n "$PROJECT_DIR" ]]; then
SESSION_DIR="${PROJECT_DIR}/.superpowers/brainstorm/${SESSION_ID}"
# Persist the bound port and key per project so a restart reuses them and an
# already-open browser tab reconnects to the same URL with a valid cookie.
export BRAINSTORM_PORT_FILE="${PROJECT_DIR}/.superpowers/brainstorm/.last-port"
export BRAINSTORM_TOKEN_FILE="${PROJECT_DIR}/.superpowers/brainstorm/.last-token"
else
SESSION_DIR="/tmp/brainstorm-${SESSION_ID}"
fi
@@ -86,10 +126,21 @@ fi
STATE_DIR="${SESSION_DIR}/state"
PID_FILE="${STATE_DIR}/server.pid"
LOG_FILE="${STATE_DIR}/server.log"
SERVER_ID_FILE="${STATE_DIR}/server-instance-id"
# Create fresh session directory with content and state peers
mkdir -p "${SESSION_DIR}/content" "$STATE_DIR"
SERVER_ID=""
if [[ -r /dev/urandom ]]; then
SERVER_ID="$(od -An -N24 -tx1 /dev/urandom 2>/dev/null | tr -d ' \n' || true)"
fi
if ! [[ "$SERVER_ID" =~ ^[A-Za-z0-9_-]{32,64}$ ]]; then
SERVER_ID="$(printf '%08x%08x%08x%08x' "$$" "$(date +%s)" "${RANDOM:-0}" "${RANDOM:-0}")"
fi
printf '%s\n' "$SERVER_ID" > "$SERVER_ID_FILE"
chmod 600 "$SERVER_ID_FILE" 2>/dev/null || true
# Kill any existing server
if [[ -f "$PID_FILE" ]]; then
old_pid=$(cat "$PID_FILE")
@@ -97,7 +148,7 @@ if [[ -f "$PID_FILE" ]]; then
rm -f "$PID_FILE"
fi
cd "$SCRIPT_DIR"
cd "$SCRIPT_DIR" || exit 1
# Resolve the harness PID (grandparent of this script).
# $PPID is the ephemeral shell the harness spawned to run us — it dies
@@ -107,22 +158,32 @@ if [[ -z "$OWNER_PID" || "$OWNER_PID" == "1" ]]; then
OWNER_PID="$PPID"
fi
# Windows/MSYS2: Node.js cannot see POSIX PIDs from the MSYS2 namespace.
# Passing a PID node cannot verify causes server to log owner-pid-invalid
# and self-terminate at the 60-second lifecycle check. Clear it so the
# watchdog is disabled and the idle timeout becomes the only shutdown trigger.
if is_windows_like_shell; then
OWNER_PID=""
fi
# Foreground mode for environments that reap detached/background processes.
if [[ "$FOREGROUND" == "true" ]]; then
echo "$$" > "$PID_FILE"
env BRAINSTORM_DIR="$SESSION_DIR" BRAINSTORM_HOST="$BIND_HOST" BRAINSTORM_URL_HOST="$URL_HOST" BRAINSTORM_OWNER_PID="$OWNER_PID" node server.cjs
env BRAINSTORM_DIR="$SESSION_DIR" BRAINSTORM_HOST="$BIND_HOST" BRAINSTORM_URL_HOST="$URL_HOST" BRAINSTORM_OWNER_PID="$OWNER_PID" node server.cjs "--brainstorm-server-id=$SERVER_ID" &
SERVER_PID=$!
echo "$SERVER_PID" > "$PID_FILE"
wait "$SERVER_PID"
exit $?
fi
# Start server, capturing output to log file
# Use nohup to survive shell exit; disown to remove from job table
nohup env BRAINSTORM_DIR="$SESSION_DIR" BRAINSTORM_HOST="$BIND_HOST" BRAINSTORM_URL_HOST="$URL_HOST" BRAINSTORM_OWNER_PID="$OWNER_PID" node server.cjs > "$LOG_FILE" 2>&1 &
nohup env BRAINSTORM_DIR="$SESSION_DIR" BRAINSTORM_HOST="$BIND_HOST" BRAINSTORM_URL_HOST="$URL_HOST" BRAINSTORM_OWNER_PID="$OWNER_PID" node server.cjs "--brainstorm-server-id=$SERVER_ID" > "$LOG_FILE" 2>&1 &
SERVER_PID=$!
disown "$SERVER_PID" 2>/dev/null
echo "$SERVER_PID" > "$PID_FILE"
# Wait for server-started message (check log file)
for i in {1..50}; do
for _ in {1..50}; do
if grep -q "server-started" "$LOG_FILE" 2>/dev/null; then
# Verify server is still alive after a short window (catches process reapers)
alive="true"
+66 -2
View File
@@ -15,15 +15,78 @@ fi
STATE_DIR="${SESSION_DIR}/state"
PID_FILE="${STATE_DIR}/server.pid"
SERVER_ID_FILE="${STATE_DIR}/server-instance-id"
mark_stopped() {
local reason="$1"
rm -f "${STATE_DIR}/server-info"
printf '{"reason":"%s","timestamp":%s}\n' "$reason" "$(date +%s)" > "${STATE_DIR}/server-stopped"
}
read_expected_server_id() {
[[ -f "$SERVER_ID_FILE" ]] || return 1
local id
id="$(tr -d '\r\n' < "$SERVER_ID_FILE" 2>/dev/null || true)"
[[ "$id" =~ ^[A-Za-z0-9_-]{32,64}$ ]] || return 1
printf '%s\n' "$id"
}
command_line_for_pid() {
local pid="$1"
if [[ -r "/proc/$pid/cmdline" ]]; then
tr '\0' '\n' < "/proc/$pid/cmdline" 2>/dev/null || true
return 0
fi
ps -ww -p "$pid" -o command= 2>/dev/null || ps -f -p "$pid" 2>/dev/null | sed '1d' || true
}
command_has_server_id() {
local pid="$1"
local expected="$2"
local expected_arg="--brainstorm-server-id=$expected"
if [[ -r "/proc/$pid/cmdline" ]]; then
local arg
while IFS= read -r -d '' arg || [[ -n "$arg" ]]; do
[[ "$arg" == "$expected_arg" ]] && return 0
done < "/proc/$pid/cmdline"
return 1
fi
local command_line
command_line="$(command_line_for_pid "$pid")"
[[ -n "$command_line" ]] || return 1
case " $command_line " in
*" $expected_arg "*) return 0 ;;
*) return 1 ;;
esac
}
# Confirm a PID has this session's per-start instance id, not just a familiar
# process name. Ambiguous or legacy metadata fails closed as stale_pid.
is_brainstorm_server() {
kill -0 "$1" 2>/dev/null || return 1
local expected_id
expected_id="$(read_expected_server_id)" || return 1
command_has_server_id "$1" "$expected_id" || return 1
return 0
}
if [[ -f "$PID_FILE" ]]; then
pid=$(cat "$PID_FILE")
# Refuse to signal a PID we can't prove is our server. A stale pid file may
# point at an unrelated process after a reboot/PID wraparound.
if ! is_brainstorm_server "$pid"; then
rm -f "$PID_FILE" "$SERVER_ID_FILE"
mark_stopped "stale_pid"
echo '{"status": "stale_pid"}'
exit 0
fi
# Try to stop gracefully, fallback to force if still alive
kill "$pid" 2>/dev/null || true
# Wait for graceful shutdown (up to ~2s)
for i in {1..20}; do
for _ in {1..20}; do
if ! kill -0 "$pid" 2>/dev/null; then
break
fi
@@ -43,7 +106,8 @@ if [[ -f "$PID_FILE" ]]; then
exit 1
fi
rm -f "$PID_FILE" "${STATE_DIR}/server.log"
rm -f "$PID_FILE" "$SERVER_ID_FILE" "${STATE_DIR}/server.log"
mark_stopped "stop-server.sh"
# Only delete ephemeral /tmp directories
if [[ "$SESSION_DIR" == /tmp/* ]]; then
+19 -9
View File
@@ -33,15 +33,25 @@ The server watches a directory for HTML files and serves the newest one to the b
## Starting a Session
```bash
# Start server with persistence (mockups saved to project)
scripts/start-server.sh --project-dir /path/to/project
# Start AFTER the user approves the companion. --open auto-opens their browser on
# the first screen; --project-dir persists mockups and enables same-port restart.
scripts/start-server.sh --project-dir /path/to/project --open
# Returns: {"type":"server-started","port":52341,"url":"http://localhost:52341",
# Returns: {"type":"server-started","port":52341,
# "url":"http://localhost:52341/?key=ab12…",
# "screen_dir":"/path/to/project/.superpowers/brainstorm/12345-1706000000/content",
# "state_dir":"/path/to/project/.superpowers/brainstorm/12345-1706000000/state"}
```
Save `screen_dir` and `state_dir` from the response. Tell user to open the URL.
Save `screen_dir` and `state_dir` from the response. With `--open`, the browser opens itself when you push the first screen — you don't need to ask the user to open it, but still share the URL as a fallback (headless/remote setups won't auto-open).
**The URL contains a session key (`?key=…`).** The server rejects any request
without it, so always give the user the **complete** URL from the `url` field —
never strip the query string, and never hand out a bare `http://host:port`. The
key gates HTTP and WebSocket access so a stray browser tab or another machine on
the network can't read the screens or inject events. After the first load the
browser remembers the key via a cookie, so reloads and `/files/*` assets work
without repeating it.
**Finding connection info:** The server writes its startup JSON to `$STATE_DIR/server-info`. If you launched the server in the background and didn't capture stdout, read that file to get the URL and port. When using `--project-dir`, check `<project>/.superpowers/brainstorm/` for the session directory.
@@ -52,7 +62,7 @@ Save `screen_dir` and `state_dir` from the response. Tell user to open the URL.
**Claude Code:**
```bash
# Default mode works — the script backgrounds the server itself.
scripts/start-server.sh --project-dir /path/to/project
scripts/start-server.sh --project-dir /path/to/project --open
```
On Windows, the script auto-detects and switches to foreground mode (which blocks the tool call). Use `run_in_background: true` on the Bash tool call so the server survives across conversation turns, then read `$STATE_DIR/server-info` on the next turn to get the URL and port.
@@ -61,14 +71,14 @@ On Windows, the script auto-detects and switches to foreground mode (which block
```bash
# Codex reaps background processes. The script auto-detects CODEX_CI and
# switches to foreground mode. Run it normally — no extra flags needed.
scripts/start-server.sh --project-dir /path/to/project
scripts/start-server.sh --project-dir /path/to/project --open
```
**Gemini CLI:**
```bash
# Use --foreground and set is_background: true on your shell tool call
# so the process survives across turns
scripts/start-server.sh --project-dir /path/to/project --foreground
scripts/start-server.sh --project-dir /path/to/project --open --foreground
```
**Copilot CLI:**
@@ -76,7 +86,7 @@ scripts/start-server.sh --project-dir /path/to/project --foreground
# Use --foreground and start the server via the bash tool with mode: "async"
# so the process survives across turns. Capture the returned shellId for
# read_bash / stop_bash if you need to interact with it later.
scripts/start-server.sh --project-dir /path/to/project --foreground
scripts/start-server.sh --project-dir /path/to/project --open --foreground
```
**Other environments:** The server must keep running in the background across conversation turns. If your environment reaps detached processes, use `--foreground` and launch the command with your platform's background execution mechanism.
@@ -95,7 +105,7 @@ Use `--url-host` to control what hostname is printed in the returned URL JSON.
## The Loop
1. **Check server is alive**, then **write HTML** to a new file in `screen_dir`:
- Before each write, check that `$STATE_DIR/server-info` exists. If it doesn't (or `$STATE_DIR/server-stopped` exists), the server has shut down restart it with `start-server.sh` before continuing. The server auto-exits after 30 minutes of inactivity.
- **Required: confirm the server is alive before referring to the URL or pushing a screen.** Check that `$STATE_DIR/server-info` exists and `$STATE_DIR/server-stopped` does not. If it has shut down, restart it with `start-server.sh` using the **same `--project-dir`** — it reuses the same port, so the user's open tab reconnects on its own (it shows a "paused" overlay while the server is down) and you don't need to send a new URL. The server auto-exits after 4 hours idle (configurable with `--idle-timeout-minutes`).
- Use semantic filenames: `platform.html`, `visual-style.html`, `layout.html`
- **Never reuse filenames** — each screen gets a fresh file
- Use your file-creation tool — **never use cat/heredoc** (dumps noise into terminal)
+176 -65
View File
@@ -5,11 +5,11 @@ description: Use when executing implementation plans with independent tasks in t
# Subagent-Driven Development
Execute plan by dispatching fresh subagent per task, with two-stage review after each: spec compliance review first, then code quality review.
Execute plan by dispatching a fresh implementer subagent per task, a task review (spec compliance + code quality) after each, and a broad whole-branch review at the end.
**Why subagents:** You delegate tasks to specialized agents with isolated context. By precisely crafting their instructions and context, you ensure they stay focused and succeed at their task. They should never inherit your session's context or history — you construct exactly what they need. This also preserves your own context for coordination work.
**Core principle:** Fresh subagent per task + two-stage review (spec then quality) = high quality, fast iteration
**Core principle:** Fresh subagent per task + task review (spec + quality) + broad final review = high quality, fast iteration
**Continuous execution:** Do not pause to check in with your human partner between tasks. Execute all tasks from the plan without stopping. The only reasons to stop are: BLOCKED status you cannot resolve, ambiguity that genuinely prevents progress, or all tasks complete. "Should I continue?" prompts and progress summaries waste their time — they asked you to execute the plan, so execute it.
@@ -36,7 +36,7 @@ digraph when_to_use {
**vs. Executing Plans (parallel session):**
- Same session (no context switch)
- Fresh subagent per task (no context pollution)
- Two-stage review after each task: spec compliance first, then code quality
- Review after each task (spec compliance + code quality), broad review at the end
- Faster iteration (no human-in-loop between tasks)
## The Process
@@ -51,38 +51,31 @@ digraph process {
"Implementer subagent asks questions?" [shape=diamond];
"Answer questions, provide context" [shape=box];
"Implementer subagent implements, tests, commits, self-reviews" [shape=box];
"Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)" [shape=box];
"Spec reviewer subagent confirms code matches spec?" [shape=diamond];
"Implementer subagent fixes spec gaps" [shape=box];
"Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" [shape=box];
"Code quality reviewer subagent approves?" [shape=diamond];
"Implementer subagent fixes quality issues" [shape=box];
"Mark task complete in todo list" [shape=box];
"Write diff file, dispatch task reviewer subagent (./task-reviewer-prompt.md)" [shape=box];
"Task reviewer reports spec ✅ and quality approved?" [shape=diamond];
"Dispatch fix subagent for Critical/Important findings" [shape=box];
"Mark task complete in todo list and progress ledger" [shape=box];
}
"Read plan, extract all tasks with full text, note context, create todos" [shape=box];
"Read plan, note context and global constraints, create todos" [shape=box];
"More tasks remain?" [shape=diamond];
"Dispatch final code reviewer subagent for entire implementation" [shape=box];
"Dispatch final code reviewer subagent (../requesting-code-review/code-reviewer.md)" [shape=box];
"Use superpowers:finishing-a-development-branch" [shape=box style=filled fillcolor=lightgreen];
"Read plan, extract all tasks with full text, note context, create todos" -> "Dispatch implementer subagent (./implementer-prompt.md)";
"Read plan, note context and global constraints, create todos" -> "Dispatch implementer subagent (./implementer-prompt.md)";
"Dispatch implementer subagent (./implementer-prompt.md)" -> "Implementer subagent asks questions?";
"Implementer subagent asks questions?" -> "Answer questions, provide context" [label="yes"];
"Answer questions, provide context" -> "Dispatch implementer subagent (./implementer-prompt.md)";
"Implementer subagent asks questions?" -> "Implementer subagent implements, tests, commits, self-reviews" [label="no"];
"Implementer subagent implements, tests, commits, self-reviews" -> "Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)";
"Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)" -> "Spec reviewer subagent confirms code matches spec?";
"Spec reviewer subagent confirms code matches spec?" -> "Implementer subagent fixes spec gaps" [label="no"];
"Implementer subagent fixes spec gaps" -> "Dispatch spec reviewer subagent (./spec-reviewer-prompt.md)" [label="re-review"];
"Spec reviewer subagent confirms code matches spec?" -> "Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" [label="yes"];
"Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" -> "Code quality reviewer subagent approves?";
"Code quality reviewer subagent approves?" -> "Implementer subagent fixes quality issues" [label="no"];
"Implementer subagent fixes quality issues" -> "Dispatch code quality reviewer subagent (./code-quality-reviewer-prompt.md)" [label="re-review"];
"Code quality reviewer subagent approves?" -> "Mark task complete in todo list" [label="yes"];
"Mark task complete in todo list" -> "More tasks remain?";
"Implementer subagent implements, tests, commits, self-reviews" -> "Write diff file, dispatch task reviewer subagent (./task-reviewer-prompt.md)";
"Write diff file, dispatch task reviewer subagent (./task-reviewer-prompt.md)" -> "Task reviewer reports spec ✅ and quality approved?";
"Task reviewer reports spec ✅ and quality approved?" -> "Dispatch fix subagent for Critical/Important findings" [label="no"];
"Dispatch fix subagent for Critical/Important findings" -> "Write diff file, dispatch task reviewer subagent (./task-reviewer-prompt.md)" [label="re-review"];
"Task reviewer reports spec ✅ and quality approved?" -> "Mark task complete in todo list and progress ledger" [label="yes"];
"Mark task complete in todo list and progress ledger" -> "More tasks remain?";
"More tasks remain?" -> "Dispatch implementer subagent (./implementer-prompt.md)" [label="yes"];
"More tasks remain?" -> "Dispatch final code reviewer subagent for entire implementation" [label="no"];
"Dispatch final code reviewer subagent for entire implementation" -> "Use superpowers:finishing-a-development-branch";
"More tasks remain?" -> "Dispatch final code reviewer subagent (../requesting-code-review/code-reviewer.md)" [label="no"];
"Dispatch final code reviewer subagent (../requesting-code-review/code-reviewer.md)" -> "Use superpowers:finishing-a-development-branch";
}
```
@@ -94,9 +87,23 @@ Use the least powerful model that can handle each role to conserve cost and incr
**Integration and judgment tasks** (multi-file coordination, pattern matching, debugging): use a standard model.
**Architecture, design, and review tasks**: use the most capable available model.
**Architecture and design tasks**: use the most capable available model.
**Task complexity signals:**
**Review tasks**: choose the model with the same judgment, scaled to the
diff's size, complexity, and risk. A small mechanical diff does not need the
most capable model; a subtle concurrency change does.
**Always specify the model explicitly when dispatching a subagent.** An
omitted model inherits your session's model — often the most capable and
most expensive — which silently defeats this section.
**Turn count beats token price.** Wall-clock and context cost scale with how
many turns a subagent takes, and the cheapest models routinely take 2-3× the
turns on multi-step work — costing more overall. Use a mid-tier model as the
floor for implementers and reviewers; reserve the cheapest tier for
single-file mechanical fixes.
**Task complexity signals (implementation tasks):**
- Touches 1-2 files with a complete spec → cheap model
- Touches multiple files with integration concerns → standard model
- Requires design judgment or broad codebase understanding → most capable model
@@ -105,7 +112,7 @@ Use the least powerful model that can handle each role to conserve cost and incr
Implementer subagents report one of four statuses. Handle each appropriately:
**DONE:** Proceed to spec compliance review.
**DONE:** Generate the review package (`scripts/review-package BASE HEAD`, from this skill's directory — it prints the unique file path it wrote; BASE is the commit you recorded before dispatching the implementer — never `HEAD~1`, which silently drops all but the last commit of a multi-commit task), then dispatch the task reviewer with the printed path.
**DONE_WITH_CONCERNS:** The implementer completed the work but flagged doubts. Read the concerns before proceeding. If the concerns are about correctness or scope, address them before review. If they're observations (e.g., "this file is getting large"), note them and proceed to review.
@@ -119,11 +126,120 @@ Implementer subagents report one of four statuses. Handle each appropriately:
**Never** ignore an escalation or force the same model to retry without changes. If the implementer said it's stuck, something needs to change.
## Handling Reviewer ⚠️ Items
The task reviewer may report "⚠️ Cannot verify from diff" items — requirements
that live in unchanged code or span tasks. These do not block the rest of the
review, but you must resolve each one yourself before marking the task
complete: you hold the plan and cross-task context the reviewer
lacks. If you confirm an item is a real gap, treat it as a failed spec
review — send it back to the implementer and re-review.
## Constructing Reviewer Prompts
Per-task reviews are task-scoped gates. The broad review happens once, at the
final whole-branch review. When you fill a reviewer template:
- Do not add open-ended directives like "check all uses" or "run race tests
if useful" without a concrete, task-specific reason
- Do not ask a reviewer to re-run tests the implementer already ran on the
same code — the implementer's report carries the test evidence
- Do not pre-judge findings for the reviewer — never instruct a reviewer to
ignore or not flag a specific issue. If you believe a finding would be a
false positive, let the reviewer raise it and adjudicate it in the review
loop. If the prompt you are writing contains "do not flag," "don't treat X
as a defect," "at most Minor," or "the plan chose" — stop: you are
pre-judging, usually to spare yourself a review loop.
- The global-constraints block you hand the reviewer is its attention
lens. Copy the binding requirements verbatim from the plan's Global
Constraints section or the spec: exact values, exact formats, and the
stated relationships between components ("same layout as X", "matches
Y"). The reviewer's template already carries the process rules (YAGNI,
test hygiene, review method) — the constraints block is for what THIS
project's spec demands.
- Hand the reviewer its diff as a file: run this skill's
`scripts/review-package BASE HEAD` and pass the reviewer the file path
it prints (or, without bash: `git log --oneline`, `git diff --stat`,
and `git diff -U10` for the range, redirected to one uniquely named
file). The output never enters your own context, and the reviewer sees
the commit list, stat summary, and full diff with context in one Read
call. Use the BASE you recorded before dispatching the implementer —
never `HEAD~1`, which silently truncates multi-commit tasks.
- A dispatch prompt describes one task, not the session's history. Do not
paste accumulated prior-task summaries ("state after Tasks 1-3") into
later dispatches — a real session's dispatch hit 42k chars of which 99%
was pasted history. A fresh subagent needs its task, the interfaces it
touches, and the global constraints. Nothing else.
- Dispatch fix subagents for Critical and Important findings. Record Minor
findings in the progress ledger as you go, and point the final
whole-branch review at that list so it can triage which must be fixed
before merge. A roll-up nobody reads is a silent discard.
- The final whole-branch review gets a package too: run
`scripts/review-package MERGE_BASE HEAD` (MERGE_BASE = the commit the
branch started from, e.g. `git merge-base main HEAD`) and include the
printed path in the final review dispatch, so the final reviewer reads
one file instead of re-deriving the branch diff with git commands.
- Every fix dispatch carries the implementer contract: the fix subagent
re-runs the tests covering its change and reports the results. Name the
covering test files in the dispatch — a one-line fix does not need the
whole suite. Before re-dispatching the reviewer, confirm the fix report
contains the covering tests, the command run, and the output; dispatch
the re-review once all three are present.
- If the final whole-branch review returns findings, dispatch ONE fix
subagent with the complete findings list — not one fixer per finding.
Per-finding fixers each rebuild context and re-run suites; a real
session's final-review fix wave cost more than all its tasks combined.
## File Handoffs
Everything you paste into a dispatch prompt — and everything a subagent
prints back — stays resident in your context for the rest of the session
and is re-read on every later turn. Hand artifacts over as files:
- **Task brief:** before dispatching an implementer, run this skill's
`scripts/task-brief PLAN_FILE N` — it extracts the task's full text to a
uniquely named file and prints the path. Compose the dispatch so the
brief stays the single source of requirements. Your dispatch should
contain: (1) one line on where this task fits in the project; (2) the
brief path, introduced as "read this first — it is your requirements,
with the exact values to use verbatim"; (3) interfaces and decisions
from earlier tasks that the brief cannot know; (4) your resolution of
any ambiguity you noticed in the brief; (5) the report-file path and
report contract. Exact values (numbers, magic strings, signatures, test
cases) appear only in the brief.
- **Report file:** name the implementer's report file after the brief
(brief `…/task-N-brief.md` → report `…/task-N-report.md`) and put it in
the dispatch prompt. The implementer writes the full report there and
returns only status, commits, a one-line test summary, and concerns.
- **Reviewer inputs:** the task reviewer gets three paths — the same brief
file, the report file, and the review package — plus the global
constraints that bind the task.
- Fix dispatches append their fix report (with test results) to the same
report file and return a short summary; re-reviews read the updated file.
## Durable Progress
Conversation memory does not survive compaction. In real sessions,
controllers that lost their place have re-dispatched entire completed task
sequences — the single most expensive failure observed. Track progress in
a ledger file, not only in todos.
- At skill start, check for a ledger:
`cat "$(git rev-parse --git-path sdd)/progress.md"`. Tasks listed there
as complete are DONE — do not re-dispatch them; resume at the first task
not marked complete.
- When a task's review comes back clean, append one line to the ledger in
the same message as your other bookkeeping:
`Task N: complete (commits <base7>..<head7>, review clean)`.
- The ledger is your recovery map: the commits it names exist in git even
when your context no longer remembers creating them. After compaction,
trust the ledger and `git log` over your own recollection.
## Prompt Templates
- [implementer-prompt.md](implementer-prompt.md) - Dispatch implementer subagent
- [spec-reviewer-prompt.md](spec-reviewer-prompt.md) - Dispatch spec compliance reviewer subagent
- [code-quality-reviewer-prompt.md](code-quality-reviewer-prompt.md) - Dispatch code quality reviewer subagent
- [task-reviewer-prompt.md](task-reviewer-prompt.md) - Dispatch task reviewer subagent (spec compliance + code quality)
- Final whole-branch review: use superpowers:requesting-code-review's [code-reviewer.md](../requesting-code-review/code-reviewer.md)
## Example Workflow
@@ -131,13 +247,11 @@ Implementer subagents report one of four statuses. Handle each appropriately:
You: I'm using Subagent-Driven Development to execute this plan.
[Read plan file once: docs/superpowers/plans/feature-plan.md]
[Extract all 5 tasks with full text and context]
[Create todos for all tasks]
Task 1: Hook installation script
[Get Task 1 text and context (already extracted)]
[Dispatch implementation subagent with full task text + context]
[Run task-brief for Task 1; dispatch implementer with brief + report paths + context]
Implementer: "Before I begin - should the hook be installed at user or system level?"
@@ -150,18 +264,15 @@ Implementer: "Got it. Implementing now..."
- Self-review: Found I missed --force flag, added it
- Committed
[Dispatch spec compliance reviewer]
Spec reviewer: Spec compliant - all requirements met, nothing extra
[Get git SHAs, dispatch code quality reviewer]
Code reviewer: Strengths: Good test coverage, clean. Issues: None. Approved.
[Run review-package, dispatch task reviewer with the printed path]
Task reviewer: Spec - all requirements met, nothing extra.
Strengths: Good test coverage, clean. Issues: None. Task quality: Approved.
[Mark Task 1 complete]
Task 2: Recovery modes
[Get Task 2 text and context (already extracted)]
[Dispatch implementation subagent with full task text + context]
[Run task-brief for Task 2; dispatch implementer with brief + report paths + context]
Implementer: [No questions, proceeds]
Implementer:
@@ -170,25 +281,17 @@ Implementer:
- Self-review: All good
- Committed
[Dispatch spec compliance reviewer]
Spec reviewer: ❌ Issues:
[Run review-package, dispatch task reviewer with the printed path]
Task reviewer: Spec ❌:
- Missing: Progress reporting (spec says "report every 100 items")
- Extra: Added --json flag (not requested)
Issues (Important): Magic number (100)
[Implementer fixes issues]
Implementer: Removed --json flag, added progress reporting
[Dispatch fix subagent with all findings]
Fixer: Removed --json flag, added progress reporting, extracted PROGRESS_INTERVAL constant
[Spec reviewer reviews again]
Spec reviewer: Spec compliant now
[Dispatch code quality reviewer]
Code reviewer: Strengths: Solid. Issues (Important): Magic number (100)
[Implementer fixes]
Implementer: Extracted PROGRESS_INTERVAL constant
[Code reviewer reviews again]
Code reviewer: ✅ Approved
[Task reviewer reviews again]
Task reviewer: Spec ✅. Task quality: Approved.
[Mark Task 2 complete]
@@ -215,20 +318,20 @@ Done!
- Review checkpoints automatic
**Efficiency gains:**
- No file reading overhead (controller provides full text)
- Controller curates exactly what context is needed
- Controller curates exactly what context is needed; bulk artifacts move
as files, not pasted text
- Subagent gets complete information upfront
- Questions surfaced before work begins (not after)
**Quality gates:**
- Self-review catches issues before handoff
- Two-stage review: spec compliance, then code quality
- Task review carries two verdicts: spec compliance and code quality
- Review loops ensure fixes actually work
- Spec compliance prevents over/under-building
- Code quality ensures implementation is well-built
**Cost:**
- More subagent invocations (implementer + 2 reviewers per task)
- More subagent invocations (implementer + reviewer per task)
- Controller does more prep work (extracting all tasks upfront)
- Review loops add iterations
- But catches issues early (cheaper than debugging later)
@@ -237,17 +340,25 @@ Done!
**Never:**
- Start implementation on main/master branch without explicit user consent
- Skip reviews (spec compliance OR code quality)
- Skip task review, or accept a report missing either verdict (spec compliance AND task quality are both required)
- Proceed with unfixed issues
- Dispatch multiple implementation subagents in parallel (conflicts)
- Make subagent read plan file (provide full text instead)
- Make a subagent read the whole plan file (hand it its task brief —
`scripts/task-brief` — instead)
- Skip scene-setting context (subagent needs to understand where task fits)
- Ignore subagent questions (answer before letting them proceed)
- Accept "close enough" on spec compliance (spec reviewer found issues = not done)
- Accept "close enough" on spec compliance (reviewer found spec issues = not done)
- Skip review loops (reviewer found issues = implementer fixes = review again)
- Let implementer self-review replace actual review (both are needed)
- **Start code quality review before spec compliance is ✅** (wrong order)
- Move to next task while either review has open issues
- Tell a reviewer what not to flag, or pre-rate a finding's severity in the
dispatch prompt ("treat it as Minor at most") — the plan's example code is
a starting point, not evidence that its weaknesses were chosen
- Dispatch a task reviewer without a diff file — generate it first
(`scripts/review-package BASE HEAD`) and name the printed path in the
prompt
- Move to next task while the review has open Critical/Important issues
- Re-dispatch a task the progress ledger already marks complete — check
the ledger (and `git log`) after any compaction or resume
**If subagent asks questions:**
- Answer clearly and completely
@@ -269,7 +380,7 @@ Done!
**Required workflow skills:**
- **superpowers:using-git-worktrees** - Ensures isolated workspace (creates one or verifies existing)
- **superpowers:writing-plans** - Creates the plan this skill executes
- **superpowers:requesting-code-review** - Code review template for reviewer subagents
- **superpowers:requesting-code-review** - Code review template for the final whole-branch review
- **superpowers:finishing-a-development-branch** - Complete development after all tasks
**Subagents should use:**
@@ -1,25 +0,0 @@
# Code Quality Reviewer Prompt Template
Use this template when dispatching a code quality reviewer subagent.
**Purpose:** Verify implementation is well-built (clean, tested, maintainable)
**Only dispatch after spec compliance review passes.**
```
Subagent (general-purpose):
Use template at ../requesting-code-review/code-reviewer.md
DESCRIPTION: [task summary, from implementer's report]
PLAN_OR_REQUIREMENTS: Task N from [plan-file]
BASE_SHA: [commit before task]
HEAD_SHA: [current commit]
```
**In addition to standard code quality concerns, the reviewer should check:**
- Does each file have one clear responsibility with a well-defined interface?
- Are units decomposed so they can be understood and tested independently?
- Is the implementation following the file structure from the plan?
- Did this implementation create new files that are already large, or significantly grow existing files? (Don't flag pre-existing file sizes — focus on what this change contributed.)
**Code reviewer returns:** Strengths, Issues (Critical/Important/Minor), Assessment
@@ -5,12 +5,15 @@ Use this template when dispatching an implementer subagent.
```
Subagent (general-purpose):
description: "Implement Task N: [task name]"
model: [MODEL — REQUIRED: choose per SKILL.md Model Selection; an omitted
model silently inherits the session's most expensive one]
prompt: |
You are implementing Task N: [task name]
## Task Description
[FULL TEXT of task from plan - paste it here, don't make subagent read file]
Read your task brief first: [BRIEF_FILE]
It contains the full task text from the plan.
## Context
@@ -41,6 +44,9 @@ Subagent (general-purpose):
**While you work:** If you encounter something unexpected or unclear, **ask questions**.
It's always OK to pause and clarify. Don't guess or make assumptions.
While iterating, run the focused test for what you're changing; run the
full suite once before committing, not after every edit.
## Code Organization
You reason best about code you can hold in context at once, and your edits are more
@@ -94,19 +100,39 @@ Subagent (general-purpose):
- Do tests actually verify behavior (not just mock behavior)?
- Did I follow TDD if required?
- Are tests comprehensive?
- Is the test output pristine (no stray warnings or noise)?
If you find issues during self-review, fix them now before reporting.
## After Review Findings
If a reviewer finds issues and you fix them, re-run the tests that cover
the amended code and append the results to your report file. Reviewers
will not re-run tests for you — your report is the test evidence.
## Report Format
When done, report:
- **Status:** DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT
Write your full report to [REPORT_FILE]:
- What you implemented (or what you attempted, if blocked)
- What you tested and test results
- **TDD Evidence** (if TDD was required for this task):
- RED: command run, relevant failing output before implementation, and why the failure was expected
- GREEN: command run and relevant passing output after implementation
- Files changed
- Self-review findings (if any)
- Any issues or concerns
Then report back with ONLY (under 15 lines — the detail lives in the
report file):
- **Status:** DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT
- Commits created (short SHA + subject)
- One-line test summary (e.g. "14/14 passing, output pristine")
- Your concerns, if any
- The report file path
If BLOCKED or NEEDS_CONTEXT, put the specifics in the final message
itself — the controller acts on it directly.
Use DONE_WITH_CONCERNS if you completed the work but have doubts about correctness.
Use BLOCKED if you cannot complete the task. Use NEEDS_CONTEXT if you need
information that wasn't provided. Never silently produce work you're unsure about.
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
# Generate a review package: commit list, stat summary, and the net
# diff with extended context, written to a file the reviewer reads in one
# call. Using the recorded per-task BASE (not HEAD~1) keeps multi-commit
# tasks intact.
#
# Usage: review-package BASE HEAD [OUTFILE]
# Default OUTFILE: <git-dir>/sdd/review-<base7>..<head7>.diff — unique per
# repo instance and per range, so concurrent sessions cannot collide and a
# re-review after fixes always gets a distinctly named fresh file.
set -euo pipefail
if [ $# -lt 2 ] || [ $# -gt 3 ]; then
echo "usage: review-package BASE HEAD [OUTFILE]" >&2
exit 2
fi
base=$1
head=$2
git rev-parse --verify --quiet "$base" >/dev/null || { echo "bad BASE: $base" >&2; exit 2; }
git rev-parse --verify --quiet "$head" >/dev/null || { echo "bad HEAD: $head" >&2; exit 2; }
if [ $# -eq 3 ]; then
out=$3
else
dir=$(git rev-parse --git-path sdd)
mkdir -p "$dir"
dir=$(cd "$dir" && pwd)
out="$dir/review-$(git rev-parse --short "$base")..$(git rev-parse --short "$head").diff"
fi
{
echo "# Review package: ${base}..${head}"
echo
echo "## Commits"
git log --oneline "${base}..${head}"
echo
echo "## Files changed"
git diff --stat "${base}..${head}"
echo
echo "## Diff"
git diff -U10 "${base}..${head}"
} > "$out"
commits=$(git rev-list --count "${base}..${head}")
echo "wrote ${out}: ${commits} commit(s), $(wc -c < "$out" | tr -d ' ') bytes"
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# Extract one task's full text from an implementation plan into a file the
# implementer reads in one call, so the task text never has to be pasted
# through the controller's context.
#
# Usage: task-brief PLAN_FILE TASK_NUMBER [OUTFILE]
# Default OUTFILE: <git-dir>/sdd/task-<N>-brief.md — unique per repo
# instance, so concurrent sessions cannot collide.
set -euo pipefail
if [ $# -lt 2 ] || [ $# -gt 3 ]; then
echo "usage: task-brief PLAN_FILE TASK_NUMBER [OUTFILE]" >&2
exit 2
fi
plan=$1
n=$2
[ -f "$plan" ] || { echo "no such plan file: $plan" >&2; exit 2; }
if [ $# -eq 3 ]; then
out=$3
else
dir=$(git rev-parse --git-path sdd)
mkdir -p "$dir"
dir=$(cd "$dir" && pwd)
out="$dir/task-${n}-brief.md"
fi
awk -v n="$n" '
/^```/ { infence = !infence }
!infence && /^#+[ \t]+Task[ \t]+[0-9]+/ {
intask = ($0 ~ ("^#+[ \t]+Task[ \t]+" n "([^0-9]|$)"))
}
intask { print }
' "$plan" > "$out"
if [ ! -s "$out" ]; then
echo "task ${n} not found in ${plan} (no heading matching 'Task ${n}')" >&2
exit 3
fi
echo "wrote ${out}: $(wc -l < "$out" | tr -d ' ') lines"
@@ -1,77 +0,0 @@
# Spec Compliance Reviewer Prompt Template
Use this template when dispatching a spec compliance reviewer subagent.
**Purpose:** Verify implementer built what was requested (nothing more, nothing less)
```
Subagent (general-purpose):
description: "Review spec compliance for Task N"
prompt: |
You are reviewing whether an implementation matches its specification.
## What Was Requested
[FULL TEXT of task requirements]
## What Implementer Claims They Built
[From implementer's report]
## Git Range to Review
**Base:** [BASE_SHA — commit before this task]
**Head:** [HEAD_SHA — current commit]
```bash
git diff --stat [BASE_SHA]..[HEAD_SHA]
git diff [BASE_SHA]..[HEAD_SHA]
```
Only read files in this diff. Do not crawl the broader codebase.
## Read-Only Review
Your review is read-only on this checkout. Do not mutate the working tree, the index, HEAD, or branch state in any way. Use tools like `git show`, `git diff`, and `git log` to inspect history. If you need a working copy of a different revision, check it out into a separate temporary directory (e.g. `git worktree add /tmp/review-[SHA] [SHA]`) — never move HEAD on this checkout.
## CRITICAL: Do Not Trust the Report
The implementer finished suspiciously quickly. Their report may be incomplete,
inaccurate, or optimistic. You MUST verify everything independently.
**DO NOT:**
- Take their word for what they implemented
- Trust their claims about completeness
- Accept their interpretation of requirements
**DO:**
- Read the actual code they wrote
- Compare actual implementation to requirements line by line
- Check for missing pieces they claimed to implement
- Look for extra features they didn't mention
## Your Job
Read the implementation code and verify:
**Missing requirements:**
- Did they implement everything that was requested?
- Are there requirements they skipped or missed?
- Did they claim something works but didn't actually implement it?
**Extra/unneeded work:**
- Did they build things that weren't requested?
- Did they over-engineer or add unnecessary features?
- Did they add "nice to haves" that weren't in spec?
**Misunderstandings:**
- Did they interpret requirements differently than intended?
- Did they solve the wrong problem?
- Did they implement the right feature but wrong way?
**Verify by reading code, not by trusting report.**
Report:
- ✅ Spec compliant (if everything matches after code inspection)
- ❌ Issues found: [list specifically what's missing or extra, with file:line references]
```
@@ -0,0 +1,178 @@
# Task Reviewer Prompt Template
Use this template when dispatching a task reviewer subagent. The reviewer
reads the task's diff once and returns two verdicts: spec compliance and
code quality.
**Purpose:** Verify one task's implementation matches its requirements (nothing
more, nothing less) and is well-built (clean, tested, maintainable)
```
Subagent (general-purpose):
description: "Review Task N (spec + quality)"
model: [MODEL — REQUIRED: choose per SKILL.md Model Selection; an omitted
model silently inherits the session's most expensive one]
prompt: |
You are reviewing one task's implementation: first whether it matches its
requirements, then whether it is well-built. This is a task-scoped gate,
not a merge review — a broad whole-branch review happens separately after
all tasks are complete.
## What Was Requested
Read the task brief: [BRIEF_FILE]
Global constraints from the spec/design that bind this task:
[GLOBAL_CONSTRAINTS]
## What the Implementer Claims They Built
Read the implementer's report: [REPORT_FILE]
## Diff Under Review
**Base:** [BASE_SHA]
**Head:** [HEAD_SHA]
**Diff file:** [DIFF_FILE]
Read the diff file once — it contains the commit list, a stat summary,
and the full diff with surrounding context, and it is your view of the
change. The diff's context lines ARE the changed files: do not Read a
changed file separately unless a hunk you must judge is cut off
mid-function — and say so in your report. Do not re-run git commands.
If the diff file is missing, fetch the diff yourself:
`git diff --stat [BASE_SHA]..[HEAD_SHA]` and `git diff [BASE_SHA]..[HEAD_SHA]`.
Do not crawl the broader codebase. Inspect code outside the diff only
to evaluate a concrete risk you can name — one focused check per named
risk, and name both the risk and what you checked in your report.
Cross-cutting changes are legitimate named risks: if the diff changes
lock ordering, a function or API contract, or shared mutable state,
checking the call sites is the right method.
Your review is read-only on this checkout. Do not mutate the working
tree, the index, HEAD, or branch state in any way.
## Do Not Trust the Report
Treat the implementer's report as unverified claims about the code. It
may be incomplete, inaccurate, or optimistic. Verify the claims against
the diff. Design rationales in the report are claims too: "left it per
YAGNI," "kept it simple deliberately," or any other justification is the
implementer grading their own work. Judge the code on its merits — a
stated rationale never downgrades a finding's severity.
## Tests
The implementer already ran the tests and reported results with TDD
evidence for exactly this code. Do not re-run the suite to confirm their
report. Run a test only when reading the code raises a specific doubt
that no existing run answers — and then a focused test, never a
package-wide suite, race detector run, or repeated/high-count loop. If
heavy validation seems warranted, recommend it in your report instead of
running it. If you cannot run commands in this environment, name the
test you would run.
Warnings or other noise in the implementer's reported test output are
findings — test output should be pristine.
## Part 1: Spec Compliance
Compare the diff against What Was Requested:
- **Missing:** requirements they skipped, missed, or claimed without
implementing
- **Extra:** features that weren't requested, over-engineering, unneeded
"nice to haves"
- **Misunderstood:** right feature built the wrong way, wrong problem
solved
If a requirement cannot be verified from this diff alone (it lives in
unchanged code or spans tasks), report it as a ⚠️ item instead of
broadening your search.
## Part 2: Code Quality
**Code quality:**
- Clean separation of concerns?
- Proper error handling?
- DRY without premature abstraction?
- Edge cases handled?
**Tests:**
- Do the new and changed tests verify real behavior, not mocks?
- Are the task's edge cases covered?
**Structure:**
- Does each file have one clear responsibility with a well-defined interface?
- Are units decomposed so they can be understood and tested independently?
- Is the implementation following the file structure from the plan?
- Did this change create new files that are already large, or
significantly grow existing files? (Don't flag pre-existing file
sizes — focus on what this change contributed.)
Your report should point at evidence: file:line references for every
finding and for any check you would otherwise answer with a bare
"yes." A tight report that cites lines gives the controller everything
it needs.
## Calibration
Categorize issues by actual severity. Not everything is Critical.
Important means this task cannot be trusted until it is fixed: incorrect
or fragile behavior, a missed requirement, or maintainability damage you
would block a merge over — verbatim duplication of a logic block,
swallowed errors, tests that assert nothing. "Coverage could be broader"
and polish suggestions are Minor.
Acknowledge what was done well before listing issues — accurate praise
helps the implementer trust the rest of the feedback.
## Output Format
### Spec Compliance
- ✅ Spec compliant | ❌ Issues found: [what's missing/extra/misunderstood,
with file:line references]
- ⚠️ Cannot verify from diff: [requirements you could not verify from the
diff alone, and what the controller should check — report alongside the
✅/❌ verdict for everything you could verify]
### Strengths
[What's well done? Be specific.]
### Issues
#### Critical (Must Fix)
#### Important (Should Fix)
#### Minor (Nice to Have)
For each issue: file:line, what's wrong, why it matters, how to fix
(if not obvious).
### Assessment
**Task quality:** [Approved | Needs fixes]
**Reasoning:** [1-2 sentence technical assessment]
```
**Placeholders:**
- `[MODEL]` — REQUIRED: reviewer model per SKILL.md Model Selection
- `[BRIEF_FILE]` — REQUIRED: the task brief file (`scripts/task-brief PLAN N`
prints the path; same file the implementer worked from)
- `[GLOBAL_CONSTRAINTS]` — the binding requirements copied verbatim from
the plan's Global Constraints section or the spec: exact values, formats,
and stated relationships between components (not process rules — those
are already in this template)
- `[REPORT_FILE]` — REQUIRED: the file the implementer wrote its detailed
report to
- `[BASE_SHA]` — commit before this task
- `[HEAD_SHA]` — current commit
- `[DIFF_FILE]` — REQUIRED: the path the controller wrote the review
package to (`scripts/review-package BASE HEAD` prints the unique path it
wrote; the package never enters the controller's context)
**Reviewer returns:** Spec Compliance verdict (✅/❌/⚠️), Strengths, Issues
(Critical/Important/Minor), Task quality verdict
A fix dispatch can address spec gaps and quality findings together;
re-review after fixes covers both verdicts.
@@ -64,7 +64,7 @@ prompt-template file (e.g. `superpowers:subagent-driven-development`'s
| Skill dispatch form | Antigravity equivalent |
|---------------------|----------------------|
| An implementer-style `*-prompt.md` template (writes code, runs tests) | Fill the template, then `invoke_subagent` with `TypeName: "self"` and the filled prompt |
| A read-only reviewer template (`spec-reviewer`, `code-quality-reviewer`, `code-reviewer`, `requesting-code-review`'s `./code-reviewer.md`) | `invoke_subagent` with `TypeName: "research"` and the filled review template |
| A read-only reviewer template (`task-reviewer`, `code-reviewer`, `requesting-code-review`'s `./code-reviewer.md`) | `invoke_subagent` with `TypeName: "research"` and the filled review template |
| Inline prompt (no template referenced) | `invoke_subagent` with `TypeName: "self"` (or `"research"` if the task only reads) and your inline prompt |
### Prompt filling
@@ -35,7 +35,7 @@ Skills dispatch with `Subagent (general-purpose):` and either reference a prompt
| Skill dispatch form | Gemini CLI equivalent |
|---------------------|----------------------|
| References a `*-prompt.md` template (implementer, spec-reviewer, code-quality-reviewer, code-reviewer, etc.) | Fill the template, then `invoke_agent` with `agent_name: "generalist"` and the filled prompt |
| References a `*-prompt.md` template (implementer, task-reviewer, code-reviewer, etc.) | Fill the template, then `invoke_agent` with `agent_name: "generalist"` and the filled prompt |
| References `superpowers:requesting-code-review`'s `./code-reviewer.md` | `invoke_agent` with `agent_name: "generalist"` and the filled review template |
| Inline prompt (no template referenced) | `invoke_agent` with `agent_name: "generalist"` and your inline prompt |
+22
View File
@@ -33,6 +33,15 @@ Before defining tasks, map out which files will be created or modified and what
This structure informs the task decomposition. Each task should produce self-contained changes that make sense independently.
## Task Right-Sizing
A task is the smallest unit that carries its own test cycle and is worth a
fresh reviewer's gate. When drawing task boundaries: fold setup,
configuration, scaffolding, and documentation steps into the task whose
deliverable needs them; split only where a reviewer could meaningfully
reject one task while approving its neighbor. Each task ends with an
independently testable deliverable.
## Bite-Sized Task Granularity
**Each step is one action (2-5 minutes):**
@@ -57,6 +66,13 @@ This structure informs the task decomposition. Each task should produce self-con
**Tech Stack:** [Key technologies/libraries]
## Global Constraints
[The spec's project-wide requirements — version floors, dependency limits,
naming and copy rules, platform requirements — one line each, with exact
values copied verbatim from the spec. Every task's requirements implicitly
include this section.]
---
```
@@ -70,6 +86,12 @@ This structure informs the task decomposition. Each task should produce self-con
- Modify: `exact/path/to/existing.py:123-145`
- Test: `tests/exact/path/to/test.py`
**Interfaces:**
- Consumes: [what this task uses from earlier tasks — exact signatures]
- Produces: [what later tasks rely on — exact function names, parameter
and return types. A task's implementer sees only their own task; this
block is how they learn the names and types neighboring tasks use.]
- [ ] **Step 1: Write the failing test**
```python
+33
View File
@@ -456,10 +456,29 @@ Different skill types need different test approaches:
**All of these mean: Test before deploying. No exceptions.**
## Match the Form to the Failure
Before writing guidance, classify the baseline failure. The form that bulletproofs one failure type measurably backfires on another.
| Baseline failure | Right form | Wrong form |
|---|---|---|
| Skips/violates a rule under pressure (knows better, does it anyway) | Prohibition + rationalization table + red flags (see Bulletproofing below) | Soft guidance ("prefer...", "consider...") |
| Complies, but output has the wrong shape (bloated prompt, buried verdict, restated spec) | Positive recipe or contract: state what the output IS — its parts, in order | Prohibition list ("don't restate", "never narrate") |
| Omits a required element from something they already produce | Structural: REQUIRED field or slot in the template they fill in | Prose reminders near the template |
| Behavior should depend on a condition | Conditional keyed to an observable predicate ("if the brief exists, reference it") | Unconditional rule + exemption clauses |
**Why prohibitions backfire on shaping problems:** under a competing incentive ("make the prompt self-contained"), agents negotiate with "don't X". In head-to-head wording tests on dispatch-prompt guidance, the prohibition arm produced clearly more of the unwanted content than the recipe arm (fully separated distributions), and trended worse than even the no-guidance control — micro-test your own case rather than assuming, but never reach for the prohibition by default. A recipe leaves nothing to negotiate: the output matches the stated shape or it doesn't.
**Rules for whichever form you pick:**
- **No nuance clauses.** "Don't X unless it matters" reopens the negotiation — appending a single nuance clause to a winning recipe degraded it from consistent to noisy in the same wording tests. Express a real exception as its own conditional on an observable predicate.
- **Exemption clauses don't scope.** "This limit doesn't apply to code blocks" still suppresses code blocks. If part of the output must be exempt, restructure so the rule can't reach it.
## Bulletproofing Skills Against Rationalization
Skills that enforce discipline (like TDD) need to resist rationalization. Agents are smart and will find loopholes when under pressure.
**Scope:** this toolkit is for discipline failures — an agent that knows the rule and skips it under pressure. For wrong-shaped output or omitted elements, prohibition-based bulletproofing backfires; use the forms in Match the Form to the Failure instead.
**Psychology note:** Understanding WHY persuasion techniques work helps you apply them systematically. See persuasion-principles.md for research foundation (Cialdini, 2021; Meincke et al., 2025) on authority, commitment, scarcity, social proof, and unity principles.
### Close Every Loophole Explicitly
@@ -553,6 +572,18 @@ Run same scenarios WITH skill. Agent should now comply.
Agent found new rationalization? Add explicit counter. Re-test until bulletproof.
### Micro-Test Wording Before Full Scenarios
Full pressure-scenario runs are the final gate, but they are slow and expensive per iteration. Verify the wording itself first with micro-tests:
1. **One fresh-context sample per call** — a raw API call, or a single-shot subagent if you don't have API access. System prompt = the realistic context the guidance will live in (the full skill or prompt template, not the guidance in isolation); user message = a task that tempts the failure.
2. **Always include a no-guidance control.** If the control doesn't exhibit the failure, there is nothing to fix — stop, don't author the guidance.
3. **5+ reps per variant.** Single samples lie.
4. **Manually read every flagged match.** Score programmatically if you like, but template echoes and quoted counter-examples masquerade as hits; automated counts alone overstate both failure and success.
5. **Variance is a metric.** When guidance lands, reps converge on the same shape. Five different interpretations across five reps means the wording isn't binding — tighten the form before adding words.
Micro-tests verify wording; they do not replace pressure scenarios for discipline skills.
**Testing methodology:** See [testing-skills-with-subagents.md](testing-skills-with-subagents.md) for the complete testing methodology:
- How to write pressure scenarios
- Pressure types (time, sunk cost, authority, exhaustion)
@@ -610,6 +641,8 @@ Deploying untested skills = deploying untested code. It's a violation of quality
- [ ] Keywords throughout for search (errors, symptoms, tools)
- [ ] Clear overview with core principle
- [ ] Address specific baseline failures identified in RED
- [ ] Guidance form matches the failure type (see Match the Form to the Failure)
- [ ] For behavior-shaping guidance: wording micro-tested against a no-guidance control (5+ reps, every flagged match read manually) — N/A for pure reference skills
- [ ] Code inline OR link to separate file
- [ ] One excellent example (not multi-language)
- [ ] Run scenarios WITH skill - verify agents now comply
+312
View File
@@ -0,0 +1,312 @@
/**
* Security tests for the brainstorm server's per-session key.
*
* The companion server is reachable by any local browser tab (default loopback
* bind) and by any host that can route to it (remote `--host 0.0.0.0` bind).
* A per-session secret key gates every endpoint so that neither a browser
* confused-deputy nor a direct remote client can read screens/files or inject
* events into state/events (prompt injection into a live agent session).
*
* Auth = a valid `?key=<token>` query param OR a valid session cookie.
*
* Uses the `ws` npm package as a test client (test-only dependency).
*/
const { spawn } = require('child_process');
const http = require('http');
const WebSocket = require('ws');
const fs = require('fs');
const path = require('path');
const assert = require('assert');
const SERVER_PATH = path.join(__dirname, '../../skills/brainstorming/scripts/server.cjs');
const TEST_PORT = 3335;
const TEST_DIR = '/tmp/brainstorm-auth-test';
const CONTENT_DIR = path.join(TEST_DIR, 'content');
const TOKEN = 'testtoken-0123456789abcdef0123456789abcdef';
const COOKIE_NAME = `brainstorm-key-${TEST_PORT}`;
const EXPECTED_SECURITY_HEADERS = {
'referrer-policy': 'no-referrer',
'cache-control': 'no-store',
'x-frame-options': 'DENY',
'content-security-policy': "frame-ancestors 'none'",
'cross-origin-resource-policy': 'same-origin'
};
function cleanup() {
if (fs.existsSync(TEST_DIR)) fs.rmSync(TEST_DIR, { recursive: true });
}
async function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
// Raw HTTP GET with optional key query and Cookie header.
function get(pathname, { key, cookie } = {}) {
const url = `http://localhost:${TEST_PORT}${pathname}` + (key !== undefined ? `?key=${key}` : '');
const headers = {};
if (cookie) headers['Cookie'] = cookie;
return new Promise((resolve, reject) => {
http.get(url, { headers }, (res) => {
let data = '';
res.on('data', chunk => data += chunk);
res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, body: data }));
}).on('error', reject);
});
}
// Try to open a WebSocket; resolve 'opened' or 'rejected'.
function wsConnect({ key, cookie, origin } = {}) {
const url = `ws://localhost:${TEST_PORT}/` + (key !== undefined ? `?key=${key}` : '');
const headers = {};
if (cookie) headers['Cookie'] = cookie;
if (origin) headers['Origin'] = origin;
const opts = Object.keys(headers).length ? { headers } : {};
const ws = new WebSocket(url, opts);
return new Promise((resolve) => {
let settled = false;
const done = (outcome) => { if (!settled) { settled = true; resolve({ outcome, ws }); } };
ws.on('open', () => done('opened'));
ws.on('error', () => done('rejected'));
ws.on('close', () => done('rejected'));
setTimeout(() => done('rejected'), 1500);
});
}
function startServer() {
return spawn('node', [SERVER_PATH], {
env: { ...process.env, BRAINSTORM_PORT: TEST_PORT, BRAINSTORM_DIR: TEST_DIR, BRAINSTORM_TOKEN: TOKEN }
});
}
function assertSecurityHeaders(headers) {
for (const [name, value] of Object.entries(EXPECTED_SECURITY_HEADERS)) {
assert.strictEqual(headers[name], value, `${name} should be ${value}`);
}
}
function runBootstrapScript(html, sessionStorage) {
const match = html.match(/<script>\n([\s\S]*?)\n<\/script>/);
assert(match, 'bootstrap response should contain a script block');
const replacements = [];
const location = { replace(url) { replacements.push(url); } };
new Function('sessionStorage', 'location', match[1])(sessionStorage, location);
return replacements;
}
async function waitForServer(server) {
let stdout = '', stderr = '';
return new Promise((resolve, reject) => {
server.stdout.on('data', (d) => {
stdout += d.toString();
if (stdout.includes('server-started')) resolve({ stdout });
});
server.stderr.on('data', (d) => { stderr += d.toString(); });
server.on('error', reject);
setTimeout(() => reject(new Error(`Server didn't start. stderr: ${stderr}`)), 5000);
});
}
function serverStartedMessage(out) {
const line = out.trim().split('\n').find(l => l.includes('server-started'));
assert(line, 'server-started JSON should be present');
return JSON.parse(line);
}
function assertStartedOnExpectedPort(out) {
const msg = serverStartedMessage(out);
assert.strictEqual(
msg.port,
TEST_PORT,
`auth.test.js expected fixed port ${TEST_PORT}, got ${msg.port}; fixed-port tests must not run through fallback`
);
return msg;
}
async function runTests() {
cleanup();
fs.mkdirSync(CONTENT_DIR, { recursive: true });
fs.writeFileSync(path.join(CONTENT_DIR, 'screen.html'), '<h2>Secret screen</h2>');
fs.writeFileSync(path.join(CONTENT_DIR, 'asset.txt'), 'secret asset');
const server = startServer();
let stdoutAccum = '';
server.stdout.on('data', (d) => { stdoutAccum += d.toString(); });
let passed = 0, failed = 0;
async function test(name, fn) {
try { await fn(); console.log(` PASS: ${name}`); passed++; }
catch (e) { console.log(` FAIL: ${name}`); console.log(` ${e.message}`); failed++; }
}
try {
const { stdout: initialStdout } = await waitForServer(server);
assertStartedOnExpectedPort(initialStdout);
console.log('\n--- Startup URL ---');
await test('server-started url includes the session key', () => {
const msg = serverStartedMessage(initialStdout);
assert(msg.url.includes(`key=${TOKEN}`), `url should carry the key, got: ${msg.url}`);
});
console.log('\n--- HTTP / gate ---');
await test('GET / without key is rejected with 403', async () => {
const res = await get('/');
assert.strictEqual(res.status, 403, 'no-key request must be 403');
});
await test('403 page names "coding agent" and the key', async () => {
const res = await get('/');
assert(/coding agent/i.test(res.body), '403 body should reference the coding agent');
assert(/key/i.test(res.body), '403 body should mention the key');
});
await test('403 responses include leak-reduction and anti-framing headers', async () => {
const res = await get('/');
assert.strictEqual(res.status, 403);
assertSecurityHeaders(res.headers);
});
await test('GET / with wrong key is rejected with 403', async () => {
const res = await get('/', { key: 'wrong-token' });
assert.strictEqual(res.status, 403);
});
await test('GET / with wrong key and valid cookie is rejected with 403', async () => {
const res = await get('/', { key: 'wrong-token', cookie: `${COOKIE_NAME}=${TOKEN}` });
assert.strictEqual(res.status, 403, 'explicit wrong query key must not fall back to cookie auth');
});
await test('GET / with valid query returns bootstrap instead of screen content', async () => {
const res = await get('/', { key: TOKEN });
assert.strictEqual(res.status, 200);
assert(res.body.includes('sessionStorage'), 'bootstrap should store the session key in tab storage');
assert(res.body.includes('location.replace'), 'bootstrap should navigate to the bare root URL');
assert(!res.body.includes('Secret screen'), 'bootstrap must not serve screen HTML at the keyed URL');
});
await test('bootstrap strips the key URL even when sessionStorage write fails', async () => {
const res = await get('/', { key: TOKEN });
assert.strictEqual(res.status, 200);
let replacements;
assert.doesNotThrow(() => {
replacements = runBootstrapScript(res.body, {
setItem() { throw new Error('storage blocked'); }
});
});
assert.deepStrictEqual(replacements, ['/']);
});
await test('HTML responses include leak-reduction and anti-framing headers', async () => {
const res = await get('/', { key: TOKEN });
assert.strictEqual(res.status, 200);
assertSecurityHeaders(res.headers);
});
await test('valid key load sets an HttpOnly SameSite=Strict cookie', async () => {
const res = await get('/', { key: TOKEN });
const setCookie = (res.headers['set-cookie'] || []).join('; ');
assert(setCookie.includes(`${COOKIE_NAME}=${TOKEN}`), `should set ${COOKIE_NAME}`);
assert(/HttpOnly/i.test(setCookie), 'cookie should be HttpOnly');
assert(/SameSite=Strict/i.test(setCookie), 'cookie should be SameSite=Strict');
});
await test('GET / with valid cookie (no query key) serves the screen', async () => {
const res = await get('/', { cookie: `${COOKIE_NAME}=${TOKEN}` });
assert.strictEqual(res.status, 200);
assert(res.body.includes('Secret screen'), 'cookie-authenticated bare root should serve the screen');
assert(!res.body.includes("location.replace('/');"), 'bare screen response should not be the bootstrap page');
});
console.log('\n--- HTTP /files gate ---');
await test('GET /files without key is rejected with 403', async () => {
const res = await get('/files/asset.txt');
assert.strictEqual(res.status, 403);
});
await test('GET /files with valid key serves the file', async () => {
const res = await get('/files/asset.txt', { key: TOKEN });
assert.strictEqual(res.status, 200);
assert(res.body.includes('secret asset'));
});
await test('/files responses include leak-reduction and anti-framing headers', async () => {
const res = await get('/files/asset.txt', { key: TOKEN });
assert.strictEqual(res.status, 200);
assertSecurityHeaders(res.headers);
});
console.log('\n--- WebSocket gate ---');
await test('WS upgrade without key is rejected', async () => {
const { outcome, ws } = await wsConnect();
ws.close();
assert.strictEqual(outcome, 'rejected', 'unauthenticated WS must not open');
});
await test('WS upgrade with valid key opens', async () => {
const { outcome, ws } = await wsConnect({ key: TOKEN });
ws.close();
assert.strictEqual(outcome, 'opened');
});
await test('WS upgrade with valid cookie opens', async () => {
const { outcome, ws } = await wsConnect({ cookie: `${COOKIE_NAME}=${TOKEN}` });
ws.close();
assert.strictEqual(outcome, 'opened');
});
await test('WS upgrade with valid cookie and same-origin Origin opens', async () => {
const { outcome, ws } = await wsConnect({
cookie: `${COOKIE_NAME}=${TOKEN}`,
origin: `http://localhost:${TEST_PORT}`
});
ws.close();
assert.strictEqual(outcome, 'opened');
});
await test('WS upgrade with valid cookie but cross-origin Origin is rejected', async () => {
const eventsFile = path.join(TEST_DIR, 'state', 'events');
if (fs.existsSync(eventsFile)) fs.unlinkSync(eventsFile);
const { outcome, ws } = await wsConnect({
cookie: `${COOKIE_NAME}=${TOKEN}`,
origin: 'http://localhost:9999'
});
if (outcome === 'opened') {
ws.send(JSON.stringify({ type: 'choice', choice: 'attacker-injected', text: 'local attacker probe' }));
await sleep(300);
}
ws.close();
assert.strictEqual(outcome, 'rejected', 'cross-origin browser WS must not open even with cookie');
assert(!fs.existsSync(eventsFile), 'cross-origin WS must not write state/events');
});
console.log('\n--- Robustness (A3) ---');
await test('null payload over an authed WS does not crash the server', async () => {
const { ws } = await wsConnect({ key: TOKEN });
ws.send('null');
await sleep(300);
const res = await get('/', { key: TOKEN });
assert.strictEqual(res.status, 200, 'server must still respond after null payload');
ws.close();
});
console.log(`\n--- Results: ${passed} passed, ${failed} failed ---`);
if (failed > 0) {
process.exitCode = 1;
return;
}
} finally {
server.kill();
await sleep(100);
cleanup();
}
}
runTests().catch(err => { console.error('Test failed:', err); process.exit(1); });
@@ -0,0 +1,66 @@
const assert = require('assert');
const {
browserLauncherForPlatform
} = require('../../skills/brainstorming/scripts/server.cjs');
let passed = 0;
let failed = 0;
async function test(name, fn) {
try {
await fn();
console.log(` PASS: ${name}`);
passed++;
} catch (e) {
console.log(` FAIL: ${name}`);
console.log(` ${e.message}`);
failed++;
}
}
(async () => {
console.log('\n--- Browser Launcher ---');
await test('Windows launcher does not route URLs through cmd.exe', () => {
const url = 'http://localhost:54122/?key=abc&x=SAFE&echo=INJECTED';
const launcher = browserLauncherForPlatform(url, {
platform: 'win32',
osRelease: '10.0.26200',
env: {}
});
assert.deepStrictEqual(launcher, {
bin: 'rundll32.exe',
args: ['url.dll,FileProtocolHandler', url]
});
assert(!launcher.args.includes('/c'), 'Windows launcher must not pass /c to a command interpreter');
});
await test('WSL launcher does not route URLs through cmd.exe', () => {
const url = 'http://localhost:54122/?key=abc&x=SAFE&echo=INJECTED';
const launcher = browserLauncherForPlatform(url, {
platform: 'linux',
osRelease: '5.15.167.4-microsoft-standard-WSL2',
env: {}
});
assert.deepStrictEqual(launcher, {
bin: 'rundll32.exe',
args: ['url.dll,FileProtocolHandler', url]
});
});
await test('Linux launcher stays headless without a display', () => {
assert.strictEqual(
browserLauncherForPlatform('http://localhost:1/', {
platform: 'linux',
osRelease: '6.0.0',
env: {}
}),
null
);
});
console.log(`\n--- Results: ${passed} passed, ${failed} failed ---`);
if (failed > 0) process.exit(1);
})();
+197
View File
@@ -0,0 +1,197 @@
/**
* Tests for the injected browser client (helper.js).
*
* helper.js runs in the browser, so its DOM behaviour is exercised live; here we
* unit-test the pure reconnect-backoff function it exports and assert that the
* reconnect / status / tombstone wiring is present.
*/
const assert = require('assert');
const fs = require('fs');
const path = require('path');
const HELPER = path.join(__dirname, '../../skills/brainstorming/scripts/helper.js');
const src = fs.readFileSync(HELPER, 'utf-8');
// helper.js is browser code, and the repo is an ES module package, so a plain
// require() won't surface its exports. Evaluate the source in a CommonJS sandbox
// with no `window`, so only the exported pure helpers run (not the browser code).
const moduleShim = { exports: {} };
new Function('module', src)(moduleShim);
const { nextReconnectDelay, MIN_RECONNECT_MS, MAX_RECONNECT_MS, TOMBSTONE_AFTER_MS } = moduleShim.exports;
let passed = 0, failed = 0;
function test(name, fn) {
try { fn(); console.log(` PASS: ${name}`); passed++; }
catch (e) { console.log(` FAIL: ${name}`); console.log(` ${e.message}`); failed++; }
}
console.log('\n--- Backoff (pure) ---');
test('doubles the delay each call', () => {
assert.strictEqual(nextReconnectDelay(500, 30000), 1000);
assert.strictEqual(nextReconnectDelay(1000, 30000), 2000);
assert.strictEqual(nextReconnectDelay(2000, 30000), 4000);
});
test('caps at the maximum', () => {
assert.strictEqual(nextReconnectDelay(20000, 30000), 30000);
assert.strictEqual(nextReconnectDelay(30000, 30000), 30000);
});
test('full progression from MIN caps at MAX and never exceeds it', () => {
const seq = [MIN_RECONNECT_MS];
let d = MIN_RECONNECT_MS;
for (let i = 0; i < 10; i++) { d = nextReconnectDelay(d, MAX_RECONNECT_MS); seq.push(d); }
assert.strictEqual(seq[0], 500);
assert.deepStrictEqual(seq.slice(0, 7), [500, 1000, 2000, 4000, 8000, 16000, 30000]);
assert(seq.every(v => v <= MAX_RECONNECT_MS), 'never exceeds max');
assert.strictEqual(seq[seq.length - 1], 30000, 'settles at the cap');
});
test('exposes sane constants', () => {
assert.strictEqual(MIN_RECONNECT_MS, 500);
assert.strictEqual(MAX_RECONNECT_MS, 30000);
assert(TOMBSTONE_AFTER_MS >= 5000, 'tombstone grace is at least a few seconds');
});
console.log('\n--- Wiring (source) ---');
test('reflects all three connection states', () => {
assert(/Connected/.test(src) && /Reconnecting/.test(src) && /Disconnected/.test(src),
'should set Connected / Reconnecting / Disconnected status');
assert(src.includes("setProperty('--status-color'"), 'drives the status dot via --status-color');
});
test('renders a tombstone overlay when paused', () => {
assert(src.includes('bs-tombstone'), 'creates the tombstone element');
assert(/Companion paused/.test(src), 'tombstone explains the companion paused');
});
test('hardens reconnection (onerror, null socket, clears pending timer)', () => {
assert(src.includes('onerror'), 'handles onerror');
assert(/ws = null/.test(src), 'nulls the socket on close so sendEvent queues');
assert(src.includes('clearTimeout'), 'clears a pending reconnect before scheduling another');
assert(src.includes('nextReconnectDelay'), 'uses exponential backoff for reconnects');
});
test('reloads on recovery and on reload messages', () => {
assert(/location\.reload\(\)/.test(src), 'reloads to pick up restarted/updated content');
});
console.log('\n--- Reconnect state machine (mocked browser) ---');
// Drive helper.js's browser code against mocked DOM/WebSocket/timers/clock so we
// can exercise the actual reconnect/status/tombstone behaviour, not just grep it.
function makeEnv() {
const state = { now: 1000, timers: [], reloads: 0, replacements: [], appended: [], sessionKey: 'stored-key-abc' };
const sockets = [];
const statusEl = { textContent: '', style: { setProperty() {} } };
class FakeWS {
constructor(url) { this.url = url; this.readyState = 0; this.onopen = this.onclose = this.onmessage = this.onerror = null; sockets.push(this); }
send() {}
close() { this.readyState = 3; if (this.onclose) this.onclose(); }
open() { this.readyState = 1; if (this.onopen) this.onopen(); }
}
FakeWS.OPEN = 1;
const env = {
module: { exports: {} },
window: {
location: {
host: 'localhost:7777',
reload() { state.reloads++; },
replace(url) { state.replacements.push(url); }
},
sessionStorage: { getItem: (key) => key === 'brainstorm-session-key' ? state.sessionKey : null }
},
document: {
querySelector: (s) => s === '.status' ? statusEl : null,
getElementById: () => null,
createElement: () => ({ style: {}, id: '' }),
addEventListener() {},
body: { appendChild: (el) => state.appended.push(el) }
},
WebSocket: FakeWS,
setTimeout: (fn, ms) => { state.timers.push({ fn, ms, fired: false, cleared: false }); return state.timers.length; },
clearTimeout: (id) => { if (state.timers[id - 1]) state.timers[id - 1].cleared = true; },
Date: { now: () => state.now },
console
};
return {
state, statusEl, sockets,
boot() { new Function(...Object.keys(env), src)(...Object.values(env)); },
advance(ms) { state.now += ms; },
last() { return sockets[sockets.length - 1]; },
fireReconnect() {
const t = [...state.timers].reverse().find(x => !x.fired && !x.cleared);
if (!t) throw new Error('no reconnect scheduled');
t.fired = true; t.fn();
}
};
}
test('uses sessionStorage key in the WebSocket URL when present', () => {
const e = makeEnv();
e.state.sessionKey = 'stored-key-abc';
e.boot();
assert.strictEqual(e.sockets[0].url, 'ws://localhost:7777/?key=stored-key-abc');
});
test('uses cookie-only WebSocket URL when no sessionStorage key is present', () => {
const e = makeEnv();
e.state.sessionKey = null;
e.boot();
assert.strictEqual(e.sockets[0].url, 'ws://localhost:7777');
});
test('on disconnect shows Reconnecting and schedules a 500ms reconnect', () => {
const e = makeEnv(); e.boot();
e.last().open();
assert.strictEqual(e.statusEl.textContent, 'Connected');
e.last().close();
assert.strictEqual(e.statusEl.textContent, 'Reconnecting…');
assert.strictEqual(e.state.timers[e.state.timers.length - 1].ms, 500);
});
test('reconnect delay backs off 500 -> 1000 -> 2000', () => {
const e = makeEnv(); e.boot();
e.last().open(); e.last().close();
e.fireReconnect(); e.last().close();
e.fireReconnect(); e.last().close();
assert.deepStrictEqual(e.state.timers.map(t => t.ms).slice(0, 3), [500, 1000, 2000]);
});
test('shows the tombstone and Disconnected after the grace period', () => {
const e = makeEnv(); e.boot();
e.last().open(); e.last().close();
e.advance(20000); // past TOMBSTONE_AFTER_MS while still down
e.fireReconnect(); e.last().close();
assert.strictEqual(e.statusEl.textContent, 'Disconnected');
assert.strictEqual(e.state.appended.length, 1, 'tombstone appended exactly once');
});
test('rebootstraps with stored key when a tombstoned connection comes back', () => {
const e = makeEnv(); e.boot();
e.last().open(); e.last().close();
e.advance(20000); e.fireReconnect(); e.last().close(); // tombstone now shown
assert.deepStrictEqual(e.state.replacements, []);
e.fireReconnect(); e.last().open(); // server back (e.g. same-port restart)
assert.strictEqual(e.state.reloads, 0, 'stored-key recovery should not reload bare /');
assert.deepStrictEqual(e.state.replacements, ['/?key=stored-key-abc']);
});
test('reloads to recover when tombstoned and no sessionStorage key is present', () => {
const e = makeEnv();
e.state.sessionKey = null;
e.boot();
e.last().open(); e.last().close();
e.advance(20000); e.fireReconnect(); e.last().close(); // tombstone now shown
assert.strictEqual(e.state.reloads, 0);
e.fireReconnect(); e.last().open(); // server back (e.g. cookie-only page)
assert.strictEqual(e.state.reloads, 1, 'reloads once on recovery');
assert.deepStrictEqual(e.state.replacements, []);
});
console.log(`\n--- Results: ${passed} passed, ${failed} failed ---`);
if (failed > 0) process.exit(1);
+515
View File
@@ -0,0 +1,515 @@
/**
* Tests for the brainstorm server's lifecycle (idle timeout + shutdown).
*
* - The idle timeout is configurable (default 4h) and reported in server-info.
* - Idle shutdown must close any open WebSocket so the process actually exits,
* not hang on a lingering connection.
* - start-server.sh exposes the timeout via --idle-timeout-minutes.
*
* Uses the `ws` npm package as a test client (test-only dependency).
*/
const { spawn, execFileSync } = require('child_process');
const WebSocket = require('ws');
const fs = require('fs');
const path = require('path');
const assert = require('assert');
const SERVER = path.join(__dirname, '../../skills/brainstorming/scripts/server.cjs');
const START = path.join(__dirname, '../../skills/brainstorming/scripts/start-server.sh');
const STOP = path.join(__dirname, '../../skills/brainstorming/scripts/stop-server.sh');
const sleep = ms => new Promise(r => setTimeout(r, ms));
function waitForExit(child, timeoutMs = 2000) {
if (child.exitCode !== null || child.signalCode !== null) return Promise.resolve(true);
return new Promise(resolve => {
let settled = false;
const finish = (exited) => {
if (settled) return;
settled = true;
resolve(exited);
};
child.once('exit', () => finish(true));
setTimeout(() => finish(false), timeoutMs);
});
}
async function killAndWait(child, timeoutMs = 2000) {
if (!child || child.exitCode !== null || child.signalCode !== null) return true;
const exited = waitForExit(child, timeoutMs);
child.kill();
if (await exited) return true;
child.kill('SIGKILL');
return waitForExit(child, 500);
}
async function waitForFile(file, timeoutMs = 3000) {
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline) {
if (fs.existsSync(file)) return true;
await sleep(50);
}
return fs.existsSync(file);
}
function firstServerStarted(out) {
return JSON.parse(out.trim().split('\n').find(l => l.includes('server-started')));
}
function openCaptureCommand(dir, marker) {
const scriptPath = path.resolve(dir, 'capture-open.cjs');
const markerPath = path.resolve(marker);
fs.writeFileSync(scriptPath,
"const fs = require('fs');\n" +
"fs.appendFileSync(process.argv[2], process.argv[3] + '\\n');\n");
return `node ${JSON.stringify(scriptPath)} ${JSON.stringify(markerPath)}`;
}
function httpStatus(port, key) {
return new Promise(resolve => {
const pathWithKey = key ? '/?key=' + encodeURIComponent(key) : '/';
require('http')
.get({ hostname: '127.0.0.1', port, path: pathWithKey }, res => {
res.resume();
resolve(res.statusCode);
})
.on('error', () => resolve(0));
});
}
function isWindowsLikeShell() {
return process.platform === 'win32' ||
/^msys|^cygwin|^mingw/i.test(process.env.OSTYPE || '') ||
!!process.env.MSYSTEM;
}
async function waitForStartedOutput(child, timeoutMs = 5000) {
let stdout = '';
let stderr = '';
child.stdout.on('data', d => { stdout += d.toString(); });
child.stderr.on('data', d => { stderr += d.toString(); });
const deadline = Date.now() + timeoutMs;
while (Date.now() < deadline && !stdout.includes('server-started') && child.exitCode === null) {
await sleep(50);
}
if (!stdout.includes('server-started')) {
throw new Error(`start-server.sh did not report server-started. exit=${child.exitCode} stdout=${stdout} stderr=${stderr}`);
}
return stdout;
}
function makeShellTempDir(prefix) {
return execFileSync('bash', ['-lc', `mktemp -d "\${TMPDIR:-/tmp}/${prefix}-XXXXXX"`], { encoding: 'utf8' }).trim();
}
function removeShellPath(p) {
execFileSync('bash', ['-lc', 'rm -rf "$1"', 'bash', p], { stdio: 'ignore' });
}
function newestSessionDir(projectDir) {
const sessionDir = execFileSync('bash', [
'-lc',
'find "$1/.superpowers/brainstorm" -mindepth 1 -maxdepth 1 -type d -print | sort | tail -1',
'bash',
projectDir
], { encoding: 'utf8' }).trim();
assert(sessionDir, `expected at least one session dir under ${projectDir}/.superpowers/brainstorm`);
return sessionDir;
}
async function runTests() {
let passed = 0, failed = 0;
async function test(name, fn) {
try { await fn(); console.log(` PASS: ${name}`); passed++; }
catch (e) { console.log(` FAIL: ${name}`); console.log(` ${e.message}`); failed++; }
}
await test('server-info reports the configured idle_timeout_ms', async () => {
const dir = fs.mkdtempSync('/tmp/bs-life-');
const srv = spawn('node', [SERVER], { env: { ...process.env, BRAINSTORM_PORT: 3401, BRAINSTORM_DIR: dir, BRAINSTORM_IDLE_TIMEOUT_MS: 1234567 } });
let out = ''; srv.stdout.on('data', d => out += d.toString());
for (let i = 0; i < 60 && !out.includes('server-started'); i++) await sleep(50);
try {
const info = firstServerStarted(out);
assert.strictEqual(info.idle_timeout_ms, 1234567, 'idle_timeout_ms should reflect the env override');
} finally {
await killAndWait(srv);
fs.rmSync(dir, { recursive: true, force: true });
}
});
await test('idle shutdown closes an open WebSocket and the process exits', async () => {
const dir = fs.mkdtempSync('/tmp/bs-life-');
const srv = spawn('node', [SERVER], { env: { ...process.env, BRAINSTORM_PORT: 3402, BRAINSTORM_DIR: dir, BRAINSTORM_TOKEN: 'lifetoken', BRAINSTORM_IDLE_TIMEOUT_MS: 200, BRAINSTORM_LIFECYCLE_CHECK_MS: 100 } });
let out = ''; srv.stdout.on('data', d => out += d.toString());
let exited = false, code = null; srv.on('exit', c => { exited = true; code = c; });
for (let i = 0; i < 60 && !out.includes('server-started'); i++) await sleep(50);
const ws = new WebSocket('ws://localhost:3402/?key=lifetoken');
await new Promise((res, rej) => { ws.on('open', res); ws.on('error', rej); });
// 200ms idle, checked every 100ms — should shut down and exit well within 4s,
// *despite* the open WS, only if shutdown() closes client sockets.
for (let i = 0; i < 40 && !exited; i++) await sleep(100);
try {
assert(exited, 'process must exit after idle shutdown even with an open WebSocket');
assert.strictEqual(code, 0, 'should exit cleanly (0)');
assert(fs.existsSync(path.join(dir, 'state', 'server-stopped')), 'should write server-stopped');
} finally {
try { ws.close(); } catch (e) {}
if (!exited) await killAndWait(srv);
fs.rmSync(dir, { recursive: true, force: true });
}
});
await test('start-server.sh --idle-timeout-minutes sets the timeout', async () => {
const dir = makeShellTempDir('bs-life');
let info = null;
let startProcess = null;
let sessionDir = null;
try {
if (isWindowsLikeShell()) {
startProcess = spawn('bash', [START, '--project-dir', dir, '--idle-timeout-minutes', '5']);
info = firstServerStarted(await waitForStartedOutput(startProcess));
} else {
const out = execFileSync('bash', [START, '--project-dir', dir, '--idle-timeout-minutes', '5', '--background'], { encoding: 'utf8' });
info = firstServerStarted(out);
}
sessionDir = newestSessionDir(dir);
assert.strictEqual(info.idle_timeout_ms, 5 * 60 * 1000, '5 minutes -> 300000 ms');
} finally {
if (sessionDir) execFileSync('bash', [STOP, sessionDir], { stdio: 'ignore' });
if (startProcess && !await waitForExit(startProcess, 3000)) {
await killAndWait(startProcess);
}
removeShellPath(dir);
}
});
await test('server-started URL brackets IPv6 URL hosts', async () => {
const dir = fs.mkdtempSync('/tmp/bs-ipv6-url-');
const srv = spawn('node', [SERVER], {
env: {
...process.env,
BRAINSTORM_PORT: 3421,
BRAINSTORM_HOST: '127.0.0.1',
BRAINSTORM_URL_HOST: '::1',
BRAINSTORM_TOKEN: 'ipv6token',
BRAINSTORM_DIR: dir,
BRAINSTORM_LIFECYCLE_CHECK_MS: 100000
}
});
let out = ''; srv.stdout.on('data', d => out += d.toString());
try {
for (let i = 0; i < 60 && !out.includes('server-started'); i++) await sleep(50);
const info = firstServerStarted(out);
assert.strictEqual(info.url, 'http://[::1]:3421/?key=ipv6token');
} finally {
await killAndWait(srv);
fs.rmSync(dir, { recursive: true, force: true });
}
});
await test('persists the bound port AND key, and restores both on restart', async () => {
const dir = fs.mkdtempSync('/tmp/bs-port-');
const portFile = path.join(dir, '.last-port');
const tokenFile = path.join(dir, '.last-token');
const env = { ...process.env, BRAINSTORM_PORT_FILE: portFile, BRAINSTORM_TOKEN_FILE: tokenFile, BRAINSTORM_LIFECYCLE_CHECK_MS: 100000 };
const a = spawn('node', [SERVER], { env: { ...env, BRAINSTORM_DIR: path.join(dir, 's1') } });
let outA = ''; a.stdout.on('data', d => outA += d.toString());
for (let i = 0; i < 60 && !outA.includes('server-started'); i++) await sleep(50);
const infoA = firstServerStarted(outA);
const keyA = new URL(infoA.url).searchParams.get('key');
assert(fs.existsSync(portFile) && fs.existsSync(tokenFile), 'should write the port and token files');
const exitedA = waitForExit(a);
a.kill();
assert(await exitedA, 'first server should exit before restart binds its port');
const b = spawn('node', [SERVER], { env: { ...env, BRAINSTORM_DIR: path.join(dir, 's2') } });
let outB = ''; b.stdout.on('data', d => outB += d.toString());
for (let i = 0; i < 60 && !outB.includes('server-started'); i++) await sleep(50);
const infoB = firstServerStarted(outB);
const keyB = new URL(infoB.url).searchParams.get('key');
await killAndWait(b);
fs.rmSync(dir, { recursive: true, force: true });
assert.strictEqual(infoB.port, infoA.port, 'restart should reuse the same port');
// Same key too — otherwise the open tab's cookie would 403 against the restart.
assert.strictEqual(keyB, keyA, 'restart should reuse the same session key');
});
await test('hardens existing persisted token file permissions', async () => {
const dir = fs.mkdtempSync('/tmp/bs-token-mode-');
const portFile = path.join(dir, '.last-port');
const tokenFile = path.join(dir, '.last-token');
const token = 'efefefefefefefefefefefefefefefef';
let srv = null;
try {
fs.writeFileSync(tokenFile, token, { mode: 0o644 });
fs.chmodSync(tokenFile, 0o644);
srv = spawn('node', [SERVER], {
env: {
...process.env,
BRAINSTORM_DIR: path.join(dir, 's1'),
BRAINSTORM_PORT_FILE: portFile,
BRAINSTORM_TOKEN_FILE: tokenFile,
BRAINSTORM_LIFECYCLE_CHECK_MS: 100000
}
});
let out = ''; srv.stdout.on('data', d => out += d.toString());
for (let i = 0; i < 60 && !out.includes('server-started'); i++) await sleep(50);
assert(out.includes('server-started'), 'server should start with persisted token');
if (process.platform !== 'win32') {
const mode = fs.statSync(tokenFile).mode & 0o777;
assert.strictEqual(mode, 0o600, `.last-token mode should be 0600, got ${mode.toString(8)}`);
} else {
assert(fs.existsSync(tokenFile), 'token file should remain present on Windows');
}
} finally {
await killAndWait(srv);
fs.rmSync(dir, { recursive: true, force: true });
}
});
await test('stored key can authenticate WebSocket after same-port restart', async () => {
const dir = fs.mkdtempSync('/tmp/bs-reconnect-');
const portFile = path.join(dir, '.last-port');
const tokenFile = path.join(dir, '.last-token');
const env = { ...process.env, BRAINSTORM_PORT_FILE: portFile, BRAINSTORM_TOKEN_FILE: tokenFile, BRAINSTORM_LIFECYCLE_CHECK_MS: 100000 };
let a = null, b = null, ws = null;
try {
a = spawn('node', [SERVER], { env: { ...env, BRAINSTORM_DIR: path.join(dir, 's1') } });
let outA = ''; a.stdout.on('data', d => outA += d.toString());
for (let i = 0; i < 60 && !outA.includes('server-started'); i++) await sleep(50);
const infoA = firstServerStarted(outA);
const keyA = new URL(infoA.url).searchParams.get('key');
const exitedA = waitForExit(a);
a.kill();
assert(await exitedA, 'first server should exit before restart binds its port');
a = null;
b = spawn('node', [SERVER], { env: { ...env, BRAINSTORM_DIR: path.join(dir, 's2') } });
let outB = ''; b.stdout.on('data', d => outB += d.toString());
for (let i = 0; i < 60 && !outB.includes('server-started'); i++) await sleep(50);
const infoB = firstServerStarted(outB);
ws = new WebSocket(`ws://localhost:${infoB.port}/?key=${keyA}`, {
headers: { Origin: `http://localhost:${infoB.port}` }
});
const opened = await new Promise(resolve => {
ws.on('open', () => resolve(true));
ws.on('error', () => resolve(false));
setTimeout(() => resolve(false), 1500);
});
assert.strictEqual(infoB.port, infoA.port, 'restart should reuse same port');
assert(opened, 'stored key should authenticate WS after restart');
} finally {
try { if (ws) ws.close(); } catch (e) {}
await killAndWait(a);
await killAndWait(b);
fs.rmSync(dir, { recursive: true, force: true });
}
});
await test('falls back to a random port when the preferred port is taken', async () => {
const dir = fs.mkdtempSync('/tmp/bs-port-');
const portFile = path.join(dir, '.last-port');
const a = spawn('node', [SERVER], { env: { ...process.env, BRAINSTORM_DIR: path.join(dir, 'a'), BRAINSTORM_PORT: 3415, BRAINSTORM_LIFECYCLE_CHECK_MS: 100000 } });
let outA = ''; a.stdout.on('data', d => outA += d.toString());
for (let i = 0; i < 60 && !outA.includes('server-started'); i++) await sleep(50);
fs.writeFileSync(portFile, '3415'); // preferred port, but it's taken by A
const b = spawn('node', [SERVER], { env: { ...process.env, BRAINSTORM_DIR: path.join(dir, 'b'), BRAINSTORM_PORT_FILE: portFile, BRAINSTORM_LIFECYCLE_CHECK_MS: 100000 } });
let outB = ''; b.stdout.on('data', d => outB += d.toString());
for (let i = 0; i < 60 && !outB.includes('server-started'); i++) await sleep(50);
const portB = firstServerStarted(outB).port;
const persisted = fs.readFileSync(portFile, 'utf8').trim();
await killAndWait(a);
await killAndWait(b);
fs.rmSync(dir, { recursive: true, force: true });
assert.notStrictEqual(portB, 3415, 'must not bind the already-taken port');
assert(portB >= 49152, 'should fall back to a random high port');
// The fallback must NOT clobber the shared port file — A still owns 3415 and
// its open tab must keep reconnecting there.
assert.strictEqual(persisted, '3415', 'fallback must not overwrite .last-port');
});
await test('fallback with persisted token generates a fresh unpersisted key', async () => {
const dir = fs.mkdtempSync('/tmp/bs-port-');
const portFile = path.join(dir, '.last-port');
const tokenFile = path.join(dir, '.last-token');
const preferredToken = 'abababababababababababababababab';
let a = null, b = null;
try {
a = spawn('node', [SERVER], {
env: {
...process.env,
BRAINSTORM_DIR: path.join(dir, 'a'),
BRAINSTORM_PORT: 3422,
BRAINSTORM_TOKEN: preferredToken,
BRAINSTORM_LIFECYCLE_CHECK_MS: 100000
}
});
let outA = ''; a.stdout.on('data', d => outA += d.toString());
for (let i = 0; i < 60 && !outA.includes('server-started'); i++) await sleep(50);
assert(outA.includes('server-started'), 'preferred-port server should start');
fs.writeFileSync(portFile, '3422');
fs.writeFileSync(tokenFile, preferredToken, { mode: 0o600 });
b = spawn('node', [SERVER], {
env: {
...process.env,
BRAINSTORM_DIR: path.join(dir, 'b'),
BRAINSTORM_PORT_FILE: portFile,
BRAINSTORM_TOKEN_FILE: tokenFile,
BRAINSTORM_LIFECYCLE_CHECK_MS: 100000
}
});
let outB = ''; b.stdout.on('data', d => outB += d.toString());
for (let i = 0; i < 60 && !outB.includes('server-started'); i++) await sleep(50);
const infoB = firstServerStarted(outB);
const fallbackKey = new URL(infoB.url).searchParams.get('key');
const persistedAfter = fs.readFileSync(tokenFile, 'utf8').trim();
const originalStatus = await httpStatus(3422, fallbackKey);
assert.notStrictEqual(infoB.port, 3422, 'fallback should use a different port');
assert.notStrictEqual(fallbackKey, preferredToken, 'fallback must not reuse persisted key');
assert.strictEqual(persistedAfter, preferredToken, 'fallback must not overwrite .last-token');
assert.strictEqual(originalStatus, 403, 'fallback key must not authenticate to original server');
} finally {
await killAndWait(a);
await killAndWait(b);
fs.rmSync(dir, { recursive: true, force: true });
}
});
await test('fallback with explicit BRAINSTORM_TOKEN fails closed', async () => {
const dir = fs.mkdtempSync('/tmp/bs-port-');
const portFile = path.join(dir, '.last-port');
const explicitToken = 'cdcdcdcdcdcdcdcdcdcdcdcdcdcdcdcd';
let a = null, b = null;
try {
a = spawn('node', [SERVER], {
env: {
...process.env,
BRAINSTORM_DIR: path.join(dir, 'a'),
BRAINSTORM_PORT: 3423,
BRAINSTORM_TOKEN: explicitToken,
BRAINSTORM_LIFECYCLE_CHECK_MS: 100000
}
});
let outA = ''; a.stdout.on('data', d => outA += d.toString());
for (let i = 0; i < 60 && !outA.includes('server-started'); i++) await sleep(50);
assert(outA.includes('server-started'), 'preferred-port server should start');
fs.writeFileSync(portFile, '3423');
b = spawn('node', [SERVER], {
env: {
...process.env,
BRAINSTORM_DIR: path.join(dir, 'b'),
BRAINSTORM_PORT_FILE: portFile,
BRAINSTORM_TOKEN: explicitToken,
BRAINSTORM_LIFECYCLE_CHECK_MS: 100000
}
});
let outB = ''; let errB = '';
b.stdout.on('data', d => outB += d.toString());
b.stderr.on('data', d => errB += d.toString());
for (let i = 0; i < 60 && !outB.includes('server-started') && b.exitCode === null; i++) await sleep(50);
const exited = await waitForExit(b, 1500);
assert(exited, 'explicit-token fallback process should exit');
assert.notStrictEqual(b.exitCode, 0, 'explicit-token fallback should fail non-zero');
assert(!outB.includes('server-started'), 'explicit-token fallback must not start on a random port');
assert(/BRAINSTORM_TOKEN/.test(errB), `stderr should explain explicit token fallback refusal, got: ${errB}`);
} finally {
await killAndWait(a);
await killAndWait(b);
fs.rmSync(dir, { recursive: true, force: true });
}
});
await test('auto-opens the browser once, on the first screen', async () => {
const dir = fs.mkdtempSync('/tmp/bs-open-');
const marker = path.join(dir, 'opened.log');
const openCmd = openCaptureCommand(dir, marker); // capture the launch instead of opening a browser
const srv = spawn('node', [SERVER], { env: { ...process.env, BRAINSTORM_PORT: 3417, BRAINSTORM_DIR: dir, BRAINSTORM_OPEN: '1', BRAINSTORM_OPEN_CMD: openCmd, BRAINSTORM_LIFECYCLE_CHECK_MS: 100000 } });
let out = ''; srv.stdout.on('data', d => out += d.toString());
for (let i = 0; i < 60 && !out.includes('server-started'); i++) await sleep(50);
// First screen, with no browser connected -> should auto-open.
fs.writeFileSync(path.join(dir, 'content', 'first.html'), '<h2>First</h2>');
await waitForFile(marker);
// Second screen -> must NOT open again.
fs.writeFileSync(path.join(dir, 'content', 'second.html'), '<h2>Second</h2>');
await sleep(700);
const lines = fs.existsSync(marker) ? fs.readFileSync(marker, 'utf8').trim().split('\n').filter(Boolean) : [];
// The opened URL must carry the key AND be reachable — a keyless URL hits 403.
let status = 0;
if (lines[0]) {
status = await new Promise(r => require('http').get(lines[0], res => { res.resume(); r(res.statusCode); }).on('error', () => r(0)));
}
await killAndWait(srv);
fs.rmSync(dir, { recursive: true, force: true });
assert.strictEqual(lines.length, 1, 'should open exactly once');
assert(lines[0].includes('3417'), `should open the server URL, got: ${lines[0]}`);
assert(/[?&]key=/.test(lines[0]), `opened URL must carry the session key, got: ${lines[0]}`);
assert.strictEqual(status, 200, 'the opened URL must be reachable (valid key), not the 403 page');
});
await test('does NOT auto-open unless approved (BRAINSTORM_OPEN unset)', async () => {
const dir = fs.mkdtempSync('/tmp/bs-open-');
const marker = path.join(dir, 'opened.log');
const openCmd = openCaptureCommand(dir, marker);
// BRAINSTORM_OPEN intentionally NOT set — auto-open must stay off.
const srv = spawn('node', [SERVER], { env: { ...process.env, BRAINSTORM_PORT: 3418, BRAINSTORM_DIR: dir, BRAINSTORM_OPEN_CMD: openCmd, BRAINSTORM_LIFECYCLE_CHECK_MS: 100000 } });
let out = ''; srv.stdout.on('data', d => out += d.toString());
for (let i = 0; i < 60 && !out.includes('server-started'); i++) await sleep(50);
fs.writeFileSync(path.join(dir, 'content', 'first.html'), '<h2>First</h2>');
await sleep(700);
await killAndWait(srv);
const opened = fs.existsSync(marker);
fs.rmSync(dir, { recursive: true, force: true });
assert(!opened, 'must not open the browser without explicit approval');
});
await test('unauthenticated requests do not defeat the idle timeout', async () => {
const dir = fs.mkdtempSync('/tmp/bs-life-');
const srv = spawn('node', [SERVER], { env: { ...process.env, BRAINSTORM_PORT: 3419, BRAINSTORM_DIR: dir, BRAINSTORM_TOKEN: 'authtok', BRAINSTORM_IDLE_TIMEOUT_MS: 400, BRAINSTORM_LIFECYCLE_CHECK_MS: 100 } });
let out = ''; srv.stdout.on('data', d => out += d.toString());
let exited = false; srv.on('exit', () => { exited = true; });
for (let i = 0; i < 60 && !out.includes('server-started'); i++) await sleep(50);
// Flood with UNAUTHENTICATED (keyless → 403) requests. These must NOT count
// as activity, so the idle timeout still fires and the process exits.
const hammer = setInterval(() => { require('http').get('http://localhost:3419/', r => r.resume()).on('error', () => {}); }, 60);
for (let i = 0; i < 40 && !exited; i++) await sleep(100);
clearInterval(hammer);
if (!exited) await killAndWait(srv);
fs.rmSync(dir, { recursive: true, force: true });
assert(exited, 'idle shutdown must still fire despite a flood of unauthenticated requests');
});
console.log(`\n--- Results: ${passed} passed, ${failed} failed ---`);
if (failed > 0) process.exit(1);
}
runTests().catch(err => { console.error('Test failed:', err); process.exit(1); });
+1 -1
View File
@@ -2,7 +2,7 @@
"name": "brainstorm-server-tests",
"version": "1.0.0",
"scripts": {
"test": "node server.test.js"
"test": "node ws-protocol.test.js && node helper.test.js && node browser-launcher.test.js && node auth.test.js && node server.test.js && node lifecycle.test.js && bash start-server.test.sh && bash stop-server.test.sh"
},
"dependencies": {
"ws": "^8.19.0"
+173 -16
View File
@@ -20,6 +20,9 @@ const TEST_PORT = 3334;
const TEST_DIR = '/tmp/brainstorm-test';
const CONTENT_DIR = path.join(TEST_DIR, 'content');
const STATE_DIR = path.join(TEST_DIR, 'state');
// Fixed session key so the test client can authenticate (see auth.test.js for
// the security behavior itself; here we just need authorized requests).
const TOKEN = 'testtoken-server-0123456789abcdef';
function cleanup() {
if (fs.existsSync(TEST_DIR)) {
@@ -33,7 +36,8 @@ async function sleep(ms) {
async function fetch(url) {
return new Promise((resolve, reject) => {
http.get(url, (res) => {
const headers = { Cookie: `brainstorm-key-${TEST_PORT}=${TOKEN}` };
http.get(url, { headers }, (res) => {
let data = '';
res.on('data', chunk => data += chunk);
res.on('end', () => resolve({
@@ -47,7 +51,7 @@ async function fetch(url) {
function startServer() {
return spawn('node', [SERVER_PATH], {
env: { ...process.env, BRAINSTORM_PORT: TEST_PORT, BRAINSTORM_DIR: TEST_DIR }
env: { ...process.env, BRAINSTORM_PORT: TEST_PORT, BRAINSTORM_DIR: TEST_DIR, BRAINSTORM_TOKEN: TOKEN }
});
}
@@ -69,6 +73,43 @@ async function waitForServer(server) {
});
}
class SkipTest extends Error {
constructor(message) {
super(message);
this.skip = true;
}
}
function skip(message) {
throw new SkipTest(message);
}
function serverStartedMessage(out) {
const line = out.trim().split('\n').find(l => l.includes('server-started'));
assert(line, 'server-started JSON should be present');
return JSON.parse(line);
}
function assertStartedOnExpectedPort(out) {
const msg = serverStartedMessage(out);
assert.strictEqual(
msg.port,
TEST_PORT,
`server.test.js expected fixed port ${TEST_PORT}, got ${msg.port}; fixed-port tests must not run through fallback`
);
return msg;
}
function ensureSymlinkWorks(target, link) {
try {
fs.symlinkSync(target, link);
fs.unlinkSync(link);
} catch (e) {
try { fs.unlinkSync(link); } catch (ignore) {}
skip(`symlink creation unavailable on this host: ${e.message}`);
}
}
async function runTests() {
cleanup();
@@ -76,15 +117,22 @@ async function runTests() {
let stdoutAccum = '';
server.stdout.on('data', (data) => { stdoutAccum += data.toString(); });
const { stdout: initialStdout } = await waitForServer(server);
let initialStdout = '';
let passed = 0;
let failed = 0;
let skipped = 0;
function test(name, fn) {
return fn().then(() => {
console.log(` PASS: ${name}`);
passed++;
}).catch(e => {
if (e.skip) {
console.log(` SKIP: ${name}`);
console.log(` ${e.message}`);
skipped++;
return;
}
console.log(` FAIL: ${name}`);
console.log(` ${e.message}`);
failed++;
@@ -92,11 +140,15 @@ async function runTests() {
}
try {
const { stdout } = await waitForServer(server);
initialStdout = stdout;
assertStartedOnExpectedPort(initialStdout);
// ========== Server Startup ==========
console.log('\n--- Server Startup ---');
await test('outputs server-started JSON on startup', () => {
const msg = JSON.parse(initialStdout.trim());
const msg = serverStartedMessage(initialStdout);
assert.strictEqual(msg.type, 'server-started');
assert.strictEqual(msg.port, TEST_PORT);
assert(msg.url, 'Should include URL');
@@ -179,6 +231,95 @@ async function runTests() {
assert(!res.body.includes('"not"'), 'Should not serve JSON');
});
await test('ignores macOS resource-fork dotfiles (._*.html) when serving', async () => {
// On macOS/ExFAT/SMB, the OS writes ._name.html sidecar files holding
// binary metadata. They end with .html but must never be served as a screen.
fs.writeFileSync(path.join(CONTENT_DIR, 'real-screen.html'), '<h2>Real Screen Content</h2>');
await sleep(100);
fs.writeFileSync(path.join(CONTENT_DIR, '._real-screen.html'), 'Mac OS X resource fork garbage');
await sleep(300);
const res = await fetch(`http://localhost:${TEST_PORT}/`);
assert(res.body.includes('Real Screen Content'), 'should serve the real screen, not the newer ._ sidecar');
assert(!res.body.includes('resource fork garbage'), 'must not serve ._*.html dotfile content');
});
await test('does not serve dotfiles via /files/', async () => {
fs.writeFileSync(path.join(CONTENT_DIR, '._secret.html'), 'dotfile body should not be served');
const res = await fetch(`http://localhost:${TEST_PORT}/files/._secret.html`);
assert.strictEqual(res.status, 404, '/files/ must 404 on dotfiles');
});
await test('GET /files/ (empty name) returns 404 and does not crash the server', async () => {
const res = await fetch(`http://localhost:${TEST_PORT}/files/`);
assert.strictEqual(res.status, 404, '/files/ (the content dir) must 404, not EISDIR-crash');
// The server must still be alive afterward.
const alive = await fetch(`http://localhost:${TEST_PORT}/`);
assert.strictEqual(alive.status, 200, 'server must survive a /files/ request');
});
await test('does not serve symlinks that escape content dir via /files/', async () => {
const target = path.join(STATE_DIR, 'server-info');
const link = path.join(CONTENT_DIR, 'linked-server-info.txt');
try { fs.unlinkSync(link); } catch (e) {}
ensureSymlinkWorks(target, link);
fs.symlinkSync(target, link);
const res = await fetch(`http://localhost:${TEST_PORT}/files/linked-server-info.txt`);
assert.strictEqual(res.status, 404, 'symlink to state/server-info must not be served');
assert(!res.body.includes('server-started'), 'response must not include server-info body');
});
await test('does not serve hard links to files outside content dir via /files/', async () => {
const target = path.join(STATE_DIR, 'server-info');
const link = path.join(CONTENT_DIR, 'hard-linked-server-info.txt');
try { fs.unlinkSync(link); } catch (e) {}
fs.linkSync(target, link);
const res = await fetch(`http://localhost:${TEST_PORT}/files/hard-linked-server-info.txt`);
assert.strictEqual(res.status, 404, 'hard link to state/server-info must not be served');
assert(!res.body.includes('server-started'), 'response must not include server-info body');
});
await test('does not serve symlinks that escape content dir via root screen selection', async () => {
const target = path.join(STATE_DIR, 'server-info');
const link = path.join(CONTENT_DIR, 'root-linked-server-info.html');
try { fs.unlinkSync(link); } catch (e) {}
ensureSymlinkWorks(target, link);
fs.symlinkSync(target, link);
const future = new Date(Date.now() + 2000);
fs.utimesSync(target, future, future);
await sleep(300);
const res = await fetch(`http://localhost:${TEST_PORT}/`);
assert.strictEqual(res.status, 200);
assert(!res.body.includes('"type":"server-started"'), 'root screen must not serve state/server-info through a symlink');
assert(!res.body.includes('"state_dir"'), 'root screen must not include server-info body');
});
await test('does not serve hard links that escape content dir via root screen selection', async () => {
const target = path.join(STATE_DIR, 'server-info');
const link = path.join(CONTENT_DIR, 'root-hard-linked-server-info.html');
try { fs.unlinkSync(link); } catch (e) {}
try {
fs.linkSync(target, link);
} catch (e) {
skip(`hardlink creation unavailable on this host: ${e.message}`);
}
const linkStat = fs.lstatSync(link);
if (linkStat.nlink <= 1) {
skip(`hardlink nlink did not expose multiple links: ${linkStat.nlink}`);
}
const future = new Date(Date.now() + 3000);
fs.utimesSync(target, future, future);
await sleep(300);
const res = await fetch(`http://localhost:${TEST_PORT}/`);
assert.strictEqual(res.status, 200);
assert(!res.body.includes('"type":"server-started"'), 'root screen must not serve state/server-info through a hardlink');
assert(!res.body.includes('"state_dir"'), 'root screen must not include server-info body');
});
await test('returns 404 for non-root paths', async () => {
const res = await fetch(`http://localhost:${TEST_PORT}/other`);
assert.strictEqual(res.status, 404);
@@ -188,7 +329,7 @@ async function runTests() {
console.log('\n--- WebSocket Communication ---');
await test('accepts WebSocket upgrade on /', async () => {
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise((resolve, reject) => {
ws.on('open', resolve);
ws.on('error', reject);
@@ -198,7 +339,7 @@ async function runTests() {
await test('relays user events to stdout with source field', async () => {
stdoutAccum = '';
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
ws.send(JSON.stringify({ type: 'click', text: 'Test Button' }));
@@ -214,7 +355,7 @@ async function runTests() {
const eventsFile = path.join(STATE_DIR, 'events');
if (fs.existsSync(eventsFile)) fs.unlinkSync(eventsFile);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
ws.send(JSON.stringify({ type: 'click', choice: 'b', text: 'Option B' }));
@@ -232,7 +373,7 @@ async function runTests() {
const eventsFile = path.join(STATE_DIR, 'events');
if (fs.existsSync(eventsFile)) fs.unlinkSync(eventsFile);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
ws.send(JSON.stringify({ type: 'hover', text: 'Something' }));
@@ -244,8 +385,8 @@ async function runTests() {
});
await test('handles multiple concurrent WebSocket clients', async () => {
const ws1 = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws2 = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws1 = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
const ws2 = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await Promise.all([
new Promise(resolve => ws1.on('open', resolve)),
new Promise(resolve => ws2.on('open', resolve))
@@ -270,7 +411,7 @@ async function runTests() {
});
await test('cleans up closed clients from broadcast list', async () => {
const ws1 = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws1 = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws1.on('open', resolve));
ws1.close();
await sleep(100);
@@ -282,7 +423,7 @@ async function runTests() {
});
await test('handles malformed JSON from client gracefully', async () => {
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
// Send invalid JSON — server should not crash
@@ -299,7 +440,7 @@ async function runTests() {
console.log('\n--- File Watching ---');
await test('sends reload on new .html file', async () => {
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
let gotReload = false;
@@ -319,7 +460,7 @@ async function runTests() {
fs.writeFileSync(filePath, '<h2>Original</h2>');
await sleep(500);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
let gotReload = false;
@@ -335,7 +476,7 @@ async function runTests() {
});
await test('does NOT send reload for non-.html files', async () => {
const ws = new WebSocket(`ws://localhost:${TEST_PORT}`);
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
let gotReload = false;
@@ -350,6 +491,22 @@ async function runTests() {
ws.close();
});
await test('does NOT send reload for ._*.html resource-fork dotfiles', async () => {
const ws = new WebSocket(`ws://localhost:${TEST_PORT}/?key=${TOKEN}`);
await new Promise(resolve => ws.on('open', resolve));
let gotReload = false;
ws.on('message', (data) => {
if (JSON.parse(data.toString()).type === 'reload') gotReload = true;
});
fs.writeFileSync(path.join(CONTENT_DIR, '._sidecar.html'), 'resource fork');
await sleep(500);
assert(!gotReload, 'a ._ dotfile appearing must not trigger a reload');
ws.close();
});
await test('clears state/events on new screen', async () => {
// Create an events file
const eventsFile = path.join(STATE_DIR, 'events');
@@ -411,7 +568,7 @@ async function runTests() {
});
// ========== Summary ==========
console.log(`\n--- Results: ${passed} passed, ${failed} failed ---`);
console.log(`\n--- Results: ${passed} passed, ${failed} failed, ${skipped} skipped ---`);
if (failed > 0) process.exit(1);
} finally {
@@ -0,0 +1,115 @@
#!/usr/bin/env bash
# Fast tests for start-server.sh shell-only platform decisions.
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
START_SCRIPT="$REPO_ROOT/skills/brainstorming/scripts/start-server.sh"
TEST_DIR="${TMPDIR:-/tmp}/brainstorm-start-test-$$"
passed=0
failed=0
cleanup() {
rm -rf "$TEST_DIR"
}
trap cleanup EXIT
pass() {
echo " PASS: $1"
passed=$((passed + 1))
}
fail() {
echo " FAIL: $1"
echo " $2"
failed=$((failed + 1))
}
make_fake_uname() {
local fake_bin="$1"
cat > "$fake_bin/uname" <<'EOF'
#!/usr/bin/env bash
if [[ "${1:-}" == "-s" ]]; then
echo "MINGW64_NT-10.0"
else
/usr/bin/uname "$@"
fi
EOF
chmod +x "$fake_bin/uname"
}
echo ""
echo "--- start-server.sh platform detection ---"
mkdir -p "$TEST_DIR/fake-bin" "$TEST_DIR/project"
make_fake_uname "$TEST_DIR/fake-bin"
cat > "$TEST_DIR/fake-bin/node" <<'EOF'
#!/usr/bin/env bash
echo "CAPTURED_OWNER_PID=${BRAINSTORM_OWNER_PID:-__UNSET__}"
printf 'CAPTURED_ARGV=%s\n' "$@"
exit 0
EOF
chmod +x "$TEST_DIR/fake-bin/node"
captured=$(
PATH="$TEST_DIR/fake-bin:$PATH" \
MSYSTEM="" \
bash "$START_SCRIPT" --project-dir "$TEST_DIR/project" --foreground 2>/dev/null || true
)
owner_pid_value=$(echo "$captured" | grep "CAPTURED_OWNER_PID=" | head -1 | sed 's/CAPTURED_OWNER_PID=//')
if [[ "$owner_pid_value" == "" || "$owner_pid_value" == "__UNSET__" ]]; then
pass "clears BRAINSTORM_OWNER_PID when uname reports a Windows-like shell"
else
fail "clears BRAINSTORM_OWNER_PID when uname reports a Windows-like shell" \
"expected empty or unset, got '$owner_pid_value'"
fi
if echo "$captured" | grep -Eq '^CAPTURED_ARGV=--brainstorm-server-id=[A-Za-z0-9_-]{32,64}$'; then
pass "passes shell-safe server instance id argv"
else
fail "passes shell-safe server instance id argv" \
"expected exact --brainstorm-server-id=<safe id> argv line, got: $captured"
fi
server_id_file=$(find "$TEST_DIR/project/.superpowers/brainstorm" -name server-instance-id -print 2>/dev/null | head -1)
server_id_value=""
if [[ -n "$server_id_file" ]]; then
server_id_value="$(tr -d '\r\n' < "$server_id_file")"
fi
if [[ "$server_id_value" =~ ^[A-Za-z0-9_-]{32,64}$ ]]; then
pass "writes shell-safe server-instance-id state file"
else
fail "writes shell-safe server-instance-id state file" \
"expected valid id in state, got '$server_id_value'"
fi
rm -rf "$TEST_DIR/project"/*
cat > "$TEST_DIR/fake-bin/node" <<'EOF'
#!/usr/bin/env bash
echo "FOREGROUND_MODE=true"
exit 0
EOF
chmod +x "$TEST_DIR/fake-bin/node"
captured=$(
PATH="$TEST_DIR/fake-bin:$PATH" \
MSYSTEM="" \
bash "$START_SCRIPT" --project-dir "$TEST_DIR/project" 2>/dev/null || true
)
if echo "$captured" | grep -q "FOREGROUND_MODE=true"; then
pass "auto-foregrounds when uname reports a Windows-like shell"
else
fail "auto-foregrounds when uname reports a Windows-like shell" \
"expected foreground node path, got: $captured"
fi
echo ""
echo "--- Results: $passed passed, $failed failed ---"
if [[ $failed -gt 0 ]]; then
exit 1
fi
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env bash
# Tests for stop-server.sh PID-ownership safety.
#
# A stale server.pid (e.g. after a reboot, when the kernel has recycled the PID)
# can point at an unrelated, live process. stop-server.sh must verify the PID is
# actually our brainstorm server before signalling it.
set -u
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
STOP="$SCRIPT_DIR/../../skills/brainstorming/scripts/stop-server.sh"
SERVER="$SCRIPT_DIR/../../skills/brainstorming/scripts/server.cjs"
PASS=0; FAIL=0
PIDS=()
DIRS=()
cleanup() {
for pid in "${PIDS[@]}"; do
kill -9 "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
done
for dir in "${DIRS[@]}"; do
rm -rf "$dir"
done
}
trap cleanup EXIT
track_dir() { DIRS+=("$1"); }
track_pid() { PIDS+=("$1"); }
untrack_pid() {
local remove="$1"
local kept=()
local pid
for pid in "${PIDS[@]}"; do
[[ "$pid" == "$remove" ]] || kept+=("$pid")
done
PIDS=("${kept[@]}")
}
new_server_id() {
printf 'testid%026d\n' "$RANDOM"
}
ok() { echo " PASS: $1"; PASS=$((PASS + 1)); }
bad() { echo " FAIL: $1"; echo " $2"; FAIL=$((FAIL + 1)); }
# --- Test 1: an unrelated, reused PID must NOT be killed ---
SESS="$(mktemp -d)"; track_dir "$SESS"; mkdir -p "$SESS/state"
sleep 600 &
UNRELATED=$!
track_pid "$UNRELATED"
disown "$UNRELATED" 2>/dev/null || true
echo "$UNRELATED" > "$SESS/state/server.pid"
OUT="$("$STOP" "$SESS")"
if kill -0 "$UNRELATED" 2>/dev/null; then
case "$OUT" in
*stale_pid*) ok "unrelated reused PID is left alone (stale_pid)" ;;
*) bad "unrelated PID survived but status was not stale_pid" "$OUT" ;;
esac
else
bad "unrelated reused PID was KILLED" "$OUT"
fi
# --- Test 2: a real brainstorm server with matching instance id IS stopped ---
SESS="$(mktemp -d)"; track_dir "$SESS"; mkdir -p "$SESS/content" "$SESS/state"
SERVER_ID="$(new_server_id)"
printf '%s\n' "$SERVER_ID" > "$SESS/state/server-instance-id"
BRAINSTORM_DIR="$SESS" BRAINSTORM_PORT=3399 node "$SERVER" "--brainstorm-server-id=$SERVER_ID" > /dev/null 2>&1 &
SRV=$!
track_pid "$SRV"
disown "$SRV" 2>/dev/null || true
for _ in $(seq 1 40); do kill -0 "$SRV" 2>/dev/null && break; sleep 0.1; done
sleep 0.4
echo "$SRV" > "$SESS/state/server.pid"
OUT="$("$STOP" "$SESS")"
sleep 0.3
if kill -0 "$SRV" 2>/dev/null; then
bad "real brainstorm server still running after stop" "$OUT"
else
wait "$SRV" 2>/dev/null || true
untrack_pid "$SRV"
case "$OUT" in
*stopped*) ok "real brainstorm server with matching instance id is stopped" ;;
*) bad "server stopped but status was not 'stopped'" "$OUT" ;;
esac
fi
# --- Test 2b: persistent sessions stop with explicit stopped metadata ---
SESS="$(mktemp -d "$SCRIPT_DIR/.stop-persistent.XXXXXX")"; track_dir "$SESS"; mkdir -p "$SESS/content" "$SESS/state"
SERVER_ID="$(new_server_id)"
printf '%s\n' "$SERVER_ID" > "$SESS/state/server-instance-id"
BRAINSTORM_DIR="$SESS" BRAINSTORM_PORT=0 node "$SERVER" "--brainstorm-server-id=$SERVER_ID" > /dev/null 2>&1 &
SRV=$!
track_pid "$SRV"
disown "$SRV" 2>/dev/null || true
for _ in $(seq 1 40); do
[[ -f "$SESS/state/server-info" ]] && break
sleep 0.1
done
echo "$SRV" > "$SESS/state/server.pid"
OUT="$("$STOP" "$SESS")"
sleep 0.3
if kill -0 "$SRV" 2>/dev/null; then
bad "persistent brainstorm server still running after stop" "$OUT"
else
wait "$SRV" 2>/dev/null || true
untrack_pid "$SRV"
if [[ -f "$SESS/state/server-info" ]]; then
bad "persistent stop clears server-info" "server-info still exists after: $OUT"
elif [[ ! -f "$SESS/state/server-stopped" ]]; then
bad "persistent stop writes server-stopped" "server-stopped missing after: $OUT"
elif grep -q '"reason":"stop-server.sh"' "$SESS/state/server-stopped"; then
ok "persistent stop clears alive metadata and writes server-stopped"
else
bad "persistent stop writes stop reason" "$(cat "$SESS/state/server-stopped" 2>/dev/null || true)"
fi
fi
# --- Test 3: no pid file ---
SESS="$(mktemp -d)"; track_dir "$SESS"; mkdir -p "$SESS/state"
OUT="$("$STOP" "$SESS")"
case "$OUT" in
*not_running*) ok "missing pid file reports not_running" ;;
*) bad "missing pid file: unexpected status" "$OUT" ;;
esac
# --- Test 4: a node server.cjs impostor with missing instance id is spared ---
SESS="$(mktemp -d)"; track_dir "$SESS"; mkdir -p "$SESS/state"
( exec -a "node server.cjs" sleep 600 ) &
IMPOSTOR=$!
track_pid "$IMPOSTOR"
disown "$IMPOSTOR" 2>/dev/null || true
echo "$IMPOSTOR" > "$SESS/state/server.pid"
OUT="$("$STOP" "$SESS")"
if kill -0 "$IMPOSTOR" 2>/dev/null; then
case "$OUT" in
*stale_pid*) ok "missing instance id leaves node server.cjs impostor alone" ;;
*) bad "impostor survived but status was not stale_pid" "$OUT" ;;
esac
else
bad "killed a node server.cjs impostor with missing instance id" "$OUT"
fi
# --- Test 5: a node server.cjs impostor with wrong instance id is spared ---
SESS="$(mktemp -d)"; track_dir "$SESS"; mkdir -p "$SESS/state"
EXPECTED_ID="$(new_server_id)"
WRONG_ID="$(new_server_id)"
printf '%s\n' "$EXPECTED_ID" > "$SESS/state/server-instance-id"
( exec -a "node server.cjs --brainstorm-server-id=$WRONG_ID" sleep 600 ) &
IMPOSTOR=$!
track_pid "$IMPOSTOR"
disown "$IMPOSTOR" 2>/dev/null || true
echo "$IMPOSTOR" > "$SESS/state/server.pid"
OUT="$("$STOP" "$SESS")"
if kill -0 "$IMPOSTOR" 2>/dev/null; then
case "$OUT" in
*stale_pid*) ok "wrong instance id leaves node server.cjs impostor alone" ;;
*) bad "wrong-id impostor survived but status was not stale_pid" "$OUT" ;;
esac
else
bad "killed a node server.cjs impostor with wrong instance id" "$OUT"
fi
# --- Test 6: malformed instance id is fail-closed ---
SESS="$(mktemp -d)"; track_dir "$SESS"; mkdir -p "$SESS/state"
printf '%s\n' 'bad id with spaces' > "$SESS/state/server-instance-id"
( exec -a "node server.cjs --brainstorm-server-id=bad-id-with-spaces" sleep 600 ) &
IMPOSTOR=$!
track_pid "$IMPOSTOR"
disown "$IMPOSTOR" 2>/dev/null || true
echo "$IMPOSTOR" > "$SESS/state/server.pid"
OUT="$("$STOP" "$SESS")"
if kill -0 "$IMPOSTOR" 2>/dev/null; then
case "$OUT" in
*stale_pid*) ok "malformed instance id is fail-closed" ;;
*) bad "malformed-id impostor survived but status was not stale_pid" "$OUT" ;;
esac
else
bad "killed process despite malformed instance id" "$OUT"
fi
echo "--- Results: $PASS passed, $FAIL failed ---"
[ "$FAIL" -eq 0 ] || exit 1
@@ -80,14 +80,23 @@ get_port_from_info() {
grep -o '"port":[0-9]*' "$1/state/server-info" | head -1 | sed 's/"port"://'
}
get_key_from_info() {
grep -o '"url":"[^"]*key=[^"]*' "$1/state/server-info" | head -1 | sed 's/.*key=//'
}
http_check() {
local port="$1"
node -e "
local key="${2:-}"
node - "$port" "$key" <<'NODE'
const http = require('http');
http.get('http://localhost:$port/', (res) => {
const port = Number(process.argv[2]);
const key = process.argv[3] || '';
const path = key ? '/?key=' + encodeURIComponent(key) : '/';
http.get({ hostname: '127.0.0.1', port, path }, (res) => {
res.resume();
process.exit(res.statusCode === 200 ? 0 : 1);
}).on('error', () => process.exit(1));
" 2>/dev/null
NODE
}
# ========== Platform Detection ==========
@@ -153,6 +162,7 @@ if [[ "$is_windows" == "true" ]]; then
cat > "$FAKE_NODE_DIR/node" <<'FAKENODE'
#!/usr/bin/env bash
echo "CAPTURED_OWNER_PID=${BRAINSTORM_OWNER_PID:-__UNSET__}"
printf 'CAPTURED_ARGV=%s\n' "$@"
exit 0
FAKENODE
chmod +x "$FAKE_NODE_DIR/node"
@@ -167,6 +177,13 @@ FAKENODE
"Expected empty or unset, got '$owner_pid_value'"
fi
if echo "$captured" | grep -Eq '^CAPTURED_ARGV=--brainstorm-server-id=[A-Za-z0-9_-]{32,64}$'; then
pass "start-server.sh passes server instance id argv on Windows"
else
fail "start-server.sh passes server instance id argv on Windows" \
"Expected --brainstorm-server-id=<safe id>, output: $captured"
fi
rm -rf "$FAKE_NODE_DIR" "$TEST_DIR/session"
else
skip "start-server.sh passes empty BRAINSTORM_OWNER_PID" "not on Windows"
@@ -227,6 +244,7 @@ else
pass "Server starts successfully with empty OWNER_PID"
SERVER_PORT=$(get_port_from_info "$TEST_DIR/survival")
SERVER_KEY=$(get_key_from_info "$TEST_DIR/survival")
sleep 75
@@ -237,11 +255,11 @@ else
"Server died. Log tail: $(tail -5 "$TEST_DIR/survival/.server.log" 2>/dev/null)"
fi
if http_check "$SERVER_PORT"; then
if http_check "$SERVER_PORT" "$SERVER_KEY"; then
pass "Server responds to HTTP after lifecycle check window"
else
fail "Server responds to HTTP after lifecycle check window" \
"HTTP request to port $SERVER_PORT failed"
"Authenticated HTTP request to port $SERVER_PORT failed"
fi
if grep -q "owner process exited" "$TEST_DIR/survival/.server.log" 2>/dev/null; then
@@ -325,23 +343,33 @@ echo ""
echo "--- Clean Shutdown ---"
mkdir -p "$TEST_DIR/stop-test/state"
STOP_TEST_ID="$(printf 'windowsstop%021d\n' "$RANDOM")"
printf '%s\n' "$STOP_TEST_ID" > "$TEST_DIR/stop-test/state/server-instance-id"
BRAINSTORM_DIR="$TEST_DIR/stop-test" \
BRAINSTORM_HOST="127.0.0.1" \
BRAINSTORM_URL_HOST="localhost" \
BRAINSTORM_OWNER_PID="" \
BRAINSTORM_PORT=$((49152 + RANDOM % 16383)) \
node "$SERVER_SCRIPT" > "$TEST_DIR/stop-test/.server.log" 2>&1 &
node "$SERVER_SCRIPT" "--brainstorm-server-id=$STOP_TEST_ID" > "$TEST_DIR/stop-test/.server.log" 2>&1 &
STOP_TEST_PID=$!
disown "$STOP_TEST_PID" 2>/dev/null || true
echo "$STOP_TEST_PID" > "$TEST_DIR/stop-test/state/server.pid"
if ! wait_for_server_info "$TEST_DIR/stop-test"; then
fail "Stop-test server starts" "Server did not start"
kill "$STOP_TEST_PID" 2>/dev/null || true
wait "$STOP_TEST_PID" 2>/dev/null || true
STOP_TEST_PID=""
else
bash "$STOP_SCRIPT" "$TEST_DIR/stop-test" >/dev/null 2>&1 || true
sleep 1
for _ in $(seq 1 10); do
if ! kill -0 "$STOP_TEST_PID" 2>/dev/null; then
wait "$STOP_TEST_PID" 2>/dev/null || true
break
fi
sleep 0.1
done
if ! kill -0 "$STOP_TEST_PID" 2>/dev/null; then
pass "stop-server.sh cleanly stops the server"
@@ -329,6 +329,21 @@ function runTests() {
assert.strictEqual(result.payload.length, 65536);
});
test('rejects oversized 64-bit frames before payload allocation', () => {
const mask = Buffer.from([0x00, 0x00, 0x00, 0x00]);
const header = Buffer.alloc(14);
header[0] = 0x81; // FIN + TEXT
header[1] = 0x80 | 127; // masked, 64-bit length
header.writeBigUInt64BE(BigInt(ws.MAX_FRAME_PAYLOAD_BYTES) + 1n, 2);
mask.copy(header, 10);
assert.throws(
() => ws.decodeFrame(header),
/exceeds maximum allowed size/i,
'oversized advertised payload must be rejected from header alone'
);
});
// ========== Close Frame with Status Code ==========
console.log('\n--- Close Frame Details ---');
@@ -175,6 +175,7 @@ write_upstream_fixture() {
mkdir -p \
"$repo/.codex-plugin" \
"$repo/.kimi-plugin" \
"$repo/.private-journal" \
"$repo/assets" \
"$repo/evals/drill" \
@@ -210,6 +211,13 @@ EOF
"name": "superpowers",
"version": "$MANIFEST_VERSION"
}
EOF
cat > "$repo/.kimi-plugin/plugin.json" <<EOF
{
"name": "superpowers",
"version": "$MANIFEST_VERSION"
}
EOF
cat > "$repo/assets/superpowers-small.svg" <<'EOF'
@@ -267,6 +275,7 @@ EOF
git -C "$repo" add \
.codex-plugin/plugin.json \
.kimi-plugin/plugin.json \
.gitignore \
assets/app-icon.png \
assets/superpowers-small.svg \
@@ -415,10 +424,15 @@ EOF
write_stale_ignored_destination_fixture() {
local repo="$1"
mkdir -p "$repo/plugins/superpowers/.private-journal"
mkdir -p \
"$repo/plugins/superpowers/.kimi-plugin" \
"$repo/plugins/superpowers/.private-journal"
printf 'fixture keep\n' > "$repo/plugins/superpowers/.fixture-keep"
printf '{"name":"stale-kimi"}\n' > "$repo/plugins/superpowers/.kimi-plugin/plugin.json"
printf 'stale ignored leak\n' > "$repo/plugins/superpowers/.private-journal/leak.txt"
git -C "$repo" add plugins/superpowers/.fixture-keep
git -C "$repo" add \
plugins/superpowers/.fixture-keep \
plugins/superpowers/.kimi-plugin/plugin.json
commit_fixture "$repo" "Initial stale ignored destination fixture"
}
@@ -618,6 +632,7 @@ main() {
assert_contains "$preview_output" "Version: $MANIFEST_VERSION" "Preview uses manifest version"
assert_not_contains "$preview_output" "Version: $PACKAGE_VERSION" "Preview does not use package.json version"
assert_contains "$preview_section" ".codex-plugin/plugin.json" "Preview includes manifest path"
assert_not_contains "$preview_section" ".kimi-plugin/plugin.json" "Preview excludes Kimi manifest from Codex sync"
assert_contains "$preview_section" "assets/superpowers-small.svg" "Preview includes SVG asset"
assert_contains "$preview_section" "assets/app-icon.png" "Preview includes PNG asset"
assert_contains "$preview_section" "hooks/hooks-codex.json" "Preview includes Codex hook manifest"
@@ -644,6 +659,7 @@ main() {
echo ""
echo "Convergence assertions..."
assert_equals "$stale_preview_status" "0" "Stale ignored destination preview exits successfully"
assert_matches "$stale_preview_section" "\\*deleting +\\.kimi-plugin/plugin\\.json" "Preview deletes stale Kimi manifest from Codex plugin"
assert_matches "$stale_preview_section" "\\*deleting +\\.private-journal/leak\\.txt" "Preview deletes stale ignored destination file"
echo ""
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
bash "$SCRIPT_DIR/test-plugin-manifest.sh"
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
MANIFEST="$REPO_ROOT/.kimi-plugin/plugin.json"
python3 - "$MANIFEST" <<'PY'
import json
import sys
from pathlib import Path
manifest_path = Path(sys.argv[1])
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
def assert_equal(actual, expected, label):
if actual != expected:
raise AssertionError(f"{label}: expected {expected!r}, got {actual!r}")
def assert_present(text, needle, label):
if needle not in text:
raise AssertionError(f"{label}: missing {needle!r}")
assert_equal(manifest.get("name"), "superpowers", "plugin name")
assert_equal(manifest.get("skills"), "./skills/", "skills path")
assert_equal(
manifest.get("sessionStart", {}).get("skill"),
"using-superpowers",
"sessionStart.skill",
)
instructions = manifest.get("skillInstructions")
if not isinstance(instructions, str) or not instructions.strip():
raise AssertionError("skillInstructions must be a non-empty string")
for token in [
"AskUserQuestion",
"TodoList",
"Agent",
"Skill",
"Read",
"Write",
"Edit",
"Bash",
"Grep",
"Glob",
"FetchURL",
"WebSearch",
]:
assert_present(instructions, token, "skillInstructions")
version_config = json.loads(
(manifest_path.parents[1] / ".version-bump.json").read_text(encoding="utf-8")
)
version_entries = version_config.get("files")
if not isinstance(version_entries, list):
raise AssertionError(".version-bump.json must contain files list")
if not any(
entry.get("path") == ".kimi-plugin/plugin.json" and entry.get("field") == "version"
for entry in version_entries
if isinstance(entry, dict)
):
raise AssertionError(
".version-bump.json must update .kimi-plugin/plugin.json version"
)
unsupported_fields = [
"tools",
"commands",
"hooks",
"apps",
"inject",
"configFile",
"config_file",
"bootstrap",
]
present_unsupported = sorted(field for field in unsupported_fields if field in manifest)
if present_unsupported:
raise AssertionError(
"unsupported Kimi runtime fields present: "
+ ", ".join(present_unsupported)
)
print("Kimi plugin manifest looks good")
PY
+179
View File
@@ -0,0 +1,179 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
SCRIPT_UNDER_TEST="$REPO_ROOT/scripts/lint-shell.sh"
FAILURES=0
TEST_ROOT="$(mktemp -d)"
cleanup() {
rm -rf "$TEST_ROOT"
}
trap cleanup EXIT
pass() {
echo " [PASS] $1"
}
fail() {
echo " [FAIL] $1"
FAILURES=$((FAILURES + 1))
}
assert_contains() {
local haystack="$1"
local needle="$2"
local description="$3"
if printf '%s' "$haystack" | grep -Fq -- "$needle"; then
pass "$description"
else
fail "$description"
echo " expected to find: $needle"
echo " in:"
printf '%s\n' "$haystack" | sed 's/^/ /'
fi
}
assert_not_contains() {
local haystack="$1"
local needle="$2"
local description="$3"
if printf '%s' "$haystack" | grep -Fq -- "$needle"; then
fail "$description"
echo " did not expect to find: $needle"
echo " in:"
printf '%s\n' "$haystack" | sed 's/^/ /'
else
pass "$description"
fi
}
configure_git_identity() {
local repo="$1"
git -C "$repo" config user.name "Test Bot"
git -C "$repo" config user.email "test@example.com"
}
write_stub_tool() {
local path="$1"
local name="$2"
cat >"$path" <<EOF
#!/usr/bin/env bash
{
printf '${name}:'
for arg in "\$@"; do
printf ' <%s>' "\$arg"
done
printf '\n'
} >> "\$SUPERPOWERS_SHELL_LINT_TEST_LOG"
exit 0
EOF
chmod +x "$path"
}
make_fixture_repo() {
local repo="$1"
git init -q -b main "$repo"
configure_git_identity "$repo"
mkdir -p "$repo/hooks"
cat >"$repo/tracked.sh" <<'EOF'
#!/usr/bin/env bash
echo "tracked"
EOF
cat >"$repo/hooks/session-start" <<'EOF'
#!/bin/sh
echo "extensionless"
EOF
cat >"$repo/README.md" <<'EOF'
# Fixture
```bash
echo "not a shell script"
```
EOF
cat >"$repo/untracked.sh" <<'EOF'
#!/usr/bin/env bash
echo "untracked"
EOF
git -C "$repo" add tracked.sh hooks/session-start README.md
git -C "$repo" commit -q -m "fixture"
printf '\necho "changed"\n' >>"$repo/tracked.sh"
printf '\necho "changed extensionless"\n' >>"$repo/hooks/session-start"
}
run_lint_shell() {
local repo="$1"
local fakebin="$2"
local log="$3"
shift 3
(
cd "$repo"
PATH="$fakebin:$PATH" \
SUPERPOWERS_SHELL_LINT_TEST_LOG="$log" \
bash "$SCRIPT_UNDER_TEST" "$@"
)
}
echo "Shell lint script tests"
fixture="$TEST_ROOT/repo"
fakebin="$TEST_ROOT/bin"
log="$TEST_ROOT/tool.log"
mkdir -p "$fixture" "$fakebin"
: >"$log"
write_stub_tool "$fakebin/shellcheck" "shellcheck"
write_stub_tool "$fakebin/shfmt" "shfmt"
make_fixture_repo "$fixture"
if output="$(run_lint_shell "$fixture" "$fakebin" "$log" 2>&1)"; then
pass "lint-shell check mode exits successfully with stub tools"
else
fail "lint-shell check mode exits successfully with stub tools"
printf '%s\n' "$output" | sed 's/^/ /'
fi
tool_log="$(cat "$log")"
assert_contains "$output" "Linting 3 shell files" "reports changed shell file count"
assert_not_contains "$tool_log" "shfmt:" "does not run shfmt in lint mode"
assert_contains "$tool_log" "shellcheck:" "runs ShellCheck"
assert_contains "$tool_log" "<--severity=warning>" "uses warning severity as the baseline"
assert_contains "$tool_log" "<--external-sources>" "allows ShellCheck to follow sourced files"
assert_contains "$tool_log" "<--source-path=SCRIPTDIR>" "resolves ShellCheck sources relative to each script"
assert_contains "$tool_log" "<hooks/session-start>" "includes changed extensionless shell shebang file"
assert_contains "$tool_log" "<tracked.sh>" "includes changed tracked .sh file"
assert_contains "$tool_log" "<untracked.sh>" "includes untracked shell files by default"
assert_not_contains "$tool_log" "README.md" "ignores Markdown with shell snippets"
: >"$log"
if output="$(run_lint_shell "$fixture" "$fakebin" "$log" --all --format 2>&1)"; then
pass "lint-shell --format exits successfully with stub tools"
else
fail "lint-shell --format exits successfully with stub tools"
printf '%s\n' "$output" | sed 's/^/ /'
fi
tool_log="$(cat "$log")"
assert_contains "$tool_log" "<-w>" "uses shfmt write mode with --format"
assert_contains "$tool_log" "shellcheck:" "runs ShellCheck after --format"
assert_contains "$tool_log" "<--severity=warning>" "keeps warning severity after --format"
assert_contains "$tool_log" "<hooks/session-start>" "--all includes tracked extensionless shell shebang file"
assert_contains "$tool_log" "<tracked.sh>" "--all includes tracked .sh file"
assert_not_contains "$tool_log" "untracked.sh" "--all ignores untracked shell files"
if [[ "$FAILURES" -eq 0 ]]; then
echo "All shell lint script tests passed"
else
echo "$FAILURES shell lint script test(s) failed"
exit 1
fi