355 lines
15 KiB
Markdown
355 lines
15 KiB
Markdown
# Spec C — Funnel Stats & No-Silent-Truncation Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans. Steps use checkbox (`- [ ]`) syntax.
|
|
> **Commits excluded** per user instruction — no `git commit` steps; each task ends with a test-green checkpoint.
|
|
|
|
**Goal:** Make coverage-bounding outputs report a balanced funnel (`found = processed + dropped`) with a mandatory `dropped_reason`, enforced at SubagentStop for the 4 key judge/research agents — so silent truncation becomes visible (audit gap G4).
|
|
|
|
**Architecture:** Extend the shared `wiki_rules.py` with `validate_stats_block` (funnel-balance + dropped-reason). `wiki_claim_gate.subagent_stop_gate` validates any subagent output carrying a `wiki-stats` marker (reuses the Spec B SubagentStop path). Add a no-silent-truncation contract to `rules/reporting-standards.md`; add a `## Stats` `wiki-stats` block to 4 agents + a `## Stats` funnel to `/ingest`. Deterministic core unit-tested; agent prose mirrored 3-platform by hand.
|
|
|
|
**Tech Stack:** Python 3 stdlib (`re`, `json`, `unittest`, `subprocess`). Claude Code hooks. Markdown agent/command specs.
|
|
|
|
**Spec:** `docs/superpowers/specs/2026-06-06-spec-c-funnel-stats-no-silent-truncation-design.md`
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
- **Modify** `.claude/hooks/wiki_rules.py` — `parse_stats_block`, `validate_stats_block`, `STATS_FENCE_RE`.
|
|
- **Modify** `.claude/hooks/test_wiki_rules.py` — `TestStatsBlock`.
|
|
- **Modify** `.claude/hooks/wiki_claim_gate.py` — `subagent_stop_gate` validates `wiki-stats`.
|
|
- **Modify** `.claude/hooks/test_wiki_claim_gate.py` — `TestSubagentStopStats`.
|
|
- **Modify** `rules/reporting-standards.md` — "No silent truncation" 절.
|
|
- **Modify** `.claude/agents/{coverage-auditor,branch-depth-auditor,wiki-decision-researcher,wiki-research-lane}.md` — `## Stats` block.
|
|
- **Modify** `.claude/commands/ingest.md` — `## Stats` funnel 계약.
|
|
- **Mirror (manual)** the 4 agents → `.agents/plugins/wiki-superpowers/agents/<name>.md` (G3) + `.codex/agents/<name>.md`/`.toml`.
|
|
|
|
---
|
|
|
|
## Task 1: `wiki_rules` — stats block parse + validate
|
|
|
|
**Files:**
|
|
- Modify: `.claude/hooks/wiki_rules.py`
|
|
- Test: `.claude/hooks/test_wiki_rules.py` (append `TestStatsBlock`)
|
|
|
|
- [ ] **Step 1: Write the failing test**
|
|
|
|
Append to `test_wiki_rules.py` before `if __name__`:
|
|
```python
|
|
def _stats(found, processed, dropped, reason=None):
|
|
body = f"agent: coverage-auditor\nfound: {found}\nprocessed: {processed}\ndropped: {dropped}"
|
|
if reason is not None:
|
|
body += f"\ndropped_reason: {reason}"
|
|
return f"```wiki-stats\n{body}\n```"
|
|
|
|
|
|
class TestStatsBlock(unittest.TestCase):
|
|
def test_no_marker_returns_none(self):
|
|
parsed, errors = wr.validate_stats_block("산문, 마커 없음")
|
|
self.assertIsNone(parsed)
|
|
self.assertEqual(errors, [])
|
|
|
|
def test_balanced_ok(self):
|
|
parsed, errors = wr.validate_stats_block(_stats(12, 10, 2, "2 out-of-scope"))
|
|
self.assertEqual(errors, [])
|
|
self.assertEqual(parsed["agent"], "coverage-auditor")
|
|
|
|
def test_imbalance_flagged(self):
|
|
_, errors = wr.validate_stats_block(_stats(12, 10, 0))
|
|
self.assertTrue(any("불균형" in e for e in errors))
|
|
|
|
def test_dropped_without_reason_flagged(self):
|
|
_, errors = wr.validate_stats_block(_stats(12, 10, 2))
|
|
self.assertTrue(any("dropped_reason" in e for e in errors))
|
|
|
|
def test_non_integer_flagged(self):
|
|
block = "```wiki-stats\nagent: x\nfound: many\nprocessed: 1\ndropped: 0\n```"
|
|
_, errors = wr.validate_stats_block(block)
|
|
self.assertTrue(any("정수" in e for e in errors))
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify it fails**
|
|
|
|
Run: `python3 .claude/hooks/test_wiki_rules.py -k Stats`
|
|
Expected: FAIL — `AttributeError: ... 'validate_stats_block'`.
|
|
|
|
- [ ] **Step 3: Implement in `wiki_rules.py`**
|
|
|
|
Append (after the `tally_quorum` function):
|
|
```python
|
|
STATS_FENCE_RE = re.compile(r"```wiki-stats\s*\n(.*?)\n```", re.S)
|
|
|
|
|
|
def parse_stats_block(text):
|
|
m = STATS_FENCE_RE.search(text or "")
|
|
if not m:
|
|
return None
|
|
out = {"agent": None, "kv": {}}
|
|
for line in m.group(1).splitlines():
|
|
line = line.strip()
|
|
if not line:
|
|
continue
|
|
kv = re.match(r"([a-z_]+):\s*(.+)$", line)
|
|
if kv:
|
|
k, v = kv.group(1), kv.group(2).strip()
|
|
if k == "agent":
|
|
out["agent"] = v
|
|
else:
|
|
out["kv"][k] = v
|
|
return out
|
|
|
|
|
|
def validate_stats_block(text):
|
|
"""(parsed, errors). parsed None → 마커 없음(통과). errors → SubagentStop 차단.
|
|
funnel 균형(found=processed+dropped) + dropped>0 시 dropped_reason 필수 (no-silent-truncation)."""
|
|
parsed = parse_stats_block(text)
|
|
if parsed is None:
|
|
return None, []
|
|
errors = []
|
|
if not parsed["agent"]:
|
|
errors.append("wiki-stats 블록에 `agent:` 누락")
|
|
nums = {}
|
|
for k in ("found", "processed", "dropped"):
|
|
try:
|
|
nums[k] = int(parsed["kv"].get(k, ""))
|
|
except ValueError:
|
|
errors.append(f"wiki-stats `{k}` 는 정수여야 함 (funnel 필수 필드)")
|
|
if len(nums) == 3:
|
|
if nums["found"] != nums["processed"] + nums["dropped"]:
|
|
errors.append(
|
|
f"funnel 불균형: found({nums['found']}) ≠ processed({nums['processed']}) "
|
|
f"+ dropped({nums['dropped']}) — 조용한 누락 의심"
|
|
)
|
|
if nums["dropped"] > 0 and not parsed["kv"].get("dropped_reason", "").strip():
|
|
errors.append("dropped>0 인데 `dropped_reason` 누락 (no-silent-truncation 위반)")
|
|
return parsed, errors
|
|
```
|
|
|
|
- [ ] **Step 4: Run test to verify it passes**
|
|
|
|
Run: `python3 .claude/hooks/test_wiki_rules.py -k Stats`
|
|
Expected: PASS (5 tests).
|
|
|
|
- [ ] **Step 5: Checkpoint (no commit)** — `python3 .claude/hooks/test_wiki_rules.py` all green.
|
|
|
|
---
|
|
|
|
## Task 2: `claim_gate.subagent_stop_gate` — wiki-stats enforcement
|
|
|
|
**Files:**
|
|
- Modify: `.claude/hooks/wiki_claim_gate.py` (`subagent_stop_gate`)
|
|
- Test: `.claude/hooks/test_wiki_claim_gate.py` (append `TestSubagentStopStats`)
|
|
|
|
- [ ] **Step 1: Write the failing test**
|
|
|
|
Append (the `_run_stop` helper already exists from Spec B):
|
|
```python
|
|
class TestSubagentStopStats(unittest.TestCase):
|
|
def test_imbalanced_stats_blocks(self):
|
|
msg = "x\n```wiki-stats\nagent: coverage-auditor\nfound: 12\nprocessed: 10\ndropped: 0\n```"
|
|
self.assertEqual(_run_stop(msg).returncode, 2)
|
|
|
|
def test_balanced_stats_allows(self):
|
|
msg = "x\n```wiki-stats\nagent: coverage-auditor\nfound: 10\nprocessed: 10\ndropped: 0\n```"
|
|
self.assertEqual(_run_stop(msg).returncode, 0)
|
|
|
|
def test_dropped_without_reason_blocks(self):
|
|
msg = "x\n```wiki-stats\nagent: coverage-auditor\nfound: 12\nprocessed: 10\ndropped: 2\n```"
|
|
self.assertEqual(_run_stop(msg).returncode, 2)
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify it fails**
|
|
|
|
Run: `python3 .claude/hooks/test_wiki_claim_gate.py -k SubagentStopStats`
|
|
Expected: FAIL — `test_imbalanced_stats_blocks` returns 0 (gate not checking stats yet).
|
|
|
|
- [ ] **Step 3: Implement — extend `subagent_stop_gate`**
|
|
|
|
In `.claude/hooks/wiki_claim_gate.py`, add a stats check right after the verdict-block check (before the final `emit_allow()`):
|
|
```python
|
|
# judge verdict 검사(위) 다음 — wiki-stats 마커가 있으면 funnel 검증(없으면 통과).
|
|
sparsed, serr = wiki_rules.validate_stats_block(message)
|
|
if sparsed is not None and serr and not event.get("stop_hook_active"):
|
|
emit_block("wiki-stats 블록 오류:\n- " + "\n- ".join(serr))
|
|
emit_allow()
|
|
```
|
|
(Replace the existing trailing `emit_allow()` with the block above so the stats check precedes it.)
|
|
|
|
- [ ] **Step 4: Run test to verify it passes**
|
|
|
|
Run: `python3 .claude/hooks/test_wiki_claim_gate.py -k SubagentStopStats`
|
|
Expected: PASS (3 tests).
|
|
|
|
- [ ] **Step 5: Checkpoint (no commit)** — `python3 .claude/hooks/test_wiki_claim_gate.py` all green.
|
|
|
|
---
|
|
|
|
## Task 3: `rules/reporting-standards.md` — No silent truncation 계약
|
|
|
|
**Files:**
|
|
- Modify: `rules/reporting-standards.md`
|
|
|
|
- [ ] **Step 1: Append the contract section**
|
|
|
|
Add at the end of `rules/reporting-standards.md`:
|
|
```markdown
|
|
## No silent truncation (funnel 계약)
|
|
|
|
출력이 캡/슬라이스/top-N/skip 으로 coverage 를 bound 하면 **드롭한 수 + 이유**를 반드시 보고한다. funnel 은 균형해야 한다:
|
|
|
|
```
|
|
found = processed + dropped
|
|
```
|
|
|
|
- `found` = 식별한 총 항목. `processed` = 실제 판정한 수(결과 무관 — covered/missing/verified/promoted 모두 포함). `dropped` = 판정하지 않고 의도 제외(이유 필수).
|
|
- **agent 출력**은 `wiki-stats` 블록으로 보고한다(SubagentStop 이 균형·dropped_reason 검증).
|
|
- **command 출력**은 `## Stats` 절로 보고한다.
|
|
- 침묵 누락은 "전부 다뤘다" 는 거짓 신호다 — 제3의 보고되지 않은 버킷을 두지 않는다.
|
|
```
|
|
|
|
- [ ] **Step 2: Verify**
|
|
|
|
Run: `grep -c "No silent truncation" rules/reporting-standards.md`
|
|
Expected: ≥1.
|
|
|
|
- [ ] **Step 3: Checkpoint (no commit).**
|
|
|
|
---
|
|
|
|
## Task 4: 4 agents — `## Stats` block (Claude)
|
|
|
|
**Files:**
|
|
- Modify: `.claude/agents/coverage-auditor.md`, `branch-depth-auditor.md`, `wiki-decision-researcher.md`, `wiki-research-lane.md`
|
|
|
|
- [ ] **Step 1: Add the block to each agent's Output section**
|
|
|
|
In each agent's Output section (for coverage-auditor / branch-depth-auditor, place it **directly after the `## Machine verdict` block** added in Spec B; for decision-researcher / research-lane place it at the end of the Output template), add — with the agent's own name and an example funnel:
|
|
````
|
|
## Stats (funnel — SubagentStop 가 균형·dropped_reason 검증)
|
|
|
|
리포트 끝에 기계 파싱용 funnel 을 **반드시** 방출한다. `found = processed + dropped` 균형 필수, `dropped>0` 면 `dropped_reason` 필수:
|
|
|
|
```wiki-stats
|
|
agent: <이 에이전트 name>
|
|
found: 12
|
|
processed: 10
|
|
dropped: 2
|
|
dropped_reason: 2 out-of-scope (사유)
|
|
```
|
|
````
|
|
Per-agent funnel 의미:
|
|
- `coverage-auditor`: found=governing 관심사 수, processed=covered+delegated+missing, dropped=범위 밖(이유).
|
|
- `branch-depth-auditor`: found=점검한 claim/결정 수, processed=판정 완료, dropped=범위 밖(이유).
|
|
- `wiki-decision-researcher`: found=식별 후보 수, processed=archive 한 수, dropped=bound(N) 초과 제외(이유).
|
|
- `wiki-research-lane`: found=슬라이스 파일 수, processed=정독+추출, dropped=무관/제외(이유).
|
|
|
|
- [ ] **Step 2: Verify all four**
|
|
|
|
Run: `for a in coverage-auditor branch-depth-auditor wiki-decision-researcher wiki-research-lane; do echo -n "$a: "; grep -c "wiki-stats" .claude/agents/$a.md; done`
|
|
Expected: each ≥1.
|
|
|
|
- [ ] **Step 3: Checkpoint (no commit).**
|
|
|
|
---
|
|
|
|
## Task 5: `/ingest` — `## Stats` funnel 계약
|
|
|
|
**Files:**
|
|
- Modify: `.claude/commands/ingest.md`
|
|
|
|
- [ ] **Step 1: Append the funnel contract**
|
|
|
|
Add a new section at the end of `.claude/commands/ingest.md`:
|
|
```markdown
|
|
## 출력: Stats funnel (no-silent-truncation)
|
|
|
|
작업 종료 시 `## Stats` 절을 보고한다 (`rules/reporting-standards.md` No silent truncation 계약):
|
|
|
|
```
|
|
## Stats
|
|
found: <식별한 promotable 항목 수>
|
|
processed: <canonical 로 promote 한 수>
|
|
dropped: <추출 안 한 수>
|
|
dropped_reason: <항목별 제외 사유 (raw 보존 / 잡담 / abandoned / 등)>
|
|
```
|
|
|
|
`found = processed + dropped` 균형 필수. daily/branch 특수처리에서 "추출 안 함" 으로 raw 에 남긴 항목도 `dropped` 에 카운트하고 사유를 적는다 — 무엇을 안 옮겼는지 보이게.
|
|
```
|
|
|
|
- [ ] **Step 2: Verify**
|
|
|
|
Run: `grep -c "Stats funnel" .claude/commands/ingest.md`
|
|
Expected: ≥1.
|
|
|
|
- [ ] **Step 3: Checkpoint (no commit).**
|
|
|
|
---
|
|
|
|
## Task 6: 3-platform manual mirror (4 agents)
|
|
|
|
**Files (mirror the `## Stats` block from Task 4):**
|
|
- `.agents/plugins/wiki-superpowers/agents/{coverage-auditor,branch-depth-auditor,wiki-decision-researcher,wiki-research-lane}.md` — integrate into the G3 Output Schema (`{{ }}` placeholder style, framed "형식 외 응답 금지").
|
|
- `.codex/agents/{...}.md` and `.codex/agents/{...}.toml` — same block as Claude (plain body, inside `developer_instructions` for `.toml`).
|
|
|
|
- [ ] **Step 1: Mirror**
|
|
|
|
For each of the 4 agents, copy the `## Stats` block into the variant files. Antigravity uses `{{ }}` placeholders (e.g. `found: {{N}}`); Codex uses the same concrete-example body as Claude. (Note: `wiki-decision-researcher` / `wiki-research-lane` — confirm they have Antigravity/Codex variants; mirror only those that exist.)
|
|
|
|
- [ ] **Step 2: Grep-verify parity**
|
|
|
|
Run:
|
|
```bash
|
|
for a in coverage-auditor branch-depth-auditor wiki-decision-researcher wiki-research-lane; do
|
|
echo "== $a =="
|
|
grep -l "wiki-stats" .claude/agents/$a.md .agents/plugins/wiki-superpowers/agents/$a.md .codex/agents/$a.md .codex/agents/$a.toml 2>/dev/null
|
|
done
|
|
```
|
|
Expected: every existing variant lists for each agent.
|
|
|
|
- [ ] **Step 3: Checkpoint (no commit).**
|
|
|
|
---
|
|
|
|
## Task 7: Full regression + acceptance smoke (spec §6)
|
|
|
|
**Files:** none (verification)
|
|
|
|
- [ ] **Step 1: All hook test suites**
|
|
|
|
Run:
|
|
```bash
|
|
for t in test_wiki_rules test_wiki_claim_gate test_wiki_structure_lint test_wiki_quorum; do
|
|
out=$(python3 .claude/hooks/$t.py 2>&1 | tail -1); echo "$t -> $out"
|
|
done
|
|
```
|
|
Expected: all `OK`.
|
|
|
|
- [ ] **Step 2: Acceptance §6.2 — live SubagentStop stats gate**
|
|
|
|
Run (printf; no `>` redirect to avoid the bash-gate):
|
|
```bash
|
|
printf '%s' '{"hook_event_name":"SubagentStop","last_assistant_message":"x\n```wiki-stats\nagent: coverage-auditor\nfound: 12\nprocessed: 10\ndropped: 0\n```"}' | python3 .claude/hooks/wiki_claim_gate.py 2>/dev/null; echo "imbalance exit=$?"
|
|
printf '%s' '{"hook_event_name":"SubagentStop","last_assistant_message":"x\n```wiki-stats\nagent: coverage-auditor\nfound: 10\nprocessed: 10\ndropped: 0\n```"}' | python3 .claude/hooks/wiki_claim_gate.py 2>/dev/null; echo "balanced exit=$?"
|
|
```
|
|
Expected: `imbalance exit=2`, `balanced exit=0`.
|
|
|
|
- [ ] **Step 3: Acceptance §6.3-6.5 — grep contracts**
|
|
|
|
Run:
|
|
```bash
|
|
grep -c "No silent truncation" rules/reporting-standards.md
|
|
for a in coverage-auditor branch-depth-auditor wiki-decision-researcher wiki-research-lane; do echo -n "$a: "; grep -c wiki-stats .claude/agents/$a.md; done
|
|
grep -c "Stats funnel" .claude/commands/ingest.md
|
|
```
|
|
Expected: reporting-standards ≥1; each agent ≥1; ingest ≥1.
|
|
|
|
- [ ] **Step 4: Final checkpoint (no commit)** — report results; leave changes in working tree.
|
|
|
|
---
|
|
|
|
## Self-Review (completed by plan author)
|
|
|
|
- **Spec coverage:** §4.2 validate_stats_block → Task 1. §4.3 SubagentStop → Task 2. §4.4 reporting-standards → Task 3. §4.5 agents → Tasks 4+6. §4.6 ingest → Task 5. §6 acceptance 1-6 → Task 1/2 tests + Task 7 smokes. No gap.
|
|
- **Placeholder scan:** deterministic-core tasks carry full code; agent/command tasks give exact block text + per-agent mapping + exact mirror file list. No "TBD"/"similar to".
|
|
- **Type/name consistency:** `parse_stats_block`/`validate_stats_block` names match Tasks 1-2 and tests; the `wiki-stats` fence + funnel fields (`found`/`processed`/`dropped`/`dropped_reason`) identical across spec, impl, tests, agent blocks, smokes; `found = processed + dropped` invariant consistent everywhere.
|