129 lines
8.6 KiB
Markdown
129 lines
8.6 KiB
Markdown
# Contracts 5 & 6 — Citation Discipline + Self-Grep Verification
|
||
|
||
Root SSOT: [`rules/advisory-depth.md`](../../../../../rules/advisory-depth.md) §"Contract 5" + §"Contract 6"
|
||
parent: [`README.md`](README.md)
|
||
관련: [`../reporting-standards/verification-rules.md`](../reporting-standards/verification-rules.md) §7.1 카운트 규칙
|
||
|
||
## Contract 5 — Citation Discipline
|
||
|
||
Every claim that names a specific number, setting, behavior, decision, or quotation must be backed by **verbatim quote + clickable file:line reference**. This applies to:
|
||
|
||
- §1 Executive Summary claims
|
||
- §2 Evidence Matrix "Extracted facts" column
|
||
- §4 Per-File Findings (every field that references the spec)
|
||
- §5 Priority Recommendations "근거 파일:라인" column
|
||
- Direct-response answers that reference any file
|
||
|
||
### Verbatim quote rules
|
||
|
||
- **Byte-for-byte copy from source.** No paraphrasing, no normalization, no translation in the quote itself.
|
||
- If the quote is too long to embed inline (>200 chars), use elided form: `"<beginning 60 chars>" [...] "<end 60 chars>"` with the `[...]` marker explicit.
|
||
- If quoting Korean text from a source, keep it Korean. If quoting English, keep it English. Mixed-language sources are quoted as-is.
|
||
- The quote must contain the specific content that supports the claim. Quoting a tangential line and then drawing an unrelated conclusion is `FILENAME_INFERENCE` adjacent and counts as a citation failure.
|
||
|
||
### Source link rules
|
||
|
||
- **Format:** `path/to/file.md:LINE` for a single line, `path/to/file.md:START-END` for a range.
|
||
- Paths are relative to the workspace root, not absolute (`/home/donghyeon/...` paths are forbidden in citations).
|
||
- IDE-clickable: `file:line` is the universal format that opens directly to the cited line in VS Code, IntelliJ, terminal grep results, GitHub, and most code review tools.
|
||
- For sources outside the workspace (e.g., external docs the user pointed to), still use `file:line` and include the absolute path in a separate `## Source roots` block at the top of the report.
|
||
|
||
### Banned citation patterns
|
||
|
||
| Pattern | Why fails | Replacement |
|
||
|---|---|---|
|
||
| `근거: <file:line>` with no quote | User cannot tell if the cited line actually says what the agent claims | Always include verbatim quote alongside the line reference |
|
||
| `(L67)` style citations without the file path | Ambiguous when multiple files are discussed | Always include path: `feature-X.md:67` |
|
||
| Paraphrased "quote" rewritten in the agent's own words | Looks authoritative but is fabrication | Copy exact bytes from source. If clarity needed, add `해석:` field separately |
|
||
| `*근거: 위 문서 본문*` / vague references | Untraceable; impossible to verify | Specific file:line + verbatim quote |
|
||
| Quoting line N when the claim is about line M | Misdirection; the cited line doesn't actually support the claim | Quote the actual supporting line, or label as `INFERENCE` |
|
||
| Citing a non-existent line | Pure fabrication | Verify the line exists before citing |
|
||
|
||
### Pre-send check (Contract 5)
|
||
|
||
송신 직전 자기 draft 점검. 하나라도 실패하면 draft `BLOCKED`.
|
||
|
||
1. 모든 구체적 사실 주장에 verbatim quote 포함?
|
||
2. 모든 verbatim quote 에 `<path>:<line>` 위치 표기?
|
||
3. 인용된 텍스트가 실제로 그 file:line 에 존재? (grep 으로 검증 가능)
|
||
4. 인용된 텍스트가 실제로 주장의 근거? (탄젠셜한 라인 인용 금지)
|
||
5. 절대 경로 (`/home/...`) 가 아닌 워크스페이스 상대 경로?
|
||
6. 외부 디렉토리 참조 시 §0 Source roots 블록에 절대 경로 명시?
|
||
|
||
근거 부족 주장은 제거 또는 `INFERENCE` 라벨링.
|
||
|
||
## Contract 6 — Self-Grep Verification
|
||
|
||
The agent must **actually execute** a grep/sed command for every verbatim quote before sending. Pasting a quote inside backticks is not citation — running the command and observing a non-empty match is citation. Contract 5 defines the format; Contract 6 enforces the truth.
|
||
|
||
### Why this contract exists
|
||
|
||
Past failure: the agent learned that the format requires `<verbatim quote>` blocks, so it wrote text in quote format. But it did not actually copy from source — it paraphrased, fabricated, or cited the wrong line. Format compliance gave false confidence. Out of 7 spot-checked quotes from a single audit, 4 were either paraphrased, at a wrong line, or did not exist in the source at all.
|
||
|
||
The fix is mechanical: before sending, the agent runs a verification command against each citation. If the command returns zero matches, the citation is fabrication and the finding is `BLOCKED`.
|
||
|
||
### Verification procedure
|
||
|
||
For every finding that includes a verbatim quote + `<path>:<line>` reference, the agent runs **one** of the following Bash commands and observes the output:
|
||
|
||
```bash
|
||
# Option A — verify quote exists at the cited line (preferred):
|
||
sed -n '<LINE>p' '<absolute path to source>'
|
||
# Output must contain the verbatim quote text.
|
||
|
||
# Option B — verify quote exists anywhere in the source:
|
||
grep -nF -- '<verbatim quote text>' '<absolute path to source>'
|
||
# Output must show at least one matching line. If the line number differs, update the citation.
|
||
|
||
# Option C — multi-line range verification:
|
||
sed -n '<START>,<END>p' '<absolute path to source>'
|
||
```
|
||
|
||
Notes:
|
||
|
||
- Quotes inside `grep -F` should be exact bytes from the agent's draft (no markdown backticks, no smart-quote conversion). Use single quotes around the pattern argument.
|
||
- For Korean or non-ASCII text, ensure UTF-8 encoding.
|
||
- For quotes containing single quotes inside, escape: `grep -nF -- "<pattern with 'inner quotes'>" '<file>'`.
|
||
|
||
### Disposition matrix
|
||
|
||
| sed/grep output | Disposition |
|
||
|---|---|
|
||
| Output contains the verbatim quote at the cited line | ✓ Citation passes. Keep finding. |
|
||
| Output exists but at a different line (line drift) | ✗ Update `<path>:<line>` to actual matching line, keep finding. |
|
||
| Output is empty (zero matches anywhere in the file) | ✗ Citation is fabrication. Finding `BLOCKED`. Rewrite quote to match file, re-verify. Paraphrases forbidden. |
|
||
| Output exists but the quote is a paraphrase (close but not byte-for-byte) | ✗ Replace with actual file bytes, re-verify. |
|
||
|
||
### Pre-send check (Contract 6)
|
||
|
||
송신 직전 자기 draft 점검. 하나라도 실패하면 draft `BLOCKED`.
|
||
|
||
1. 모든 verbatim quote 에 대해 실제로 `sed` 또는 `grep -F` 명령 실행 (Bash tool 호출에서 명시적으로 확인 가능)?
|
||
2. 각 명령 출력이 cited line 에서 quote 와 일치? 불일치 시 finding 폐기 또는 line 정정?
|
||
3. 정정 시 §3 Coverage Reconciliation 에 "라인 정정: X → Y" 기록?
|
||
4. 남은 모든 verbatim quote 가 self-grep 통과 상태? 미통과 finding 이 §4 / §5 어디에도 안 남음?
|
||
5. Verdict 산식의 `G` (self-grep 통과 finding 수) = `T` (전체 finding 수)? 불일치 시 `PARTIAL` 또는 `BLOCKED`.
|
||
|
||
송신 시 "검증 완료" 같은 표현 금지 — §7 Verification 또는 §3 Coverage 에 실제로 실행한 grep 명령 중 대표 사례 1~3개를 결과와 함께 기록 (예: `grep -nF '<wiki>/feature-X.md' '<quote>'` → `116: ...`). 이것이 self-grep 을 했다는 유일한 증거.
|
||
|
||
### Sampling honesty
|
||
|
||
The agent is allowed to sample a subset of quotes, but §7.1 statistics MUST reflect what was actually verified:
|
||
|
||
- `V` (검증한 quote 수) = **exact count of sed/grep commands written in §7.1**. Not "I ran more, I just didn't write them all." If the command isn't in §7.1, it isn't counted.
|
||
- `P` (통과 수) ≤ `V`. Never `P > V`.
|
||
- "통과 43" with only 2 sed commands in §7.1 is fabrication, draft `BLOCKED`.
|
||
- Unverified quotes are reported as `U = N − V` and labeled `UNVERIFIED`. They are not "통과" and not "실패" — simply not yet tested.
|
||
|
||
If the agent wants a high pass rate without writing 43 sed commands, reduce `V` by trimming quotes from §4 (i.e., trimming the report) or accept the lower verification ratio in §1.
|
||
|
||
### When the agent cannot run grep
|
||
|
||
If the execution environment or permissions prevent running self-grep, **all verbatim quote-based findings are `BLOCKED`**. Only findings without quotes (e.g., critique of absence) can ship, with Original goal labeled "해당 라인 없음 — 명세 자체에 누락".
|
||
|
||
## Antigravity-specific 메모
|
||
|
||
- Hook G1 (PreToolUse on `docs/superpowers/specs/*.md` write) 가 §7.1 에 `$ sed -n` / `$ grep -nF` 명령 존재 여부 검사. 없으면 deny.
|
||
- **단 hook 은 형식 검사만**. 실제 sed/grep 이 실행됐는지, 출력이 진짜인지는 검증 못 한다. agent 가 위조 출력을 적어도 hook 통과 → **진실성은 agent 자체 책임**.
|
||
- Disposition matrix 의 fabrication 케이스 ("Output is empty") 는 agent 가 의도적으로 위반하지 않는 한 self-detected. agent 가 grep 안 돌리고 fake 출력 적으면 hook 은 막지 못함.
|