init: llm-wiki-haness 하네스 설계

This commit is contained in:
DongHyeonka
2026-07-24 14:21:35 +09:00
parent 42bf3db4fd
commit 6c53ded9cb
2436 changed files with 194486 additions and 1 deletions
@@ -0,0 +1,100 @@
---
title: official-doc / OWASP Cross-Site Request Forgery Prevention Cheat Sheet
source_type: official-doc
url: https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
archive_url: http://web.archive.org/web/20260722202907/https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
related_branches: [feature-keycloak-bff-csrf-samesite-defense]
related_projects: []
tags: [official-doc, keycloak-patterns, security, owasp, csrf]
created: 2026-07-23
---
# official-doc / OWASP Cross-Site Request Forgery Prevention Cheat Sheet
> Layer: `raw/` — 외부 자료(공식 문서)의 **원문 발췌·출처 기록**.
> 본 템플릿은 `raw/official-docs/` 와 `raw/company-tech-blogs/` 두 폴더가 공유.
> 검증된 요약은 `/ingest` 후 `wiki/concepts/`에 `source-summary-template` 형식으로 별도 작성. 원본은 raw에 영구 보관.
## source_type 허용값
`official-doc` — OWASP Foundation 공식 Cheat Sheet Series (커뮤니티 관리형 공식 보안 레퍼런스).
## Parent / 활용 branch (필수, 최소 1개+)
| Branch | 이 자료가 정당화하는 결정 |
|---|---|
| [[raw/branch-notes/feature-keycloak-bff-csrf-samesite-defense]] | branch가 (D1) 쿠키-세션 BFF 대상 CSRF 공격을 재현하는 방법과 (D4) SameSite 단독으로는 불충분해서 SameSite AND CSRF token 을 defense-in-depth 로 함께 쓰는 이유를 결정하는 데 대한 1차 근거 — 이 cheat sheet 가 CSRF 공격 모델, synchronizer/double-submit token 패턴, "SameSite는 defense-in-depth 이지 단독 CSRF mitigation 이 아니다" 라는 OWASP 의 명시적 입장의 authoritative source. |
## 출처
- 원본 URL: https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
- 아카이브 URL: http://web.archive.org/web/20260722202907/https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html (Wayback Machine, 2026-07-22 스냅샷)
- 저자 / 조직: OWASP Foundation (Cheat Sheet Series 커뮤니티 편집)
- 발행일: 지속 갱신되는 living document (cheat sheet 자체에 단일 발행일 명시 없음)
- 마지막 확인일: 2026-07-23
## 왜 저장했는지
`feature-keycloak-bff-csrf-samesite-defense` branch가 CSRF 공격을 재현하고 SameSite + CSRF token 이중 방어를 검증하려면, "CSRF가 왜 성립하는가"와 "SameSite가 왜 단독으로 불충분한가"에 대한 벤더 중립적 authoritative 근거가 필요하다. 이 문서는 OWASP 공식 cheat sheet로 두 질문 모두에 명시적으로 답한다.
## 핵심 인용
> [§Introduction] "A Cross-Site Request Forgery (CSRF) attack occurs when a malicious web site, email, blog, instant message, or program tricks an authenticated user's web browser into performing an unwanted action on a trusted site."
> [§Introduction] "browser requests automatically include all cookies including session cookies"
> [§Token-Based Mitigation] "The synchronizer token pattern is one of the most popular and recommended methods to mitigate CSRF."
> [§Limitations of SameSite] "SameSite is useful as a defense-in-depth control but it does not replace a proper CSRF defense in most deployments."
> [§Synchronizer Token Pattern, HMAC validation pseudo-code] `response.sendError(403, "Invalid CSRF token")`
## 추출된 주장 / Claims Extracted
| Claim ID | Claim (이 자료가 직접 말하는 것) | Evidence quote | Strength | Applies to | Does not prove | Usage Boundary |
|---|---|---|---|---|---|---|
| OWASP-CSRF-C1 | CSRF 공격은 악성 사이트/메일/메시지가 인증된 사용자의 브라우저를 속여 신뢰된 사이트에 원치 않는 action을 수행시키는 것이다 | [§Introduction] "A Cross-Site Request Forgery (CSRF) attack occurs when a malicious web site, email, blog, instant message, or program tricks an authenticated user's web browser into performing an unwanted action on a trusted site." | `official-reference` | CSRF 공격의 정의 자체 (공격 모델) | 특정 프레임워크/브라우저의 구현 세부는 증명 안 함 | branch의 "CSRF 재현" 시나리오 서술에만 사용 — 특정 상용 프레임워크의 방어 여부 주장에는 사용 불가 |
| OWASP-CSRF-C2 | 브라우저는 세션 쿠키를 포함한 모든 쿠키를 cross-site 요청에도 자동으로 첨부한다 | [§Introduction] "browser requests automatically include all cookies including session cookies" | `official-reference` | 쿠키-세션 인증이 CSRF에 취약한 근본 전제 조건 | 이 문장만으로 SameSite/HttpOnly 등 특정 쿠키 속성이 이 동작을 바꾼다고까지는 말하지 않음 (그 내용은 별도 섹션 C4) | branch D1(공격 재현) 의 전제 조건 근거로만 사용 |
| OWASP-CSRF-C3 | Synchronizer token pattern은 CSRF를 완화하는 가장 널리 쓰이고 권장되는 방법 중 하나다 | [§Token-Based Mitigation] "The synchronizer token pattern is one of the most popular and recommended methods to mitigate CSRF." | `official-reference` | CSRF token을 1차 방어로 채택하는 결정의 근거 | 이 문장이 SameSite를 배제해야 한다고 말하지는 않음 (OWASP는 병행을 권고 — C4 참조) | branch D4의 "CSRF token을 쓴다" 절반 근거. "SameSite는 필요 없다"는 결론에는 사용 불가 |
| OWASP-CSRF-C4 | SameSite는 defense-in-depth 통제로 유용하지만 대부분의 배포 환경에서 제대로 된 CSRF 방어를 대체하지 않는다 | [§Limitations of SameSite] "SameSite is useful as a defense-in-depth control but it does not replace a proper CSRF defense in most deployments." | `official-reference` | branch D4 "SameSite 단독 불충분 → SameSite AND CSRF token" 결정의 직접 근거 | 모든 배포 환경에서 예외 없이 불충분하다고까지는 말하지 않음 (cheat sheet는 "SameSite May Be Sufficient On Its Own"인 좁은 조건도 별도 서술 — 이 인용은 그 조건 밖 일반 원칙만 증명) | branch D4의 defense-in-depth 결정 근거로 사용. "SameSite 단독으로 충분한 예외 조건이 전혀 없다"는 과잉 일반화에는 사용 불가 |
| OWASP-CSRF-C5 | Token 검증(HMAC 비교)이 실패하면 서버는 요청을 거부하고 403을 반환해야 한다 | [§Synchronizer Token Pattern, pseudo-code] `response.sendError(403, "Invalid CSRF token")` | `official-reference` | CSRF token 검증 실패 시 기대되는 서버 응답 코드(403) 근거 | 이 pseudo-code 자체가 특정 프레임워크(Spring 등)의 실제 구현 코드라는 뜻은 아님 — 개념 설명용 예시 코드 | branch가 "토큰 검증 실패 → 403" 을 검증 기준으로 채택하는 근거로만 사용. 실제 구현 코드 그대로 복사해도 된다는 근거로는 사용 불가 |
### Strength 허용값
- `official-standard` — RFC, 표준 사양, 언어/프로토콜 표준
- `official-vendor-doc` — Spring, Keycloak, AWS, Google 등 공식 벤더 문서
- `official-reference` — 공식 reference/API 문서
- `company-case-study` — 대기업/실무 기술 블로그의 특정 사례
- `engineering-blog` — 개인/팀 블로그의 엔지니어링 해설
- `tutorial` — 튜토리얼/가이드. 일반화 금지
- `needs-confirmation` — 원문만으로는 적용 판단 불가
## 적용 경계
- 이 자료가 직접 증명하는 것:
- `OWASP-CSRF-C1`: CSRF 공격의 정의와 성립 조건 (인증된 브라우저를 속여 신뢰 사이트에 원치 않는 action 수행)
- `OWASP-CSRF-C2`: 쿠키 자동 첨부가 CSRF의 근본 전제 조건이라는 것
- `OWASP-CSRF-C3`: synchronizer token pattern이 OWASP가 권고하는 주요 CSRF 방어라는 것
- `OWASP-CSRF-C4`: SameSite는 defense-in-depth 이지 단독 CSRF 방어 대체 수단이 아니라는 OWASP의 명시적 입장
- `OWASP-CSRF-C5`: token 검증 실패 시 403 거부가 기대되는 서버 동작이라는 것
- 이 자료가 증명하지 않는 것:
- ca-tmpl / ca-skeleton의 실제 Spring Security 설정이 이 패턴대로 구현되어 있다는 것 (별도 코드 검증 필요)
- "모든" 배포 환경에서 SameSite가 예외 없이 불충분하다는 것 (cheat sheet는 좁은 예외 조건도 서술)
- 특정 프레임워크(Spring Security 등)의 CSRF 필터가 정확히 이 pseudo-code와 동일하게 구현되어 있다는 것
- 내 프로젝트에 적용하려면 추가 확인이 필요한 것:
- 실제 BFF 세션 구현에서 CSRF token이 double-submit cookie 방식인지 synchronizer session-store 방식인지 (branch 구현 단계에서 결정)
- SameSite 속성 값(Lax/Strict)이 실제 배포 도메인 구조(서브도메인 공유 여부)에서 어떤 gap을 남기는지 로컬 재현으로 검증
## 메모
> 나중에 wiki로 옮길 때 참고할 짧은 메모. 검증되지 않은 내 추론은 여기에 두지 말 것.
- 인용 1~2 해석 후보 (미검증): 쿠키 자동 첨부(C2) + 인증된 세션(C1)의 조합이 곧 "왜 GET/POST 상태변경 요청이 위조 가능한가"의 근본 원인이라는 해석은 이 문서가 직접 하는 말이지만, ca-tmpl의 구체 엔드포인트에 어떤 요청이 취약한지는 branch에서 실제 엔드포인트 목록을 대조해야 확정된다 (미검증).
- 추가로 봐야 할 동일 출처 페이지: 같은 cheat sheet 내 "Using Standard Headers to Verify Origin" 섹션 (Origin/Referer 검증) — 이번 인용 범위 밖이라 이 파일에는 포함하지 않음. 필요 시 별도 인용 라운드에서 추가.
## 관련
> 같은 주제의 다른 raw 자료, 또는 이 자료를 인용한 wiki 문서.
- 같은 주제 다른 official-doc / company-tech-blog: 없음 (이 branch의 첫 CSRF 근거 자료)
- 이 자료를 인용한 wiki 요약: 없음 (생성 시 추가)
@@ -0,0 +1,15 @@
{
"schema_version": "document-commit/v1",
"candidate": {
"path": "harness/runtime/_staging_owasp_csrf/candidate.md",
"sha256": "4a273cb35da5005b6ced7f3d0aedf30a8436156e23f757802818f1cc66884699"
},
"target": {
"path": "raw/official-docs/csrf-prevention-owasp-official.md",
"must_not_exist": true
},
"proof_manifest": {
"path": "harness/runtime/_staging_owasp_csrf/proof-manifest.json",
"sha256": "7a8363a9eb49922909a29d4962ab0af80ed445d252c366fe1861079e168df5d4"
}
}
@@ -0,0 +1,146 @@
{
"proofs": [
{
"execution": {
"argv": [
"proof-runner/exact-utf8-v1",
"repo",
"harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"7:7"
],
"exact_match": true,
"exit_code": 0,
"stdout_sha256": "bc957d9b7c984cbf757bfe98c51f740e9c7cf2791b3d835e9ff3daf8091ae7d7",
"stdout_utf8": "A Cross-Site Request Forgery (CSRF) attack occurs when a malicious web site, email, blog, instant message, or program tricks an authenticated user's web browser into performing an unwanted action on a trusted site."
},
"finding": {
"id": "OWASP-CSRF-C1",
"role": "quote"
},
"source": {
"line_end": 7,
"line_start": 7,
"namespace": "repo",
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "A Cross-Site Request Forgery (CSRF) attack occurs when a malicious web site, email, blog, instant message, or program tricks an authenticated user's web browser into performing an unwanted action on a trusted site.",
"sha256": "8ec2079f784a3cf6976f8b204394591063f8f256ad0b87d1ab9c22255fac18e5"
}
},
{
"execution": {
"argv": [
"proof-runner/exact-utf8-v1",
"repo",
"harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"9:9"
],
"exact_match": true,
"exit_code": 0,
"stdout_sha256": "17822e6c9d428f8acaa6c02394ee1521d59619e67357a557c686444389b3c89f",
"stdout_utf8": "browser requests automatically include all cookies including session cookies"
},
"finding": {
"id": "OWASP-CSRF-C2",
"role": "quote"
},
"source": {
"line_end": 9,
"line_start": 9,
"namespace": "repo",
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "browser requests automatically include all cookies including session cookies",
"sha256": "8ec2079f784a3cf6976f8b204394591063f8f256ad0b87d1ab9c22255fac18e5"
}
},
{
"execution": {
"argv": [
"proof-runner/exact-utf8-v1",
"repo",
"harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"13:13"
],
"exact_match": true,
"exit_code": 0,
"stdout_sha256": "51ca093dac0e693bed7f4e1cf99b28335553097832ea16a16c10dbb7f98338e8",
"stdout_utf8": "The synchronizer token pattern is one of the most popular and recommended methods to mitigate CSRF."
},
"finding": {
"id": "OWASP-CSRF-C3",
"role": "quote"
},
"source": {
"line_end": 13,
"line_start": 13,
"namespace": "repo",
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "The synchronizer token pattern is one of the most popular and recommended methods to mitigate CSRF.",
"sha256": "8ec2079f784a3cf6976f8b204394591063f8f256ad0b87d1ab9c22255fac18e5"
}
},
{
"execution": {
"argv": [
"proof-runner/exact-utf8-v1",
"repo",
"harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"26:26"
],
"exact_match": true,
"exit_code": 0,
"stdout_sha256": "07fc06db95c057ad86f62577cfff470b572e9d96a26fb17141629bd201e0ceed",
"stdout_utf8": "SameSite is useful as a defense-in-depth control but it does not replace a proper CSRF defense in most deployments."
},
"finding": {
"id": "OWASP-CSRF-C4",
"role": "quote"
},
"source": {
"line_end": 26,
"line_start": 26,
"namespace": "repo",
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "SameSite is useful as a defense-in-depth control but it does not replace a proper CSRF defense in most deployments.",
"sha256": "8ec2079f784a3cf6976f8b204394591063f8f256ad0b87d1ab9c22255fac18e5"
}
},
{
"execution": {
"argv": [
"proof-runner/exact-utf8-v1",
"repo",
"harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"19:19"
],
"exact_match": true,
"exit_code": 0,
"stdout_sha256": "770c7ed7576a1360f5992918da62806aa8ba3f4f12ea905b3373911f8f4bc4ef",
"stdout_utf8": "response.sendError(403, \"Invalid CSRF token\")"
},
"finding": {
"id": "OWASP-CSRF-C5",
"role": "quote"
},
"source": {
"line_end": 19,
"line_start": 19,
"namespace": "repo",
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "response.sendError(403, \"Invalid CSRF token\")",
"sha256": "8ec2079f784a3cf6976f8b204394591063f8f256ad0b87d1ab9c22255fac18e5"
}
}
],
"run": {
"id": "wiki-source-summarizer-owasp-csrf-2026-07-23",
"profile": "capture"
},
"schema_version": "proof-manifest/v1",
"verification": {
"fail_count": 0,
"pass_count": 5,
"proof_count": 5,
"schema_version": "proof-manifest-result/v1",
"status": "PASS"
}
}
@@ -0,0 +1,44 @@
{
"schema_version": "proof-request/v1",
"run": {
"id": "wiki-source-summarizer-owasp-csrf-2026-07-23",
"profile": "capture"
},
"proofs": [
{
"finding": {"id": "OWASP-CSRF-C1", "role": "quote"},
"source": {
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "A Cross-Site Request Forgery (CSRF) attack occurs when a malicious web site, email, blog, instant message, or program tricks an authenticated user's web browser into performing an unwanted action on a trusted site."
}
},
{
"finding": {"id": "OWASP-CSRF-C2", "role": "quote"},
"source": {
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "browser requests automatically include all cookies including session cookies"
}
},
{
"finding": {"id": "OWASP-CSRF-C3", "role": "quote"},
"source": {
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "The synchronizer token pattern is one of the most popular and recommended methods to mitigate CSRF."
}
},
{
"finding": {"id": "OWASP-CSRF-C4", "role": "quote"},
"source": {
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "SameSite is useful as a defense-in-depth control but it does not replace a proper CSRF defense in most deployments."
}
},
{
"finding": {"id": "OWASP-CSRF-C5", "role": "quote"},
"source": {
"path": "harness/runtime/_staging_owasp_csrf/source-fetch.txt",
"quote_utf8": "response.sendError(403, \"Invalid CSRF token\")"
}
}
]
}
@@ -0,0 +1,7 @@
## 증명 결과
- Manifest: `harness/runtime/_staging_owasp_csrf/proof-manifest.json`
- Manifest SHA-256: `7a8363a9eb49922909a29d4962ab0af80ed445d252c366fe1861079e168df5d4`
- Proof: 5
- PASS: 5
- FAIL: 0
@@ -0,0 +1,26 @@
Cross-Site Request Forgery Prevention Cheat Sheet
Source: https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html
Fetched: 2026-07-23 (curl direct fetch, HTML stripped to plain text; excerpt limited to the sentences cited by raw/official-docs/csrf-prevention-owasp-official.md)
Introduction
A Cross-Site Request Forgery (CSRF) attack occurs when a malicious web site, email, blog, instant message, or program tricks an authenticated user's web browser into performing an unwanted action on a trusted site.
Since browser requests automatically include all cookies including session cookies, this attack works unless proper authorization is used.
Token-Based Mitigation
The synchronizer token pattern is one of the most popular and recommended methods to mitigate CSRF.
Synchronizer Token Pattern
if (!constantTimeEquals(hmacFromRequest, expectedHmac)) {
// HMAC validation failed, reject the request
response.sendError(403, "Invalid CSRF token")
logError("Invalid CSRF token", hmacFromRequest, expectedHmac)
return
}
Limitations of SameSite
SameSite is useful as a defense-in-depth control but it does not replace a proper CSRF defense in most deployments.
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""Efficiently lint active project and branch documents with one shared index."""
from __future__ import annotations
import argparse
import importlib.util
import json
from pathlib import Path
import sys
from typing import Any
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
RESULT_SCHEMA = "active-structure-result/v1"
class ActiveStructureError(RuntimeError):
pass
def _module(path: Path) -> Any:
spec = importlib.util.spec_from_file_location("wiki_structure_lint_active", path)
if spec is None or spec.loader is None:
raise ActiveStructureError(f"cannot load structure lint: {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def check(root: Path) -> dict[str, Any]:
root = root.resolve(strict=True)
lint = _module(root / ".claude/hooks/wiki_structure_lint.py")
authority = lint.authority_mapping(root)
by_st, by_file = lint.build_template_index(root)
vault_paths, vault_bases = lint.build_vault_index(root)
cache: dict[Path, str] = {}
if authority["mode"] == "canonical":
paths = sorted(
root / canonical
for legacy, canonical in authority["legacy_to_canonical"].items()
if (
legacy.startswith("raw/branch-notes/")
or legacy.startswith("raw/project-notes/")
)
and legacy.endswith(".md")
)
else:
paths = sorted((root / "raw/branch-notes").glob("*.md")) + sorted(
(root / "raw/project-notes").glob("*.md")
)
paths = [path for path in paths if path.is_file()]
if not paths:
raise ActiveStructureError("no active project/branch documents found")
findings: list[dict[str, Any]] = []
for path in paths:
relative = path.relative_to(root).as_posix()
mode = lint.classify(relative, root)
lint_findings, _source_type = lint.lint_file(
path,
root,
by_st,
by_file,
vault_paths,
vault_bases,
cache,
mode=mode,
)
findings.extend(
{"code": code, "path": relative, "line": line, "message": message}
for code, line, message in lint_findings
)
findings.sort(key=lambda item: (item["path"], item["line"], item["code"]))
return {
"schema_version": RESULT_SCHEMA,
"status": "PASS" if not findings else "FAIL",
"mode": authority["mode"],
"namespace": "canonical" if authority["mode"] == "canonical" else "legacy",
"checked": len(paths),
"findings": findings,
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
args = parser.parse_args(argv)
try:
result = check(args.root)
exit_code = 0 if result["status"] == "PASS" else 1
except (ActiveStructureError, OSError, UnicodeError, ValueError, ImportError) as exc:
result = {
"schema_version": RESULT_SCHEMA,
"status": "ERROR",
"errors": [{"code": "ACTIVE_STRUCTURE_ERROR", "message": str(exc)}],
}
exit_code = 2
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+335
View File
@@ -0,0 +1,335 @@
#!/usr/bin/env python3
"""Deterministic preflight/postflight validation for project Work Item branches."""
from __future__ import annotations
import argparse
import hashlib
import importlib.util
import json
from pathlib import Path
import re
import sys
import tempfile
from typing import Any
from contract_markdown import as_list, cell, clean, parse_frontmatter, parse_tables, table_for
import fs_transaction
import migrate_graph_contracts
import quality_gate
import template_renderer
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEC_REF_RE = re.compile(r"DEC-[A-Z0-9][A-Z0-9-]*-\d{3}@[1-9]\d*")
WI_RE = re.compile(r"WI-[A-Z0-9][A-Z0-9-]*-\d{3}")
EDITABLE_SECTION_IDS = ("branch-parent", "branch-goal", "branch-scope")
class ContractCheckError(RuntimeError):
pass
def _module(name: str, path: Path) -> Any:
spec = importlib.util.spec_from_file_location(name, path)
if spec is None or spec.loader is None:
raise ContractCheckError(f"checker cannot be loaded: {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _refs(value: object) -> set[str]:
text = " ".join(as_list(value))
return set(DEC_REF_RE.findall(text))
def _wis(value: object) -> set[str]:
return set(WI_RE.findall(" ".join(as_list(value))))
def _finding(code: str, path: str, message: str, line: int = 0) -> dict[str, Any]:
return {"code": code, "path": path, "line": line, "message": message}
def validate(root: Path, branch_path: str | Path, *, postflight: bool = False) -> dict[str, Any]:
try:
root = root.resolve(strict=True)
path = Path(branch_path)
path = path if path.is_absolute() else root / path
# vault cutover 이후 raw/branch-notes/*.md 는 vault 정본을 가리키는 심링크다.
# resolve() 한 경로로 부모를 검사하면 모든 branch 가 "raw/branch-notes 직계가
# 아니다"로 거부돼 postflight 자체가 돌지 않는다 — 그래서 packet digest drift 가
# 조용히 쌓였다. 위치 판정은 심링크를 따라가기 *전* 경로로, 읽기는 정본으로 한다.
legacy = path if path.is_absolute() else root / path
if legacy.parent.resolve() != (root / "raw/branch-notes").resolve():
raise ContractCheckError(
f"branch must be a direct raw/branch-notes child: {legacy.relative_to(root).as_posix()}"
)
path = path.resolve(strict=True)
rel = legacy.relative_to(root).as_posix()
text = path.read_text(encoding="utf-8")
fm = parse_frontmatter(text)
findings: list[dict[str, Any]] = []
project = str(fm.get("project", "")).strip()
work_item = str(fm.get("work_item", "")).strip()
project_path = root / "raw/project-notes" / f"{project}.md"
if not project or not project_path.is_file():
findings.append(_finding("PROJECT_NOT_FOUND", rel, f"project does not exist: {project}"))
if not WI_RE.fullmatch(work_item):
findings.append(_finding("WORK_ITEM_NOT_FOUND", rel, f"invalid Work Item: {work_item}"))
registry, _summaries, registry_blocked = migrate_graph_contracts._registries(root)
findings.extend(
_finding(item["code"], item["path"], item["message"]) for item in registry_blocked
)
kind = str(fm.get("kind", "")).strip()
item = registry.get(path.stem)
if item is None and kind == "branch-child":
candidates = [candidate for candidate in registry.values() if candidate["work_item"] == work_item]
item = candidates[0] if len(candidates) == 1 else None
if item is None:
findings.append(_finding("BRANCH_SLUG_MISMATCH", rel, "branch cannot be resolved to one Work Item row"))
elif kind != "branch-child" and registry.get(path.stem) is None:
findings.append(_finding("BRANCH_SLUG_MISMATCH", rel, "project Work Item branch slug differs from filename"))
elif item["project"] != project or item["work_item"] != work_item:
findings.append(
_finding(
"WORK_ITEM_BINDING_MISMATCH",
rel,
f"expected {item['project']}/{item['work_item']}, observed {project}/{work_item}",
)
)
project_prefix = project.upper()
for reference in sorted(_refs(fm.get("inherits")) | _refs(fm.get("refines")) | _refs(fm.get("overrides"))):
if not reference.startswith(f"DEC-{project_prefix}-"):
findings.append(_finding("FOREIGN_PROJECT_PREFIX", rel, reference))
for dependency in sorted(_wis(fm.get("depends_on")) | ({work_item} if work_item else set())):
if not dependency.startswith(f"WI-{project_prefix}-"):
findings.append(_finding("FOREIGN_PROJECT_PREFIX", rel, dependency))
tables = parse_tables(text)
inherited_table = table_for(tables, "section-id:inherited-project-decisions")
local_table = table_for(tables, "section-id:branch-local-decisions")
overrides_table = table_for(tables, "section-id:declared-overrides")
packet_refs = {
reference
for _line, row in (inherited_table.rows if inherited_table else [])
for reference in _refs(cell(row, "Decision Ref"))
}
fm_inherits = _refs(fm.get("inherits"))
if packet_refs != fm_inherits:
findings.append(
_finding("INHERITED_DECISION_MISMATCH", rel, f"frontmatter={sorted(fm_inherits)}, packet={sorted(packet_refs)}")
)
if item is not None:
expected_refs = set(item["decisions"])
if fm_inherits != expected_refs:
findings.append(
_finding("INHERITED_DECISION_MISMATCH", rel, f"Work Item={sorted(expected_refs)}, branch={sorted(fm_inherits)}")
)
dependencies = _wis(fm.get("depends_on"))
if dependencies != set(item["dependencies"]):
findings.append(
_finding("DEPENDENCY_MISMATCH", rel, f"Work Item={sorted(item['dependencies'])}, branch={sorted(dependencies)}")
)
revision_match = re.search(r"^-\s*\*\*생성 시 프로젝트 개정\*\*:\s*`?([1-9]\d*)`?\s*$", text, re.MULTILINE)
observed_revision = int(revision_match.group(1)) if revision_match else None
if observed_revision != item["project_revision"]:
findings.append(
_finding("STALE_PROJECT_REVISION", rel, f"expected {item['project_revision']}, observed {observed_revision}")
)
completion_match = re.search(r"^-\s*\*\*완료 조건\*\*:\s*(.*?)\s*$", text, re.MULTILINE)
observed_completion = clean(completion_match.group(1)) if completion_match else ""
if observed_completion != clean(item["completion"]):
findings.append(
_finding("COMPLETION_CRITERION_MISMATCH", rel, f"expected {item['completion']!r}, observed {observed_completion!r}")
)
fm_refines = _refs(fm.get("refines"))
relation_refines: set[str] = set()
for _line, row in (local_table.rows if local_table else []):
relation = cell(row, "Relation")
if re.search(r"\brefines\b", relation, re.IGNORECASE):
relation_refines.update(_refs(relation))
if fm_refines != relation_refines:
findings.append(
_finding("REFINES_RELATION_MISMATCH", rel, f"frontmatter={sorted(fm_refines)}, rows={sorted(relation_refines)}")
)
fm_overrides = _refs(fm.get("overrides"))
table_overrides: set[str] = set()
unapproved: set[str] = set()
invalid_approvals = {"", "pending", "tbd", "none", "needs-confirmation", "unapproved", "needs-approval"}
for _line, row in (overrides_table.rows if overrides_table else []):
refs = _refs(cell(row, "Overrides"))
table_overrides.update(refs)
if clean(cell(row, "Approval")).lower() in invalid_approvals:
unapproved.update(refs)
if fm_overrides != table_overrides or unapproved:
findings.append(
_finding(
"OVERRIDE_APPROVAL_MISMATCH",
rel,
f"frontmatter={sorted(fm_overrides)}, rows={sorted(table_overrides)}, unapproved={sorted(unapproved)}",
)
)
expected_hash = str(fm.get("contract_packet_sha256", "")).strip()
actual_hash = ""
try:
actual_hash = template_renderer.generated_sha256(text)
if not re.fullmatch(r"[0-9a-f]{64}", expected_hash) or expected_hash != actual_hash:
findings.append(
_finding("GENERATED_REGION_DRIFT", rel, f"expected hash {expected_hash or '(missing)'}, actual {actual_hash}")
)
except template_renderer.TemplateRenderError as exc:
findings.append(_finding(exc.code, rel, str(exc)))
graph = _module("wiki_graph_contract_check_branch_contract", DEFAULT_ROOT / ".claude/hooks/wiki_graph_contract_check.py")
graph_findings, _warnings, _stats = graph.scan(root, include_expected_edges=True)
findings.extend(_finding(code, finding_path, message, line) for code, finding_path, line, message in graph_findings)
if postflight:
# R1 branch workflows validate their reverse view through the graph
# checker above. The generalized all-document MOC is an R2 gate and
# may contain unrelated legacy edges that must not disable R1 edits.
gate = quality_gate.run(
root,
[path],
structure_paths=[path],
template_root=DEFAULT_ROOT,
require_moc_convergence=False,
)
findings.extend(gate["findings"])
unique = {
(item["code"], item["path"], item.get("line", 0), item["message"]): item
for item in findings
}
findings = [unique[key] for key in sorted(unique)]
if any(item["code"] == "OVERRIDE_APPROVAL_MISMATCH" for item in findings):
findings.append(
{
**next(item for item in findings if item["code"] == "OVERRIDE_APPROVAL_MISMATCH"),
"code": "UNDECLARED_OVERRIDE",
"alias_of": "OVERRIDE_APPROVAL_MISMATCH",
}
)
findings.sort(key=lambda item: (item["path"], item.get("line", 0), item["code"], item["message"]))
return {
"schema_version": "branch-contract-check-result/v1",
"status": "PASS" if not findings else "FAIL",
"phase": "postflight" if postflight else "preflight",
"branch": rel,
"project": project,
"work_item": work_item,
"project_revision": item["project_revision"] if item is not None else None,
"inherits": sorted(fm_inherits),
"editable_sections": list(EDITABLE_SECTION_IDS),
"generated_hashes": {
"declared_sha256": expected_hash,
"observed_sha256": actual_hash,
},
"failure_code_aliases": {"OVERRIDE_APPROVAL_MISMATCH": ["UNDECLARED_OVERRIDE"]},
"findings": findings,
}
except ContractCheckError:
raise
except quality_gate.QualityGateError as exc:
raise ContractCheckError(str(exc)) from exc
except (OSError, UnicodeError, ValueError, ImportError) as exc:
raise ContractCheckError(str(exc)) from exc
def _stage_repository(root: Path, destination: Path) -> None:
fs_transaction.stage_repository(root, destination)
def validate_candidate(
root: Path,
branch_path: str | Path,
candidate_path: str | Path,
*,
postflight: bool = True,
) -> dict[str, Any]:
"""Validate candidate bytes in an isolated repository without touching target."""
root = root.resolve(strict=True)
target = Path(branch_path)
target = target if target.is_absolute() else root / target
# validate() 와 같은 규율 — 위치 판정은 심링크를 따라가기 *전* 경로로 한다.
# 예전에는 resolve() 후의 경로로 relative 를 뽑아 validate() 에 되먹였고, validate()
# 의 수정(pre-resolve 판정)이 그대로 상쇄돼 candidate 경로의 postflight 는 여전히
# "raw/branch-notes 직계가 아니다"로 거부됐다 — 고친 함수의 쌍둥이가 안 고쳐진 사례다.
legacy = target
try:
relative = legacy.relative_to(root)
except ValueError as exc:
raise ContractCheckError(f"branch escapes repository: {target}") from exc
target = target.resolve(strict=True)
candidate = Path(candidate_path)
candidate = candidate if candidate.is_absolute() else root / candidate
candidate = candidate.resolve(strict=True)
if not candidate.is_file():
raise ContractCheckError(f"candidate is not a file: {candidate}")
original = target.read_bytes()
original_sha256 = hashlib.sha256(original).hexdigest()
with tempfile.TemporaryDirectory(prefix="branch-contract-stage-") as directory:
stage = Path(directory) / "repo"
stage.mkdir()
_stage_repository(root, stage)
staged_target = stage / relative
staged_target.write_bytes(candidate.read_bytes())
result = validate(stage, relative, postflight=postflight)
if target.read_bytes() != original:
raise ContractCheckError("branch target changed during candidate validation")
result["candidate"] = candidate.relative_to(root).as_posix() if candidate.is_relative_to(root) else candidate.as_posix()
result["target_sha256"] = original_sha256
result["candidate_sha256"] = hashlib.sha256(candidate.read_bytes()).hexdigest()
result["staged"] = True
return result
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("branch")
parser.add_argument("--candidate", type=Path, help="candidate bytes to validate in an isolated staged repository")
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--preflight", action="store_true")
mode.add_argument("--postflight", action="store_true")
args = parser.parse_args(argv)
try:
if args.candidate is not None and not args.postflight:
raise ContractCheckError("--candidate requires --postflight")
result = (
validate_candidate(args.root, args.branch, args.candidate, postflight=True)
if args.candidate is not None
else validate(args.root, args.branch, postflight=args.postflight)
)
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 0 if result["status"] == "PASS" else 1
except ContractCheckError as exc:
json.dump(
{
"schema_version": "branch-contract-check-result/v1",
"status": "ERROR",
"errors": [{"code": "CONTRACT_CHECK_ERROR", "message": str(exc)}],
},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+389
View File
@@ -0,0 +1,389 @@
#!/usr/bin/env python3
"""Materialize a project Work Item as a validated branch contract packet."""
from __future__ import annotations
import argparse
from datetime import date
import hashlib
import json
from pathlib import Path
import re
import shutil
import sys
import tempfile
from typing import Any
from contract_markdown import cell, clean, parse_frontmatter, parse_tables, replace_table_cell, table_for
from fs_transaction import ReplacementValue, SymlinkValue, replace_many
import layout_check
import moc_indexer
import quality_gate
import semantic_certificate
import semantic_surface_extractor
import template_renderer
import typed_contract_check
import vault_migrate
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEC_REF_RE = re.compile(r"^(DEC-[A-Z0-9][A-Z0-9-]*-\d{3})@([1-9]\d*)$")
DEC_ID_RE = re.compile(r"^DEC-[A-Z0-9][A-Z0-9-]*-\d{3}$")
WI_RE = re.compile(r"^WI-[A-Z0-9][A-Z0-9-]*-\d{3}$")
SLUG_RE = re.compile(r"^(feature|fix|chore|experiment)-[a-z0-9]+(?:-[a-z0-9]+)*$")
class BranchError(ValueError):
def __init__(self, code: str, message: str, location: str = "") -> None:
self.code = code
self.location = location
super().__init__(message)
def _refs(value: str) -> dict[str, int]:
items = [clean(item) for item in value.split(",") if clean(item) and clean(item) != "-"]
if not items:
raise BranchError("MISSING_APPLIED_DECISION", "Applies Decisions must contain at least one pinned reference")
parsed: dict[str, int] = {}
for item in items:
match = DEC_REF_RE.fullmatch(item)
if not match:
raise BranchError("INVALID_DECISION_REF", f"invalid pinned decision reference: {item}")
if match.group(1) in parsed:
raise BranchError("DUPLICATE_DECISION_REF", f"duplicate pinned decision reference: {item}")
parsed[match.group(1)] = int(match.group(2))
return parsed
def _dependencies(value: str) -> list[str]:
items = [clean(item) for item in value.split(",") if clean(item) and clean(item) != "-"]
for item in items:
if not WI_RE.fullmatch(item):
raise BranchError("INVALID_WORK_ITEM_DEPENDENCY", f"invalid Work Item dependency: {item}")
return sorted(set(items))
def _inline(values: list[str]) -> str:
return "[" + ", ".join(values) + "]"
def _table_cell(value: str) -> str:
return value.replace("|", "\\|")
def _render_branch(
root: Path,
project: str,
wi_id: str,
slug: str,
revision: int,
completion: str,
decisions: list[tuple[str, int, str]],
dependencies: list[str],
) -> str:
branch_id = wi_id.replace("WI-", "BR-", 1)
inherited = [f"{decision_id}@{decision_revision}" for decision_id, decision_revision, _ in decisions]
rows = "\n".join(
f"| `{decision_id}@{decision_revision}` | {_table_cell(summary)} | `{wi_id}` 완료 조건에 적용 | "
f"`[[raw/project-notes/{project}]]` |"
for decision_id, decision_revision, summary in decisions
)
template_path = root / "templates/branch-note-template.md"
if not template_path.is_file():
template_path = DEFAULT_ROOT / "templates/branch-note-template.md"
values = {
"branch_slug": slug,
"branch_id": branch_id,
"project": project,
"project_parent_link": f"- [[raw/project-notes/{project}]]",
"work_item": wi_id,
"inherits_yaml": _inline(inherited),
"depends_on_yaml": _inline(dependencies),
"created": date.today().isoformat(),
"contract_packet_sha256": "0" * 64,
"project_revision": str(revision),
"completion": completion,
"inherited_rows": rows,
"dependency_display": ", ".join(f"`{item}`" for item in dependencies) if dependencies else "해당 없음",
}
try:
provisional = template_renderer.render_branch_note(template_path, values)
values["contract_packet_sha256"] = template_renderer.generated_sha256(provisional)
return template_renderer.render_branch_note(template_path, values)
except template_renderer.TemplateRenderError as exc:
raise BranchError(exc.code, str(exc), str(template_path)) from exc
def _moc_staging_roots() -> set[Path]:
"""staging 에 담을 문서 root — moc_indexer 가 스캔하는 모든 relation root.
staging 이 이보다 좁으면 stage 에 없는 child_root(raw/official-docs 등)의
문서로 파생되던 generated region 이 *빈 목록* 으로 재생성되고, 그 파괴적
결과가 실 저장소에 그대로 반영된다(2026-07-23 실측 — 무관 문서 100+개의
sources region 소실). relations config 를 읽지 못하면 기존 최소 2개 root 로
되돌아간다 — 그 경우 build_updates 가 같은 파일을 읽다 스스로 실패한다.
"""
roots = {Path("raw/project-notes"), Path("raw/branch-notes")}
try:
relations = json.loads(moc_indexer.DEFAULT_RELATIONS.read_text(encoding="utf-8"))
except (OSError, ValueError):
return roots
for relation in relations.get("relations", []):
for key in ("child_roots", "parent_roots"):
for value in relation.get(key) or []:
roots.add(Path(str(value)))
return roots
def _plan_sha256(root: Path, changes: dict[Path, ReplacementValue]) -> str:
digest = hashlib.sha256()
for path in sorted(changes, key=lambda item: item.relative_to(root).as_posix()):
relative = path.relative_to(root).as_posix().encode("utf-8")
payload = changes[path]
if isinstance(payload, SymlinkValue):
# canonical 모드 신규 문서의 호환 심링크. vault_migrate 의
# _replacement_fingerprint 와 같은 "symlink\0<target>" 인코딩으로
# bytes 와 구분해 결정론 해시에 넣는다.
payload = ("symlink\0" + payload.target).encode("utf-8")
digest.update(len(relative).to_bytes(8, "big"))
digest.update(relative)
digest.update(len(payload).to_bytes(8, "big"))
digest.update(payload)
return digest.hexdigest()
def prepare(
root: Path,
project: str,
wi_id: str,
*,
layout_path: Path = layout_check.DEFAULT_MANIFEST,
) -> tuple[dict[Path, bytes], dict[str, Any]]:
root = root.resolve()
# Isolated runtime fixtures may omit copied harness sources. In that case
# validate their documents with the packaged schema. For the real repo
# this resolves to the same local path, so a missing schema still fails
# closed instead of silently disabling the gate.
typed_schema = (
typed_contract_check.DEFAULT_SCHEMA
if (root / typed_contract_check.DEFAULT_SCHEMA).is_file()
else DEFAULT_ROOT / typed_contract_check.DEFAULT_SCHEMA
)
try:
typed_result = typed_contract_check.check(root, typed_schema)
except (typed_contract_check.TypedContractError, OSError, UnicodeError) as exc:
raise BranchError("TYPED_CONTRACT_ERROR", str(exc)) from exc
if typed_result["status"] != "PASS":
codes = ",".join(
sorted({str(item.get("code", "UNKNOWN")) for item in typed_result.get("findings", [])})
)
raise BranchError("TYPED_CONTRACT_FAILED", codes or "typed contract gate failed")
if not re.fullmatch(r"[a-z0-9]+(?:-[a-z0-9]+)*", project):
raise BranchError("INVALID_PROJECT_SLUG", f"invalid project slug: {project}")
expected_wi = re.compile(rf"^WI-{re.escape(project.upper())}-\d{{3}}$")
if not expected_wi.fullmatch(wi_id):
raise BranchError("INVALID_WORK_ITEM_ID", f"Work Item must match WI-{project.upper()}-NNN")
project_path = root / "raw/project-notes" / f"{project}.md"
# cutover 이후 legacy 경로는 정본을 가리키는 심링크다. lexical 경로 그대로
# write-root 집행에 넘기면 canonical 모드에서 "raw/… 는 write root 밖" 으로
# 거부돼 **모든 입력에서 실패**한다 — 실제로 그 상태였다. 위치 판정은 정본으로 한다.
project_authority_path = project_path.resolve() if project_path.is_symlink() else project_path
# 신규 branch 노트의 목적지는 이제 vault_migrate.plan_new_documents 가 계산한다
# (정본 + 호환 심링크 + manifest 2행). 존재하지 않는 합성 경로로 write root 를
# 미리 찔러보던 authority-check probe 는 그 계산을 우회하므로 제거한다.
try:
authority = layout_check.resolve_authority(root, layout_path)
layout_check.enforce_write_paths(root, [project_authority_path], authority)
except layout_check.LayoutContractError as exc:
raise BranchError(exc.code, str(exc), exc.location) from exc
if not project_path.is_file():
raise BranchError("PROJECT_NOT_FOUND", f"project note does not exist: {project_path}")
if (root / semantic_surface_extractor.DEFAULT_POLICY).is_file():
try:
semantic_parent = semantic_certificate.check(root, mode="hub", paths=[project_path])
except (
semantic_certificate.SemanticCertificateError,
semantic_surface_extractor.SemanticSurfaceError,
typed_contract_check.TypedContractError,
OSError,
UnicodeError,
) as exc:
raise BranchError("PARENT_SEMANTIC_CERTIFICATE_ERROR", str(exc), project_path.relative_to(root).as_posix()) from exc
if semantic_parent["status"] != "PASS":
codes = ",".join(sorted({str(item.get("code", "UNKNOWN")) for item in semantic_parent["findings"]}))
raise BranchError(
"PARENT_SEMANTIC_CERTIFICATE_INVALID",
codes or "parent project hub certificate is missing, stale, or blocking",
project_path.relative_to(root).as_posix(),
)
project_text = project_path.read_text(encoding="utf-8")
frontmatter = parse_frontmatter(project_text)
revision_raw = str(frontmatter.get("project_revision", ""))
if not revision_raw.isdigit() or int(revision_raw) < 1:
raise BranchError("LEGACY_PROJECT_CONTRACT", "project_revision must be a positive integer")
revision = int(revision_raw)
tables = parse_tables(project_text)
decision_table = table_for(tables, "section-id:project-decisions", "안정 결정 레지스트리", "Project Decision Registry")
work_table = table_for(tables, "section-id:project-work-items", "실행계획", "Work Item Registry")
if decision_table is None or work_table is None:
raise BranchError("LEGACY_PROJECT_CONTRACT", "project decision/work-item registry is required")
registry: dict[str, tuple[int, str]] = {}
for line, row in decision_table.rows:
decision_id = clean(cell(row, "Decision ID"))
decision_revision = clean(cell(row, "Revision"))
if not DEC_ID_RE.fullmatch(decision_id) or not decision_revision.isdigit() or int(decision_revision) < 1:
raise BranchError("INVALID_PROJECT_DECISION", "invalid decision registry row", f"{project_path}:{line}")
if decision_id in registry:
raise BranchError("DUPLICATE_DECISION_OWNER", f"duplicate decision: {decision_id}", f"{project_path}:{line}")
registry[decision_id] = (int(decision_revision), clean(cell(row, "Decision Summary")))
work_rows = [(line, row) for line, row in work_table.rows if clean(cell(row, "Work Item ID")) == wi_id]
if len(work_rows) != 1:
code = "WORK_ITEM_NOT_FOUND" if not work_rows else "DUPLICATE_WORK_ITEM"
raise BranchError(code, f"expected exactly one {wi_id} row, observed {len(work_rows)}")
work_line, work_row = work_rows[0]
all_work_ids = {clean(cell(row, "Work Item ID")) for _line, row in work_table.rows}
slug = clean(cell(work_row, "branch slug"))
if not SLUG_RE.fullmatch(slug) or not 4 <= len(slug.split("-")[1:]) <= 8:
raise BranchError("INVALID_BRANCH_SLUG", f"branch slug violates naming-conventions: {slug}")
target = root / "raw/branch-notes" / f"{slug}.md"
if target.exists():
raise BranchError("TARGET_EXISTS", f"branch target already exists: {target}")
completion = clean(cell(work_row, "완료 조건 (측정가능)"))
if not completion:
raise BranchError("MISSING_COMPLETION_CRITERION", f"{wi_id} has no completion criterion")
applied = _refs(cell(work_row, "Applies Decisions"))
inherited: list[tuple[str, int, str]] = []
for decision_id, pinned_revision in applied.items():
current = registry.get(decision_id)
if current is None:
raise BranchError("MISSING_PROJECT_DECISION", f"{decision_id} is not in the project registry")
if pinned_revision != current[0]:
raise BranchError(
"STALE_INHERITANCE_REVISION",
f"{decision_id}@{pinned_revision} does not match current @{current[0]}",
)
inherited.append((decision_id, pinned_revision, current[1]))
dependencies = _dependencies(cell(work_row, "Dependencies"))
missing_dependencies = sorted(set(dependencies) - all_work_ids)
if missing_dependencies:
raise BranchError("MISSING_WORK_ITEM_DEPENDENCY", f"unknown dependencies: {missing_dependencies}")
updated_project = project_text
if clean(cell(work_row, "Status")) == "planned":
updated_project = replace_table_cell(updated_project, work_table, work_line, "Status", "`in-progress`")
branch_text = _render_branch(root, project, wi_id, slug, revision, completion, sorted(inherited), dependencies)
with tempfile.TemporaryDirectory(prefix=".branch-from-project-stage-", dir=root) as directory:
stage = Path(directory)
for rel in sorted(_moc_staging_roots()):
source = root / rel
if source.exists():
shutil.copytree(source, stage / rel)
staged_project = stage / project_path.relative_to(root)
staged_target = stage / target.relative_to(root)
staged_project.parent.mkdir(parents=True, exist_ok=True)
staged_target.parent.mkdir(parents=True, exist_ok=True)
staged_project.write_text(updated_project, encoding="utf-8")
staged_target.write_text(branch_text, encoding="utf-8")
moc_updates, _moc_stats = moc_indexer.build_updates(stage)
for path, text in moc_updates.items():
path.write_text(text, encoding="utf-8")
remaining, _ = moc_indexer.build_updates(stage)
if remaining:
raise BranchError("MOC_VALIDATION_FAILED", "MOC indexer did not converge")
try:
gate = quality_gate.run(
stage,
[staged_project, staged_target],
structure_paths=[staged_target],
template_root=DEFAULT_ROOT,
)
except quality_gate.QualityGateError as exc:
raise BranchError("QUALITY_GATE_ERROR", str(exc)) from exc
if gate["status"] != "PASS":
summary = "; ".join(
f"{item['code']} {item['path']}: {item['message']}" for item in gate["findings"][:10]
)
raise BranchError("QUALITY_GATE_FAILED", summary)
changes = {project_path: staged_project.read_bytes(), target: staged_target.read_bytes()}
for staged_path in moc_updates:
actual_path = root / staged_path.relative_to(stage)
changes[actual_path] = staged_path.read_bytes()
try:
changes, authority = vault_migrate.expand_authoritative_changes(
root,
changes,
layout_path=layout_path,
)
except vault_migrate.MigrationError as exc:
raise BranchError(exc.code, str(exc), exc.location) from exc
plan_sha256 = _plan_sha256(root, changes)
return changes, {
"project": project,
"work_item": wi_id,
"project_revision": revision,
"target": target.relative_to(root).as_posix(),
"inherits_count": len(inherited),
"depends_on_count": len(dependencies),
"changed_paths": [path.relative_to(root).as_posix() for path in sorted(changes)],
"must_not_exist_paths": [
path.relative_to(root).as_posix() for path in sorted(changes) if not path.exists()
],
"active_layout": {
"mode": authority["mode"],
"authority": authority["authority"],
"write_roots": authority["write_roots"],
"manifest_sha256": authority["manifest_sha256"],
},
"plan_sha256": plan_sha256,
}
def _emit(document: dict[str, Any]) -> None:
json.dump(document, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("project")
parser.add_argument("work_item")
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--layout", type=Path, default=layout_check.DEFAULT_MANIFEST)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--dry-run", action="store_true")
mode.add_argument("--apply", action="store_true")
parser.add_argument("--expected-plan-sha256")
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
changes, result = prepare(root, args.project, args.work_item, layout_path=args.layout)
if args.apply:
expected = args.expected_plan_sha256
if expected is not None:
if not re.fullmatch(r"[0-9a-f]{64}", expected):
raise BranchError("INVALID_PLAN_SHA256", "expected plan hash must be 64 lowercase hex characters")
if expected != result["plan_sha256"]:
raise BranchError(
"PLAN_HASH_MISMATCH",
f"expected {expected}, current plan is {result['plan_sha256']}",
)
forbidden = {root / path for path in result["must_not_exist_paths"]}
replace_many(changes, must_not_exist=forbidden)
_emit({"schema_version": "branch-from-project-result/v1", "status": "APPLIED" if args.apply else "DRY_RUN", "findings": [], **result})
return 0
except BranchError as exc:
error = {"code": exc.code, "location": exc.location, "message": str(exc)}
_emit({"schema_version": "branch-from-project-result/v1", "status": "FAIL", "errors": [error]})
return 1
except (OSError, UnicodeError) as exc:
_emit({"schema_version": "branch-from-project-result/v1", "status": "ERROR", "errors": [{"code": "IO_ERROR", "location": "", "message": str(exc)}]})
return 2
if __name__ == "__main__":
raise SystemExit(main())
+193
View File
@@ -0,0 +1,193 @@
#!/usr/bin/env python3
"""Small, dependency-free parsers for the wiki's structured Markdown contracts."""
from __future__ import annotations
from dataclasses import dataclass
import re
from typing import Iterable
FM_RE = re.compile(r"^([A-Za-z_][\w-]*):\s*(.*)$")
HEADING_RE = re.compile(r"^(#{1,6})\s+(.+?)\s*$")
TABLE_SEP_RE = re.compile(r"^:?-{3,}:?$")
SECTION_ID_RE = re.compile(r"^\s*<!--\s*section-id:\s*([a-z0-9][a-z0-9-]*)\s*-->\s*$")
# 셀 전체가 하나의 코드스팬/강조일 때만 벗긴다. 예전에는 strip("`* ") 로 양끝 문자를
# 무조건 깎았는데, 그러면 코드스팬으로 *시작만* 하는 셀이 여는 백틱을 잃는다 —
# "`domain <- application` 의존 방향과 …" 가 "domain <- application` 의존 방향과 …" 가 돼
# 투영된 표 11곳(hub 자신의 생성 블록 포함)에서 코드스팬이 깨져 있었다.
SINGLE_CODE_SPAN_RE = re.compile(r"^`([^`]*)`$")
SINGLE_EMPHASIS_RE = re.compile(r"^(\*{1,2})([^*]*)\1$")
def clean(value: str) -> str:
value = value.strip()
while True:
match = SINGLE_CODE_SPAN_RE.match(value) or SINGLE_EMPHASIS_RE.match(value)
if match is None:
break
value = match.group(match.lastindex).strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in "'\"":
value = value[1:-1]
return value.strip()
def parse_frontmatter(text: str) -> dict[str, object]:
lines = text.splitlines()
if not lines or lines[0].strip() != "---":
return {}
values: dict[str, object] = {}
current: str | None = None
for line in lines[1:]:
if line.strip() == "---":
break
match = FM_RE.match(line)
if match:
current = match.group(1)
raw = match.group(2).strip()
if raw.startswith("[") and raw.endswith("]"):
values[current] = [clean(item) for item in raw[1:-1].split(",") if clean(item)]
else:
values[current] = clean(raw)
continue
item = re.match(r"^\s+-\s+(.+?)\s*$", line)
if item and current:
if not isinstance(values.get(current), list):
values[current] = []
assert isinstance(values[current], list)
values[current].append(clean(item.group(1)))
return values
def as_list(value: object) -> list[str]:
if isinstance(value, list):
return [str(item).strip() for item in value if str(item).strip()]
if isinstance(value, str) and value.strip():
return [value.strip()]
return []
def split_row(line: str) -> list[str]:
token = "\x00PIPE\x00"
return [
cell.strip().replace(token, "|")
for cell in line.strip().strip("|").replace("\\|", token).split("|")
]
def header_key(value: str) -> str:
return re.sub(r"[^0-9a-zA-Z가-힣]+", "", value).lower()
@dataclass(frozen=True)
class MarkdownTable:
headings: tuple[str, ...]
section_ids: tuple[str, ...]
headers: tuple[str, ...]
header_line: int
rows: tuple[tuple[int, dict[str, str]], ...]
def parse_tables(text: str) -> list[MarkdownTable]:
lines = text.splitlines()
headings: dict[int, str] = {}
section_ids: dict[int, str] = {}
pending_section_id = ""
tables: list[MarkdownTable] = []
index = 0
while index < len(lines):
section_id = SECTION_ID_RE.match(lines[index])
if section_id:
pending_section_id = section_id.group(1)
index += 1
continue
heading = HEADING_RE.match(lines[index])
if heading:
level = len(heading.group(1))
headings = {key: value for key, value in headings.items() if key < level}
section_ids = {key: value for key, value in section_ids.items() if key < level}
headings[level] = heading.group(2).strip()
if pending_section_id:
section_ids[level] = pending_section_id
pending_section_id = ""
index += 1
continue
if (
lines[index].lstrip().startswith("|")
and index + 1 < len(lines)
and lines[index + 1].lstrip().startswith("|")
):
headers = split_row(lines[index])
separators = split_row(lines[index + 1])
if len(headers) == len(separators) and all(TABLE_SEP_RE.fullmatch(item) for item in separators):
rows: list[tuple[int, dict[str, str]]] = []
cursor = index + 2
keys = [header_key(header) for header in headers]
while cursor < len(lines) and lines[cursor].lstrip().startswith("|"):
cells = split_row(lines[cursor])
cells += [""] * (len(headers) - len(cells))
rows.append((cursor + 1, dict(zip(keys, cells))))
cursor += 1
tables.append(
MarkdownTable(
headings=tuple(headings.values()),
section_ids=tuple(section_ids.values()),
headers=tuple(headers),
header_line=index + 1,
rows=tuple(rows),
)
)
index = cursor
continue
index += 1
return tables
def table_for(tables: Iterable[MarkdownTable], *headings: str) -> MarkdownTable | None:
"""Return the first table under any accepted localized heading.
Structured column names remain stable machine schema. Section headings are
presentation text, so readers and migration tools accept both the current
Korean title and the legacy English title during rollout.
"""
needles = tuple(heading.casefold() for heading in headings)
return next(
(
table
for table in tables
if any(
needle in item.casefold()
for needle in needles
for item in (*table.headings, *(f"section-id:{value}" for value in table.section_ids))
)
),
None,
)
def cell(row: dict[str, str], name: str) -> str:
return row.get(header_key(name), "")
def replace_table_cell(
text: str,
table: MarkdownTable,
row_line: int,
header_name: str,
value: str,
) -> str:
keys = [header_key(header) for header in table.headers]
target_key = header_key(header_name)
if target_key not in keys:
raise ValueError(f"table has no {header_name!r} column")
lines = text.splitlines(keepends=True)
original = lines[row_line - 1]
newline = "\n" if original.endswith("\n") else ""
cells = split_row(original.rstrip("\n"))
cells += [""] * (len(keys) - len(cells))
cells[keys.index(target_key)] = value
rendered = [item.replace("|", "\\|") for item in cells[: len(keys)]]
lines[row_line - 1] = "| " + " | ".join(rendered) + " |" + newline
return "".join(lines)
+287
View File
@@ -0,0 +1,287 @@
#!/usr/bin/env python3
"""Generate deterministic typed-contract projections for consumer documents."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
from fs_transaction import replace_many
import typed_contract_check
from typed_contract_check import Graph, Record
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_SCHEMA = Path("harness/source/typed-contracts.json")
RESULT_SCHEMA = "contract-projection-result/v1"
CLUSTER_HEADING = re.compile(r"^##\s+.*(?:Cluster|묶음).*$", re.MULTILINE | re.IGNORECASE)
class ProjectionError(ValueError):
def __init__(self, code: str, message: str, path: str = "") -> None:
self.code = code
self.path = path
super().__init__(message)
def _finding(code: str, path: str, message: str, line: int = 0) -> dict[str, Any]:
return {"code": code, "path": path, "line": line, "message": message}
def _marker_pair(marker: str) -> tuple[str, str]:
return f"<!-- GENERATED: {marker}:start -->", f"<!-- GENERATED: {marker}:end -->"
def _escape(value: str) -> str:
return value.replace("|", "\\|").replace("\n", " ")
def _link(graph: Graph, slug: str) -> str:
matches = graph.by_slug.get(slug, ())
return f"[[{matches[0].relative[:-3]}]]" if len(matches) == 1 else slug
def _records_by_id(graph: Graph) -> dict[str, Record]:
grouped: dict[str, list[Record]] = {}
for records in graph.records.values():
for record in records:
grouped.setdefault(record.identifier, []).append(record)
return {identifier: records[0] for identifier, records in grouped.items() if len(records) == 1}
def _artifact_block(graph: Graph, marker: str, records: Iterable[Record]) -> str:
rows = [
f"| `{item.identifier}@{item.revision}` | {_link(graph, item.owner)} | "
f"{_link(graph, typed_contract_check._owner_slug(item.values.get('producer', '')))} | "
f"`{_escape(item.values.get('schemaref', ''))}` |"
for item in sorted(records, key=lambda value: value.identifier)
]
start, end = _marker_pair(marker)
return "\n".join(
[
start,
"### 가져온 artifact 계약",
"",
"| Artifact Ref | Owner | Producer | Schema Ref |",
"|---|---|---|---|",
*rows,
end,
]
)
def _contract_block(graph: Graph, marker: str, records: Iterable[Record]) -> str:
rows = [
f"| `{item.identifier}@{item.revision}` | {_link(graph, item.owner)} | "
f"{_escape(item.values.get('requiredeffect', ''))} | import 참조로 적용 |"
for item in sorted(records, key=lambda value: value.identifier)
]
start, end = _marker_pair(marker)
return "\n".join(
[
start,
"## 가져온 프로젝트 계약",
"",
"| Ref | Owner | 요약 | Branch 적용 |",
"|---|---|---|---|",
*rows,
end,
]
)
def _delegation_block(graph: Graph, marker: str, records: Iterable[Record]) -> str:
rows = [
f"| `{item.identifier}@{item.revision}` | {_link(graph, typed_contract_check._owner_slug(item.values.get('delegator', '')))} | "
f"`{_escape(item.values.get('concernkey', ''))}` | {_escape(item.values.get('status', ''))} |"
for item in sorted(records, key=lambda value: value.identifier)
]
start, end = _marker_pair(marker)
return "\n".join(
[
start,
"### 수신한 위임",
"",
"| Delegation Ref | From | Concern | Status |",
"|---|---|---|---|",
*rows,
end,
]
)
def _flow_block(graph: Graph, marker: str, records: Iterable[Record]) -> str:
rows = [
f"| `{item.identifier}@{item.revision}` | {item.values.get('order', '')} | {_link(graph, item.owner)} | "
f"{_escape(item.values.get('input', ''))} | {_escape(item.values.get('action', ''))} | "
f"{_escape(item.values.get('output', ''))} |"
for item in sorted(records, key=lambda value: (int(value.values.get("order", "0") or 0), value.identifier))
]
start, end = _marker_pair(marker)
return "\n".join(
[
start,
"### 가져온 흐름 단계",
"",
"| Stage Ref | Order | Owner | Input | Action | Output |",
"|---|---:|---|---|---|---|",
*rows,
end,
]
)
def _replace_or_insert(text: str, marker: str, block: str, relative: str) -> tuple[str, bool]:
start, end = _marker_pair(marker)
starts = [item.start() for item in re.finditer(re.escape(start), text)]
ends = [item.start() for item in re.finditer(re.escape(end), text)]
if starts or ends:
if len(starts) != 1 or len(ends) != 1 or starts[0] >= ends[0]:
raise ProjectionError("INVALID_CONTRACT_PROJECTION_MARKERS", f"{marker} markers must be one ordered pair", relative)
end_at = ends[0] + len(end)
updated = text[: starts[0]] + block + text[end_at:]
return updated, updated != text
heading = CLUSTER_HEADING.search(text)
if heading is not None:
insert_at = text.find("\n", heading.end())
if insert_at < 0:
return text.rstrip() + "\n\n" + block + "\n", True
return text[: insert_at + 1] + "\n" + block + "\n" + text[insert_at + 1 :], True
return text.rstrip() + "\n\n" + block + "\n", True
def _drift_codes(kind: str) -> tuple[str, ...]:
return {
"artifacts": ("ARTIFACT_PROJECTION_DRIFT",),
"contracts": ("GENERATED_CONTRACT_PROJECTION_DRIFT", "CONTRACT_PROJECTION_DRIFT"),
"delegations": ("MISSING_RECEIVED_DELEGATION_PROJECTION",),
"flow_stages": ("CONTRACT_PROJECTION_DRIFT",),
}[kind]
def plan(graph: Graph) -> dict[str, Any]:
by_id = _records_by_id(graph)
markers = graph.config["projection_markers"]
updates: dict[Path, str] = {}
findings: list[dict[str, Any]] = []
projection_count = 0
for document in graph.documents:
imported: dict[str, list[Record]] = {
"artifacts": [],
"contracts": [],
"flow_stages": [],
}
for identifier, revision in graph.imports[document.relative]:
record = by_id.get(identifier)
if record is not None and record.revision == revision and record.kind in imported:
imported[record.kind].append(record)
delegations = [
record
for record in graph.records["delegations"]
if record.values.get("status") == "accepted"
and typed_contract_check._owner_slug(record.values.get("delegate", "")) == document.slug
and (record.identifier, record.revision) in set(graph.accepts[document.relative])
]
desired: dict[str, list[Record]] = {**imported, "delegations": delegations}
text = document.text
for kind in ("artifacts", "contracts", "delegations", "flow_stages"):
marker = markers[kind]
start, end = _marker_pair(marker)
has_marker = start in text or end in text
rows = desired[kind]
if not rows and not has_marker:
continue
projection_count += len(rows)
if kind == "artifacts":
block = _artifact_block(graph, marker, rows)
elif kind == "contracts":
block = _contract_block(graph, marker, rows)
elif kind == "delegations":
block = _delegation_block(graph, marker, rows)
else:
block = _flow_block(graph, marker, rows)
try:
updated, changed = _replace_or_insert(text, marker, block, document.relative)
except ProjectionError as exc:
findings.append(_finding(exc.code, exc.path, str(exc)))
continue
if changed:
for code in _drift_codes(kind):
findings.append(_finding(code, document.relative, f"{marker} projection differs from registry authority"))
text = updated
if text != document.text:
updates[document.path] = text
findings.sort(key=lambda item: (item["path"], item["code"], item["message"]))
return {
"updates": updates,
"projection_count": projection_count,
"findings": findings,
}
def build_updates(
root: Path,
schema_path: Path = DEFAULT_SCHEMA,
) -> tuple[dict[Path, str], dict[str, Any]]:
graph, schema_findings = typed_contract_check.build_graph(root, schema_path)
base = typed_contract_check.check(root, schema_path, include_projection=False)
if base["status"] != "PASS":
return {}, {
"status": "FAIL",
"findings": base["findings"],
"projection_count": 0,
"changed_paths": [],
}
result = plan(graph)
return result["updates"], {
"status": "DRIFT" if result["updates"] or result["findings"] else "CURRENT",
"findings": [*schema_findings, *result["findings"]],
"projection_count": result["projection_count"],
"changed_paths": [path.relative_to(graph.root).as_posix() for path in sorted(result["updates"])],
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--schema", type=Path, default=DEFAULT_SCHEMA)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--check", action="store_true")
mode.add_argument("--write", action="store_true")
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
updates, result = build_updates(root, args.schema)
if args.write and result["status"] != "FAIL" and updates:
# 문서 스캔은 legacy 경로(raw/…)로 하지만 쓰기는 정본으로 해야 한다.
# resolve() 없이 쓰면 atomic replace 가 심링크를 실파일로 갈아치워
# canonical/legacy 사본이 갈라진다(layout_check INVALID_COMPATIBILITY_STUB).
replace_many({path.resolve(): text.encode("utf-8") for path, text in updates.items()})
result["status"] = "UPDATED"
result["findings"] = []
payload = {"schema_version": RESULT_SCHEMA, **result}
exit_code = 0 if payload["status"] in {"CURRENT", "UPDATED"} else 1
except (typed_contract_check.TypedContractError, ProjectionError, OSError, UnicodeError, json.JSONDecodeError) as exc:
payload = {
"schema_version": RESULT_SCHEMA,
"status": "ERROR",
"errors": [
{
"code": getattr(exc, "code", "CONTRACT_PROJECTION_ERROR"),
"path": getattr(exc, "path", ""),
"message": str(exc),
}
],
}
exit_code = 2
json.dump(payload, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+518
View File
@@ -0,0 +1,518 @@
#!/usr/bin/env python3
"""Atomically commit one candidate document and all generated reverse views."""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
import sys
import tempfile
from typing import Any, Callable, Mapping
from fs_transaction import replace_many, stage_repository
import contract_projection
import layout_check
import moc_indexer
import proof_manifest
import quality_gate
import semantic_audit
import semantic_certificate
import semantic_surface_extractor
import typed_contract_check
import vault_migrate
SCHEMA_VERSION = "document-commit/v1"
RESULT_SCHEMA = "document-commit-result/v1"
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
HEX_SHA256 = proof_manifest.HEX_SHA256
QualityRunner = Callable[..., dict[str, Any]]
class DocumentCommitError(ValueError):
def __init__(self, code: str, message: str, location: str = "") -> None:
self.code = code
self.location = location
super().__init__(message)
class PreparedChanges(dict[Path, bytes]):
"""Final bytes plus optimistic preconditions for concurrent-edit detection."""
def __init__(self, values: Mapping[Path, bytes], expected: Mapping[Path, str | None]) -> None:
super().__init__(values)
self.expected = dict(expected)
def _projection_quality_result(staged_root: Path) -> dict[str, Any]:
updates, result = contract_projection.build_updates(staged_root)
current = result.get("status") == "CURRENT" and not updates
return {
"schema_version": contract_projection.RESULT_SCHEMA,
"status": "PASS" if current else "FAIL",
"findings": result.get("findings", []),
}
def _repo_path(root: Path, value: Any, location: str, *, must_exist: bool) -> Path:
if not isinstance(value, str) or not value or "\\" in value:
raise DocumentCommitError("INVALID_PATH", "path must be a non-empty repo-relative POSIX path", location)
candidate = Path(value)
if candidate.is_absolute():
raise DocumentCommitError("PATH_OUTSIDE_REPO", "absolute path is not allowed", location)
resolved = (root / candidate).resolve()
try:
resolved.relative_to(root)
except ValueError as exc:
raise DocumentCommitError("PATH_OUTSIDE_REPO", "path escapes repository root", location) from exc
if must_exist and not resolved.is_file():
raise DocumentCommitError("FILE_NOT_FOUND", "file does not exist", location)
return resolved
def _sha256(value: Any, location: str) -> str:
if not isinstance(value, str) or not HEX_SHA256.fullmatch(value):
raise DocumentCommitError("INVALID_SHA256", "expected 64 lowercase hexadecimal characters", location)
return value
def _plan_sha256(
root: Path,
changes: "PreparedChanges",
*,
candidate_hash: str,
proof_hash: str,
target: Path,
authority: Mapping[str, Any],
) -> str:
document = {
"schema_version": "document-commit-plan/v1",
"candidate_sha256": candidate_hash,
"proof_manifest_sha256": proof_hash,
"target": target.relative_to(root).as_posix(),
"active_layout": {
"authority": authority["authority"],
"manifest_sha256": authority["manifest_sha256"],
"mode": authority["mode"],
"write_roots": authority["write_roots"],
},
"writes": [
{
"path": path.relative_to(root).as_posix(),
"sha256": hashlib.sha256(changes[path]).hexdigest(),
"expected_sha256": changes.expected[path],
}
for path in sorted(changes)
],
}
payload = (json.dumps(document, ensure_ascii=False, separators=(",", ":"), sort_keys=True) + "\n").encode("utf-8")
return hashlib.sha256(payload).hexdigest()
def _request_parts(root: Path, request: Any) -> tuple[Path, str, Path, bool, Path, str]:
required = {"schema_version", "candidate", "target", "proof_manifest"}
if not isinstance(request, dict) or not required.issubset(request) or set(request) - required - {"semantic_audit"}:
raise DocumentCommitError("INVALID_REQUEST", "request has missing or unknown top-level fields")
if request.get("schema_version") != SCHEMA_VERSION:
raise DocumentCommitError("REQUEST_SCHEMA_MISMATCH", f"expected {SCHEMA_VERSION}")
candidate = request.get("candidate")
target = request.get("target")
proof = request.get("proof_manifest")
if not isinstance(candidate, dict) or set(candidate) != {"path", "sha256"}:
raise DocumentCommitError("INVALID_CANDIDATE", "candidate must contain path and sha256", "candidate")
if not isinstance(target, dict) or set(target) != {"path", "must_not_exist"}:
raise DocumentCommitError("INVALID_TARGET", "target must contain path and must_not_exist", "target")
if not isinstance(proof, dict) or set(proof) != {"path", "sha256"}:
raise DocumentCommitError("INVALID_PROOF_MANIFEST", "proof_manifest must contain path and sha256", "proof_manifest")
candidate_path = _repo_path(root, candidate.get("path"), "candidate.path", must_exist=True)
candidate_hash = _sha256(candidate.get("sha256"), "candidate.sha256")
target_path = _repo_path(root, target.get("path"), "target.path", must_exist=False)
if target_path.suffix != ".md" or target_path == candidate_path:
raise DocumentCommitError("INVALID_TARGET", "target must be a distinct Markdown path", "target.path")
if not isinstance(target.get("must_not_exist"), bool):
raise DocumentCommitError("INVALID_TARGET", "must_not_exist must be boolean", "target.must_not_exist")
proof_path = _repo_path(root, proof.get("path"), "proof_manifest.path", must_exist=True)
proof_hash = _sha256(proof.get("sha256"), "proof_manifest.sha256")
return candidate_path, candidate_hash, target_path, target["must_not_exist"], proof_path, proof_hash
def _semantic_parts(root: Path, request: Mapping[str, Any], run_root: Path | None) -> tuple[Path, str, Path, str] | None:
value = request.get("semantic_audit")
if value is None:
return None
if not isinstance(value, dict) or set(value) != {"request", "result"}:
raise DocumentCommitError("INVALID_SEMANTIC_AUDIT", "semantic_audit must contain request and result", "semantic_audit")
parsed: list[Path | str] = []
for key in ("request", "result"):
reference = value.get(key)
if not isinstance(reference, dict) or set(reference) not in ({"path", "sha256"}, {"namespace", "path", "sha256"}):
raise DocumentCommitError("INVALID_SEMANTIC_AUDIT", f"semantic_audit.{key} must contain namespace/path/sha256", f"semantic_audit.{key}")
namespace = reference.get("namespace", "repo")
if namespace not in {"repo", "run"}:
raise DocumentCommitError("INVALID_SEMANTIC_AUDIT", "semantic audit namespace must be repo or run", f"semantic_audit.{key}.namespace")
source_root = root if namespace == "repo" else run_root
if source_root is None:
raise DocumentCommitError("SEMANTIC_RUN_ROOT_REQUIRED", "run namespace requires semantic_run_root", f"semantic_audit.{key}.namespace")
parsed.extend((
_repo_path(source_root, reference.get("path"), f"semantic_audit.{key}.path", must_exist=True),
_sha256(reference.get("sha256"), f"semantic_audit.{key}.sha256"),
))
return parsed[0], parsed[1], parsed[2], parsed[3] # type: ignore[return-value]
def _verify_hash(path: Path, expected: str, code: str, location: str) -> bytes:
content = path.read_bytes()
observed = hashlib.sha256(content).hexdigest()
if observed != expected:
raise DocumentCommitError(code, f"expected {expected}, observed {observed}", location)
return content
def _verify_proof(path: Path, expected_hash: str, root: Path, profiles_path: Path) -> dict[str, Any]:
content = _verify_hash(path, expected_hash, "PROOF_MANIFEST_HASH_MISMATCH", "proof_manifest.sha256")
try:
manifest = json.loads(content.decode("utf-8"))
except (UnicodeError, json.JSONDecodeError) as exc:
raise DocumentCommitError("INVALID_PROOF_MANIFEST", str(exc), "proof_manifest.path") from exc
verification = manifest.get("verification") if isinstance(manifest, dict) else None
if not isinstance(verification, dict) or verification.get("status") != "PASS" or verification.get("fail_count") != 0:
raise DocumentCommitError("PROOF_NOT_PASS", "persisted proof manifest must have PASS and fail_count=0")
profiles = proof_manifest.load_allowed_profiles(profiles_path.resolve(strict=True))
try:
verified = proof_manifest.verify_manifest(manifest, root, profiles)
except proof_manifest.ManifestValidationError as exc:
codes = ",".join(sorted({issue["code"] for issue in exc.issues}))
raise DocumentCommitError("PROOF_REVALIDATION_FAILED", codes, "proof_manifest.path") from exc
if verified["verification"]["fail_count"] != 0:
raise DocumentCommitError("PROOF_NOT_PASS", "failed proofs block document completion")
return verified
def _stage_repository(root: Path, destination: Path) -> None:
stage_repository(root, destination)
def prepare(
root: Path,
request: Any,
*,
profiles_path: Path = proof_manifest.DEFAULT_PROFILES,
relations_path: Path = moc_indexer.DEFAULT_RELATIONS,
layout_path: Path = layout_check.DEFAULT_MANIFEST,
quality_runner: QualityRunner = quality_gate.run,
semantic_run_root: Path | None = None,
) -> tuple[PreparedChanges, dict[str, Any], set[Path]]:
"""Prepare verified final bytes without changing the repository."""
root = root.resolve(strict=True)
candidate, candidate_hash, target, must_not_exist, proof_path, proof_hash = _request_parts(root, request)
# 정체성 판정(TARGET_EXISTS 위치 표기·subject 대조·staging·touched/report)은 심링크를
# 따라가기 *전* 논리 경로로 한다 — canonical cutover 에서 target 을 resolve() 하면
# vault 철자가 되어 인증서의 logical(raw/…) subject 와 영구 mismatch 였다
# (branch_contract_check 가 고친 pre-resolve 규율의 쌍둥이, 2026-07-23 실측).
# resolve() 된 `target` 은 실제 바이트가 쓰일 목적지로만 쓴다.
logical_target = layout_check._lexical_absolute(root / Path(str(request["target"]["path"])))
logical_rel = logical_target.relative_to(root).as_posix()
resolved_run_root = semantic_run_root.resolve(strict=True) if semantic_run_root is not None else None
semantic_parts = _semantic_parts(root, request, resolved_run_root)
try:
authority = layout_check.resolve_authority(root, layout_path)
enforcement_targets = [target]
if (
authority["mode"] == "canonical"
and not logical_target.exists()
and not logical_target.is_symlink()
and vault_migrate._is_legacy_content_path(root, logical_target, layout_path)
):
# 신규 legacy 문서 — 목적지·호환 심링크·manifest 는 expand 의 planner 가
# 계산·검증한다(정본은 그 단계에서 계속 write-root 집행 대상).
enforcement_targets = []
layout_check.enforce_write_paths(root, enforcement_targets, authority)
except layout_check.LayoutContractError as exc:
raise DocumentCommitError(exc.code, str(exc), exc.location) from exc
if must_not_exist and target.exists():
raise DocumentCommitError("TARGET_EXISTS", "target already exists", logical_rel)
candidate_bytes = _verify_hash(candidate, candidate_hash, "CANDIDATE_HASH_MISMATCH", "candidate.sha256")
verified_proof = _verify_proof(proof_path, proof_hash, root, profiles_path)
with tempfile.TemporaryDirectory(prefix="document-commit-stage-") as directory:
stage = Path(directory) / "repo"
stage.mkdir()
_stage_repository(root, stage)
staged_target = stage / logical_rel
staged_target.parent.mkdir(parents=True, exist_ok=True)
target_original_hash = hashlib.sha256(staged_target.read_bytes()).hexdigest() if staged_target.is_file() else None
staged_target.write_bytes(candidate_bytes)
moc_updates, moc_stats = moc_indexer.build_updates(stage, relations_path)
moc_original_hashes = {
path.relative_to(stage).as_posix(): hashlib.sha256(path.read_bytes()).hexdigest()
for path in moc_updates
}
for path, text in moc_updates.items():
path.write_text(text, encoding="utf-8")
remaining, _ = moc_indexer.build_updates(stage, relations_path)
if remaining:
raise DocumentCommitError("MOC_NOT_CONVERGED", "relation indexer did not converge")
projection_updates, projection_result = contract_projection.build_updates(stage)
if projection_result["status"] == "FAIL":
codes = ",".join(sorted({str(item.get("code", "UNKNOWN")) for item in projection_result["findings"]}))
raise DocumentCommitError("TYPED_CONTRACT_FAILED", codes)
projection_original_hashes = {
path.relative_to(stage).as_posix(): hashlib.sha256(path.read_bytes()).hexdigest()
for path in projection_updates
}
for path, text in projection_updates.items():
path.write_text(text, encoding="utf-8")
projection_remaining, projection_after = contract_projection.build_updates(stage)
if projection_remaining or projection_after["status"] != "CURRENT":
raise DocumentCommitError("CONTRACT_PROJECTION_NOT_CONVERGED", "typed projections did not converge")
staged_policy = stage / semantic_surface_extractor.DEFAULT_POLICY
target_frontmatter = semantic_surface_extractor.parse_frontmatter(staged_target.read_text(encoding="utf-8"))
semantic_required = False
if str(target_frontmatter.get("source_type", "")) in {"project-note", "branch-note"}:
if not staged_policy.is_file():
raise DocumentCommitError("SEMANTIC_POLICY_SOURCE_MISSING", "design-bearing document requires semantic surface policy")
policy = semantic_surface_extractor.load_policy(stage)
semantic_required = semantic_surface_extractor.is_required(target_frontmatter, policy)
certificate_stage_path: Path | None = None
certificate_root_path: Path | None = None
certificate_original_hash: str | None = None
certificate_document: dict[str, Any] | None = None
if semantic_parts is not None:
audit_request_path, audit_request_hash, audit_result_path, audit_result_hash = semantic_parts
request_bytes = _verify_hash(audit_request_path, audit_request_hash, "SEMANTIC_AUDIT_REQUEST_HASH_MISMATCH", "semantic_audit.request.sha256")
result_bytes = _verify_hash(audit_result_path, audit_result_hash, "SEMANTIC_AUDIT_RESULT_HASH_MISMATCH", "semantic_audit.result.sha256")
try:
audit_request = json.loads(request_bytes.decode("utf-8"))
audit_result = json.loads(result_bytes.decode("utf-8"))
validated_audit = semantic_audit.validate_result(
stage,
audit_request,
audit_result,
profiles_path=profiles_path,
run_root=resolved_run_root,
)
certificate_stage_path, certificate_bytes, certificate_document = semantic_certificate.prepare_certificate(
stage,
validated_audit,
audit_request=audit_request,
audit_result=audit_result,
run_root=resolved_run_root,
)
except (
UnicodeError,
json.JSONDecodeError,
semantic_audit.SemanticAuditError,
semantic_certificate.SemanticCertificateError,
proof_manifest.ManifestValidationError,
) as exc:
raise DocumentCommitError("SEMANTIC_AUDIT_FAILED", str(exc), "semantic_audit") from exc
if certificate_document["subject"] != logical_rel:
raise DocumentCommitError("SEMANTIC_AUDIT_SUBJECT_MISMATCH", "semantic audit subject must equal target path", "semantic_audit")
if certificate_document["verdict"] != "PASS":
raise DocumentCommitError("SEMANTIC_BLOCKING_VERDICT", "semantic audit contains blocking or readiness-blocking findings", "semantic_audit")
certificate_stage_path.parent.mkdir(parents=True, exist_ok=True)
certificate_stage_path.write_bytes(certificate_bytes)
certificate_root_path = root / certificate_stage_path.relative_to(stage)
certificate_original_hash = hashlib.sha256(certificate_root_path.read_bytes()).hexdigest() if certificate_root_path.is_file() else None
elif semantic_required:
raise DocumentCommitError("SEMANTIC_CERTIFICATE_MISSING", "required design-bearing candidate has no semantic audit", logical_rel)
touched_rel = {logical_rel}
touched_rel.update(path.relative_to(stage).as_posix() for path in moc_updates)
touched_rel.update(path.relative_to(stage).as_posix() for path in projection_updates)
extensions = [
quality_gate.QualityExtension(
"typed-contract",
lambda staged: typed_contract_check.check(
staged,
include_projection=False,
),
),
quality_gate.QualityExtension(
"contract-projection",
_projection_quality_result,
),
]
if certificate_stage_path is not None:
extensions.append(
quality_gate.QualityExtension(
"semantic-certificate",
lambda staged: semantic_certificate.quality_extension(staged, [staged_target]),
)
)
try:
gate = quality_runner(
stage,
sorted(touched_rel),
structure_paths=[logical_rel],
template_root=root,
include_graph=True,
require_moc_convergence=True,
extensions=extensions,
)
except quality_gate.QualityGateError as exc:
raise DocumentCommitError("QUALITY_GATE_ERROR", str(exc)) from exc
if not isinstance(gate, dict) or gate.get("schema_version") != "quality-gate-result/v1":
raise DocumentCommitError("QUALITY_GATE_ERROR", "unexpected quality gate result schema")
if gate.get("status") != "PASS":
codes = ",".join(sorted({str(item.get("code", "UNKNOWN")) for item in gate.get("findings", [])}))
raise DocumentCommitError("QUALITY_GATE_FAILED", codes)
raw_changes = {root / rel: (stage / rel).read_bytes() for rel in sorted(touched_rel)}
expected_before_expansion = {
root / rel: (
target_original_hash
if rel == logical_rel
else moc_original_hashes.get(rel)
if rel in moc_original_hashes
else projection_original_hashes.get(rel)
)
for rel in sorted(touched_rel)
}
try:
expanded, authority = vault_migrate.expand_authoritative_changes(
root,
raw_changes,
layout_path=layout_path,
relations_path=relations_path,
)
except vault_migrate.MigrationError as exc:
raise DocumentCommitError(exc.code, str(exc), exc.location) from exc
if certificate_stage_path is not None and certificate_root_path is not None:
expanded[certificate_root_path] = certificate_stage_path.read_bytes()
expected_before_expansion[certificate_root_path] = certificate_original_hash
expected = {
path: expected_before_expansion.get(
path,
hashlib.sha256(path.read_bytes()).hexdigest() if path.is_file() else None,
)
for path in expanded
}
changes = PreparedChanges(expanded, expected)
forbidden = {path for path, digest in changes.expected.items() if digest is None} if must_not_exist else set()
plan_sha256 = _plan_sha256(
root,
changes,
candidate_hash=candidate_hash,
proof_hash=proof_hash,
target=root / logical_rel,
authority=authority,
)
result = {
"target": logical_rel,
"candidate_sha256": candidate_hash,
"proof_manifest": proof_path.relative_to(root).as_posix(),
"proof_manifest_sha256": proof_hash,
"proof_count": verified_proof["verification"]["proof_count"],
"semantic_certificate": (
certificate_root_path.relative_to(root).as_posix()
if certificate_root_path is not None
else None
),
"relation_edges": moc_stats["canonical_edges"],
"changed_paths": [path.relative_to(root).as_posix() for path in sorted(changes)],
"active_layout": {
"mode": authority["mode"],
"authority": authority["authority"],
"write_roots": authority["write_roots"],
"manifest_sha256": authority["manifest_sha256"],
},
"plan_sha256": plan_sha256,
}
return changes, result, forbidden
def commit(changes: PreparedChanges, forbidden: set[Path]) -> None:
"""Check snapshot preconditions, then perform exactly one multi-file replace."""
for path, expected_hash in changes.expected.items():
if expected_hash is None:
if path.exists():
raise DocumentCommitError("CONCURRENT_MODIFICATION", "new target appeared after staging", path.as_posix())
continue
if not path.is_file():
raise DocumentCommitError("CONCURRENT_MODIFICATION", "existing target disappeared after staging", path.as_posix())
observed = hashlib.sha256(path.read_bytes()).hexdigest()
if observed != expected_hash:
raise DocumentCommitError(
"CONCURRENT_MODIFICATION",
f"expected pre-commit hash {expected_hash}, observed {observed}",
path.as_posix(),
)
replace_many(changes, must_not_exist=forbidden)
def _failure(exc: Exception) -> dict[str, Any]:
return {
"schema_version": RESULT_SCHEMA,
"status": "FAIL",
"error": {
"code": getattr(exc, "code", "IO_ERROR"),
"location": getattr(exc, "location", ""),
"message": str(exc),
},
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("request", type=Path)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--profiles", type=Path, default=proof_manifest.DEFAULT_PROFILES)
parser.add_argument("--relations", type=Path, default=moc_indexer.DEFAULT_RELATIONS)
parser.add_argument("--layout", type=Path, default=layout_check.DEFAULT_MANIFEST)
parser.add_argument("--semantic-run-root", type=Path)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--dry-run", action="store_true")
mode.add_argument("--apply", action="store_true")
parser.add_argument("--expected-plan-sha256")
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
request = json.loads(args.request.read_text(encoding="utf-8"))
changes, result, forbidden = prepare(
root,
request,
profiles_path=args.profiles,
relations_path=args.relations,
layout_path=args.layout,
semantic_run_root=args.semantic_run_root,
)
if args.apply:
if args.expected_plan_sha256 is not None:
expected = args.expected_plan_sha256
if not HEX_SHA256.fullmatch(expected):
raise DocumentCommitError("INVALID_PLAN_SHA256", "expected plan hash must be 64 lowercase hex characters")
if expected != result["plan_sha256"]:
raise DocumentCommitError(
"PLAN_HASH_MISMATCH",
f"expected {expected}, current plan is {result['plan_sha256']}",
)
commit(changes, forbidden)
json.dump(
{"schema_version": RESULT_SCHEMA, "status": "APPLIED" if args.apply else "DRY_RUN", **result},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 0
except (DocumentCommitError, proof_manifest.ManifestValidationError) as exc:
json.dump(_failure(exc), sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 1
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
json.dump(_failure(exc), sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+322
View File
@@ -0,0 +1,322 @@
#!/usr/bin/env python3
"""Resolve risk-based semantic and adversarial review requirements."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import sys
from typing import Any
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_PROFILES = DEFAULT_ROOT / "harness/source/execution-profiles.json"
DEFAULT_WORKFLOW_DIR = DEFAULT_ROOT / "harness/source/workflows"
SCHEMA_VERSION = "execution-profile-result/v1"
OUTPUT_MODES = {
"failures-only",
"decision-risk-summary",
"detailed-artifact",
"public-claim-verification",
}
WORKFLOW_KINDS = {"orchestrated", "deterministic"}
class ProfileError(ValueError):
pass
def load_profiles(path: Path) -> dict[str, Any]:
document = json.loads(path.read_text(encoding="utf-8"))
if document.get("schema_version") != "execution-profiles/v1":
raise ProfileError("expected execution-profiles/v1")
levels = document.get("risk_levels")
profiles = document.get("profiles")
checks = document.get("always_checks")
mandatory = document.get("mandatory_gates")
if not isinstance(levels, list) or not levels or len(set(levels)) != len(levels):
raise ProfileError("risk_levels must be a non-empty unique list")
if not isinstance(profiles, dict) or not profiles:
raise ProfileError("profiles must be a non-empty object")
if not isinstance(checks, list) or not checks or not all(isinstance(item, str) for item in checks):
raise ProfileError("always_checks must be a non-empty string list")
if mandatory != {
"typed_contract": "required_for_design_bearing",
"semantic_coherence": "required_for_design_bearing",
"proof_manifest": "required_when_claims_present",
}:
raise ProfileError("mandatory_gates must declare the v1 typed/semantic/proof policies")
for profile, config in profiles.items():
if not isinstance(profile, str) or not isinstance(config, dict):
raise ProfileError("profile entries must be named objects")
dispatch = config.get("dispatch_contract")
if not isinstance(dispatch, dict) or dispatch != {
"semantic_review": "when-required",
"adversarial_review": "when-required",
}:
raise ProfileError(f"{profile}: invalid dispatch_contract")
output = config.get("output_contract")
if not isinstance(output, dict) or output.get("mode") not in OUTPUT_MODES:
raise ProfileError(f"{profile}: invalid output_contract.mode")
include = output.get("include")
if (
not isinstance(include, list)
or not include
or not all(isinstance(item, str) and item for item in include)
or len(set(include)) != len(include)
):
raise ProfileError(f"{profile}: output_contract.include must be unique strings")
intensity = config.get("review_intensity")
if (
not isinstance(intensity, dict)
or set(intensity) != {"semantic_passes", "adversarial_findings", "impact_scope"}
or not isinstance(intensity.get("semantic_passes"), int)
or isinstance(intensity.get("semantic_passes"), bool)
or intensity["semantic_passes"] < 1
or not isinstance(intensity.get("adversarial_findings"), bool)
or intensity.get("impact_scope") not in {"direct", "transitive", "full-hub"}
):
raise ProfileError(f"{profile}: invalid review_intensity")
return document
def load_workflow_contract(workflow: str, workflow_dir: Path = DEFAULT_WORKFLOW_DIR) -> tuple[str, dict[str, Any]]:
if not workflow or Path(workflow).name != workflow or workflow.endswith(".json"):
raise ProfileError("workflow must be an id without path separators or extension")
path = workflow_dir / f"{workflow}.json"
data = json.loads(path.read_text(encoding="utf-8"))
if data.get("schema_version") != 2 or data.get("source_kind") != "workflow":
raise ProfileError(f"{path}: expected workflow schema_version 2")
if data.get("id") != workflow:
raise ProfileError(f"{path}: workflow id mismatch")
if "profile" in data or "default_risk" in data:
raise ProfileError(f"{path}: profile/default_risk must be nested")
contract = data.get("execution_contract")
if not isinstance(contract, dict):
raise ProfileError(f"{path}: missing execution_contract")
allowed = {
"kind",
"profile",
"default_risk",
"entrypoint",
"dry_run_first",
"result_schema",
"design_bearing",
}
if set(contract) - allowed:
raise ProfileError(f"{path}: unsupported execution_contract fields")
kind = contract.get("kind")
if kind not in WORKFLOW_KINDS:
raise ProfileError(f"{path}: unsupported execution kind")
deterministic = {"entrypoint", "dry_run_first", "result_schema"}
if kind == "deterministic":
if not deterministic.issubset(contract):
raise ProfileError(f"{path}: incomplete deterministic execution contract")
entrypoint = contract.get("entrypoint")
result_schema = contract.get("result_schema")
if (
not isinstance(entrypoint, str)
or not entrypoint.endswith(".py")
or Path(entrypoint).is_absolute()
or ".." in Path(entrypoint).parts
or contract.get("dry_run_first") is not True
or not isinstance(result_schema, str)
or not result_schema
):
raise ProfileError(f"{path}: invalid deterministic execution contract")
elif deterministic.intersection(contract):
raise ProfileError(f"{path}: orchestrated workflow declares deterministic fields")
profile = contract.get("profile")
risk = contract.get("default_risk")
design_bearing = contract.get("design_bearing")
if not isinstance(profile, str) or not isinstance(risk, str) or not isinstance(design_bearing, bool):
raise ProfileError(f"{path}: execution_contract requires profile/default_risk/design_bearing")
return workflow, dict(contract)
def _condition_matches(condition: str, context: dict[str, Any], levels: list[str]) -> bool:
if condition == "public_claims_present":
return bool(context["public_claims_present"])
if condition.startswith("finding_count>="):
try:
threshold = int(condition.split(">=", 1)[1])
except ValueError as exc:
raise ProfileError(f"invalid condition: {condition}") from exc
return context["finding_count"] >= threshold
if condition.startswith("risk>="):
threshold = condition.split(">=", 1)[1]
if threshold not in levels:
raise ProfileError(f"unknown risk threshold: {threshold}")
return levels.index(context["risk"]) >= levels.index(threshold)
raise ProfileError(f"unsupported condition: {condition}")
def _required(rule: Any, context: dict[str, Any], levels: list[str]) -> tuple[bool, list[str]]:
if not isinstance(rule, dict):
raise ProfileError("review rule must be an object")
mode = rule.get("mode")
if mode == "not_required":
return False, []
if mode == "required":
return True, ["profile requires review"]
if mode == "required_at_or_above_risk":
threshold = rule.get("minimum_risk")
if threshold not in levels:
raise ProfileError(f"unknown minimum_risk: {threshold}")
matched = levels.index(context["risk"]) >= levels.index(threshold)
return matched, [f"risk>={threshold}"] if matched else []
if mode == "required_when":
conditions = rule.get("conditions")
if not isinstance(conditions, list) or not conditions or not all(
isinstance(item, str) for item in conditions
):
raise ProfileError("required_when.conditions must be a non-empty string list")
matched = [item for item in conditions if _condition_matches(item, context, levels)]
return bool(matched), matched
raise ProfileError(f"unsupported review mode: {mode}")
def resolve_policy(
document: dict[str, Any],
profile: str,
risk: str,
finding_count: int,
public_claims_present: bool,
design_bearing: bool = False,
claims_present: bool = False,
) -> dict[str, Any]:
levels = document["risk_levels"]
profiles = document["profiles"]
if profile not in profiles:
raise ProfileError(f"unknown profile: {profile}")
if risk not in levels:
raise ProfileError(f"unknown risk: {risk}")
if finding_count < 0:
raise ProfileError("finding_count must be >= 0")
context = {
"risk": risk,
"finding_count": finding_count,
"public_claims_present": public_claims_present,
"design_bearing": design_bearing,
"claims_present": claims_present,
}
config = profiles[profile]
semantic, semantic_reasons = _required(config.get("semantic_review"), context, levels)
if design_bearing and not semantic:
semantic = True
semantic_reasons = ["mandatory_gates.semantic_coherence"]
adversarial, adversarial_reasons = _required(config.get("adversarial_review"), context, levels)
output_contract = dict(config["output_contract"])
base_intensity = dict(config["review_intensity"])
risk_index = levels.index(risk)
if risk_index >= levels.index("high"):
base_intensity["semantic_passes"] += 1
if base_intensity["impact_scope"] == "direct":
base_intensity["impact_scope"] = "transitive"
base_intensity["adversarial_findings"] = bool(
base_intensity["adversarial_findings"] or adversarial
)
mandatory_gates = {
"typed_contract": "required" if design_bearing else "skip",
"semantic_coherence": "required" if design_bearing else "skip",
"proof_manifest": "required" if claims_present or public_claims_present else "skip",
}
return {
"schema_version": SCHEMA_VERSION,
"status": "RESOLVED",
"profile": profile,
"context": context,
"always_checks": list(document["always_checks"]),
"mandatory_gates": mandatory_gates,
"review_intensity": base_intensity,
"semantic_review": {"required": semantic, "matched_conditions": semantic_reasons},
"adversarial_review": {"required": adversarial, "matched_conditions": adversarial_reasons},
"dispatch": {
"semantic_review": "dispatch" if semantic else "skip",
"adversarial_review": "dispatch" if adversarial else "skip",
},
"output_contract": output_contract,
"output_mode": output_contract["mode"],
}
def resolve_workflow_policy(
document: dict[str, Any],
workflow: str,
workflow_dir: Path,
risk: str | None,
finding_count: int,
public_claims_present: bool,
claims_present: bool = False,
) -> dict[str, Any]:
workflow_id, contract = load_workflow_contract(workflow, workflow_dir)
selected_risk = risk or contract["default_risk"]
result = resolve_policy(
document,
contract["profile"],
selected_risk,
finding_count,
public_claims_present,
bool(contract["design_bearing"]),
claims_present,
)
result["workflow"] = workflow_id
result["execution_contract"] = contract
return result
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("profile", nargs="?")
parser.add_argument("--workflow", help="resolve profile/default risk from neutral workflow metadata")
parser.add_argument("--risk", help="override workflow default risk; required with positional profile")
parser.add_argument("--finding-count", type=int, default=0)
parser.add_argument("--public-claims-present", action="store_true")
parser.add_argument("--claims-present", action="store_true")
parser.add_argument("--profiles", type=Path, default=DEFAULT_PROFILES)
parser.add_argument("--workflow-dir", type=Path, default=DEFAULT_WORKFLOW_DIR)
args = parser.parse_args(argv)
try:
document = load_profiles(args.profiles.resolve(strict=True))
if bool(args.profile) == bool(args.workflow):
raise ProfileError("provide exactly one positional profile or --workflow")
if args.workflow:
result = resolve_workflow_policy(
document,
args.workflow,
args.workflow_dir.resolve(strict=True),
args.risk,
args.finding_count,
args.public_claims_present,
args.claims_present,
)
else:
if args.risk is None:
raise ProfileError("--risk is required with positional profile")
result = resolve_policy(
document,
args.profile,
args.risk,
args.finding_count,
args.public_claims_present,
False,
args.claims_present,
)
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 0
except (OSError, UnicodeError, json.JSONDecodeError, ProfileError) as exc:
json.dump(
{"schema_version": SCHEMA_VERSION, "status": "FAIL", "error": str(exc)},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""Atomically convert checker-confirmed bare branch decision/owner refs to wikilinks."""
from __future__ import annotations
import argparse
import importlib.util
import json
from pathlib import Path
import re
import sys
from typing import Any
from fs_transaction import replace_many
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
def _load_checker(root: Path):
path = root / ".claude/hooks/wiki_consistency_check.py"
hooks = str(path.parent)
if hooks not in sys.path:
sys.path.insert(0, hooks)
spec = importlib.util.spec_from_file_location("wiki_consistency_fix_source", path)
if spec is None or spec.loader is None:
raise OSError(f"cannot load checker: {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _replace_slug(line: str, slug: str, branch_dir: str) -> tuple[str, bool]:
pattern = re.compile(
rf"(?<![\/\w-])`?{re.escape(slug)}`?(?![\w-])"
)
rendered, count = pattern.subn(f"[[{branch_dir}/{slug}]]", line, count=1)
return rendered, bool(count)
def build_updates(root: Path) -> tuple[dict[Path, str], dict[str, Any]]:
checker = _load_checker(root)
updates: dict[Path, str] = {}
decision_fixes = 0
owner_fixes = 0
for branch_path in sorted((root / checker.BRANCH_DIR).glob("*.md")):
text = branch_path.read_text(encoding="utf-8")
lines = text.splitlines(keepends=True)
for line_number, slug, _ref_id, kind in checker.extract_refs(text, branch_path.stem):
if kind != "bare":
continue
original = lines[line_number - 1]
lines[line_number - 1], changed = _replace_slug(original, slug, checker.BRANCH_DIR)
if changed:
decision_fixes += 1
current = "".join(lines)
coverage = checker.coverage_rows(current)
for line_number, _concern, status, owner in coverage:
if "delegated" not in status or not owner:
continue
slugs = [match.group(1) for match in checker.BARE_SLUG_RE.finditer(owner)]
for slug in slugs:
original = lines[line_number - 1]
lines[line_number - 1], changed = _replace_slug(original, slug, checker.BRANCH_DIR)
if changed:
owner_fixes += 1
rendered = "".join(lines)
if rendered != text:
updates[branch_path] = rendered
return updates, {
"decision_ref_fixes": decision_fixes,
"owner_ref_fixes": owner_fixes,
"changed_files": [path.relative_to(root).as_posix() for path in sorted(updates)],
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--check", action="store_true")
mode.add_argument("--write", action="store_true")
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
updates, counts = build_updates(root)
if args.write and updates:
replace_many({path: text.encode("utf-8") for path, text in updates.items()})
result = {
"schema_version": "bare-ref-migration-result/v1",
"status": "DRIFT" if args.check and updates else "UPDATED" if updates else "CURRENT",
**counts,
}
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 1 if args.check and updates else 0
except (OSError, UnicodeError, ValueError) as exc:
json.dump({"schema_version": "bare-ref-migration-result/v1", "status": "FAIL", "error": str(exc)}, sys.stdout, ensure_ascii=False, indent=2)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+179
View File
@@ -0,0 +1,179 @@
#!/usr/bin/env python3
"""Recoverable multi-file replacement using same-filesystem ``os.replace``."""
from __future__ import annotations
import os
import shutil
from dataclasses import dataclass
from pathlib import Path
import tempfile
from typing import Mapping
class TransactionError(OSError):
"""A commit failed; rollback details are included in the message."""
@dataclass(frozen=True)
class SymlinkValue:
"""A lexical symlink target to install with the surrounding transaction."""
target: str
ReplacementValue = bytes | SymlinkValue
OriginalValue = tuple[str, bytes | str, int | None]
def _lexical_absolute(path: Path) -> Path:
"""Make a path absolute without following an existing symlink."""
return Path(os.path.abspath(path))
def _temporary_bytes(target: Path, data: bytes, mode: int | None = None) -> Path:
target.parent.mkdir(parents=True, exist_ok=True)
descriptor, temporary_name = tempfile.mkstemp(prefix=f".{target.name}.", suffix=".tmp", dir=target.parent)
temporary = Path(temporary_name)
try:
with os.fdopen(descriptor, "wb") as stream:
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
if mode is not None:
os.chmod(temporary, mode)
return temporary
except Exception:
temporary.unlink(missing_ok=True)
raise
def _temporary_symlink(target: Path, link_target: str) -> Path:
target.parent.mkdir(parents=True, exist_ok=True)
descriptor, temporary_name = tempfile.mkstemp(prefix=f".{target.name}.", suffix=".tmp", dir=target.parent)
os.close(descriptor)
temporary = Path(temporary_name)
temporary.unlink()
try:
os.symlink(link_target, temporary)
return temporary
except Exception:
temporary.unlink(missing_ok=True)
raise
def _temporary_value(target: Path, value: ReplacementValue, mode: int | None = None) -> Path:
if isinstance(value, SymlinkValue):
return _temporary_symlink(target, value.target)
return _temporary_bytes(target, value, mode)
def _capture_original(target: Path) -> OriginalValue | None:
if target.is_symlink():
return ("symlink", os.readlink(target), None)
if target.exists():
return ("bytes", target.read_bytes(), target.stat().st_mode & 0o777)
return None
def _write_target(path: Path, value: ReplacementValue, follow_symlinks: bool) -> Path:
"""Pick the entry this value should land on.
vault cutover 이후 ``raw/**``·``wiki/**`` 의 문서는 ``vault/**`` 정본을 가리키는
심링크다. ``os.replace`` 는 심링크를 *따라가지 않고* 그 자리를 실파일로 갈아치우므로,
호출자가 심링크 경로를 그대로 넘기면 (1) 호환 계층이 끊기고 (2) 정본은 낡은 채로
남는 split-brain 이 된다 — 그리고 그 사실이 조용하다. 그래서 bytes 쓰기는 기본적으로
심링크를 따라 정본에 쓴다.
``SymlinkValue`` 는 링크 *자체* 를 설치하는 것이므로 언제나 lexical 경로에 쓴다.
``follow_symlinks=False`` 는 cutover/rollback 처럼 심링크를 실파일로 되돌리는 것이
목적인 호출자(``vault_migrate``)를 위한 예외다.
"""
if isinstance(value, SymlinkValue) or not follow_symlinks:
return _lexical_absolute(path)
if path.is_symlink():
return _lexical_absolute(path.resolve())
return _lexical_absolute(path)
_STAGE_IGNORE = shutil.ignore_patterns(".git", "__pycache__", "*.pyc", ".DS_Store")
def stage_repository(root: Path, destination: Path) -> None:
"""Copy the working tree into ``destination`` *preserving symlinks*.
canonical 모드에서 ``raw/``·``wiki/`` 는 ``vault/`` 정본을 가리키는 상대 심링크
디렉토리다. 기본 ``copytree``(``symlinks=False``)는 각 링크를 따라가 실파일로 복제하므로
스테이지가 split-brain(정본·링크가 독립된 실파일 2개)이 되고, candidate 가 legacy 경로로
정본을 우회 편집해도 투영/레이아웃 검증이 그 사실을 못 잡는다. ``symlinks=True`` 로
링크를 링크 그대로 복제해 실제 저장소 구조를 재현한다.
(검증 스테이징의 단일 구현 — 예전엔 document_commit / branch_contract_check /
migrate_graph_contracts 에 같은 함수가 세 벌 복사돼 있었고, 그 중 하나만 고치면
나머지가 조용히 어긋났다.)
"""
for child in root.iterdir():
if child.name == ".git":
continue
target = destination / child.name
if child.is_dir():
shutil.copytree(child, target, ignore=_STAGE_IGNORE, symlinks=True)
elif child.is_file():
shutil.copy2(child, target, follow_symlinks=False)
def replace_many(
changes: Mapping[Path, ReplacementValue],
*,
must_not_exist: set[Path] | None = None,
follow_symlinks: bool = True,
) -> None:
"""Replace bytes/symlinks atomically and restore the original entry kind on failure."""
forbidden = {_lexical_absolute(path) for path in (must_not_exist or set())}
normalized = {
_write_target(path, data, follow_symlinks): data for path, data in changes.items()
}
for target in forbidden:
if target.exists() or target.is_symlink():
raise FileExistsError(f"target already exists: {target}")
originals: dict[Path, OriginalValue | None] = {}
pending: dict[Path, Path] = {}
committed: list[Path] = []
try:
for target, value in normalized.items():
originals[target] = _capture_original(target)
original_mode = originals[target][2] if originals[target] is not None else None
pending[target] = _temporary_value(target, value, original_mode)
except Exception:
for temporary in pending.values():
temporary.unlink(missing_ok=True)
raise
try:
for target in sorted(pending, key=lambda item: item.as_posix()):
os.replace(pending[target], target)
committed.append(target)
except Exception as exc:
rollback_errors: list[str] = []
for target in reversed(committed):
original = originals[target]
try:
if original is None:
target.unlink(missing_ok=True)
else:
kind, payload, mode = original
restore = (
_temporary_symlink(target, str(payload))
if kind == "symlink"
else _temporary_bytes(target, bytes(payload), mode)
)
os.replace(restore, target)
except Exception as rollback_exc: # pragma: no cover - catastrophic filesystem failure
rollback_errors.append(f"{target}: {rollback_exc}")
detail = f"; rollback failures: {rollback_errors}" if rollback_errors else "; rollback completed"
raise TransactionError(f"multi-file commit failed: {exc}{detail}") from exc
finally:
for temporary in pending.values():
temporary.unlink(missing_ok=True)
+450
View File
@@ -0,0 +1,450 @@
#!/usr/bin/env python3
"""Validate compatibility, shadow, and canonical project-first vault layouts."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
from contract_markdown import parse_frontmatter
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_MANIFEST = Path("harness/source/vault-layout.json")
CONTENT_ROOTS = ("raw", "wiki")
MODES = {"compatibility", "shadow", "canonical"}
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
WIKILINK = re.compile(r"\[\[([^\]|#]+)(?:#[^\]|]+)?(?:\|[^\]]+)?\]\]")
FENCE = re.compile(r"^[ \t]{0,3}(`{3,}|~{3,})")
INLINE_CODE = re.compile(r"(`+)(.*?)\1")
class LayoutContractError(ValueError):
def __init__(self, code: str, message: str, location: str = "") -> None:
self.code = code
self.location = location
super().__init__(message)
def _finding(code: str, path: str, detail: str = "") -> dict[str, str]:
item = {"code": code, "path": path}
if detail:
item["detail"] = detail
return item
def _lexical_absolute(path: Path) -> Path:
"""Return an absolute repository path without following redirects."""
return Path(os.path.abspath(path))
def _load_embedded_or_path(root: Path, value: Any, expected_schema: str, findings: list[dict[str, str]], location: str) -> Mapping[str, Any] | None:
document = value
if isinstance(value, str):
candidate = (root / value).resolve()
try:
candidate.relative_to(root)
document = json.loads(candidate.read_text(encoding="utf-8"))
except (ValueError, OSError, UnicodeError, json.JSONDecodeError) as exc:
findings.append(_finding("CUTOVER_SOURCE_ERROR", value, str(exc)))
return None
if not isinstance(document, dict) or document.get("schema_version") != expected_schema:
findings.append(_finding("INVALID_CUTOVER_SCHEMA", location, f"expected {expected_schema}"))
return None
return document
def _safe_repo_path(root: Path, value: Any, findings: list[dict[str, str]], location: str) -> Path | None:
if not isinstance(value, str) or not value or "\\" in value or Path(value).is_absolute():
findings.append(_finding("INVALID_MIGRATION_PATH", location, str(value)))
return None
resolved = _lexical_absolute(root / value)
try:
resolved.relative_to(root)
except ValueError:
findings.append(_finding("INVALID_MIGRATION_PATH", location, str(value)))
return None
if resolved.is_symlink():
try:
resolved.resolve().relative_to(root)
except ValueError:
findings.append(_finding("INVALID_MIGRATION_PATH", location, "symlink target escapes repository"))
return None
return resolved
def _content_files(root: Path, assignments: Mapping[str, str]) -> set[Path]:
files: set[Path] = set()
for source in assignments:
base = root / source
if base.is_dir():
files.update(
path for path in base.rglob("*")
if path.is_file() and path.name != ".gitkeep"
)
return files
def _migration_entries(
root: Path,
manifest: Mapping[str, Any],
findings: list[dict[str, str]],
) -> tuple[dict[Path, tuple[Path, str]], dict[Path, Path]]:
entries = manifest.get("entries")
if not isinstance(entries, list):
findings.append(_finding("INVALID_CUTOVER_SCHEMA", "migration_manifest.entries", "must be an array"))
return {}, {}
by_legacy: dict[Path, tuple[Path, str]] = {}
by_canonical: dict[Path, Path] = {}
for index, item in enumerate(entries):
location = f"migration_manifest.entries[{index}]"
if not isinstance(item, dict) or set(item) != {"legacy_path", "canonical_path", "sha256"}:
findings.append(_finding("INVALID_MIGRATION_ENTRY", location))
continue
legacy = _safe_repo_path(root, item.get("legacy_path"), findings, f"{location}.legacy_path")
canonical = _safe_repo_path(root, item.get("canonical_path"), findings, f"{location}.canonical_path")
digest = item.get("sha256")
if not isinstance(digest, str) or not HEX_SHA256.fullmatch(digest):
findings.append(_finding("INVALID_MIGRATION_HASH", location, str(digest)))
continue
if legacy is None or canonical is None:
continue
if legacy in by_legacy:
findings.append(_finding("DUPLICATE_LEGACY_OWNER", legacy.relative_to(root).as_posix()))
if canonical in by_canonical:
findings.append(_finding("DUPLICATE_CANONICAL_OWNER", canonical.relative_to(root).as_posix()))
by_legacy[legacy] = (canonical, digest)
by_canonical[canonical] = legacy
return by_legacy, by_canonical
def _validate_links(
root: Path,
files: Iterable[Path],
findings: list[dict[str, str]],
*,
namespace: Iterable[Path] | None = None,
) -> None:
namespace_items = list(namespace if namespace is not None else root.rglob("*"))
all_files = [path for path in namespace_items if path.is_file() and ".git" not in path.parts]
namespace_paths = {path.resolve() for path in all_files}
by_basename: dict[str, list[Path]] = {}
by_name: dict[str, list[Path]] = {}
for path in all_files:
by_basename.setdefault(path.stem, []).append(path)
by_name.setdefault(path.name, []).append(path)
for source in sorted(path for path in set(files) if path.suffix == ".md"):
lines: list[str] = []
fence_char = ""
fence_length = 0
for line in source.read_text(encoding="utf-8").splitlines():
if fence_char:
if re.fullmatch(rf"[ \t]{{0,3}}{re.escape(fence_char)}{{{fence_length},}}[ \t]*", line):
fence_char, fence_length = "", 0
continue
match = FENCE.match(line)
if match:
token = match.group(1)
if not fence_char:
fence_char, fence_length = token[0], len(token)
continue
lines.append(INLINE_CODE.sub("", line))
text = "\n".join(lines).replace(r"\|", "|")
for target_value in WIKILINK.findall(text):
target = target_value.strip()
if not target:
continue
if "/" not in target:
matches = (
by_name.get(Path(target).name, [])
if Path(target).suffix
else by_basename.get(Path(target).stem, [])
)
if not matches:
findings.append(_finding("BROKEN_LINK", source.relative_to(root).as_posix(), target))
elif len(matches) > 1:
findings.append(_finding("AMBIGUOUS_BASENAME_LINK", source.relative_to(root).as_posix(), target))
continue
candidate = root / target
if not candidate.is_file():
markdown_candidate = Path(str(candidate) + ".md")
if markdown_candidate.is_file():
candidate = markdown_candidate
if not candidate.is_file():
findings.append(_finding("BROKEN_LINK", source.relative_to(root).as_posix(), target))
elif namespace is not None and candidate.resolve() not in namespace_paths:
# STALE_AUTHORITY_LINK 은 "권한 밖 사본을 가리키는 링크"를 잡는다. shadow 모드
# (namespace=legacy|external)에서 canonical(vault/) 경로를 *미리* 가리키면 발화한다.
# canonical 모드(namespace=canonical|external)에서는 [[raw/…]]·[[wiki/…]] 레거시
# 경로가 호환 심링크를 통해 canonical 로 resolve() 되므로 여기 걸리지 않는다 —
# 이는 의도된 것이다(raw/·wiki/ 는 영구 호환 별칭). 회귀 방지:
# test_vault_migrate.test_canonical_symlink_preserves_escaped_alias_wikilink_without_stale_authority.
findings.append(_finding("STALE_AUTHORITY_LINK", source.relative_to(root).as_posix(), target))
def _validate_cutover(
root: Path,
mode: str,
vault_root: Path,
assignments: Mapping[str, str],
manifest: Mapping[str, Any],
findings: list[dict[str, str]],
) -> dict[str, int]:
migration = _load_embedded_or_path(root, manifest.get("migration_manifest"), "vault-migration/v1", findings, "migration_manifest")
rollback = _load_embedded_or_path(root, manifest.get("rollback_mapping"), "vault-rollback/v1", findings, "rollback_mapping")
if migration is None or rollback is None:
return {"migration_entries": 0, "canonical_files": 0}
by_legacy, by_canonical = _migration_entries(root, migration, findings)
legacy_files = _content_files(root, assignments)
canonical_files = {
path for path in vault_root.rglob("*")
if path.is_file() and path.name != ".gitkeep" and path != vault_root / "README.md"
}
if set(by_legacy) != legacy_files:
for path in sorted(legacy_files - set(by_legacy)):
findings.append(_finding("MIGRATION_ENTRY_MISSING", path.relative_to(root).as_posix()))
for path in sorted(set(by_legacy) - legacy_files):
findings.append(_finding("MIGRATION_LEGACY_EXTRA", path.relative_to(root).as_posix()))
if set(by_canonical) != canonical_files:
for path in sorted(canonical_files - set(by_canonical)):
findings.append(_finding("CANONICAL_OWNER_MISSING", path.relative_to(root).as_posix()))
for path in sorted(set(by_canonical) - canonical_files):
findings.append(_finding("MIGRATION_CANONICAL_MISSING", path.relative_to(root).as_posix()))
for legacy, (canonical, expected_hash) in sorted(by_legacy.items()):
if not canonical.is_file():
continue
if mode == "shadow":
# cutover 검증 단계에서만 콘텐츠 해시를 대조한다. 이 단계의 목적은
# "이관이 내용을 그대로 옮겼는가"를 증명하는 것이다.
observed_hash = hashlib.sha256(canonical.read_bytes()).hexdigest()
if observed_hash != expected_hash:
findings.append(_finding("MIGRATION_HASH_MISMATCH", canonical.relative_to(root).as_posix(), f"expected {expected_hash}, observed {observed_hash}"))
# canonical 모드에서는 문서가 계속 편집되는 것이 정상이므로 해시를 고정하지 않는다.
# 이 모드에서 manifest 는 legacy→canonical 매핑과 rollback 근거로만 쓰인다
# (매핑 완전성은 위 MIGRATION_ENTRY_MISSING / CANONICAL_OWNER_MISSING 이 검사한다).
#
# shadow 불변식: legacy·canonical 은 서로 *독립된 실파일* 바이트 동일 미러다.
# 둘 중 하나라도 심링크면 is_file()/read_bytes() 가 상대를 따라가 항상 '동일'로
# 읽혀 drift 를 못 잡는다(symlink-blind). 그래서 심링크를 먼저 loud 하게 잡는다.
if legacy.is_symlink() or canonical.is_symlink():
findings.append(_finding(
"SHADOW_MIRROR_SYMLINK", legacy.relative_to(root).as_posix(),
"shadow 미러는 독립 실파일이어야 함 — 심링크는 byte-identical 검사를 무력화한다"))
elif not legacy.is_file():
findings.append(_finding("SHADOW_SOURCE_MISSING", legacy.relative_to(root).as_posix()))
elif legacy.read_bytes() != canonical.read_bytes():
findings.append(_finding("SHADOW_MIRROR_DRIFT", canonical.relative_to(root).as_posix()))
elif mode == "canonical" and legacy.is_symlink():
expected_target = canonical.resolve()
observed_target = legacy.resolve()
if observed_target != expected_target:
findings.append(
_finding(
"INVALID_COMPATIBILITY_SYMLINK",
legacy.relative_to(root).as_posix(),
f"expected target {canonical.relative_to(root).as_posix()}",
)
)
elif mode == "canonical" and legacy.is_file():
if legacy.read_bytes() == canonical.read_bytes():
findings.append(_finding("OLD_NEW_FULL_CONTENT_DUPLICATE", legacy.relative_to(root).as_posix()))
if legacy.suffix == ".md":
canonical_value = parse_frontmatter(legacy.read_text(encoding="utf-8")).get("canonical_path")
expected_path = canonical.relative_to(root).as_posix()
if canonical_value != expected_path:
findings.append(_finding("INVALID_COMPATIBILITY_STUB", legacy.relative_to(root).as_posix(), f"canonical_path must be {expected_path}"))
else:
findings.append(
_finding(
"NON_MARKDOWN_LEGACY_REMAINS",
legacy.relative_to(root).as_posix(),
"canonical mode requires a symlink redirect for non-Markdown legacy assets",
)
)
rollback_entries = rollback.get("entries")
rollback_pairs: dict[Path, Path] = {}
if not isinstance(rollback_entries, list):
findings.append(_finding("INVALID_CUTOVER_SCHEMA", "rollback_mapping.entries", "must be an array"))
else:
for index, item in enumerate(rollback_entries):
location = f"rollback_mapping.entries[{index}]"
if not isinstance(item, dict) or set(item) != {"canonical_path", "legacy_path"}:
findings.append(_finding("INVALID_ROLLBACK_ENTRY", location))
continue
canonical = _safe_repo_path(root, item.get("canonical_path"), findings, f"{location}.canonical_path")
legacy = _safe_repo_path(root, item.get("legacy_path"), findings, f"{location}.legacy_path")
if canonical is not None and legacy is not None:
if canonical in rollback_pairs:
findings.append(_finding("DUPLICATE_ROLLBACK_MAPPING", canonical.relative_to(root).as_posix()))
rollback_pairs[canonical] = legacy
for canonical, legacy in by_canonical.items():
if rollback_pairs.get(canonical) != legacy:
findings.append(_finding("ROLLBACK_MAPPING_MISSING", canonical.relative_to(root).as_posix()))
for canonical in rollback_pairs.keys() - by_canonical.keys():
findings.append(_finding("ROLLBACK_MAPPING_EXTRA", canonical.relative_to(root).as_posix()))
all_repository_files = {
path for path in root.rglob("*")
if path.is_file() and ".git" not in path.parts
}
external_files = all_repository_files - legacy_files - canonical_files
link_namespace = (legacy_files if mode == "shadow" else canonical_files) | external_files
_validate_links(root, canonical_files, findings, namespace=link_namespace)
return {"migration_entries": len(by_legacy), "canonical_files": len(canonical_files)}
def resolve_authority(
root: Path,
manifest_path: Path = DEFAULT_MANIFEST,
*,
require_clean: bool = True,
) -> dict[str, Any]:
"""Resolve the active write authority and fail closed on invalid layout state."""
root = root.resolve(strict=True)
manifest_file = manifest_path if manifest_path.is_absolute() else root / manifest_path
try:
result = check_layout(root, manifest_path)
manifest_bytes = manifest_file.read_bytes()
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise LayoutContractError("LAYOUT_IO_ERROR", str(exc), str(manifest_path)) from exc
if require_clean and result.get("status") != "PASS":
codes = ",".join(sorted({item.get("code", "UNKNOWN") for item in result.get("findings", [])}))
raise LayoutContractError("LAYOUT_NOT_READY", codes or "layout validation failed", str(manifest_path))
mode = result.get("mode")
write_roots = result.get("write_roots")
if mode not in MODES or not isinstance(write_roots, list) or not write_roots:
raise LayoutContractError("INVALID_WRITE_AUTHORITY", "active mode has no valid write roots", str(manifest_path))
return {
"mode": mode,
"authority": result.get("authority"),
"write_roots": list(write_roots),
"manifest_path": manifest_file.relative_to(root).as_posix(),
"manifest_sha256": hashlib.sha256(manifest_bytes).hexdigest(),
}
def enforce_write_paths(root: Path, paths: Iterable[Path], authority: Mapping[str, Any]) -> None:
"""Require every repository write to live under the active authoritative roots."""
root = _lexical_absolute(root)
allowed = [_lexical_absolute(root / value) for value in authority.get("write_roots", [])]
for path in paths:
resolved = _lexical_absolute(path)
try:
resolved.relative_to(root)
except ValueError as exc:
raise LayoutContractError("WRITE_OUTSIDE_REPOSITORY", str(path), str(path)) from exc
if not any(resolved == prefix or prefix in resolved.parents for prefix in allowed):
relative = resolved.relative_to(root).as_posix()
raise LayoutContractError(
"WRITE_ROOT_VIOLATION",
f"{relative} is outside active write roots {authority.get('write_roots', [])}",
relative,
)
def check_layout(root: Path, manifest_path: Path = DEFAULT_MANIFEST) -> dict[str, Any]:
root = root.resolve()
manifest_file = manifest_path if manifest_path.is_absolute() else root / manifest_path
manifest = json.loads(manifest_file.read_text(encoding="utf-8"))
findings: list[dict[str, str]] = []
if manifest.get("schema_version") != "vault-layout/v1":
findings.append(_finding("INVALID_LAYOUT_SCHEMA", str(manifest_path)))
mode = manifest.get("mode")
if mode not in MODES:
findings.append(_finding("INVALID_LAYOUT_MODE", str(manifest_path)))
vault_root = root / str(manifest.get("vault_root", "vault"))
assignments: dict[str, str] = {}
areas = manifest.get("areas")
if not isinstance(areas, dict):
findings.append(_finding("INVALID_LAYOUT_SCHEMA", "areas"))
areas = {}
for area, sources in areas.items():
target = vault_root / area
if not target.is_dir():
findings.append(_finding("MISSING_VAULT_AREA", target.relative_to(root).as_posix()))
if not isinstance(sources, list):
findings.append(_finding("INVALID_LAYOUT_SCHEMA", f"areas.{area}"))
continue
for raw_source in sources:
source = Path(str(raw_source)).as_posix().rstrip("/")
if source in assignments:
findings.append(_finding("DUPLICATE_LAYOUT_OWNER", source, f"{assignments[source]},{area}"))
assignments[source] = area
if not (root / source).is_dir():
findings.append(_finding("MISSING_COMPATIBILITY_SOURCE", source))
if source == "harness" or source.startswith("harness/"):
findings.append(_finding("HARNESS_INSIDE_VAULT", source))
actual = {
path.relative_to(root).as_posix()
for content_root in CONTENT_ROOTS
if (root / content_root).is_dir()
for path in (root / content_root).iterdir()
if path.is_dir()
}
for path in sorted(actual - set(assignments)):
findings.append(_finding("UNASSIGNED_CONTENT_ROOT", path))
for required in ("harness/source", "harness/adapters", "harness/runtime", "harness/tests"):
if not (root / required).is_dir():
findings.append(_finding("MISSING_HARNESS_AREA", required))
write_roots = manifest.get("write_roots")
if not isinstance(write_roots, dict) or set(write_roots) != MODES:
findings.append(_finding("INVALID_WRITE_ROOTS", "write_roots", "all three modes are required"))
else:
expected = [Path(str(manifest.get("vault_root", "vault"))).as_posix()] if mode == "canonical" else ["raw", "wiki"]
observed = write_roots.get(mode)
if observed != expected:
findings.append(_finding("INVALID_AUTHORITATIVE_WRITE_ROOT", f"write_roots.{mode}", f"expected {expected}, observed {observed}"))
cutover_stats = {"migration_entries": 0, "canonical_files": 0}
if mode in {"shadow", "canonical"}:
cutover_stats = _validate_cutover(root, mode, vault_root, assignments, manifest, findings)
findings.sort(key=lambda item: (item["path"], item["code"], item.get("detail", "")))
return {
"schema_version": "layout-check-result/v2",
"status": "PASS" if not findings else "FAIL",
"mode": mode or "",
"authority": "vault" if mode == "canonical" else "legacy",
"write_roots": write_roots.get(mode, []) if isinstance(write_roots, dict) and mode in MODES else [],
"assigned_sources": len(assignments),
"discovered_content_roots": len(actual),
**cutover_stats,
"findings": findings,
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--manifest", type=Path, default=DEFAULT_MANIFEST)
args = parser.parse_args(argv)
try:
result = check_layout(args.root.resolve(strict=True), args.manifest)
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
result = {
"schema_version": "layout-check-result/v2",
"status": "ERROR",
"findings": [_finding("LAYOUT_IO_ERROR", str(exc))],
}
exit_code = 2
else:
exit_code = 0 if result["status"] == "PASS" else 1
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+520
View File
@@ -0,0 +1,520 @@
#!/usr/bin/env python3
"""Bulk-migrate direct project Work Item branches to graph-contract v2.
The project decision/work-item registries are the only input authority. The
command deliberately skips branch children and already migrated notes; those
need an explicit parent/work-item decision instead of a guessed binding.
"""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import re
import sys
import tempfile
from typing import Any, Callable
from contract_markdown import cell, clean, parse_frontmatter, parse_tables, table_for
from fs_transaction import replace_many, stage_repository
import moc_indexer
import quality_gate
import template_renderer
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
PROJECT_DIR = Path("raw/project-notes")
BRANCH_DIR = Path("raw/branch-notes")
DEC_REF_RE = re.compile(r"\b(DEC-[A-Z0-9][A-Z0-9-]*-\d{3})@(\d+)\b")
DEC_ID_RE = re.compile(r"\bDEC-[A-Z0-9][A-Z0-9-]*-\d{3}\b")
WI_RE = re.compile(r"\bWI-[A-Z0-9][A-Z0-9-]*-\d{3}\b")
class MigrationError(ValueError):
pass
QualityRunner = Callable[..., dict[str, Any]]
def _refs(value: str) -> list[str]:
return [f"{match.group(1)}@{match.group(2)}" for match in DEC_REF_RE.finditer(value)]
def _wis(value: str) -> list[str]:
return WI_RE.findall(value)
def _plain(value: str) -> str:
# contract_markdown.clean() 과 같은 규칙 — 셀 전체가 단일 코드스팬/강조일 때만 벗긴다.
# strip("`* ") 로 양끝을 무조건 깎으면 코드스팬으로 시작만 하는 셀이 여는 백틱을 잃는다.
return clean(value)
def _yaml_list(values: list[str]) -> str:
return "[" + ", ".join(values) + "]"
def _replace_frontmatter(text: str, values: dict[str, str]) -> str:
lines = text.splitlines(keepends=True)
if not lines or lines[0].strip() != "---":
raise MigrationError("branch note has no YAML frontmatter")
try:
end = next(index for index, line in enumerate(lines[1:], 1) if line.strip() == "---")
except StopIteration as exc:
raise MigrationError("branch note has unterminated YAML frontmatter") from exc
pending = dict(values)
rendered: list[str] = [lines[0]]
for line in lines[1:end]:
match = re.match(r"^([A-Za-z_][\w-]*):", line)
key = match.group(1) if match else ""
if key in pending:
value = pending.pop(key)
rendered.append(f"{key}:{' ' + value if value else ''}\n")
else:
rendered.append(line)
for key, value in pending.items():
rendered.append(f"{key}:{' ' + value if value else ''}\n")
rendered.extend(lines[end:])
return "".join(rendered)
def _packet(project: str, item: dict[str, Any], summaries: dict[str, str]) -> str:
rows = []
for ref in item["decisions"]:
decision_id = ref.rsplit("@", 1)[0]
summary = summaries.get(decision_id, "")
rows.append(
f"| `{ref}` | {summary} | Work Item 완료 조건에 적용 | "
f"`[[raw/project-notes/{project}]]` |"
)
inherited_rows = "\n".join(rows)
if not inherited_rows:
inherited_rows = "| - | - | - | - |"
return f"""<!-- section-id: branch-contract-packet -->
## 브랜치 계약 패킷
- **생성 시 프로젝트 개정**: `{item['project_revision']}`
- **패킷 스키마**: `contract_packet: 1`
- **완료 조건**: {item['completion']}
<!-- section-id: inherited-project-decisions -->
### 상속한 프로젝트 결정
| Decision Ref | Project Summary | Branch Application | Source |
|---|---|---|---|
{inherited_rows}
<!-- section-id: branch-local-decisions -->
### 브랜치 지역 결정
> 기존 branch-local 결정은 아래 `## 결정-근거 매핑`의 D-row가 소유하며 이 packet에서 복제하지 않는다.
| Decision ID | Decision | Relation | Supporting Claims | Status |
|---|---|---|---|---|
<!-- section-id: declared-overrides -->
### 선언한 예외
| Override ID | Overrides | Reason | Approval | Status |
|---|---|---|---|---|
"""
def _insert_packet(text: str, packet: str) -> str:
if "## 브랜치 계약 패킷" in text or "## Branch Contract Packet" in text:
return text
goal = re.search(r"^##\s+.*(?:목표|WHY).*$", text, re.MULTILINE | re.IGNORECASE)
if not goal:
raise MigrationError("branch note has no goal heading for packet insertion")
return text[: goal.start()] + packet + text[goal.start() :]
def _upgrade_generated_packet(text: str, project_revision: int, completion: str) -> str:
"""Seal an existing v2 packet without rewriting any of its owned bytes."""
start_token = template_renderer.GENERATED_START
end_token = template_renderer.GENERATED_END
if start_token in text or end_token in text:
# Exact marker validation is delegated to the shared renderer helper.
template_renderer.generated_region(text)
wrapped = text
else:
packet = re.search(
r"^(?:<!--\s*section-id:\s*branch-contract-packet\s*-->\s*\n)?##\s+(?:브랜치 계약 패킷|Branch Contract Packet)\s*$",
text,
re.MULTILINE,
)
declared = re.search(
r"^<!--\s*section-id:\s*declared-overrides\s*-->\s*$",
text,
re.MULTILINE,
)
if packet is None or declared is None or declared.start() <= packet.start():
raise MigrationError("v2 branch packet has no ordered packet/override sections")
next_heading = re.search(r"^##\s+", text[declared.end() :], re.MULTILINE)
if next_heading is None:
raise MigrationError("v2 branch packet has no editable section after declared overrides")
end_at = declared.end() + next_heading.start()
prefix = text[: packet.start()] + start_token + "\n"
packet_bytes = text[packet.start() : end_at].rstrip("\n")
suffix = text[end_at:].lstrip("\n")
wrapped = prefix + packet_bytes + "\n" + end_token + "\n\n" + suffix
wrapped = re.sub(
r"(^-\s*\*\*생성 시 프로젝트 개정\*\*:\s*`?)[1-9]\d*(`?\s*$)",
rf"\g<1>{project_revision}\g<2>",
wrapped,
count=1,
flags=re.MULTILINE,
)
wrapped, completion_count = re.subn(
r"(^-\s*\*\*완료 조건\*\*:\s*).*$",
lambda match: match.group(1) + completion,
wrapped,
count=1,
flags=re.MULTILINE,
)
if completion_count != 1:
raise MigrationError("v2 branch packet has no completion criterion")
digest = template_renderer.generated_sha256(wrapped)
return _replace_frontmatter(wrapped, {"contract_packet_sha256": digest})
def _blocked(code: str, path: Path | str, message: str) -> dict[str, str]:
return {"code": code, "path": path.as_posix() if isinstance(path, Path) else path, "message": message}
def _pinned_refs(value: str) -> tuple[list[str], list[str]]:
refs: list[str] = []
invalid: list[str] = []
for raw in value.split(","):
item = _plain(raw)
if not item or item == "-":
continue
match = re.fullmatch(r"(DEC-[A-Z0-9][A-Z0-9-]*-\d{3})@([1-9]\d*)", item)
if match:
refs.append(f"{match.group(1)}@{match.group(2)}")
else:
invalid.append(item)
return refs, invalid
def _dependency_ids(value: str) -> tuple[list[str], list[str]]:
dependencies: list[str] = []
invalid: list[str] = []
for raw in value.split(","):
item = _plain(raw)
if not item or item == "-":
continue
if re.fullmatch(r"WI-[A-Z0-9][A-Z0-9-]*-\d{3}", item):
dependencies.append(item)
else:
invalid.append(item)
return dependencies, invalid
def _registries(root: Path) -> tuple[dict[str, dict[str, Any]], dict[str, str], list[dict[str, str]]]:
by_slug: dict[str, dict[str, Any]] = {}
summaries: dict[str, str] = {}
decision_revisions: dict[str, int] = {}
work_by_id: dict[str, dict[str, Any]] = {}
work_records: list[dict[str, Any]] = []
blocked: list[dict[str, str]] = []
for project_path in sorted((root / PROJECT_DIR).glob("*.md")):
text = project_path.read_text(encoding="utf-8")
fm = parse_frontmatter(text)
if "project_revision" not in fm:
continue
revision_raw = str(fm.get("project_revision", ""))
if not revision_raw.isdigit() or int(revision_raw) < 1:
blocked.append(_blocked("INVALID_PROJECT_REVISION", project_path.relative_to(root), "project_revision must be a positive integer"))
continue
project_revision = int(revision_raw)
project_prefix = project_path.stem.upper()
tables = parse_tables(text)
decisions = table_for(tables, "section-id:project-decisions", "안정 결정 레지스트리", "Project Decision Registry")
work_items = table_for(tables, "section-id:project-work-items", "실행계획", "Work Item Registry")
if decisions is None or work_items is None:
blocked.append(_blocked("INCOMPLETE_PROJECT_REGISTRY", project_path.relative_to(root), "decision/work-item registry is required"))
continue
for line, row in decisions.rows:
decision_id = next(iter(DEC_ID_RE.findall(cell(row, "Decision ID"))), "")
revision = _plain(cell(row, "Revision"))
location = f"{project_path.relative_to(root)}:{line}"
if not decision_id or not revision.isdigit() or int(revision) < 1:
blocked.append(_blocked("INVALID_PROJECT_DECISION", location, "decision id/revision is invalid"))
continue
if not decision_id.startswith(f"DEC-{project_prefix}-"):
blocked.append(_blocked("FOREIGN_PROJECT_PREFIX", location, f"{decision_id} does not belong to {project_path.stem}"))
if decision_id in decision_revisions:
blocked.append(_blocked("DUPLICATE_PROJECT_DECISION", location, f"duplicate decision id: {decision_id}"))
else:
decision_revisions[decision_id] = int(revision)
summaries[decision_id] = cell(row, "Decision Summary")
for line, row in work_items.rows:
wi = next(iter(_wis(cell(row, "Work Item ID"))), "")
slug = _plain(cell(row, "branch slug"))
location = f"{project_path.relative_to(root)}:{line}"
if not wi or not slug:
blocked.append(_blocked("INVALID_WORK_ITEM", location, "Work Item ID and branch slug are required"))
continue
if not wi.startswith(f"WI-{project_prefix}-"):
blocked.append(_blocked("FOREIGN_PROJECT_PREFIX", location, f"{wi} does not belong to {project_path.stem}"))
if wi in work_by_id:
blocked.append(_blocked("DUPLICATE_WORK_ITEM", location, f"duplicate Work Item id: {wi}"))
if slug in by_slug:
blocked.append(_blocked("DUPLICATE_BRANCH_SLUG", location, f"duplicate Work Item branch slug: {slug}"))
decisions_refs, invalid_decisions = _pinned_refs(cell(row, "Applies Decisions"))
dependencies, invalid_dependencies = _dependency_ids(cell(row, "Dependencies"))
if invalid_decisions:
blocked.append(_blocked("INVALID_APPLIED_DECISION", location, f"invalid references: {invalid_decisions}"))
if not decisions_refs:
blocked.append(_blocked("MISSING_APPLIED_DECISION", location, f"{wi} has no applied decision"))
if invalid_dependencies:
blocked.append(_blocked("INVALID_DEPENDENCY", location, f"invalid dependencies: {invalid_dependencies}"))
record = {
"project": project_path.stem,
"project_revision": project_revision,
"work_item": wi,
"completion": cell(row, "완료 조건 (측정가능)"),
"decisions": decisions_refs,
"dependencies": dependencies,
"location": location,
}
by_slug.setdefault(slug, record)
work_by_id.setdefault(wi, record)
work_records.append(record)
for item in work_records:
wi = item["work_item"]
expected_prefix = f"DEC-{item['project'].upper()}-"
for ref in item["decisions"]:
decision_id, revision_raw = ref.rsplit("@", 1)
if not decision_id.startswith(expected_prefix):
blocked.append(_blocked("FOREIGN_PROJECT_PREFIX", item["location"], f"{decision_id} does not belong to {item['project']}"))
current = decision_revisions.get(decision_id)
if current is None:
blocked.append(_blocked("MISSING_APPLIED_DECISION", item["location"], f"{decision_id} is not declared"))
elif int(revision_raw) != current:
blocked.append(_blocked("STALE_REVISION", item["location"], f"{ref} != current {decision_id}@{current}"))
for dependency in item["dependencies"]:
if dependency == wi:
blocked.append(_blocked("SELF_DEPENDENCY", item["location"], f"{wi} depends on itself"))
elif dependency not in work_by_id:
blocked.append(_blocked("MISSING_DEPENDENCY", item["location"], f"dependency does not exist: {dependency}"))
visiting: set[str] = set()
visited: set[str] = set()
def visit(wi: str, trail: list[str]) -> None:
if wi in visited:
return
if wi in visiting:
cycle = trail[trail.index(wi):] + [wi]
blocked.append(_blocked("DEPENDENCY_CYCLE", work_by_id[wi]["location"], " -> ".join(cycle)))
return
visiting.add(wi)
for dependency in work_by_id[wi]["dependencies"]:
if dependency in work_by_id:
visit(dependency, trail + [dependency])
visiting.remove(wi)
visited.add(wi)
for wi in sorted(work_by_id):
visit(wi, [wi])
unique = {(item["code"], item["path"], item["message"]): item for item in blocked}
return by_slug, summaries, [unique[key] for key in sorted(unique)]
def build_updates(root: Path) -> tuple[dict[Path, str], dict[str, Any]]:
registry, summaries, blocked = _registries(root)
updates: dict[Path, str] = {}
eligible: list[str] = []
already_v2: list[str] = []
unmapped: list[str] = []
for path in sorted((root / BRANCH_DIR).glob("*.md")):
text = path.read_text(encoding="utf-8")
fm = parse_frontmatter(text)
if "contract_packet" in fm or "## 브랜치 계약 패킷" in text or "## Branch Contract Packet" in text:
already_v2.append(path.stem)
item = registry.get(path.stem)
if item is None:
work_item = str(fm.get("work_item", "")).strip()
matches = [candidate for candidate in registry.values() if candidate["work_item"] == work_item]
item = matches[0] if len(matches) == 1 else None
if item is None:
blocked.append(_blocked("UNMAPPED_V2_BRANCH", path.relative_to(root), "cannot resolve v2 branch to one Work Item"))
continue
try:
migrated = _upgrade_generated_packet(text, item["project_revision"], item["completion"])
except (MigrationError, template_renderer.TemplateRenderError) as exc:
blocked.append(_blocked("INVALID_V2_PACKET", path.relative_to(root), str(exc)))
continue
if migrated != text:
updates[path] = migrated
continue
item = registry.get(path.stem)
if item is None:
unmapped.append(path.stem)
continue
eligible.append(path.stem)
values = {
"id": item["work_item"].replace("WI-", "BR-", 1),
"kind": "project-work-item",
"project": item["project"],
"work_item": item["work_item"],
"inherits": _yaml_list(item["decisions"]),
"refines": "[]",
"overrides": "[]",
"depends_on": _yaml_list(item["dependencies"]),
"contract_packet": "1",
"parent_branch": "",
"branch": path.stem,
}
migrated = _replace_frontmatter(text, values)
migrated = _insert_packet(migrated, _packet(item["project"], item, summaries))
migrated = _upgrade_generated_packet(migrated, item["project_revision"], item["completion"])
if migrated != text:
updates[path] = migrated
if blocked:
updates = {}
return updates, {
"eligible": eligible,
"already_v2": already_v2,
"unmapped": unmapped,
"blocked": blocked,
"changed": [path.relative_to(root).as_posix() for path in sorted(updates)],
}
def _stage_repository(root: Path, destination: Path) -> None:
stage_repository(root, destination)
def prepare_updates(
root: Path,
*,
quality_runner: QualityRunner = quality_gate.run,
relations_path: Path = moc_indexer.DEFAULT_RELATIONS,
) -> tuple[dict[Path, str], dict[str, Any]]:
"""Stage the full migration scope and return bytes only after all gates pass."""
root = root.resolve(strict=True)
branch_updates, stats = build_updates(root)
stats = dict(stats)
stats["migration_changed"] = list(stats["changed"])
stats["moc_changed"] = []
stats["quality"] = None
if stats["blocked"] or not branch_updates:
return {}, stats
with tempfile.TemporaryDirectory(prefix="graph-contract-migration-stage-") as directory:
stage = Path(directory) / "repo"
stage.mkdir()
_stage_repository(root, stage)
staged_branches: list[Path] = []
for path, text in branch_updates.items():
staged = stage / path.relative_to(root)
staged.write_text(text, encoding="utf-8")
staged_branches.append(staged)
moc_updates, _moc_stats = moc_indexer.build_updates(stage, relations_path)
for path, text in moc_updates.items():
path.write_text(text, encoding="utf-8")
remaining_moc, _ = moc_indexer.build_updates(stage, relations_path)
if remaining_moc:
stats["blocked"] = [
_blocked("MIGRATION_MOC_NOT_IDEMPOTENT", path.relative_to(stage), "relation projection did not converge")
for path in sorted(remaining_moc)
]
return {}, stats
touched_rel = sorted(
{
*(path.relative_to(stage).as_posix() for path in staged_branches),
*(path.relative_to(stage).as_posix() for path in moc_updates),
}
)
try:
gate = quality_runner(
stage,
touched_rel,
structure_paths=[path.relative_to(stage).as_posix() for path in staged_branches],
template_root=DEFAULT_ROOT,
include_graph=True,
require_moc_convergence=True,
)
except quality_gate.QualityGateError as exc:
raise MigrationError(f"migration quality gate error: {exc}") from exc
if not isinstance(gate, dict) or gate.get("schema_version") != "quality-gate-result/v1":
raise MigrationError("migration quality gate returned an unexpected schema")
stats["quality"] = {
"status": gate.get("status"),
"checks": gate.get("checks", []),
"touched_paths": gate.get("touched_paths", gate.get("checked_paths", [])),
}
if gate.get("status") != "PASS":
codes = sorted({str(item.get("code", "UNKNOWN")) for item in gate.get("findings", [])})
stats["blocked"] = [
_blocked("MIGRATION_QUALITY_GATE_FAILED", "<staged-scope>", ",".join(codes) or "quality gate failed")
]
return {}, stats
repeated_branches, repeated_stats = build_updates(stage)
if repeated_branches or repeated_stats["blocked"]:
stats["blocked"] = [
_blocked("MIGRATION_NOT_IDEMPOTENT", "<staged-scope>", "second migration pass was not current")
]
return {}, stats
all_relative = set(touched_rel)
prepared = {root / relative: (stage / relative).read_text(encoding="utf-8") for relative in all_relative}
stats["moc_changed"] = sorted(path.relative_to(stage).as_posix() for path in moc_updates)
stats["changed"] = sorted(all_relative)
return prepared, stats
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--check", action="store_true")
mode.add_argument("--write", action="store_true")
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
updates, stats = prepare_updates(root)
if args.write and updates and not stats["blocked"]:
replace_many({path: text.encode("utf-8") for path, text in updates.items()})
status = (
"BLOCKED"
if stats["blocked"]
else "DRIFT"
if args.check and updates
else "UPDATED"
if updates
else "CURRENT"
)
json.dump(
{"schema_version": "graph-contract-migration/v1", "status": status, **stats},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 1 if stats["blocked"] or (args.check and updates) else 0
except (MigrationError, OSError, UnicodeError) as exc:
json.dump(
{"schema_version": "graph-contract-migration/v1", "status": "ERROR", "errors": [{"code": "MIGRATION_ERROR", "message": str(exc)}]},
sys.stdout,
ensure_ascii=False,
indent=2,
)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+451
View File
@@ -0,0 +1,451 @@
#!/usr/bin/env python3
"""Build generated reverse MOC views from canonical child frontmatter edges."""
from __future__ import annotations
import argparse
from collections import defaultdict
import json
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
from contract_markdown import as_list, parse_frontmatter
from fs_transaction import replace_many
import vault_migrate
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_RELATIONS = Path(__file__).resolve().parents[1] / "source/document-relations.json"
DEFAULT_LAYOUT = Path("harness/source/vault-layout.json")
RELATION_SCHEMA = "document-relations/v1"
GEN_START = "<!-- GENERATED: branches:start -->"
GEN_END = "<!-- GENERATED: branches:end -->"
SAFE_NAME = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$")
SAFE_RELATIVE_ROOT = re.compile(r"^(?:[a-zA-Z0-9._-]+/)*[a-zA-Z0-9._-]+$")
CLUSTER_HEADING = re.compile(r"^##\s+.*(?:Cluster|묶음).*$", re.MULTILINE | re.IGNORECASE)
FIELD_LINE = re.compile(r"^([A-Za-z_][\w-]*):\s*(.*)$")
class MocError(ValueError):
def __init__(self, code: str, message: str, path: str = "") -> None:
self.code = code
self.path = path
super().__init__(message)
def _authority_mapping(root: Path) -> dict[str, Any]:
manifest_path = root / DEFAULT_LAYOUT
if not manifest_path.is_file():
return {"mode": "compatibility", "legacy_to_canonical": {}}
try:
layout = json.loads(manifest_path.read_text(encoding="utf-8"))
mode = layout.get("mode")
migration = layout.get("migration_manifest")
if isinstance(migration, str):
migration_path = (root / migration).resolve()
migration_path.relative_to(root)
migration = json.loads(migration_path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError, ValueError) as exc:
raise MocError("LAYOUT_SOURCE_ERROR", str(exc), DEFAULT_LAYOUT.as_posix()) from exc
if mode not in {"compatibility", "shadow", "canonical"}:
raise MocError("INVALID_LAYOUT_MODE", str(mode), DEFAULT_LAYOUT.as_posix())
if not isinstance(migration, dict) or migration.get("schema_version") != "vault-migration/v1":
raise MocError("INVALID_MIGRATION_SCHEMA", "expected vault-migration/v1", DEFAULT_LAYOUT.as_posix())
entries = migration.get("entries")
if not isinstance(entries, list):
raise MocError("INVALID_MIGRATION_SCHEMA", "entries must be an array", DEFAULT_LAYOUT.as_posix())
mapping: dict[str, str] = {}
reverse: set[str] = set()
for index, item in enumerate(entries):
if not isinstance(item, dict):
raise MocError("INVALID_MIGRATION_ENTRY", str(index), DEFAULT_LAYOUT.as_posix())
legacy, canonical = item.get("legacy_path"), item.get("canonical_path")
if not isinstance(legacy, str) or not isinstance(canonical, str):
raise MocError("INVALID_MIGRATION_ENTRY", str(index), DEFAULT_LAYOUT.as_posix())
for value in (legacy, canonical):
if Path(value).is_absolute() or ".." in Path(value).parts or "\\" in value:
raise MocError("INVALID_MIGRATION_PATH", value, DEFAULT_LAYOUT.as_posix())
if legacy in mapping or canonical in reverse:
raise MocError("DUPLICATE_MIGRATION_OWNER", f"entry {index}", DEFAULT_LAYOUT.as_posix())
mapping[legacy] = canonical
reverse.add(canonical)
return {"mode": mode, "legacy_to_canonical": mapping}
def _authority_paths(root: Path, legacy_root: str, authority: Mapping[str, Any]) -> list[Path]:
if authority["mode"] == "canonical":
return sorted(
root / canonical
for legacy, canonical in authority["legacy_to_canonical"].items()
if Path(legacy).is_relative_to(Path(legacy_root)) and legacy.endswith(".md")
)
base = root / legacy_root
return sorted(base.rglob("*.md")) if base.is_dir() else []
def _relative(path: Path, root: Path) -> str:
return path.resolve().relative_to(root.resolve()).as_posix()
def _logical_relative(path: Path, root: Path, authority: Mapping[str, Any]) -> str:
"""Render stable legacy wikilinks for manifest-owned canonical files."""
relative = _relative(path, root)
if authority["mode"] != "canonical":
return relative
matches = [
legacy
for legacy, canonical in authority["legacy_to_canonical"].items()
if canonical == relative
]
if len(matches) != 1:
raise MocError("AMBIGUOUS_LOGICAL_PATH", f"expected one legacy owner: {relative}", relative)
return matches[0]
def _safe_root(value: Any, location: str) -> Path:
if not isinstance(value, str) or not SAFE_RELATIVE_ROOT.fullmatch(value) or ".." in Path(value).parts:
raise MocError("INVALID_RELATION_ROOT", f"invalid repo-relative root: {value!r}", location)
return Path(value)
def _marker_pair(marker: str) -> tuple[str, str]:
return f"<!-- GENERATED: {marker}:start -->", f"<!-- GENERATED: {marker}:end -->"
def _frontmatter_values(text: str, field: str) -> list[str]:
"""Return scalar/list frontmatter values, including bracket lists split over lines."""
parsed = as_list(parse_frontmatter(text).get(field))
if parsed and parsed != ["["]:
return parsed
lines = text.splitlines()
if not lines or lines[0].strip() != "---":
return []
for index, line in enumerate(lines[1:], 1):
if line.strip() == "---":
break
match = FIELD_LINE.match(line)
if not match or match.group(1) != field:
continue
raw = match.group(2).strip()
if raw == "[":
values: list[str] = []
for continuation in lines[index + 1 :]:
token = continuation.strip()
if token == "]":
return values
token = token.lstrip("- ").rstrip(",").strip().strip("'\"")
if token:
values.append(token)
return []
return parsed
return []
def _load_relations(path: Path) -> list[dict[str, Any]]:
try:
document = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise MocError("RELATION_SOURCE_ERROR", str(exc), str(path)) from exc
if not isinstance(document, dict) or document.get("schema_version") != RELATION_SCHEMA:
raise MocError("INVALID_RELATION_SCHEMA", f"expected {RELATION_SCHEMA}", str(path))
raw_relations = document.get("relations")
if not isinstance(raw_relations, list) or not raw_relations:
raise MocError("INVALID_RELATION_SCHEMA", "relations must be a non-empty array", str(path))
result: list[dict[str, Any]] = []
seen: set[str] = set()
allowed = {
"id", "child_roots", "parent_field", "parent_roots", "marker", "marker_aliases",
"parent_aliases", "reference_kind", "require_fields", "when",
}
for index, value in enumerate(raw_relations):
location = f"{path}:relations[{index}]"
if not isinstance(value, dict) or set(value) - allowed:
raise MocError("INVALID_RELATION", "relation is not an object or has unknown fields", location)
relation_id = value.get("id")
marker = value.get("marker")
parent_field = value.get("parent_field")
if not isinstance(relation_id, str) or not SAFE_NAME.fullmatch(relation_id) or relation_id in seen:
raise MocError("INVALID_RELATION_ID", f"invalid or duplicate relation id: {relation_id!r}", location)
if not isinstance(marker, str) or not SAFE_NAME.fullmatch(marker):
raise MocError("INVALID_RELATION_MARKER", f"invalid marker: {marker!r}", location)
if not isinstance(parent_field, str) or not re.fullmatch(r"[A-Za-z_][\w-]*", parent_field):
raise MocError("INVALID_PARENT_FIELD", f"invalid parent field: {parent_field!r}", location)
if value.get("reference_kind", "slug") not in {"slug", "path"}:
raise MocError("INVALID_REFERENCE_KIND", "reference_kind must be slug or path", location)
for key in ("child_roots", "parent_roots"):
roots = value.get(key)
if not isinstance(roots, list) or not roots:
raise MocError("INVALID_RELATION_ROOT", f"{key} must be non-empty", location)
value[key] = [_safe_root(item, location).as_posix() for item in roots]
aliases = value.get("marker_aliases", [])
parent_aliases = value.get("parent_aliases", {})
requires = value.get("require_fields", [])
if not isinstance(aliases, list) or any(not isinstance(item, str) or not SAFE_NAME.fullmatch(item) for item in aliases):
raise MocError("INVALID_RELATION_MARKER", "marker_aliases contains an invalid marker", location)
if not isinstance(requires, list) or any(not isinstance(item, str) for item in requires):
raise MocError("INVALID_RELATION", "require_fields must be a string array", location)
if (
not isinstance(parent_aliases, dict)
or any(
not isinstance(key, str)
or not SAFE_NAME.fullmatch(key)
or not isinstance(target, str)
or not SAFE_NAME.fullmatch(target)
for key, target in parent_aliases.items()
)
):
raise MocError("INVALID_PARENT_ALIAS", "parent_aliases must map safe names to safe names", location)
when = value.get("when")
if when is not None:
if (
not isinstance(when, dict)
or set(when) != {"field", "operator"}
or when.get("operator") not in {"empty", "nonempty"}
or not isinstance(when.get("field"), str)
):
raise MocError("INVALID_RELATION_CONDITION", "when must contain field and empty/nonempty operator", location)
seen.add(relation_id)
result.append(dict(value))
return result
def _condition_matches(text: str, relation: Mapping[str, Any]) -> bool:
condition = relation.get("when")
if not condition:
return True
values = _frontmatter_values(text, str(condition["field"]))
return bool(values) if condition["operator"] == "nonempty" else not values
def _generated_block(marker: str, children: Iterable[str]) -> str:
start, end = _marker_pair(marker)
return "\n".join([start, *(f"- [[{child}]]" for child in sorted(set(children))), end])
def _replace_or_insert_marker(
text: str,
children: set[str],
marker: str,
rel: str,
aliases: Iterable[str] = (),
) -> str:
candidates = [marker, *aliases]
located: list[tuple[str, int, int]] = []
for candidate in candidates:
start_token, end_token = _marker_pair(candidate)
starts = [match.start() for match in re.finditer(re.escape(start_token), text)]
ends = [match.start() for match in re.finditer(re.escape(end_token), text)]
if starts or ends:
if len(starts) != 1 or len(ends) != 1 or starts[0] >= ends[0]:
raise MocError("INVALID_GENERATED_BLOCK", f"{candidate} markers must form one ordered pair", rel)
located.append((candidate, starts[0], ends[0] + len(end_token)))
if len(located) > 1:
raise MocError("DUPLICATE_GENERATED_VIEW", f"multiple marker variants exist for {marker}", rel)
block = _generated_block(marker, children)
if located:
_candidate, start, end = located[0]
return text[:start] + block + text[end:]
heading = CLUSTER_HEADING.search(text)
if heading:
insert_at = text.find("\n", heading.end())
if insert_at < 0:
return text + "\n\n" + block + "\n"
return text[: insert_at + 1] + "\n" + block + "\n" + text[insert_at + 1 :]
suffix = "" if text.endswith("\n") else "\n"
return text + suffix + "\n## Cluster / 묶음\n\n" + block + "\n"
def _replace_or_insert(text: str, children: set[str], rel: str) -> str:
"""Backward-compatible branches-view helper used by older callers/tests."""
return _replace_or_insert_marker(text, children, "branches", rel, ("children",))
def _resolve_parent(
root: Path,
roots: Iterable[str],
reference: str,
child_rel: str,
aliases: Mapping[str, str] | None = None,
authority: Mapping[str, Any] | None = None,
reference_kind: str = "slug",
) -> Path:
reference = (aliases or {}).get(reference, reference)
active = authority or {"mode": "compatibility", "legacy_to_canonical": {}}
if reference_kind == "slug":
if not SAFE_NAME.fullmatch(reference):
raise MocError("INVALID_PARENT_REFERENCE", f"invalid parent reference: {reference!r}", child_rel)
candidates = [
candidate
for parent_root in roots
for candidate in _authority_paths(root, parent_root, active)
if candidate.stem == reference
]
else:
candidate_path = Path(reference)
if candidate_path.is_absolute() or ".." in candidate_path.parts or "\\" in reference:
raise MocError("INVALID_PARENT_REFERENCE", f"invalid parent path: {reference!r}", child_rel)
legacy = candidate_path.with_suffix(".md") if not candidate_path.suffix else candidate_path
if not any(legacy.is_relative_to(Path(parent_root)) for parent_root in roots):
raise MocError("INVALID_PARENT_REFERENCE", f"parent path is outside configured roots: {reference!r}", child_rel)
resolved_relative = (
Path(active["legacy_to_canonical"].get(legacy.as_posix(), legacy.as_posix()))
if active["mode"] == "canonical"
else legacy
)
candidates = [root / resolved_relative]
matches = [candidate for candidate in candidates if candidate.is_file()]
if len(matches) != 1:
code = "MISSING_PARENT_HUB" if not matches else "AMBIGUOUS_PARENT_HUB"
rendered = ", ".join(_relative(item, root) for item in candidates)
raise MocError(code, f"expected one canonical parent for {reference}: {rendered}", child_rel)
return matches[0]
def build_updates(
root: Path,
relations_path: Path | None = None,
) -> tuple[dict[Path, str], dict[str, Any]]:
root = root.resolve()
source = relations_path or DEFAULT_RELATIONS
if not source.is_absolute():
source = root / source
relations = _load_relations(source.resolve(strict=True))
authority = _authority_mapping(root)
expected: dict[tuple[Path, str], set[str]] = defaultdict(set)
marker_aliases: dict[tuple[Path, str], set[str]] = defaultdict(set)
relation_edges: dict[str, int] = defaultdict(int)
skipped = 0
for relation in relations:
for child_root_value in relation["child_roots"]:
for child in _authority_paths(root, child_root_value, authority):
if not child.is_file():
raise MocError(
"MISSING_AUTHORITATIVE_DOCUMENT",
"manifest owner is missing",
child.relative_to(root).as_posix(),
)
text = child.read_text(encoding="utf-8")
if any(not _frontmatter_values(text, field) for field in relation.get("require_fields", [])):
skipped += 1
continue
if not _condition_matches(text, relation):
continue
references = _frontmatter_values(text, relation["parent_field"])
if not references:
continue
child_link = Path(_logical_relative(child, root, authority)).with_suffix("").as_posix()
resolved_parents = {
_resolve_parent(
root,
relation["parent_roots"],
reference,
child_link,
relation.get("parent_aliases"),
authority,
relation.get("reference_kind", "slug"),
)
for reference in references
}
for parent in resolved_parents:
key = (parent, relation["marker"])
expected[key].add(child_link)
marker_aliases[key].update(relation.get("marker_aliases", []))
relation_edges[relation["id"]] += 1
# Existing generated views remain managed even when their last child disappears.
marker_names = [relation["marker"], *relation.get("marker_aliases", [])]
for parent_root_value in relation["parent_roots"]:
for hub in _authority_paths(root, parent_root_value, authority):
if not hub.is_file():
raise MocError(
"MISSING_AUTHORITATIVE_DOCUMENT",
"manifest owner is missing",
hub.relative_to(root).as_posix(),
)
hub_text = hub.read_text(encoding="utf-8")
if any(any(token in hub_text for token in _marker_pair(name)) for name in marker_names):
key = (hub, relation["marker"])
expected.setdefault(key, set())
marker_aliases[key].update(relation.get("marker_aliases", []))
by_hub: dict[Path, list[tuple[str, set[str], set[str]]]] = defaultdict(list)
for (hub, marker), children in expected.items():
by_hub[hub].append((marker, children, marker_aliases[(hub, marker)]))
updates: dict[Path, str] = {}
indexed: list[str] = []
view_count = 0
for hub in sorted(by_hub, key=lambda path: _relative(path, root)):
original = hub.read_text(encoding="utf-8")
rendered = original
rel = _relative(hub, root)
for marker, children, aliases in sorted(by_hub[hub], key=lambda item: item[0]):
rendered = _replace_or_insert_marker(rendered, children, marker, rel, sorted(aliases))
view_count += 1
indexed.append(rel)
if rendered != original:
updates[hub] = rendered
return updates, {
"mode": authority["mode"],
"namespace": "canonical" if authority["mode"] == "canonical" else "legacy",
"canonical_edges": sum(relation_edges.values()),
"structured_children": relation_edges.get("branch-to-branch", 0) + relation_edges.get("branch-to-project", 0),
"skipped_legacy": skipped,
"relation_edges": dict(sorted(relation_edges.items())),
"indexed_views": view_count,
"indexed_hubs": indexed,
"changed_hubs": [_relative(path, root) for path in sorted(updates)],
}
def _emit(document: dict[str, Any]) -> None:
json.dump(document, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
def _parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--relations", type=Path, default=DEFAULT_RELATIONS)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--check", action="store_true", help="report drift without writing")
mode.add_argument("--apply", action="store_true", help="atomically replace changed hubs")
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = _parse_args(argv)
try:
root = args.root.resolve(strict=True)
updates, stats = build_updates(root, args.relations)
if args.apply and updates:
encoded = {path: text.encode("utf-8") for path, text in updates.items()}
expanded, _authority = vault_migrate.expand_authoritative_changes(
root,
encoded,
relations_path=args.relations,
)
replace_many(expanded)
status = "DRIFT" if args.check and updates else "UPDATED" if updates else "CURRENT"
_emit({"schema_version": "moc-indexer-result/v2", "status": status, **stats})
return 1 if args.check and updates else 0
except (MocError, vault_migrate.MigrationError, OSError, UnicodeError) as exc:
_emit({
"schema_version": "moc-indexer-result/v2",
"status": "FAIL",
"error": {
"code": getattr(exc, "code", "IO_ERROR"),
"path": getattr(exc, "path", ""),
"message": str(exc),
},
})
return 2
if __name__ == "__main__":
raise SystemExit(main())
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env python3
"""Validate a persisted proof manifest reference as a standalone hard gate."""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
import re
import sys
from typing import Any
import proof_manifest
SCHEMA = "proof-hard-gate-result/v1"
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
class ProofGateFailure(ValueError):
def __init__(self, code: str, message: str) -> None:
self.code = code
super().__init__(message)
def _allowed_path(path: Path, repo_root: Path, run_root: Path | None) -> Path:
resolved = path.resolve(strict=True)
allowed = [repo_root, *( [run_root] if run_root is not None else [] )]
if not any(resolved.is_relative_to(root) for root in allowed):
raise ProofGateFailure("MANIFEST_OUTSIDE_ALLOWED_ROOT", str(resolved))
if not resolved.is_file():
raise ProofGateFailure("MANIFEST_NOT_FILE", str(resolved))
return resolved
def validate_reference(
manifest_path: Path,
*,
expected_sha256: str,
expected_proof_count: int,
expected_pass_count: int,
expected_fail_count: int,
repo_root: Path,
allowed_profiles: set[str],
run_root: Path | None = None,
) -> dict[str, Any]:
repo_root = repo_root.resolve(strict=True)
run_root = run_root.resolve(strict=True) if run_root is not None else None
path = _allowed_path(manifest_path, repo_root, run_root)
if not HEX_SHA256.fullmatch(expected_sha256):
raise ProofGateFailure("INVALID_EXPECTED_MANIFEST_SHA256", expected_sha256)
if any(isinstance(value, bool) or not isinstance(value, int) or value < 0 for value in (expected_proof_count, expected_pass_count, expected_fail_count)):
raise ProofGateFailure("INVALID_EXPECTED_COUNT", "proof/pass/fail counts must be non-negative integers")
content = path.read_bytes()
observed_sha256 = hashlib.sha256(content).hexdigest()
if observed_sha256 != expected_sha256:
raise ProofGateFailure("MANIFEST_HASH_MISMATCH", f"expected {expected_sha256}, observed {observed_sha256}")
manifest = json.loads(content.decode("utf-8"))
if not isinstance(manifest, dict) or manifest.get("schema_version") != proof_manifest.SCHEMA_VERSION:
raise ProofGateFailure("MANIFEST_SCHEMA_MISMATCH", f"expected {proof_manifest.SCHEMA_VERSION}")
verified = proof_manifest.verify_manifest(manifest, repo_root, allowed_profiles, run_root=run_root)
verification = verified["verification"]
observed = (
verification["proof_count"],
verification["pass_count"],
verification["fail_count"],
)
expected = (expected_proof_count, expected_pass_count, expected_fail_count)
if observed != expected:
raise ProofGateFailure("MANIFEST_COUNT_MISMATCH", f"expected {expected}, observed {observed}")
if verification.get("status") != "PASS" or expected_fail_count != 0 or expected_pass_count != expected_proof_count:
raise ProofGateFailure("PROOF_GATE_NOT_PASS", f"verification={verification}")
return {
"schema_version": SCHEMA,
"status": "PASS",
"manifest": {"path": path.as_posix(), "sha256": observed_sha256, "schema_version": manifest["schema_version"]},
"proof_count": observed[0],
"pass_count": observed[1],
"fail_count": observed[2],
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("manifest", type=Path)
parser.add_argument("--manifest-sha256", required=True)
parser.add_argument("--proof-count", required=True, type=int)
parser.add_argument("--pass-count", required=True, type=int)
parser.add_argument("--fail-count", required=True, type=int)
parser.add_argument("--repo-root", type=Path, default=proof_manifest.DEFAULT_REPO_ROOT)
parser.add_argument("--run-root", type=Path)
parser.add_argument("--profiles", type=Path, default=proof_manifest.DEFAULT_PROFILES)
args = parser.parse_args(argv)
try:
repo_root = args.repo_root.resolve(strict=True)
run_root = args.run_root.resolve(strict=True) if args.run_root is not None else None
profiles = proof_manifest.load_allowed_profiles(args.profiles.resolve(strict=True))
result = validate_reference(
args.manifest,
expected_sha256=args.manifest_sha256,
expected_proof_count=args.proof_count,
expected_pass_count=args.pass_count,
expected_fail_count=args.fail_count,
repo_root=repo_root,
run_root=run_root,
allowed_profiles=profiles,
)
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 0
except (ProofGateFailure, proof_manifest.ManifestValidationError) as exc:
errors = exc.issues if isinstance(exc, proof_manifest.ManifestValidationError) else [{"code": exc.code, "message": str(exc)}]
json.dump({"schema_version": SCHEMA, "status": "FAIL", "errors": errors}, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 1
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
json.dump({"schema_version": SCHEMA, "status": "ERROR", "errors": [{"code": "PROOF_GATE_ERROR", "message": str(exc)}]}, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+420
View File
@@ -0,0 +1,420 @@
#!/usr/bin/env python3
"""Fail-closed verifier for machine-readable quote proof manifests.
The verifier never executes the recorded argv. It validates a captured execution
record against repository source bytes. Disk output is opt-in via ``--output``.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import sys
import tempfile
from typing import Any, Iterable, Mapping
SCHEMA_VERSION = "proof-manifest/v1"
RESULT_SCHEMA_VERSION = "proof-manifest-result/v1"
DEFAULT_REPO_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_PROFILES = DEFAULT_REPO_ROOT / "harness/source/execution-profiles.json"
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
SAFE_IDENTIFIER = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
SAFE_ROLE = re.compile(r"^[a-z][a-z0-9_-]*$")
class ManifestValidationError(Exception):
"""Raised when one or more fail-closed proof checks fail."""
def __init__(self, issues: Iterable[Mapping[str, str]]) -> None:
self.issues = [dict(issue) for issue in issues]
super().__init__(f"proof manifest validation failed ({len(self.issues)} issue(s))")
def _issue(issues: list[dict[str, str]], code: str, location: str, message: str) -> None:
issues.append({"code": code, "location": location, "message": message})
def _is_int(value: Any) -> bool:
return isinstance(value, int) and not isinstance(value, bool)
def _require_object(
value: Any,
location: str,
required: set[str],
optional: set[str],
issues: list[dict[str, str]],
) -> Mapping[str, Any] | None:
if not isinstance(value, dict):
_issue(issues, "INVALID_TYPE", location, "must be a JSON object")
return None
keys = set(value)
for missing in sorted(required - keys):
_issue(issues, "MISSING_FIELD", f"{location}.{missing}", "required field is missing")
for unknown in sorted(keys - required - optional):
_issue(issues, "UNKNOWN_FIELD", f"{location}.{unknown}", "unknown field is not allowed")
return value
def _load_json(path: Path) -> Any:
with path.open("r", encoding="utf-8") as stream:
return json.load(stream)
def load_allowed_profiles(path: Path) -> set[str]:
try:
document = _load_json(path)
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise ManifestValidationError(
[{"code": "PROFILE_SOURCE_ERROR", "location": str(path), "message": str(exc)}]
) from exc
if not isinstance(document, dict) or document.get("schema_version") != "execution-profiles/v1":
raise ManifestValidationError(
[
{
"code": "PROFILE_SOURCE_SCHEMA",
"location": str(path),
"message": "expected execution-profiles/v1",
}
]
)
profiles = document.get("profiles")
if not isinstance(profiles, dict) or not profiles:
raise ManifestValidationError(
[{"code": "PROFILE_SOURCE_SCHEMA", "location": str(path), "message": "profiles must be a non-empty object"}]
)
return set(profiles)
def _resolve_source(root: Path, relative_path: str) -> Path | None:
candidate = Path(relative_path)
if candidate.is_absolute():
return None
resolved = (root / candidate).resolve()
try:
resolved.relative_to(root)
except ValueError:
return None
return resolved
def _validate_proof(
proof: Any,
index: int,
repo_root: Path,
run_root: Path | None,
seen_finding_roles: set[tuple[str, str]],
issues: list[dict[str, str]],
) -> None:
base = f"proofs[{index}]"
proof_obj = _require_object(proof, base, {"finding", "source", "execution"}, set(), issues)
if proof_obj is None:
return
finding = _require_object(proof_obj.get("finding"), f"{base}.finding", {"id", "role"}, set(), issues)
source = _require_object(
proof_obj.get("source"),
f"{base}.source",
{"path", "sha256", "line_start", "line_end", "quote_utf8"},
{"namespace"},
issues,
)
execution = _require_object(
proof_obj.get("execution"),
f"{base}.execution",
{"argv", "exit_code", "stdout_utf8", "stdout_sha256", "exact_match"},
set(),
issues,
)
finding_id: str | None = None
role: str | None = None
if finding is not None:
finding_id_value = finding.get("id")
role_value = finding.get("role")
if not isinstance(finding_id_value, str) or not SAFE_IDENTIFIER.fullmatch(finding_id_value):
_issue(issues, "INVALID_FINDING_ID", f"{base}.finding.id", "must match [A-Za-z0-9][A-Za-z0-9._-]*")
else:
finding_id = finding_id_value
if not isinstance(role_value, str) or not SAFE_ROLE.fullmatch(role_value):
_issue(issues, "INVALID_FINDING_ROLE", f"{base}.finding.role", "must be a lowercase role identifier")
else:
role = role_value
if finding_id is not None and role is not None:
key = (finding_id, role)
if key in seen_finding_roles:
_issue(issues, "DUPLICATE_FINDING_ROLE", f"{base}.finding", f"duplicate pair: {finding_id}/{role}")
else:
seen_finding_roles.add(key)
quote_utf8: str | None = None
source_bytes: bytes | None = None
selected_bytes: bytes | None = None
if source is not None:
namespace = source.get("namespace", "repo")
relative_path = source.get("path")
expected_sha256 = source.get("sha256")
line_start = source.get("line_start")
line_end = source.get("line_end")
quote_value = source.get("quote_utf8")
if namespace not in {"repo", "run"}:
_issue(issues, "INVALID_SOURCE_NAMESPACE", f"{base}.source.namespace", "must be repo or run")
source_root = None
elif namespace == "run" and run_root is None:
_issue(issues, "RUN_ROOT_REQUIRED", f"{base}.source.namespace", "run namespace requires a run root")
source_root = None
else:
source_root = repo_root if namespace == "repo" else run_root
if not isinstance(relative_path, str) or not relative_path or "\\" in relative_path:
_issue(issues, "INVALID_SOURCE_PATH", f"{base}.source.path", "must be a non-empty repo-relative POSIX path")
resolved_source = None
elif source_root is None:
resolved_source = None
else:
resolved_source = _resolve_source(source_root, relative_path)
if resolved_source is None:
_issue(issues, "SOURCE_OUTSIDE_NAMESPACE", f"{base}.source.path", f"path escapes the {namespace} root")
elif not resolved_source.is_file():
_issue(issues, "SOURCE_NOT_FOUND", f"{base}.source.path", "source file does not exist")
else:
try:
source_bytes = resolved_source.read_bytes()
except OSError as exc:
_issue(issues, "SOURCE_READ_ERROR", f"{base}.source.path", str(exc))
if not isinstance(expected_sha256, str) or not HEX_SHA256.fullmatch(expected_sha256):
_issue(issues, "INVALID_SOURCE_SHA256", f"{base}.source.sha256", "must be 64 lowercase hexadecimal characters")
elif source_bytes is not None:
actual_source_sha256 = hashlib.sha256(source_bytes).hexdigest()
if actual_source_sha256 != expected_sha256:
_issue(
issues,
"SOURCE_HASH_MISMATCH",
f"{base}.source.sha256",
f"expected {expected_sha256}, observed {actual_source_sha256}",
)
valid_range = True
if not _is_int(line_start) or line_start < 1:
_issue(issues, "INVALID_LINE_RANGE", f"{base}.source.line_start", "must be an integer >= 1")
valid_range = False
if not _is_int(line_end) or (_is_int(line_start) and line_end < line_start):
_issue(issues, "INVALID_LINE_RANGE", f"{base}.source.line_end", "must be an integer >= line_start")
valid_range = False
if not isinstance(quote_value, str) or not quote_value:
_issue(issues, "INVALID_QUOTE", f"{base}.source.quote_utf8", "must be a non-empty UTF-8 string")
else:
quote_utf8 = quote_value
if source_bytes is not None:
try:
source_bytes.decode("utf-8", errors="strict")
except UnicodeDecodeError as exc:
_issue(issues, "SOURCE_NOT_UTF8", f"{base}.source.path", str(exc))
source_bytes = None
if source_bytes is not None and valid_range:
source_lines = source_bytes.splitlines(keepends=True)
if line_end > len(source_lines):
_issue(
issues,
"LINE_RANGE_OUT_OF_BOUNDS",
f"{base}.source.line_end",
f"source has {len(source_lines)} line(s)",
)
else:
selected_bytes = b"".join(source_lines[line_start - 1 : line_end])
if quote_utf8 is not None and quote_utf8.encode("utf-8") not in selected_bytes:
_issue(
issues,
"QUOTE_MISMATCH",
f"{base}.source.quote_utf8",
"exact quote bytes were not found inside the declared line range",
)
if execution is not None:
argv = execution.get("argv")
exit_code = execution.get("exit_code")
stdout_utf8 = execution.get("stdout_utf8")
stdout_sha256 = execution.get("stdout_sha256")
exact_match = execution.get("exact_match")
if (
not isinstance(argv, list)
or not argv
or any(not isinstance(arg, str) or not arg for arg in argv)
):
_issue(issues, "INVALID_ARGV", f"{base}.execution.argv", "must be a non-empty array of non-empty strings")
if not _is_int(exit_code):
_issue(issues, "INVALID_EXIT_CODE", f"{base}.execution.exit_code", "must be an integer")
elif exit_code != 0:
_issue(issues, "COMMAND_FAILED", f"{base}.execution.exit_code", "recorded verification command did not exit 0")
if not isinstance(stdout_utf8, str):
_issue(issues, "INVALID_STDOUT", f"{base}.execution.stdout_utf8", "must be a UTF-8 string")
if not isinstance(stdout_sha256, str) or not HEX_SHA256.fullmatch(stdout_sha256):
_issue(issues, "INVALID_STDOUT_SHA256", f"{base}.execution.stdout_sha256", "must be 64 lowercase hexadecimal characters")
elif isinstance(stdout_utf8, str):
actual_stdout_sha256 = hashlib.sha256(stdout_utf8.encode("utf-8")).hexdigest()
if actual_stdout_sha256 != stdout_sha256:
_issue(
issues,
"STDOUT_HASH_MISMATCH",
f"{base}.execution.stdout_sha256",
f"expected {stdout_sha256}, observed {actual_stdout_sha256}",
)
if not isinstance(exact_match, bool):
_issue(issues, "INVALID_EXACT_MATCH", f"{base}.execution.exact_match", "must be a boolean")
elif not exact_match:
_issue(issues, "EXACT_MATCH_FALSE", f"{base}.execution.exact_match", "proof cannot pass with exact_match=false")
if isinstance(stdout_utf8, str) and quote_utf8 is not None and stdout_utf8 != quote_utf8:
_issue(
issues,
"STDOUT_QUOTE_MISMATCH",
f"{base}.execution.stdout_utf8",
"recorded stdout is not byte-for-byte equal to quote_utf8",
)
def verify_manifest(
manifest: Any,
repo_root: Path,
allowed_profiles: set[str],
*,
run_root: Path | None = None,
) -> dict[str, Any]:
issues: list[dict[str, str]] = []
root = _require_object(manifest, "$", {"schema_version", "run", "proofs"}, {"verification"}, issues)
if root is None:
raise ManifestValidationError(issues)
if root.get("schema_version") != SCHEMA_VERSION:
_issue(issues, "SCHEMA_VERSION_MISMATCH", "$.schema_version", f"expected {SCHEMA_VERSION}")
run = _require_object(root.get("run"), "$.run", {"id", "profile"}, set(), issues)
if run is not None:
run_id = run.get("id")
profile = run.get("profile")
if not isinstance(run_id, str) or not SAFE_IDENTIFIER.fullmatch(run_id):
_issue(issues, "INVALID_RUN_ID", "$.run.id", "must match [A-Za-z0-9][A-Za-z0-9._-]*")
if not isinstance(profile, str) or profile not in allowed_profiles:
_issue(issues, "INVALID_PROFILE", "$.run.profile", f"must be one of {sorted(allowed_profiles)}")
proofs = root.get("proofs")
if not isinstance(proofs, list) or not proofs:
_issue(issues, "INVALID_PROOFS", "$.proofs", "must be a non-empty array")
else:
seen_finding_roles: set[tuple[str, str]] = set()
for index, proof in enumerate(proofs):
_validate_proof(proof, index, repo_root, run_root, seen_finding_roles, issues)
if issues:
raise ManifestValidationError(issues)
verified = dict(root)
verified["verification"] = {
"schema_version": RESULT_SCHEMA_VERSION,
"status": "PASS",
"proof_count": len(proofs),
"pass_count": len(proofs),
"fail_count": 0,
}
return verified
def manifest_bytes(document: Mapping[str, Any]) -> bytes:
"""Return the canonical bytes used for persisted manifest hashing."""
return (json.dumps(document, ensure_ascii=False, indent=2, sort_keys=True) + "\n").encode("utf-8")
def _failure_document(exc: ManifestValidationError) -> dict[str, Any]:
return {
"schema_version": RESULT_SCHEMA_VERSION,
"status": "FAIL",
"errors": exc.issues,
}
def _atomic_write_json(path: Path, document: Mapping[str, Any]) -> None:
# ``os.replace`` 는 대상 심링크를 *따라가지 않고* 그 자리를 실파일로 갈아치운다.
# cutover 이후 raw/·wiki/ 경로는 vault 정본을 가리키는 심링크이므로, 그런 경로를
# 그대로 받으면 링크가 끊겨 정본과 분리된다(split-brain, 그리고 조용하다). 심링크면
# 정본 경로에 write-through 해 링크를 보존한다.
target = Path(os.path.abspath(path.resolve())) if path.is_symlink() else path
parent = target.parent.resolve()
if not parent.is_dir():
raise OSError(f"output parent directory does not exist: {parent}")
temporary_name: str | None = None
try:
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
dir=parent,
prefix=f".{target.name}.",
suffix=".tmp",
delete=False,
) as stream:
temporary_name = stream.name
stream.write(manifest_bytes(document).decode("utf-8"))
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary_name, target)
except Exception:
if temporary_name is not None:
try:
os.unlink(temporary_name)
except FileNotFoundError:
pass
raise
def _emit(document: Mapping[str, Any]) -> None:
json.dump(document, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
def _parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("manifest", type=Path, help="input proof manifest JSON")
parser.add_argument("--repo-root", type=Path, default=DEFAULT_REPO_ROOT, help="source path root")
parser.add_argument("--run-root", type=Path, help="source root for source.namespace=run")
parser.add_argument("--profiles", type=Path, default=DEFAULT_PROFILES, help="execution profile JSON source")
parser.add_argument("--output", type=Path, help="atomically write the verified manifest; omitted by default")
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = _parse_args(argv)
try:
repo_root = args.repo_root.resolve(strict=True)
if not repo_root.is_dir():
raise OSError(f"repo root is not a directory: {repo_root}")
allowed_profiles = load_allowed_profiles(args.profiles.resolve(strict=True))
manifest = _load_json(args.manifest)
run_root = args.run_root.resolve(strict=True) if args.run_root is not None else None
if run_root is not None and not run_root.is_dir():
raise OSError(f"run root is not a directory: {run_root}")
verified = verify_manifest(manifest, repo_root, allowed_profiles, run_root=run_root)
if args.output is not None:
_atomic_write_json(args.output, verified)
_emit(verified)
return 0
except ManifestValidationError as exc:
_emit(_failure_document(exc))
return 2
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
failure = ManifestValidationError(
[{"code": "IO_OR_JSON_ERROR", "location": str(args.manifest), "message": str(exc)}]
)
_emit(_failure_document(failure))
return 2
if __name__ == "__main__":
raise SystemExit(main())
+241
View File
@@ -0,0 +1,241 @@
#!/usr/bin/env python3
"""Create fixed exact-quote proofs and verify them without executing request argv."""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
import sys
from typing import Any
import proof_manifest
from fs_transaction import replace_many
REQUEST_SCHEMA = "proof-request/v1"
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
RESULT_SCHEMA = "proof-runner-result/v1"
class ProofRequestError(ValueError):
def __init__(self, code: str, message: str, location: str = "") -> None:
self.code = code
self.location = location
super().__init__(message)
def _source_path(
repo_root: Path,
run_root: Path | None,
source: dict[str, Any],
) -> tuple[str, str, Path]:
namespace = source.get("namespace", "repo")
if namespace not in {"repo", "run"}:
raise ProofRequestError("INVALID_SOURCE_NAMESPACE", "source.namespace must be repo or run")
value = source.get("path")
if not isinstance(value, str) or not value or "\\" in value:
raise ProofRequestError("INVALID_SOURCE_PATH", "source.path must be a namespace-relative POSIX path")
candidate = Path(value)
if candidate.is_absolute():
raise ProofRequestError("SOURCE_OUTSIDE_NAMESPACE", "absolute source path is not allowed", value)
base = repo_root if namespace == "repo" else run_root
if base is None:
raise ProofRequestError("RUN_ROOT_REQUIRED", "run namespace requires --run-root", value)
resolved = (base / candidate).resolve()
try:
resolved.relative_to(base)
except ValueError as exc:
raise ProofRequestError("SOURCE_OUTSIDE_NAMESPACE", f"source path escapes {namespace} root", value) from exc
if not resolved.is_file():
raise ProofRequestError("SOURCE_NOT_FOUND", "source file does not exist", value)
return namespace, candidate.as_posix(), resolved
def _line_range(text: str, quote: str, source: dict[str, Any], location: str) -> tuple[int, int]:
start_value = source.get("line_start")
end_value = source.get("line_end")
if (start_value is None) != (end_value is None):
raise ProofRequestError("INCOMPLETE_LINE_RANGE", "line_start and line_end must be supplied together", location)
if start_value is not None:
if (
not isinstance(start_value, int)
or isinstance(start_value, bool)
or not isinstance(end_value, int)
or isinstance(end_value, bool)
or start_value < 1
or end_value < start_value
):
raise ProofRequestError("INVALID_LINE_RANGE", "line range must contain positive ordered integers", location)
lines = text.splitlines(keepends=True)
if end_value > len(lines) or quote not in "".join(lines[start_value - 1 : end_value]):
raise ProofRequestError("QUOTE_MISMATCH", "quote is not inside the requested line range", location)
return start_value, end_value
positions: list[int] = []
cursor = text.find(quote)
while cursor >= 0:
positions.append(cursor)
cursor = text.find(quote, cursor + 1)
if not positions:
raise ProofRequestError("QUOTE_MISMATCH", "expected quote was not found", location)
if len(positions) != 1:
raise ProofRequestError("AMBIGUOUS_QUOTE", "quote occurs more than once; provide a line range", location)
start_index = positions[0]
end_index = start_index + len(quote) - 1
return text.count("\n", 0, start_index) + 1, text.count("\n", 0, end_index) + 1
def build_manifest(request: Any, root: Path, run_root: Path | None = None) -> dict[str, Any]:
if not isinstance(request, dict) or request.get("schema_version") != REQUEST_SCHEMA:
raise ProofRequestError("REQUEST_SCHEMA_MISMATCH", f"expected {REQUEST_SCHEMA}")
run = request.get("run")
proofs = request.get("proofs")
if not isinstance(run, dict) or set(run) != {"id", "profile"}:
raise ProofRequestError("INVALID_RUN", "run must contain only id and profile")
if not isinstance(proofs, list) or not proofs:
raise ProofRequestError("INVALID_PROOFS", "proofs must be a non-empty array")
generated: list[dict[str, Any]] = []
for index, item in enumerate(proofs):
location = f"proofs[{index}]"
if not isinstance(item, dict) or set(item) != {"finding", "source"}:
raise ProofRequestError("INVALID_PROOF_REQUEST", "proof must contain finding and source", location)
finding = item.get("finding")
source = item.get("source")
if not isinstance(finding, dict) or set(finding) != {"id", "role"} or not isinstance(source, dict):
raise ProofRequestError("INVALID_PROOF_REQUEST", "invalid finding/source object", location)
allowed_source = {"namespace", "path", "quote_utf8", "line_start", "line_end"}
if set(source) - allowed_source or not {"path", "quote_utf8"}.issubset(source):
raise ProofRequestError("INVALID_PROOF_REQUEST", "invalid source fields", location)
namespace, relative, resolved = _source_path(root, run_root, source)
quote = source.get("quote_utf8")
if not isinstance(quote, str) or not quote:
raise ProofRequestError("INVALID_QUOTE", "quote_utf8 must be non-empty", location)
source_bytes = resolved.read_bytes()
try:
source_text = source_bytes.decode("utf-8", errors="strict")
except UnicodeDecodeError as exc:
raise ProofRequestError("SOURCE_NOT_UTF8", str(exc), relative) from exc
line_start, line_end = _line_range(source_text, quote, source, location)
generated.append(
{
"finding": {"id": finding.get("id"), "role": finding.get("role")},
"source": {
"namespace": namespace,
"path": relative,
"sha256": hashlib.sha256(source_bytes).hexdigest(),
"line_start": line_start,
"line_end": line_end,
"quote_utf8": quote,
},
"execution": {
"argv": ["proof-runner/exact-utf8-v1", namespace, relative, f"{line_start}:{line_end}"],
"exit_code": 0,
"stdout_utf8": quote,
"stdout_sha256": hashlib.sha256(quote.encode("utf-8")).hexdigest(),
"exact_match": True,
},
}
)
return {"schema_version": proof_manifest.SCHEMA_VERSION, "run": dict(run), "proofs": generated}
def _failure(exc: Exception) -> dict[str, Any]:
if isinstance(exc, proof_manifest.ManifestValidationError):
errors = exc.issues
else:
errors = [{"code": getattr(exc, "code", "IO_OR_JSON_ERROR"), "location": getattr(exc, "location", ""), "message": str(exc)}]
return {"schema_version": proof_manifest.RESULT_SCHEMA_VERSION, "status": "FAIL", "errors": errors}
def _display_path(path: Path, root: Path) -> str:
resolved = path.resolve()
try:
return resolved.relative_to(root).as_posix()
except ValueError:
return resolved.as_posix()
def _constrained_output(path: Path, repo_root: Path, run_root: Path | None) -> Path:
output = path.resolve()
roots = [repo_root, *( [run_root] if run_root is not None else [] )]
if not any(output.is_relative_to(root) for root in roots):
raise ProofRequestError(
"OUTPUT_OUTSIDE_ALLOWED_ROOT",
"proof output must be under --repo-root or --run-root",
output.as_posix(),
)
if not output.parent.is_dir():
raise ProofRequestError("OUTPUT_PARENT_NOT_FOUND", "proof output parent does not exist", output.parent.as_posix())
return output
def render_report_summary(manifest_path: str, manifest_sha256: str, proof_count: int) -> str:
"""Render the compact proof section consumed by report workflows."""
return (
"## 증명 결과\n\n"
f"- Manifest: `{manifest_path}`\n"
f"- Manifest SHA-256: `{manifest_sha256}`\n"
f"- Proof: {proof_count}\n"
f"- PASS: {proof_count}\n"
"- FAIL: 0\n"
)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("request", type=Path, help="proof-request/v1 JSON")
parser.add_argument("--repo-root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--run-root", type=Path, help="isolated root for source.namespace=run and run artifacts")
parser.add_argument("--profiles", type=Path, default=proof_manifest.DEFAULT_PROFILES)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument(
"--summary-output",
type=Path,
help="compact Markdown proof summary (default: proof-summary.md beside --output)",
)
args = parser.parse_args(argv)
try:
root = args.repo_root.resolve(strict=True)
run_root = args.run_root.resolve(strict=True) if args.run_root is not None else None
if run_root is not None and not run_root.is_dir():
raise ProofRequestError("INVALID_RUN_ROOT", "--run-root must be a directory", str(run_root))
with args.request.open("r", encoding="utf-8") as stream:
request = json.load(stream)
manifest = build_manifest(request, root, run_root)
profiles = proof_manifest.load_allowed_profiles(args.profiles.resolve(strict=True))
verified = proof_manifest.verify_manifest(manifest, root, profiles, run_root=run_root)
manifest_content = proof_manifest.manifest_bytes(verified)
manifest_hash = hashlib.sha256(manifest_content).hexdigest()
output = _constrained_output(args.output, root, run_root)
summary_output = _constrained_output(args.summary_output or args.output.with_name("proof-summary.md"), root, run_root)
if output == summary_output:
raise ProofRequestError("OUTPUT_PATH_COLLISION", "manifest and summary outputs must differ", str(output))
manifest_display = _display_path(output, root)
summary_content = render_report_summary(
manifest_display,
manifest_hash,
verified["verification"]["proof_count"],
).encode("utf-8")
replace_many({output: manifest_content, summary_output: summary_content})
result = {
"schema_version": RESULT_SCHEMA,
"status": "PASS",
"manifest": {"path": manifest_display, "sha256": manifest_hash},
"report_summary": {"path": _display_path(summary_output, root)},
"proof_count": verified["verification"]["proof_count"],
"pass_count": verified["verification"]["pass_count"],
"fail_count": verified["verification"]["fail_count"],
}
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 0
except (ProofRequestError, proof_manifest.ManifestValidationError, OSError, UnicodeError, json.JSONDecodeError) as exc:
json.dump(_failure(exc), sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+370
View File
@@ -0,0 +1,370 @@
#!/usr/bin/env python3
"""Fail-closed, read-only quality gate for staged wiki document writes."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
import hashlib
import importlib.util
import json
from pathlib import Path
import re
import sys
from typing import Any, Callable, Iterable, Mapping
import moc_indexer
from contract_markdown import parse_frontmatter
import template_renderer
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
SECTION_ID_RE = re.compile(r"^\s*<!--\s*section-id:\s*([a-z0-9][a-z0-9-]*)\s*-->\s*$", re.MULTILINE)
UNRESOLVED_RE = re.compile(r"\{\{[^{}\n]+\}\}")
TRANSPORT_RE = re.compile(
r"(?:</?content>|</?file(?:\s[^>]*)?>|^(?:<<<<<<<(?:\s.*)?|=======|>>>>>>>(?:\s.*)?)$)",
re.MULTILINE,
)
BRANCH_SECTION_ORDER = (
"branch-parent",
"branch-contract-packet",
"inherited-project-decisions",
"branch-local-decisions",
"declared-overrides",
"branch-goal",
"branch-scope",
)
@dataclass(frozen=True)
class QualityExtension:
"""A fail-closed external gate evaluated against the staged repository.
Typed contracts, projections, and semantic certificates can plug into the
common gate without creating imports from quality_gate back into those
independently versioned runtimes.
"""
name: str
runner: Callable[[Path], Mapping[str, Any]]
required: bool = True
class QualityGateError(RuntimeError):
"""A schema, I/O, or checker-loading failure prevented a quality decision."""
def _module(name: str, path: Path) -> Any:
spec = importlib.util.spec_from_file_location(name, path)
if spec is None or spec.loader is None:
raise QualityGateError(f"checker cannot be loaded: {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _resolved_paths(root: Path, values: Iterable[str | Path]) -> list[Path]:
resolved: set[Path] = set()
for value in values:
path = Path(value)
path = path if path.is_absolute() else root / path
path = path.resolve(strict=True)
try:
path.relative_to(root)
except ValueError as exc:
raise QualityGateError(f"path escapes staged root: {path}") from exc
if not path.is_file():
raise QualityGateError(f"touched path is not a file: {path}")
resolved.add(path)
if not resolved:
raise QualityGateError("at least one touched path is required")
return sorted(resolved)
def _finding(
code: str,
path: str,
message: str,
line: int = 0,
*,
check: str = "",
) -> dict[str, Any]:
result: dict[str, Any] = {"code": code, "path": path, "line": line, "message": message}
if check:
result["check"] = check
return result
def _extension_result(
staged_root: Path,
extension: QualityExtension,
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
if not re.fullmatch(r"[a-z][a-z0-9-]*", extension.name):
raise QualityGateError(f"invalid extension name: {extension.name!r}")
raw = extension.runner(staged_root)
if not isinstance(raw, Mapping):
raise QualityGateError(f"extension {extension.name} returned a non-object")
status = raw.get("status")
if status not in {"PASS", "FAIL", "SKIP"}:
raise QualityGateError(f"extension {extension.name} returned invalid status: {status!r}")
raw_findings = raw.get("findings", [])
if not isinstance(raw_findings, list) or any(not isinstance(item, Mapping) for item in raw_findings):
raise QualityGateError(f"extension {extension.name} findings must be an array of objects")
findings: list[dict[str, Any]] = []
for item in raw_findings:
findings.append(
_finding(
str(item.get("code", "EXTERNAL_CHECK_FAILED")),
str(item.get("path", "<repository>")),
str(item.get("message", item.get("code", "external check failed"))),
int(item.get("line", 0) or 0),
check=extension.name,
)
)
if status == "FAIL" and not findings:
findings.append(
_finding(
"EXTERNAL_CHECK_FAILED",
"<repository>",
f"{extension.name} returned FAIL without findings",
check=extension.name,
)
)
if status == "SKIP" and extension.required:
findings.append(
_finding(
"REQUIRED_EXTENSION_SKIPPED",
"<repository>",
f"required extension was skipped: {extension.name}",
check=extension.name,
)
)
effective_status = "FAIL" if findings else status
return {
"name": extension.name,
"status": effective_status,
"finding_count": len(findings),
"schema_version": raw.get("schema_version"),
"required": extension.required,
}, findings
def scan_hygiene(paths: Iterable[Path], *, root: Path | None = None) -> list[dict[str, Any]]:
"""Return transport-wrapper findings for the supplied text files."""
base = root.resolve() if root is not None else None
findings: list[dict[str, Any]] = []
for path in sorted(set(paths)):
text = path.read_text(encoding="utf-8")
rel = path.relative_to(base).as_posix() if base is not None else path.as_posix()
for match in TRANSPORT_RE.finditer(text):
findings.append(
_finding(
"SOURCE_HYGIENE_VIOLATION",
rel,
f"transport wrapper token is forbidden: {match.group(0)[:80]!r}",
text.count("\n", 0, match.start()) + 1,
)
)
return findings
def run(
root: Path,
touched_paths: Iterable[str | Path],
*,
structure_paths: Iterable[str | Path] | None = None,
template_root: Path | None = None,
include_graph: bool = True,
require_moc_convergence: bool = True,
extensions: Iterable[QualityExtension] = (),
) -> dict[str, Any]:
"""Validate staged bytes without modifying them.
The caller is responsible for committing only after this function returns
``status=PASS``. Operational failures raise :class:`QualityGateError` so a
gateway cannot accidentally treat an incomplete check as a quality finding.
"""
try:
staged_root = root.resolve(strict=True)
templates = (template_root or DEFAULT_ROOT).resolve(strict=True)
touched = _resolved_paths(staged_root, touched_paths)
structure = _resolved_paths(staged_root, structure_paths if structure_paths is not None else touched_paths)
before = {path: hashlib.sha256(path.read_bytes()).hexdigest() for path in touched}
findings: list[dict[str, Any]] = []
checks: list[dict[str, Any]] = []
hygiene = scan_hygiene(touched, root=staged_root)
for item in hygiene:
item["check"] = "source-hygiene"
findings.extend(hygiene)
checks.append({"name": "source-hygiene", "status": "FAIL" if hygiene else "PASS", "finding_count": len(hygiene)})
document_findings: list[dict[str, Any]] = []
for path in touched:
rel = path.relative_to(staged_root).as_posix()
text = path.read_text(encoding="utf-8")
for match in UNRESOLVED_RE.finditer(text):
document_findings.append(
_finding(
"UNRESOLVED_PLACEHOLDER",
rel,
match.group(0),
text.count("\n", 0, match.start()) + 1,
)
)
section_ids = SECTION_ID_RE.findall(text)
duplicates = sorted({item for item in section_ids if section_ids.count(item) > 1})
for section_id in duplicates:
document_findings.append(_finding("DUPLICATE_SECTION_ID", rel, section_id))
if "contract_packet" in parse_frontmatter(text) or "branch-contract-packet" in section_ids:
observed = [item for item in section_ids if item in BRANCH_SECTION_ORDER]
if tuple(observed) != BRANCH_SECTION_ORDER:
document_findings.append(
_finding(
"SECTION_ID_ORDER",
rel,
f"expected {list(BRANCH_SECTION_ORDER)}, observed {observed}",
)
)
try:
template_renderer.generated_region(text)
except template_renderer.TemplateRenderError as exc:
document_findings.append(_finding(exc.code, rel, str(exc)))
# 한국어 문체·자연스러움 검사는 이 하네스의 책임이 아니다.
# 별도 하네스 im-not-ai(`/humanize-korean`)가 문서 작성이 끝난 뒤 일괄 처리한다.
for item in document_findings:
item["check"] = "document-schema"
findings.extend(document_findings)
checks.append({"name": "document-schema", "status": "FAIL" if document_findings else "PASS", "finding_count": len(document_findings)})
structure_lint = _module(
"wiki_structure_lint_quality_gate",
templates / ".claude/hooks/wiki_structure_lint.py",
)
by_st, by_file = structure_lint.build_template_index(templates)
vault_paths, vault_bases = structure_lint.build_vault_index(staged_root)
cache: dict[Path, str] = {}
structure_findings: list[dict[str, Any]] = []
for path in structure:
rel = path.relative_to(staged_root).as_posix()
mode = structure_lint.classify(rel, staged_root)
lint_findings, _source_type = structure_lint.lint_file(
path,
staged_root,
by_st,
by_file,
vault_paths,
vault_bases,
cache,
mode=mode,
)
structure_findings.extend(_finding(code, rel, message, line, check="structure") for code, line, message in lint_findings)
findings.extend(structure_findings)
checks.append({"name": "structure", "status": "FAIL" if structure_findings else "PASS", "finding_count": len(structure_findings)})
if require_moc_convergence:
moc_findings: list[dict[str, Any]] = []
moc_updates, _moc_stats = moc_indexer.build_updates(staged_root)
if moc_updates:
for path in sorted(moc_updates):
moc_findings.append(
_finding(
"MOC_NOT_CONVERGED",
path.relative_to(staged_root).as_posix(),
"generated reverse view differs from canonical child edges",
check="moc",
)
)
findings.extend(moc_findings)
checks.append({"name": "moc", "status": "FAIL" if moc_findings else "PASS", "finding_count": len(moc_findings)})
else:
checks.append({"name": "moc", "status": "SKIP", "finding_count": 0})
if include_graph:
graph = _module(
"wiki_graph_contract_check_quality_gate",
templates / ".claude/hooks/wiki_graph_contract_check.py",
)
graph_findings, _warnings, _stats = graph.scan(staged_root, include_expected_edges=True)
normalized_graph = [_finding(code, rel, message, line, check="graph") for code, rel, line, message in graph_findings]
findings.extend(normalized_graph)
checks.append({"name": "graph", "status": "FAIL" if normalized_graph else "PASS", "finding_count": len(normalized_graph)})
else:
checks.append({"name": "graph", "status": "SKIP", "finding_count": 0})
seen_extensions: set[str] = set()
for extension in extensions:
if not isinstance(extension, QualityExtension):
raise QualityGateError("extensions must contain QualityExtension values")
if extension.name in seen_extensions:
raise QualityGateError(f"duplicate extension name: {extension.name}")
seen_extensions.add(extension.name)
check_result, extension_findings = _extension_result(staged_root, extension)
checks.append(check_result)
findings.extend(extension_findings)
after = {path: hashlib.sha256(path.read_bytes()).hexdigest() for path in touched}
for path in touched:
if before[path] != after[path]:
findings.append(
_finding(
"TOUCHED_PATH_MUTATED_DURING_GATE",
path.relative_to(staged_root).as_posix(),
"a read-only checker changed staged bytes",
check="touched-paths",
)
)
mutation_count = sum(1 for item in findings if item.get("check") == "touched-paths")
checks.append({"name": "touched-paths", "status": "FAIL" if mutation_count else "PASS", "finding_count": mutation_count})
findings.sort(key=lambda item: (item["path"], item["line"], item["code"], item["message"]))
return {
"schema_version": "quality-gate-result/v1",
"status": "PASS" if not findings else "FAIL",
"findings": findings,
"checked_paths": [path.relative_to(staged_root).as_posix() for path in touched],
"touched_paths": [path.relative_to(staged_root).as_posix() for path in touched],
"checks": checks,
}
except QualityGateError:
raise
except (OSError, UnicodeError, ValueError, ImportError) as exc:
raise QualityGateError(str(exc)) from exc
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--path", action="append", required=True)
parser.add_argument(
"--branch-scope",
action="store_true",
help="validate project/branch MOC through the graph gate without the R2 all-document relation index",
)
args = parser.parse_args(argv)
try:
result = run(args.root, args.path, require_moc_convergence=not args.branch_scope)
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 0 if result["status"] == "PASS" else 1
except QualityGateError as exc:
json.dump(
{
"schema_version": "quality-gate-result/v1",
"status": "ERROR",
"errors": [{"code": "QUALITY_GATE_ERROR", "message": str(exc)}],
},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+385
View File
@@ -0,0 +1,385 @@
#!/usr/bin/env python3
"""Cumulative, machine-readable release completion gate for harness v2."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
import json
from pathlib import Path
import subprocess
import sys
from typing import Any, Callable, Iterable, Sequence
import quality_gate
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
LEVELS = {"r1": 1, "r2": 2, "r3": 3}
RESULT_SCHEMA = "harness-release-gate/v2"
@dataclass(frozen=True)
class GateCommand:
name: str
argv: tuple[str, ...]
release: int = 1
json_requirements: tuple[tuple[str, object], ...] = ()
Runner = Callable[..., subprocess.CompletedProcess[str]]
def _command_status(code: int, stdout: str) -> str:
if code == 0:
return "PASS"
if code == 1:
return "FAIL"
# Some older deterministic checkers used exit 2 for both I/O and a
# fail-closed contract finding. Preserve the release-gate 0/1/2 contract
# by recognizing only an explicit, non-I/O JSON finding as quality FAIL.
try:
document = json.loads(stdout)
except (json.JSONDecodeError, TypeError):
return "ERROR"
error = document.get("error") if isinstance(document, dict) else None
error_code = error.get("code", "") if isinstance(error, dict) else ""
if document.get("status") == "FAIL" and error_code and error_code not in {
"IO_ERROR",
"RELATION_SOURCE_ERROR",
"QUALITY_GATE_ERROR",
}:
return "FAIL"
return "ERROR"
def _existing_test_modules(root: Path, names: Iterable[str]) -> list[str]:
return [name for name in names if (root / (name.replace(".", "/") + ".py")).is_file()]
def default_commands(root: Path) -> list[GateCommand]:
py = sys.executable
commands = [
GateCommand(
"typed_contract_gate",
(py, "harness/runtime/typed_contract_check.py", "--root", str(root)),
),
GateCommand(
"projection_gate",
(py, "harness/runtime/contract_projection.py", "--root", str(root), "--check"),
),
GateCommand(
"semantic_surface_gate",
(py, "harness/runtime/semantic_surface_extractor.py", "--root", str(root), "--check"),
),
GateCommand(
"hub_semantic_gate",
(
py,
"harness/runtime/semantic_certificate.py",
"--root",
str(root),
"--check",
"--mode",
"hub",
),
),
GateCommand(
"semantic_certificate_gate",
(py, "harness/runtime/semantic_certificate.py", "--root", str(root), "--check"),
),
GateCommand(
"local_semantic_gate",
(
py,
"harness/runtime/semantic_certificate.py",
"--root",
str(root),
"--check",
"--mode",
"local",
),
release=2,
),
GateCommand(
"semantic_regression_gate",
(py, "harness/runtime/semantic_regression.py", "--root", str(root), "--check"),
release=2,
),
GateCommand("source-hygiene-contract", (py, "harness/runtime/source_hygiene.py", "--check", "--root", str(root))),
GateCommand("workflow-adapters", (py, "harness/adapters/generate.py", "--check", "--root", str(root))),
GateCommand("rule-adapters", (py, "harness/adapters/generate_rules.py", "--check", "--root", str(root))),
GateCommand("moc-convergence", (py, "harness/runtime/moc_indexer.py", "--root", str(root), "--check"), release=2),
GateCommand("migration-convergence", (py, "harness/runtime/migrate_graph_contracts.py", "--root", str(root), "--check")),
GateCommand("graph-contract", (py, ".claude/hooks/wiki_graph_contract_check.py", "--root", str(root), "--all")),
GateCommand(
"workflow-dispatch-contract",
(py, "harness/runtime/workflow_dispatch.py", "--root", str(root), "--check-all"),
release=2,
),
GateCommand(
"workflow-source-connections",
(py, "harness/runtime/workflow_connection_check.py", "--root", str(root)),
release=2,
),
GateCommand(
"consistency-contract",
(py, ".claude/hooks/wiki_consistency_check.py", "--root", str(root), "--all"),
release=2,
),
GateCommand(
"full-links",
(py, ".claude/hooks/wiki_structure_lint.py", "--root", str(root), "--all", "--links-only"),
release=2,
),
GateCommand(
"active-structure",
(py, "harness/runtime/active_structure_check.py", "--root", str(root)),
release=2,
),
]
# 한국어 문체 검사는 별도 하네스 im-not-ai(`/humanize-korean`)로 이관했다.
# 이 게이트는 구조·계약·의미 정합만 판정한다.
r1_tests = _existing_test_modules(
root,
(
"harness.tests.test_template_renderer",
"harness.tests.test_quality_gate",
"harness.tests.test_branch_from_project",
"harness.tests.test_migrate_graph_contracts",
"harness.tests.test_typed_contract_check",
"harness.tests.test_contract_projection",
),
)
if r1_tests:
commands.append(GateCommand("r1-runtime-tests", (py, "-m", "unittest", *r1_tests)))
if (root / ".claude/hooks").is_dir():
commands.append(
GateCommand(
"hook-tests",
(py, "-m", "unittest", "discover", "-s", ".claude/hooks", "-p", "test_*.py"),
)
)
for profile, risk in (("capture", "low"), ("design", "medium"), ("audit", "high"), ("publish", "high")):
commands.append(
GateCommand(
f"execution-profile-{profile}",
(py, "harness/runtime/execution_profile.py", profile, "--risk", risk),
release=2,
)
)
r2_tests = _existing_test_modules(
root,
(
"harness.tests.test_moc_indexer",
"harness.tests.test_document_commit",
"harness.tests.test_execution_profile",
"harness.tests.test_proof_manifest",
"harness.tests.test_proof_runner",
"harness.tests.test_workflow_dispatch",
"harness.tests.test_workflow_connection_check",
"harness.tests.test_active_structure_check",
),
)
if r2_tests:
commands.append(GateCommand("r2-runtime-tests", (py, "-m", "unittest", *r2_tests), release=2))
commands.append(
GateCommand(
"vault-layout",
(py, "harness/runtime/layout_check.py", "--root", str(root)),
release=3,
json_requirements=(
("mode", "canonical"),
("authority", "vault"),
("write_roots", ("vault",)),
),
)
)
r3_tests = _existing_test_modules(root, ("harness.tests.test_layout_check",))
if r3_tests:
commands.append(GateCommand("r3-runtime-tests", (py, "-m", "unittest", *r3_tests), release=3))
return commands
def hygiene_paths(root: Path) -> list[Path]:
patterns = (
"harness/source/skills/*.md",
".agents/skills/*/SKILL.md",
".agents/workflows/*.md",
".claude/commands/*.md",
)
return sorted({path for pattern in patterns for path in root.glob(pattern) if path.is_file()})
def run_gate(
root: Path,
level: str,
*,
commands: Sequence[GateCommand] | None = None,
runner: Runner = subprocess.run,
source_paths: Sequence[Path] | None = None,
) -> dict[str, Any]:
if level not in LEVELS:
raise ValueError(f"unsupported release level: {level}")
root = root.resolve(strict=True)
selected_level = LEVELS[level]
checks: list[dict[str, Any]] = []
try:
hygiene_findings = quality_gate.scan_hygiene(
source_paths if source_paths is not None else hygiene_paths(root),
root=root,
)
checks.append(
{
"name": "source-hygiene",
"status": "PASS" if not hygiene_findings else "FAIL",
"exit_code": 0 if not hygiene_findings else 1,
"findings": hygiene_findings,
}
)
except (OSError, UnicodeError, ValueError) as exc:
checks.append(
{
"name": "source-hygiene",
"status": "ERROR",
"exit_code": 2,
"errors": [{"code": "HYGIENE_IO_ERROR", "message": str(exc)}],
}
)
for command in commands if commands is not None else default_commands(root):
if command.release > selected_level:
continue
if (
len(command.argv) >= 2
and command.argv[0] == sys.executable
and command.argv[1].endswith(".py")
and not (root / command.argv[1]).is_file()
):
checks.append(
{
"name": command.name,
"status": "FAIL",
"exit_code": 1,
"findings": [
{
"code": "COMMAND_MISSING",
"message": f"required release gate command is missing: {command.argv[1]}",
}
],
}
)
continue
try:
completed = runner(
list(command.argv),
cwd=root,
check=False,
capture_output=True,
text=True,
timeout=120,
)
code = completed.returncode
status = _command_status(code, completed.stdout)
contract_findings: list[dict[str, str]] = []
child_document: dict[str, Any] | None = None
try:
parsed_stdout = json.loads(completed.stdout)
except (json.JSONDecodeError, TypeError):
pass
else:
if isinstance(parsed_stdout, dict):
child_document = parsed_stdout
if code == 0 and command.json_requirements:
if child_document is None:
status = "ERROR"
contract_findings.append(
{"code": "RESULT_SCHEMA_ERROR", "message": "command returned non-object JSON"}
)
else:
for key, expected in command.json_requirements:
observed = child_document.get(key)
comparable = tuple(observed) if isinstance(expected, tuple) and isinstance(observed, list) else observed
if comparable != expected:
status = "FAIL"
contract_findings.append(
{
"code": "RELEASE_CONTRACT_MISMATCH",
"message": f"{key}: expected {expected!r}, observed {observed!r}",
}
)
result = {
"name": command.name,
"status": status,
"exit_code": code,
"stdout": completed.stdout[-4000:],
"stderr": completed.stderr[-4000:],
}
if contract_findings:
result["findings"] = contract_findings
elif child_document is not None and isinstance(child_document.get("findings"), list):
result["findings"] = child_document["findings"]
if child_document is not None and isinstance(child_document.get("errors"), list):
result["errors"] = child_document["errors"]
checks.append(result)
except (OSError, subprocess.SubprocessError) as exc:
checks.append(
{
"name": command.name,
"status": "ERROR",
"exit_code": 2,
"errors": [{"code": "COMMAND_ENVIRONMENT_ERROR", "message": str(exc)}],
}
)
status = "ERROR" if any(item["status"] == "ERROR" for item in checks) else "FAIL" if any(
item["status"] == "FAIL" for item in checks
) else "PASS"
result = {
"schema_version": RESULT_SCHEMA,
"status": status,
"level": level,
"cumulative": True,
"gates": checks,
"summary": {
"total": len(checks),
"passed": sum(item["status"] == "PASS" for item in checks),
"failed": sum(item["status"] == "FAIL" for item in checks),
"errors": sum(item["status"] == "ERROR" for item in checks),
},
}
# Transitional alias for existing automation; new consumers must use gates.
result["checks"] = result["gates"]
return result
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--level", choices=sorted(LEVELS), required=True)
args = parser.parse_args(argv)
try:
result = run_gate(args.root, args.level)
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 0 if result["status"] == "PASS" else 1 if result["status"] == "FAIL" else 2
except (OSError, UnicodeError, ValueError) as exc:
json.dump(
{
"schema_version": RESULT_SCHEMA,
"status": "ERROR",
"errors": [{"code": "RELEASE_GATE_ERROR", "message": str(exc)}],
},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+372
View File
@@ -0,0 +1,372 @@
#!/usr/bin/env python3
"""Build semantic auditor requests and validate grounded verdict results.
This runtime deliberately does not infer assertions or verdicts. It binds the
auditor's work to deterministic surface/candidate bytes, revalidates proof
manifests, and applies the blocking policy from the ontology.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
import proof_manifest
import semantic_candidate_builder
import semantic_surface_extractor
ASSERTION_REQUEST_SCHEMA = "semantic-assertion-request/v1"
VERDICT_REQUEST_SCHEMA = "semantic-verdict-request/v1"
AUDIT_RESULT_SCHEMA = "semantic-audit-result/v1"
VALIDATED_RESULT_SCHEMA = "semantic-audit-validation-result/v1"
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
class SemanticAuditError(ValueError):
"""Audit request/result bytes violate the deterministic contract."""
def canonical_json_bytes(value: Any) -> bytes:
return semantic_surface_extractor.canonical_json_bytes(value)
def _sha(value: Any) -> str:
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
def build_assertion_request(extraction: Mapping[str, Any], ontology: Mapping[str, Any]) -> dict[str, Any]:
coverage = extraction.get("coverage", {})
if extraction.get("findings") and any(item.get("severity") == "error" for item in extraction["findings"]):
raise SemanticAuditError("cannot request assertions for uncovered semantic surfaces")
if coverage.get("eligible_surface_blocks") != coverage.get("extracted_surface_blocks", 0) + coverage.get("explicitly_excluded_blocks", 0):
raise SemanticAuditError("surface coverage is incomplete")
return {
"schema_version": ASSERTION_REQUEST_SCHEMA,
"subject": extraction["path"],
"mode": extraction["mode"],
"document_sha256": extraction["document_sha256"],
"surface_manifest_sha256": _sha(extraction),
"ontology_sha256": _sha(ontology),
"predicate_ontology": list(ontology["predicates"]),
"surfaces": list(extraction["surfaces"]),
"explicitly_excluded": list(extraction["excluded"]),
"output_schema": semantic_candidate_builder.ASSERTION_SCHEMA,
}
def build_verdict_request(candidate_result: Mapping[str, Any], *, explicit_blocking: Iterable[Mapping[str, Any]] = ()) -> dict[str, Any]:
if candidate_result.get("schema_version") != semantic_candidate_builder.RESULT_SCHEMA or candidate_result.get("status") != "PASS":
raise SemanticAuditError("candidate result must be semantic-candidate-result/v1 PASS")
blocking: list[dict[str, str]] = []
for index, item in enumerate(explicit_blocking):
if not isinstance(item, Mapping) or set(item) != {"code", "message"}:
raise SemanticAuditError(f"explicit_blocking[{index}] must contain code and message")
code, message = item.get("code"), item.get("message")
if not isinstance(code, str) or not code or not isinstance(message, str) or not message:
raise SemanticAuditError(f"explicit_blocking[{index}] fields must be non-empty strings")
blocking.append({"code": code, "message": message})
return {
"schema_version": VERDICT_REQUEST_SCHEMA,
"subject": candidate_result["subject"],
"mode": candidate_result["mode"],
"document_sha256": candidate_result["document_sha256"],
"candidate_manifest_sha256": _sha(candidate_result),
"ontology_sha256": candidate_result["ontology_sha256"],
"coverage": dict(candidate_result["coverage"]),
"assertions": list(candidate_result["assertions"]),
"candidates": list(candidate_result["candidates"]),
"explicit_blocking": sorted(blocking, key=lambda item: (item["code"], item["message"])),
"output_schema": AUDIT_RESULT_SCHEMA,
}
def _resolve_reference(root: Path, reference: Mapping[str, Any], run_root: Path | None) -> tuple[Path, str]:
if set(reference) != {"namespace", "path", "sha256"}:
raise SemanticAuditError("proof_manifest must contain namespace, path, and sha256")
namespace, value, expected = reference.get("namespace"), reference.get("path"), reference.get("sha256")
if namespace not in {"repo", "run"}:
raise SemanticAuditError("proof manifest namespace must be repo or run")
base = root if namespace == "repo" else run_root
if base is None:
raise SemanticAuditError("run proof manifest requires run_root")
if not isinstance(value, str) or not value or "\\" in value or Path(value).is_absolute():
raise SemanticAuditError("proof manifest path must be relative POSIX")
path = (base / value).resolve()
try:
path.relative_to(base.resolve())
except ValueError as exc:
raise SemanticAuditError("proof manifest path escapes its namespace") from exc
if not path.is_file():
raise SemanticAuditError("proof manifest does not exist")
if not isinstance(expected, str) or not HEX_SHA256.fullmatch(expected):
raise SemanticAuditError("proof manifest sha256 is invalid")
observed = hashlib.sha256(path.read_bytes()).hexdigest()
if observed != expected:
raise SemanticAuditError("proof manifest hash mismatch")
return path, expected
def _verify_evidence(verdict: Mapping[str, Any], candidate: Mapping[str, Any], assertions: Mapping[str, Mapping[str, Any]]) -> None:
for key, assertion_key in (("evidence_a", "assertion_a"), ("evidence_b", "assertion_b")):
evidence = verdict.get(key)
assertion = assertions[candidate[assertion_key]]
if not isinstance(evidence, Mapping) or set(evidence) != {"quote", "line_start", "line_end"}:
raise SemanticAuditError(f"{key} must contain quote and exact line range")
if (
evidence.get("quote") != assertion["quote"]
or evidence.get("line_start") != assertion["line_start"]
or evidence.get("line_end") != assertion["line_end"]
):
raise SemanticAuditError(f"{key} does not match its grounded assertion")
def _verify_negative_proof(
root: Path,
run_root: Path | None,
profiles: set[str],
verdict: Mapping[str, Any],
candidate: Mapping[str, Any],
assertions: Mapping[str, Mapping[str, Any]],
) -> str:
reference = verdict.get("proof_manifest")
if not isinstance(reference, Mapping):
raise SemanticAuditError("negative semantic verdict requires proof_manifest")
path, digest = _resolve_reference(root, reference, run_root)
try:
manifest = json.loads(path.read_text(encoding="utf-8"))
verified = proof_manifest.verify_manifest(manifest, root, profiles, run_root=run_root)
except (OSError, UnicodeError, json.JSONDecodeError, proof_manifest.ManifestValidationError) as exc:
raise SemanticAuditError(f"proof manifest revalidation failed: {exc}") from exc
if verified["verification"]["status"] != "PASS" or verified["verification"]["fail_count"] != 0:
raise SemanticAuditError("proof manifest is not PASS")
expected = {
(candidate["candidate_id"], "assertion_a", assertions[candidate["assertion_a"]]["quote"]),
(candidate["candidate_id"], "assertion_b", assertions[candidate["assertion_b"]]["quote"]),
}
observed = {
(item["finding"]["id"], item["finding"]["role"], item["source"]["quote_utf8"])
for item in verified["proofs"]
}
if not expected.issubset(observed):
raise SemanticAuditError("proof manifest does not bind both candidate assertions")
return digest
def validate_result(
root: Path,
request: Mapping[str, Any],
result: Any,
*,
ontology_path: Path = semantic_candidate_builder.DEFAULT_ONTOLOGY,
profiles_path: Path = proof_manifest.DEFAULT_PROFILES,
run_root: Path | None = None,
) -> dict[str, Any]:
root = root.resolve(strict=True)
ontology = semantic_candidate_builder.load_ontology(root, ontology_path)
request_fields = {
"schema_version", "subject", "mode", "document_sha256", "candidate_manifest_sha256",
"ontology_sha256", "coverage", "assertions", "candidates", "explicit_blocking", "output_schema",
}
if not isinstance(request, Mapping) or set(request) != request_fields or request.get("schema_version") != VERDICT_REQUEST_SCHEMA:
raise SemanticAuditError(f"expected {VERDICT_REQUEST_SCHEMA}")
subject = request.get("subject")
if not isinstance(subject, str) or not subject or Path(subject).is_absolute() or ".." in Path(subject).parts:
raise SemanticAuditError("verdict request subject must be a canonical repo-relative path")
document = Path(os.path.abspath(root / subject))
try:
document.relative_to(root)
document.resolve(strict=True).relative_to(root)
except (ValueError, FileNotFoundError) as exc:
raise SemanticAuditError("verdict request subject escapes repository") from exc
if not document.is_file():
raise SemanticAuditError("verdict request subject does not exist")
current_document_sha = hashlib.sha256(document.read_bytes()).hexdigest()
if request.get("document_sha256") != current_document_sha:
raise SemanticAuditError("verdict request is stale for current document bytes")
policy = semantic_surface_extractor.load_policy(root)
extraction = semantic_surface_extractor.extract_document(root, document, policy)
if extraction["mode"] != request.get("mode"):
raise SemanticAuditError("verdict request mode differs from current document policy")
assertion_result = {
"schema_version": semantic_candidate_builder.ASSERTION_SCHEMA,
"subject": subject,
"mode": request["mode"],
"surface_manifest_sha256": _sha(extraction),
"assertions": request.get("assertions"),
}
rebuilt = semantic_candidate_builder.build(root, extraction, assertion_result, ontology_path)
for field in ("ontology_sha256", "coverage", "assertions", "candidates"):
if request.get(field) != rebuilt[field]:
raise SemanticAuditError(f"verdict request {field} differs from deterministic reconstruction")
if request.get("candidate_manifest_sha256") != _sha(rebuilt):
raise SemanticAuditError("verdict request candidate manifest hash is stale or forged")
if request.get("output_schema") != AUDIT_RESULT_SCHEMA:
raise SemanticAuditError("verdict request output schema mismatch")
required = {"schema_version", "request_sha256", "subject", "mode", "auditor", "verdicts"}
if not isinstance(result, dict) or set(result) != required or result.get("schema_version") != AUDIT_RESULT_SCHEMA:
raise SemanticAuditError(f"audit result must contain exact {AUDIT_RESULT_SCHEMA} fields")
if result.get("request_sha256") != _sha(request):
raise SemanticAuditError("audit result is not bound to current verdict request")
if result.get("subject") != request.get("subject") or result.get("mode") != request.get("mode"):
raise SemanticAuditError("audit result subject/mode mismatch")
auditor = result.get("auditor")
if not isinstance(auditor, dict) or set(auditor) != {"contract_version", "model_id", "run_id"}:
raise SemanticAuditError("auditor must contain contract_version/model_id/run_id")
if auditor.get("contract_version") != ontology["auditor_contract_version"]:
raise SemanticAuditError("auditor contract version mismatch")
if any(not isinstance(auditor.get(key), str) or not auditor[key] for key in ("model_id", "run_id")):
raise SemanticAuditError("auditor model_id/run_id must be non-empty")
raw_verdicts = result.get("verdicts")
if not isinstance(raw_verdicts, list):
raise SemanticAuditError("verdicts must be an array")
candidates = {item["candidate_id"]: item for item in request["candidates"]}
assertions = {item["assertion_id"]: item for item in request["assertions"]}
seen: set[str] = set()
verified_findings: list[dict[str, Any]] = []
dropped: list[dict[str, Any]] = []
positive = {"CONSISTENT", "COMPLEMENTARY", "CONTEXTUAL_VARIANT"}
negative = {"AMBIGUOUS_AUTHORITY", "RESTATEMENT_DRIFT", "CONTRADICTION"}
profiles_source = profiles_path if profiles_path.is_absolute() else root / profiles_path
profiles = proof_manifest.load_allowed_profiles(profiles_source.resolve(strict=True))
for index, item in enumerate(raw_verdicts):
fields = {"candidate_id", "verdict", "rationale", "evidence_a", "evidence_b", "proof_manifest"}
if not isinstance(item, dict) or set(item) != fields:
raise SemanticAuditError(f"verdicts[{index}] has missing or unknown fields")
candidate_id = item.get("candidate_id")
verdict = item.get("verdict")
if candidate_id not in candidates or candidate_id in seen:
raise SemanticAuditError(f"verdicts[{index}] has unknown or duplicate candidate_id")
seen.add(candidate_id)
if verdict not in ontology["verdicts"]:
raise SemanticAuditError(f"verdicts[{index}] is outside exact verdict set")
if not isinstance(item.get("rationale"), str) or not item["rationale"].strip():
raise SemanticAuditError(f"verdicts[{index}].rationale must be non-empty")
candidate = candidates[candidate_id]
_verify_evidence(item, candidate, assertions)
if verdict in positive:
if item.get("proof_manifest") is not None:
raise SemanticAuditError("passing verdict must not claim a finding proof")
continue
try:
proof_sha = _verify_negative_proof(root, run_root, profiles, item, candidate, assertions)
except SemanticAuditError as exc:
dropped.append({"candidate_id": candidate_id, "verdict": verdict, "reason": str(exc)})
continue
verified_findings.append({
"candidate_id": candidate_id,
"verdict": verdict,
"rationale": item["rationale"],
"evidence_a": dict(item["evidence_a"]),
"evidence_b": dict(item["evidence_b"]),
"proof_manifest_sha256": proof_sha,
})
missing = sorted(set(candidates) - seen)
if missing:
raise SemanticAuditError(f"candidate verdict coverage is incomplete: {missing}")
mode = str(request["mode"])
explicit = list(request.get("explicit_blocking", []))
blocking = len(explicit) + sum(
item["verdict"] == "CONTRADICTION" or (mode == "hub" and item["verdict"] == "AMBIGUOUS_AUTHORITY")
for item in verified_findings
)
readiness = sum(item["verdict"] == "RESTATEMENT_DRIFT" for item in verified_findings)
# A negative judgment without replayable proof is not evidence of a
# contradiction, but it is also not a certifiable clean audit. Treat
# dropped candidates as fail-closed in every mode so a local certificate
# cannot hide an auditor-raised contradiction merely because its proof
# reference was omitted or stale.
status = "PASS" if blocking == 0 and readiness == 0 and not dropped else "FAIL"
return {
"schema_version": VALIDATED_RESULT_SCHEMA,
"status": status,
"subject": request["subject"],
"mode": mode,
"document_sha256": request["document_sha256"],
"request_sha256": _sha(request),
"ontology_sha256": request["ontology_sha256"],
"coverage": {
"eligible_surfaces": request["coverage"]["eligible_surfaces"],
"processed_surfaces": request["coverage"]["processed_surfaces"],
"candidate_pairs": len(candidates),
"processed_pairs": len(seen),
"dropped_pairs": len(dropped),
},
"findings": verified_findings,
"dropped": dropped,
"explicit_blocking": explicit,
"counts": {
"blocking": blocking,
"readiness_blocking": readiness,
"verified_findings": len(verified_findings),
"dropped_pairs": len(dropped),
},
"auditor": dict(auditor),
}
def _load(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def _one_extraction(value: Any) -> Mapping[str, Any]:
if isinstance(value, dict) and {"path", "mode", "surfaces", "coverage"}.issubset(value):
return value
documents = value.get("documents") if isinstance(value, dict) else None
if not isinstance(documents, list) or len(documents) != 1 or not isinstance(documents[0], dict):
raise SemanticAuditError("assertion request input must contain exactly one extracted document")
return documents[0]
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("phase", choices=("assertion-request", "verdict-request", "validate"))
parser.add_argument("input", type=Path)
parser.add_argument("result", type=Path, nargs="?")
parser.add_argument("--root", type=Path, default=Path(__file__).resolve().parents[2])
parser.add_argument("--run-root", type=Path)
parser.add_argument("--explicit-blocking", type=Path)
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
if args.phase == "assertion-request":
extraction = _one_extraction(_load(args.input))
ontology = semantic_candidate_builder.load_ontology(root)
output = build_assertion_request(extraction, ontology)
elif args.phase == "verdict-request":
blocking: Iterable[Mapping[str, Any]] = ()
if args.explicit_blocking is not None:
value = _load(args.explicit_blocking)
if not isinstance(value, list):
raise SemanticAuditError("explicit blocking input must be an array")
blocking = value
output = build_verdict_request(_load(args.input), explicit_blocking=blocking)
else:
if args.result is None:
raise SemanticAuditError("validate requires request and result paths")
output = validate_result(root, _load(args.input), _load(args.result), run_root=args.run_root)
exit_code = 0 if output.get("status", "PASS") == "PASS" else 1
except (
SemanticAuditError,
semantic_candidate_builder.SemanticCandidateError,
semantic_surface_extractor.SemanticSurfaceError,
proof_manifest.ManifestValidationError,
OSError,
UnicodeError,
json.JSONDecodeError,
) as exc:
output = {"schema_version": VALIDATED_RESULT_SCHEMA, "status": "ERROR", "errors": [{"code": "SEMANTIC_AUDIT_ERROR", "message": str(exc)}]}
exit_code = 2
json.dump(output, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,254 @@
#!/usr/bin/env python3
"""Validate auditor assertions and deterministically construct semantic candidate pairs."""
from __future__ import annotations
import argparse
from collections import defaultdict
from itertools import combinations
import hashlib
import json
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
import semantic_surface_extractor
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_ONTOLOGY = Path("harness/source/semantic-ontology.json")
RESULT_SCHEMA = "semantic-candidate-result/v1"
ASSERTION_SCHEMA = "semantic-assertion-result/v1"
ONTOLOGY_SCHEMA = "semantic-ontology/v1"
STABLE_REF_RE = re.compile(r"\b(?:ART|DELEG|FLOW|[A-Z][A-Z0-9]*)(?:-[A-Z0-9]+)+-\d{3}(?:@[1-9]\d*)?\b")
SHARED_LITERAL_RE = re.compile(r"`([^`\n]+)`|(?<!\w)(--[a-z0-9][a-z0-9-]*)|(?:https?://[^\s)]+)|(?:\b[a-zA-Z0-9_.-]+/[a-zA-Z0-9_./{}-]+)")
class SemanticCandidateError(ValueError):
"""Assertions or ontology are malformed or cannot be grounded."""
def _canonical(value: Any) -> bytes:
return semantic_surface_extractor.canonical_json_bytes(value)
def load_ontology(root: Path, ontology_path: Path = DEFAULT_ONTOLOGY) -> Mapping[str, Any]:
source = ontology_path if ontology_path.is_absolute() else root / ontology_path
try:
data = json.loads(source.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise SemanticCandidateError(f"semantic ontology error: {exc}") from exc
required = {"schema_version", "auditor_contract_version", "predicates", "modalities", "verdicts", "blocking", "candidate_rules"}
if not isinstance(data, dict) or set(data) != required or data.get("schema_version") != ONTOLOGY_SCHEMA:
raise SemanticCandidateError(f"expected exact {ONTOLOGY_SCHEMA} schema")
expected_predicates = {
"owns", "produces", "consumes", "returns", "validates", "maps_to", "runs_before", "runs_after",
"uses", "requires", "forbids", "enforces", "delegates", "has_schema", "has_threshold",
"has_cardinality", "has_failure_behavior", "other",
}
expected_verdicts = {
"CONSISTENT", "COMPLEMENTARY", "CONTEXTUAL_VARIANT", "AMBIGUOUS_AUTHORITY",
"RESTATEMENT_DRIFT", "CONTRADICTION",
}
if set(data["predicates"]) != expected_predicates:
raise SemanticCandidateError("predicate ontology differs from the required exact set")
if set(data["verdicts"]) != expected_verdicts:
raise SemanticCandidateError("semantic verdict set differs from the required exact set")
rule_ids = [item.get("id") for item in data["candidate_rules"] if isinstance(item, dict)]
if rule_ids != [f"C{index}" for index in range(1, 8)]:
raise SemanticCandidateError("candidate rules must be exactly C1..C7 in order")
return data
def _assertion(value: Any, index: int, surfaces: Mapping[str, Mapping[str, Any]], ontology: Mapping[str, Any], root: Path) -> dict[str, Any]:
fields = {
"assertion_id", "source_surface", "subject", "predicate", "object", "condition",
"modality", "scope", "quote", "line_start", "line_end",
}
if not isinstance(value, dict) or set(value) != fields:
raise SemanticCandidateError(f"assertions[{index}] must contain exactly {sorted(fields)}")
for field in fields - {"line_start", "line_end"}:
if not isinstance(value[field], str) or not value[field].strip():
raise SemanticCandidateError(f"assertions[{index}].{field} must be non-empty")
if value["predicate"] not in ontology["predicates"]:
raise SemanticCandidateError(f"assertions[{index}].predicate is outside ontology")
if value["modality"] not in ontology["modalities"]:
raise SemanticCandidateError(f"assertions[{index}].modality is outside ontology")
if not isinstance(value["line_start"], int) or not isinstance(value["line_end"], int) or value["line_start"] < 1 or value["line_end"] < value["line_start"]:
raise SemanticCandidateError(f"assertions[{index}] has invalid line range")
surface = surfaces.get(value["source_surface"])
if surface is None:
raise SemanticCandidateError(f"assertions[{index}] references unknown source surface")
if value["line_start"] < surface["line_start"] or value["line_end"] > surface["line_end"]:
raise SemanticCandidateError(f"assertions[{index}] quote range escapes its source surface")
document = root / str(surface["path"])
lines = document.read_text(encoding="utf-8").splitlines()
observed = "\n".join(lines[value["line_start"] - 1 : value["line_end"]])
if observed != value["quote"]:
raise SemanticCandidateError(f"assertions[{index}] exact UTF-8 quote/line verification failed")
return {field: value[field] for field in sorted(fields)}
def validate_assertions(root: Path, extraction: Mapping[str, Any], assertion_result: Any, ontology: Mapping[str, Any]) -> list[dict[str, Any]]:
if not isinstance(assertion_result, dict) or set(assertion_result) != {"schema_version", "subject", "mode", "surface_manifest_sha256", "assertions"}:
raise SemanticCandidateError("assertion result has missing or unknown fields")
if assertion_result.get("schema_version") != ASSERTION_SCHEMA:
raise SemanticCandidateError(f"expected {ASSERTION_SCHEMA}")
if assertion_result.get("subject") != extraction.get("path") or assertion_result.get("mode") != extraction.get("mode"):
raise SemanticCandidateError("assertion result subject/mode does not match extraction")
manifest_hash = hashlib.sha256(_canonical(extraction)).hexdigest()
if assertion_result.get("surface_manifest_sha256") != manifest_hash:
raise SemanticCandidateError("assertion result is not bound to current surface manifest")
raw_assertions = assertion_result.get("assertions")
if not isinstance(raw_assertions, list):
raise SemanticCandidateError("assertions must be an array")
surfaces = {item["surface_id"]: item for item in extraction.get("surfaces", [])}
assertions = [_assertion(item, index, surfaces, ontology, root) for index, item in enumerate(raw_assertions)]
identifiers = [item["assertion_id"] for item in assertions]
if len(identifiers) != len(set(identifiers)):
raise SemanticCandidateError("assertion_id values must be unique")
covered = {item["source_surface"] for item in assertions}
missing = sorted(set(surfaces) - covered)
if missing:
raise SemanticCandidateError(f"auditor silently dropped surfaces without assertions: {missing}")
return sorted(assertions, key=lambda item: item["assertion_id"])
def _norm(value: str) -> str:
return re.sub(r"\s+", " ", value.strip().casefold())
def _literals(assertion: Mapping[str, Any]) -> set[str]:
text = f"{assertion['object']} {assertion['quote']}"
result: set[str] = set()
for match in SHARED_LITERAL_RE.finditer(text):
captured = next((group for group in match.groups() if group is not None), match.group(0))
result.add(_norm(captured))
return {item for item in result if len(item) >= 3}
def _pairs(values: Iterable[Mapping[str, Any]]) -> Iterable[tuple[Mapping[str, Any], Mapping[str, Any]]]:
yield from combinations(sorted(values, key=lambda item: str(item["assertion_id"])), 2)
def build(root: Path, extraction: Mapping[str, Any], assertion_result: Any, ontology_path: Path = DEFAULT_ONTOLOGY) -> dict[str, Any]:
root = root.resolve(strict=True)
ontology = load_ontology(root, ontology_path)
assertions = validate_assertions(root, extraction, assertion_result, ontology)
reasons: dict[tuple[str, str], set[str]] = defaultdict(set)
by_base: dict[tuple[str, str, str, str], list[dict[str, Any]]] = defaultdict(list)
for item in assertions:
by_base[(_norm(item["subject"]), item["predicate"], _norm(item["condition"]), _norm(item["scope"]))].append(item)
for values in by_base.values():
for left, right in _pairs(values):
reasons[(left["assertion_id"], right["assertion_id"])].add("BASE")
for left, right in _pairs(assertions):
predicates = {left["predicate"], right["predicate"]}
same_subject = _norm(left["subject"]) == _norm(right["subject"])
same_object = _norm(left["object"]) == _norm(right["object"])
same_context = _norm(left["condition"]) == _norm(right["condition"]) and _norm(left["scope"]) == _norm(right["scope"])
pair = (left["assertion_id"], right["assertion_id"])
if same_object and predicates <= {"produces", "consumes"} and predicates == {"produces", "consumes"}:
reasons[pair].add("C1")
if same_subject and "stage" in _norm(left["subject"]) and predicates <= {"owns", "consumes", "produces", "returns", "runs_before", "runs_after"}:
reasons[pair].add("C2")
if same_subject and ("gate" in _norm(left["subject"]) or "contract" in _norm(left["subject"])) and predicates <= {"requires", "enforces", "uses", "forbids"}:
reasons[pair].add("C3")
if same_subject and same_object and same_context and {left["modality"], right["modality"]} == {"must", "must_not"}:
reasons[pair].add("C4")
if same_subject and predicates <= {"returns", "validates", "maps_to"}:
reasons[pair].add("C5")
if _literals(left).intersection(_literals(right)):
reasons[pair].add("C6")
left_refs = set(STABLE_REF_RE.findall(f"{left['object']} {left['quote']}"))
right_refs = set(STABLE_REF_RE.findall(f"{right['object']} {right['quote']}"))
if left_refs.intersection(right_refs):
reasons[pair].add("C7")
by_id = {item["assertion_id"]: item for item in assertions}
candidates: list[dict[str, Any]] = []
for pair, rule_ids in sorted(reasons.items()):
left, right = (by_id[pair[0]], by_id[pair[1]])
digest = hashlib.sha256((pair[0] + "\0" + pair[1] + "\0" + ",".join(sorted(rule_ids))).encode("utf-8")).hexdigest()
candidates.append({
"candidate_id": "SEM-" + digest[:20].upper(),
"assertion_a": pair[0],
"assertion_b": pair[1],
"rule_ids": sorted(rule_ids, key=lambda item: (item != "BASE", item)),
"grouping_key": {
"subject": left["subject"] if _norm(left["subject"]) == _norm(right["subject"]) else "",
"predicate": left["predicate"] if left["predicate"] == right["predicate"] else "",
"condition": left["condition"] if _norm(left["condition"]) == _norm(right["condition"]) else "",
"scope": left["scope"] if _norm(left["scope"]) == _norm(right["scope"]) else "",
},
})
return {
"schema_version": RESULT_SCHEMA,
"status": "PASS",
"subject": extraction["path"],
"mode": extraction["mode"],
"document_sha256": extraction["document_sha256"],
"surface_manifest_sha256": hashlib.sha256(_canonical(extraction)).hexdigest(),
"ontology_sha256": hashlib.sha256(_canonical(ontology)).hexdigest(),
"assertions_sha256": hashlib.sha256(_canonical(assertions)).hexdigest(),
"coverage": {
"eligible_surfaces": extraction["coverage"]["eligible_surface_blocks"],
"processed_surfaces": len({item["source_surface"] for item in assertions}) + extraction["coverage"]["explicitly_excluded_blocks"],
"assertions": len(assertions),
"candidate_pairs": len(candidates),
},
"assertions": assertions,
"candidates": candidates,
"findings": [],
}
def structural_check(root: Path, ontology_path: Path = DEFAULT_ONTOLOGY) -> dict[str, Any]:
root = root.resolve(strict=True)
ontology = load_ontology(root, ontology_path)
surfaces = semantic_surface_extractor.check(root)
return {
"schema_version": RESULT_SCHEMA,
"status": surfaces["status"],
"check_kind": "STRUCTURE_ONLY",
"semantic_verdict": "NOT_EVALUATED",
"ontology_sha256": hashlib.sha256(_canonical(ontology)).hexdigest(),
"surface_document_count": surfaces["document_count"],
"coverage": surfaces["coverage"],
"findings": surfaces["findings"],
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--ontology", type=Path, default=DEFAULT_ONTOLOGY)
parser.add_argument("--document", type=Path)
parser.add_argument("--assertions", type=Path)
parser.add_argument("--check", action="store_true")
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
if args.document is None and args.assertions is None:
result = structural_check(root, args.ontology)
elif args.document is not None and args.assertions is not None:
policy = semantic_surface_extractor.load_policy(root)
document = args.document if args.document.is_absolute() else root / args.document
extraction = semantic_surface_extractor.extract_document(root, document, policy)
assertion_result = json.loads(args.assertions.read_text(encoding="utf-8"))
result = build(root, extraction, assertion_result, args.ontology)
else:
raise SemanticCandidateError("--document and --assertions must be supplied together")
exit_code = 0 if result["status"] == "PASS" else 1
except (SemanticCandidateError, semantic_surface_extractor.SemanticSurfaceError, OSError, UnicodeError, json.JSONDecodeError) as exc:
result = {"schema_version": RESULT_SCHEMA, "status": "ERROR", "errors": [{"code": "SEMANTIC_CANDIDATE_ERROR", "message": str(exc)}]}
exit_code = 2
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+414
View File
@@ -0,0 +1,414 @@
#!/usr/bin/env python3
"""Issue and validate byte-bound semantic certificates for design-bearing documents."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
import semantic_audit
import semantic_candidate_builder
import semantic_surface_extractor
import typed_contract_check
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_STATE = Path("harness/state/semantic-certificates")
DEFAULT_AGENT_METADATA = Path("harness/source/agents/wiki-semantic-coherence-auditor.json")
DEFAULT_AGENT_BODY = Path("harness/source/agents/bodies/wiki-semantic-coherence-auditor.md")
SCHEMA_VERSION = "semantic-certificate/v1"
RESULT_SCHEMA = "semantic-certificate-result/v1"
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
class SemanticCertificateError(ValueError):
def __init__(self, code: str, message: str, path: str = "") -> None:
self.code = code
self.path = path
super().__init__(message)
def _source_path(root: Path, value: Path) -> Path:
return value if value.is_absolute() else root / value
def _file_sha(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def _canonical_sha(value: Any) -> str:
return hashlib.sha256(semantic_audit.canonical_json_bytes(value)).hexdigest()
def document_id(relative: str) -> str:
candidate = Path(relative)
if not relative or candidate.is_absolute() or ".." in candidate.parts or "\\" in relative:
raise SemanticCertificateError("INVALID_CERTIFICATE_SUBJECT", "subject must be a canonical repo-relative POSIX path", relative)
return hashlib.sha256(relative.encode("utf-8")).hexdigest()
def certificate_path(root: Path, subject: str, document_sha256: str, state_path: Path = DEFAULT_STATE) -> Path:
if not HEX_SHA256.fullmatch(document_sha256):
raise SemanticCertificateError("INVALID_DOCUMENT_SHA256", "document sha256 is invalid", subject)
state = _source_path(root, state_path)
return state / document_id(subject) / f"{document_sha256}.json"
def logical_subject(root: Path, document: Path) -> str:
"""Return the stable pre-cutover identity for an active document path."""
relative = document.resolve().relative_to(root.resolve()).as_posix()
layout_path = root / "harness/source/vault-layout.json"
if not layout_path.is_file():
return relative
try:
layout = json.loads(layout_path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError):
return relative
if layout.get("mode") != "canonical":
return relative
migration = layout.get("migration_manifest")
entries = migration.get("entries") if isinstance(migration, Mapping) else None
if not isinstance(entries, list):
return relative
matches = [
str(item.get("legacy_path"))
for item in entries
if isinstance(item, Mapping) and item.get("canonical_path") == relative
]
if len(matches) != 1:
return relative
return matches[0]
def _policy_hashes(
root: Path,
*,
policy_path: Path = semantic_surface_extractor.DEFAULT_POLICY,
ontology_path: Path = semantic_candidate_builder.DEFAULT_ONTOLOGY,
agent_metadata_path: Path = DEFAULT_AGENT_METADATA,
agent_body_path: Path = DEFAULT_AGENT_BODY,
) -> dict[str, str]:
policy = _source_path(root, policy_path)
ontology = _source_path(root, ontology_path)
metadata = _source_path(root, agent_metadata_path)
body = _source_path(root, agent_body_path)
for source in (policy, ontology, metadata, body):
if not source.is_file():
raise SemanticCertificateError("SEMANTIC_POLICY_SOURCE_MISSING", "certificate binding source is missing", source.as_posix())
contract_payload = metadata.read_bytes() + b"\0" + body.read_bytes()
return {
"policy_sha256": _file_sha(policy),
"ontology_sha256": _file_sha(ontology),
"auditor_contract_sha256": hashlib.sha256(contract_payload).hexdigest(),
}
def _typed_hash(root: Path) -> str:
result = typed_contract_check.check(root)
if result.get("status") != "PASS":
codes = sorted({str(item.get("code", "UNKNOWN")) for item in result.get("findings", [])})
raise SemanticCertificateError("TYPED_CONTRACT_FAILED", ",".join(codes))
return str(result["typed_contract_graph_sha256"])
def build_certificate(
root: Path,
validated_audit: Mapping[str, Any],
*,
audit_request: Mapping[str, Any],
audit_result: Mapping[str, Any],
run_root: Path | None = None,
policy_path: Path = semantic_surface_extractor.DEFAULT_POLICY,
ontology_path: Path = semantic_candidate_builder.DEFAULT_ONTOLOGY,
agent_metadata_path: Path = DEFAULT_AGENT_METADATA,
agent_body_path: Path = DEFAULT_AGENT_BODY,
) -> dict[str, Any]:
root = root.resolve(strict=True)
try:
revalidated = semantic_audit.validate_result(root, audit_request, audit_result, run_root=run_root)
except semantic_audit.SemanticAuditError as exc:
raise SemanticCertificateError("INVALID_SEMANTIC_AUDIT", str(exc)) from exc
if semantic_audit.canonical_json_bytes(revalidated) != semantic_audit.canonical_json_bytes(validated_audit):
raise SemanticCertificateError("INVALID_SEMANTIC_AUDIT", "validated audit differs from request/result replay")
if validated_audit.get("schema_version") != semantic_audit.VALIDATED_RESULT_SCHEMA:
raise SemanticCertificateError("INVALID_SEMANTIC_AUDIT", "validated semantic audit schema mismatch")
subject = str(validated_audit.get("subject", ""))
doc_id = document_id(subject)
document = Path(os.path.abspath(root / subject))
try:
document.relative_to(root)
document.resolve(strict=True).relative_to(root)
except ValueError as exc:
raise SemanticCertificateError("INVALID_CERTIFICATE_SUBJECT", "subject escapes repository", subject) from exc
if not document.is_file():
raise SemanticCertificateError("CERTIFICATE_SUBJECT_MISSING", "subject document does not exist", subject)
document_sha = _file_sha(document)
if document_sha != validated_audit.get("document_sha256"):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "audit is not bound to current document bytes", subject)
coverage = validated_audit.get("coverage")
counts = validated_audit.get("counts")
if not isinstance(coverage, Mapping) or not isinstance(counts, Mapping):
raise SemanticCertificateError("INVALID_SEMANTIC_AUDIT", "audit coverage/counts are missing", subject)
if coverage.get("eligible_surfaces") != coverage.get("processed_surfaces"):
raise SemanticCertificateError("SEMANTIC_SURFACE_UNCOVERED", "audit did not process every eligible surface", subject)
if coverage.get("candidate_pairs") != coverage.get("processed_pairs"):
raise SemanticCertificateError("SEMANTIC_PAIR_UNCOVERED", "audit did not process every candidate pair", subject)
mode = validated_audit.get("mode")
if mode not in {"local", "hub"}:
raise SemanticCertificateError("INVALID_SEMANTIC_AUDIT", "audit mode is invalid", subject)
proof_hashes = sorted({str(item["proof_manifest_sha256"]) for item in validated_audit.get("findings", [])})
hashes = _policy_hashes(
root,
policy_path=policy_path,
ontology_path=ontology_path,
agent_metadata_path=agent_metadata_path,
agent_body_path=agent_body_path,
)
verdict = "PASS" if validated_audit.get("status") == "PASS" else "FAIL"
return {
"schema_version": SCHEMA_VERSION,
"subject": subject,
"document_id": doc_id,
"document_sha256": document_sha,
**hashes,
"typed_contract_graph_sha256": _typed_hash(root),
"mode": mode,
"verdict": verdict,
"coverage": {
"eligible_surfaces": int(coverage["eligible_surfaces"]),
"processed_surfaces": int(coverage["processed_surfaces"]),
"candidate_pairs": int(coverage["candidate_pairs"]),
"processed_pairs": int(coverage["processed_pairs"]),
"dropped_pairs": int(coverage["dropped_pairs"]),
},
"findings": {
"blocking": int(counts["blocking"]),
"readiness_blocking": int(counts["readiness_blocking"]),
"verified": int(counts["verified_findings"]),
},
"proof_manifest_sha256": _canonical_sha(proof_hashes),
"audit_request_sha256": str(validated_audit["request_sha256"]),
"semantic_audit_sha256": _canonical_sha(validated_audit),
"audit_request": dict(audit_request),
"audit_result": dict(audit_result),
"auditor": dict(validated_audit["auditor"]),
}
def prepare_certificate(
root: Path,
validated_audit: Mapping[str, Any],
*,
state_path: Path = DEFAULT_STATE,
**kwargs: Any,
) -> tuple[Path, bytes, dict[str, Any]]:
certificate = build_certificate(root, validated_audit, **kwargs)
path = certificate_path(root, certificate["subject"], certificate["document_sha256"], state_path)
return path, semantic_audit.canonical_json_bytes(certificate), certificate
def _load_certificate(path: Path) -> Mapping[str, Any]:
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise SemanticCertificateError("INVALID_SEMANTIC_CERTIFICATE", str(exc), path.as_posix()) from exc
required = {
"schema_version", "subject", "document_id", "document_sha256", "policy_sha256", "ontology_sha256",
"auditor_contract_sha256", "typed_contract_graph_sha256", "mode", "verdict", "coverage", "findings",
"proof_manifest_sha256", "audit_request_sha256", "semantic_audit_sha256", "audit_request", "audit_result",
"auditor",
}
if not isinstance(value, dict) or set(value) != required or value.get("schema_version") != SCHEMA_VERSION:
raise SemanticCertificateError("INVALID_SEMANTIC_CERTIFICATE", "certificate has missing or unknown fields", path.as_posix())
return value
def validate_certificate(
root: Path,
path: Path,
*,
state_path: Path = DEFAULT_STATE,
policy_path: Path = semantic_surface_extractor.DEFAULT_POLICY,
ontology_path: Path = semantic_candidate_builder.DEFAULT_ONTOLOGY,
agent_metadata_path: Path = DEFAULT_AGENT_METADATA,
agent_body_path: Path = DEFAULT_AGENT_BODY,
) -> dict[str, Any]:
root = root.resolve(strict=True)
path = path.resolve(strict=True)
certificate = _load_certificate(path)
subject = str(certificate["subject"])
expected_path = certificate_path(root, subject, str(certificate["document_sha256"]), state_path).resolve()
if path != expected_path:
raise SemanticCertificateError("INVALID_CERTIFICATE_PATH", "certificate path does not match document-id/document sha", path.as_posix())
if certificate["document_id"] != document_id(subject):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "document id mismatch", subject)
audit_request = certificate.get("audit_request")
audit_result = certificate.get("audit_result")
if not isinstance(audit_request, Mapping) or not isinstance(audit_result, Mapping):
raise SemanticCertificateError("INVALID_SEMANTIC_CERTIFICATE", "embedded audit artifacts must be objects", subject)
try:
replayed = semantic_audit.validate_result(root, audit_request, audit_result)
except semantic_audit.SemanticAuditError as exc:
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", f"embedded audit replay failed: {exc}", subject) from exc
if certificate.get("audit_request_sha256") != _canonical_sha(audit_request):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "audit request hash mismatch", subject)
if certificate.get("semantic_audit_sha256") != _canonical_sha(replayed):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "semantic audit hash mismatch", subject)
replayed_hashes = sorted({str(item["proof_manifest_sha256"]) for item in replayed.get("findings", [])})
if certificate.get("proof_manifest_sha256") != _canonical_sha(replayed_hashes):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "proof manifest binding changed", subject)
document = Path(os.path.abspath(root / subject))
try:
document.relative_to(root)
document.resolve(strict=True).relative_to(root)
except (ValueError, FileNotFoundError) as exc:
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "document subject is missing or escapes repository", subject) from exc
if not document.is_file() or _file_sha(document) != certificate["document_sha256"]:
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "document bytes changed", subject)
hashes = _policy_hashes(
root,
policy_path=policy_path,
ontology_path=ontology_path,
agent_metadata_path=agent_metadata_path,
agent_body_path=agent_body_path,
)
for key, expected in hashes.items():
if certificate.get(key) != expected:
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", f"{key} changed", subject)
if certificate.get("typed_contract_graph_sha256") != _typed_hash(root):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "typed contract graph changed", subject)
expected_verdict = "PASS" if replayed.get("status") == "PASS" else "FAIL"
if certificate.get("verdict") != expected_verdict:
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "audit verdict changed", subject)
if certificate.get("auditor") != replayed.get("auditor"):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "auditor identity changed", subject)
if certificate.get("mode") != replayed.get("mode") or certificate.get("document_sha256") != replayed.get("document_sha256"):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "audit subject binding changed", subject)
policy = semantic_surface_extractor.load_policy(root, policy_path)
extraction = semantic_surface_extractor.extract_document(root, document, policy)
if extraction["mode"] != certificate.get("mode"):
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "semantic mode changed", subject)
coverage = certificate.get("coverage")
findings = certificate.get("findings")
if not isinstance(coverage, Mapping) or not isinstance(findings, Mapping):
raise SemanticCertificateError("INVALID_SEMANTIC_CERTIFICATE", "coverage/findings must be objects", subject)
if coverage.get("eligible_surfaces") != extraction["coverage"]["eligible_surface_blocks"]:
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "eligible surface count changed", subject)
replayed_coverage = replayed.get("coverage", {})
replayed_counts = replayed.get("counts", {})
expected_coverage = {
"eligible_surfaces": replayed_coverage.get("eligible_surfaces"),
"processed_surfaces": replayed_coverage.get("processed_surfaces"),
"candidate_pairs": replayed_coverage.get("candidate_pairs"),
"processed_pairs": replayed_coverage.get("processed_pairs"),
"dropped_pairs": replayed_coverage.get("dropped_pairs"),
}
expected_findings = {
"blocking": replayed_counts.get("blocking"),
"readiness_blocking": replayed_counts.get("readiness_blocking"),
"verified": replayed_counts.get("verified_findings"),
}
if dict(coverage) != expected_coverage or dict(findings) != expected_findings:
raise SemanticCertificateError("SEMANTIC_CERTIFICATE_STALE", "audit coverage or finding counts changed", subject)
if coverage.get("eligible_surfaces") != coverage.get("processed_surfaces"):
raise SemanticCertificateError("SEMANTIC_SURFACE_UNCOVERED", "certificate surface coverage is incomplete", subject)
if coverage.get("candidate_pairs") != coverage.get("processed_pairs"):
raise SemanticCertificateError("SEMANTIC_PAIR_UNCOVERED", "certificate pair coverage is incomplete", subject)
if certificate.get("verdict") != "PASS" or findings.get("blocking") != 0 or findings.get("readiness_blocking") != 0:
raise SemanticCertificateError("SEMANTIC_BLOCKING_VERDICT", "certificate contains a blocking semantic result", subject)
if certificate["mode"] == "hub" and coverage.get("dropped_pairs") != 0:
raise SemanticCertificateError("SEMANTIC_PAIR_DROPPED", "hub certificate contains dropped candidates", subject)
return dict(certificate)
def check(
root: Path,
*,
mode: str | None = None,
paths: Iterable[Path] | None = None,
state_path: Path = DEFAULT_STATE,
) -> dict[str, Any]:
root = root.resolve(strict=True)
policy = semantic_surface_extractor.load_policy(root)
selected = tuple(paths) if paths is not None else semantic_surface_extractor.eligible_documents(root, policy, required_only=True, mode=mode)
findings: list[dict[str, Any]] = []
current: list[dict[str, Any]] = []
# 자동 탐색이 0건이면 "요구 문서가 없어 전부 최신" 처럼 PASS 로 보이지만, 실제로는
# 탐색이 깨져 인증 검사를 *조용히 건너뛴* 상태일 수 있다. 명시적 paths 는 호출자 책임이나,
# 자동 탐색의 0건은 loud FAIL. 단 mode 필터(local/hub)의 0건은 "그 모드 문서가 없을 뿐"
# 이라 정상일 수 있으므로(예: local 만 있고 hub 는 없음), 필터 없는 전수 탐색에만 적용한다.
if paths is None and mode is None and not selected:
findings.append({
"code": "NO_REQUIRED_DOCUMENTS",
"path": "",
"line": 0,
"message": "자동 탐색이 인증 대상(설계-보유) 문서를 0건 발견 — 탐색이 깨졌을 수 있음.",
})
for raw in selected:
document = raw if raw.is_absolute() else root / raw
extraction = semantic_surface_extractor.extract_document(root, document, policy)
if mode is not None and extraction["mode"] != mode:
continue
subject = logical_subject(root, document)
path = certificate_path(root, subject, extraction["document_sha256"], state_path)
if not path.is_file():
findings.append({
"code": "SEMANTIC_CERTIFICATE_MISSING",
"path": extraction["path"],
"line": 0,
"message": f"current {extraction['mode']} certificate is missing",
})
continue
try:
certificate = validate_certificate(root, path, state_path=state_path)
current.append({"subject": certificate["subject"], "mode": certificate["mode"], "path": path.relative_to(root).as_posix()})
except SemanticCertificateError as exc:
findings.append({"code": exc.code, "path": exc.path or extraction["path"], "line": 0, "message": str(exc)})
return {
"schema_version": RESULT_SCHEMA,
"status": "PASS" if not findings else "FAIL",
"mode": mode or "all",
"required_documents": len(selected),
"current_certificates": len(current),
"coverage": {
"required": len(selected),
"current": len(current),
"missing_or_stale": len(findings),
},
"certificates": current,
"findings": sorted(findings, key=lambda item: (item["path"], item["code"])),
}
def quality_extension(root: Path, paths: Iterable[Path]) -> dict[str, Any]:
"""QualityExtension-compatible current-certificate validation."""
return check(root, paths=paths)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--mode", choices=("local", "hub"))
parser.add_argument("--path", type=Path, action="append")
parser.add_argument("--check", action="store_true")
args = parser.parse_args(argv)
try:
result = check(args.root, mode=args.mode, paths=args.path)
exit_code = 0 if result["status"] == "PASS" else 1
except (SemanticCertificateError, semantic_surface_extractor.SemanticSurfaceError, OSError, UnicodeError, json.JSONDecodeError) as exc:
result = {"schema_version": RESULT_SCHEMA, "status": "ERROR", "errors": [{"code": getattr(exc, "code", "SEMANTIC_CERTIFICATE_ERROR"), "message": str(exc)}]}
exit_code = 2
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+479
View File
@@ -0,0 +1,479 @@
#!/usr/bin/env python3
"""Evaluate the 70-case typed and semantic consistency regression corpus.
The deterministic half is executed against the real typed-contract checker.
The semantic half remains release-blocking until three truthful live auditor
runs are recorded; fixture completeness is never reported as an LLM result.
"""
from __future__ import annotations
import argparse
from collections import Counter
from datetime import datetime
import hashlib
import json
from pathlib import Path
import re
import shutil
import sys
import tempfile
from typing import Any, Iterable, Mapping
import contract_projection
import semantic_candidate_builder
import semantic_surface_extractor
import typed_contract_check
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_MANIFEST = Path("harness/tests/fixtures/semantic-consistency/manifest.json")
RESULT_SCHEMA = "semantic-regression-result/v1"
MANIFEST_SCHEMA = "semantic-consistency-corpus/v1"
FIXTURE_SCHEMA = "semantic-regression-fixture/v1"
RUN_SCHEMA = "semantic-evaluation-run/v1"
TYPES = ("A4", "E1", "DELEG", "D7", "A1", "HUB")
DETERMINISTIC_TYPES = frozenset({"A4", "E1", "DELEG", "D7"})
SEMANTIC_TYPES = frozenset({"A1", "HUB"})
EXPECTED_CODES = {
"A4": "MANUAL_ARTIFACT_SCHEMA_RESTATEMENT",
"E1": "DUPLICATE_CONCERN_OWNER",
"DELEG": "UNACCEPTED_DELEGATION",
"D7": "GENERATED_CONTRACT_PROJECTION_DRIFT",
"A1": "CONTRADICTION",
"HUB": "CONTRADICTION",
}
CASE_ID_RE = re.compile(r"^(A4|E1|DELEG|D7|A1|HUB)-\d{3}$")
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
AUDITOR_PROMPT = Path("harness/source/agents/bodies/wiki-semantic-coherence-auditor.md")
class SemanticRegressionError(ValueError):
"""The corpus schema, path set, or evaluation record is unusable."""
def _safe_path(root: Path, value: Any, location: str) -> Path:
if not isinstance(value, str) or not value or "\\" in value:
raise SemanticRegressionError(f"{location}: expected repo-relative POSIX path")
relative = Path(value)
if relative.is_absolute() or ".." in relative.parts:
raise SemanticRegressionError(f"{location}: path escapes repository")
path = (root / relative).resolve()
try:
path.relative_to(root)
except ValueError as exc:
raise SemanticRegressionError(f"{location}: path escapes repository") from exc
if not path.is_file():
raise SemanticRegressionError(f"{location}: file does not exist: {value}")
return path
def _load_json(path: Path, schema: str) -> dict[str, Any]:
try:
document = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise SemanticRegressionError(f"cannot read {path}: {exc}") from exc
if not isinstance(document, dict) or document.get("schema_version") != schema:
raise SemanticRegressionError(f"{path}: expected {schema}")
return document
def load_manifest(root: Path, manifest_path: Path = DEFAULT_MANIFEST) -> dict[str, Any]:
root = root.resolve(strict=True)
source = manifest_path if manifest_path.is_absolute() else root / manifest_path
manifest = _load_json(source, MANIFEST_SCHEMA)
cases = manifest.get("cases")
if manifest.get("case_count") != 70 or not isinstance(cases, list) or len(cases) != 70:
raise SemanticRegressionError("manifest must declare exactly 70 cases")
thresholds = manifest.get("thresholds")
expected_thresholds = {
"deterministic_recall": 1.0,
"deterministic_false_negatives": 0,
"deterministic_negative_false_positives": 0,
"semantic_critical_high_recall": 1.0,
"semantic_overall_recall": 0.95,
"semantic_precision": 0.90,
"semantic_dropped_pairs": 0,
"live_runs": 3,
}
if thresholds != expected_thresholds:
raise SemanticRegressionError("manifest thresholds do not match the WP17 release contract")
seen: set[str] = set()
distribution: Counter[str] = Counter()
required = {
"case_id",
"type",
"severity",
"documents",
"surface_a",
"surface_b",
"expected",
"counterexample",
"provenance",
}
for index, case in enumerate(cases):
if not isinstance(case, dict) or set(case) != required:
raise SemanticRegressionError(f"cases[{index}] must contain exactly {sorted(required)}")
case_id = case["case_id"]
case_type = case["type"]
if not isinstance(case_id, str) or not CASE_ID_RE.fullmatch(case_id):
raise SemanticRegressionError(f"cases[{index}].case_id is invalid")
if case_id in seen:
raise SemanticRegressionError(f"duplicate case id: {case_id}")
seen.add(case_id)
if case_type not in TYPES or not case_id.startswith(f"{case_type}-"):
raise SemanticRegressionError(f"{case_id}: type does not match case id")
distribution[case_type] += 1
if case.get("severity") not in {"Critical", "High", "Medium", "Low"}:
raise SemanticRegressionError(f"{case_id}: invalid severity")
if case.get("expected") != EXPECTED_CODES[case_type]:
raise SemanticRegressionError(f"{case_id}: unexpected expected code")
if case.get("provenance") != "design-fixture":
raise SemanticRegressionError(f"{case_id}: provenance must be design-fixture")
documents = case.get("documents")
if not isinstance(documents, list) or len(documents) != 1:
raise SemanticRegressionError(f"{case_id}: documents must name one positive fixture")
for field in ("surface_a", "surface_b"):
surface = case.get(field)
if (
not isinstance(surface, dict)
or set(surface) != {"section", "quote"}
or not all(isinstance(surface[key], str) and surface[key] for key in surface)
):
raise SemanticRegressionError(f"{case_id}: invalid {field}")
for path_index, value in enumerate(documents):
_safe_path(root, value, f"{case_id}.documents[{path_index}]")
_safe_path(root, case["counterexample"], f"{case_id}.counterexample")
if set(distribution) != set(TYPES) or any(distribution[item] == 0 for item in TYPES):
raise SemanticRegressionError("all six case types must be represented")
declared_distribution = manifest.get("type_distribution")
if declared_distribution != {name: distribution[name] for name in TYPES}:
raise SemanticRegressionError("type_distribution does not match cases")
runs = manifest.get("evaluation_runs")
if not isinstance(runs, list) or len(runs) != 3:
raise SemanticRegressionError("evaluation_runs must name exactly three records")
for index, value in enumerate(runs):
_safe_path(root, value, f"evaluation_runs[{index}]")
return manifest
def _fixture(root: Path, value: str, case_id: str, polarity: str) -> dict[str, Any]:
path = _safe_path(root, value, f"{case_id}.{polarity}")
fixture = _load_json(path, FIXTURE_SCHEMA)
if fixture.get("case_id") != case_id or fixture.get("polarity") != polarity:
raise SemanticRegressionError(f"{path}: case_id/polarity mismatch")
files = fixture.get("files")
if not isinstance(files, dict) or not files:
raise SemanticRegressionError(f"{path}: files must be a non-empty object")
for relative, content in files.items():
if not isinstance(content, str):
raise SemanticRegressionError(f"{path}: fixture contents must be strings")
candidate = Path(relative)
if candidate.is_absolute() or ".." in candidate.parts or "\\" in relative:
raise SemanticRegressionError(f"{path}: unsafe fixture path {relative}")
if not isinstance(fixture.get("materialize_projections", False), bool):
raise SemanticRegressionError(f"{path}: materialize_projections must be boolean")
mutations = fixture.get("mutations", [])
if not isinstance(mutations, list):
raise SemanticRegressionError(f"{path}: mutations must be an array")
for mutation in mutations:
if not isinstance(mutation, dict) or set(mutation) != {"path", "old", "new"}:
raise SemanticRegressionError(f"{path}: invalid mutation")
if not all(isinstance(mutation[key], str) for key in mutation):
raise SemanticRegressionError(f"{path}: mutation values must be strings")
return fixture
def _materialize(root: Path, fixture: Mapping[str, Any]) -> Path:
stage = Path(tempfile.mkdtemp(prefix="semantic-regression-"))
for relative, content in fixture["files"].items():
path = stage / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
schema = stage / typed_contract_check.DEFAULT_SCHEMA
schema.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(root / typed_contract_check.DEFAULT_SCHEMA, schema)
if fixture.get("materialize_projections"):
updates, result = contract_projection.build_updates(stage)
if result["status"] == "FAIL":
raise SemanticRegressionError(
"fixture projection precondition failed: "
+ ",".join(sorted({item["code"] for item in result["findings"]}))
)
for path, text in updates.items():
path.write_text(text, encoding="utf-8")
for mutation in fixture.get("mutations", []):
path = stage / mutation["path"]
if not path.is_file():
raise SemanticRegressionError(f"mutation target does not exist: {mutation['path']}")
text = path.read_text(encoding="utf-8")
if mutation["old"] not in text:
raise SemanticRegressionError(f"mutation source not found: {mutation['old']!r}")
path.write_text(text.replace(mutation["old"], mutation["new"], 1), encoding="utf-8")
return stage
def evaluate_deterministic(root: Path, cases: Iterable[Mapping[str, Any]]) -> dict[str, Any]:
by_type: dict[str, dict[str, int | float]] = {}
findings: list[dict[str, Any]] = []
total_positive = true_positive = false_negative = negative_fp = 0
for case in cases:
if case["type"] not in DETERMINISTIC_TYPES:
continue
case_id = str(case["case_id"])
positive = _fixture(root, case["documents"][0], case_id, "positive")
negative = _fixture(root, case["counterexample"], case_id, "negative")
metrics = by_type.setdefault(case["type"], {"cases": 0, "true_positive": 0, "false_negative": 0, "negative_false_positive": 0})
metrics["cases"] += 1
total_positive += 1
positive_stage = _materialize(root, positive)
negative_stage = _materialize(root, negative)
try:
positive_codes = {item["code"] for item in typed_contract_check.check(positive_stage)["findings"]}
negative_result = typed_contract_check.check(negative_stage)
finally:
shutil.rmtree(positive_stage)
shutil.rmtree(negative_stage)
if case["expected"] in positive_codes:
true_positive += 1
metrics["true_positive"] += 1
else:
false_negative += 1
metrics["false_negative"] += 1
findings.append({
"code": "DETERMINISTIC_FALSE_NEGATIVE",
"case_id": case_id,
"message": f"expected {case['expected']}, observed {sorted(positive_codes)}",
})
if negative_result["status"] != "PASS":
negative_fp += 1
metrics["negative_false_positive"] += 1
findings.append({
"code": "DETERMINISTIC_FALSE_POSITIVE",
"case_id": case_id,
"message": f"counterexample findings: {[item['code'] for item in negative_result['findings']]}",
})
for metrics in by_type.values():
count = int(metrics["cases"])
metrics["recall"] = int(metrics["true_positive"]) / count if count else 0.0
recall = true_positive / total_positive if total_positive else 0.0
status = "PASS" if recall == 1.0 and false_negative == 0 and negative_fp == 0 else "FAIL"
return {
"status": status,
"case_count": total_positive,
"metrics": {
"recall": recall,
"true_positive": true_positive,
"false_negative": false_negative,
"negative_false_positive": negative_fp,
},
"by_type": by_type,
"findings": findings,
}
def validate_semantic_corpus(root: Path, cases: Iterable[Mapping[str, Any]]) -> dict[str, Any]:
findings: list[dict[str, Any]] = []
count = 0
high_critical = 0
for case in cases:
if case["type"] not in SEMANTIC_TYPES:
continue
count += 1
high_critical += case["severity"] in {"Critical", "High"}
positive = _fixture(root, case["documents"][0], case["case_id"], "positive")
negative = _fixture(root, case["counterexample"], case["case_id"], "negative")
positive_text = "\n".join(positive["files"].values())
negative_text = "\n".join(negative["files"].values())
for name, surface in (("surface_a", case["surface_a"]), ("surface_b", case["surface_b"])):
if surface["quote"] not in positive_text:
findings.append({
"code": "SEMANTIC_PAIR_DROPPED",
"case_id": case["case_id"],
"message": f"{name} quote is absent from positive fixture",
})
if case["surface_a"]["quote"] not in negative_text or case["surface_b"]["quote"] in negative_text:
# The counterexample must retain the authority claim but replace
# the contradictory claim with a compatible variant.
findings.append({
"code": "INVALID_SEMANTIC_COUNTEREXAMPLE",
"case_id": case["case_id"],
"message": "counterexample does not preserve A while replacing B",
})
return {
"status": "READY" if not findings else "FAIL",
"case_count": count,
"critical_high_count": high_critical,
"dropped_pairs": sum(item["code"] == "SEMANTIC_PAIR_DROPPED" for item in findings),
"findings": findings,
}
def _median(values: list[float]) -> float:
ordered = sorted(values)
return ordered[len(ordered) // 2]
def evaluate_live_runs(
semantic_cases: Iterable[Mapping[str, Any]],
runs: Iterable[Mapping[str, Any]],
) -> dict[str, Any]:
cases = list(semantic_cases)
run_list = list(runs)
not_run = [run for run in run_list if run.get("status") == "NOT_RUN"]
if not_run:
return {
"status": "NOT_RUN",
"required_runs": 3,
"completed_runs": len(run_list) - len(not_run),
"metrics": None,
"findings": [{
"code": "LIVE_EVALUATION_NOT_RUN",
"run_id": str(run.get("run_id", "")),
"message": str(run.get("reason", "live semantic evaluation has not run")),
} for run in not_run],
}
if len(run_list) != 3:
raise SemanticRegressionError("live evaluation requires exactly three runs")
case_map = {case["case_id"]: case for case in cases}
per_run: list[dict[str, Any]] = []
findings: list[dict[str, Any]] = []
for run in run_list:
if run.get("status") != "COMPLETED":
raise SemanticRegressionError(f"unsupported evaluation status: {run.get('status')}")
for field in ("model_id", "auditor_contract_version", "ontology_sha256", "prompt_sha256"):
if not isinstance(run.get(field), str) or not run[field]:
raise SemanticRegressionError(f"{run.get('run_id')}: {field} is required")
if not HEX_SHA256.fullmatch(str(run["ontology_sha256"])) or not HEX_SHA256.fullmatch(str(run["prompt_sha256"])):
raise SemanticRegressionError(f"{run.get('run_id')}: ontology/prompt sha256 is invalid")
executed_at = run.get("executed_at")
if not isinstance(executed_at, str) or not executed_at:
raise SemanticRegressionError(f"{run.get('run_id')}: executed_at is required")
try:
datetime.fromisoformat(executed_at)
except ValueError as exc:
raise SemanticRegressionError(f"{run.get('run_id')}: executed_at is not ISO-8601") from exc
predictions = run.get("predictions")
if not isinstance(predictions, list):
raise SemanticRegressionError(f"{run.get('run_id')}: predictions must be an array")
by_case = {item.get("case_id"): item for item in predictions if isinstance(item, dict)}
if set(by_case) != set(case_map) or len(predictions) != len(case_map):
raise SemanticRegressionError(f"{run.get('run_id')}: predictions must cover every semantic case exactly once")
tp = fn = fp = dropped = critical_high_tp = critical_high_total = 0
for case_id, case in case_map.items():
item = by_case[case_id]
if set(item) != {"case_id", "positive", "counterexample", "dropped"}:
raise SemanticRegressionError(f"{run.get('run_id')}/{case_id}: invalid prediction fields")
if item["dropped"]:
dropped += 1
positive_hit = item["positive"] == case["expected"]
tp += positive_hit
fn += not positive_hit
fp += item["counterexample"] not in {None, "CONSISTENT", "CONTEXTUAL_VARIANT", "COMPLEMENTARY"}
if case["severity"] in {"Critical", "High"}:
critical_high_total += 1
critical_high_tp += positive_hit
recall = tp / len(case_map) if case_map else 0.0
critical_high_recall = critical_high_tp / critical_high_total if critical_high_total else 0.0
precision = tp / (tp + fp) if tp + fp else 0.0
per_run.append({
"run_id": run["run_id"],
"recall": recall,
"critical_high_recall": critical_high_recall,
"precision": precision,
"dropped_pairs": dropped,
"true_positive": tp,
"false_negative": fn,
"false_positive": fp,
})
if critical_high_recall != 1.0:
findings.append({"code": "SEMANTIC_CRITICAL_HIGH_RECALL_FAILED", "run_id": run["run_id"], "message": str(critical_high_recall)})
if dropped:
findings.append({"code": "SEMANTIC_PAIR_DROPPED", "run_id": run["run_id"], "message": str(dropped)})
median_recall = _median([item["recall"] for item in per_run])
median_precision = _median([item["precision"] for item in per_run])
if median_recall < 0.95:
findings.append({"code": "SEMANTIC_RECALL_BELOW_THRESHOLD", "message": str(median_recall)})
if median_precision < 0.90:
findings.append({"code": "SEMANTIC_PRECISION_BELOW_THRESHOLD", "message": str(median_precision)})
return {
"status": "PASS" if not findings else "FAIL",
"required_runs": 3,
"completed_runs": 3,
"metrics": {
"median_overall_recall": median_recall,
"median_precision": median_precision,
"runs": per_run,
},
"findings": findings,
}
def check(root: Path, manifest_path: Path = DEFAULT_MANIFEST) -> dict[str, Any]:
root = root.resolve(strict=True)
manifest = load_manifest(root, manifest_path)
cases = manifest["cases"]
deterministic = evaluate_deterministic(root, cases)
semantic_corpus = validate_semantic_corpus(root, cases)
runs = [
_load_json(_safe_path(root, value, f"evaluation_runs[{index}]"), RUN_SCHEMA)
for index, value in enumerate(manifest["evaluation_runs"])
]
ontology = semantic_candidate_builder.load_ontology(root)
expected_ontology_sha = hashlib.sha256(
semantic_surface_extractor.canonical_json_bytes(ontology)
).hexdigest()
expected_prompt_sha = hashlib.sha256((root / AUDITOR_PROMPT).read_bytes()).hexdigest()
for run in runs:
if run.get("status") == "COMPLETED" and (
run.get("auditor_contract_version") != ontology["auditor_contract_version"]
or run.get("ontology_sha256") != expected_ontology_sha
or run.get("prompt_sha256") != expected_prompt_sha
):
raise SemanticRegressionError(
f"{run.get('run_id')}: live evaluation contract, ontology, or prompt is stale"
)
live = evaluate_live_runs((case for case in cases if case["type"] in SEMANTIC_TYPES), runs)
findings = [*deterministic["findings"], *semantic_corpus["findings"], *live["findings"]]
status = (
"PASS"
if deterministic["status"] == "PASS"
and semantic_corpus["status"] == "READY"
and live["status"] == "PASS"
else "FAIL"
)
return {
"schema_version": RESULT_SCHEMA,
"status": status,
"case_count": len(cases),
"type_distribution": manifest["type_distribution"],
"deterministic": deterministic,
"semantic_corpus": semantic_corpus,
"live_evaluation": live,
"findings": findings,
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--manifest", type=Path, default=DEFAULT_MANIFEST)
parser.add_argument("--check", action="store_true", required=True)
args = parser.parse_args(argv)
try:
result = check(args.root, args.manifest)
exit_code = 0 if result["status"] == "PASS" else 1
except (SemanticRegressionError, typed_contract_check.TypedContractError, OSError, UnicodeError, json.JSONDecodeError) as exc:
result = {
"schema_version": RESULT_SCHEMA,
"status": "ERROR",
"errors": [{"code": "SEMANTIC_REGRESSION_ERROR", "message": str(exc)}],
}
exit_code = 2
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,417 @@
#!/usr/bin/env python3
"""Extract policy-declared semantic surfaces without performing semantic judgment."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
import hashlib
import json
import os
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
from contract_markdown import as_list, parse_frontmatter
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_POLICY = Path("harness/source/document-semantic-surfaces.json")
RESULT_SCHEMA = "semantic-surface-extractor-result/v1"
POLICY_SCHEMA = "document-semantic-surfaces/v1"
SECTION_MARKER_RE = re.compile(r"^\s*<!--\s*section-id:\s*([a-z0-9][a-z0-9-]*)\s*-->\s*$")
HEADING_RE = re.compile(r"^(#{1,6})\s+(.+?)\s*$")
class SemanticSurfaceError(ValueError):
"""The policy or an input document cannot be interpreted safely."""
@dataclass(frozen=True)
class Section:
section_id: str | None
heading: str
level: int
line_start: int
line_end: int
text: str
def canonical_json_bytes(value: Any) -> bytes:
return (json.dumps(value, ensure_ascii=False, separators=(",", ":"), sort_keys=True) + "\n").encode("utf-8")
def _safe_rel(value: Any, location: str) -> str:
if not isinstance(value, str) or not value or "\\" in value:
raise SemanticSurfaceError(f"{location}: expected a non-empty repo-relative POSIX path")
path = Path(value)
if path.is_absolute() or ".." in path.parts:
raise SemanticSurfaceError(f"{location}: path escapes repository")
return path.as_posix()
def load_policy(root: Path, policy_path: Path = DEFAULT_POLICY) -> Mapping[str, Any]:
source = policy_path if policy_path.is_absolute() else root / policy_path
try:
data = json.loads(source.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise SemanticSurfaceError(f"surface policy error: {exc}") from exc
if not isinstance(data, dict) or data.get("schema_version") != POLICY_SCHEMA:
raise SemanticSurfaceError(f"expected {POLICY_SCHEMA}")
if not isinstance(data.get("required_statuses"), list) or not all(isinstance(x, str) for x in data["required_statuses"]):
raise SemanticSurfaceError("required_statuses must be a string list")
if not isinstance(data.get("always_required_source_types"), list) or not all(
isinstance(x, str) for x in data["always_required_source_types"]
):
raise SemanticSurfaceError("always_required_source_types must be a string list")
documents = data.get("documents")
if not isinstance(documents, dict) or not documents:
raise SemanticSurfaceError("documents must be a non-empty object")
for source_type, config in documents.items():
if not isinstance(source_type, str) or not isinstance(config, dict):
raise SemanticSurfaceError("invalid document policy")
if config.get("mode") not in {"local", "hub"}:
raise SemanticSurfaceError(f"documents.{source_type}.mode must be local or hub")
roots = config.get("roots")
surfaces = config.get("required_surfaces")
if not isinstance(roots, list) or not roots:
raise SemanticSurfaceError(f"documents.{source_type}.roots must be non-empty")
for index, value in enumerate(roots):
_safe_rel(value, f"documents.{source_type}.roots[{index}]")
if not isinstance(surfaces, list) or not surfaces:
raise SemanticSurfaceError(f"documents.{source_type}.required_surfaces must be non-empty")
ids: set[str] = set()
for index, surface in enumerate(surfaces):
if not isinstance(surface, dict) or set(surface) not in (
{"section_id", "legacy_heading"},
{"section_id", "section_aliases", "legacy_heading"},
):
raise SemanticSurfaceError(f"documents.{source_type}.required_surfaces[{index}] has invalid fields")
section_id = surface.get("section_id")
pattern = surface.get("legacy_heading")
if not isinstance(section_id, str) or not re.fullmatch(r"[a-z0-9][a-z0-9-]*", section_id):
raise SemanticSurfaceError(f"invalid semantic section id: {section_id!r}")
if section_id in ids:
raise SemanticSurfaceError(f"duplicate semantic section id: {section_id}")
ids.add(section_id)
aliases = surface.get("section_aliases", [])
if not isinstance(aliases, list) or any(
not isinstance(alias, str) or not re.fullmatch(r"[a-z0-9][a-z0-9-]*", alias)
for alias in aliases
):
raise SemanticSurfaceError(f"invalid section_aliases for {section_id}")
try:
re.compile(str(pattern), re.IGNORECASE)
except re.error as exc:
raise SemanticSurfaceError(f"invalid legacy heading regex for {section_id}: {exc}") from exc
return data
def _sections(text: str) -> tuple[Section, ...]:
lines = text.splitlines()
headings: list[tuple[int, int, str, str | None]] = []
pending_id: str | None = None
for line_number, line in enumerate(lines, 1):
marker = SECTION_MARKER_RE.fullmatch(line)
if marker:
pending_id = marker.group(1)
continue
heading = HEADING_RE.match(line)
if heading:
headings.append((line_number, len(heading.group(1)), heading.group(2).strip(), pending_id))
pending_id = None
elif line.strip() and not line.lstrip().startswith("<!--"):
pending_id = None
result: list[Section] = []
for index, (start, level, heading, section_id) in enumerate(headings):
end = len(lines)
for next_start, next_level, _next_heading, _next_id in headings[index + 1 :]:
if next_level <= level:
end = next_start - 1
break
result.append(Section(section_id, heading, level, start, end, "\n".join(lines[start - 1 : end])))
return tuple(result)
def _exclusions(frontmatter: Mapping[str, object], field: str) -> tuple[dict[str, str], list[dict[str, Any]]]:
values: dict[str, str] = {}
findings: list[dict[str, Any]] = []
for raw in as_list(frontmatter.get(field)):
section_id, separator, reason = raw.partition("|")
section_id = section_id.strip()
reason = reason.strip()
if not separator or not section_id or not reason:
findings.append({
"code": "INVALID_SEMANTIC_SURFACE_EXCLUSION",
"path": "",
"line": 0,
"message": f"{field} entries must be '<section-id>|<reason>': {raw!r}",
"severity": "error",
})
continue
if section_id in values:
findings.append({
"code": "DUPLICATE_SEMANTIC_SURFACE_EXCLUSION",
"path": "",
"line": 0,
"message": f"duplicate exclusion: {section_id}",
"severity": "error",
})
continue
values[section_id] = reason
return values, findings
def is_required(frontmatter: Mapping[str, object], policy: Mapping[str, Any]) -> bool:
if str(frontmatter.get("source_type", "")) in set(policy.get("always_required_source_types", [])):
return True
explicit_field = str(policy.get("explicit_gate_field", "semantic_gate"))
explicit = str(frontmatter.get(explicit_field, "")).strip().casefold()
if explicit in {"required", "true", "yes"}:
return True
return str(frontmatter.get("status", "")) in set(policy["required_statuses"])
def _layout(root: Path) -> tuple[str, str]:
path = root / "harness/source/vault-layout.json"
if not path.is_file():
return "compatibility", "vault"
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise SemanticSurfaceError(f"vault layout source error: {exc}") from exc
mode = value.get("mode") if isinstance(value, dict) else None
vault_root = value.get("vault_root", "vault") if isinstance(value, dict) else "vault"
if mode not in {"compatibility", "shadow", "canonical"} or not isinstance(vault_root, str) or not vault_root:
raise SemanticSurfaceError("invalid vault layout semantic authority")
return mode, vault_root
def document_policy(
relative: str,
frontmatter: Mapping[str, object],
policy: Mapping[str, Any],
*,
root: Path | None = None,
) -> tuple[str, Mapping[str, Any]] | None:
source_type = str(frontmatter.get("source_type", ""))
config = policy["documents"].get(source_type)
if not isinstance(config, Mapping):
return None
mode, vault_root = _layout(root) if root is not None else ("compatibility", "vault")
if mode == "canonical":
parts = Path(relative).parts
categories = {Path(value).name for value in config["roots"]}
canonical_match = bool(parts) and parts[0] == vault_root and bool(categories.intersection(parts))
# Canonical cutover keeps legacy paths as read-only compatibility
# symlinks. Embedded semantic audits deliberately retain that stable
# logical subject, so policy coverage must recognize both spellings
# while document discovery continues to select only vault authority.
legacy_match = any(
relative == policy_root or relative.startswith(f"{policy_root}/")
for policy_root in config["roots"]
)
if not canonical_match and not legacy_match:
return None
elif not any(relative == policy_root or relative.startswith(f"{policy_root}/") for policy_root in config["roots"]):
return None
return source_type, config
def extract_document(root: Path, path: Path, policy: Mapping[str, Any]) -> dict[str, Any]:
root = root.resolve(strict=True)
# Preserve the caller's logical compatibility path while separately
# validating the resolved target. Canonical cutover symlinks may point to
# identical bytes under ``vault/`` without changing a certificate's
# stable subject identity.
path = Path(os.path.abspath(path))
try:
relative = path.relative_to(root).as_posix()
path.resolve(strict=True).relative_to(root)
except (ValueError, FileNotFoundError) as exc:
raise SemanticSurfaceError(f"document escapes repository: {path}") from exc
if not path.is_file():
raise SemanticSurfaceError(f"document does not exist: {path}")
text = path.read_text(encoding="utf-8")
frontmatter = parse_frontmatter(text)
matched = document_policy(relative, frontmatter, policy, root=root)
if matched is None:
raise SemanticSurfaceError(f"document is not covered by semantic policy: {relative}")
source_type, config = matched
sections = _sections(text)
exclusions, findings = _exclusions(frontmatter, str(policy.get("exclusion_field", "semantic_surface_exclusions")))
for item in findings:
item["path"] = relative
required_ids = {str(item["section_id"]) for item in config["required_surfaces"]}
for unknown in sorted(set(exclusions) - required_ids):
findings.append({
"code": "UNKNOWN_SEMANTIC_SURFACE_EXCLUSION",
"path": relative,
"line": 0,
"message": f"exclusion does not name a required surface: {unknown}",
"severity": "error",
})
surfaces: list[dict[str, Any]] = []
excluded: list[dict[str, str]] = []
for requirement in config["required_surfaces"]:
section_id = str(requirement["section_id"])
accepted_ids = {section_id, *map(str, requirement.get("section_aliases", []))}
exact = [section for section in sections if section.section_id in accepted_ids]
legacy = [
section for section in sections
if section.section_id is None and re.search(str(requirement["legacy_heading"]), section.heading, re.IGNORECASE)
]
matches = exact if exact else legacy
if len(matches) > 1:
findings.append({
"code": "AMBIGUOUS_SEMANTIC_SURFACE",
"path": relative,
"line": matches[0].line_start,
"message": f"multiple sections match required surface {section_id}",
"severity": "error",
})
continue
if not matches:
if section_id in exclusions:
excluded.append({"section_id": section_id, "reason": exclusions[section_id]})
else:
findings.append({
"code": "SEMANTIC_SURFACE_UNCOVERED",
"path": relative,
"line": 0,
"message": f"required surface is neither extracted nor explicitly excluded: {section_id}",
"severity": "error",
})
continue
section = matches[0]
if not exact:
findings.append({
"code": "LEGACY_SEMANTIC_SURFACE",
"path": relative,
"line": section.line_start,
"message": f"legacy heading fallback matched {section_id}: {section.heading}",
"severity": "warning",
})
surface_key = f"{relative}\0{section_id}\0{section.line_start}\0{section.line_end}".encode("utf-8")
surfaces.append({
"surface_id": "SURF-" + hashlib.sha256(surface_key).hexdigest()[:20].upper(),
"path": relative,
"section_id": section_id,
"heading": section.heading,
"line_start": section.line_start,
"line_end": section.line_end,
"authority": "high" if source_type == "project-note" else "medium",
"content_sha256": hashlib.sha256(section.text.encode("utf-8")).hexdigest(),
"text": section.text,
})
eligible = len(config["required_surfaces"])
extracted = len(surfaces)
excluded_count = len(excluded)
uncovered = eligible - extracted - excluded_count
if uncovered != sum(1 for item in findings if item["code"] in {"SEMANTIC_SURFACE_UNCOVERED", "AMBIGUOUS_SEMANTIC_SURFACE"}):
raise SemanticSurfaceError(f"coverage accounting drift for {relative}")
return {
"path": relative,
"source_type": source_type,
"mode": config["mode"],
"document_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(),
"required": is_required(frontmatter, policy),
"coverage": {
"eligible_surface_blocks": eligible,
"extracted_surface_blocks": extracted,
"explicitly_excluded_blocks": excluded_count,
"uncovered_surface_blocks": uncovered,
},
"surfaces": sorted(surfaces, key=lambda item: item["section_id"]),
"excluded": sorted(excluded, key=lambda item: item["section_id"]),
"findings": sorted(findings, key=lambda item: (item["severity"], item["code"], item["line"])),
}
def eligible_documents(root: Path, policy: Mapping[str, Any], *, required_only: bool = False, mode: str | None = None) -> tuple[Path, ...]:
root = root.resolve(strict=True)
found: set[Path] = set()
layout_mode, vault_root = _layout(root)
for source_type, config in policy["documents"].items():
if mode is not None and config["mode"] != mode:
continue
scan_roots = [root / vault_root] if layout_mode == "canonical" else [root / value for value in config["roots"]]
categories = {Path(value).name for value in config["roots"]}
for directory in scan_roots:
if not directory.is_dir():
continue
for path in directory.rglob("*.md"):
if layout_mode == "canonical" and not categories.intersection(path.relative_to(root).parts):
continue
frontmatter = parse_frontmatter(path.read_text(encoding="utf-8"))
if str(frontmatter.get("source_type", "")) != source_type:
continue
if required_only and not is_required(frontmatter, policy):
continue
found.add(path.resolve())
return tuple(sorted(found))
def check(root: Path, policy_path: Path = DEFAULT_POLICY, *, paths: Iterable[Path] | None = None) -> dict[str, Any]:
root = root.resolve(strict=True)
policy = load_policy(root, policy_path)
selected = tuple(paths) if paths is not None else eligible_documents(root, policy, required_only=True)
documents = [extract_document(root, path if path.is_absolute() else root / path, policy) for path in selected]
findings = [item for document in documents for item in document["findings"]]
# 자동 탐색(paths=None)이 0건을 반환하면 "검사할 게 없어 PASS" 처럼 보이지만, 실제로는
# 레이아웃 모드 오판·카테고리 필터·frontmatter 탐색이 깨져 *조용히 아무것도 안 검사한*
# 상태일 수 있다. 명시적 paths=[] 는 호출자가 의도한 공허참(그대로 PASS)이지만, 자동
# 탐색의 0건은 loud FAIL 로 드러낸다(진짜 빈 저장소면 paths=[] 로 호출).
if paths is None and not selected:
findings.append({
"code": "NO_ELIGIBLE_DOCUMENTS",
"path": "",
"line": 0,
"message": "자동 탐색이 대상 문서를 0건 발견 — 탐색(레이아웃/카테고리/source_type)이 "
"깨졌을 수 있음. 빈 저장소가 맞다면 paths=[] 로 명시 호출하세요.",
"severity": "error",
})
errors = [item for item in findings if item["severity"] == "error"]
coverage = {
key: sum(int(document["coverage"][key]) for document in documents)
for key in (
"eligible_surface_blocks",
"extracted_surface_blocks",
"explicitly_excluded_blocks",
"uncovered_surface_blocks",
)
}
if coverage["eligible_surface_blocks"] != coverage["extracted_surface_blocks"] + coverage["explicitly_excluded_blocks"] + coverage["uncovered_surface_blocks"]:
raise SemanticSurfaceError("global semantic surface coverage invariant failed")
return {
"schema_version": RESULT_SCHEMA,
"status": "PASS" if not errors else "FAIL",
"policy_sha256": hashlib.sha256(canonical_json_bytes(policy)).hexdigest(),
"document_count": len(documents),
"coverage": coverage,
"documents": documents,
"findings": sorted(findings, key=lambda item: (item["path"], item["line"], item["code"])),
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--policy", type=Path, default=DEFAULT_POLICY)
parser.add_argument("--path", type=Path, action="append")
parser.add_argument("--check", action="store_true", help="validate without writing (the only supported mode)")
args = parser.parse_args(argv)
try:
result = check(args.root, args.policy, paths=args.path)
exit_code = 0 if result["status"] == "PASS" else 1
except (SemanticSurfaceError, OSError, UnicodeError, json.JSONDecodeError) as exc:
result = {"schema_version": RESULT_SCHEMA, "status": "ERROR", "errors": [{"code": "SEMANTIC_SURFACE_ERROR", "message": str(exc)}]}
exit_code = 2
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+249
View File
@@ -0,0 +1,249 @@
#!/usr/bin/env python3
"""Fail closed on transport wrappers, merge debris, and retired scratch references."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import re
import sys
from typing import Any
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
MANIFEST = Path("harness/source/generation-manifest.json")
REPOMIX_IGNORE = Path(".repomixignore")
SCHEMA = "source-hygiene-result/v1"
TEXT_SUFFIXES = {".json", ".md", ".py", ".toml", ".txt", ".yaml", ".yml"}
TRANSPORT_RE = re.compile(r"^\s*</?(?:content|file)(?:\s[^>]*)?>\s*$", re.IGNORECASE)
MERGE_RE = re.compile(r"^(?:<{7}|={7}|>{7})(?:\s|$)")
SCRATCH_RE = re.compile(
r"(?:\.agents/plugins/wiki-superpowers/scratch(?:/|\b)|"
r"(?:analyze_audit_results|build_master_report|build_per_file_findings|"
r"extract_findings_details|generate_markdown_tables|generate_sed_proofs)\.py)"
)
CONTROL_RE = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]")
PLACEHOLDER_RE = re.compile(r"\{\{([^{}\n]*)\}\}")
REQUIRED_REPOMIX_RULES = {
".agents/plugins/wiki-superpowers/scratch/**",
"**/__pycache__/**",
"**/*.pyc",
}
RETIRED_SCRATCH = Path(".agents/plugins/wiki-superpowers/scratch")
class HygieneError(ValueError):
"""The hygiene scan could not be constructed safely."""
def _safe_rel(value: Any, field: str) -> Path:
if not isinstance(value, str) or not value:
raise HygieneError(f"{field} must be a non-empty repository-relative path")
path = Path(value)
if path.is_absolute() or ".." in path.parts:
raise HygieneError(f"{field} escapes repository: {value}")
return path
def _manifest_paths(root: Path) -> tuple[set[Path], set[Path]]:
manifest_path = root / MANIFEST
data = json.loads(manifest_path.read_text(encoding="utf-8"))
if data.get("schema_version") != 1:
raise HygieneError("generation manifest must use schema_version 1")
sources = data.get("sources")
if not isinstance(sources, list) or not sources:
raise HygieneError("generation manifest sources must be non-empty")
neutral: set[Path] = set()
for item in sources:
if not isinstance(item, dict):
raise HygieneError("generation manifest source entry must be an object")
neutral.add(_safe_rel(item.get("metadata"), "source metadata"))
neutral.add(_safe_rel(item.get("body"), "source body"))
neutral.update(
path.relative_to(root)
for path in (root / "harness/source").rglob("*")
if path.is_file()
)
targets: set[Path] = set()
for metadata in sorted(path for path in neutral if path.suffix == ".json"):
absolute = root / metadata
document = json.loads(absolute.read_text(encoding="utf-8"))
if document.get("source_kind") not in {"workflow", "agent"}:
continue
raw_targets = document.get("targets")
if not isinstance(raw_targets, list):
raise HygieneError(f"{metadata}: targets must be a list")
for target in raw_targets:
if not isinstance(target, dict):
raise HygieneError(f"{metadata}: target must be an object")
targets.add(_safe_rel(target.get("path"), "generated target"))
return neutral, targets
def _scan_text(path: Path, relative: Path, *, neutral_source: bool = False, generated_target: bool = False) -> list[dict[str, Any]]:
if path.suffix.lower() not in TEXT_SUFFIXES:
return []
text = path.read_text(encoding="utf-8")
findings: list[dict[str, Any]] = []
for line_number, line in enumerate(text.splitlines(), 1):
code: str | None = None
if TRANSPORT_RE.fullmatch(line):
code = "TRANSPORT_WRAPPER"
elif MERGE_RE.match(line):
code = "MERGE_MARKER"
elif SCRATCH_RE.search(line):
code = "SCRATCH_REFERENCE"
if code:
findings.append(
{
"code": code,
"path": relative.as_posix(),
"line": line_number,
}
)
for match in CONTROL_RE.finditer(text):
findings.append(
{
"code": "FORBIDDEN_CONTROL_CHARACTER",
"path": relative.as_posix(),
"line": text.count("\n", 0, match.start()) + 1,
"codepoint": f"U+{ord(match.group(0)):04X}",
}
)
for match in PLACEHOLDER_RE.finditer(text):
placeholder = match.group(1).strip().casefold()
if generated_target and placeholder == "arguments":
findings.append(
{
"code": "UNRESOLVED_GENERATOR_PLACEHOLDER",
"path": relative.as_posix(),
"line": text.count("\n", 0, match.start()) + 1,
}
)
elif neutral_source and placeholder != "arguments":
findings.append(
{
"code": "UNAPPROVED_NEUTRAL_PLACEHOLDER",
"path": relative.as_posix(),
"line": text.count("\n", 0, match.start()) + 1,
"placeholder": match.group(0),
}
)
return findings
def _content_roots(root: Path) -> list[Path]:
"""콘텐츠 실파일이 놓인 활성 write root 를 레이아웃 계약에서 읽어 돌려준다.
"vault" 를 하드코딩하면 canonical 모드에서만 맞고, compatibility/shadow(cutover 이전)
에서는 콘텐츠가 raw/·wiki/ 실파일이라 스캔 대상이 0건이 되어 hygiene 검사가 조용히
아무것도 안 훑는다. 모드에 맞는 write root(canonical→vault, 그 외→raw·wiki)만 골라
심링크 이중 스캔도 피한다. 레이아웃을 못 읽으면 존재하는 표준 콘텐츠 루트로 폴백한다.
"""
try:
import layout_check
authority = layout_check.resolve_authority(root, require_clean=False)
roots = [root / r for r in authority.get("write_roots", []) if isinstance(r, str)]
roots = [r for r in roots if r.is_dir()]
if roots:
return roots
except Exception:
pass
if (root / "vault").is_dir():
return [root / "vault"]
return [root / name for name in ("raw", "wiki") if (root / name).is_dir()]
def check(root: Path) -> dict[str, Any]:
root = root.resolve(strict=True)
neutral, targets = _manifest_paths(root)
# 콘텐츠 문서도 transport 잔여물 검사 대상이다. 서브에이전트가 작성한 문서에
# `</content>` 같은 wrapper 가 남는 사례가 실제로 발생했으므로 활성 write root 를 훑는다.
content: set[Path] = {
path.relative_to(root)
for base in _content_roots(root)
for path in base.rglob("*.md")
if path.is_file()
}
findings: list[dict[str, Any]] = []
scanned = 0
for relative in sorted(neutral | targets | content, key=lambda item: item.as_posix()):
absolute = root / relative
if not absolute.is_file():
findings.append({"code": "MISSING_CONTEXT_FILE", "path": relative.as_posix()})
continue
scanned += 1
findings.extend(
_scan_text(
absolute,
relative,
neutral_source=relative in neutral,
generated_target=relative in targets,
)
)
ignore_path = root / REPOMIX_IGNORE
if not ignore_path.is_file():
findings.append({"code": "MISSING_REPOMIX_IGNORE", "path": REPOMIX_IGNORE.as_posix()})
else:
rules = {
line.strip()
for line in ignore_path.read_text(encoding="utf-8").splitlines()
if line.strip() and not line.lstrip().startswith("#")
}
for rule in sorted(REQUIRED_REPOMIX_RULES - rules):
findings.append(
{
"code": "MISSING_REPOMIX_EXCLUSION",
"path": REPOMIX_IGNORE.as_posix(),
"rule": rule,
}
)
scratch = root / RETIRED_SCRATCH
if scratch.is_dir():
for path in sorted(scratch.rglob("*")):
if path.is_file() and path.suffix.lower() in TEXT_SUFFIXES:
findings.append(
{
"code": "ACTIVE_SCRATCH_FILE",
"path": path.relative_to(root).as_posix(),
}
)
findings.sort(key=lambda item: (item.get("path", ""), item.get("line", 0), item["code"]))
return {
"schema_version": SCHEMA,
"status": "PASS" if not findings else "FAIL",
"scanned_files": scanned,
"findings": findings,
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--check", action="store_true", help="explicit no-write mode")
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
args = parser.parse_args(argv)
try:
result = check(args.root)
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return 0 if result["status"] == "PASS" else 1
except (OSError, UnicodeError, json.JSONDecodeError, HygieneError) as exc:
json.dump(
{"schema_version": SCHEMA, "status": "ERROR", "error": str(exc)},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
"""Strict, dependency-free rendering for deterministic harness templates."""
from __future__ import annotations
import hashlib
from pathlib import Path
import re
from typing import Mapping
PLACEHOLDER_RE = re.compile(r"\{\{([a-z][a-z0-9_]*)\}\}")
REGION_START = "<!-- RUNTIME-TEMPLATE: branch-from-project:start -->"
REGION_END = "<!-- RUNTIME-TEMPLATE: branch-from-project:end -->"
GENERATED_START = "<!-- GENERATED: branch-contract:start -->"
GENERATED_END = "<!-- GENERATED: branch-contract:end -->"
class TemplateRenderError(ValueError):
def __init__(self, code: str, message: str) -> None:
self.code = code
super().__init__(message)
def extract_region(text: str, start: str, end: str) -> str:
"""Return one ordered marker region, excluding the marker lines."""
if text.count(start) != 1 or text.count(end) != 1:
raise TemplateRenderError("INVALID_TEMPLATE_REGION", f"expected one marker pair: {start} / {end}")
start_at = text.index(start) + len(start)
end_at = text.index(end, start_at)
if start_at >= end_at:
raise TemplateRenderError("INVALID_TEMPLATE_REGION", "template region markers are reversed")
return text[start_at:end_at].strip("\n") + "\n"
def generated_region(text: str) -> str:
"""Return the runtime-owned branch contract including its marker lines."""
if text.count(GENERATED_START) != 1 or text.count(GENERATED_END) != 1:
raise TemplateRenderError("GENERATED_REGION_DRIFT", "generated branch-contract markers must occur exactly once")
start_at = text.index(GENERATED_START)
end_at = text.index(GENERATED_END, start_at) + len(GENERATED_END)
if start_at >= end_at:
raise TemplateRenderError("GENERATED_REGION_DRIFT", "generated branch-contract markers are reversed")
return text[start_at:end_at]
def generated_sha256(text: str) -> str:
return hashlib.sha256(generated_region(text).encode("utf-8")).hexdigest()
def render(text: str, values: Mapping[str, str], *, allowed: set[str]) -> str:
"""Render only allowlisted placeholders and reject missing or extra input."""
discovered = set(PLACEHOLDER_RE.findall(text))
unknown = sorted(discovered - allowed)
if unknown:
raise TemplateRenderError("UNKNOWN_PLACEHOLDER", f"template contains non-allowlisted placeholders: {unknown}")
missing = sorted(discovered - set(values))
if missing:
raise TemplateRenderError("UNRESOLVED_PLACEHOLDER", f"placeholder values are missing: {missing}")
extra = sorted(set(values) - allowed)
if extra:
raise TemplateRenderError("UNEXPECTED_TEMPLATE_VALUE", f"values were supplied for unknown placeholders: {extra}")
rendered = PLACEHOLDER_RE.sub(lambda match: values[match.group(1)], text)
unresolved = sorted(set(PLACEHOLDER_RE.findall(rendered)))
if unresolved:
raise TemplateRenderError("UNRESOLVED_PLACEHOLDER", f"unresolved placeholders remain: {unresolved}")
return rendered
def render_branch_note(template_path: Path, values: Mapping[str, str]) -> str:
"""Render the deterministic branch-from-project region of the branch template."""
template = template_path.read_text(encoding="utf-8")
body = extract_region(template, REGION_START, REGION_END)
allowed = {
"branch_slug",
"branch_id",
"project",
"project_parent_link",
"work_item",
"inherits_yaml",
"depends_on_yaml",
"created",
"contract_packet_sha256",
"project_revision",
"completion",
"inherited_rows",
"dependency_display",
}
return render(body, values, allowed=allowed)
+632
View File
@@ -0,0 +1,632 @@
#!/usr/bin/env python3
"""Validate typed artifact, gate, delegation, and flow contracts.
The checker treats registry rows and pinned frontmatter references as the only
authority. Generated projections are verified byte-for-byte through
``contract_projection``; this module never writes repository files.
"""
from __future__ import annotations
import argparse
from collections import defaultdict
from dataclasses import dataclass
import hashlib
import json
from pathlib import Path
import re
import sys
from typing import Any, Iterable, Mapping
from contract_markdown import MarkdownTable, as_list, cell, clean, parse_frontmatter, parse_tables
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_SCHEMA = Path("harness/source/typed-contracts.json")
RESULT_SCHEMA = "typed-contract-check-result/v1"
SOURCE_SCHEMA = "typed-contracts/v1"
CONCERN_RE = re.compile(
r"^[a-z0-9]+(?:-[a-z0-9]+)*(?:\.[a-z0-9]+(?:-[a-z0-9]+)*)*$"
)
ARTIFACT_ID_RE = re.compile(r"^ART-[A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}$")
CONTRACT_ID_RE = re.compile(r"^[A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}$")
DELEGATION_ID_RE = re.compile(r"^DELEG-[A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}$")
FLOW_ID_RE = re.compile(r"^FLOW-[A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3}$")
PINNED_RE = re.compile(r"^([A-Z0-9]+(?:-[A-Z0-9]+)*-\d{3})@([1-9]\d*)$")
WIKILINK_RE = re.compile(r"\[\[([^\]|#]+)(?:#[^\]|]+)?(?:\|[^\]]+)?\]\]")
GENERATED_RE = re.compile(
r"<!-- GENERATED: (?:artifact-imports|project-contract-imports|received-delegations|flow):start -->.*?"
r"<!-- GENERATED: (?:artifact-imports|project-contract-imports|received-delegations|flow):end -->",
re.DOTALL,
)
class TypedContractError(ValueError):
"""A schema or I/O problem prevented a typed-contract decision."""
@dataclass(frozen=True)
class Document:
path: Path
relative: str
slug: str
text: str
frontmatter: Mapping[str, object]
tables: tuple[MarkdownTable, ...]
@dataclass(frozen=True)
class Record:
kind: str
identifier: str
revision: int
owner: str
path: str
line: int
values: Mapping[str, str]
@dataclass(frozen=True)
class Graph:
root: Path
config: Mapping[str, Any]
documents: tuple[Document, ...]
by_slug: Mapping[str, tuple[Document, ...]]
records: Mapping[str, tuple[Record, ...]]
imports: Mapping[str, tuple[tuple[str, int], ...]]
overrides: Mapping[str, tuple[tuple[str, int], ...]]
delegates: Mapping[str, tuple[tuple[str, int], ...]]
accepts: Mapping[str, tuple[tuple[str, int], ...]]
def canonical_json_bytes(value: Any) -> bytes:
"""Return stable UTF-8 bytes for hashes shared by all typed-contract tools."""
return (json.dumps(value, ensure_ascii=False, separators=(",", ":"), sort_keys=True) + "\n").encode("utf-8")
def _finding(code: str, path: str, message: str, line: int = 0) -> dict[str, Any]:
return {"code": code, "path": path, "line": line, "message": message}
def _safe_root(value: Any, location: str) -> Path:
if not isinstance(value, str) or not value or "\\" in value:
raise TypedContractError(f"{location}: expected repo-relative POSIX path")
path = Path(value)
if path.is_absolute() or ".." in path.parts:
raise TypedContractError(f"{location}: path escapes repository")
return path
def load_config(root: Path, schema_path: Path = DEFAULT_SCHEMA) -> Mapping[str, Any]:
source = schema_path if schema_path.is_absolute() else root / schema_path
try:
document = json.loads(source.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise TypedContractError(f"typed contract source error: {exc}") from exc
if not isinstance(document, dict) or document.get("schema_version") != SOURCE_SCHEMA:
raise TypedContractError(f"expected {SOURCE_SCHEMA}")
roots = document.get("document_roots")
registries = document.get("registries")
frontmatter = document.get("frontmatter")
markers = document.get("projection_markers")
expected_registry_names = {"artifacts", "contracts", "delegations", "flow_stages"}
if not isinstance(roots, list) or not roots:
raise TypedContractError("document_roots must be a non-empty list")
for index, value in enumerate(roots):
_safe_root(value, f"document_roots[{index}]")
if not isinstance(registries, dict) or set(registries) != expected_registry_names:
raise TypedContractError("registries must declare artifacts/contracts/delegations/flow_stages")
for name, registry in registries.items():
if not isinstance(registry, dict):
raise TypedContractError(f"registries.{name} must be an object")
section_id = registry.get("section_id")
columns = registry.get("required_columns")
if not isinstance(section_id, str) or not section_id:
raise TypedContractError(f"registries.{name}.section_id is required")
if not isinstance(columns, list) or not columns or any(not isinstance(item, str) for item in columns):
raise TypedContractError(f"registries.{name}.required_columns must be strings")
required_fields = {"imports", "overrides", "delegates", "accepts_delegations"}
if not isinstance(frontmatter, dict) or set(frontmatter) != required_fields:
raise TypedContractError("frontmatter field map is incomplete")
if not isinstance(markers, dict) or set(markers) != expected_registry_names:
raise TypedContractError("projection marker map is incomplete")
return document
def _documents(root: Path, config: Mapping[str, Any]) -> tuple[Document, ...]:
documents: list[Document] = []
for root_value in config["document_roots"]:
directory = root / root_value
if not directory.is_dir():
continue
# rglob: 현재 document_roots(raw/branch-notes·raw/project-notes)는 flat 이라
# glob 과 결과가 같지만, 루트가 nested 트리(예: vault/10-projects)를 가리키게
# 바뀌면 non-recursive glob 은 조용히 0건을 훑는다. 재귀로 그 함정을 없앤다.
for path in sorted(directory.rglob("*.md")):
text = path.read_text(encoding="utf-8")
documents.append(
Document(
path=path,
relative=path.relative_to(root).as_posix(),
slug=path.stem,
text=text,
frontmatter=parse_frontmatter(text),
tables=tuple(parse_tables(text)),
)
)
return tuple(documents)
def _table_for_section(document: Document, section_id: str) -> MarkdownTable | None:
matches = [table for table in document.tables if section_id in table.section_ids]
if len(matches) > 1:
raise TypedContractError(f"{document.relative}: duplicate registry section {section_id}")
return matches[0] if matches else None
def _owner_slug(value: str) -> str:
match = WIKILINK_RE.search(value)
if match:
return Path(match.group(1)).stem
token = clean(value)
if "/" in token:
token = Path(token).stem
return token
def _split_values(value: str) -> list[str]:
normalized = re.sub(r"<br\s*/?>", ",", value, flags=re.IGNORECASE)
return [clean(item) for item in re.split(r"[,;]", normalized) if clean(item) not in {"", "-"}]
def _positive_revision(value: str) -> int | None:
token = clean(value)
return int(token) if re.fullmatch(r"[1-9]\d*", token) else None
def _pinned_values(value: object) -> tuple[tuple[str, int], ...]:
result: list[tuple[str, int]] = []
for raw in as_list(value):
for token in _split_values(raw):
match = PINNED_RE.fullmatch(token)
if match:
result.append((match.group(1), int(match.group(2))))
return tuple(sorted(set(result)))
def _headers_missing(table: MarkdownTable, required: Iterable[str]) -> list[str]:
def key(value: str) -> str:
return re.sub(r"[^0-9a-zA-Z가-힣]+", "", value).casefold()
observed = {key(item) for item in table.headers}
return [item for item in required if key(item) not in observed]
def _record(
kind: str,
document: Document,
line: int,
row: Mapping[str, str],
identifier_field: str,
revision_field: str,
owner_field: str,
) -> Record | None:
identifier = clean(cell(dict(row), identifier_field))
revision = _positive_revision(cell(dict(row), revision_field))
if not identifier and all(not clean(value) or clean(value) == "-" for value in row.values()):
return None
return Record(
kind=kind,
identifier=identifier,
revision=revision or 0,
owner=_owner_slug(cell(dict(row), owner_field)) if owner_field else document.slug,
path=document.relative,
line=line,
values={key: clean(value) for key, value in row.items()},
)
def build_graph(root: Path, schema_path: Path = DEFAULT_SCHEMA) -> tuple[Graph, list[dict[str, Any]]]:
root = root.resolve(strict=True)
config = load_config(root, schema_path)
documents = _documents(root, config)
by_slug_lists: dict[str, list[Document]] = defaultdict(list)
for document in documents:
by_slug_lists[document.slug].append(document)
by_slug = {key: tuple(value) for key, value in by_slug_lists.items()}
findings: list[dict[str, Any]] = []
records: dict[str, list[Record]] = {name: [] for name in config["registries"]}
definitions = {
"artifacts": ("Artifact ID", "Revision", "Schema Owner"),
"contracts": ("Contract ID", "Revision", "Owner"),
"delegations": ("Delegation ID", "Revision", "Delegator"),
"flow_stages": ("Stage ID", "Revision", "Owner"),
}
for document in documents:
for kind, registry in config["registries"].items():
table = _table_for_section(document, registry["section_id"])
if table is None:
continue
missing = _headers_missing(table, registry["required_columns"])
if missing:
findings.append(
_finding("INVALID_REGISTRY_SCHEMA", document.relative, f"{kind}: missing columns {missing}", table.header_line)
)
continue
identifier_field, revision_field, owner_field = definitions[kind]
for line, row in table.rows:
item = _record(kind, document, line, row, identifier_field, revision_field, owner_field)
if item is not None:
records[kind].append(item)
fm = config["frontmatter"]
imports = {document.relative: _pinned_values(document.frontmatter.get(fm["imports"])) for document in documents}
overrides = {document.relative: _pinned_values(document.frontmatter.get(fm["overrides"])) for document in documents}
delegates = {document.relative: _pinned_values(document.frontmatter.get(fm["delegates"])) for document in documents}
accepts = {document.relative: _pinned_values(document.frontmatter.get(fm["accepts_delegations"])) for document in documents}
graph = Graph(
root=root,
config=config,
documents=documents,
by_slug=by_slug,
records={key: tuple(value) for key, value in records.items()},
imports=imports,
overrides=overrides,
delegates=delegates,
accepts=accepts,
)
return graph, findings
def _one_document(graph: Graph, slug: str) -> Document | None:
matches = graph.by_slug.get(slug, ())
return matches[0] if len(matches) == 1 else None
def _record_maps(graph: Graph) -> tuple[dict[str, Record], dict[str, list[Record]]]:
all_records: dict[str, list[Record]] = defaultdict(list)
for values in graph.records.values():
for record in values:
all_records[record.identifier].append(record)
unique = {identifier: values[0] for identifier, values in all_records.items() if len(values) == 1}
return unique, all_records
def _validate_records(graph: Graph, findings: list[dict[str, Any]]) -> None:
unique, all_records = _record_maps(graph)
patterns = {
"artifacts": ARTIFACT_ID_RE,
"contracts": CONTRACT_ID_RE,
"delegations": DELEGATION_ID_RE,
"flow_stages": FLOW_ID_RE,
}
duplicate_codes = {
"artifacts": "DUPLICATE_ARTIFACT_OWNER",
"contracts": "DUPLICATE_CONTRACT_OWNER",
"delegations": "DELEGATION_SCOPE_COLLISION",
"flow_stages": "DUPLICATE_CONTRACT_OWNER",
}
for kind, values in graph.records.items():
for record in values:
if not patterns[kind].fullmatch(record.identifier) or record.revision < 1:
findings.append(_finding("INVALID_REGISTRY_ROW", record.path, f"invalid {kind} id/revision: {record.identifier}@{record.revision}", record.line))
grouped: dict[str, list[Record]] = defaultdict(list)
for record in values:
grouped[record.identifier].append(record)
for identifier, duplicates in grouped.items():
if len(duplicates) > 1:
for record in duplicates:
findings.append(_finding(duplicate_codes[kind], record.path, f"duplicate id: {identifier}", record.line))
artifact_concerns: dict[str, list[Record]] = defaultdict(list)
for record in graph.records["artifacts"]:
name = record.values.get("name", "")
concern = re.sub(r"[^a-z0-9]+", "-", name.casefold()).strip("-")
if concern:
artifact_concerns[concern].append(record)
if _one_document(graph, record.owner) is None:
findings.append(_finding("MISSING_ARTIFACT_OWNER", record.path, f"schema owner does not resolve uniquely: {record.owner}", record.line))
schema_ref = record.values.get("schemaref", "")
schema_path = (graph.root / schema_ref).resolve() if schema_ref else None
if not schema_ref or Path(schema_ref).is_absolute() or ".." in Path(schema_ref).parts or schema_path is None or not schema_path.is_file():
findings.append(_finding("MISSING_ARTIFACT_SCHEMA", record.path, f"schema ref does not exist: {schema_ref or '(empty)'}", record.line))
for concern, values in artifact_concerns.items():
if len({item.owner for item in values}) > 1:
for record in values:
findings.append(_finding("DUPLICATE_ARTIFACT_CONCERN", record.path, f"artifact concern has multiple owners: {concern}", record.line))
concerns: dict[str, list[Record]] = defaultdict(list)
for record in graph.records["contracts"]:
concern = record.values.get("concernkey", "")
if not CONCERN_RE.fullmatch(concern):
findings.append(_finding("INVALID_CONCERN_KEY", record.path, f"not normalized lowercase kebab segments: {concern}", record.line))
else:
concerns[concern].append(record)
if _one_document(graph, record.owner) is None:
findings.append(_finding("MISSING_CONTRACT_OWNER", record.path, f"owner does not resolve uniquely: {record.owner}", record.line))
for concern, values in concerns.items():
if len({item.owner for item in values}) > 1 or len(values) > 1:
for record in values:
findings.append(_finding("DUPLICATE_CONCERN_OWNER", record.path, f"concern key collision: {concern}", record.line))
for record in graph.records["flow_stages"]:
if _one_document(graph, record.owner) is None:
findings.append(_finding("MISSING_CONTRACT_OWNER", record.path, f"flow owner does not resolve uniquely: {record.owner}", record.line))
order = _positive_revision(record.values.get("order", ""))
if order is None:
findings.append(_finding("INVALID_REGISTRY_ROW", record.path, f"flow order must be positive: {record.values.get('order', '')}", record.line))
for identifier, records in all_records.items():
kinds = {record.kind for record in records}
if len(kinds) > 1:
for record in records:
findings.append(_finding("DUPLICATE_CONTRACT_OWNER", record.path, f"id reused across registry kinds: {identifier}", record.line))
def _refs_by_identifier(values: Iterable[tuple[str, int]]) -> dict[str, int]:
return {identifier: revision for identifier, revision in values}
def _validate_imports(graph: Graph, findings: list[dict[str, Any]]) -> None:
unique, _all = _record_maps(graph)
artifact_ids = {item.identifier for item in graph.records["artifacts"]}
contract_ids = {item.identifier for item in graph.records["contracts"]}
flow_ids = {item.identifier for item in graph.records["flow_stages"]}
delegation_ids = {item.identifier for item in graph.records["delegations"]}
for document in graph.documents:
for identifier, revision in graph.imports[document.relative]:
record = unique.get(identifier)
if record is None:
code = "MISSING_ARTIFACT_IMPORT" if identifier.startswith("ART-") else "UNREGISTERED_GATE_CONTRACT"
findings.append(_finding(code, document.relative, f"unknown import: {identifier}@{revision}"))
if not identifier.startswith("ART-"):
findings.append(_finding("MISSING_CONTRACT_IMPORT", document.relative, f"unknown contract import: {identifier}@{revision}"))
continue
if record.kind == "artifacts" and revision != record.revision:
findings.append(_finding("STALE_ARTIFACT_REVISION", document.relative, f"{identifier}@{revision} != current @{record.revision}"))
elif record.kind in {"contracts", "flow_stages"} and revision != record.revision:
findings.append(_finding("STALE_CONTRACT_REVISION", document.relative, f"{identifier}@{revision} != current @{record.revision}"))
findings.append(_finding("STALE_IMPORTED_CONTRACT", document.relative, f"{identifier}@{revision} != current @{record.revision}"))
elif record.kind == "delegations":
findings.append(_finding("UNKNOWN_DELEGATION", document.relative, f"delegations cannot use imports: {identifier}@{revision}"))
for identifier, revision in graph.overrides[document.relative]:
if identifier not in contract_ids:
# Existing DEC overrides belong to the graph-contract checker.
if identifier.startswith("DEC-"):
continue
findings.append(_finding("UNDECLARED_CONTRACT_OVERRIDE", document.relative, f"unknown contract override: {identifier}@{revision}"))
continue
record = unique.get(identifier)
if record is not None and revision != record.revision:
findings.append(_finding("STALE_IMPORTED_CONTRACT", document.relative, f"override {identifier}@{revision} != current @{record.revision}"))
for record in graph.records["artifacts"]:
expected_ref = (record.identifier, record.revision)
for consumer in _split_values(record.values.get("consumers", "")):
target = _one_document(graph, _owner_slug(consumer))
if target is None:
findings.append(_finding("MISSING_ARTIFACT_IMPORT", record.path, f"consumer does not resolve uniquely: {consumer}", record.line))
continue
observed = _refs_by_identifier(graph.imports[target.relative])
if record.identifier not in observed:
findings.append(_finding("MISSING_ARTIFACT_IMPORT", target.relative, f"consumer is missing {record.identifier}@{record.revision}"))
elif observed[record.identifier] != record.revision:
findings.append(_finding("STALE_ARTIFACT_REVISION", target.relative, f"{record.identifier}@{observed[record.identifier]} != current @{record.revision}"))
def _delegation_cycles(edges: Mapping[str, set[str]]) -> list[list[str]]:
cycles: list[list[str]] = []
visiting: list[str] = []
visited: set[str] = set()
def visit(node: str) -> None:
if node in visiting:
start = visiting.index(node)
cycles.append(visiting[start:] + [node])
return
if node in visited:
return
visiting.append(node)
for target in sorted(edges.get(node, set())):
visit(target)
visiting.pop()
visited.add(node)
for node in sorted(edges):
visit(node)
return cycles
def _validate_delegations(graph: Graph, findings: list[dict[str, Any]]) -> None:
rows = graph.records["delegations"]
unique = {record.identifier: record for record in rows if sum(item.identifier == record.identifier for item in rows) == 1}
active_scope: dict[tuple[str, str], list[Record]] = defaultdict(list)
edges: dict[str, set[str]] = defaultdict(set)
for document in graph.documents:
for identifier, revision in (*graph.delegates[document.relative], *graph.accepts[document.relative]):
record = unique.get(identifier)
if record is None:
findings.append(_finding("UNKNOWN_DELEGATION", document.relative, f"unknown delegation: {identifier}@{revision}"))
elif revision != record.revision:
findings.append(_finding("STALE_DELEGATION_ACCEPTANCE", document.relative, f"{identifier}@{revision} != current @{record.revision}"))
for record in rows:
concern = record.values.get("concernkey", "")
delegate = _owner_slug(record.values.get("delegate", ""))
delegator = _owner_slug(record.values.get("delegator", ""))
scope = record.values.get("scope", "")
status = record.values.get("status", "")
if not CONCERN_RE.fullmatch(concern):
findings.append(_finding("INVALID_CONCERN_KEY", record.path, f"not normalized: {concern}", record.line))
if _one_document(graph, delegator) is None or _one_document(graph, delegate) is None:
findings.append(_finding("DELEGATION_TARGET_MISMATCH", record.path, f"delegator/delegate does not resolve: {delegator}->{delegate}", record.line))
continue
delegator_doc = _one_document(graph, delegator)
delegate_doc = _one_document(graph, delegate)
assert delegator_doc is not None and delegate_doc is not None
expected = (record.identifier, record.revision)
delegator_refs = set(graph.delegates[delegator_doc.relative])
delegate_refs = set(graph.accepts[delegate_doc.relative])
if expected not in delegator_refs:
findings.append(_finding("UNKNOWN_DELEGATION", delegator_doc.relative, f"delegator must declare {record.identifier}@{record.revision}"))
if status == "accepted" and expected not in delegate_refs:
findings.append(_finding("UNACCEPTED_DELEGATION", delegate_doc.relative, f"delegate has not accepted {record.identifier}@{record.revision}"))
elif status == "proposed":
findings.append(_finding("UNACCEPTED_DELEGATION", record.path, f"delegation remains proposed: {record.identifier}", record.line))
if status not in graph.config["registries"]["delegations"]["statuses"]:
findings.append(_finding("INVALID_REGISTRY_ROW", record.path, f"invalid delegation status: {status}", record.line))
for document in graph.documents:
if expected in set(graph.delegates[document.relative]) and document.slug != delegator:
findings.append(_finding("DELEGATION_TARGET_MISMATCH", document.relative, f"{record.identifier} belongs to delegator {delegator}"))
if expected in set(graph.accepts[document.relative]) and document.slug != delegate:
findings.append(_finding("DELEGATION_TARGET_MISMATCH", document.relative, f"{record.identifier} belongs to delegate {delegate}"))
if status in {"proposed", "accepted"}:
active_scope[(concern, scope)].append(record)
edges[delegator].add(delegate)
for (concern, scope), collisions in active_scope.items():
if len(collisions) > 1:
for record in collisions:
findings.append(_finding("DELEGATION_SCOPE_COLLISION", record.path, f"active scope collision: {concern}/{scope}", record.line))
for cycle in _delegation_cycles(edges):
findings.append(_finding("DELEGATION_CYCLE", "", " -> ".join(cycle)))
def _table_text(document: Document, table: MarkdownTable) -> str:
lines = document.text.splitlines()
end = table.header_line + 1 + len(table.rows)
return "\n".join(lines[table.header_line - 1 : end])
# registry / gate matrix 가 소유한 정의 열. 비-owner 표에 이 중 2개 이상이 나타나면
# 그 표는 owner 의 정의를 옮겨 적은 것이다.
DEFINITIONAL_HEADERS = (
"blocking scope", "covered fe-oc", "covered fe-nfr", "pass condition",
"required fixture", "evidence artifact", "required effect", "enforcement",
"trigger", "차단 범위", "통과 조건", "필수 fixture", "증거 artifact",
)
def _validate_manual_restatements(graph: Graph, findings: list[dict[str, Any]]) -> None:
registry_sections = {value["section_id"] for value in graph.config["registries"].values()}
# 정의 registry — 계약을 '정의'하는 표이지 남의 계약을 옮겨 적은 것이 아니다.
# 스키마 검증은 wiki_graph_contract_check 가 별도로 수행한다.
registry_sections |= set(graph.config.get("definition_sections", ()))
artifacts = {record.identifier: record for record in graph.records["artifacts"]}
contracts = {record.identifier: record for record in graph.records["contracts"]}
for document in graph.documents:
generated_lines = [
(
document.text.count("\n", 0, match.start()) + 1,
document.text.count("\n", 0, match.end()) + 1,
)
for match in GENERATED_RE.finditer(document.text)
]
for table in document.tables:
if registry_sections.intersection(table.section_ids):
continue
if any(start <= table.header_line <= end for start, end in generated_lines):
continue
rendered = _table_text(document, table)
headers = " ".join(table.headers).casefold()
for identifier, record in artifacts.items():
if document.slug != record.owner and (identifier in rendered or record.values.get("name", "") in rendered):
if any(token in headers for token in ("field", "property", "schema", "producer", "consumer", "필드", "속성")):
findings.append(_finding("MANUAL_ARTIFACT_SCHEMA_RESTATEMENT", document.relative, f"non-owner table restates {identifier}", table.header_line))
for identifier, record in contracts.items():
if document.slug == record.owner or identifier not in rendered:
continue
imported = identifier in _refs_by_identifier(graph.imports[document.relative])
# 재진술은 owner 의 *정의 열*을 옮겨 적었을 때다. 계약 ID 를 행 키나
# owner 참조로만 쓰는 표(자기 fixture·배선·소유 선언)는 Reference-Only 가
# 요구하는 형태이므로 걸지 않는다. 정의 열이 2개 이상 재현될 때만 발동한다.
if sum(token in headers for token in DEFINITIONAL_HEADERS) >= 2:
findings.append(_finding("MANUAL_GATE_RESTATEMENT", document.relative, f"non-owner table restates {identifier}", table.header_line))
findings.append(_finding("FOREIGN_CONTRACT_RESTATEMENT", document.relative, f"foreign contract table: {identifier}", table.header_line))
if not imported:
findings.append(_finding("MISSING_CONTRACT_IMPORT", document.relative, f"table references {identifier} without a pinned import", table.header_line))
override_table = next((table for table in document.tables if "declared-overrides" in table.section_ids), None)
if override_table is not None:
rendered = _table_text(document, override_table)
declared = _refs_by_identifier(graph.overrides[document.relative])
for identifier, record in contracts.items():
if identifier in rendered and declared.get(identifier) != record.revision:
findings.append(_finding("UNDECLARED_CONTRACT_OVERRIDE", document.relative, f"override table is not pinned in frontmatter: {identifier}@{record.revision}", override_table.header_line))
def _graph_payload(graph: Graph) -> dict[str, Any]:
return {
kind: [
{
"id": record.identifier,
"revision": record.revision,
"owner": record.owner,
"path": record.path,
"values": dict(sorted(record.values.items())),
}
for record in sorted(values, key=lambda item: (item.identifier, item.path, item.line))
]
for kind, values in sorted(graph.records.items())
}
def check(
root: Path,
schema_path: Path = DEFAULT_SCHEMA,
*,
include_projection: bool = True,
) -> dict[str, Any]:
graph, findings = build_graph(root, schema_path)
_validate_records(graph, findings)
_validate_imports(graph, findings)
_validate_delegations(graph, findings)
_validate_manual_restatements(graph, findings)
projections = 0
if include_projection:
# Lazy import avoids a module cycle: the projection engine reuses this
# module's parsed graph and canonical hashing rules.
import contract_projection
projection_result = contract_projection.plan(graph)
projections = projection_result["projection_count"]
findings.extend(projection_result["findings"])
unique = {
(item["code"], item["path"], item.get("line", 0), item["message"]): item
for item in findings
}
findings = [unique[key] for key in sorted(unique)]
payload = _graph_payload(graph)
counts = {kind: len(values) for kind, values in graph.records.items()}
return {
"schema_version": RESULT_SCHEMA,
"status": "PASS" if not findings else "FAIL",
"registries": counts,
"imports": sum(len(value) for value in graph.imports.values()),
"projections": projections,
"typed_contract_graph_sha256": hashlib.sha256(canonical_json_bytes(payload)).hexdigest(),
"findings": findings,
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--schema", type=Path, default=DEFAULT_SCHEMA)
parser.add_argument("--without-projection", action="store_true")
args = parser.parse_args(argv)
try:
result = check(args.root, args.schema, include_projection=not args.without_projection)
exit_code = 0 if result["status"] == "PASS" else 1
except (TypedContractError, OSError, UnicodeError, json.JSONDecodeError) as exc:
result = {
"schema_version": RESULT_SCHEMA,
"status": "ERROR",
"errors": [{"code": "TYPED_CONTRACT_ERROR", "message": str(exc)}],
}
exit_code = 2
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+860
View File
@@ -0,0 +1,860 @@
#!/usr/bin/env python3
"""Plan and atomically apply project-first vault authority transitions."""
from __future__ import annotations
import argparse
import copy
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
import tempfile
from typing import Any, Iterable, Mapping
from contract_markdown import parse_frontmatter
from fs_transaction import ReplacementValue, SymlinkValue, replace_many
import layout_check
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_LAYOUT = Path("harness/source/vault-layout.json")
DEFAULT_RELATIONS = Path("harness/source/document-relations.json")
SCHEMA_VERSION = "vault-migration-plan/v1"
RESULT_SCHEMA = "vault-migration-result/v1"
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
# Obsidian table cells commonly escape the alias separator as ``\|``. Keep
# the separator in its own group so the target lookup never receives the
# trailing escape character and preserve the author's original spelling when
# replacing only the authority path.
WIKILINK = re.compile(r"\[\[([^\]|#]+?)(#[^\]|]+)?((?:\\)?\|[^\]]+)?\]\]")
class MigrationError(ValueError):
def __init__(self, code: str, message: str, location: str = "") -> None:
self.code = code
self.location = location
super().__init__(message)
class MigrationPlan(dict[Path, ReplacementValue]):
def __init__(
self,
values: Mapping[Path, ReplacementValue],
*,
expected: Mapping[Path, str | None],
forbidden: Iterable[Path],
result: Mapping[str, Any],
) -> None:
super().__init__(values)
self.expected = dict(expected)
self.forbidden = set(forbidden)
self.result = dict(result)
def _sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _lexical_absolute(path: Path) -> Path:
"""Return an absolute path without following a compatibility symlink."""
return Path(os.path.abspath(path))
def _entry_fingerprint(path: Path) -> str | None:
if path.is_symlink():
return _sha256(("symlink\0" + os.readlink(path)).encode("utf-8"))
if path.is_file():
return _sha256(path.read_bytes())
return None
def _replacement_fingerprint(value: ReplacementValue) -> str:
if isinstance(value, SymlinkValue):
return _sha256(("symlink\0" + value.target).encode("utf-8"))
return _sha256(value)
def _config_path(root: Path, value: Path) -> Path:
return value if value.is_absolute() else root / value
def _load_json(path: Path, schema: str) -> dict[str, Any]:
try:
document = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
raise MigrationError("CONFIG_IO_ERROR", str(exc), str(path)) from exc
if not isinstance(document, dict) or document.get("schema_version") != schema:
raise MigrationError("CONFIG_SCHEMA_MISMATCH", f"expected {schema}", str(path))
return document
def _document_bytes(document: Mapping[str, Any]) -> bytes:
return (json.dumps(document, ensure_ascii=False, indent=2, sort_keys=True) + "\n").encode("utf-8")
def _assignments(config: Mapping[str, Any]) -> dict[str, str]:
areas = config.get("areas")
if not isinstance(areas, dict):
raise MigrationError("INVALID_LAYOUT_SCHEMA", "areas must be an object", "areas")
result: dict[str, str] = {}
for area, roots in areas.items():
if not isinstance(area, str) or not isinstance(roots, list):
raise MigrationError("INVALID_LAYOUT_SCHEMA", "area roots must be arrays", f"areas.{area}")
for raw_root in roots:
source = Path(str(raw_root)).as_posix().rstrip("/")
if source in result:
raise MigrationError("DUPLICATE_LAYOUT_OWNER", source, source)
result[source] = area
return result
def _content_paths(root: Path, assignments: Mapping[str, str], overrides: Mapping[Path, bytes]) -> set[Path]:
paths: set[Path] = set()
for source in assignments:
base = root / source
if base.is_dir():
paths.update(path for path in base.rglob("*") if path.is_file() and path.name != ".gitkeep")
paths.update(path for path in overrides if path.name != ".gitkeep")
return paths
def _source_owner(root: Path, legacy: Path, assignments: Mapping[str, str]) -> tuple[str, Path]:
owners: list[tuple[int, str, Path]] = []
for source, area in assignments.items():
base = (root / source).resolve()
try:
legacy.resolve().relative_to(base)
except ValueError:
continue
owners.append((len(base.parts), area, base))
if len(owners) != 1:
relative = legacy.relative_to(root).as_posix()
code = "MISSING_LAYOUT_OWNER" if not owners else "AMBIGUOUS_LAYOUT_OWNER"
raise MigrationError(code, f"expected one assigned source root, observed {len(owners)}", relative)
_length, area, base = owners[0]
return area, base
def _mapping_policy(config: Mapping[str, Any]) -> dict[str, str]:
policy = config.get("canonical_mapping")
expected = {
"schema_version": "project-first-paths/v1",
"project_relation": "branch-to-project",
"project_member_pattern": "{vault_root}/{area}/{project}/{category}/{relative_path}",
"default_pattern": "{vault_root}/{area}/{category}/{relative_path}",
}
if policy != expected:
raise MigrationError(
"INVALID_CANONICAL_MAPPING_POLICY",
"canonical_mapping must declare the project-first-paths/v1 deterministic patterns",
"canonical_mapping",
)
return expected
def _project_contract(relations: Mapping[str, Any], relation_id: str) -> tuple[set[str], dict[str, str]]:
project_roots: set[str] = set()
members: dict[str, str] = {}
rows = relations.get("relations")
if not isinstance(rows, list):
raise MigrationError("INVALID_RELATION_SCHEMA", "relations must be an array", "relations")
for row in rows:
if not isinstance(row, dict):
continue
if row.get("id") != relation_id:
continue
parent_roots = row.get("parent_roots")
child_roots = row.get("child_roots")
field = row.get("parent_field")
if not isinstance(parent_roots, list) or not isinstance(child_roots, list) or not isinstance(field, str):
raise MigrationError("INVALID_RELATION_SCHEMA", "branch-to-project relation is incomplete")
project_roots.update(Path(str(value)).as_posix().rstrip("/") for value in parent_roots)
for child in child_roots:
source = Path(str(child)).as_posix().rstrip("/")
if source in members and members[source] != field:
raise MigrationError("AMBIGUOUS_PROJECT_RELATION", source, source)
members[source] = field
if not project_roots or not members:
raise MigrationError("PROJECT_RELATION_MISSING", f"{relation_id} relation is required")
return project_roots, members
def _frontmatter_for(path: Path, overrides: Mapping[Path, bytes]) -> dict[str, Any]:
try:
data = overrides[path] if path in overrides else path.read_bytes()
return parse_frontmatter(data.decode("utf-8"))
except (OSError, UnicodeError) as exc:
raise MigrationError("PROJECT_METADATA_UNREADABLE", str(exc), str(path)) from exc
def _canonical_path(
root: Path,
legacy: Path,
config: Mapping[str, Any],
assignments: Mapping[str, str],
project_roots: set[str],
project_members: Mapping[str, str],
overrides: Mapping[Path, bytes],
) -> Path:
area, source_base = _source_owner(root, legacy, assignments)
source = source_base.relative_to(root).as_posix()
relative = legacy.relative_to(source_base)
vault = root / str(config.get("vault_root", "vault")) / area
category_parts = Path(source).parts
if category_parts and category_parts[0] in {"raw", "wiki"}:
category_parts = category_parts[1:]
category = Path(*category_parts)
project: str | None = None
if source in project_roots:
if relative.parent != Path(".") or legacy.suffix != ".md":
raise MigrationError("INVALID_PROJECT_OWNER_PATH", "project notes must be top-level Markdown files", legacy.relative_to(root).as_posix())
project = legacy.stem
elif source in project_members:
metadata = _frontmatter_for(legacy, overrides)
raw_project = metadata.get(project_members[source])
if not isinstance(raw_project, str) or not re.fullmatch(r"[a-z0-9]+(?:-[a-z0-9]+)*", raw_project):
raise MigrationError("PROJECT_OWNER_MISSING", f"{project_members[source]} must name exactly one project", legacy.relative_to(root).as_posix())
project = raw_project
if not any((root / parent_root / f"{project}.md").is_file() for parent_root in project_roots):
raise MigrationError("PROJECT_OWNER_NOT_FOUND", project, legacy.relative_to(root).as_posix())
if project is not None:
return vault / project / category / relative
return vault / category / relative
def build_mapping(
root: Path,
config: Mapping[str, Any],
relations: Mapping[str, Any],
*,
overrides: Mapping[Path, bytes] | None = None,
) -> dict[Path, Path]:
"""Derive one canonical owner for every configured content file."""
root = root.resolve()
overrides = {path.resolve(): data for path, data in (overrides or {}).items()}
assignments = _assignments(config)
policy = _mapping_policy(config)
project_roots, project_members = _project_contract(relations, policy["project_relation"])
mapping: dict[Path, Path] = {}
reverse: dict[Path, Path] = {}
for legacy in sorted(_content_paths(root, assignments, overrides)):
legacy = legacy.resolve()
canonical = _canonical_path(
root,
legacy,
config,
assignments,
project_roots,
project_members,
overrides,
).resolve()
if canonical in reverse:
first = reverse[canonical].relative_to(root).as_posix()
second = legacy.relative_to(root).as_posix()
raise MigrationError("CANONICAL_PATH_COLLISION", f"{first}, {second}", canonical.relative_to(root).as_posix())
mapping[legacy] = canonical
reverse[canonical] = legacy
return mapping
def _embedded_documents(config: dict[str, Any]) -> tuple[dict[str, Any], dict[str, Any]]:
migration = config.get("migration_manifest")
rollback = config.get("rollback_mapping")
if not isinstance(migration, dict) or not isinstance(rollback, dict):
raise MigrationError("EXTERNAL_CUTOVER_MANIFEST_UNSUPPORTED", "writer updates require embedded migration and rollback documents")
if migration.get("schema_version") != "vault-migration/v1" or rollback.get("schema_version") != "vault-rollback/v1":
raise MigrationError("INVALID_CUTOVER_SCHEMA", "invalid embedded migration or rollback schema")
return migration, rollback
def _entries(
mapping: Mapping[Path, Path],
root: Path,
payloads: Mapping[Path, ReplacementValue],
) -> tuple[list[dict[str, str]], list[dict[str, str]]]:
migration: list[dict[str, str]] = []
rollback: list[dict[str, str]] = []
for legacy, canonical in sorted(mapping.items(), key=lambda item: item[0].relative_to(root).as_posix()):
content = payloads.get(canonical)
if isinstance(content, SymlinkValue):
raise MigrationError(
"CANONICAL_OWNER_SYMLINK",
"canonical owner must contain bytes",
canonical.relative_to(root).as_posix(),
)
if content is None:
if not canonical.is_file():
raise MigrationError("CANONICAL_CONTENT_MISSING", "canonical owner does not exist", canonical.relative_to(root).as_posix())
content = canonical.read_bytes()
migration.append({
"legacy_path": legacy.relative_to(root).as_posix(),
"canonical_path": canonical.relative_to(root).as_posix(),
"sha256": _sha256(content),
})
rollback.append({
"canonical_path": canonical.relative_to(root).as_posix(),
"legacy_path": legacy.relative_to(root).as_posix(),
})
return migration, rollback
def _configured_mapping(root: Path, config: Mapping[str, Any]) -> dict[Path, Path]:
migration = config.get("migration_manifest")
if not isinstance(migration, dict) or migration.get("schema_version") != "vault-migration/v1":
raise MigrationError("INVALID_CUTOVER_SCHEMA", "embedded migration manifest required")
rows = migration.get("entries")
if not isinstance(rows, list):
raise MigrationError("INVALID_CUTOVER_SCHEMA", "migration entries must be an array")
result: dict[Path, Path] = {}
for index, row in enumerate(rows):
if not isinstance(row, dict) or set(row) != {"legacy_path", "canonical_path", "sha256"}:
raise MigrationError("INVALID_MIGRATION_ENTRY", str(index))
legacy = _lexical_absolute(root / str(row["legacy_path"]))
canonical = (root / str(row["canonical_path"])).resolve()
try:
legacy.relative_to(root)
canonical.relative_to(root)
except ValueError as exc:
raise MigrationError("INVALID_MIGRATION_PATH", str(index)) from exc
if legacy in result:
raise MigrationError("DUPLICATE_LEGACY_OWNER", row["legacy_path"])
result[legacy] = canonical
return result
def expand_authoritative_changes(
root: Path,
changes: Mapping[Path, bytes],
*,
layout_path: Path = DEFAULT_LAYOUT,
relations_path: Path = DEFAULT_RELATIONS,
) -> tuple[dict[Path, bytes], dict[str, Any]]:
"""Enforce active roots and add shadow mirrors plus manifest hash updates."""
root = root.resolve(strict=True)
normalized = {path.resolve(): data for path, data in changes.items()}
try:
authority = layout_check.resolve_authority(root, layout_path)
except layout_check.LayoutContractError as exc:
raise MigrationError(exc.code, str(exc), exc.location) from exc
planner_surface: set[Path] = set()
if authority["mode"] == "canonical":
# 기존 문서는 legacy 경로가 심링크라 위의 resolve() 가 정본으로 번역해 준다.
# 신규 문서에는 그 심링크가 아직 없어 raw/… 그대로 남고, write root 집행에
# 걸려 **문서 생성 자체가 막혀 있었다**. 여기서 목적지를 계산해 정본·호환
# 심링크·manifest 2행을 한 트랜잭션으로 만든다(셋 중 하나만 빠져도 layout FAIL).
pending = {
path: data
for path, data in normalized.items()
if isinstance(data, bytes)
and not path.exists()
and not path.is_symlink()
and _is_legacy_content_path(root, path, layout_path)
}
if pending:
planned = plan_new_documents(
root, pending, layout_path=layout_path, relations_path=relations_path
)
normalized = {
path: data for path, data in normalized.items() if path not in pending
}
normalized.update(planned)
# write-root 집행 대상은 caller 가 고른 목적지다. planner 가 layout 계약
# (①~③ 동시 성립)을 위해 스스로 도출한 ②호환 심링크(raw/…)와 ③manifest
# (harness/source/…)는 canonical write root 밖에 있는 것이 정상이므로
# 집행에서 면제한다 — 면제 없이는 신규 문서 생성 전체가 여기서
# WRITE_ROOT_VIOLATION 으로 죽는다(2026-07-23 branch_from_project 실측).
# ①정본(bytes)은 계속 집행 대상이다 — vault 밖이면 여전히 위반.
planner_surface = {
path for path, value in planned.items() if isinstance(value, SymlinkValue)
}
planner_surface.add(_config_path(root, layout_path).resolve())
try:
layout_check.enforce_write_paths(
root,
[path for path in normalized if path not in planner_surface],
authority,
)
except layout_check.LayoutContractError as exc:
raise MigrationError(exc.code, str(exc), exc.location) from exc
if authority["mode"] != "shadow":
return dict(normalized), authority
config_path = _config_path(root, layout_path).resolve()
config = _load_json(config_path, "vault-layout/v1")
relations = _load_json(_config_path(root, relations_path), "document-relations/v1")
derived = build_mapping(root, config, relations, overrides=normalized)
configured = _configured_mapping(root, config)
for legacy, canonical in configured.items():
if derived.get(legacy) != canonical:
raise MigrationError("MAPPING_POLICY_DRIFT", canonical.relative_to(root).as_posix(), legacy.relative_to(root).as_posix())
expanded = dict(normalized)
for legacy, content in normalized.items():
canonical = derived.get(legacy)
if canonical is None:
raise MigrationError("SHADOW_MAPPING_MISSING", "write has no canonical mirror", legacy.relative_to(root).as_posix())
if canonical in expanded and expanded[canonical] != content:
raise MigrationError("SHADOW_WRITE_CONFLICT", "legacy and mirror bytes differ", canonical.relative_to(root).as_posix())
expanded[canonical] = content
updated = copy.deepcopy(config)
migration, rollback = _embedded_documents(updated)
migration_entries, rollback_entries = _entries(derived, root, expanded)
migration["entries"] = migration_entries
rollback["entries"] = rollback_entries
expanded[config_path] = _document_bytes(updated)
authority = dict(authority)
authority["control_plane_updates"] = [config_path.relative_to(root).as_posix()]
return expanded, authority
def _rewrite_links(text: str, root: Path, mapping: Mapping[Path, Path]) -> str:
by_value: dict[str, str] = {}
for legacy, canonical in mapping.items():
legacy_value = legacy.relative_to(root).as_posix()
canonical_value = canonical.relative_to(root).as_posix()
by_value[legacy_value] = canonical_value
if legacy.suffix == ".md":
by_value[legacy_value[:-3]] = canonical_value[:-3] if canonical.suffix == ".md" else canonical_value
def replace(match: re.Match[str]) -> str:
target, anchor, alias = match.groups()
rewritten = by_value.get(target, target)
return f"[[{rewritten}{anchor or ''}{alias or ''}]]"
return WIKILINK.sub(replace, text)
def _stub(legacy: Path, canonical: Path, root: Path) -> bytes:
title = legacy.stem
canonical_value = canonical.relative_to(root).as_posix()
return (
"---\n"
f"title: {title} (compatibility stub)\n"
"status: stale\n"
f"canonical_path: {canonical_value}\n"
"---\n\n"
f"# {title}\n\n"
f"Canonical document: [[{canonical_value[:-3] if canonical_value.endswith('.md') else canonical_value}]]\n"
).encode("utf-8")
def _is_legacy_content_path(root: Path, path: Path, layout_path: Path) -> bool:
"""Is this a legacy content path (raw/… · wiki/…) that a new document could claim?"""
try:
config = _load_json(_config_path(root, layout_path).resolve(), "vault-layout/v1")
assignments = _assignments(config)
except MigrationError:
return False
if path.suffix != ".md":
return False
for source in assignments:
base = _lexical_absolute(root / source)
try:
path.relative_to(base)
except ValueError:
continue
return True
return False
def plan_new_documents(
root: Path,
documents: Mapping[Path, bytes],
*,
layout_path: Path = DEFAULT_LAYOUT,
relations_path: Path = DEFAULT_RELATIONS,
) -> dict[Path, ReplacementValue]:
"""Batch form of :func:`plan_new_document`.
manifest 항목은 누적돼야 한다 — 문서마다 디스크의 config 를 새로 읽으면 두 번째가
첫 번째의 항목을 덮어써 한쪽이 조용히 사라진다.
"""
root = root.resolve(strict=True)
config_path = _config_path(root, layout_path).resolve()
config = _load_json(config_path, "vault-layout/v1")
changes: dict[Path, ReplacementValue] = {}
for legacy, content in sorted(documents.items(), key=lambda item: item[0].as_posix()):
planned, config = _plan_new_document(
root, legacy, content, config=config, relations_path=relations_path
)
changes.update(planned)
changes[config_path] = _document_bytes(config)
return changes
def plan_new_document(
root: Path,
legacy: Path,
content: bytes,
*,
layout_path: Path = DEFAULT_LAYOUT,
relations_path: Path = DEFAULT_RELATIONS,
) -> dict[Path, ReplacementValue]:
"""Return the atomic change set that creates one new document under canonical authority.
canonical 모드에서 문서 하나를 새로 만들려면 세 가지가 *동시에* 있어야 한다
(2026-07-22 실측: 하나라도 빠지면 layout_check 가 FAIL):
① 정본 파일 vault/<area>/<project>/<category>/<name>.md 없으면 MIGRATION_ENTRY_MISSING
② 호환 심링크 raw/<category>/<name>.md → 정본 없으면 CANONICAL_OWNER_MISSING
③ manifest 2행 migration_manifest + rollback_mapping
기존 문서가 그냥 되는 건 권한이 있어서가 아니라 ②가 이미 있어서 ``resolve()`` 가
번역기 노릇을 하기 때문이다. 신규 문서는 그 번역기가 없으므로 여기서 목적지를
계산해 준다. ``build_mapping`` 을 그대로 쓰지 못하는 이유는 canonical 모드에서
legacy 가 심링크라 ``legacy.resolve()`` 가 canonical 로 접혀 키가 무너지기 때문이다 —
그래서 신규 경로 하나만 ``_canonical_path`` 로 직접 계산한다.
"""
root = root.resolve(strict=True)
config_path = _config_path(root, layout_path).resolve()
config = _load_json(config_path, "vault-layout/v1")
changes, updated = _plan_new_document(
root, legacy, content, config=config, relations_path=relations_path
)
changes[config_path] = _document_bytes(updated)
return changes
def _plan_new_document(
root: Path,
legacy: Path,
content: bytes,
*,
config: Mapping[str, Any],
relations_path: Path,
) -> tuple[dict[Path, ReplacementValue], dict[str, Any]]:
"""Compute one document's change set and the manifest it leaves behind."""
legacy = _lexical_absolute(legacy if legacy.is_absolute() else root / legacy)
try:
legacy_rel = legacy.relative_to(root)
except ValueError as exc:
raise MigrationError("PATH_OUTSIDE_REPO", "legacy path escapes repository", str(legacy)) from exc
if legacy.exists() or legacy.is_symlink():
raise MigrationError("LEGACY_ALREADY_EXISTS", "document already exists", legacy_rel.as_posix())
relations = _load_json(_config_path(root, relations_path), "document-relations/v1")
assignments = _assignments(config)
policy = _mapping_policy(config)
project_roots, project_members = _project_contract(relations, policy["project_relation"])
overrides = {legacy: content}
canonical = _canonical_path(
root, legacy, config, assignments, project_roots, project_members, overrides
)
canonical = _lexical_absolute(canonical)
canonical_rel = canonical.relative_to(root)
if canonical.exists():
raise MigrationError("CANONICAL_ALREADY_EXISTS", "canonical owner already exists", canonical_rel.as_posix())
updated = copy.deepcopy(config)
migration, rollback = _embedded_documents(updated)
for document, extra in ((migration, {"sha256": _sha256(content)}), (rollback, {})):
rows = document.get("entries")
if not isinstance(rows, list):
raise MigrationError("INVALID_CUTOVER_SCHEMA", "entries must be an array")
rows.append({
"canonical_path": canonical_rel.as_posix(),
"legacy_path": legacy_rel.as_posix(),
**extra,
})
document["entries"] = sorted(rows, key=lambda row: row["canonical_path"])
link_target = os.path.relpath(canonical, start=legacy.parent)
return {
canonical: content,
legacy: SymlinkValue(Path(link_target).as_posix()),
}, updated
def _stage_repository(root: Path, destination: Path) -> None:
ignored = shutil.ignore_patterns(".git", "__pycache__", "*.pyc", ".DS_Store")
for child in root.iterdir():
if child.name == ".git":
continue
target = destination / child.name
if child.is_dir():
shutil.copytree(child, target, ignore=ignored, symlinks=True)
elif child.is_file():
shutil.copy2(child, target)
def _plan_hash(
root: Path,
current_mode: str,
target_mode: str,
expected: Mapping[Path, str | None],
changes: Mapping[Path, ReplacementValue],
authority: Mapping[str, Any],
) -> str:
document = {
"schema_version": SCHEMA_VERSION,
"from_mode": current_mode,
"to_mode": target_mode,
"active_layout": {
"authority": authority["authority"],
"manifest_sha256": authority["manifest_sha256"],
"mode": authority["mode"],
"write_roots": authority["write_roots"],
},
"preconditions": [
{
"path": path.relative_to(root).as_posix(),
"expected_sha256": expected[path],
}
for path in sorted(expected)
],
"writes": [
{
"path": path.relative_to(root).as_posix(),
"kind": "symlink" if isinstance(changes[path], SymlinkValue) else "bytes",
"sha256": _replacement_fingerprint(changes[path]),
"expected_sha256": expected[path],
}
for path in sorted(changes)
],
}
return _sha256(_document_bytes(document))
def _validate_stage(
stage: Path,
layout_path: Path,
target_mode: str,
*,
run_release_gate: bool = True,
) -> None:
result = layout_check.check_layout(stage, layout_path)
if result.get("status") != "PASS":
codes = ",".join(sorted({item.get("code", "UNKNOWN") for item in result.get("findings", [])}))
sample = json.dumps(result.get("findings", [])[:20], ensure_ascii=False, sort_keys=True)
raise MigrationError("LAYOUT_POSTFLIGHT_FAILED", f"{codes}; sample={sample}")
release_gate = stage / "harness/runtime/release_gate.py"
if run_release_gate and release_gate.is_file():
level = "r3" if target_mode == "canonical" else "r2"
completed = subprocess.run(
[sys.executable, str(release_gate), "--root", str(stage), "--level", level],
check=False,
capture_output=True,
text=True,
)
if completed.returncode != 0:
try:
release_result = json.loads(completed.stdout)
nonpass = [
{
"name": item.get("name"),
"status": item.get("status"),
"exit_code": item.get("exit_code"),
}
for item in release_result.get("checks", [])
if item.get("status") not in {"PASS", "SKIP"}
]
detail = json.dumps(
{
"summary": release_result.get("summary"),
"nonpass": nonpass,
},
ensure_ascii=False,
sort_keys=True,
)
except (json.JSONDecodeError, AttributeError):
detail = completed.stdout[-4000:]
raise MigrationError(f"{level.upper()}_POSTFLIGHT_FAILED", detail)
def prepare(
root: Path,
target_mode: str,
*,
layout_path: Path = DEFAULT_LAYOUT,
relations_path: Path = DEFAULT_RELATIONS,
) -> MigrationPlan:
root = root.resolve(strict=True)
if target_mode not in {"shadow", "canonical"}:
raise MigrationError("INVALID_TARGET_MODE", "target mode must be shadow or canonical")
try:
# A shadow tree is expected to become temporarily stale when the
# authoritative legacy corpus gains a document or changes outside a
# harness-aware writer. Permit only that repairable class of drift
# for an explicit shadow refresh; every other transition remains
# fail-closed on a clean layout.
requested_config = _load_json(_config_path(root, layout_path), "vault-layout/v1")
requested_mode = requested_config.get("mode")
refresh_shadow = requested_mode == "shadow" and target_mode == "shadow"
authority = layout_check.resolve_authority(root, layout_path, require_clean=not refresh_shadow)
if refresh_shadow:
layout_result = layout_check.check_layout(root, layout_path)
recoverable = {"MIGRATION_ENTRY_MISSING", "SHADOW_MIRROR_DRIFT", "MIGRATION_HASH_MISMATCH"}
observed = {str(item.get("code")) for item in layout_result.get("findings", [])}
unsupported = sorted(observed - recoverable)
if unsupported:
raise MigrationError(
"LAYOUT_NOT_REFRESHABLE",
",".join(unsupported),
str(layout_path),
)
except layout_check.LayoutContractError as exc:
raise MigrationError(exc.code, str(exc), exc.location) from exc
current_mode = authority["mode"]
if (current_mode, target_mode) not in {
("compatibility", "shadow"),
("shadow", "shadow"),
("shadow", "canonical"),
("canonical", "shadow"),
}:
raise MigrationError("INVALID_MODE_TRANSITION", f"{current_mode} -> {target_mode}")
config_path = _config_path(root, layout_path).resolve()
config = _load_json(config_path, "vault-layout/v1")
relations = _load_json(_config_path(root, relations_path), "document-relations/v1")
configured = _configured_mapping(root, config)
derived: dict[Path, Path] = {}
changes: dict[Path, ReplacementValue] = {}
if current_mode == "compatibility" or (current_mode, target_mode) == ("shadow", "shadow"):
derived = build_mapping(root, config, relations)
mapping = derived
for legacy, canonical in mapping.items():
changes[canonical] = legacy.read_bytes()
else:
if current_mode == "shadow":
derived = build_mapping(root, config, relations)
if configured != derived:
raise MigrationError("MAPPING_POLICY_DRIFT", "configured migration mapping differs from deterministic project-first mapping")
mapping = configured
if current_mode == "shadow" and target_mode == "canonical":
for legacy, canonical in mapping.items():
# Preserve document bytes across an authority-only cutover.
# A compatibility symlink keeps external wikilinks and
# repository-owned readers of rules/templates functional,
# while write-root enforcement still rejects old-path writes.
# Because bytes do not change, semantic certificates remain
# current under their stable logical (legacy) subject IDs.
changes[canonical] = legacy.read_bytes()
relative_target = os.path.relpath(canonical, start=legacy.parent)
changes[legacy] = SymlinkValue(Path(relative_target).as_posix())
elif current_mode == "canonical" and target_mode == "shadow":
for legacy, canonical in mapping.items():
changes[legacy] = canonical.read_bytes()
updated = copy.deepcopy(config)
updated["mode"] = target_mode
migration, rollback = _embedded_documents(updated)
migration_entries, rollback_entries = _entries(mapping, root, changes)
migration["entries"] = migration_entries
rollback["entries"] = rollback_entries
changes[config_path] = _document_bytes(updated)
expected = {path: _entry_fingerprint(path) for path in changes}
for legacy in mapping:
expected.setdefault(legacy, _entry_fingerprint(legacy))
relations_file = _config_path(root, relations_path).resolve()
expected.setdefault(relations_file, _entry_fingerprint(relations_file))
forbidden = {path for path in changes if expected[path] is None}
with tempfile.TemporaryDirectory(prefix="vault-migration-stage-") as directory:
stage = Path(directory) / "repo"
stage.mkdir()
_stage_repository(root, stage)
for path, content in changes.items():
staged = stage / path.relative_to(root)
staged.parent.mkdir(parents=True, exist_ok=True)
if isinstance(content, SymlinkValue):
staged.unlink(missing_ok=True)
os.symlink(content.target, staged)
else:
# 실제 apply 는 replace_many(follow_symlinks=False) 로 *경로 자체* 를 실파일로
# 만든다(롤백 시 심링크→실파일). 스테이지는 심링크를 보존(_stage_repository
# symlinks=True)하므로, 심링크 위에 write_bytes 하면 정본으로 write-through 돼
# 스테이지가 실제 apply 와 어긋난다(롤백 후에도 legacy 가 심링크로 남는 것처럼
# 보임). 링크를 먼저 끊어 apply 의미를 그대로 재현한다.
if staged.is_symlink():
staged.unlink()
staged.write_bytes(content)
_validate_stage(
stage,
layout_path,
target_mode,
# A same-mode refresh only restores the shadow invariant. R2 is
# still the mandatory prerequisite for the later canonical
# transition, but must not make the repair operation depend on
# unrelated semantic certificates or corpus-level gates.
run_release_gate=(current_mode, target_mode) != ("shadow", "shadow"),
)
plan_sha256 = _plan_hash(root, current_mode, target_mode, expected, changes, authority)
result = {
"from_mode": current_mode,
"to_mode": target_mode,
"plan_sha256": plan_sha256,
"migration_entries": len(mapping),
"changed_paths": [path.relative_to(root).as_posix() for path in sorted(changes)],
"rollback_mapping_complete": True,
}
return MigrationPlan(changes, expected=expected, forbidden=forbidden, result=result)
def apply(plan: MigrationPlan) -> None:
for path, expected in plan.expected.items():
observed = _entry_fingerprint(path)
if observed != expected:
raise MigrationError("CONCURRENT_MODIFICATION", f"expected {expected}, observed {observed}", str(path))
# cutover/rollback 은 legacy 경로의 *엔트리 종류 자체* 를 바꾸는 것이 목적이다
# (실파일 → 심링크, 롤백 시 심링크 → 실파일). 여기서 심링크를 따라가면 롤백이
# 정본만 덮어쓰고 심링크는 남겨 마이그레이션이 성립하지 않는다.
replace_many(plan, must_not_exist=plan.forbidden, follow_symlinks=False)
def _emit(document: Mapping[str, Any]) -> None:
json.dump(document, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--layout", type=Path, default=DEFAULT_LAYOUT)
parser.add_argument("--relations", type=Path, default=DEFAULT_RELATIONS)
parser.add_argument("--to", choices=("shadow", "canonical"), required=True)
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--dry-run", action="store_true")
mode.add_argument("--apply", action="store_true")
parser.add_argument("--expected-plan-sha256")
args = parser.parse_args(argv)
try:
plan = prepare(args.root, args.to, layout_path=args.layout, relations_path=args.relations)
if args.apply:
expected = args.expected_plan_sha256
if expected is None:
raise MigrationError("EXPECTED_PLAN_SHA256_REQUIRED", "apply requires --expected-plan-sha256")
if not HEX_SHA256.fullmatch(expected):
raise MigrationError("INVALID_PLAN_SHA256", "expected plan hash must be 64 lowercase hex characters")
if expected != plan.result["plan_sha256"]:
raise MigrationError("PLAN_HASH_MISMATCH", f"expected {expected}, current {plan.result['plan_sha256']}")
apply(plan)
_emit({"schema_version": RESULT_SCHEMA, "status": "APPLIED" if args.apply else "DRY_RUN", **plan.result})
return 0
except (MigrationError, layout_check.LayoutContractError) as exc:
_emit({
"schema_version": RESULT_SCHEMA,
"status": "FAIL",
"errors": [{"code": getattr(exc, "code", "MIGRATION_FAILED"), "location": getattr(exc, "location", ""), "message": str(exc)}],
})
return 1
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
_emit({"schema_version": RESULT_SCHEMA, "status": "ERROR", "errors": [{"code": "IO_ERROR", "location": "", "message": str(exc)}]})
return 2
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,157 @@
#!/usr/bin/env python3
"""Verify that R2 workflow sources are connected to deterministic gateways."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import sys
from typing import Any, Iterable
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
RESULT_SCHEMA = "workflow-connection-result/v1"
WRITER_SOURCES = (
Path("harness/source/agents/bodies/wiki-doc-author.md"),
Path("harness/source/agents/bodies/wiki-source-summarizer.md"),
)
GLOBAL_REPORT_SOURCE = Path(".agents/plugins/wiki-superpowers/skills/wiki-workflow/SKILL.md")
WRITER_REQUIRED = (
"harness/runtime/document_commit.py",
"document-commit/v1",
"document-commit-result/v1",
"--dry-run",
"plan_sha256",
"--expected-plan-sha256",
"--apply",
)
WRITER_FORBIDDEN = (
"자동 rollback 미구현",
"**C4. Parent hub Cluster 갱신**",
"**M5. Parent hub Cluster 점검**",
"### Step 6: Parent hub Cluster 갱신",
)
REPORT_REQUIRED = (
"proof-request/v1",
"harness/runtime/proof_runner.py",
"proof-runner-result/v1",
"proof-manifest/v1",
"manifest_sha256",
"proof_count",
"pass_count",
"fail_count",
"1~3",
"실패",
)
SEMANTIC_WORKFLOW_SOURCES = (
Path("harness/source/skills/project-spec.md"),
Path("harness/source/skills/branch-spec.md"),
Path("harness/source/skills/sync.md"),
)
SEMANTIC_REQUIRED = (
"semantic_surface_extractor.py",
"semantic_candidate_builder.py",
"wiki-semantic-coherence-auditor",
"semantic_audit.py",
"semantic certificate",
)
PARENT_CERTIFICATE_SOURCE = Path("harness/source/skills/branch-from-project.md")
PARENT_CERTIFICATE_REQUIRED = (
"semantic_certificate.py",
"--mode hub",
"--path raw/project-notes/<project>.md",
)
class ConnectionCheckError(RuntimeError):
pass
def _finding(code: str, path: Path, token: str) -> dict[str, str]:
return {"code": code, "path": path.as_posix(), "token": token}
def _require_tokens(path: Path, text: str, tokens: Iterable[str]) -> list[dict[str, str]]:
return [_finding("MISSING_REQUIRED_CONNECTION", path, token) for token in tokens if token not in text]
def _report_sources(root: Path) -> list[Path]:
body_root = root / "harness/source/agents/bodies"
sources = [
path.relative_to(root)
for path in body_root.glob("*.md")
if "§7.1" in path.read_text(encoding="utf-8") or "Self-Grep" in path.read_text(encoding="utf-8")
]
sources.append(GLOBAL_REPORT_SOURCE)
return sorted(set(sources), key=lambda item: item.as_posix())
def check(root: Path) -> dict[str, Any]:
root = root.resolve(strict=True)
findings: list[dict[str, str]] = []
for relative in WRITER_SOURCES:
path = root / relative
if not path.is_file():
findings.append(_finding("MISSING_CONNECTION_SOURCE", relative, ""))
continue
text = path.read_text(encoding="utf-8")
findings.extend(_require_tokens(relative, text, WRITER_REQUIRED))
findings.extend(
_finding("FORBIDDEN_DIRECT_WRITE_CONTRACT", relative, token)
for token in WRITER_FORBIDDEN
if token in text
)
for relative in SEMANTIC_WORKFLOW_SOURCES:
path = root / relative
if not path.is_file():
findings.append(_finding("MISSING_CONNECTION_SOURCE", relative, ""))
continue
findings.extend(_require_tokens(relative, path.read_text(encoding="utf-8"), SEMANTIC_REQUIRED))
parent_path = root / PARENT_CERTIFICATE_SOURCE
if not parent_path.is_file():
findings.append(_finding("MISSING_CONNECTION_SOURCE", PARENT_CERTIFICATE_SOURCE, ""))
else:
findings.extend(_require_tokens(PARENT_CERTIFICATE_SOURCE, parent_path.read_text(encoding="utf-8"), PARENT_CERTIFICATE_REQUIRED))
reporters = _report_sources(root)
for relative in reporters:
path = root / relative
if not path.is_file():
findings.append(_finding("MISSING_CONNECTION_SOURCE", relative, ""))
continue
findings.extend(_require_tokens(relative, path.read_text(encoding="utf-8"), REPORT_REQUIRED))
findings.sort(key=lambda item: (item["path"], item["code"], item["token"]))
return {
"schema_version": RESULT_SCHEMA,
"status": "PASS" if not findings else "FAIL",
"writer_sources": len(WRITER_SOURCES),
"semantic_workflow_sources": len(SEMANTIC_WORKFLOW_SOURCES) + 1,
"report_sources": len(reporters),
"findings": findings,
}
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
args = parser.parse_args(argv)
try:
result = check(args.root)
exit_code = 0 if result["status"] == "PASS" else 1
except (ConnectionCheckError, OSError, UnicodeError, ValueError) as exc:
result = {
"schema_version": RESULT_SCHEMA,
"status": "ERROR",
"errors": [{"code": "WORKFLOW_CONNECTION_ERROR", "message": str(exc)}],
}
exit_code = 2
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
if __name__ == "__main__":
raise SystemExit(main())
+316
View File
@@ -0,0 +1,316 @@
#!/usr/bin/env python3
"""Resolve and optionally execute neutral workflow execution contracts."""
from __future__ import annotations
import argparse
import json
from pathlib import Path
import re
import subprocess
import sys
from typing import Any, Callable
import execution_profile
DEFAULT_ROOT = Path(__file__).resolve().parents[2]
MANIFEST = Path("harness/source/generation-manifest.json")
RESULT_SCHEMA = "workflow-dispatch-result/v1"
HEX_SHA256 = re.compile(r"^[0-9a-f]{64}$")
Runner = Callable[..., subprocess.CompletedProcess[str]]
class DispatchError(ValueError):
"""A workflow declaration or runtime result is not safe to dispatch."""
def _safe_path(root: Path, value: Any, *, field: str, must_exist: bool = True) -> Path:
if not isinstance(value, str) or not value or "\\" in value:
raise DispatchError(f"{field} must be a non-empty repo-relative POSIX path")
relative = Path(value)
if relative.is_absolute() or ".." in relative.parts:
raise DispatchError(f"{field} escapes repository: {value}")
resolved = (root / relative).resolve()
try:
resolved.relative_to(root)
except ValueError as exc:
raise DispatchError(f"{field} escapes repository: {value}") from exc
if must_exist and not resolved.is_file():
raise DispatchError(f"{field} does not exist: {value}")
return resolved
def _workflow_inventory(root: Path) -> dict[str, Path]:
manifest_path = root / MANIFEST
document = json.loads(manifest_path.read_text(encoding="utf-8"))
if document.get("schema_version") != 1 or not isinstance(document.get("sources"), list):
raise DispatchError("generation manifest must use schema_version 1 with sources")
workflows: dict[str, Path] = {}
for index, item in enumerate(document["sources"]):
if not isinstance(item, dict):
raise DispatchError(f"sources[{index}] must be an object")
metadata = _safe_path(root, item.get("metadata"), field=f"sources[{index}].metadata")
data = json.loads(metadata.read_text(encoding="utf-8"))
if data.get("source_kind") != "workflow":
continue
identifier = data.get("id")
if not isinstance(identifier, str) or not identifier:
raise DispatchError(f"{metadata.relative_to(root)}: workflow id is required")
if identifier in workflows:
raise DispatchError(f"duplicate workflow id: {identifier}")
workflows[identifier] = metadata
if not workflows:
raise DispatchError("generation manifest declares no workflows")
return workflows
def _contract(root: Path, workflow: str) -> tuple[dict[str, Any], dict[str, Any]]:
inventory = _workflow_inventory(root)
metadata = inventory.get(workflow)
if metadata is None:
raise DispatchError(f"workflow is not declared by generation manifest: {workflow}")
expected = (root / "harness/source/workflows" / f"{workflow}.json").resolve()
if metadata != expected:
raise DispatchError(f"workflow metadata must use neutral workflow directory: {metadata}")
_identifier, contract = execution_profile.load_workflow_contract(
workflow,
root / "harness/source/workflows",
)
if contract["kind"] == "deterministic":
entrypoint = _safe_path(root, contract.get("entrypoint"), field="execution_contract.entrypoint")
if entrypoint.suffix != ".py":
raise DispatchError("deterministic entrypoint must be a Python source file")
return contract, json.loads(metadata.read_text(encoding="utf-8"))
def build_plan(
root: Path,
workflow: str,
*,
risk: str | None = None,
finding_count: int = 0,
public_claims_present: bool = False,
claims_present: bool = False,
phase: str = "baseline",
) -> dict[str, Any]:
if phase not in {"baseline", "final"}:
raise DispatchError(f"unsupported resolution phase: {phase}")
if phase == "baseline" and (finding_count or public_claims_present or claims_present):
raise DispatchError("baseline resolution cannot declare findings or claims")
root = root.resolve(strict=True)
contract, _metadata = _contract(root, workflow)
profiles = execution_profile.load_profiles(root / "harness/source/execution-profiles.json")
policy = execution_profile.resolve_workflow_policy(
profiles,
workflow,
root / "harness/source/workflows",
risk,
finding_count,
public_claims_present,
claims_present,
)
execution: dict[str, Any] = {
"kind": contract["kind"],
"explicit_execute_required": contract["kind"] == "deterministic",
}
if contract["kind"] == "deterministic":
execution.update(
{
"entrypoint": contract["entrypoint"],
"dry_run_first": contract["dry_run_first"],
"result_schema": contract["result_schema"],
}
)
return {
"schema_version": RESULT_SCHEMA,
"status": "PLANNED",
"workflow": workflow,
"phase": phase,
"profile": policy["profile"],
"context": policy["context"],
"always_checks": policy["always_checks"],
"mandatory_gates": policy["mandatory_gates"],
"review_intensity": policy["review_intensity"],
"dispatch": policy["dispatch"],
"semantic_review": policy["semantic_review"],
"adversarial_review": policy["adversarial_review"],
"output_contract": policy["output_contract"],
"execution": execution,
}
def check_all(root: Path) -> dict[str, Any]:
root = root.resolve(strict=True)
findings: list[dict[str, str]] = []
workflows: list[str] = []
try:
workflows = sorted(_workflow_inventory(root))
except (DispatchError, OSError, UnicodeError, json.JSONDecodeError) as exc:
findings.append({"code": "WORKFLOW_INVENTORY_ERROR", "workflow": "", "message": str(exc)})
for workflow in workflows:
try:
build_plan(root, workflow)
except (DispatchError, execution_profile.ProfileError, OSError, UnicodeError, json.JSONDecodeError) as exc:
findings.append({"code": "WORKFLOW_CONTRACT_ERROR", "workflow": workflow, "message": str(exc)})
return {
"schema_version": RESULT_SCHEMA,
"status": "PASS" if not findings else "FAIL",
"workflow_count": len(workflows),
"findings": findings,
}
def _child_json(completed: subprocess.CompletedProcess[str], schema: str, phase: str) -> dict[str, Any]:
try:
document = json.loads(completed.stdout)
except (json.JSONDecodeError, TypeError) as exc:
raise DispatchError(f"{phase} returned non-JSON output") from exc
if not isinstance(document, dict) or document.get("schema_version") != schema:
raise DispatchError(f"{phase} result schema mismatch: expected {schema}")
return document
def _phase_outcome(
plan: dict[str, Any],
completed: subprocess.CompletedProcess[str],
document: dict[str, Any],
phase: str,
success_status: str,
) -> tuple[dict[str, Any] | None, int]:
"""Preserve the shared 0/1/2 CLI envelope across child processes."""
if completed.returncode not in {0, 1, 2}:
raise DispatchError(f"{phase} returned unsupported exit code {completed.returncode}")
if completed.returncode == 2:
return {
**plan,
"status": "ERROR",
"phase": phase,
"runtime_result": document,
}, 2
if completed.returncode == 1:
return {
**plan,
"status": "FAIL",
"phase": phase,
"runtime_result": document,
}, 1
if document.get("status") != success_status:
raise DispatchError(f"{phase} exit 0 must return status {success_status}")
return None, 0
def execute_deterministic(
root: Path,
plan: dict[str, Any],
arguments: list[str],
*,
runner: Runner = subprocess.run,
) -> tuple[dict[str, Any], int]:
if plan["execution"]["kind"] != "deterministic":
return {**plan, "status": "FAIL", "error": {"code": "AGENTIC_EXECUTION_NOT_SUPPORTED"}}, 1
required_reviews = [name for name, action in plan["dispatch"].items() if action == "dispatch"]
if required_reviews:
return {
**plan,
"status": "REVIEW_REQUIRED",
"error": {"code": "REVIEW_DISPATCH_REQUIRED", "reviews": required_reviews},
}, 1
entrypoint = (root / plan["execution"]["entrypoint"]).resolve()
schema = plan["execution"]["result_schema"]
dry_command = [sys.executable, str(entrypoint), *arguments, "--dry-run"]
dry = runner(dry_command, cwd=root, check=False, capture_output=True, text=True, timeout=120)
dry_document = _child_json(dry, schema, "dry-run")
outcome, exit_code = _phase_outcome(plan, dry, dry_document, "dry-run", "DRY_RUN")
if outcome is not None:
return outcome, exit_code
plan_hash = dry_document.get("plan_sha256")
if not isinstance(plan_hash, str) or not HEX_SHA256.fullmatch(plan_hash):
raise DispatchError("dry-run did not return a valid plan_sha256")
apply_command = [
sys.executable,
str(entrypoint),
*arguments,
"--apply",
"--expected-plan-sha256",
plan_hash,
]
applied = runner(apply_command, cwd=root, check=False, capture_output=True, text=True, timeout=120)
applied_document = _child_json(applied, schema, "apply")
outcome, exit_code = _phase_outcome(plan, applied, applied_document, "apply", "APPLIED")
if outcome is not None:
return outcome, exit_code
if applied_document.get("plan_sha256") != plan_hash:
raise DispatchError("apply result plan_sha256 does not match dry-run")
return {
**plan,
"status": "APPLIED",
"plan_sha256": plan_hash,
"runtime_result": applied_document,
}, 0
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("workflow", nargs="?")
parser.add_argument("arguments", nargs="*")
parser.add_argument("--root", type=Path, default=DEFAULT_ROOT)
parser.add_argument("--risk")
parser.add_argument("--finding-count", type=int, default=0)
parser.add_argument("--public-claims-present", action="store_true")
parser.add_argument("--claims-present", action="store_true")
parser.add_argument("--phase", choices=("baseline", "final"), default="baseline")
parser.add_argument("--execute", action="store_true")
parser.add_argument("--check-all", action="store_true")
args = parser.parse_args(argv)
try:
root = args.root.resolve(strict=True)
if args.check_all:
if args.workflow or args.arguments or args.execute:
raise DispatchError("--check-all cannot be combined with a workflow or --execute")
result = check_all(root)
exit_code = 0 if result["status"] == "PASS" else 1
else:
if not args.workflow:
raise DispatchError("workflow is required unless --check-all is used")
plan = build_plan(
root,
args.workflow,
risk=args.risk,
finding_count=args.finding_count,
public_claims_present=args.public_claims_present,
claims_present=args.claims_present,
phase=args.phase,
)
if args.execute:
result, exit_code = execute_deterministic(root, plan, args.arguments)
else:
result, exit_code = plan, 0
json.dump(result, sys.stdout, ensure_ascii=False, indent=2, sort_keys=True)
sys.stdout.write("\n")
return exit_code
except (
DispatchError,
execution_profile.ProfileError,
OSError,
UnicodeError,
json.JSONDecodeError,
subprocess.SubprocessError,
) as exc:
json.dump(
{
"schema_version": RESULT_SCHEMA,
"status": "ERROR",
"errors": [{"code": "WORKFLOW_DISPATCH_ERROR", "message": str(exc)}],
},
sys.stdout,
ensure_ascii=False,
indent=2,
sort_keys=True,
)
sys.stdout.write("\n")
return 2
if __name__ == "__main__":
raise SystemExit(main())