document-haness/docs/_design/2026-08-28-tech-log-document-pipeline-design.md

# Tech Log Document Pipeline Design

## Goal

Build a durable, project-scoped pipeline that turns a codebase into a deeply evidenced analysis document, decomposes that analysis into a root tree of Tech Log records, generates the records and evidence assets, and later performs an editorial pass without depending on any legacy workspace.

## Non-goals

- Do not publish records to Tech Log automatically.
- Do not invent incidents, decisions, measurements, or first-person experiences that are not supported by the codebase, command output, browser evidence, Git history, or explicitly supplied source material.
- Do not make the pipeline depend on any legacy document repository or path.
- Do not treat the root tree as a brainstorming list. Every node must be traceable to analysis evidence.

## Workspace contract

```text
/shared/
├── codebase/
│   └── <project>/
│
├── document-detail/
│   └── <project>/
│       ├── README.md
│       ├── state.json
│       ├── source-index.md
│       ├── analysis/
│       │   ├── 00-project-overview.md
│       │   └── <module-or-scope>.md
│       ├── final/
│       │   └── document.md
│       ├── root-tree.md
│       ├── notes/
│       ├── checkpoints/
│       └── evidence/
│           ├── raw/
│           ├── terminal/
│           ├── browser/
│           ├── svg/
│           └── meta/
│
└── Tech-Log-Document/
    ├── AGENTS.md
    ├── README.md
    ├── .agents/skills/
    │   ├── writing-tech-log-from-analysis/
    │   └── humanizing-korean-tech-writing/
    ├── tools/terminal-evidence/
    ├── research/korean-tech-writing/
    ├── _templates/project/
    └── <project>/
        ├── case/
        ├── reference/
        ├── openquestion/
        ├── decision/
        ├── assets/
        │   ├── raw/
        │   ├── terminal/
        │   ├── browser/
        │   └── svg/
        └── _meta/
```

## Pipeline stages

### Stage A — Detailed codebase analysis

Input: `/shared/codebase/<project>`.

Output: `/shared/document-detail/<project>`.

For a small codebase, analysis may converge in one run. For a large codebase, analyze one bounded module or subsystem per run and update `state.json` and `source-index.md`. The final document is a synthesis of completed module analyses, not a fresh rewrite that discards their provenance.

Required analysis properties:

- map project/module/package boundaries and dependency direction;
- trace representative request, state, persistence, messaging, error, and operational paths when present;
- identify implemented behavior separately from declared-but-unwired contracts;
- inspect tests, build rules, configuration, Git history, and runtime behavior when they materially change the interpretation;
- distinguish observed facts, code-derived inference, hypotheses, and external knowledge;
- capture command/browser evidence for claims that benefit from execution verification;
- preserve exact versions, paths, commands, status codes, measurements, and identifiers in evidence.

### Stage B — Root tree derivation

Input: `final/document.md`, module analyses, source index, evidence.

Output: `root-tree.md`.

The root tree is the decomposition contract for all downstream Tech Log records. It groups records by Topic and by kind: CASE, REFERENCE, OPEN QUESTION, DECISION.

A node is not valid merely because its title sounds useful. Each node records:

- slug;
- source anchors into the detailed analysis;
- code/evidence references when relevant;
- why it belongs to that record kind;
- readiness status;
- missing verification, if any;
- relations to sibling nodes.

Allowed readiness values:

- `READY`: enough grounded material exists to author the record;
- `NEEDS_EVIDENCE`: the idea is grounded, but a material claim still needs execution or browser evidence;
- `NEEDS_DECISION`: a Decision title is plausible but no project decision has actually been made;
- `OPEN`: valid Question with unresolved unknowns;
- `BLOCKED`: source material is insufficient or contradictory;
- `REJECTED`: candidate must not become a record.

Only `READY` Case/Reference nodes, actual adopted/proposed project Decision nodes with explicit decision evidence, and legitimate `OPEN` Question nodes may enter Stage C.

### Record classification contract

**CASE** — a concrete incident, implementation experiment, failure, diagnosis, or verification sequence exists. It must have a specific observed problem/condition, evidence, and bounded conclusion. Case is the only record kind that may carry rich body Markdown such as code, tables, diagrams, and images.

**REFERENCE** — a reusable criterion, distinction, or operating/design rule can be extracted from one or more grounded cases or code observations. It must generalize beyond retelling one incident.

**OPEN QUESTION** — a material design or operational uncertainty remains unresolved. It must state known facts, unknowns, constraints, candidate directions when grounded, and the next verification/decision criterion. It must not smuggle in an answer.

**DECISION** — the project has actually selected or proposed a direction. It requires explicit decision evidence and at least one supporting relation. A best-practice recommendation is not a project Decision.

### Stage C — Tech Log record generation

Input: `root-tree.md` plus cited analysis/evidence.

Output: `/shared/Tech-Log-Document/<project>/{case,reference,openquestion,decision}` plus assets.

Generation rules:

- read the local writing skill before authoring;
- generate only root-tree nodes whose status permits generation;
- re-open the cited source anchors instead of relying on the root-tree title alone;
- never invent a technical reason merely because a technology is present;
- never invent first-person experience;
- preserve protected literals exactly: numbers, dates, versions, units, source paths, code, commands, URLs, status codes, identifiers, quoted text;
- Case rich evidence must be backed by actual raw evidence or a diagram whose semantics are derived from grounded sources;
- Reference/Question/Decision fields remain plain text unless the target Tech Log contract changes;
- relation metadata is generated from root-tree relations and source provenance.

### Stage D — Editorial refinement

Input: generated Tech Log record.

Output: same record, content-preserving editorial revision.

The editorial pass must read `humanizing-korean-tech-writing` first. It may alter diction, sentence rhythm, paragraphing, headings, repetition, and awkward connective phrases. It may not delete technical facts for concision, change evidence, change status/decision semantics, manufacture personal experience, or silently broaden/narrow a claim.

Research on Korean engineering writing is stored under `research/korean-tech-writing/` and distilled into the skill. Runtime editing must not depend on a specific external blog being reachable.

## Evidence model

### Raw first

Evidence is always captured in a raw form before presentation assets are produced.

```text
real command / browser observation
          ↓
evidence/raw/<artifact>
          ↓
renderer or curated diagram
          ↓
evidence/terminal | browser | svg
```

### Terminal evidence

A command run is stored with command, cwd, execution time, exit code, and output. A deterministic renderer converts that real output into an SVG terminal card. The renderer must:

- XML-escape all output;
- preserve the original raw output separately;
- redact obvious secret-bearing environment assignments and authorization/token values from the visual output;
- visually mark truncation when the renderer caps lines;
- never fabricate output lines.

### Browser evidence

Use browser automation only when an application can actually be run and the UI/network behavior is relevant. Store screenshots under the project evidence path and record the URL/state/assertion that makes the screenshot evidentiary rather than decorative.

### Diagrams

SVG diagrams may explain architecture, boundaries, sequences, state, or before/after behavior. A diagram is explanatory evidence, not primary proof. Its labels and relationships must be traceable to code or observed behavior.

## State and idempotency

Each project has state files so scheduled runs can resume safely. A run must inspect Git status and existing state before editing. It must not overwrite uncommitted user work.

Detailed-analysis state records at least:

- project path;
- current code revision when Git is available;
- analyzed scopes;
- pending scopes;
- final synthesis status;
- root-tree status;
- evidence tasks.

Tech-Log generation state records at least:

- root-tree revision/hash;
- generated nodes;
- pending nodes;
- editorial status per record;
- last validation result.

## Safety boundaries

- No automatic `git reset`, `git clean`, branch deletion, push, merge, or destructive filesystem operation.
- No automatic production changes.
- Do not display or persist credentials in evidence.
- Do not silently overwrite source code while doing documentation analysis.
- If the codebase changes materially after analysis, mark affected analysis/root-tree records stale before generating new records.

## Independence requirement

The new pipeline must be self-contained. Its instructions, skills, templates, tools, and scheduled prompts must not reference or require any legacy document workspace. Existing historical material may be consulted once during migration, but all durable rules must live under `/shared/document-detail` or `/shared/Tech-Log-Document` afterward.