init: 클린 아키텍처 백엔드
This commit is contained in:
DongHyeonka
@@ -0,0 +1,120 @@
|
||||
# Registry: Repository Access Capabilities
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/capabilities.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Owner branch: feature-repository-access-permission-contract
|
||||
# Last updated: 2026-06-05
|
||||
#
|
||||
# Notes
|
||||
# - capability는 사용자 권한이 아니라 application use case가 infrastructure capability를
|
||||
# 사용할 수 있는지에 대한 계약 (feature-repository-access-permission-contract).
|
||||
# - enforcement default = ArchUnit annotation-based rule. compile-time annotation processor는
|
||||
# alternative. runtime AOP는 forbidden.
|
||||
# - capability 제거는 항상 breaking change. 추가는 additive (registry row 동반 시).
|
||||
# - annotation 표기 (as-built, F1/F2 reconciled 2026-06-05): 코드 SSOT는 단일
|
||||
# `@UseCaseCapability` (TYPE target, typed attribute). 노트 D2/D11의 flat
|
||||
# `@UseCaseRepositoryAccess(Capability[])` 모델은 superseded. 7 capability ↔ as-built 매핑:
|
||||
# READ_REPOSITORY/WRITE_REPOSITORY → repositoryAccess, TRANSACTION_REQUIRED → transactionMode,
|
||||
# EXTERNAL_OUTBOUND_ALLOWED → externalOutboundAllowed, SENSITIVE_READ → sensitiveRead,
|
||||
# BULK_WRITE → bulkWrite, CROSS_TENANT_ADMIN → crossTenantAdmin.
|
||||
# 각 row의 annotation: 필드는 아래에서 as-built 표기로 정합됨.
|
||||
|
||||
capabilities:
|
||||
|
||||
# source: feature-repository-access-permission-contract — 판정 기준 "Required capability: READ_REPOSITORY"
|
||||
# source: feature-application-port-usecase-contract — "read-only query use case는 readOnly 와 READ_REPOSITORY capability만 선언 가능"
|
||||
- name: READ_REPOSITORY
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(repositoryAccess = READ_REPOSITORY)"
|
||||
semantics: "use case가 read-only repository operation을 호출하는 것을 허용. query use case의 기본 capability. write/sensitive/bulk 작업은 별도 capability 선언이 없으면 forbidden."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:read-repository-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — 판정 기준 "Required capability: WRITE_REPOSITORY"
|
||||
# source: feature-application-port-usecase-contract — "write use case는 transactionMode, idempotency, repositoryAccess를 명시해야 함"
|
||||
- name: WRITE_REPOSITORY
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(repositoryAccess = WRITE_REPOSITORY)"
|
||||
semantics: "use case가 mutating repository operation(insert/update/delete)을 호출하는 것을 허용. 단일/소량 write 기준이며 batch size > 100은 BULK_WRITE 별도 선언 필요. read-only use case에서 이 capability 없이 write repository 접근하면 fail."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:write-repository-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "SENSITIVE_READ marker = registry-managed metadata table (entity FQN + field name 단위)"
|
||||
- name: SENSITIVE_READ
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(sensitiveRead = true)"
|
||||
semantics: "PII/credential 등 sensitive field를 읽는 use case가 선언해야 하는 capability. marker는 registry-managed metadata table(entity FQN + field name 단위)에서 lookup. domain annotation 또는 JPA entity annotation 형태는 forbidden(domain에 framework 의존 회피). pseudonymized data read는 documented 시에만 예외 허용."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:sensitive-read-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "BULK_WRITE threshold = N > 100 또는 batch size > 100. 미만은 일반 WRITE_REPOSITORY로 충분"
|
||||
- name: BULK_WRITE
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(bulkWrite = true)"
|
||||
semantics: "단일 transaction 내 N > 100 또는 batch size > 100 mutating operation을 수행하는 use case가 선언해야 하는 capability. 이 미만이면 일반 WRITE_REPOSITORY로 충분. lock 점유 시간, pool 영향, retry 비용이 큰 작업을 명시화."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: WRITE_REPOSITORY
|
||||
threshold: 100
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:bulk-write-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "TRANSACTION_REQUIRED는 application-port branch의 TransactionPort contract와 연결되어야 하며 Spring @Transactional 직접 import로 충족하지 않음"
|
||||
# source: feature-application-port-usecase-contract — TransactionPort Contract
|
||||
- name: TRANSACTION_REQUIRED
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(transactionMode = WRITE | READ_ONLY | REQUIRES_NEW)"
|
||||
semantics: "use case가 TransactionPort(또는 TransactionalUseCaseRunner)를 통해 transactional boundary를 갖는 것을 강제. Spring @Transactional의 application package 직접 import는 forbidden. infrastructure가 Spring transaction implementation을 제공하고 application은 port만 호출."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:transaction-required-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "EXTERNAL_OUTBOUND_ALLOWED 분류 = outbox row INSERT는 in-process(불요), polling publisher의 broker publish는 outbound(필요)"
|
||||
# source: feature-application-port-usecase-contract — "outbound adapter 호출 use case에 EXTERNAL_OUTBOUND_ALLOWED가 없으면 실패"
|
||||
- name: EXTERNAL_OUTBOUND_ALLOWED
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(externalOutboundAllowed = true)"
|
||||
semantics: "use case가 외부 HTTP/message broker로 outbound 호출을 발생시키는 것을 허용. outbox claim 분류: outbox row INSERT는 in-process이므로 본 capability 불요. polling publisher의 broker publish는 outbound이므로 필요. domain event without transport detail은 outbound 호출이 아니므로 별도 분류."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:external-outbound-capability
|
||||
|
||||
# source: feature-repository-access-permission-contract — decisions 2026-05-22
|
||||
# "CROSS_TENANT_ADMIN capability를 capability vocabulary에 추가 (tenant branch feature-tenant-context-policy와 cross-link)"
|
||||
- name: CROSS_TENANT_ADMIN
|
||||
scope: use_case_method
|
||||
enforcement: archunit
|
||||
annotation: "@UseCaseCapability(crossTenantAdmin = true)"
|
||||
semantics: "tenant 경계를 넘어 데이터에 접근/변경하는 admin use case가 선언해야 하는 capability. tenant-context-policy의 cross-tenant 정책과 cross-link되어야 하며, 단일 tenant 범위 use case에서 이 capability를 선언하면 review에서 reject. SENSITIVE_READ가 동반될 가능성이 높지만 자동 결합은 아님."
|
||||
owner_branch: feature-repository-access-permission-contract
|
||||
bound_to_capability: null
|
||||
threshold: null
|
||||
compatibility_impact: breaking
|
||||
required_test: architecture-enforcement:cross-tenant-admin-capability
|
||||
|
||||
# Row count verification
|
||||
# - feature-repository-access-permission-contract 판정 기준 "Required capability" 표에 명시된 7개:
|
||||
# READ_REPOSITORY, WRITE_REPOSITORY, SENSITIVE_READ, BULK_WRITE, TRANSACTION_REQUIRED,
|
||||
# EXTERNAL_OUTBOUND_ALLOWED, CROSS_TENANT_ADMIN.
|
||||
# - source에 명시되지 않은 capability는 본 registry에 추가하지 않음 (추측 금지).
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,918 @@
|
||||
# Registry: Error Codes
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/error-codes.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Category enum owner: feature-operational-error-observability-foundation
|
||||
# Last updated: 2026-05-22
|
||||
# Note: 이 파일은 Phase B 산출물. Phase C2(ca-tmpl 실 코드)에서 generated Java constants의 source.
|
||||
#
|
||||
# Schema (per row):
|
||||
# code: UPPER_SNAKE_CASE
|
||||
# category: VALIDATION | AUTH | AUTHZ | NOT_FOUND | CONFLICT |
|
||||
# RATE_LIMIT | TRANSIENT_DEPENDENCY | PERMANENT_DEPENDENCY |
|
||||
# DATA_INTEGRITY | INTERNAL
|
||||
# http_status: int (async-only failures use 500 placeholder)
|
||||
# retryable: bool
|
||||
# retry_after_seconds: int | null (RATE_LIMIT/TRANSIENT 권고 backoff)
|
||||
# owner_branch: source branch (raw/branch-notes/feature-*.md)
|
||||
# owner_layer: presentation | application | domain | infrastructure | crosscut
|
||||
# client_safe_message: no token / no principal raw / no internal path / no stack trace
|
||||
# log_level: ERROR | WARN | INFO
|
||||
# runbook_link: runbook://area/scenario OR null (client-error만 null 허용)
|
||||
# compatibility_impact: none | additive | behavior-change | breaking
|
||||
# required_test: owning contract test identifier
|
||||
#
|
||||
# Runbook policy (operational-runbook-contract L80):
|
||||
# retryable=false + category ∈ {AUTH, AUTHZ, RATE_LIMIT, INTERNAL,
|
||||
# TRANSIENT_DEPENDENCY, PERMANENT_DEPENDENCY} ⇒ runbook_link 필수.
|
||||
# VALIDATION/NOT_FOUND/CONFLICT/DATA_INTEGRITY는 client-error로 runbook 면제 가능.
|
||||
# retryable=true 인 모든 row는 runbook_link 필수.
|
||||
|
||||
errors:
|
||||
|
||||
# ============================================================
|
||||
# AUTH (feature-security-operational-baseline / Decision Matrix)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-security-operational-baseline L82 — "token 누락 | 401 | AUTH_TOKEN_MISSING | AUTH"
|
||||
- code: AUTH_TOKEN_MISSING
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication required"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/token-missing"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L83 — "token malformed (parse fail) | 401 | AUTH_TOKEN_MALFORMED | AUTH"
|
||||
- code: AUTH_TOKEN_MALFORMED
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/token-malformed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L84 — "token expired (clock skew tolerance 60s 초과) | 401 | AUTH_TOKEN_EXPIRED | AUTH"
|
||||
- code: AUTH_TOKEN_EXPIRED
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication expired"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/token-expired"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L85 — "invalid signature | 401 | AUTH_TOKEN_INVALID_SIGNATURE | AUTH"
|
||||
- code: AUTH_TOKEN_INVALID_SIGNATURE
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/token-invalid-signature"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L86 — "issuer mismatch | 401 | AUTH_ISSUER_MISMATCH | AUTH"
|
||||
- code: AUTH_ISSUER_MISMATCH
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/issuer-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L87 — "audience mismatch | 401 | AUTH_AUDIENCE_MISMATCH | AUTH"
|
||||
- code: AUTH_AUDIENCE_MISMATCH
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/audience-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L88 — "unknown kid (JWKS 미캐시) | 401 + Retry-After 5s | AUTH_KID_UNKNOWN | AUTH"
|
||||
- code: AUTH_KID_UNKNOWN
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: true # 2026-06-01: false→true. JWKS 키 회전 중 unknown kid 는 ~5s 후 JWKS refresh 로 해소 가능(transient). retry_after_seconds=5 + client_safe_message "please retry" 와 정합. 키 고정 정책으로 전환 시 false 복귀.
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://auth/kid-unknown"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L89 — "JWKS endpoint outage ... | AUTH_JWKS_UNAVAILABLE | TRANSIENT_DEPENDENCY"
|
||||
- code: AUTH_JWKS_UNAVAILABLE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 30
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Authentication service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/jwks-unavailable"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-security-operational-baseline L90 — "claim mapping failure ... | 401 | AUTH_CLAIM_MAPPING_FAILED | AUTH"
|
||||
- code: AUTH_CLAIM_MAPPING_FAILED
|
||||
category: AUTH
|
||||
http_status: 401
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Authentication failed"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/claim-mapping-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# ============================================================
|
||||
# AUTHZ (feature-security-operational-baseline)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-security-operational-baseline L91 — "valid token + 권한 부족 | 403 | AUTHZ_INSUFFICIENT_PERMISSION | AUTHZ"
|
||||
- code: AUTHZ_INSUFFICIENT_PERMISSION
|
||||
category: AUTHZ
|
||||
http_status: 403
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Permission denied"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://authz/insufficient-permission"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:authz-category
|
||||
|
||||
# source: feature-security-operational-baseline L92 — "valid token + tenant cross-access | 403 | AUTHZ_TENANT_MISMATCH | AUTHZ"
|
||||
- code: AUTHZ_TENANT_MISMATCH
|
||||
category: AUTHZ
|
||||
http_status: 403
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Permission denied"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://authz/tenant-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:authz-category
|
||||
|
||||
# ============================================================
|
||||
# INTERNAL (feature-security-operational-baseline + container-runtime)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-security-operational-baseline L93 — "public path misconfiguration ... | 500 + P1 alert | INTERNAL_AUTH_MISCONFIGURATION | INTERNAL"
|
||||
- code: INTERNAL_AUTH_MISCONFIGURATION
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
owner_layer: crosscut
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://auth/public-path-misconfiguration"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:auth-category
|
||||
|
||||
# source: feature-container-runtime-contract L113 — "JVM OutOfMemoryError → ExitOnOutOfMemoryError로 137 exit, log에 error.code=JVM_OOM 명시"
|
||||
- code: JVM_OOM
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-container-runtime-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://runtime/jvm-oom"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:container-runtime-oom
|
||||
|
||||
# ============================================================
|
||||
# DB / Persistence (feature-persistence-failure-baseline / SQLState Matrix)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-persistence-failure-baseline L85 — "08* | all | TRANSIENT_DEPENDENCY | DB_UNAVAILABLE | true"
|
||||
- code: DB_UNAVAILABLE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://db/unavailable"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L86 — "40001 | Postgres/MySQL | CONFLICT | DB_SERIALIZATION_FAILURE | true"
|
||||
- code: DB_SERIALIZATION_FAILURE
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request conflicted with another transaction, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://db/serialization-failure"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L87 — "40P01 | Postgres | CONFLICT | DB_DEADLOCK | true (backoff)"
|
||||
- code: DB_DEADLOCK
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request conflicted, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://db/deadlock"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L88 — "23502 | Postgres | DATA_INTEGRITY | DB_NULL_VIOLATION | false"
|
||||
- code: DB_NULL_VIOLATION
|
||||
category: DATA_INTEGRITY
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request violates a required field constraint"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L89 — "23503 | Postgres | DATA_INTEGRITY | DB_FK_VIOLATION | false"
|
||||
- code: DB_FK_VIOLATION
|
||||
category: DATA_INTEGRITY
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request references missing resource"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L90 — "23505 | Postgres | CONFLICT | DB_UNIQUE_VIOLATION | false (business mapping)"
|
||||
- code: DB_UNIQUE_VIOLATION
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Resource already exists"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L91 — "23514 | Postgres | DATA_INTEGRITY | DB_CHECK_VIOLATION | false"
|
||||
- code: DB_CHECK_VIOLATION
|
||||
category: DATA_INTEGRITY
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request violates a value constraint"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L92 — "25P03 | Postgres | TRANSIENT_DEPENDENCY | DB_IDLE_IN_TX_TIMEOUT | true"
|
||||
- code: DB_IDLE_IN_TX_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://db/idle-in-tx-timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# source: feature-persistence-failure-baseline L93 — "57014 | Postgres | TRANSIENT_DEPENDENCY | DB_QUERY_CANCELED | false"
|
||||
- code: DB_QUERY_CANCELED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Request was canceled, please retry later"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://db/query-canceled"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:persistence-mapping
|
||||
|
||||
# ============================================================
|
||||
# Rate limit / Idempotency (feature-rate-limit-idempotency-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract — rate limit response/log 기준 / Retry-After header 기준 (scope L29, L33)
|
||||
- code: RATE_LIMIT_EXCEEDED
|
||||
category: RATE_LIMIT
|
||||
http_status: 429
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Too many requests, please retry after the indicated interval"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://rate-limit/exceeded"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:rate-limit
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract L71 — "200ms 초과 시 409 IDEMPOTENT_IN_FLIGHT (retryable=false, client는 polling)"
|
||||
- code: IDEMPOTENT_IN_FLIGHT
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
owner_layer: application
|
||||
client_safe_message: "A previous identical request is still being processed, please poll for result"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:idempotency
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract L72 — "fingerprint mismatch (same key + different body) = 422 IDEMPOTENT_REQUEST_MISMATCH"
|
||||
- code: IDEMPOTENT_REQUEST_MISMATCH
|
||||
category: VALIDATION
|
||||
http_status: 422
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
owner_layer: application
|
||||
client_safe_message: "Idempotency key reused with different request body"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:idempotency
|
||||
|
||||
# ============================================================
|
||||
# File / Resource (feature-file-resource-handling-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-file-resource-handling-contract L69 — "spring.servlet.multipart.max-file-size 10MB ... Spring 단의 enforcement가 실패 시 envelope 응답 보장" / 테스트 계약 "oversized upload가 generic 500으로 처리되면 실패"
|
||||
- code: UPLOAD_SIZE_EXCEEDED
|
||||
category: VALIDATION
|
||||
http_status: 413
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Uploaded file exceeds maximum size"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-upload
|
||||
|
||||
# source: feature-file-resource-handling-contract L72 — "allowed content-type allowlist starting set ..."
|
||||
- code: UPLOAD_CONTENT_TYPE_REJECTED
|
||||
category: VALIDATION
|
||||
http_status: 415
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Uploaded content type is not allowed"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-upload
|
||||
|
||||
# source: feature-file-resource-handling-contract — Decisionized Work Items "path traversal | normalized storage key only ... | traversal test"
|
||||
- code: PATH_TRAVERSAL_DETECTED
|
||||
category: VALIDATION
|
||||
http_status: 400
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Invalid file path"
|
||||
log_level: ERROR
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-upload
|
||||
|
||||
# source: feature-file-resource-handling-contract L73 — "streaming download backpressure = response timeout 60s, max stream 100MB. 초과 시 truncate + ERROR log"
|
||||
- code: DOWNLOAD_STREAMING_FAILURE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-file-resource-handling-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Download failed, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://file/download-streaming-failure"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:file-download
|
||||
|
||||
# ============================================================
|
||||
# API contract transport-standard codes (feature-api-contract-baseline)
|
||||
# ============================================================
|
||||
# NOTE: feature-api-contract-baseline owns the transport-shape failure
|
||||
# classification (D8 413/414, D9 406/415, D12 405, D15 412). These rows mirror
|
||||
# dev.caskeleton.shared.error.OperationalError; the D11 status-mapping
|
||||
# consistency test (owner: this branch, producer) fails the build when a code's
|
||||
# registry http_status and the enum httpStatus() drift apart.
|
||||
|
||||
# source: feature-api-contract-baseline.md D12 — "405 Method Not Allowed + Allow header 의무"
|
||||
- code: METHOD_NOT_ALLOWED
|
||||
category: VALIDATION
|
||||
http_status: 405
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "HTTP method not allowed for this resource"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D9 — "406 Not Acceptable = 응답 표현 협상 실패"
|
||||
- code: NOT_ACCEPTABLE
|
||||
category: VALIDATION
|
||||
http_status: 406
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "No acceptable representation for the requested Accept header"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D15 — "If-Match mismatch 시 412 Precondition Failed"
|
||||
- code: PRECONDITION_FAILED
|
||||
category: CONFLICT
|
||||
http_status: 412
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Resource was modified by another request; refetch and retry"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D8 — "request size limit 실패 분류 (413)"
|
||||
- code: PAYLOAD_TOO_LARGE
|
||||
category: VALIDATION
|
||||
http_status: 413
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Request payload is too large"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D8 형제 — "URI 길이 실패 분류 (414)"
|
||||
# NOTE: enforcement is Tomcat/gateway-owned (rejected before Spring dispatch);
|
||||
# this row + code exist for status-mapping consistency. End-to-end 414 contract
|
||||
# test is `planned` (gateway/Tomcat maxHttpHeaderSize 8KB boundary).
|
||||
- code: URI_TOO_LONG
|
||||
category: VALIDATION
|
||||
http_status: 414
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Request URI is too long"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# source: feature-api-contract-baseline.md D9 — "415 Unsupported Media Type = 요청 본문 format 미지원"
|
||||
- code: UNSUPPORTED_MEDIA_TYPE
|
||||
category: VALIDATION
|
||||
http_status: 415
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Request Content-Type is not supported"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:api-contract-status-mapping
|
||||
|
||||
# ============================================================
|
||||
# Tenant (feature-tenant-context-policy)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-tenant-context-policy L71 — "tenant 미지원 모드에서 X-Tenant-Id 헤더 수신 시 400 TENANT_NOT_SUPPORTED (filter 단계)"
|
||||
- code: TENANT_NOT_SUPPORTED
|
||||
category: VALIDATION
|
||||
http_status: 400
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-tenant-context-policy
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Tenant context is not supported by this deployment"
|
||||
log_level: WARN
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:tenant-policy
|
||||
|
||||
# ============================================================
|
||||
# Validation / Business rule (feature-business-rule-validation-contract)
|
||||
# ============================================================
|
||||
# NOTE: business-rule-validation branch는 mapping 규칙 SSOT (syntax→VALIDATION,
|
||||
# policy→AUTHZ/CONFLICT, invariant→CONFLICT/VALIDATION, persistence→PERSISTENCE/CONFLICT)
|
||||
# 이며 구체 code는 example로 VALIDATION_EMAIL_FORMAT만 등장
|
||||
# (feature-operational-error-observability-foundation L110). 실제 도메인별 code는
|
||||
# Phase D(도메인 feature 적용) 시 본 registry에 추가.
|
||||
|
||||
# source: feature-operational-error-observability-foundation L110 — "code: VALIDATION_EMAIL_FORMAT, // registry-registered code" (validation field error JSON shape example)
|
||||
- code: VALIDATION_EMAIL_FORMAT
|
||||
category: VALIDATION
|
||||
http_status: 400
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Invalid email format"
|
||||
log_level: INFO
|
||||
runbook_link: null
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:validation-envelope
|
||||
|
||||
# ============================================================
|
||||
# Cache (feature-cache-consistency-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-cache-consistency-contract — Decisionized Work Items "Redis unavailable | degrade only if declared | fail-fast for required cache | generic INTERNAL | unavailable mapping" / 테스트 "Redis unavailable이 degrade 가능 여부 없이 INTERNAL로 처리되면 실패"
|
||||
- code: CACHE_UNAVAILABLE
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://cache/unavailable"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:cache-consistency
|
||||
|
||||
# source: feature-cache-consistency-contract L70 — "stampede 방지 default = single-instance Caffeine local lock, multi-instance HPA 시 Redisson RLock distributed mutex" / 테스트 "동일 key에 대해 동시 cache miss 시 backend 호출이 1회로 제한되는지 verify (stampede). 미충족 시 실패"
|
||||
- code: CACHE_STAMPEDE_LOCK_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://cache/stampede-lock-timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:cache-consistency
|
||||
|
||||
# ============================================================
|
||||
# Outbound HTTP (feature-outbound-http-client-baseline)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-outbound-http-client-baseline L70 — "outbound HTTP timeout default = connect 2s / read 5s / global call 10s" + scope "timeout/connect/DNS failure 분류" / 테스트 "upstream timeout은 retryable dependency failure로 분류되어야 함"
|
||||
- code: DEPENDENCY_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 504
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service did not respond in time, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "timeout/connect/DNS failure 분류" + L70 connect=2s timeout
|
||||
- code: DEPENDENCY_CONNECT_FAILED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service unreachable, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/connect-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "timeout/connect/DNS failure 분류"
|
||||
- code: DEPENDENCY_DNS_FAILED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service unreachable, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/dns-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "upstream 4xx/5xx 분류" / 테스트 "401/403은 credential/scope/config 문제로 분류되어야 함"
|
||||
- code: DEPENDENCY_4XX_CLIENT
|
||||
category: PERMANENT_DEPENDENCY
|
||||
http_status: 502
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service rejected the request"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/4xx-client"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline — scope "upstream 4xx/5xx 분류"
|
||||
- code: DEPENDENCY_5XX_SERVER
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 502
|
||||
retryable: true
|
||||
retry_after_seconds: 2
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service error, please retry"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://dependency/5xx-server"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# source: feature-outbound-http-client-baseline L69 — "circuit breaker metric은 dependency.name, dependency.type, outcome까지만 tag로 허용" + Decisionized "circuit breaker | Resilience4j optional env"
|
||||
- code: DEPENDENCY_CIRCUIT_OPEN
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 10
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Upstream service temporarily unavailable, please retry later"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://dependency/circuit-open"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbound-http
|
||||
|
||||
# ============================================================
|
||||
# Outbox (feature-domain-event-outbox-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-domain-event-outbox-contract L67 — "outbox row status enum = PENDING / IN_FLIGHT / PUBLISHED / FAILED / DEAD" + scope "publish 실패 분류" / 판정 "publish 실패가 retry/DLQ/log/runbook 기준 없이 삼켜지면 실패"
|
||||
- code: OUTBOX_PUBLISH_FAILED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 500
|
||||
retryable: true
|
||||
retry_after_seconds: 30
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://outbox/publish-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbox-publish
|
||||
|
||||
# source: feature-domain-event-outbox-contract L67 — outbox status enum "DEAD" / Outbox Defaults "DLQ | background-job branch owner"
|
||||
- code: OUTBOX_DEAD_LETTER
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://outbox/dead-letter"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:outbox-dlq
|
||||
|
||||
# ============================================================
|
||||
# Background job / Async (feature-background-job-async-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-background-job-async-contract — Decisionized "saturation | bounded executor + rejection log" / L72 "saturation policy default = AbortPolicy" / 테스트 "executor rejection이 structured log 없이 발생하면 실패"
|
||||
- code: JOB_EXECUTOR_REJECTED
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 503
|
||||
retryable: true
|
||||
retry_after_seconds: 5
|
||||
owner_branch: feature-background-job-async-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Service temporarily unavailable"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://job/executor-rejected"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:async-saturation
|
||||
|
||||
# source: feature-background-job-async-contract L69 — "기본 backoff는 exponential backoff with jitter, max attempts 3, DLQ after exhausted attempts" + scope "shutdown 중 job 처리 기준" / L73 graceful shutdown ≤19s
|
||||
- code: JOB_TIMEOUT
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
http_status: 500
|
||||
retryable: true
|
||||
retry_after_seconds: 10
|
||||
owner_branch: feature-background-job-async-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://job/timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:async-timeout
|
||||
|
||||
# source: feature-background-job-async-contract L69 — "DLQ after exhausted attempts" + Decisionized "retry/DLQ | exp backoff jitter, max 3, DLQ exhausted | ... | infinite retry"
|
||||
- code: JOB_DEAD_LETTER
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-background-job-async-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://job/dead-letter"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:async-dlq
|
||||
|
||||
# ============================================================
|
||||
# Distributed Lock (feature-distributed-lock-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-distributed-lock-contract D7 — "lock 획득 실패/timeout 의 error code =
|
||||
# LOCK_ACQUISITION_TIMEOUT (category CONFLICT, retryable true, client_safe true) + metric
|
||||
# lock.acquisition" / D5 — "try-lock + 유한 waitTime + lease(TTL) 필수, 무한 blocking 금지".
|
||||
# category CONFLICT 는 기존 enum 재사용; retryable=true — 락 보유자가 임계 구역을 빠져나오면
|
||||
# 동일 요청 재시도로 해소된다(transient contention). DB_DEADLOCK / DB_SERIALIZATION_FAILURE 와
|
||||
# 같은 retryable CONFLICT 계열(409). 본 코드는 distributedLockProvider 획득 timeout 전용이며
|
||||
# cache stampede lock 의 CACHE_STAMPEDE_LOCK_TIMEOUT(cache-consistency, TRANSIENT_DEPENDENCY 503)
|
||||
# 과 의미가 구분된다 — 후자는 캐시 백엔드 의존성 timeout, 전자는 분산 상호배제 contention.
|
||||
- code: LOCK_ACQUISITION_TIMEOUT
|
||||
category: CONFLICT
|
||||
http_status: 409
|
||||
retryable: true
|
||||
retry_after_seconds: 1
|
||||
owner_branch: feature-distributed-lock-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Resource is busy, please retry"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://lock/acquisition-timeout"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:lock-acquisition-timeout
|
||||
|
||||
# ============================================================
|
||||
# Migration / Startup (feature-migration-startup-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = ... migration 실패=70 ..." + Decisionized "startup failure log | structured log with startup.phase, error.code, error.category"
|
||||
- code: MIGRATION_FAILED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://migration/failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = env 누락/malformed=78 ..." / 테스트 "required env 누락 시 startup이 성공하면 실패"
|
||||
- code: STARTUP_VALIDATION_FAILED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://startup/validation-failed"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = ... required adapter disabled=72" / 테스트 "disabled required adapter로 app이 뜨면 실패"
|
||||
- code: REQUIRED_ADAPTER_DISABLED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://startup/required-adapter-disabled"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# source: feature-integration-adapter-templates §구현 가이드 §4 (Layer 3) + §Audit A2.
|
||||
# Runtime-lifecycle fail-fast for an invoke against a DISABLED optional adapter
|
||||
# (Kafka/Redis/Slack/Google Email). Deliberately distinct from the startup-lifecycle
|
||||
# REQUIRED_ADAPTER_DISABLED above (exit 72): a runtime invoke ≠ a startup validation,
|
||||
# so reusing the startup code would conflate two lifecycles (A2 resolution — new
|
||||
# runtime code owned by this branch). retryable=false: the adapter stays disabled
|
||||
# until redeploy, so retrying the same call never clears it.
|
||||
- code: ADAPTER_DISABLED
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-integration-adapter-templates
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://adapter/adapter-disabled"
|
||||
compatibility_impact: none
|
||||
required_test: adapter-contract:adapter-disabled-runtime-call
|
||||
|
||||
# source: feature-migration-startup-contract L71 — "startup exit code 표준 = ... profile mismatch=71" / 테스트 "prod profile에서 local-only 설정이 켜지면 실패"
|
||||
- code: PROFILE_MISMATCH
|
||||
category: INTERNAL
|
||||
http_status: 500
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-migration-startup-contract
|
||||
owner_layer: infrastructure
|
||||
client_safe_message: "Internal server error"
|
||||
log_level: ERROR
|
||||
runbook_link: "runbook://startup/profile-mismatch"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:migration-startup
|
||||
|
||||
# ============================================================
|
||||
# Management / Actuator (feature-management-actuator-security-contract)
|
||||
# ============================================================
|
||||
|
||||
# source: feature-management-actuator-security-contract — Exposure Policy "env/configprops | forbidden" "heapdump/threaddump | forbidden unless break-glass runbook" "shutdown | forbidden" / 테스트 "prod에서 env/configprops endpoint가 노출되면 실패"
|
||||
- code: ACTUATOR_FORBIDDEN
|
||||
category: AUTHZ
|
||||
http_status: 403
|
||||
retryable: false
|
||||
retry_after_seconds: null
|
||||
owner_branch: feature-management-actuator-security-contract
|
||||
owner_layer: presentation
|
||||
client_safe_message: "Permission denied"
|
||||
log_level: WARN
|
||||
runbook_link: "runbook://management/actuator-forbidden"
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:management-actuator
|
||||
@@ -0,0 +1,220 @@
|
||||
# Registry: HTTP Headers
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/headers.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Conventions:
|
||||
# - HTTP header name: kebab-case (X-Request-Id, X-Tenant-Id)
|
||||
# - W3C standard headers: lowercase (traceparent, tracestate)
|
||||
# - mdc_key: snake_case (foundation SSOT)
|
||||
# - envelope_meta_field: camelCase (envelope SSOT)
|
||||
|
||||
headers:
|
||||
# source: feature-operational-error-observability-foundation.md L97
|
||||
# "request_id | inbound filter (생성 또는 X-Request-Id 헤더) | response header X-Request-Id"
|
||||
- name: X-Request-Id
|
||||
direction: both
|
||||
type: ulid
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: request_id
|
||||
envelope_meta_field: requestId
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:envelope-headers
|
||||
|
||||
# source: feature-api-contract-baseline.md L67
|
||||
# "X-Api-Version은 실험/compatibility 보조 header이며 path version과 충돌하면 path가 우선"
|
||||
- name: X-Api-Version
|
||||
direction: inbound
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-api-contract-baseline
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:api-versioning
|
||||
|
||||
# source: feature-api-contract-baseline.md L77 / feature-rate-limit-idempotency-contract.md L66-67
|
||||
# "idempotency header 이름은 Idempotency-Key" / "기본 scope는 (authenticatedPrincipal, idempotencyKey, useCaseName)"
|
||||
- name: Idempotency-Key
|
||||
direction: inbound
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:idempotency-replay
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md L85 / foundation L85
|
||||
# "RATE_LIMIT | ... | 429 | true (Retry-After 이후)" / "retry-after 기준 없이 429를 반환하면 실패"
|
||||
- name: Retry-After
|
||||
direction: outbound
|
||||
type: duration-seconds
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md scope L26 (rate limit response/log 기준)
|
||||
# rate-limit 응답 표면 (limit/remaining/reset 3종은 표준 rate-limit signaling)
|
||||
- name: X-RateLimit-Limit
|
||||
direction: outbound
|
||||
type: numeric
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md scope L26 (rate limit response/log 기준)
|
||||
- name: X-RateLimit-Remaining
|
||||
direction: outbound
|
||||
type: numeric
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-rate-limit-idempotency-contract.md scope L26 (rate limit response/log 기준)
|
||||
- name: X-RateLimit-Reset
|
||||
direction: outbound
|
||||
type: rfc3339-date
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-rate-limit-idempotency-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:rate-limit-headers
|
||||
|
||||
# source: feature-api-compatibility-deprecation-contract.md L87
|
||||
# "deprecation marker | OpenAPI deprecated: true + branch note | response header optional"
|
||||
- name: Deprecation
|
||||
direction: outbound
|
||||
type: rfc3339-date
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-api-compatibility-deprecation-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:deprecation-marker
|
||||
|
||||
# source: feature-api-compatibility-deprecation-contract.md L87
|
||||
# "deprecation marker | OpenAPI deprecated: true + branch note | response header optional" (RFC 8594 Sunset)
|
||||
- name: Sunset
|
||||
direction: outbound
|
||||
type: rfc3339-date
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-api-compatibility-deprecation-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:deprecation-marker
|
||||
|
||||
# source: feature-distributed-tracing-contract.md L64, L85
|
||||
# "propagation header는 W3C traceparent default" / "HTTP | traceparent, tracestate (W3C)"
|
||||
- name: traceparent
|
||||
direction: both
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: trace_id
|
||||
envelope_meta_field: traceId
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:trace-propagation
|
||||
|
||||
# source: feature-distributed-tracing-contract.md L66, L85
|
||||
# "propagation format = W3C traceparent + tracestate only. B3 propagation은 forbidden"
|
||||
- name: tracestate
|
||||
direction: both
|
||||
type: comma-separated
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:trace-propagation
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L100
|
||||
# "correlation_id | inbound header X-Correlation-Id 또는 생성 | HTTP X-Correlation-Id, message header correlation_id"
|
||||
- name: X-Correlation-Id
|
||||
direction: both
|
||||
type: ulid
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: correlation_id
|
||||
envelope_meta_field: correlationId
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:envelope-headers
|
||||
|
||||
# source: feature-tenant-context-policy.md L69, L101 (foundation)
|
||||
# "tenant resolution 우선순위 = ... (2) 명시적 X-Tenant-Id 헤더 (admin/internal API only)" /
|
||||
# "tenant_id | tenant context (활성 시) | downstream HTTP X-Tenant-Id (with allowlist)"
|
||||
- name: X-Tenant-Id
|
||||
direction: both
|
||||
type: ulid
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: tenant_id
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-tenant-context-policy
|
||||
case_style: kebab
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:tenant-header-policy
|
||||
|
||||
# source: feature-security-operational-baseline.md L66
|
||||
# "JWT Resource Server를 baseline security model로 둠" (Bearer token via Authorization header)
|
||||
- name: Authorization
|
||||
direction: inbound
|
||||
type: bearer-token
|
||||
required: false
|
||||
generated_if_missing: false
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:jwt-resource-server
|
||||
|
||||
# source: feature-security-operational-baseline.md L83-90 (AuthN/AuthZ Decision Matrix)
|
||||
# 401 응답 시 WWW-Authenticate (Bearer realm/error) — Spring Security JWT Resource Server 표준 challenge header
|
||||
- name: WWW-Authenticate
|
||||
direction: outbound
|
||||
type: string
|
||||
required: false
|
||||
generated_if_missing: true
|
||||
mdc_key: null
|
||||
envelope_meta_field: null
|
||||
owner_branch: feature-security-operational-baseline
|
||||
case_style: kebab
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:jwt-resource-server
|
||||
@@ -0,0 +1,294 @@
|
||||
# Registry: MDC / Log Keys
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/mdc-keys.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# MDC SSOT: feature-operational-error-observability-foundation
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Conventions:
|
||||
# - MDC key naming: snake_case (foundation L93 "snake_case 강제. camelCase / dot.case 금지.")
|
||||
# - cardinality_safe_for_metric=true 인 key만 metric tag로 사용 가능
|
||||
# - foundation L93-102 표 "MDC Key Standard (final)" 6개가 core SSOT
|
||||
|
||||
mdc_keys:
|
||||
# source: feature-operational-error-observability-foundation.md L97
|
||||
# "request_id | inbound filter (생성 또는 X-Request-Id 헤더) | response header X-Request-Id"
|
||||
- key: request_id
|
||||
type: ulid
|
||||
source: inbound_filter
|
||||
required_in: [request, dependency, security, application]
|
||||
http_header_mapping: X-Request-Id
|
||||
envelope_field: meta.requestId
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L98
|
||||
# "trace_id | Micrometer Tracing | W3C traceparent header"
|
||||
- key: trace_id
|
||||
type: string
|
||||
source: observation_context
|
||||
required_in: [request, dependency, application]
|
||||
http_header_mapping: traceparent
|
||||
envelope_field: meta.traceId
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L99
|
||||
# "span_id | Micrometer Tracing | W3C traceparent"
|
||||
# NOTE: background-job-async-contract L71 "span_id는 Micrometer Observation context에서 자동 전파(MDC explicit copy 불필요)"
|
||||
- key: span_id
|
||||
type: string
|
||||
source: observation_context
|
||||
required_in: [request, dependency]
|
||||
http_header_mapping: traceparent
|
||||
envelope_field: null
|
||||
propagation: [http, async]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L100
|
||||
# "correlation_id | inbound header X-Correlation-Id 또는 생성 | HTTP X-Correlation-Id, message header correlation_id"
|
||||
- key: correlation_id
|
||||
type: ulid
|
||||
source: inbound_filter
|
||||
required_in: [request, dependency, application]
|
||||
http_header_mapping: X-Correlation-Id
|
||||
envelope_field: meta.correlationId
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L101 + feature-tenant-context-policy.md L70
|
||||
# "tenant_id | tenant context (활성 시) | downstream HTTP X-Tenant-Id (with allowlist)" /
|
||||
# "tenant ID format = opaque ULID (26 chars Crockford base32)"
|
||||
# NOTE: tenant L73 "tenant_id ULID 원본은 metric tag에 직접 사용 금지"
|
||||
- key: tenant_id
|
||||
type: ulid
|
||||
source: security_context
|
||||
required_in: [request, dependency, security, audit]
|
||||
http_header_mapping: X-Tenant-Id
|
||||
envelope_field: null
|
||||
propagation: [http, async, message]
|
||||
owner_branch: feature-tenant-context-policy
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:tenant-leakage
|
||||
|
||||
# source: feature-operational-error-observability-foundation.md L102
|
||||
# "user_principal | security context (pseudonymized only) | log only, headers forbidden"
|
||||
- key: user_principal
|
||||
type: string
|
||||
source: security_context
|
||||
required_in: [security, audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-operational-error-observability-foundation
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-mdc-keys
|
||||
|
||||
# ── log type extensions (log-management-contract L101-109 "Log Type별 필수 필드") ──
|
||||
|
||||
# source: feature-log-management-contract.md L105 "request | request_id, trace_id, method, uri_template, status, duration_ms"
|
||||
# NOTE: application-port-usecase-contract / business 측 operation 식별자 (uri_template과 별도 application-set)
|
||||
- key: operation
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [application, dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L105 "request | request_id, trace_id, method, uri_template, status, duration_ms"
|
||||
- key: method
|
||||
type: string
|
||||
source: inbound_filter
|
||||
required_in: [request]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L105 "request | request_id, trace_id, method, uri_template, status, duration_ms"
|
||||
# NOTE: metrics L86 "status_code | 7 (1xx-5xx + ok/other)" — bounded
|
||||
- key: status
|
||||
type: numeric
|
||||
source: inbound_filter
|
||||
required_in: [request]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L105-106 "request | ... duration_ms" / "dependency | ... duration_ms"
|
||||
- key: duration_ms
|
||||
type: numeric
|
||||
source: application_set
|
||||
required_in: [request, dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 "dependency | dependency_name, dependency_type, duration_ms, outcome, error_code"
|
||||
# NOTE: metrics L88 "dependency_name | 50" — bounded
|
||||
- key: dependency_name
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 "dependency | dependency_name, dependency_type, ..."
|
||||
- key: dependency_type
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 + metrics L91 "outcome (resilience4j) | 5 (SUCCESS/FAILURE/CIRCUIT_OPEN/TIMEOUT/REJECTED)"
|
||||
- key: outcome
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L106 "dependency | ... error_code (실패 시)"
|
||||
# NOTE: metrics L89 "error_code | 100 — error registry row 상한과 정합" — bounded
|
||||
- key: error_code
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [dependency]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L107
|
||||
# "security | event_type, user_principal (pseudonymized), source_ip (anonymized — last octet zeroed)"
|
||||
- key: event_type
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [security, audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L107 "security | ... source_ip (anonymized — last octet zeroed)"
|
||||
# NOTE: metrics L93 "high-cardinality 금지 tag: ... ip_address"
|
||||
- key: source_ip_anon
|
||||
type: string
|
||||
source: inbound_filter
|
||||
required_in: [security]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L108 "audit | actor, action, target, before_hash, after_hash, occurred_at"
|
||||
- key: actor
|
||||
type: string
|
||||
source: security_context
|
||||
required_in: [audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L108 "audit | actor, action, target, ..."
|
||||
- key: action
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: true
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
|
||||
# source: feature-log-management-contract.md L108 "audit | actor, action, target, ..."
|
||||
- key: target
|
||||
type: string
|
||||
source: application_set
|
||||
required_in: [audit]
|
||||
http_header_mapping: null
|
||||
envelope_field: null
|
||||
propagation: [none]
|
||||
owner_branch: feature-log-management-contract
|
||||
cardinality_safe_for_metric: false
|
||||
case_style: snake
|
||||
compatibility_impact: none
|
||||
required_test: contract-verification:log-fields
|
||||
@@ -0,0 +1,540 @@
|
||||
# Registry: Metrics
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/metrics.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Owner branch: feature-metrics-alerting-contract
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Notes
|
||||
# - Naming: Micrometer dot.case + unit suffix (.seconds | .bytes | .total).
|
||||
# - Tag cardinality bounds are SSOT of feature-metrics-alerting-contract "Cardinality Bounds" table.
|
||||
# - High-cardinality tags forbidden globally: user_id, request_id, raw_url, raw_query,
|
||||
# raw_header_value, ip_address. These MUST NOT appear in any row.
|
||||
# - tenant_id label is bounded mapping table id OR cohort bucket only (ULID raw forbidden).
|
||||
# - error_code tag cardinality_limit follows error-codes.yaml row count (max 100).
|
||||
|
||||
metrics:
|
||||
|
||||
# === HTTP server (inbound) ===
|
||||
# source: feature-metrics-alerting-contract — Metric/Alert Defaults
|
||||
# "HTTP metric | http.server.requests with method/status/uri-template | raw URL or user id tag"
|
||||
- name: http.server.requests
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: method
|
||||
cardinality_limit: 8
|
||||
allowed_values: [GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS, OTHER]
|
||||
- name: status
|
||||
cardinality_limit: 7
|
||||
allowed_values: [1xx, 2xx, 3xx, 4xx, 5xx, ok, other]
|
||||
- name: uri_template
|
||||
cardinality_limit: 200
|
||||
validation: must_be_template_not_raw_uri
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "error_rate > 5% for 5m OR > 10% for 1m"
|
||||
p2: "error_rate > 1% for 10m"
|
||||
p3: "error_rate > 0.1% for 1h"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [method, status, uri_template]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — P1/P2/P3 정량 기준 (HTTP latency p99)
|
||||
# "P1: p99 > 5s 5분 / P2: p99 > 1s 10분 / P3: p99 > 500ms 30분"
|
||||
- name: http.server.requests.latency
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: method
|
||||
cardinality_limit: 8
|
||||
allowed_values: [GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS, OTHER]
|
||||
- name: uri_template
|
||||
cardinality_limit: 200
|
||||
validation: must_be_template_not_raw_uri
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "p99 > 5s for 5m"
|
||||
p2: "p99 > 1s for 10m"
|
||||
p3: "p99 > 500ms for 30m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [method, uri_template, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === HTTP client (outbound dependency) ===
|
||||
# source: feature-metrics-alerting-contract — Metric/Alert Defaults
|
||||
# "dependency metric | dependency.client.requests with dependency.name/type/outcome | endpoint with secret tag"
|
||||
# source: feature-outbound-http-client-baseline — "circuit breaker metric은 dependency.name, dependency.type, outcome까지만 tag로 허용"
|
||||
- name: dependency.client.requests
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: dependency_name
|
||||
cardinality_limit: 50
|
||||
- name: dependency_type
|
||||
cardinality_limit: 10
|
||||
allowed_values: [http, grpc, db, cache, queue, broker, other]
|
||||
- name: outcome
|
||||
cardinality_limit: 5
|
||||
allowed_values: [SUCCESS, FAILURE, CIRCUIT_OPEN, TIMEOUT, REJECTED]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "required dep unavailable for 2m"
|
||||
p2: "optional dep degraded for 5m"
|
||||
p3: "spike alert (10x baseline)"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [dependency_name, dependency_type, outcome, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-outbound-http-client-baseline — decisions
|
||||
# "retry/circuit breaker 기본 라이브러리는 Resilience4j"
|
||||
# source: feature-metrics-alerting-contract — "retry/CB minimum: resilience4j.retry.calls{outcome}"
|
||||
- name: resilience4j.retry.calls
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 5
|
||||
allowed_values: [SUCCESS, FAILURE, CIRCUIT_OPEN, TIMEOUT, REJECTED]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "retry exhaustion rate > 1% for 10m"
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
log_field_mapping: [dependency_name, outcome, retry_attempt]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — "resilience4j.circuitbreaker.state"
|
||||
- name: resilience4j.circuitbreaker.state
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: name
|
||||
cardinality_limit: 50
|
||||
- name: state
|
||||
cardinality_limit: 6
|
||||
allowed_values: [CLOSED, OPEN, HALF_OPEN, DISABLED, FORCED_OPEN, METRICS_ONLY]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "state == OPEN for required dependency for 2m"
|
||||
p2: "state == OPEN for optional dependency for 5m"
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
log_field_mapping: [dependency_name]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — "resilience4j.circuitbreaker.calls{outcome}"
|
||||
- name: resilience4j.circuitbreaker.calls
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 5
|
||||
allowed_values: [SUCCESS, FAILURE, CIRCUIT_OPEN, TIMEOUT, REJECTED]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "CIRCUIT_OPEN rate > 1% for 10m"
|
||||
owner_branch: feature-outbound-http-client-baseline
|
||||
log_field_mapping: [dependency_name, outcome, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === DB connection pool ===
|
||||
# source: feature-persistence-failure-baseline — Hikari Alert Threshold
|
||||
# "pool wait p99 > 100ms 5분 지속 → P2 / pool exhaustion (active = max) > 1분 → P1"
|
||||
# source: feature-metrics-alerting-contract — "hikaricp.connections.acquire{outcome='timeout'} p99 > 100ms"
|
||||
- name: hikaricp.connections.acquire
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: pool
|
||||
cardinality_limit: 5
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [SUCCESS, TIMEOUT, FAILURE]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p1: "pool exhaustion (active == max) for 1m"
|
||||
p2: "acquire p99 > 100ms for 5m"
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
log_field_mapping: [pool, outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-persistence-failure-baseline — In scope "Hikari metric 노출 기준" + Hikari Alert Threshold
|
||||
- name: hikaricp.connections.usage
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: pool
|
||||
cardinality_limit: 5
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "usage p99 elevated > 10m"
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
log_field_mapping: [pool]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-persistence-failure-baseline — In scope "Hikari metric 노출 기준"
|
||||
- name: hikaricp.connections.active
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: pool
|
||||
cardinality_limit: 5
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "active == max for 1m"
|
||||
owner_branch: feature-persistence-failure-baseline
|
||||
log_field_mapping: [pool]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — Histogram Buckets/Percentile "DB query: same"
|
||||
- name: db.query.duration
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: operation
|
||||
cardinality_limit: 20
|
||||
allowed_values: [select, insert, update, delete, batch, ddl, other]
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [SUCCESS, FAILURE, TIMEOUT]
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "p99 > 1s for 10m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [operation, outcome, duration_ms]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Background job / async executor ===
|
||||
# source: feature-background-job-async-contract — Decisionized Work Items "saturation policy"
|
||||
# "AbortPolicy default (core=10, max=50, queue=200)"
|
||||
- name: executor.saturation
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: executor_name
|
||||
cardinality_limit: 10
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "queue size > 80% capacity for 5m"
|
||||
p1: "rejection rate > 0 for 1m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [executor_name]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-background-job-async-contract — Decisionized Work Items "saturation | bounded executor + rejection log"
|
||||
- name: executor.rejected.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: executor_name
|
||||
cardinality_limit: 10
|
||||
- name: policy
|
||||
cardinality_limit: 3
|
||||
allowed_values: [AbortPolicy, CallerRunsPolicy, DiscardPolicy]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "rejection_count > 0 for 1m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [executor_name, policy]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-background-job-async-contract — Decisionized Work Items "retry/DLQ | exp backoff jitter, max 3, DLQ exhausted"
|
||||
- name: job.retry.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: job_name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 4
|
||||
allowed_values: [SUCCESS, RETRY, EXHAUSTED, DLQ]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "EXHAUSTED rate > 1% for 10m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [job_name, outcome, retry_attempt]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-background-job-async-contract — "DLQ after exhausted attempts"
|
||||
- name: job.dlq.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: job_name
|
||||
cardinality_limit: 50
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "DLQ rate sustained > 0 for 5m"
|
||||
owner_branch: feature-background-job-async-contract
|
||||
log_field_mapping: [job_name]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Distributed lock ===
|
||||
# source: feature-distributed-lock-contract D7 — "metric lock.acquisition (tag: outcome =
|
||||
# acquired/timeout/error) — 신규 제안" / D5 — try-lock + 유한 waitTime + lease(TTL). 분산
|
||||
# 상호배제(distributedLockProvider) 획득 시도 결과를 센다. key 는 tag 로 넣지 않는다
|
||||
# (무한 cardinality — 위 전역 금지 규칙). timeout outcome 은 LOCK_ACQUISITION_TIMEOUT 발생과 1:1.
|
||||
- name: lock.acquisition
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [acquired, timeout, error]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "timeout rate > 5% for 10m"
|
||||
owner_branch: feature-distributed-lock-contract
|
||||
log_field_mapping: [outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-distributed-lock-contract §Edge / D5 (SI-LOCK-C5) — "lease 만료 후 unlock →
|
||||
# ConcurrentModificationException — 삼킴 금지, 로그+metric 후 정상 흐름 복귀". Counts releases that
|
||||
# found the lease already expired (the JdbcLock row was reclaimed by another instance before
|
||||
# the holder called close()). A sustained nonzero rate means lease TTL is shorter than real
|
||||
# critical-section duration — raise APP/lease TTL or shorten the protected work. Not an
|
||||
# acquisition outcome, hence a separate counter from lock.acquisition.
|
||||
- name: lock.lease.expired
|
||||
type: counter
|
||||
unit: total
|
||||
tags: []
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "lease-expired rate sustained > 0 for 10m"
|
||||
owner_branch: feature-distributed-lock-contract
|
||||
log_field_mapping: []
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Outbox publisher ===
|
||||
# source: feature-domain-event-outbox-contract — Outbox Defaults
|
||||
# "DB outbox table with eventId, aggregateId, eventType, payload, occurredAt, status, attemptCount, nextAttemptAt"
|
||||
- name: outbox.publisher.published.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: event_type
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 4
|
||||
allowed_values: [PUBLISHED, FAILED, DEAD, IN_FLIGHT]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "FAILED rate > 1% for 10m"
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
log_field_mapping: [event_type, outcome, event_id]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-domain-event-outbox-contract — row status enum PENDING/IN_FLIGHT/PUBLISHED/FAILED/DEAD
|
||||
- name: outbox.publisher.lag
|
||||
type: gauge
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: event_type
|
||||
cardinality_limit: 50
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "lag > 60s for 10m"
|
||||
p1: "lag > 300s for 5m"
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
log_field_mapping: [event_type]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-domain-event-outbox-contract — row status enum + Outbox Defaults attemptCount
|
||||
- name: outbox.pending.size
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: status
|
||||
cardinality_limit: 5
|
||||
allowed_values: [PENDING, IN_FLIGHT, PUBLISHED, FAILED, DEAD]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "PENDING size growing for 10m"
|
||||
owner_branch: feature-domain-event-outbox-contract
|
||||
log_field_mapping: [status]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Cache ===
|
||||
# source: feature-cache-consistency-contract — Decisionized Work Items "cache pattern | cache-aside default"
|
||||
- name: cache.gets.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: cache_name
|
||||
cardinality_limit: 50
|
||||
- name: result
|
||||
cardinality_limit: 3
|
||||
allowed_values: [hit, miss, error]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p3: "hit_ratio < baseline 0.5x for 1h"
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
log_field_mapping: [cache_name, result]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-cache-consistency-contract — "invalidation = after-commit only", "invalidation 실패가 조용히 무시되면 실패"
|
||||
- name: cache.invalidations.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: cache_name
|
||||
cardinality_limit: 50
|
||||
- name: outcome
|
||||
cardinality_limit: 3
|
||||
allowed_values: [SUCCESS, FAILURE, SKIPPED]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "FAILURE rate > 0 for 5m"
|
||||
owner_branch: feature-cache-consistency-contract
|
||||
log_field_mapping: [cache_name, outcome]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Log appender ===
|
||||
# source: feature-log-management-contract — Sampling Policy (final)
|
||||
# "async appender overflow default: drop oldest INFO/DEBUG with counter metric (log.appender.dropped.total)"
|
||||
- name: log.appender.dropped.total
|
||||
type: counter
|
||||
unit: total
|
||||
tags:
|
||||
- name: appender
|
||||
cardinality_limit: 5
|
||||
- name: level
|
||||
cardinality_limit: 2
|
||||
allowed_values: [INFO, DEBUG]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "dropped > 0 sustained for 10m"
|
||||
owner_branch: feature-log-management-contract
|
||||
log_field_mapping: [appender, level]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === Distributed tracing ===
|
||||
# source: feature-distributed-tracing-contract — decisions
|
||||
# "trace sampling rate default = prod 1%, staging 10%, dev/local 100%"
|
||||
- name: tracing.sampling.rate
|
||||
type: gauge
|
||||
unit: total
|
||||
tags:
|
||||
- name: profile
|
||||
cardinality_limit: 4
|
||||
allowed_values: [prod, staging, dev, local]
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p3: "effective rate deviates from configured for 1h"
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
log_field_mapping: [profile]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# === JVM baseline ===
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric"
|
||||
- name: jvm.memory.used
|
||||
type: gauge
|
||||
unit: bytes
|
||||
tags:
|
||||
- name: area
|
||||
cardinality_limit: 2
|
||||
allowed_values: [heap, nonheap]
|
||||
- name: id
|
||||
cardinality_limit: 10
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p2: "heap used / max > 0.85 for 10m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [area, id]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric"
|
||||
- name: jvm.gc.pause
|
||||
type: timer
|
||||
unit: seconds
|
||||
tags:
|
||||
- name: action
|
||||
cardinality_limit: 10
|
||||
- name: cause
|
||||
cardinality_limit: 10
|
||||
percentiles: [0.5, 0.9, 0.95, 0.99]
|
||||
histogram_buckets: slo_driven
|
||||
alert_severity_thresholds:
|
||||
p2: "p99 pause > 500ms for 10m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: [action, cause]
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric"
|
||||
- name: jvm.threads.live
|
||||
type: gauge
|
||||
unit: total
|
||||
tags: []
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p3: "thread count > 2x baseline for 30m"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: []
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
|
||||
# source: feature-metrics-alerting-contract — In scope "JVM/process metric" (process uptime)
|
||||
- name: process.uptime
|
||||
type: gauge
|
||||
unit: seconds
|
||||
tags: []
|
||||
percentiles: null
|
||||
histogram_buckets: null
|
||||
alert_severity_thresholds:
|
||||
p1: "uptime reset unexpectedly < 60s (crash loop signal)"
|
||||
owner_branch: feature-metrics-alerting-contract
|
||||
log_field_mapping: []
|
||||
compatibility_impact: additive
|
||||
required_test: contract-verification:metrics-cardinality
|
||||
@@ -0,0 +1,203 @@
|
||||
# Registry: Secrets Classification
|
||||
# SSOT: wiki/projects/ca-tmpl/registries/secrets-classification.yaml
|
||||
# Schema owner: feature-contract-registry-governance
|
||||
# Owner branch: feature-secrets-config-source-contract
|
||||
# Last updated: 2026-05-22
|
||||
#
|
||||
# Conventions:
|
||||
# - 3-tier classification (feature-secrets-config-source-contract 2026-05-22):
|
||||
# public-config | sensitive-config | secret
|
||||
# - `secret` rows: prod_default 항상 null. dev fake 식별자는 `__LOCAL_DEV_` prefix
|
||||
# (feature-secrets-config-source-contract 2026-05-22: "dev/local sentinel value prefix = __LOCAL_DEV_").
|
||||
# - prod profile에서 `__LOCAL_DEV_` prefix 발견 시 startup fail
|
||||
# (feature-secrets-config-source-contract 2026-05-22).
|
||||
# - Masking 기본 = `full_except_last_4` (feature-secrets-config-source-contract 2026-05-22:
|
||||
# "full mask except last 4 chars for non-secret tokens"). 진짜 secret(password/private key)은 `full`.
|
||||
# - Naming suffix는 보조 신호 (feature-secrets-config-source-contract: "_TOKEN, _KEY, _PASSWORD").
|
||||
# - public-config 항목은 env-keys.yaml에서 직접 정의되며 본 파일에는 reference row만 둠.
|
||||
|
||||
secrets:
|
||||
|
||||
# === Tier 3: secret (true secret — password/private-key/HMAC-salt) ===
|
||||
|
||||
- name: APP_DATASOURCE_PASSWORD
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "DB credential은 dual-bind 60s" + "__LOCAL_DEV_FAKE_DB_PASSWORD" 예시
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: dual-bind-60s
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:db-password-no-leak-in-actuator
|
||||
|
||||
- name: APP_SECURITY_JWT_SIGNING_KEY
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "JWT signing key는 24h overlap window 유지 (security branch와 cross-link)"
|
||||
# + feature-security-operational-baseline "rotation overlap window = 새 kid 도입 → 24h 동안 old kid 병행"
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: overlap-24h
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:jwt-signing-key-rotation-overlap
|
||||
|
||||
- name: APP_SECURITY_OAUTH_CLIENT_SECRET
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "secret classification은 ... naming pattern은 보조(suffix _TOKEN, _KEY, _PASSWORD)"
|
||||
# + feature-security-operational-baseline "JWT Resource Server를 baseline security model" (OAuth 자격 증명 분류)
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:oauth-client-secret-no-leak
|
||||
|
||||
- name: APP_EXTERNAL_API_KEY
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "external API key는 application restart 시 reload"
|
||||
# (per-dependency suffix는 adapter 등록 시 추가; 본 row는 baseline 분류 정의)
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:external-api-key-no-leak
|
||||
|
||||
- name: APP_CACHE_REDIS_PASSWORD
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# 3-tier classification "secret" + feature-integration-adapter-templates "Redis | disabled optional module"
|
||||
# (Redis enabled + auth 사용 시 secret으로 분류)
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:redis-password-no-leak
|
||||
|
||||
- name: APP_PRIVACY_PSEUDONYMIZATION_SALT
|
||||
# source: feature-data-retention-privacy-contract 2026-05-22
|
||||
# "pseudonymization key = HMAC-SHA-256 with rotating salt. salt rotation interval = 90일.
|
||||
# rotation 시 old salt 90일 retain (lookup 가능)."
|
||||
# + feature-tenant-context-policy "tenant identifier는 raw PII가 아니어야 하며 ... pseudonymized id"
|
||||
classification: secret
|
||||
source: secret-manager
|
||||
rotation_policy: salt-rotation-90d
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-data-retention-privacy-contract
|
||||
masking_rule: full
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:pseudonymization-salt-rotation
|
||||
|
||||
# === Tier 2: sensitive-config (token-bearing URL or id with exposure restriction) ===
|
||||
|
||||
- name: APP_NOTIFICATION_SLACK_WEBHOOK_URL
|
||||
# source: feature-integration-adapter-templates 2026-05-22
|
||||
# "Slack | disabled optional module | notification failure policy"
|
||||
# Slack webhook URL은 token을 path에 포함하므로 sensitive-config (URL 형태이지만 secret과 동급 취급)
|
||||
classification: sensitive-config
|
||||
source: secret-manager
|
||||
rotation_policy: manual
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:slack-webhook-no-leak
|
||||
|
||||
- name: APP_SECURITY_GOOGLE_OAUTH_CLIENT_ID
|
||||
# source: feature-secrets-config-source-contract 2026-05-22
|
||||
# "sensitive-config" tier (id이지만 노출 제한)
|
||||
# + feature-integration-adapter-templates "Google Email | disabled optional module"
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: manual
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: behavior-change
|
||||
required_test: secrets-contract:google-oauth-client-id-masked
|
||||
|
||||
- name: APP_DATASOURCE_USERNAME
|
||||
# source: feature-secrets-config-source-contract 2026-05-22 — "sensitive-config" tier
|
||||
# (DB user는 password와 함께 노출되면 위험하므로 sensitive-config)
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: dual-bind-60s
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:datasource-username-masked-in-actuator
|
||||
|
||||
- name: APP_DATASOURCE_URL
|
||||
# source: feature-secrets-config-source-contract 2026-05-22 — JDBC URL은 host/db 포함하므로 sensitive-config
|
||||
# (env-keys.yaml에서는 public-config 처리; 본 파일에서는 노출 통제 관점에서 sensitive로 재분류 — masking 기준 명시 목적)
|
||||
classification: sensitive-config
|
||||
source: mounted-env
|
||||
rotation_policy: restart-only
|
||||
prod_default: null
|
||||
dev_sentinel_prefix: __LOCAL_DEV_
|
||||
owner_branch: feature-secrets-config-source-contract
|
||||
masking_rule: full_except_last_4
|
||||
compatibility_impact: breaking
|
||||
required_test: secrets-contract:datasource-url-masked-in-actuator
|
||||
|
||||
# === Tier 1: public-config (reference only — full row in env-keys.yaml) ===
|
||||
|
||||
- name: APP_PROFILE
|
||||
# source: feature-env-driven-runtime-configuration — public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#APP_PROFILE
|
||||
|
||||
- name: APP_NAME
|
||||
# source: feature-env-driven-runtime-configuration — public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#APP_NAME
|
||||
|
||||
- name: SERVER_PORT
|
||||
# source: feature-env-driven-runtime-configuration — Spring native, public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#SERVER_PORT
|
||||
|
||||
- name: SPRING_PROFILES_ACTIVE
|
||||
# source: feature-env-driven-runtime-configuration — Spring native, public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-env-driven-runtime-configuration
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#SPRING_PROFILES_ACTIVE
|
||||
|
||||
- name: OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
# source: feature-distributed-tracing-contract — public-config tier reference
|
||||
classification: public-config
|
||||
source: application-yml
|
||||
owner_branch: feature-distributed-tracing-contract
|
||||
masking_rule: none
|
||||
reference: env-keys.yaml#OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — ADAPTER_DISABLED (런타임 어댑터 비활성화 호출)
|
||||
category: INTERNAL
|
||||
error_codes: [ADAPTER_DISABLED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: ADAPTER_DISABLED (`runbook://adapter/adapter-disabled`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 500 with `error.code=ADAPTER_DISABLED`
|
||||
- Code invoked an optional adapter (Kafka/Redis/Slack/Email) that is disabled in this deployment
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check adapter name in log (`adapter_name` field)
|
||||
- Review deployment config — which optional adapters are enabled?
|
||||
|
||||
## Action
|
||||
|
||||
- Enable the adapter in deployment configuration (env flag)
|
||||
- Or update application logic to skip disabled-adapter paths
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to deployment team if adapter should be enabled but isn't
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_AUDIENCE_MISMATCH (대상 불일치)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_AUDIENCE_MISMATCH]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_AUDIENCE_MISMATCH (`runbook://auth/audience-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_AUDIENCE_MISMATCH`
|
||||
- Token `aud` claim does not include this service's expected audience
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check token `aud` claim value
|
||||
- Compare against configured `spring.security.oauth2.resourceserver.jwt.audiences`
|
||||
|
||||
## Action
|
||||
|
||||
- Verify client is requesting tokens scoped to the correct audience
|
||||
- Update audience configuration if service identifier changed
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to auth-platform team if misconfiguration is system-wide
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_CLAIM_MAPPING_FAILED (클레임 매핑 실패)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_CLAIM_MAPPING_FAILED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_CLAIM_MAPPING_FAILED (`runbook://auth/claim-mapping-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_CLAIM_MAPPING_FAILED`
|
||||
- Token validated but required claims (sub, roles, tenant) missing or unexpected type
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Inspect token payload claims via logs
|
||||
- Check claim extractor configuration
|
||||
|
||||
## Action
|
||||
|
||||
- Verify IdP token template includes required claims
|
||||
- Update claim mapping configuration if IdP schema changed
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to auth-platform team if IdP changed claim schema
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_ISSUER_MISMATCH (발급자 불일치)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_ISSUER_MISMATCH]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_ISSUER_MISMATCH (`runbook://auth/issuer-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_ISSUER_MISMATCH`
|
||||
- Token `iss` claim does not match configured expected issuer
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Compare token `iss` against `spring.security.oauth2.resourceserver.jwt.issuer-uri`
|
||||
- Check if IdP environment changed
|
||||
|
||||
## Action
|
||||
|
||||
- Update issuer config if IdP migrated
|
||||
- Reject tokens from unexpected issuers
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to platform-security if unexpected issuer detected
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_JWKS_UNAVAILABLE (JWKS 엔드포인트 장애)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [AUTH_JWKS_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_JWKS_UNAVAILABLE (`runbook://auth/jwks-unavailable`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=AUTH_JWKS_UNAVAILABLE`
|
||||
- All authentication failing; JWKS refresh attempts failing
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check IdP JWKS endpoint health: `curl -sf https://<idp-host>/.well-known/jwks.json`
|
||||
- Check network connectivity from app pods to IdP
|
||||
|
||||
## Action
|
||||
|
||||
- Enable cached JWKS fallback if available
|
||||
- Coordinate with IdP team for restoration
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 page: IdP team immediately if JWKS endpoint unreachable > 2 minutes
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_KID_UNKNOWN (키 ID 미인식)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_KID_UNKNOWN]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_KID_UNKNOWN (`runbook://auth/kid-unknown`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_KID_UNKNOWN`, `retryable=true`
|
||||
- Token `kid` header not present in cached JWKS
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check if IdP key rotation occurred recently
|
||||
- Verify JWKS cache TTL and refresh timing
|
||||
|
||||
## Action
|
||||
|
||||
- Force JWKS cache refresh
|
||||
- Confirm new key is published in IdP JWKS endpoint
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to IdP team if new kid not appearing in JWKS after 10 minutes
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — INTERNAL_AUTH_MISCONFIGURATION (공개 경로 설정 오류)
|
||||
category: INTERNAL
|
||||
error_codes: [INTERNAL_AUTH_MISCONFIGURATION]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: INTERNAL_AUTH_MISCONFIGURATION (`runbook://auth/public-path-misconfiguration`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 500 with `error.code=INTERNAL_AUTH_MISCONFIGURATION`
|
||||
- Security filter misconfiguration detected at runtime
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `verifyPublicPathSnapshot` output in CI
|
||||
- Review recent changes to `SecurityConfig` or `application.yml` public path list
|
||||
|
||||
## Action
|
||||
|
||||
- Revert misconfigured public path change
|
||||
- Run `./gradlew verifyPublicPathSnapshot` to compare snapshot
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 immediate: if auth bypass is possible due to misconfiguration
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_EXPIRED (토큰 만료)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_EXPIRED]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_EXPIRED (`runbook://auth/token-expired`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_TOKEN_EXPIRED`
|
||||
- Spike may indicate clock skew or long-lived token usage
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `exp` claim vs server clock
|
||||
- Check NTP sync on token-issuing host
|
||||
|
||||
## Action
|
||||
|
||||
- Client must refresh tokens before expiry
|
||||
- Verify clock skew tolerance is configured (default 60s)
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if spike is widespread or clock drift is confirmed
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_INVALID_SIGNATURE (서명 검증 실패)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_INVALID_SIGNATURE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_INVALID_SIGNATURE (`runbook://auth/token-invalid-signature`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 with `error.code=AUTH_TOKEN_INVALID_SIGNATURE`
|
||||
- `log_level=ERROR` — may indicate forged tokens or wrong signing key
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check if JWKS endpoint returned a new key set
|
||||
- Check for token forgery attempts in logs
|
||||
|
||||
## Action
|
||||
|
||||
- Verify JWKS key IDs match token headers
|
||||
- Alert security team if forgery suspected
|
||||
|
||||
## Escalation
|
||||
|
||||
- Immediate P1 escalation if forgery indicators present
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_MALFORMED (토큰 파싱 실패)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_MALFORMED]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_MALFORMED (`runbook://auth/token-malformed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 responses with `error.code=AUTH_TOKEN_MALFORMED`
|
||||
- Token present but fails JWT parse (not 3-part, non-base64, etc.)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Inspect raw Authorization header value in logs
|
||||
- Check if token generation tooling has a bug
|
||||
|
||||
## Action
|
||||
|
||||
- Identify source of malformed tokens
|
||||
- Fix or update client token generation
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if spike suggests infrastructure issue
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTH_TOKEN_MISSING (인증 토큰 누락)
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_MISSING]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTH_TOKEN_MISSING (`runbook://auth/token-missing`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 401 responses with `error.code=AUTH_TOKEN_MISSING`
|
||||
- Client missing Authorization header or Bearer token
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check request logs for missing Authorization header
|
||||
- Verify client SDK configuration
|
||||
|
||||
## Action
|
||||
|
||||
- Confirm API clients are sending Authorization header
|
||||
- Check gateway/proxy configuration for header stripping
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if widespread or affecting critical workflows
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: Runbook — JWT key rotation 시 인증 실패 spike
|
||||
category: AUTH
|
||||
error_codes: [AUTH_TOKEN_EXPIRED, AUTH_KID_UNKNOWN, AUTH_JWKS_UNAVAILABLE, AUTH_TOKEN_INVALID_SIGNATURE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: JWT key rotation 시 인증 실패 spike
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `auth_401_error_rate_high` 또는 `jwks_refresh_failure_spike`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `runbook_link`, `dependency_name`
|
||||
- 임계: 401 error rate > 5% 5분 지속 OR JWKS refresh failure count > 10건/분
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. JWKS endpoint health check: `curl -sf https://<idp-host>/.well-known/jwks.json | jq '.keys | length'`
|
||||
2. log query에서 `error.code` 분포 확인 — `AUTH_KID_UNKNOWN` 비중이 높으면 rotation 원인 강력 시사
|
||||
3. IdP rotation schedule 확인 (직전 24h 내 rotation 이벤트가 있었는지)
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- JWKS cache TTL을 짧게(예: 60s) 강제하여 새 kid 전파 가속
|
||||
- 새 kid가 JWKS에 publish되어 있는지 확인. 누락이면 IdP에 republish 요청
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query (Loki/CloudWatch): `{service="auth"} | error.category="AUTH" | dependency_name="jwks-endpoint"`
|
||||
- metric panel: `auth_jwks_cache_hit_ratio`, `auth_jwks_refresh_failure_total`, `auth_kid_unknown_total`
|
||||
- trace: 실패한 request 1건에서 `traceId` 추출 → IdP outbound span 확인
|
||||
- 가능한 원인:
|
||||
- 새 kid가 JWKS에 publish되기 전 token 발급 → 24h overlap window 안에 있는지 확인
|
||||
- JWKS endpoint 장애 (5xx, timeout) → IdP status page 확인
|
||||
- 시계 skew로 인한 만료 오판 → NTP sync 상태 확인
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: old kid를 임시 재허용 (rollback). overlap window를 48h로 일시 확장
|
||||
- IdP에 새 JWKS publish 재시도 요청
|
||||
- 장기: rotation 절차에 "publish → 24h 대기 → switch" 단계 강제. observability에 kid 분포 metric 추가
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- P2 → P1 격상 조건: 401 error rate > 20% 또는 다중 tenant에 동시 발생
|
||||
- 다음 on-call로 page: 10분 내 회복 안 되면 IdP team 또는 platform-security team page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: `auth_401_error_rate < 1%` 5분 지속, `AUTH_KID_UNKNOWN` 건수 0
|
||||
- post-incident:
|
||||
- rotation 절차 RCA 작성
|
||||
- JWKS overlap window 정책 문서 업데이트
|
||||
- kid 분포 dashboard 영구화
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `AUTH_TOKEN_EXPIRED`, `AUTH_KID_UNKNOWN`, `AUTH_JWKS_UNAVAILABLE`, `AUTH_TOKEN_INVALID_SIGNATURE`
|
||||
- metrics.yaml: `auth_jwks_cache_hit_ratio`, `auth_jwks_refresh_failure_total`
|
||||
- 관련 branch: [[feature-security-operational-baseline]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 IdP 종류·rotation 정책·JWKS endpoint URL·dashboard 링크로 보강 필요.
|
||||
@@ -0,0 +1,72 @@
|
||||
---
|
||||
title: Runbook — cross-tenant 접근 시도 감지
|
||||
category: AUTHZ
|
||||
error_codes: [AUTHZ_INSUFFICIENT_PERMISSION, AUTHZ_TENANT_MISMATCH]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: cross-tenant 접근 시도 감지
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `authz_cross_tenant_violation` 또는 `authz_403_spike`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `principal_id_pseudonymized`, `tenant_id`, `runbook_link`
|
||||
- 임계:
|
||||
- P2: 403 with `error.code=AUTHZ_TENANT_MISMATCH` > 10건/5분
|
||||
- P1 격상: 동일 principal에서 3개 이상 tenant 시도 OR 5분 내 100건 초과
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. log query로 위반 principal 식별 (pseudonymized): `error.code=AUTHZ_TENANT_MISMATCH`
|
||||
2. principal의 정상 tenant scope 확인 (IdP claim 또는 entitlement table)
|
||||
3. `CROSS_TENANT_ADMIN` capability 보유 여부 확인 — 보유자라면 false positive 가능성
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- 명백한 위반 패턴이면 principal session 강제 만료 (token revocation list 추가)
|
||||
- security incident channel 통보 (`#sec-incident`)
|
||||
- 위반 request의 source IP / user-agent 기록
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="api"} | error.category="AUTHZ" | principal_id_pseudonymized="<hash>"`
|
||||
- metric panel: `authz_denied_total{reason="tenant_mismatch"}`, `authz_principal_tenant_distribution`
|
||||
- trace: 위반 request의 `traceId`로 호출 chain 확인. token claim의 `tenant_id`와 요청 path의 `tenant_id` 비교
|
||||
- 가능한 원인:
|
||||
- account takeover (계정 탈취) → 즉시 session revoke + 비밀번호 reset 요구
|
||||
- client bug (잘못된 tenant id 전송) → product team에 통보
|
||||
- 정상 admin operation 누락된 capability → entitlement 보정
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: principal session revoke, source IP rate-limit 강화
|
||||
- 위반이 client bug면 client patch release 협조
|
||||
- 장기: tenant boundary 검증 layer를 controller가 아닌 repository 진입점에서 강제 ([[feature-repository-access-permission-contract]])
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: 보안 incident channel 즉시 page. 5분 내 security on-call 응답 없으면 CISO escalation
|
||||
- legal/compliance 통보 필요 여부 판단 (개인정보 noted시)
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: `AUTHZ_TENANT_MISMATCH` 건수 정상 baseline 복귀
|
||||
- post-incident:
|
||||
- account takeover면 forensics 수행 + audit log 보존
|
||||
- cross-tenant 검증 unit test 추가
|
||||
- 위반 패턴 detection rule 영구화
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `AUTHZ_INSUFFICIENT_PERMISSION`, `AUTHZ_TENANT_MISMATCH`
|
||||
- metrics.yaml: `authz_denied_total`, `authz_principal_tenant_distribution`
|
||||
- 관련 branch: [[feature-tenant-context-policy]], [[feature-repository-access-permission-contract]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 tenant 모델·capability 정의·security team 연락 체계로 보강 필요.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTHZ_INSUFFICIENT_PERMISSION (권한 부족)
|
||||
category: AUTHZ
|
||||
error_codes: [AUTHZ_INSUFFICIENT_PERMISSION]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTHZ_INSUFFICIENT_PERMISSION (`runbook://authz/insufficient-permission`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 403 with `error.code=AUTHZ_INSUFFICIENT_PERMISSION`
|
||||
- Valid token but missing required role or permission
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check user's assigned roles in IdP
|
||||
- Review endpoint's required permission annotation
|
||||
|
||||
## Action
|
||||
|
||||
- Grant correct role/permission to user
|
||||
- Verify endpoint permission requirement is correct
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to access-management team if bulk users affected
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — AUTHZ_TENANT_MISMATCH (테넌트 cross-access 시도)
|
||||
category: AUTHZ
|
||||
error_codes: [AUTHZ_TENANT_MISMATCH]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: AUTHZ_TENANT_MISMATCH (`runbook://authz/tenant-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 403 with `error.code=AUTHZ_TENANT_MISMATCH`
|
||||
- `log_level=ERROR` — cross-tenant access attempt detected
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Extract `traceId`, check `X-Tenant-Id` vs token tenant claim
|
||||
- Determine if this is misconfigured client or intentional attack
|
||||
|
||||
## Action
|
||||
|
||||
- Block repeat offenders at gateway level
|
||||
- Alert security team for investigation
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if confirmed malicious cross-tenant access attempt
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — CACHE_STAMPEDE_LOCK_TIMEOUT (캐시 스탬피드 락 타임아웃)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [CACHE_STAMPEDE_LOCK_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: CACHE_STAMPEDE_LOCK_TIMEOUT (`runbook://cache/stampede-lock-timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=CACHE_STAMPEDE_LOCK_TIMEOUT`
|
||||
- Multiple concurrent cache misses on same key; lock contention
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check cache hit ratio metrics
|
||||
- Identify cache keys with high miss rates
|
||||
|
||||
## Action
|
||||
|
||||
- Verify stampede lock TTL is configured appropriately
|
||||
- Pre-warm cache for high-traffic keys on startup
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if backend load spike accompanies stampede
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — CACHE_UNAVAILABLE (캐시 연결 불가)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [CACHE_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: CACHE_UNAVAILABLE (`runbook://cache/unavailable`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=CACHE_UNAVAILABLE`
|
||||
- Redis connection errors in logs
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check Redis cluster health
|
||||
- Verify network connectivity from app to Redis
|
||||
|
||||
## Action
|
||||
|
||||
- Check Redis sentinel/cluster status
|
||||
- Enable cache degradation path if configured for optional caches
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if required cache is down and no degradation path exists
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_DEADLOCK (데드락)
|
||||
category: CONFLICT
|
||||
error_codes: [DB_DEADLOCK]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_DEADLOCK (`runbook://db/deadlock`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 409 with `error.code=DB_DEADLOCK`
|
||||
- SQLState 40P01 in Postgres logs
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `pg_locks` and `pg_stat_activity` during deadlock
|
||||
- Identify conflicting transaction lock order
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry (retryable=true)
|
||||
- Fix lock ordering in code if recurring
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA if deadlock rate is sustained > 1% of transactions
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_IDLE_IN_TX_TIMEOUT (트랜잭션 idle 타임아웃)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DB_IDLE_IN_TX_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_IDLE_IN_TX_TIMEOUT (`runbook://db/idle-in-tx-timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DB_IDLE_IN_TX_TIMEOUT`
|
||||
- SQLState 25P03; transaction held open too long without activity
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `idle_in_transaction_session_timeout` Postgres setting
|
||||
- Look for application-level long-running transaction holders
|
||||
|
||||
## Action
|
||||
|
||||
- Reduce transaction scope in application code
|
||||
- Verify `spring.jpa.properties.hibernate.connection.timeout` is bounded
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA if connection pool exhaustion results
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_QUERY_CANCELED (쿼리 취소)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DB_QUERY_CANCELED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_QUERY_CANCELED (`runbook://db/query-canceled`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DB_QUERY_CANCELED`
|
||||
- SQLState 57014; query exceeds statement timeout
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `statement_timeout` in Postgres
|
||||
- Identify slow queries in `pg_stat_statements`
|
||||
|
||||
## Action
|
||||
|
||||
- Optimize slow query or add index
|
||||
- Adjust statement timeout if query is legitimately long
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA for query optimization if recurring
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_SERIALIZATION_FAILURE (직렬화 실패)
|
||||
category: CONFLICT
|
||||
error_codes: [DB_SERIALIZATION_FAILURE]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_SERIALIZATION_FAILURE (`runbook://db/serialization-failure`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 409 with `error.code=DB_SERIALIZATION_FAILURE`
|
||||
- SQLState 40001; high concurrent transaction contention
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check DB transaction isolation level
|
||||
- Identify hot rows / hot tables under high concurrency
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry with exponential backoff (retryable=true)
|
||||
- Optimize transaction scope if spike is sustained
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to DBA if sustained serialization failure rate > 5%
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DB_UNAVAILABLE (데이터베이스 연결 불가)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DB_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DB_UNAVAILABLE (`runbook://db/unavailable`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DB_UNAVAILABLE`
|
||||
- SQLState 08* connection errors in logs
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check DB server health and connection pool exhaustion
|
||||
- Review network connectivity from app pods to DB
|
||||
|
||||
## Action
|
||||
|
||||
- Check DB primary health; failover to replica if available
|
||||
- Drain connection pool and reconnect
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: immediate if DB primary is down
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_4XX_CLIENT (업스트림 클라이언트 오류)
|
||||
category: PERMANENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_4XX_CLIENT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_4XX_CLIENT (`runbook://dependency/4xx-client`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 502 with `error.code=DEPENDENCY_4XX_CLIENT`
|
||||
- Upstream returned 401/403/400 — credential, scope, or request format issue
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check upstream response body in logs for error detail
|
||||
- Verify API credentials and scopes are valid
|
||||
|
||||
## Action
|
||||
|
||||
- Rotate credentials if expired
|
||||
- Fix request format if API contract changed
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to upstream API owner if contract change is suspected
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_5XX_SERVER (업스트림 서버 오류)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_5XX_SERVER]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_5XX_SERVER (`runbook://dependency/5xx-server`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 502 with `error.code=DEPENDENCY_5XX_SERVER`
|
||||
- Upstream returned 5xx; transient server-side failure
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `dependency_name` tag for which upstream is failing
|
||||
- Review upstream service status page
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry (retryable=true)
|
||||
- Monitor upstream recovery
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if critical upstream is in sustained 5xx state
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_CIRCUIT_OPEN (서킷 브레이커 개방)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_CIRCUIT_OPEN]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_CIRCUIT_OPEN (`runbook://dependency/circuit-open`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DEPENDENCY_CIRCUIT_OPEN`
|
||||
- Circuit breaker (Resilience4j) in OPEN state for a dependency
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check Resilience4j circuit breaker metrics for the dependency
|
||||
- Check upstream health; circuit opens after failure threshold breached
|
||||
|
||||
## Action
|
||||
|
||||
- Wait for circuit half-open probe (automatic after wait duration)
|
||||
- Resolve upstream issue to allow circuit to close
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if circuit remains open > 5 minutes on a critical dependency
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_CONNECT_FAILED (외부 의존성 연결 실패)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_CONNECT_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_CONNECT_FAILED (`runbook://dependency/connect-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DEPENDENCY_CONNECT_FAILED`
|
||||
- TCP connection refused or network unreachable to upstream
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `dependency_name` tag for which upstream is unreachable
|
||||
- Verify network path and firewall rules
|
||||
|
||||
## Action
|
||||
|
||||
- Check upstream service availability
|
||||
- Verify service discovery / DNS resolution
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if upstream is a critical service dependency
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_DNS_FAILED (DNS 조회 실패)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_DNS_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_DNS_FAILED (`runbook://dependency/dns-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DEPENDENCY_DNS_FAILED`
|
||||
- DNS resolution failure for upstream hostname
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Test DNS resolution from app pod: `nslookup <upstream-host>`
|
||||
- Check cluster DNS (CoreDNS) health
|
||||
|
||||
## Action
|
||||
|
||||
- Verify upstream hostname configuration
|
||||
- Check CoreDNS / cluster DNS health
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if cluster DNS is degraded
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DEPENDENCY_TIMEOUT (외부 의존성 타임아웃)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_TIMEOUT]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DEPENDENCY_TIMEOUT (`runbook://dependency/timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 504 with `error.code=DEPENDENCY_TIMEOUT`
|
||||
- Upstream service did not respond within configured timeout (default: global 10s)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `dependency_name` in log for which upstream is timing out
|
||||
- Review upstream service latency metrics
|
||||
|
||||
## Action
|
||||
|
||||
- Check upstream service health
|
||||
- Verify timeout settings match SLA expectations
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if critical upstream is timing out at scale
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: Runbook — 외부 의존성 unavailable
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DEPENDENCY_TIMEOUT, DEPENDENCY_CONNECT_FAILED, DEPENDENCY_DNS_FAILED, DEPENDENCY_CIRCUIT_OPEN, DEPENDENCY_5XX_SERVER, CACHE_UNAVAILABLE, DB_UNAVAILABLE]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: 외부 의존성 unavailable
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `dependency_error_rate_critical` 또는 `circuit_breaker_open`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `dependency_name`, `dependency_kind`(required|optional), `runbook_link`
|
||||
- 임계:
|
||||
- P1: required dependency의 error rate > 50% 1분 OR circuit_open state 활성
|
||||
- P2: optional dependency degraded (fail-open으로 동작 중)
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. `dependency_name` 별 status page 확인 (외부 SaaS면 vendor status, internal이면 해당 service dashboard)
|
||||
2. log query로 실패 패턴 확인: timeout / connect / DNS / 5xx 중 어떤 모드인지
|
||||
3. runtime-health Dependency Matrix에서 required vs optional 분류 확인
|
||||
4. circuit breaker state 확인 (Resilience4j metric)
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- required dep이면 readiness probe로 traffic 차단 (회복 대기) — cascade failure 방지
|
||||
- optional dep이면 fail-open with degraded mode 확인. degraded banner를 client에 노출
|
||||
- DNS failure면 resolver/coredns 상태 확인. cache 강제 flush 검토
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | dependency_name="<name>" | stats count by error.code`
|
||||
- metric panel:
|
||||
- `resilience4j_circuitbreaker_state{name="<name>"}`
|
||||
- `resilience4j_retry_calls_total{kind="failed_without_retry"}`
|
||||
- `hikaricp_connections_active`, `hikaricp_connections_pending` (DB_UNAVAILABLE)
|
||||
- `http_client_requests_seconds_count{outcome="SERVER_ERROR"}`
|
||||
- trace: 실패 request의 outbound span에서 timeout/connect/DNS 분류, target endpoint 확인
|
||||
- 가능한 원인:
|
||||
- vendor outage → status page 확인, 회복 대기
|
||||
- 네트워크 문제 (DNS, security group, NAT) → infra team 확인
|
||||
- connection pool 고갈 (Hikari) → pool size/timeout 점검
|
||||
- circuit breaker open 후 half-open 전환 실패 → 수동 reset 검토
|
||||
- retry-storm으로 인한 self-DoS → retry budget 축소
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: required면 회복 대기 + traffic 차단, optional이면 degraded mode로 유지
|
||||
- pool 고갈이면 일시 pool size 상향 + leak detection 활성화
|
||||
- circuit이 stuck이면 수동 reset (`actuator/circuitbreakerevents`)
|
||||
- 장기: retry budget·timeout·circuit 임계 재조정, fallback path 보강, vendor SLA 재협상
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: required dep 5분 내 회복 안 되면 외부 dep team 또는 vendor에 page
|
||||
- 다중 dep 동시 장애면 incident commander 호출 (네트워크 전반 문제 의심)
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: dependency error rate < 1% 5분 지속, circuit_breaker_state = CLOSED, pool utilization 정상
|
||||
- post-incident:
|
||||
- vendor postmortem 요청 (외부 SaaS면)
|
||||
- timeout/retry/circuit 설정 재검토
|
||||
- degraded mode가 사용자 경험에 미친 영향 측정
|
||||
- chaos test에 해당 시나리오 추가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `DEPENDENCY_TIMEOUT`, `DEPENDENCY_CONNECT_FAILED`, `DEPENDENCY_DNS_FAILED`, `DEPENDENCY_CIRCUIT_OPEN`, `DEPENDENCY_5XX_SERVER`, `CACHE_UNAVAILABLE`, `DB_UNAVAILABLE`
|
||||
- metrics.yaml: `resilience4j_circuitbreaker_state`, `hikaricp_connections_active`, `http_client_requests_seconds_count`
|
||||
- 관련 branch: [[feature-outbound-http-client-baseline]], [[feature-persistence-failure-baseline]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 dependency 목록·required/optional 분류·vendor 연락 체계·circuit/timeout 임계로 보강 필요.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — DOWNLOAD_STREAMING_FAILURE (스트리밍 다운로드 실패)
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [DOWNLOAD_STREAMING_FAILURE]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: DOWNLOAD_STREAMING_FAILURE (`runbook://file/download-streaming-failure`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 503 with `error.code=DOWNLOAD_STREAMING_FAILURE`
|
||||
- Streaming response truncated; backpressure or timeout (60s / 100MB limit)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check streaming response timeout configuration
|
||||
- Review download size vs 100MB limit
|
||||
|
||||
## Action
|
||||
|
||||
- Verify storage backend is reachable
|
||||
- Check for network congestion on download path
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate to infra if storage backend is degraded
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,74 @@
|
||||
---
|
||||
title: Runbook — 5xx Internal error spike
|
||||
category: INTERNAL
|
||||
error_codes: [INTERNAL_ERROR, INTERNAL_AUTH_MISCONFIGURATION, JVM_OOM]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: 5xx Internal error spike
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `http_5xx_error_rate_critical`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `request_id`, `traceId`, `runbook_link`
|
||||
- 임계: 5xx error rate > 5% 5분 지속 OR > 10% 1분
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. 가장 최근 deploy 시각 확인 (CI/CD dashboard, artifact registry digest)
|
||||
2. JVM metric 확인: heap usage, GC pause, CPU, thread count
|
||||
3. log에서 실패 request 1건 추출 → `request_id`, `traceId` 확보
|
||||
4. error.code 분포 확인: `INTERNAL_ERROR` vs `JVM_OOM` vs `INTERNAL_AUTH_MISCONFIGURATION`
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- 직전 deploy가 의심되면 즉시 rollback (artifact registry에서 직전 image digest pin)
|
||||
- OOM 패턴이면 affected pod evict → ASG/HPA로 replacement 유도
|
||||
- LB에서 unhealthy pod 격리 (readiness probe failure 유도)
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | http.status>=500 | stats count by error.code`
|
||||
- metric panel: `jvm_memory_used_bytes{area="heap"}`, `jvm_gc_pause_seconds`, `process_cpu_seconds_total`, `http_server_requests_seconds_count{status=~"5.."}`
|
||||
- trace: 실패 request의 `traceId`로 span chain 확인 → stack trace에서 root exception 추출
|
||||
- heap dump 위치: `/var/tmp/heap/heapdump-<pid>.hprof` (JVM ergonomics: `-XX:MaxRAMPercentage=75 -XX:+HeapDumpOnOutOfMemoryError`)
|
||||
- 가능한 원인:
|
||||
- 직전 deploy의 회귀 버그 → rollback
|
||||
- JVM OOM (메모리 leak 또는 부하 증가) → heap dump 분석
|
||||
- 외부 의존성 설정 오류 (`INTERNAL_AUTH_MISCONFIGURATION`) → config secret 확인
|
||||
- thread starvation (pool 고갈) → thread dump (`jstack <pid>`)
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: 직전 deploy rollback, OOM pod replacement, traffic 일시 감소(scale-out 또는 rate-limit 강화)
|
||||
- config 오류면 secret/configmap rollback
|
||||
- 장기: heap dump 기반 leak 수정, capacity planning 재검토
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: 10분 내 회복 안 되면 incident commander 호출, severity 1 incident 선언
|
||||
- 데이터 손상 의심되면 DBA team page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: 5xx rate < 0.5% 5분 지속, JVM heap usage < 70%, GC pause p99 < 500ms
|
||||
- post-incident:
|
||||
- rollback 원인 RCA 작성 (배포 게이트 강화 필요 여부)
|
||||
- heap dump 분석 결과 공유
|
||||
- JVM ergonomics(`-XX:MaxRAMPercentage`) 재검토
|
||||
- rollback 자동화 절차 점검
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `INTERNAL_ERROR`, `INTERNAL_AUTH_MISCONFIGURATION`, `JVM_OOM`
|
||||
- metrics.yaml: `jvm_memory_used_bytes`, `jvm_gc_pause_seconds`, `http_server_requests_seconds_count`
|
||||
- 관련 branch: [[feature-operational-error-observability-foundation]], [[feature-container-runtime-contract]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 deploy 파이프라인·heap dump 보관 경로·rollback 자동화 명령으로 보강 필요.
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
title: Runbook — background job dead letter
|
||||
category: INTERNAL
|
||||
error_codes: [JOB_DEAD_LETTER]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-13
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: background job dead letter (`runbook://job/dead-letter`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `job_dead_letter`
|
||||
- alert payload 필수 field: `error.code=JOB_DEAD_LETTER`, `job_name`, `correlation_id`, `runbook_link`
|
||||
- 임계: `job.dlq.total` > 0 for 5m (p1) — retry 소진 후 DLQ 진입은 자동 회복이 없으므로 점검 대상
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `JOB_DEAD_LETTER` 라인 확인: `job_name`, 최종 실패 원인 예외, `correlation_id` 추출
|
||||
2. `job.retry.total{outcome=EXHAUSTED}` 추이로 DLQ 유입 규모 파악
|
||||
3. DLQ 적재 위치(향후 retry carrier 확정 시 DB 테이블/큐) 확인 — 현재 skeleton은 vocabulary 단계
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- DLQ는 max attempts(3) 소진의 최종 상태 — 자동 재시도 없음, 수동 개입 필수
|
||||
- 비즈니스 크리티컬 job이면 §4의 수동 처분(재처리 또는 폐기)을 우선 수행
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="JOB_DEAD_LETTER" | stats count by job_name`
|
||||
- metric panel: `job.dlq.total{job_name}`, `job.retry.total{job_name, outcome}`
|
||||
- 최종 실패 원인 분류:
|
||||
- poison input(직렬화/계약 위반) → 입력 결함, 재처리해도 실패 — 수정 후 재처리 또는 폐기
|
||||
- 외부 의존성 장기 outage 중 attempts 소진 → 의존성 회복 후 재처리로 해결 가능
|
||||
- non-transient error(권한/도메인/스키마)인데 retry된 경우 → 분류기 보강 필요(WAF-REL05-C3: 즉시 DLQ가 정답)
|
||||
|
||||
## 4. Mitigation (수동 처분 — 둘 중 하나)
|
||||
|
||||
- **재처리 (기본)**: 원인 해소 후 해당 job을 다시 enqueue. 소비자는 멱등(idempotencyKey dedupe) 의무가 있으므로 중복 처리 안전
|
||||
- **폐기 (영구)**: 작업이 더 이상 유효하지 않으면 DLQ에서 제거. ⚠ 비즈니스 오너 승인 후에만 수행하고 incident 기록에 남김
|
||||
- 장기: poison input 재발 방지(입력 계약 테스트 보강), non-transient error는 retry 없이 즉시 DLQ로 분류
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 처분 판단(재처리 vs 폐기)이 불가하면 해당 job의 비즈니스 오너에게 escalate
|
||||
- DLQ 누적이 특정 `job_name`에 집중되면 해당 job 코드 오너에게 page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `job.dlq.total` 증가 멈춤, 재처리분의 소비자 dedupe 동작 확인
|
||||
- post-incident: DLQ 원인 분류 기록, 같은 원인의 재발 방지 테스트 추가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `JOB_DEAD_LETTER` (INTERNAL, 500, retryable=false)
|
||||
- metrics.yaml: `job.dlq.total{job_name}`, `job.retry.total{job_name, outcome=DLQ}`
|
||||
- 코드: `app-bootstrap` `async/BackgroundJobMetrics`(retry/DLQ vocabulary 기록 seam — D2/D4)
|
||||
- 관련 runbook: [[job-executor-rejected]], [[job-timeout]], [[outbox-dead-letter]]
|
||||
- 관련 branch: [[feature-background-job-async-contract]] (D4 retry/DLQ vocabulary SSOT — outbox/outbound가 consume)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. retry carrier(Spring Retry / Resilience4j / 자체) 확정 후 DLQ 저장소·재처리 절차 보강 필요.
|
||||
@@ -0,0 +1,71 @@
|
||||
---
|
||||
title: Runbook — async executor rejected
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [JOB_EXECUTOR_REJECTED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-13
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: async executor rejected (`runbook://job/executor-rejected`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `executor_rejected`
|
||||
- alert payload 필수 field: `error.code=JOB_EXECUTOR_REJECTED`, `executor_name`, `policy`, `runbook_link`
|
||||
- 임계: `executor.rejected.total` > 0 for 1m (p1) — bounded pool이 saturation으로 task를 거부
|
||||
- 보조 신호: `executor.saturation` gauge > queue capacity의 80% for 5m (p2)
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `JOB_EXECUTOR_REJECTED` 라인 확인: `executor_name`, `policy=AbortPolicy`, `queue_size` 추출
|
||||
2. `executor.saturation` 패널에서 큐 점유율 추이 확인 — 일시적 burst인지 지속 saturation인지 판별
|
||||
3. 동시 유입 원인 파악: 신규 배포 / 트래픽 spike / 다운스트림 지연으로 worker가 장기 점유되는지
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- AbortPolicy 거부는 호출부에 `RejectedExecutionException`으로 surface됨 — fire-and-forget `@Async` 호출이면 호출부의 async-exception 처리(log/metric)로 흡수됐는지 확인
|
||||
- 지속 saturation이면 유입 측(트래픽/스케줄러 빈도)을 우선 감속
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="JOB_EXECUTOR_REJECTED" | stats count by executor_name`
|
||||
- metric panel: `executor.saturation{executor_name}`, `executor.rejected.total{executor_name, policy}`
|
||||
- 가능한 원인 우선순위:
|
||||
- 다운스트림 의존성 지연 → worker가 반납되지 않아 큐 포화 (가장 흔함)
|
||||
- 트래픽 spike → 정상 부하 한계 초과
|
||||
- pool 과소 설정 (`APP_ASYNC_EXECUTOR_*`)
|
||||
- non-idempotent 작업이 retry로 누적
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: 유입 감속(상위 rate-limit / 스케줄러 interval 확대) 또는 다운스트림 의존성 회복
|
||||
- pool 재조정(restart-only): `APP_ASYNC_EXECUTOR_CORE_SIZE` / `APP_ASYNC_EXECUTOR_MAX_SIZE` / `APP_ASYNC_EXECUTOR_QUEUE_CAPACITY`
|
||||
— ⚠ queue를 무한정 키우지 말 것(unbounded 금지, D7). 부하테스트로 수치 검증 후 변경
|
||||
- CallerRunsPolicy로의 전환은 use-case 차원의 명시적 결정 필요(request thread latency 침식 — TPE-JDK21-C6)
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다운스트림 의존성 장애가 근본 원인이면 해당 의존성 오너에게 escalate
|
||||
- pool 재조정으로도 saturation이 지속되면 용량 계획(capacity planning) 오너에게 page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `executor.rejected.total` 증가 멈춤, `executor.saturation` < 80% 정상화
|
||||
- 거부된 작업의 재처리 경로(멱등 retry / 다음 스케줄 cycle) 정상 동작 확인
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `JOB_EXECUTOR_REJECTED` (TRANSIENT_DEPENDENCY, 503, retryable=true, retry_after 5s)
|
||||
- metrics.yaml: `executor.rejected.total{executor_name, policy}`, `executor.saturation{executor_name}`
|
||||
- 코드: `app-bootstrap` `async/AsyncExecutorConfig`(bounded executor), `async/LoggingAbortPolicy`(reject log+metric), `async/BackgroundJobMetrics`
|
||||
- env: `APP_ASYNC_EXECUTOR_CORE_SIZE` / `APP_ASYNC_EXECUTOR_MAX_SIZE` / `APP_ASYNC_EXECUTOR_QUEUE_CAPACITY`
|
||||
- 관련 runbook: [[job-timeout]], [[job-dead-letter]]
|
||||
- 관련 branch: [[feature-background-job-async-contract]] (D7 saturation policy)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 부하 프로파일·alert 채널·pool 수치 확정 시 보강 필요.
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: Runbook — background job timeout
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [JOB_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-13
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: background job timeout (`runbook://job/timeout`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `job_timeout`
|
||||
- alert payload 필수 field: `error.code=JOB_TIMEOUT`, `job_name`, `correlation_id`, `runbook_link`
|
||||
- 임계: `job.retry.total{outcome=RETRY}` 급증 또는 graceful-shutdown 중 in-flight job interrupt 발생
|
||||
- 연관: shutdown phase에서 19s await 초과로 interrupt된 job (D8)
|
||||
|
||||
## 2. First Response (10분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `JOB_TIMEOUT` 라인 확인: `job_name`, 마지막 단계, 소요 시간 추출
|
||||
2. timeout이 정상 실행 중 발생인지, graceful-shutdown(배포/스케일다운) 중 interrupt인지 구분
|
||||
3. 해당 job이 멱등(retry-on-next-cycle 안전)인지 확인 — 비멱등이면 §4에서 신중히 처리
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- shutdown 중 interrupt면: 다음 기동 시 재시도 대상인지(멱등 전제) 확인, 중복 부작용 여부 점검
|
||||
- 정상 실행 중 timeout이면: 해당 job의 외부 의존성(DB/HTTP) 지연 여부 확인
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="JOB_TIMEOUT" | stats count by job_name`
|
||||
- metric panel: `job.retry.total{job_name, outcome}`
|
||||
- 가능한 원인 우선순위:
|
||||
- 외부 의존성(DB lock / 느린 HTTP) 지연으로 job p99 상승
|
||||
- job 작업량 증가로 단일 cycle이 19s 예산 초과 (D8 — interrupt 노출)
|
||||
- interrupt 미반응 blocking call(JDBC 등) → awaitTermination 초과 (K8S-POD-LC-C2 SIGKILL 경로)
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: 의존성 회복 / job 입력 배치 크기 축소
|
||||
- job p99가 구조적으로 19s를 넘으면: 작업을 분할하거나, grace period 연장 검토(parent project 운영 계약 소유자 승인 필요 — OUT_OF_BRANCH_SCOPE)
|
||||
- 비멱등 job이 재시도로 중복 부작용을 내면 멱등키/dedupe 도입 우선
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 의존성 지연이 근본 원인이면 해당 의존성 오너에게 escalate
|
||||
- shutdown 예산(20s) vs k8s `terminationGracePeriodSeconds`(30s) 정합 이슈면 플랫폼/런타임 오너에게 escalate
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `JOB_TIMEOUT` 신규 발생 멈춤, `job.retry.total{outcome=SUCCESS}` 정상 비율 회복
|
||||
- 멱등 재시도분의 부작용 중복 없음 확인
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `JOB_TIMEOUT` (TRANSIENT_DEPENDENCY, 500, retryable=true, retry_after 10s)
|
||||
- metrics.yaml: `job.retry.total{job_name, outcome}`
|
||||
- 코드: `app-bootstrap` `async/AsyncExecutorConfig`(awaitTermination 19s — D8 graceful shutdown)
|
||||
- env: `APP_SERVER_SHUTDOWN_TIMEOUT`(owner: feature-env-driven-runtime-configuration D2)
|
||||
- 관련 runbook: [[job-executor-rejected]], [[job-dead-letter]]
|
||||
- 관련 branch: [[feature-background-job-async-contract]] (D4 retry / D8 shutdown)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 retry carrier·job p99·shutdown 예산 확정 시 보강 필요.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — LOCK_ACQUISITION_TIMEOUT (분산 락 획득 타임아웃)
|
||||
category: CONFLICT
|
||||
error_codes: [LOCK_ACQUISITION_TIMEOUT]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: LOCK_ACQUISITION_TIMEOUT (`runbook://lock/acquisition-timeout`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 409 with `error.code=LOCK_ACQUISITION_TIMEOUT`
|
||||
- Distributed lock wait exceeded configured timeout; high contention on a resource
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check `lock.acquisition` metric for lock name and duration
|
||||
- Identify lock holders (check DB `integration_lock` table)
|
||||
|
||||
## Action
|
||||
|
||||
- Client should retry with backoff (retryable=true)
|
||||
- Optimize critical section holding time if lock contention is systemic
|
||||
|
||||
## Escalation
|
||||
|
||||
- Escalate if lock holder appears stuck (potential deadlock in distributed lock)
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — ACTUATOR_FORBIDDEN (Actuator 접근 거부)
|
||||
category: AUTHZ
|
||||
error_codes: [ACTUATOR_FORBIDDEN]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: ACTUATOR_FORBIDDEN (`runbook://management/actuator-forbidden`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- HTTP 403 with `error.code=ACTUATOR_FORBIDDEN`
|
||||
- Attempt to access restricted actuator endpoint (env/configprops/heapdump/shutdown)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Identify which actuator endpoint was accessed
|
||||
- Check caller identity (internal tooling vs external)
|
||||
|
||||
## Action
|
||||
|
||||
- Verify management port is not exposed externally
|
||||
- For heapdump/threaddump: follow break-glass runbook procedure
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 if forbidden actuator access appears to be external attack
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Runbook — MIGRATION_FAILED (DB 마이그레이션 실패)
|
||||
category: INTERNAL
|
||||
error_codes: [MIGRATION_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: MIGRATION_FAILED (`runbook://migration/failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 70 (migration failure exit)
|
||||
- Structured log with `error.code=MIGRATION_FAILED`, `startup.phase=migration`
|
||||
- App refuses to start (fail-fast)
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check Flyway migration log for which script failed and why
|
||||
- Review latest migration script for SQL errors
|
||||
|
||||
## Action
|
||||
|
||||
- Fix migration script or roll back to previous migration version
|
||||
- Run migration manually in repair mode if checksum mismatch
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1 immediate: app cannot start until migration is resolved
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
title: Runbook — outbox dead letter
|
||||
category: INTERNAL
|
||||
error_codes: [OUTBOX_DEAD_LETTER]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-11
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: outbox dead letter (`runbook://outbox/dead-letter`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `outbox_dead_letter`
|
||||
- alert payload 필수 field: `error.code=OUTBOX_DEAD_LETTER`, `event_type`, `event_id`, `correlation_id`, `runbook_link`
|
||||
- 임계: `outbox.publisher.published.total{outcome=DEAD}` > 0 (DEAD 전이는 자동 회복이 없으므로 단건도 점검 대상)
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `OUTBOX_DEAD_LETTER` 라인 확인: `event_id`, `event_type`, `correlation_id`, 마지막 실패 원인 예외 추출
|
||||
2. DB에서 DEAD row 확인: `SELECT * FROM outbox_event WHERE status = 'DEAD' ORDER BY occurred_at;`
|
||||
3. **차단 영향 파악 (중요)**: strict per-aggregate FIFO 정책상 DEAD row는 같은 `aggregate_id`의 후행 이벤트를 계속 차단함 —
|
||||
`SELECT count(*) FROM outbox_event b WHERE b.status <> 'PUBLISHED' AND EXISTS (SELECT 1 FROM outbox_event d WHERE d.status='DEAD' AND d.aggregate_id=b.aggregate_id AND d.occurred_at < b.occurred_at);`
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- DEAD는 max attempts(3) 소진의 최종 상태 — 자동 재시도 없음, 수동 개입 필수
|
||||
- 차단된 aggregate가 비즈니스 크리티컬하면 아래 §4의 수동 처분(재발행 또는 skip)을 우선 수행
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="OUTBOX_DEAD_LETTER" | stats count by event_type`
|
||||
- 마지막 실패 원인 분류:
|
||||
- poison event (payload 직렬화/계약 위반) → payload 자체 결함, 재발행해도 실패 — 수정 후 재발행 또는 skip
|
||||
- broker 장기 outage 중 attempts 소진 → broker 회복 후 재발행으로 해결 가능
|
||||
- 구성 오류 (Kafka disabled 상태에서 producer 활성) → 구성 수정 후 재발행
|
||||
- 가능한 원인 우선순위: 구성 오류 > broker outage > poison payload
|
||||
|
||||
## 4. Mitigation (수동 처분 — 둘 중 하나)
|
||||
|
||||
- **재발행 (기본)**: 원인 해소 후 해당 row를 다시 claim 가능 상태로 되돌림 —
|
||||
`UPDATE outbox_event SET status = 'PENDING', attempt_count = 0, next_attempt_at = now() WHERE event_id = '<id>' AND status = 'DEAD';`
|
||||
(consumer는 at-least-once + idempotencyKey dedupe 의무가 있으므로 중복 발행은 안전)
|
||||
- **skip (영구 폐기)**: 이벤트가 더 이상 유효하지 않으면 PUBLISHED로 마킹해 FIFO 차단을 해제 —
|
||||
`UPDATE outbox_event SET status = 'PUBLISHED' WHERE event_id = '<id>' AND status = 'DEAD';`
|
||||
⚠️ skip은 다운스트림에 영구 이벤트 갭을 만든다 — 비즈니스 오너 승인 후에만 수행하고 incident 기록에 남김
|
||||
- 장기: poison event 재발 방지(payload 계약 테스트 보강), DEAD 빈발 event_type의 producer 검증 강화
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 처분 판단(재발행 vs skip)이 불가하면 해당 이벤트의 비즈니스 오너에게 escalate
|
||||
- DEAD 누적이 특정 event_type에 집중되면 producer 코드 오너에게 page
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인: `SELECT count(*) FROM outbox_event WHERE status='DEAD';` = 0, 차단됐던 aggregate의 후행 이벤트가 PUBLISHED로 전이
|
||||
- `outbox.publisher.lag` 정상화(< 60s), 재발행분의 consumer dedupe 동작 확인
|
||||
- post-incident: DEAD 원인 분류 기록, 같은 원인의 재발 방지 테스트 추가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `OUTBOX_DEAD_LETTER` (INTERNAL, retryable=false)
|
||||
- metrics.yaml: `outbox.publisher.published.total{outcome=DEAD}`, `outbox.pending.size{status=DEAD}`, `outbox.publisher.lag`
|
||||
- 코드: `application-core` `PublishPendingOutboxEventsUseCase`(FAILED→DEAD 전이), `adapter-persistence` `outbox/OutboxEventJpaRepository`(FIFO 게이트 — DEAD가 후행 차단)
|
||||
- 관련 runbook: [[outbox-publish-failed]]
|
||||
- 관련 branch: [[feature-domain-event-outbox-contract]], [[feature-background-job-async-contract]] (max attempts/DLQ vocabulary SSOT)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 broker·DLQ 토픽·승인 체계 확정 시 보강 필요.
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
title: Runbook — outbox publish 일시 실패
|
||||
category: TRANSIENT_DEPENDENCY
|
||||
error_codes: [OUTBOX_PUBLISH_FAILED]
|
||||
severity: P2
|
||||
owner: oncall
|
||||
last_updated: 2026-06-11
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: outbox publish 일시 실패 (`runbook://outbox/publish-failed`)
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `outbox_publish_failed_rate` 또는 `outbox_publisher_lag`
|
||||
- alert payload 필수 field: `error.code=OUTBOX_PUBLISH_FAILED`, `event_type`, `correlation_id`, `runbook_link`
|
||||
- 임계 (metrics.yaml verbatim):
|
||||
- P2: `outbox.publisher.published.total{outcome=FAILED}` rate > 1% for 10m
|
||||
- P2: `outbox.publisher.lag` > 60s for 10m / P1: > 300s for 5m
|
||||
- P2: `outbox.pending.size{status=PENDING}` growing for 10m
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. ERROR log에서 `OUTBOX_PUBLISH_FAILED` 라인 확인: `event_type`, `event_id`, `correlation_id`, `attempt_count` 추출
|
||||
2. broker(기본 Kafka adapter) 상태 확인: `APP_MESSAGING_KAFKA_ENABLED` 값과 broker endpoint 가용성
|
||||
- Kafka disabled(default) 상태에서 outbox 이벤트가 append 되고 있으면 publish 경로가 `AdapterDisabledException`으로 전부 실패하는 구성 오류 — 이 경우 producer use case 쪽 활성화/구성을 먼저 의심
|
||||
3. `outbox.pending.size` status 분포 확인 (FAILED 누적 vs PENDING 누적)
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- 일시 실패는 자동 backoff 재시도(30s × 2^(attempt-1) + jitter, max attempts 3)가 동작 — 즉시 수동 개입 불필요
|
||||
- broker 장기 다운이면 DEAD 전이 누적 전에 broker 회복을 우선 (max attempts 소진 시 `runbook://outbox/dead-letter`로 이관)
|
||||
- relay 자체를 멈춰야 하면 `ca-skeleton.outbox.relay-enabled=false`로 스케줄러 비활성 (이벤트는 outbox 테이블에 안전하게 보존됨 — 유실 없음)
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="app"} | error.code="OUTBOX_PUBLISH_FAILED" | stats count by event_type`
|
||||
- metric panel:
|
||||
- `outbox.publisher.published.total{outcome}` — FAILED 비율
|
||||
- `outbox.publisher.lag{event_type}` — 최고령 미발행 이벤트 age
|
||||
- `outbox.pending.size{status}` — 상태별 분포
|
||||
- DB 확인: `SELECT status, count(*) FROM outbox_event GROUP BY status;`
|
||||
- 가능한 원인:
|
||||
- broker outage/네트워크 → broker 측 회복 대기
|
||||
- Kafka adapter 미구성(enabled인데 brokers 누락은 기동 시 차단됨) / disabled 상태에서 producer 활성화
|
||||
- poison event (직렬화 불가/payload 계약 위반) → 재시도 무의미, attempts 소진 후 DEAD로 흘러감 (의도된 동작)
|
||||
- 동일 aggregate head 실패로 후행 이벤트가 FIFO 게이트에 차단되어 lag 증가 (strict per-aggregate FIFO — 설계 의도)
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: broker 회복 후 backoff 만료 시 자동 재발행 — `outcome=PUBLISHED` 회복 확인
|
||||
- IN_FLIGHT orphan(claim 후 crash)은 in-flight-timeout(기본 PT5M) 경과 후 자동 재claim — at-least-once이므로 중복 발행 가능, consumer dedupe(idempotencyKey)가 흡수
|
||||
- 장기: `ca-skeleton.outbox.poll-interval`/`batch-size` 조정, broker 가용성 SLA 점검, 빈발 event_type의 payload 계약 검토
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- P1 lag(>300s 5m) 지속 + broker 회복 불가면 broker/infra 팀에 page
|
||||
- DEAD 전이가 발생하기 시작하면 `runbook://outbox/dead-letter` 절차로 이관
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: `outcome=FAILED` rate < 1% 10분 지속, `outbox.publisher.lag` < 60s, `outbox.pending.size{status=FAILED}` 감소 추세
|
||||
- post-incident: 실패 구간의 DEAD row 유무 확인, consumer 측 중복 처리량 확인(dedupe 동작 검증), backoff/attempts 상수 재평가
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `OUTBOX_PUBLISH_FAILED` (TRANSIENT_DEPENDENCY, retryable=true, retry_after 30s)
|
||||
- metrics.yaml: `outbox.publisher.published.total`, `outbox.publisher.lag`, `outbox.pending.size`
|
||||
- 코드: `application-core` `PublishPendingOutboxEventsUseCase`(상태머신), `adapter-persistence` `outbox/OutboxEventJpaRepository`(SKIP LOCKED claim + FIFO 게이트), `adapter-outbound` `messaging/outbox/KafkaOutboxMessagePublishAdapter`(fail-closed)
|
||||
- 관련 runbook: [[outbox-dead-letter]]
|
||||
- 관련 branch: [[feature-domain-event-outbox-contract]], [[feature-background-job-async-contract]] (retry/DLQ vocabulary SSOT)
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 실제 broker 채택·alert 라우팅·대시보드 링크 확정 시 보강 필요.
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
title: Runbook — Rate limit 초과 spike
|
||||
category: RATE_LIMIT
|
||||
error_codes: [RATE_LIMIT_EXCEEDED, IDEMPOTENT_IN_FLIGHT]
|
||||
severity: P3
|
||||
owner: oncall
|
||||
last_updated: 2026-05-22
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: Rate limit 초과 spike
|
||||
|
||||
## 1. Trigger
|
||||
|
||||
이 runbook은 다음 alert에서 발동됩니다.
|
||||
|
||||
- alert name: `rate_limit_429_high`
|
||||
- alert payload 필수 field: `operation`, `error.code`, `error.category`, `rate_limit_key_type`(ip|principal|tenant), `runbook_link`
|
||||
- 임계:
|
||||
- P3: 429 rate > 1% 10분 지속 (일상적 abuse 차단 효과 정상)
|
||||
- P2 격상: 정상 client(known principal/tenant)에서 spike 또는 spike와 함께 5xx 동반
|
||||
|
||||
## 2. First Response (5분 이내)
|
||||
|
||||
### Step 1 — 확인
|
||||
1. rate-limit key 분포 확인: IP/principal/tenant 중 어디서 spike가 발생했는지
|
||||
- log query: `error.code=RATE_LIMIT_EXCEEDED | stats count by rate_limit_key_type, rate_limit_key`
|
||||
2. top-N offending key 추출 (상위 10건)
|
||||
3. 정상 client 식별 — 알려진 partner/internal service면 P2 격상
|
||||
|
||||
### Step 2 — 임시 격리
|
||||
- abuse traffic 패턴이면 WAF/gateway에서 IP block (geo, ASN 단위)
|
||||
- IDEMPOTENT_IN_FLIGHT 다발이면 client의 retry-storm 의심 → client에 retry-after 협조 요청
|
||||
|
||||
## 3. Diagnosis
|
||||
|
||||
- log query: `{service="gateway"} | error.code="RATE_LIMIT_EXCEEDED" | stats count by rate_limit_key`
|
||||
- metric panel: `gateway_rate_limit_dropped_total`, `gateway_rate_limit_bucket_utilization`
|
||||
- trace: 429 응답의 `Retry-After` 헤더 값, `rate_limit_remaining` header 확인
|
||||
- 가능한 원인:
|
||||
- abuse / bot traffic → IP/ASN block
|
||||
- 정상 client의 traffic 증가 (캠페인, 신규 feature) → limit 일시 상향
|
||||
- retry-storm (client backoff 미적용) → client에 idempotency-key + exponential backoff 권고
|
||||
- limit 설정 오류 (잘못된 정량 threshold) → config rollback
|
||||
|
||||
## 4. Mitigation
|
||||
|
||||
- 단기: abuse면 IP/ASN block, 정상 client면 해당 key의 limit 일시 상향(예: 2x, 1시간 TTL)
|
||||
- IDEMPOTENT_IN_FLIGHT 다발: idempotency-key 정책 점검, client 협조 요청
|
||||
- 장기: limit 정책을 tenant tier별 차등으로 재설계, abuse pattern detection 자동화
|
||||
|
||||
## 5. Escalation
|
||||
|
||||
- 다음 on-call로 page: 30분 내 정상 client 회복 안 되면 product team 통보
|
||||
- 정상 client에 SLO 위반 가능성 있으면 CSM/계정담당 통보
|
||||
|
||||
## 6. Recovery / Verification
|
||||
|
||||
- 회복 확인 metric: 429 rate < 0.5% 10분 지속, 정상 client의 success rate 정상화
|
||||
- post-incident:
|
||||
- 일시 상향한 limit 원복 (TTL 만료 확인)
|
||||
- abuse pattern을 detection rule에 영구 등록
|
||||
- retry-storm이면 client SDK 가이드 보완
|
||||
|
||||
## 7. Related
|
||||
|
||||
- error-codes.yaml rows: `RATE_LIMIT_EXCEEDED`, `IDEMPOTENT_IN_FLIGHT`
|
||||
- metrics.yaml: `gateway_rate_limit_dropped_total`, `gateway_rate_limit_bucket_utilization`
|
||||
- 관련 branch: [[feature-rate-limit-idempotency-contract]]
|
||||
|
||||
---
|
||||
|
||||
> **Stub 상태 안내**: 이 runbook은 skeleton 단계의 stub. 도메인 도입 시 실제 gateway 제품(NGINX/Envoy/Kong 등)·tenant tier 정책·WAF 연동 절차로 보강 필요.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: Runbook — JVM_OOM (JVM OutOfMemoryError)
|
||||
category: INTERNAL
|
||||
error_codes: [JVM_OOM]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: JVM_OOM (`runbook://runtime/jvm-oom`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 137 (ExitOnOutOfMemoryError triggered)
|
||||
- Structured log entry with `error.code=JVM_OOM` before exit
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check heap dump if `-XX:HeapDumpOnOutOfMemoryError` is configured
|
||||
- Review memory usage trends before crash
|
||||
- Check for memory leaks: large cache growth, unbounded lists, session accumulation
|
||||
|
||||
## Action
|
||||
|
||||
- Restart container immediately (k8s will auto-restart with liveness probe)
|
||||
- If recurring: increase heap `-Xmx` or fix memory leak
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: immediate if multiple pods crashing simultaneously
|
||||
- Page SRE / infra team for heap analysis
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
title: Runbook — PROFILE_MISMATCH (프로파일 불일치)
|
||||
category: INTERNAL
|
||||
error_codes: [PROFILE_MISMATCH]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: PROFILE_MISMATCH (`runbook://startup/profile-mismatch`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 71 (profile mismatch exit)
|
||||
- Structured log with `error.code=PROFILE_MISMATCH`, `startup.phase=profile-check`
|
||||
- Production profile active with local-only settings enabled
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check active Spring profiles (`spring.profiles.active`)
|
||||
- Identify which local-only setting is incorrectly enabled in prod profile
|
||||
|
||||
## Action
|
||||
|
||||
- Remove local-only setting from production deployment config
|
||||
- Ensure prod profile does not inherit local/dev profile settings
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: security risk if local settings expose debug endpoints in production
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — REQUIRED_ADAPTER_DISABLED (필수 어댑터 비활성화)
|
||||
category: INTERNAL
|
||||
error_codes: [REQUIRED_ADAPTER_DISABLED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: REQUIRED_ADAPTER_DISABLED (`runbook://startup/required-adapter-disabled`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 72 (required adapter disabled exit)
|
||||
- Structured log with `error.code=REQUIRED_ADAPTER_DISABLED`, `startup.phase=adapter-check`
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Identify which adapter is disabled but required
|
||||
- Check adapter enable flags in environment config
|
||||
|
||||
## Action
|
||||
|
||||
- Enable required adapter in deployment configuration
|
||||
- If adapter is intentionally disabled, update the required/optional designation
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: app cannot start; coordinate with deployment team
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
title: Runbook — STARTUP_VALIDATION_FAILED (환경 변수 검증 실패)
|
||||
category: INTERNAL
|
||||
error_codes: [STARTUP_VALIDATION_FAILED]
|
||||
severity: P1
|
||||
owner: oncall
|
||||
last_updated: 2026-06-15
|
||||
status: stub
|
||||
---
|
||||
|
||||
# Runbook: STARTUP_VALIDATION_FAILED (`runbook://startup/validation-failed`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Container exits with code 78 (env validation failure exit)
|
||||
- Structured log with `error.code=STARTUP_VALIDATION_FAILED`, `startup.phase=env-validation`
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Check which required env variable is missing or malformed
|
||||
- Review container environment and secrets injection
|
||||
|
||||
## Action
|
||||
|
||||
- Supply missing environment variables to deployment
|
||||
- Verify secrets are correctly mounted / injected
|
||||
|
||||
## Escalation
|
||||
|
||||
- P1: app cannot start; coordinate with deployment/secrets team
|
||||
|
||||
---
|
||||
|
||||
> **Stub**: Phase D2 — author body after domain adoption. (feature-operational-runbook-contract D9)
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: Runbook — <TITLE>
|
||||
category: <CATEGORY>
|
||||
error_codes: [<ERROR_CODE_1>, <ERROR_CODE_2>]
|
||||
severity: <P1|P2|P3>
|
||||
owner: oncall
|
||||
last_updated: <YYYY-MM-DD>
|
||||
status: <stub|active>
|
||||
---
|
||||
|
||||
# Runbook: <TITLE> (`runbook://<area>/<scenario>`)
|
||||
|
||||
## Symptoms
|
||||
|
||||
- What observable signals trigger this runbook?
|
||||
- Alert name, metric thresholds, log patterns
|
||||
|
||||
## Diagnosis
|
||||
|
||||
- Step-by-step diagnostic commands and queries
|
||||
- Log queries (Loki/CloudWatch)
|
||||
- Metric panels to check
|
||||
- Trace investigation approach
|
||||
|
||||
## Action
|
||||
|
||||
- Immediate mitigation steps
|
||||
- Configuration changes
|
||||
- Manual intervention procedures
|
||||
|
||||
## Escalation
|
||||
|
||||
- Conditions for severity upgrade (e.g., P2 → P1)
|
||||
- Who to page and when
|
||||
- Fallback procedures if on-call cannot resolve
|
||||
|
||||
---
|
||||
|
||||
> **Note**: This is the canonical runbook template.
|
||||
> Copy this file, rename it to match the `runbook://area/scenario` pattern (→ `area-scenario.md`),
|
||||
> fill in the frontmatter fields, replace section bodies with operational content,
|
||||
> then set `status: active` and remove from `STUB_ALLOWLIST` in `RunbookCoverageContractTest`.
|
||||
@@ -0,0 +1,127 @@
|
||||
# Harness Policy Engine Implementation Plan
|
||||
|
||||
> **Spec:** `docs/superpowers/specs/2026-07-20-harness-policy-engine-design.md`
|
||||
|
||||
**Goal:** Replace topology- and platform-specific duplicated harness rules with a registry,
|
||||
strict evidence validators, generated platform variants, and risk-based review policies.
|
||||
|
||||
**Working policy:** human-only commits. Each task leaves changes in the working tree.
|
||||
|
||||
## Task 1 — Registry, resolver, and Gradle SSOT
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/project/modules.yaml`
|
||||
- Add `.harness/lib/module_registry.py`
|
||||
- Add `.harness/validators/validate_modules.py`
|
||||
- Add `.harness/tests/test_module_registry.py`
|
||||
- Modify `src/settings.gradle`
|
||||
- Modify the dependency-verifier section of `src/build.gradle`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Write failing tests for 19-leaf loading, nested owner resolution, nearest `CLAUDE.md`,
|
||||
unknown paths, and settings/registry parity.
|
||||
- [ ] Add the registry and stdlib loader/resolver.
|
||||
- [ ] Make Gradle settings and dependency verification consume registry data.
|
||||
- [ ] Run Python tests and `./gradlew projects verifyCleanArchitectureDependencies`.
|
||||
|
||||
## Task 2 — Registry-driven import gate and mutation suite
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify `.claude/hooks/ca_import_gate.py`
|
||||
- Modify `.claude/hooks/test_ca_import_gate.py`
|
||||
- Add `.harness/tests/test_import_gate_mutations.py`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Add failing real-path tests for every registered production module.
|
||||
- [ ] Replace flat-path regex/prefix rules with registry owner and role policy.
|
||||
- [ ] Normalize Claude snake_case and Antigravity camelCase tool events.
|
||||
- [ ] Fail closed on malformed in-scope events and marker failures.
|
||||
- [ ] Run all import-gate tests.
|
||||
|
||||
## Task 3 — Verdict schema, evidence artifacts, and platform adapters
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/schemas/verdict.schema.json`
|
||||
- Add `.harness/schemas/evidence.schema.json`
|
||||
- Add `.harness/lib/verdict.py`
|
||||
- Add `.harness/validators/validate_verdict.py`
|
||||
- Add `.harness/validators/validate_evidence.py`
|
||||
- Add `.harness/adapters/antigravity_hook.py`
|
||||
- Add `.harness/tests/test_verdict.py`
|
||||
- Modify `.claude/hooks/ca_verdict_gate.py`
|
||||
- Modify `.claude/hooks/test_ca_verdict_gate.py`
|
||||
- Add `.agents/plugins/ca-superpowers/hooks.json`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Write negative tests for missing required enums, negative counts, Gradle arithmetic,
|
||||
behavior change without red, missing upstream artifacts, malformed input, and revision
|
||||
mismatch.
|
||||
- [ ] Implement strict validation and evidence recording with source/diff hashes.
|
||||
- [ ] Adapt Claude fenced verdicts to the common model.
|
||||
- [ ] Add Antigravity Stop/pre-tool adapter and plugin hook wiring.
|
||||
- [ ] Run validator, adapter, and JSON syntax tests.
|
||||
|
||||
## Task 4 — Canonical agents and deterministic rendering
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/agents/*.md`
|
||||
- Add `.harness/project/platforms.yaml`
|
||||
- Add `.harness/generators/render_agents.py`
|
||||
- Add `.harness/tests/test_platform_parity.py`
|
||||
- Regenerate `.claude/agents/*`, `.codex/agents/*.toml`, `.agents/agents/*/agent.json`
|
||||
- Update `.agents/plugins/ca-superpowers/README.md` and `plugin.json`
|
||||
- Update `.codex/agents/README.md`
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Seed canonical sources from the newest human-only Claude policy, then update module
|
||||
discovery and runner validation to use the registry.
|
||||
- [ ] Add generated metadata and stable output ordering.
|
||||
- [ ] Render all variants and add a `--check` parity mode.
|
||||
- [ ] Assert commit policy, source hashes, tool permissions, and body parity in tests.
|
||||
|
||||
## Task 5 — Risk/profile policies and guidance drift cleanup
|
||||
|
||||
**Files:**
|
||||
|
||||
- Add `.harness/manifest.yaml`
|
||||
- Add `.harness/core/risk-policy.yaml`, `.harness/core/evidence-policy.yaml`
|
||||
- Add current architecture/language/build/framework/capability profile files
|
||||
- Add `.harness/validators/resolve_task.py` and tests
|
||||
- Modify `AGENTS.md`, root `CLAUDE.md`, clean-architecture rule, workflow skill,
|
||||
advisory-depth rule, reporting-standards rule, and plugin README
|
||||
- Modify stale module `CLAUDE.md` files and add missing leaf-module guidance where useful
|
||||
|
||||
**Steps:**
|
||||
|
||||
- [ ] Add failing task-classification tests for high-risk one-file changes and low-risk
|
||||
multi-file fixture/docs changes.
|
||||
- [ ] Implement profile resolution.
|
||||
- [ ] Replace `N!`, routine all-quote grep, file-count report split, and unconditional
|
||||
counterargument policies with the design profiles.
|
||||
- [ ] Replace flat module documentation and focused commands with registry-backed nested names.
|
||||
- [ ] Run policy grep assertions and harness tests.
|
||||
|
||||
## Task 6 — Full review and verification
|
||||
|
||||
- [ ] Run harness unit/mutation/parity suite.
|
||||
- [ ] Run `./gradlew projects` and `./gradlew verifyCleanArchitectureDependencies`.
|
||||
- [ ] Run the focused ArchUnit suite.
|
||||
- [ ] Run `./gradlew check`.
|
||||
- [ ] Audit the working-tree diff in order: architecture → spec → quality.
|
||||
- [ ] Fix findings and restart the review chain, up to three loops.
|
||||
|
||||
## Task 7 — LLM Wiki capture
|
||||
|
||||
- [ ] Read the LLM Wiki authority and branch-note template.
|
||||
- [ ] Update/create the detached-HEAD branch note with implementation decisions, changed files,
|
||||
verification evidence, failures, and open risks.
|
||||
- [ ] Create/link derived error, interview, or blog-topic raw notes only when supported by the
|
||||
completed work; otherwise record an explicit “none” judgment in the branch note.
|
||||
@@ -0,0 +1,187 @@
|
||||
# Harness Policy Engine Refactoring Design
|
||||
|
||||
- **Date:** 2026-07-20
|
||||
- **Status:** Approved by user request
|
||||
- **Scope:** repository-local development harness (`.harness`, `.agents`, `.claude`, `.codex`, root/module guidance, Gradle module registry integration)
|
||||
- **Source:** user-provided “개발 하네스 분석·리뷰” plus repository evidence gathered on 2026-07-20
|
||||
|
||||
## 1. Problem Statement
|
||||
|
||||
The repository now has 19 nested Gradle leaf modules, but the write-time import gate,
|
||||
agent prompts, runner allowlist, and root guidance still contain parts of the previous flat
|
||||
module topology. Platform variants are copied manually, so commit policy and orchestration
|
||||
already differ between Claude, Codex, and Antigravity. Verdict validation checks a text
|
||||
summary but does not consistently require enum fields, non-negative counts, or arithmetic
|
||||
balance.
|
||||
|
||||
The harness must move from duplicated platform prompts to a small policy engine with one
|
||||
project manifest, deterministic renderers, strict validators, and platform adapters.
|
||||
|
||||
## 2. Goals
|
||||
|
||||
1. Make the actual nested Gradle topology a single machine-readable source of truth.
|
||||
2. Resolve a touched file to its nearest owning leaf module without assuming `src/<module>`.
|
||||
3. Generate write-time import policy and focused Gradle task validation from that registry.
|
||||
4. Validate machine verdicts with required fields, non-negative integers, arithmetic rules,
|
||||
upstream evidence, revision identity, and TDD red evidence for behavior changes.
|
||||
5. Materialize validated evidence as JSON artifacts that platform hooks can share.
|
||||
6. Render Claude, Codex, and Antigravity agent variants from one canonical source and fail
|
||||
parity checks when generated files drift.
|
||||
7. Use one human-only commit policy on every platform.
|
||||
8. Replace file-count and exhaustive-report rules with risk and review profiles.
|
||||
9. Add mutation and cross-platform static parity tests.
|
||||
|
||||
## 3. Non-Goals
|
||||
|
||||
- This change does not run authenticated end-to-end golden tasks inside all three external
|
||||
products. It supplies the deterministic fixtures and validators those runs will consume.
|
||||
- It does not add application features or alter production Java behavior.
|
||||
- It does not require PyYAML, jsonschema, Pydantic, or another runtime dependency. Harness
|
||||
data files use JSON syntax, which is valid YAML, and validators use Python stdlib only.
|
||||
- It does not make natural-language agent self-reports authoritative. Hooks convert accepted
|
||||
reports into evidence artifacts; validators remain authoritative.
|
||||
|
||||
## 4. Architecture
|
||||
|
||||
```text
|
||||
.harness/project/modules.yaml ──┬── Gradle settings/includes
|
||||
├── Gradle dependency verification
|
||||
├── owning-module resolver
|
||||
├── import gate
|
||||
└── Gradle command validator
|
||||
|
||||
.harness/agents/*.md + platforms.yaml
|
||||
└── render_agents.py
|
||||
├── .claude/agents/*.md
|
||||
├── .codex/agents/*.toml
|
||||
└── .agents/agents/*/agent.json
|
||||
|
||||
Claude hook ───────────────┐
|
||||
Antigravity hook adapter ──┼── verdict validator ── evidence JSON
|
||||
Codex validation command ──┘
|
||||
```
|
||||
|
||||
### 4.1 Project registry
|
||||
|
||||
`modules.yaml` contains, per leaf module:
|
||||
|
||||
- stable module id
|
||||
- repository-relative source path
|
||||
- Gradle path
|
||||
- role
|
||||
- Java package roots (informational and import-policy lookup only)
|
||||
- allowed project dependencies
|
||||
- focused test command
|
||||
- profiles/capabilities
|
||||
- owning `CLAUDE.md` when present
|
||||
- an intentional mutation import used by gate tests
|
||||
|
||||
`src/settings.gradle` reads the registry to declare projects. The
|
||||
`verifyCleanArchitectureDependencies` task reads the same registry instead of maintaining a
|
||||
second dependency map.
|
||||
|
||||
### 4.2 Owning-module resolution
|
||||
|
||||
Owner selection uses the longest filesystem-boundary match among registered leaf source
|
||||
paths. Package prefixes never decide ownership because `support` owns a broad
|
||||
`dev.caskeleton.adapter.outbound` package and sample code mirrors production packages.
|
||||
|
||||
Instruction discovery walks upward from the touched file and returns the nearest
|
||||
`CLAUDE.md`; if a leaf has none, root `CLAUDE.md` and `AGENTS.md` are the explicit fallback.
|
||||
|
||||
### 4.3 Import gate
|
||||
|
||||
The import gate first resolves the registered leaf module, then applies:
|
||||
|
||||
- dependency-derived sibling module isolation
|
||||
- role-specific framework rules for domain, application, inbound, outbound, persistence,
|
||||
identifier, shared-contract, bootstrap, and sample roles
|
||||
- global unsafe-pattern checks
|
||||
|
||||
All registered production modules receive a mutation test using their real nested source
|
||||
path. Sample-only exemptions are explicit registry data, not accidental regex misses.
|
||||
|
||||
### 4.4 Verdict and evidence
|
||||
|
||||
The canonical verdict schema requires `agent`, `verdict`, `task_id`, `revision`, and agent
|
||||
specific evidence. Non-blocked verdicts require every declared field. Counts are non-negative.
|
||||
Required equations include:
|
||||
|
||||
- spec totals balance
|
||||
- Gradle `run = passed + failed + skipped`
|
||||
- ready Gradle results include at least one command and no failed command
|
||||
- behavior-changing implementation requires at least one observed red test
|
||||
- quality-ready references validated architecture and spec artifacts
|
||||
|
||||
Claude's fenced `ca-verdict` remains a compatibility input, but its accepted form is converted
|
||||
to the same JSON evidence model. Missing or malformed payloads for detected CA agents fail
|
||||
closed. Evidence records include a source-message hash and current revision/diff identity.
|
||||
|
||||
### 4.5 Platform rendering and hook adapters
|
||||
|
||||
Canonical agent Markdown lives under `.harness/agents/`; platform metadata lives in
|
||||
`.harness/project/platforms.yaml`. Generated files carry `generated_from`, `source_hash`,
|
||||
`generator_version`, and `do_not_edit` metadata.
|
||||
|
||||
Antigravity gains a plugin `hooks.json` and a platform adapter using the documented camelCase
|
||||
stdin/stdout contract. Claude keeps its native hook entry points but calls the common library.
|
||||
Codex variants instruct the runner/reviewer to invoke the common validation command because
|
||||
the repository has no equivalent local lifecycle-hook registration surface.
|
||||
|
||||
### 4.6 Risk and review profiles
|
||||
|
||||
Risk is determined by change surface, not file count:
|
||||
|
||||
- high: security, migration/schema, public contract, module dependency, architecture rule,
|
||||
transaction/concurrency, CI/deployment
|
||||
- medium: behavior, multiple modules, external integration
|
||||
- low: docs/comments, test fixture, local refactor protected by characterization tests
|
||||
|
||||
Review profiles:
|
||||
|
||||
- `review-lite`: direct diff references; no saved report by default
|
||||
- `review-standard`: verify blocking citations; one report only when risk or findings justify it
|
||||
- `audit-deep`: verify all quotes and persist detailed findings
|
||||
- `regulated`: immutable evidence and full traceability
|
||||
|
||||
Option analysis uses a dependency DAG and at most 3–5 materially distinct alternatives.
|
||||
Counterarguments are required for judgment-dependent findings, not deterministic failures.
|
||||
|
||||
## 5. Commit Policy
|
||||
|
||||
All platforms use `human-only`. Implementers never stage or commit. Reviewers may inspect a
|
||||
working-tree diff before commit or an explicit immutable range after the human commits.
|
||||
|
||||
## 6. Verification Strategy
|
||||
|
||||
1. Stdlib unit tests for registry loading and owner resolution.
|
||||
2. Mutation tests for every registered production module path.
|
||||
3. Strict verdict negative tests: missing fields, negatives, arithmetic imbalance, missing
|
||||
upstream evidence, revision mismatch, and behavior change without red evidence.
|
||||
4. Golden renderer tests and `--check` parity validation.
|
||||
5. JSON validation of generated Antigravity hook and agent files.
|
||||
6. Gradle `projects`, architecture dependency verification, focused ArchUnit test, and full
|
||||
`check` after harness tests pass.
|
||||
|
||||
## 7. Migration and Compatibility
|
||||
|
||||
- Existing fenced verdicts remain parseable only when they satisfy the new required fields.
|
||||
- Generated platform files are overwritten only by the renderer and documented as generated.
|
||||
- Root and module guidance is updated to the registered nested topology.
|
||||
- Actual external cross-platform golden executions remain a follow-up; static parity and seeded
|
||||
mutation coverage become mandatory in this change.
|
||||
|
||||
## 8. Acceptance Criteria
|
||||
|
||||
- A seeded forbidden import under every nested production module is rejected.
|
||||
- No legacy flat adapter path remains in gate tests or agent task allowlists.
|
||||
- `settings.gradle`, dependency verification, import gate, and Gradle runner resolve the same
|
||||
19 leaf modules from `modules.yaml`.
|
||||
- Missing/negative/inconsistent ready verdicts fail validation.
|
||||
- Claude and Antigravity adapters invoke the shared validator; accepted verdicts produce JSON
|
||||
evidence.
|
||||
- Rendering followed by `--check` reports no platform drift.
|
||||
- Agent variants uniformly state human-only commit policy and risk-based orchestration.
|
||||
- `N!` enumeration, all-quote routine grep, file-count report splitting, and unconditional
|
||||
counterargument requirements are absent from active rules.
|
||||
- Harness tests and Gradle checks pass, or every unrun/failing command is reported with risk.
|
||||
Reference in New Issue
Block a user