Add platform infrastructure configuration

This commit is contained in:
donghyeon-ka
2026-08-28 17:35:41 +09:00
parent fa76531e5b
commit 16c337bcc9
302 changed files with 83259 additions and 1 deletions
+287
View File
@@ -0,0 +1,287 @@
# Traefik 경계 계약과 적용 기록
Traefik은 k3s가 관리하는 클러스터 내부 Ingress Controller다. 공개 요청은 반드시
Host Nginx에서 TLS를 종료한 뒤 loopback NodePort를 통해 Traefik의 `web`
entrypoint로 들어온다. 이 저장소는 두 번째 Ingress Controller를 설치하지 않으며,
애플리케이션 Ingress에 클러스터 내부 TLS를 중복 구성하지 않는다.
2026-07-23 현재 `kube-system/traefik` `HelmChartConfig`에는 trust overlay가 실제로
적용돼 있다. Host Nginx 경유 관측에서 확인한 `ClientHost` `10.42.0.1` 한 주소만
`10.42.0.1/32`로 신뢰하며, Gitea의 site manifest도 외부 HTTPS URL을 생성한다.
## 현재 live 상태
| 항목 | 확인된 값 |
|---|---|
| k3s Traefik Chart | `40.1.3+up40.1.0` |
| Traefik 이미지 | `v3.7.4` |
| `HelmChartConfig` | trust overlay와 일치, live 적용됨 |
| Service 유형 | `NodePort`, `externalTrafficPolicy: Cluster` |
| `web` | Service `80`, NodePort `30080` |
| `websecure` | Service `443`, NodePort `30443` |
| NodePort bind 범위 | `127.0.0.0/8` |
| JSON access log | 활성화, request header 기록 제외 |
| 관측 `ClientHost` | `10.42.0.1` |
| `web` trusted CIDR | `10.42.0.1/32` |
| `websecure` forwarded-header trust | 없음 |
| `forwardedHeaders.insecure` | 없음 |
| Gitea site manifest | `start_url`과 icon URL 모두 `https://git.learn.hyeonworks.com/` 기준 |
| 실제 ingress 경로 | `Host Nginx :443 -> 127.0.0.1:30080 -> Traefik web` |
다음 명령으로 변할 수 있는 live 상태를 다시 확인한다.
```sh
kubectl -n kube-system get helmchartconfig.helm.cattle.io/traefik
kubectl -n kube-system get service/traefik \
-o custom-columns='NAME:.metadata.name,TYPE:.spec.type,PORTS:.spec.ports[*].port,NODEPORTS:.spec.ports[*].nodePort'
kubectl -n kube-system get deployment/traefik -o json |
jq -r '.spec.template.spec.containers[] | select(.name == "traefik") | .args[]'
```
`web=30080`, `websecure=30443`, Service `NodePort` 중 하나라도 다르면 Host Nginx를
새 포트로 임의 변경하지 말고 중지한다. 선언과 live 상태가 왜 달라졌는지 먼저
확인한다.
## 트래픽과 노출 경계
- 애플리케이션 Ingress가 hostname에서 Service로 이어지는 routing을 소유한다.
- 모든 Ingress는 `spec.ingressClassName: traefik``web` entrypoint를 명시한다.
- 공개 TLS는 Host Nginx가 종료하므로 애플리케이션 Ingress에 `spec.tls`를 넣지 않는다.
- `websecure` NodePort `30443`은 Service 계약상 고정하지만 현재 Host Nginx upstream은
사용하지 않는다. 이 entrypoint에는 forwarded-header trust도 설정하지 않는다.
- Traefik Dashboard와 관리 endpoint는 공개하지 않는다.
- k3s drop-in의 `nodeport-addresses=127.0.0.0/8`이 LAN에서 NodePort에 직접
접근하는 우회 경로를 차단한다.
서버 node IP와 별도 LAN 클라이언트에서는 다음 연결이 거부되거나 timeout이어야
한다. Traefik `404`도 TCP 연결에 성공했다는 뜻이므로 실패다.
```sh
nc -vz -w 3 192.168.0.107 30080
nc -vz -w 3 192.168.0.107 30443
```
반대로 서버 loopback에서는 두 포트가 listening 상태여야 하며 Host 기반 Gitea
health가 통과해야 한다.
```sh
nc -vz -w 3 127.0.0.1 30080
nc -vz -w 3 127.0.0.1 30443
curl --fail-with-body \
--header 'Host: git.learn.hyeonworks.com' \
http://127.0.0.1:30080/api/healthz
```
UFW는 현재 inactive다. 인터넷 측 고포트 차단 여부는 LAN 결과에서 추론하지 않고
router 규칙 또는 별도 외부망 검사로 확인한다.
## 선언 구조와 각 overlay의 역할
```text
infrastructure/networking/traefik/
├── base/
│ └── helm-chart-config.yaml
├── overlays/
│ ├── baseline/
│ │ └── service-boundary-only-patch.yaml
│ ├── observe/
│ └── trust/
│ └── trusted-proxy-cidr-patch.yaml
└── scripts/
├── apply-observe.sh
├── observe-client-host.sh
├── apply-trust.sh
├── rollback-to-observe.sh
└── validate.sh
```
세 overlay는 모두 Service `NodePort`, `externalTrafficPolicy: Cluster`
`30080/30443`을 명시적으로 소유한다.
- `baseline`: Service 경계만 남긴다. access log와 forwarded-header trust는 없다.
- `observe`: Service 경계와 header를 버리는 JSON access log를 적용한다. trust는 없다.
- `trust`: observe 설정에 `web.forwardedHeaders.trustedIPs=10.42.0.1/32`만 추가한다.
루트 `kustomization.yaml`은 의도적으로 안전한 `observe` overlay를 가리킨다. 현재
live 상태는 `trust`이므로 루트에 단순히 `kubectl apply -k`를 실행하면 trust 제거를
요청하게 된다. 상태 전환은 아래 guarded script와 정확한 overlay를 사용한다.
Chart 원본이나 k3s가 소유한 `HelmChart`는 직접 수정하지 않는다.
## 첫 observe 적용 실패와 복구
첫 observe 적용 때 `HelmChartConfig`에는 access log만 있고 Traefik Service values가
없었다. k3s Helm Controller가 전체 Chart를 기본값으로 다시 조정하면서 다음 drift가
발생했다.
```text
기존: NodePort web=30080, websecure=30443
변경: LoadBalancer web=31251, websecure=30997
```
이어진 loopback listener 검사가 실패했다. 당시 실패 처리도 새
`HelmChartConfig`를 삭제했을 뿐, desired state에 없던 수동 Service spec은 복원하지
못했다. Gitea·PostgreSQL·PV/PVC는 건드리지 않고 Traefik Service만 다음 명령으로
즉시 원래 경계에 복구했다.
```sh
kubectl -n kube-system patch service traefik \
--type=merge \
--patch '{"spec":{"type":"NodePort","externalTrafficPolicy":"Cluster","ports":[{"name":"web","port":80,"protocol":"TCP","targetPort":"web","nodePort":30080},{"name":"websecure","port":443,"protocol":"TCP","targetPort":"websecure","nodePort":30443}]}}'
```
그 뒤 다음을 영구 보완했다.
- `base`, `baseline`, `observe`, `trust`가 Service type과 정확한 NodePort를 선언한다.
- `baseline` overlay를 추가해 access log나 trust 없이도 NodePort desired state를
유지한다.
- observe 실패 시 `HelmChartConfig`를 삭제하지 않고 baseline을 적용한다.
- trust 실패 또는 표준 trust 롤백 시 observe를 적용한다.
- rollout 뒤 NodePort listener와 Gitea health가 수렴할 때까지 bounded wait를 한다.
- 검증기는 세 overlay에서 LoadBalancer 부재와 `30080/30443`을 강제한다.
따라서 `HelmChartConfig` 삭제는 더 이상 롤백 방법이 아니다. 삭제하면 Chart 기본값이
다시 Service를 소유해 같은 drift를 재발시킬 수 있다.
## 전달 헤더 최소 신뢰 적용 결과
Host Nginx는 외부 요청의 기존 forwarded chain을 이어 붙이지 않고 신뢰 경계에서
다음 값을 새로 만든다.
- `Host`는 선택한 공개 hostname으로 고정한다.
- `X-Real-IP``X-Forwarded-For`는 Nginx가 실제로 본 client address로 교체한다.
- `X-Forwarded-Proto``https`, `X-Forwarded-Port``443`으로 고정한다.
observe 단계에서 다음 probe가 Host Nginx를 반드시 통과하는 고유 요청을 만들고
Traefik JSON access log의 한 router 기록만 읽었다. request header와 자격 증명은
로그에 남기지 않았다.
```sh
bash infrastructure/networking/traefik/scripts/observe-client-host.sh
```
확인 결과는 다음과 같다.
```text
ClientHost: 10.42.0.1
Minimum trusted CIDR: 10.42.0.1/32
```
Pod CIDR 전체, loopback 전체 또는 LAN CIDR을 추정해 넓히지 않고 이 한 주소만 trust
overlay에 기록했다. 적용 명령과 승인 문자열은 다음과 같았다.
```sh
bash infrastructure/networking/traefik/scripts/apply-trust.sh \
--observed-client-host '10.42.0.1' \
--execute
```
```text
APPLY default TRUST 10.42.0.1/32
```
현재 runtime에는 다음 trust 인자 하나만 존재한다.
```text
--entryPoints.web.forwardedHeaders.trustedIPs=10.42.0.1/32
```
`entryPoints.websecure.forwardedHeaders.*``forwardedHeaders.insecure` 인자는 없다.
적용 후 `/assets/site-manifest.json``start_url`과 두 icon URL이 모두 HTTPS로
확인됐고 Gitea health의 status·database·cache 검사도 통과했다.
## 검증과 상태 전환
소스와 세 overlay의 정적 계약은 다음 명령으로 검증한다.
```sh
cd /home/donghyeon/workspace/platform
bash infrastructure/networking/traefik/scripts/validate.sh
```
검증기는 다음 조건을 강제한다.
- 세 overlay의 Service가 `NodePort`, `externalTrafficPolicy: Cluster`,
`30080/30443`을 유지한다.
- observe와 trust access log는 JSON이고 request header를 기록하지 않는다.
- trust CIDR은 관측한 단일 host `/32` 또는 `/128` 형식이다.
- `forwardedHeaders.insecure`, `websecure` trust, `LoadBalancer`가 없다.
새 설치처럼 `HelmChartConfig`가 없거나 이미 observe 상태인 경우에는 다음 guarded
script로 observe 구성을 확인하거나 적용한다.
```sh
bash infrastructure/networking/traefik/scripts/apply-observe.sh --execute
# 승인: APPLY <현재-context> OBSERVE
```
현재 live trust에서 다시 관측하려면 먼저 아래 표준 롤백으로 observe를 적용한 뒤
probe를 실행한다. trust 상태에서 `apply-observe.sh`를 바로 실행하지 않는다.
```sh
bash infrastructure/networking/traefik/scripts/rollback-to-observe.sh --execute
bash infrastructure/networking/traefik/scripts/observe-client-host.sh
```
관측값이 달라지면 기존 CIDR을 넓히지 말고 trust patch를 exact host CIDR로 갱신한 뒤
`apply-trust.sh`를 실행한다. Chart, 이미지, context, API server, NodePort 경계 또는
재관측 값이 기대와 다르면 스크립트가 적용을 중단한다.
### 롤백
trust만 제거하고 JSON access log를 남기는 표준 롤백은 observe overlay를 적용한다.
```sh
bash infrastructure/networking/traefik/scripts/rollback-to-observe.sh --execute
# 승인: ROLLBACK <현재-context> OBSERVE
```
observe 적용 자체가 실패하면 `apply-observe.sh`가 NodePort-only baseline overlay를
적용한다. access log와 trust를 모두 제거해야 하는 명시적 유지보수에서는 live
context와 대상 overlay를 재확인한 뒤 baseline을 적용한다.
```sh
kubectl apply --kustomize \
infrastructure/networking/traefik/overlays/baseline
```
어느 경우에도 `HelmChartConfig`를 삭제해 롤백하지 않는다. baseline 또는 observe를
적용해 Service `30080/30443`을 계속 desired state로 남긴다.
k3s·kube-proxy·CNI·Service traffic policy나 Host Nginx 경로를 바꾸면
`ClientHost`가 달라질 수 있다. 이때는 observe로 돌아가 다시 관측하고 정확한 한
주소만 trust한다.
## 종단 간 인수 조건
다음 로컬 검사는 Host Nginx와 Traefik을 함께 통과해야 한다.
```sh
curl --fail-with-body \
--resolve git.learn.hyeonworks.com:443:127.0.0.1 \
https://git.learn.hyeonworks.com/api/healthz
curl --fail-with-body \
--resolve git.learn.hyeonworks.com:443:127.0.0.1 \
https://git.learn.hyeonworks.com/assets/site-manifest.json |
jq -e '
.start_url == "https://git.learn.hyeonworks.com/" and
([.icons[].src | startswith("https://git.learn.hyeonworks.com/")] | all)
'
```
별도 LAN 클라이언트에서 `192.168.0.107:30080/30443`이 거부되는지 다시 확인하고,
독립 외부망에서는 공개 HTTP→HTTPS redirect와 두 서비스의 HTTPS 응답을 검사한다.
서버에서 공인 FQDN으로 향하는 NAT hairpin timeout만으로 공개 실패를 판정하지 않는다.
구현 근거는 [k3s HelmChartConfig](https://docs.k3s.io/helm),
[k3s 내장 Traefik](https://docs.k3s.io/networking/networking-services),
[Traefik forwarded headers](https://doc.traefik.io/traefik/reference/install-configuration/entrypoints/),
[Traefik access log](https://doc.traefik.io/traefik/observe/logs-and-access-logs/),
[Traefik Chart 40.1.0 values](https://github.com/traefik/traefik-helm-chart/blob/v40.1.0/traefik/values.yaml)다.
첫 실패, 수동 복구, 영구 보완, 관측값과 trust 적용의 전체 명령·출력은
[중앙 실행 기록](../../../../docs/platform/runbooks/2026-07-23-traefik-forwarded-header-trust-boundary.md)에
보존한다.
@@ -0,0 +1,45 @@
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
labels:
app.kubernetes.io/part-of: platform
app.kubernetes.io/managed-by: kustomize
spec:
failurePolicy: abort
valuesContent: |-
deployment:
podAnnotations:
prometheus.io/path: /metrics
prometheus.io/port: "9100"
prometheus.io/scrape: "true"
service:
spec:
type: NodePort
externalTrafficPolicy: Cluster
ports:
web:
nodePort: 30080
websecure:
nodePort: 30443
metrics:
prometheus:
service:
enabled: true
serviceMonitor:
enabled: true
additionalLabels:
observability.hyeonworks.com/instance: home
jobLabel: app.kubernetes.io/name
interval: 30s
scrapeTimeout: 10s
logs:
access:
enabled: true
format: json
fields:
general:
defaultmode: keep
headers:
defaultmode: drop
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- helm-chart-config.yaml
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- overlays/observe
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
- path: service-boundary-only-patch.yaml
target:
group: helm.cattle.io
version: v1
kind: HelmChartConfig
name: traefik
namespace: kube-system
@@ -0,0 +1,28 @@
- op: replace
path: /spec/valuesContent
value: |-
deployment:
podAnnotations:
prometheus.io/path: /metrics
prometheus.io/port: "9100"
prometheus.io/scrape: "true"
service:
spec:
type: NodePort
externalTrafficPolicy: Cluster
ports:
web:
nodePort: 30080
websecure:
nodePort: 30443
metrics:
prometheus:
service:
enabled: true
serviceMonitor:
enabled: true
additionalLabels:
observability.hyeonworks.com/instance: home
jobLabel: app.kubernetes.io/name
interval: 30s
scrapeTimeout: 10s
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
- path: trusted-proxy-cidr-patch.yaml
target:
group: helm.cattle.io
version: v1
kind: HelmChartConfig
name: traefik
namespace: kube-system
@@ -0,0 +1,41 @@
- op: replace
path: /spec/valuesContent
value: |-
deployment:
podAnnotations:
prometheus.io/path: /metrics
prometheus.io/port: "9100"
prometheus.io/scrape: "true"
service:
spec:
type: NodePort
externalTrafficPolicy: Cluster
ports:
web:
nodePort: 30080
forwardedHeaders:
trustedIPs:
# 2026-07-23 Host Nginx 경유 probe에서 관측한 Traefik ClientHost이다.
- "10.42.0.1/32"
websecure:
nodePort: 30443
metrics:
prometheus:
service:
enabled: true
serviceMonitor:
enabled: true
additionalLabels:
observability.hyeonworks.com/instance: home
jobLabel: app.kubernetes.io/name
interval: 30s
scrapeTimeout: 10s
logs:
access:
enabled: true
format: json
fields:
general:
defaultmode: keep
headers:
defaultmode: drop
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env bash
set -Eeuo pipefail
set +x
umask 077
readonly SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
usage() {
cat <<'USAGE'
Usage: bash infrastructure/networking/traefik/scripts/apply-observe.sh --execute
Applies JSON access logging to the k3s-managed Traefik HelmChartConfig.
It does not trust any forwarded header. The existing loopback-only
NodePort 30080/30443 boundary is declared explicitly so Helm reconciliation
cannot replace it with chart defaults.
USAGE
}
[[ "${1:-}" == "--execute" && "$#" -eq 1 ]] || {
usage
exit 2
}
require_commands kubectl jq rg curl nc python3 awk cmp find bash
bash "${SCRIPT_DIR}/validate.sh"
work_dir="$(mktemp -d "${TMPDIR:-/tmp}/platform-traefik-observe.XXXXXX")"
rollback_required=false
selected_context=""
cleanup() {
local exit_code=$?
trap - EXIT
if [[ "$exit_code" -ne 0 && "$rollback_required" == "true" ]]; then
printf '\nROLLBACK: restoring the durable NodePort-only baseline.\n' >&2
set +e
kubectl --context "$selected_context" apply --kustomize "$BASELINE_OVERLAY"
wait_for_runtime baseline
kubectl --context "$selected_context" --namespace kube-system \
rollout status deployment/traefik --timeout=5m
wait_for_nodeport_boundary_and_health
printf 'ROLLBACK complete. NodePort 30080/30443 remains pinned.\n' >&2
set -e
fi
case "$work_dir" in
/tmp/platform-traefik-observe.*|"${TMPDIR:-/tmp}"/platform-traefik-observe.*)
rm -rf -- "$work_dir"
;;
*)
printf 'WARNING: refusing to remove unexpected directory: %s\n' \
"$work_dir" >&2
;;
esac
exit "$exit_code"
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
observe_render="${work_dir}/observe.yaml"
render_overlay "$OBSERVE_OVERLAY" "$observe_render"
assert_live_baseline
assert_nodeport_boundary_and_health
selected_context="$(current_context)"
selected_api_server="$(current_api_server)"
printf '\nKubernetes context: %s\nAPI server: %s\nTarget node: %s\n' \
"$selected_context" "$selected_api_server" "$TARGET_NODE"
already_applied=false
if kubectl --namespace kube-system \
get helmchartconfig.helm.cattle.io traefik >/dev/null 2>&1; then
assert_live_hcc_matches_overlay "$OBSERVE_OVERLAY" \
"${work_dir}/live-observe-compare.yaml"
already_applied=true
printf 'The live HelmChartConfig already matches the observation overlay.\n'
fi
printf 'Type APPLY %s OBSERVE to enable JSON access logs: ' "$selected_context"
read -r confirmation
[[ "$confirmation" == "APPLY ${selected_context} OBSERVE" ]] || \
fail "cancelled"
[[ "$(current_context)" == "$selected_context" ]] || \
fail "kubectl context changed after confirmation"
[[ "$(current_api_server)" == "$selected_api_server" ]] || \
fail "Kubernetes API server changed after confirmation"
assert_live_baseline
assert_nodeport_boundary_and_health
if [[ "$already_applied" == "false" ]]; then
rollback_required=true
kubectl apply --filename "$observe_render"
fi
wait_for_runtime observe
kubectl --namespace kube-system rollout status deployment/traefik --timeout=5m
assert_runtime observe
assert_live_hcc_matches_overlay "$OBSERVE_OVERLAY" \
"${work_dir}/post-apply-observe.yaml"
assert_live_baseline
wait_for_nodeport_boundary_and_health
manifest_start_url="$(
curl --fail-with-body --silent --show-error \
--resolve "${GITEA_HOST}:443:127.0.0.1" \
"$GITEA_MANIFEST_URL" |
jq --raw-output '.start_url'
)"
rollback_required=false
printf '\nOBSERVATION PHASE READY\n'
printf 'Traefik JSON access logging: enabled\n'
printf 'forwardedHeaders trust: absent\n'
printf 'Current Gitea manifest start_url: %s\n' "$manifest_start_url"
printf 'Next: bash %s/observe-client-host.sh\n' "$SCRIPT_DIR"
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env bash
set -Eeuo pipefail
set +x
umask 077
readonly SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
usage() {
cat <<'USAGE'
Usage:
bash infrastructure/networking/traefik/scripts/apply-trust.sh \
--observed-client-host <IP> --execute
The IP must be the ClientHost printed by observe-client-host.sh. The trust
overlay must already contain that exact IP as /32 (IPv4) or /128 (IPv6).
USAGE
}
[[ "$#" -eq 3 && "$1" == "--observed-client-host" && "$3" == "--execute" ]] || {
usage
exit 2
}
readonly REVIEWED_CLIENT_HOST="$2"
require_commands kubectl jq rg curl nc python3 awk sort date cmp find bash
reviewed_cidr="$(host_to_exact_cidr "$REVIEWED_CLIENT_HOST")"
declared_cidr="$(source_trusted_proxy_cidr)"
declared_cidr="$(normalize_exact_host_cidr "$declared_cidr")"
[[ "$declared_cidr" != "$SENTINEL_TRUSTED_PROXY_CIDR" ]] || \
fail "trust overlay still contains the non-routable sentinel CIDR"
[[ "$declared_cidr" == "$reviewed_cidr" ]] || \
fail "declared CIDR ${declared_cidr} does not match ClientHost ${REVIEWED_CLIENT_HOST}"
bash "${SCRIPT_DIR}/validate.sh"
work_dir="$(mktemp -d "${TMPDIR:-/tmp}/platform-traefik-trust.XXXXXX")"
rollback_required=false
selected_context=""
cleanup() {
local exit_code=$?
trap - EXIT
if [[ "$exit_code" -ne 0 && "$rollback_required" == "true" ]]; then
printf '\nROLLBACK: restoring the access-log-only observation overlay.\n' >&2
set +e
kubectl --context "$selected_context" apply --kustomize "$OBSERVE_OVERLAY"
wait_for_runtime observe
kubectl --context "$selected_context" --namespace kube-system \
rollout status deployment/traefik --timeout=5m
wait_for_nodeport_boundary_and_health
printf 'ROLLBACK complete. Forwarded-header trust removal was requested.\n' >&2
set -e
fi
case "$work_dir" in
/tmp/platform-traefik-trust.*|"${TMPDIR:-/tmp}"/platform-traefik-trust.*)
rm -rf -- "$work_dir"
;;
*)
printf 'WARNING: refusing to remove unexpected directory: %s\n' \
"$work_dir" >&2
;;
esac
exit "$exit_code"
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
trust_render="${work_dir}/trust.yaml"
render_overlay "$TRUST_OVERLAY" "$trust_render"
assert_live_baseline
assert_runtime observe
assert_live_hcc_matches_overlay "$OBSERVE_OVERLAY" \
"${work_dir}/live-observe.yaml"
assert_nodeport_boundary_and_health
fresh_client_host="$(observe_host_nginx_client_host)"
fresh_cidr="$(host_to_exact_cidr "$fresh_client_host")"
[[ "$fresh_cidr" == "$reviewed_cidr" ]] || \
fail "fresh ClientHost ${fresh_client_host} differs from reviewed ${REVIEWED_CLIENT_HOST}"
selected_context="$(current_context)"
selected_api_server="$(current_api_server)"
printf '\nKubernetes context: %s\nAPI server: %s\nTarget node: %s\n' \
"$selected_context" "$selected_api_server" "$TARGET_NODE"
printf 'Fresh ClientHost: %s\nExact trusted CIDR: %s\n' \
"$fresh_client_host" "$declared_cidr"
printf 'Type APPLY %s TRUST %s to continue: ' \
"$selected_context" "$declared_cidr"
read -r confirmation
[[ "$confirmation" == "APPLY ${selected_context} TRUST ${declared_cidr}" ]] || \
fail "cancelled"
[[ "$(current_context)" == "$selected_context" ]] || \
fail "kubectl context changed after confirmation"
[[ "$(current_api_server)" == "$selected_api_server" ]] || \
fail "Kubernetes API server changed after confirmation"
assert_live_baseline
assert_runtime observe
assert_nodeport_boundary_and_health
assert_live_hcc_matches_overlay "$OBSERVE_OVERLAY" \
"${work_dir}/pre-apply-observe.yaml"
rollback_required=true
kubectl apply --filename "$trust_render"
wait_for_runtime trust "$declared_cidr"
kubectl --namespace kube-system rollout status deployment/traefik --timeout=5m
assert_runtime trust "$declared_cidr"
assert_live_hcc_matches_overlay "$TRUST_OVERLAY" \
"${work_dir}/post-apply-trust.yaml"
assert_live_baseline
wait_for_nodeport_boundary_and_health
assert_manifest_https
rollback_required=false
printf '\nTRUST PHASE READY\n'
printf 'Traefik web trusted CIDR: %s\n' "$declared_cidr"
printf 'Traefik websecure trusted CIDR: absent\n'
printf 'forwardedHeaders.insecure: absent\n'
printf 'Gitea health and HTTPS site-manifest checks: PASS\n'
printf 'Repeat the 30080/30443 refusal check from a separate LAN client.\n'
+464
View File
@@ -0,0 +1,464 @@
#!/usr/bin/env bash
# 이 파일은 같은 디렉터리의 실행 스크립트에서만 source한다.
readonly TRAEFIK_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd -P)"
readonly BASELINE_OVERLAY="${TRAEFIK_ROOT}/overlays/baseline"
readonly OBSERVE_OVERLAY="${TRAEFIK_ROOT}/overlays/observe"
readonly TRUST_OVERLAY="${TRAEFIK_ROOT}/overlays/trust"
readonly TRUST_PATCH="${TRUST_OVERLAY}/trusted-proxy-cidr-patch.yaml"
readonly SENTINEL_TRUSTED_PROXY_CIDR="192.0.2.1/32"
readonly TARGET_NODE="donghyeon-system-product-name"
readonly K3S_NODEPORT_CONFIG="/etc/rancher/k3s/config.yaml.d/30-nodeport-loopback.yaml"
readonly EXPECTED_K3S_CHART="https://%{KUBERNETES_API}%/static/charts/traefik-40.1.3+up40.1.0.tgz"
readonly EXPECTED_CHART_LABEL="traefik-40.1.3_up40.1.0"
readonly EXPECTED_TRAEFIK_IMAGE="rancher/mirrored-library-traefik:3.7.4"
readonly GITEA_HOST="git.learn.hyeonworks.com"
readonly GITEA_HEALTH_URL="http://127.0.0.1:30080/api/healthz"
readonly GITEA_HTTPS_HEALTH_URL="https://${GITEA_HOST}/api/healthz"
readonly GITEA_MANIFEST_URL="https://${GITEA_HOST}/assets/site-manifest.json"
fail() {
printf 'ERROR: %s\n' "$*" >&2
return 1
}
require_commands() {
local command_name
for command_name in "$@"; do
command -v "$command_name" >/dev/null 2>&1 || \
fail "${command_name} is required"
done
}
render_overlay() {
local overlay="$1"
local output="$2"
kubectl kustomize "$overlay" >"$output"
[[ -s "$output" ]] || fail "rendered manifest is empty: ${overlay}"
}
manifest_values_content() {
local manifest="$1"
awk '
/^ valuesContent: \|-$/ {
found = 1
next
}
found {
sub(/^ /, "")
print
}
' "$manifest"
}
source_trusted_proxy_cidr() {
local -a values=()
mapfile -t values < <(
awk -F'"' '/^[[:space:]]*-[[:space:]]*"/ { print $2 }' "$TRUST_PATCH"
)
[[ "${#values[@]}" -eq 1 ]] || \
fail "trust patch must contain exactly one quoted trusted CIDR"
printf '%s\n' "${values[0]}"
}
host_to_exact_cidr() {
local host="$1"
python3 - "$host" <<'PY'
import ipaddress
import sys
value = sys.argv[1]
if "/" in value:
raise SystemExit("ClientHost must be one IP address, not a CIDR")
address = ipaddress.ip_address(value)
if address.is_unspecified or address.is_multicast:
raise SystemExit("ClientHost cannot be unspecified or multicast")
prefix = 32 if address.version == 4 else 128
print(f"{address.compressed}/{prefix}")
PY
}
normalize_exact_host_cidr() {
local cidr="$1"
python3 - "$cidr" <<'PY'
import ipaddress
import sys
network = ipaddress.ip_network(sys.argv[1], strict=True)
required_prefix = 32 if network.version == 4 else 128
if network.prefixlen != required_prefix:
raise SystemExit(
f"trusted proxy range must be one exact host /{required_prefix}, "
f"not {network.with_prefixlen}"
)
print(network.with_prefixlen)
PY
}
current_context() {
kubectl config current-context
}
current_api_server() {
kubectl config view --minify --output=jsonpath='{.clusters[0].cluster.server}'
}
deployment_args() {
kubectl --namespace kube-system get deployment traefik --output=json |
jq --raw-output '
.spec.template.spec.containers[]
| select(.name == "traefik")
| .args[]
'
}
runtime_matches() {
local mode="$1"
local trusted_cidr="${2:-}"
local args
args="$(deployment_args 2>/dev/null)" || return 1
if rg --quiet --ignore-case -- 'forwardedheaders\.insecure' <<<"$args"; then
return 1
fi
if [[ "$mode" != "baseline" ]]; then
rg --quiet --fixed-strings --line-regexp -- '--accesslog=true' <<<"$args" || \
return 1
rg --quiet --fixed-strings --line-regexp -- '--accesslog.format=json' <<<"$args" || \
return 1
fi
case "$mode" in
observe)
! rg --quiet --ignore-case -- 'forwardedheaders\.trustedips' <<<"$args"
;;
trust)
rg --quiet --fixed-strings --line-regexp -- \
"--entryPoints.web.forwardedHeaders.trustedIPs=${trusted_cidr}" <<<"$args" || \
return 1
! rg --quiet --ignore-case -- \
'entrypoints\.websecure\.forwardedheaders\.(trustedips|insecure)' <<<"$args"
;;
baseline)
! rg --quiet --ignore-case -- \
'accesslog|forwardedheaders\.(trustedips|insecure)' <<<"$args"
;;
*)
return 1
;;
esac
}
assert_runtime() {
local mode="$1"
local trusted_cidr="${2:-}"
local args
local trusted_count
args="$(deployment_args)"
rg --quiet --fixed-strings --line-regexp -- '--accesslog=true' <<<"$args" || \
fail "Traefik runtime is missing --accesslog=true"
rg --quiet --fixed-strings --line-regexp -- '--accesslog.format=json' <<<"$args" || \
fail "Traefik runtime is missing JSON access-log format"
if rg --quiet --ignore-case -- 'forwardedheaders\.insecure' <<<"$args"; then
fail "forwardedHeaders.insecure must never be present"
fi
trusted_count="$(
rg --count --ignore-case -- 'forwardedheaders\.trustedips' <<<"$args" || true
)"
trusted_count="${trusted_count:-0}"
case "$mode" in
observe)
[[ "$trusted_count" == "0" ]] || \
fail "observation phase must not trust forwarded headers"
;;
trust)
[[ "$trusted_count" == "1" ]] || \
fail "trust phase must render exactly one trustedIPs argument"
rg --quiet --fixed-strings --line-regexp -- \
"--entryPoints.web.forwardedHeaders.trustedIPs=${trusted_cidr}" <<<"$args" || \
fail "web entrypoint does not contain the reviewed exact-host CIDR"
if rg --quiet --ignore-case -- \
'entrypoints\.websecure\.forwardedheaders\.(trustedips|insecure)' <<<"$args"; then
fail "websecure must not receive forwarded-header trust"
fi
;;
*)
fail "unsupported runtime assertion mode: ${mode}"
;;
esac
}
wait_for_runtime() {
local mode="$1"
local trusted_cidr="${2:-}"
local attempt
for ((attempt = 1; attempt <= 120; attempt++)); do
if runtime_matches "$mode" "$trusted_cidr"; then
return 0
fi
sleep 2
done
fail "Traefik runtime did not reach ${mode} state within 240 seconds"
}
assert_live_baseline() {
local chart
local chart_label
local image
local nodeport_matches
local service_json
kubectl get node "$TARGET_NODE" >/dev/null
chart="$(
kubectl --namespace kube-system get helmchart.helm.cattle.io traefik \
--output=jsonpath='{.spec.chart}'
)"
[[ "$chart" == "$EXPECTED_K3S_CHART" ]] || \
fail "unexpected packaged Traefik chart: ${chart}"
chart_label="$(
kubectl --namespace kube-system get deployment traefik \
--output=jsonpath='{.metadata.labels.helm\.sh/chart}'
)"
[[ "$chart_label" == "$EXPECTED_CHART_LABEL" ]] || \
fail "unexpected live Traefik chart label: ${chart_label}"
image="$(
kubectl --namespace kube-system get deployment traefik --output=json |
jq --raw-output '
.spec.template.spec.containers[]
| select(.name == "traefik")
| .image
'
)"
[[ "$image" == "$EXPECTED_TRAEFIK_IMAGE" ]] || \
fail "unexpected live Traefik image: ${image}"
[[ -r "$K3S_NODEPORT_CONFIG" ]] || \
fail "cannot read the k3s nodeport-addresses drop-in: ${K3S_NODEPORT_CONFIG}"
rg --quiet --fixed-strings --line-regexp -- \
' - "nodeport-addresses=127.0.0.0/8"' "$K3S_NODEPORT_CONFIG" || \
fail "k3s nodeport-addresses is not pinned to 127.0.0.0/8"
nodeport_matches="$(
rg --no-heading --line-number -- 'nodeport-addresses[=:]' \
/etc/rancher/k3s/config.yaml \
/etc/rancher/k3s/config.yaml.d 2>/dev/null || true
)"
[[ "$(wc -l <<<"$nodeport_matches" | tr -d '[:space:]')" == "1" ]] || \
fail "nodeport-addresses must have exactly one k3s configuration owner"
rg --quiet --fixed-strings -- "$K3S_NODEPORT_CONFIG" <<<"$nodeport_matches" || \
fail "nodeport-addresses is owned by an unexpected k3s configuration file"
service_json="$(
kubectl --namespace kube-system get service traefik --output=json
)"
jq --exit-status '
.spec.type == "NodePort"
and .spec.externalTrafficPolicy == "Cluster"
and (.spec.ports | length) == 2
and any(.spec.ports[];
.name == "web"
and .port == 80
and .nodePort == 30080
and .protocol == "TCP")
and any(.spec.ports[];
.name == "websecure"
and .port == 443
and .nodePort == 30443
and .protocol == "TCP")
' >/dev/null <<<"$service_json" || \
fail "Traefik Service no longer matches the 80/30080 and 443/30443 boundary"
}
assert_health_body() {
local body="$1"
jq --exit-status '
.status == "pass"
and (.checks["database:ping"] | length) > 0
and all(.checks["database:ping"][]; .status == "pass")
and (.checks["cache:ping"] | length) > 0
and all(.checks["cache:ping"][]; .status == "pass")
' >/dev/null <<<"$body" || fail "Gitea database/cache health is not pass"
}
assert_nodeport_boundary_and_health() {
local body
local node_ip
local port
node_ip="$(
kubectl get node "$TARGET_NODE" --output=json |
jq --raw-output '
[.status.addresses[] | select(.type == "InternalIP") | .address]
| if length == 1 then .[0] else empty end
'
)"
[[ -n "$node_ip" && "$node_ip" != "127.0.0.1" ]] || \
fail "could not resolve exactly one non-loopback node InternalIP"
for port in 30080 30443; do
nc -z -w 3 127.0.0.1 "$port" >/dev/null 2>&1 || \
fail "loopback NodePort is not listening: 127.0.0.1:${port}"
if nc -z -w 3 "$node_ip" "$port" >/dev/null 2>&1; then
fail "NodePort escaped the loopback boundary: ${node_ip}:${port}"
fi
done
body="$(
curl --fail-with-body --silent --show-error \
--header "Host: ${GITEA_HOST}" \
"$GITEA_HEALTH_URL"
)"
assert_health_body "$body"
body="$(
curl --fail-with-body --silent --show-error \
--resolve "${GITEA_HOST}:443:127.0.0.1" \
"$GITEA_HTTPS_HEALTH_URL"
)"
assert_health_body "$body"
}
wait_for_nodeport_boundary_and_health() {
local attempt
for ((attempt = 1; attempt <= 60; attempt++)); do
if assert_nodeport_boundary_and_health >/dev/null 2>&1; then
return 0
fi
sleep 2
done
# 마지막 검사는 오류 원인을 숨기지 않고 그대로 출력한다.
assert_nodeport_boundary_and_health
fail "Traefik NodePort boundary and Gitea health did not recover within 120 seconds"
}
observe_host_nginx_client_host() {
local http_code
local logs
local matched
local nonce
local observed
local probe_path
local router_count
local since
local -a client_hosts=()
local attempt
since="$(date --utc '+%Y-%m-%dT%H:%M:%SZ')"
nonce="$(date --utc '+%Y%m%dT%H%M%S')-${BASHPID}"
probe_path="/api/healthz/traefik-source-${nonce}"
http_code="$(
curl --silent --show-error \
--resolve "${GITEA_HOST}:443:127.0.0.1" \
--output /dev/null \
--write-out '%{http_code}' \
"https://${GITEA_HOST}${probe_path}"
)"
[[ "$http_code" == "404" ]] || \
fail "unique Host Nginx observation request returned HTTP ${http_code}, expected 404"
matched=""
for ((attempt = 1; attempt <= 20; attempt++)); do
logs="$(
kubectl --namespace kube-system logs deployment/traefik \
--since-time "$since"
)"
matched="$(
jq --raw-input --compact-output --arg path "$probe_path" '
fromjson?
| select(.RequestPath == $path)
' <<<"$logs"
)"
[[ -n "$matched" ]] && break
sleep 1
done
[[ -n "$matched" ]] || \
fail "the unique request was not found in Traefik JSON access logs"
router_count="$(
jq --slurp '
[
.[]
| select(
((.RouterName // "") | ascii_downcase | contains("gitea"))
)
]
| length
' <<<"$matched"
)"
[[ "$router_count" -ge 1 ]] || \
fail "the observation log did not traverse a Gitea router"
mapfile -t client_hosts < <(
jq --raw-output '
select((.RouterName // "") | ascii_downcase | contains("gitea"))
| .ClientHost // empty
' <<<"$matched" |
sort --unique
)
[[ "${#client_hosts[@]}" -eq 1 && -n "${client_hosts[0]}" ]] || \
fail "expected one distinct Traefik ClientHost for the unique request"
observed="${client_hosts[0]}"
host_to_exact_cidr "$observed" >/dev/null
printf '%s\n' "$observed"
}
assert_manifest_https() {
local body
local attempt
for ((attempt = 1; attempt <= 30; attempt++)); do
body="$(
curl --fail-with-body --silent --show-error \
--resolve "${GITEA_HOST}:443:127.0.0.1" \
"$GITEA_MANIFEST_URL"
)"
if jq --exit-status --arg expected "https://${GITEA_HOST}/" '
.start_url == $expected
and all(.icons[]; (.src | startswith("https://")))
' >/dev/null <<<"$body"; then
return 0
fi
sleep 2
done
fail "Gitea site manifest did not stabilize on HTTPS URLs"
}
assert_live_hcc_matches_overlay() {
local overlay="$1"
local render_file="$2"
local actual
local expected
render_overlay "$overlay" "$render_file"
expected="$(manifest_values_content "$render_file")"
actual="$(
kubectl --namespace kube-system \
get helmchartconfig.helm.cattle.io traefik --output=json |
jq --raw-output '.spec.valuesContent'
)"
[[ "$actual" == "$expected" ]] || \
fail "live Traefik HelmChartConfig does not match the expected overlay"
}
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
set -Eeuo pipefail
set +x
umask 077
readonly SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
usage() {
cat <<'USAGE'
Usage: bash infrastructure/networking/traefik/scripts/observe-client-host.sh
Sends one unique HTTPS request through Host Nginx and extracts the corresponding
ClientHost from Traefik JSON access logs. It performs no cluster mutation.
USAGE
}
[[ "$#" -eq 0 ]] || {
usage
exit 2
}
require_commands kubectl jq rg curl nc python3 awk sort date
work_dir="$(mktemp -d "${TMPDIR:-/tmp}/platform-traefik-clienthost.XXXXXX")"
cleanup() {
case "$work_dir" in
/tmp/platform-traefik-clienthost.*|"${TMPDIR:-/tmp}"/platform-traefik-clienthost.*)
rm -rf -- "$work_dir"
;;
*)
printf 'WARNING: refusing to remove unexpected directory: %s\n' \
"$work_dir" >&2
;;
esac
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
bash "${SCRIPT_DIR}/validate.sh"
assert_live_baseline
assert_runtime observe
assert_live_hcc_matches_overlay "$OBSERVE_OVERLAY" \
"${work_dir}/live-observe.yaml"
assert_nodeport_boundary_and_health
observed_client_host="$(observe_host_nginx_client_host)"
trusted_proxy_cidr="$(host_to_exact_cidr "$observed_client_host")"
printf '\nTRAEFIK SOURCE OBSERVED\n'
printf 'ClientHost: %s\n' "$observed_client_host"
printf 'Minimum trusted CIDR: %s\n' "$trusted_proxy_cidr"
printf 'Record that CIDR in:\n%s\n' "$TRUST_PATCH"
printf 'Replace only the sentinel %s, then run apply-trust.sh with this ClientHost.\n' \
"$SENTINEL_TRUSTED_PROXY_CIDR"
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
set -Eeuo pipefail
set +x
umask 077
readonly SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
usage() {
cat <<'USAGE'
Usage:
bash infrastructure/networking/traefik/scripts/rollback-to-observe.sh --execute
Removes forwarded-header trust while retaining JSON access logging.
USAGE
}
[[ "${1:-}" == "--execute" && "$#" -eq 1 ]] || {
usage
exit 2
}
require_commands kubectl jq rg curl nc python3 awk cmp find bash
bash "${SCRIPT_DIR}/validate.sh"
assert_live_baseline
kubectl --namespace kube-system \
get helmchartconfig.helm.cattle.io traefik >/dev/null
assert_nodeport_boundary_and_health
selected_context="$(current_context)"
selected_api_server="$(current_api_server)"
printf '\nKubernetes context: %s\nAPI server: %s\nTarget node: %s\n' \
"$selected_context" "$selected_api_server" "$TARGET_NODE"
printf 'Type ROLLBACK %s OBSERVE to remove forwarded-header trust: ' \
"$selected_context"
read -r confirmation
[[ "$confirmation" == "ROLLBACK ${selected_context} OBSERVE" ]] || \
fail "cancelled"
[[ "$(current_context)" == "$selected_context" ]] || \
fail "kubectl context changed after confirmation"
[[ "$(current_api_server)" == "$selected_api_server" ]] || \
fail "Kubernetes API server changed after confirmation"
assert_live_baseline
kubectl apply --kustomize "$OBSERVE_OVERLAY"
wait_for_runtime observe
kubectl --namespace kube-system rollout status deployment/traefik --timeout=5m
assert_runtime observe
work_dir="$(mktemp -d "${TMPDIR:-/tmp}/platform-traefik-rollback.XXXXXX")"
cleanup() {
case "$work_dir" in
/tmp/platform-traefik-rollback.*|"${TMPDIR:-/tmp}"/platform-traefik-rollback.*)
rm -rf -- "$work_dir"
;;
*)
printf 'WARNING: refusing to remove unexpected directory: %s\n' \
"$work_dir" >&2
;;
esac
}
trap cleanup EXIT
assert_live_hcc_matches_overlay "$OBSERVE_OVERLAY" \
"${work_dir}/post-rollback-observe.yaml"
assert_live_baseline
wait_for_nodeport_boundary_and_health
printf '\nROLLBACK COMPLETE\n'
printf 'JSON access logging remains enabled.\n'
printf 'forwardedHeaders trust is absent from both entrypoints.\n'
+161
View File
@@ -0,0 +1,161 @@
#!/usr/bin/env bash
set -Eeuo pipefail
set +x
umask 077
readonly SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
assert_count() {
local file="$1"
local pattern="$2"
local expected="$3"
local description="$4"
local actual
actual="$(rg --count --no-filename -- "$pattern" "$file" || true)"
actual="${actual:-0}"
[[ "$actual" == "$expected" ]] || \
fail "${description}: expected ${expected}, found ${actual}"
}
require_commands kubectl rg awk cmp python3 find bash
render_dir="$(mktemp -d "${TMPDIR:-/tmp}/platform-traefik-render.XXXXXX")"
cleanup() {
case "$render_dir" in
/tmp/platform-traefik-render.*|"${TMPDIR:-/tmp}"/platform-traefik-render.*)
rm -rf -- "$render_dir"
;;
*)
printf 'WARNING: refusing to remove unexpected directory: %s\n' \
"$render_dir" >&2
;;
esac
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
root_render="${render_dir}/root.yaml"
baseline_render="${render_dir}/baseline.yaml"
observe_render="${render_dir}/observe.yaml"
trust_render="${render_dir}/trust.yaml"
render_overlay "$TRAEFIK_ROOT" "$root_render"
render_overlay "$BASELINE_OVERLAY" "$baseline_render"
render_overlay "$OBSERVE_OVERLAY" "$observe_render"
render_overlay "$TRUST_OVERLAY" "$trust_render"
cmp --silent "$root_render" "$observe_render" || \
fail "the Traefik root must render the observation phase"
for manifest in "$baseline_render" "$observe_render" "$trust_render"; do
assert_count "$manifest" '^apiVersion: helm\.cattle\.io/v1$' 1 \
"HelmChartConfig API version"
assert_count "$manifest" '^kind: HelmChartConfig$' 1 \
"HelmChartConfig kind"
assert_count "$manifest" '^ name: traefik$' 1 \
"HelmChartConfig name"
assert_count "$manifest" '^ namespace: kube-system$' 1 \
"HelmChartConfig namespace"
assert_count "$manifest" '^ failurePolicy: abort$' 1 \
"Helm failure policy must preserve the running release on upgrade failure"
assert_count "$manifest" '^ deployment:$' 1 \
"Traefik Deployment values root"
assert_count "$manifest" '^ podAnnotations:$' 1 \
"Traefik Pod annotations block"
assert_count "$manifest" '^ prometheus\.io/path: /metrics$' 1 \
"Traefik legacy scrape path preservation"
assert_count "$manifest" '^ prometheus\.io/port: "9100"$' 1 \
"Traefik legacy scrape port preservation"
assert_count "$manifest" '^ prometheus\.io/scrape: "true"$' 1 \
"Traefik legacy scrape enablement preservation"
assert_count "$manifest" '[Ii]nsecure' 0 \
"insecure forwarded-header mode"
assert_count "$manifest" '^ service:$' 1 \
"Traefik Service values root"
assert_count "$manifest" '^ type: NodePort$' 1 \
"loopback NodePort Service type preservation"
assert_count "$manifest" '^ externalTrafficPolicy: Cluster$' 1 \
"Traefik externalTrafficPolicy preservation"
assert_count "$manifest" '^ nodePort: 30080$' 1 \
"Traefik web NodePort preservation"
assert_count "$manifest" '^ nodePort: 30443$' 1 \
"Traefik websecure NodePort preservation"
assert_count "$manifest" '^ metrics:$' 1 \
"Traefik metrics values root"
assert_count "$manifest" '^ prometheus:$' 1 \
"Traefik Prometheus metrics block"
assert_count "$manifest" '^ serviceMonitor:$' 1 \
"Traefik ServiceMonitor block"
assert_count "$manifest" '^ jobLabel: app\.kubernetes\.io/name$' 1 \
"Traefik ServiceMonitor job label"
assert_count "$manifest" '^ observability\.hyeonworks\.com/instance: home$' 1 \
"Traefik ServiceMonitor selector label"
assert_count "$manifest" '^ interval: 30s$' 1 \
"Traefik ServiceMonitor interval"
assert_count "$manifest" '^ scrapeTimeout: 10s$' 1 \
"Traefik ServiceMonitor timeout"
assert_count "$manifest" '^ enabled: true$' 2 \
"Traefik metrics Service and ServiceMonitor enablement"
assert_count "$manifest" '^[[:space:]]*(type|serviceType):[[:space:]]*LoadBalancer' 0 \
"LoadBalancer exposure"
done
assert_count "$baseline_render" '^[[:space:]]*logs:' 0 \
"baseline access logs"
assert_count "$baseline_render" '^[[:space:]]*forwardedHeaders:' 0 \
"baseline forwarded-header trust"
for manifest in "$observe_render" "$trust_render"; do
assert_count "$manifest" '^ enabled: true$' 1 \
"access log enablement"
assert_count "$manifest" '^ format: json$' 1 \
"JSON access log format"
assert_count "$manifest" '^ defaultmode: keep$' 1 \
"access-log general-field policy"
assert_count "$manifest" '^ defaultmode: drop$' 1 \
"access-log header policy"
done
assert_count "$observe_render" '^[[:space:]]*forwardedHeaders:' 0 \
"observation-phase forwarded-header trust"
assert_count "$observe_render" '^[[:space:]]*trustedIPs:' 0 \
"observation-phase trusted IP list"
assert_count "$trust_render" '^ ports:$' 1 \
"trust-phase ports values root"
assert_count "$trust_render" '^ web:$' 1 \
"trust-phase web entrypoint"
assert_count "$trust_render" '^ forwardedHeaders:$' 1 \
"trust-phase forwarded-header block"
assert_count "$trust_render" '^ trustedIPs:$' 1 \
"trust-phase trusted IP list"
trusted_cidr="$(source_trusted_proxy_cidr)"
normalized_cidr="$(normalize_exact_host_cidr "$trusted_cidr")"
[[ "$trusted_cidr" == "$normalized_cidr" ]] || \
fail "trusted proxy CIDR must use canonical exact-host notation"
assert_count "$trust_render" \
"^[[:space:]]*-[[:space:]]*\"${trusted_cidr//./\\.}\"[[:space:]]*$" 1 \
"rendered exact-host trusted proxy CIDR"
while IFS= read -r -d '' script_path; do
bash -n "$script_path"
done < <(
find "$SCRIPT_DIR" -maxdepth 1 -type f -name '*.sh' -print0
)
printf 'Traefik observation and trust overlays rendered successfully.\n'
printf 'Access logs are JSON and request headers are dropped.\n'
printf 'No insecure mode, websecure trust, or LoadBalancer exposure was found.\n'
printf 'The existing NodePort 30080/30443 boundary is declared in both phases.\n'
if [[ "$trusted_cidr" == "$SENTINEL_TRUSTED_PROXY_CIDR" ]]; then
printf 'Trust overlay remains intentionally blocked by sentinel CIDR %s.\n' \
"$SENTINEL_TRUSTED_PROXY_CIDR"
else
printf 'Trust overlay contains reviewed exact-host CIDR %s.\n' "$trusted_cidr"
fi