579 lines
14 KiB
Markdown
579 lines
14 KiB
Markdown
# network / ingress / TLS 예시
|
|
|
|
모든 YAML은 `kubectl apply --dry-run=server -f -` 기준 clean. Traefik은 `ingress-traefik` namespace에 IngressClass `traefik`으로 설치되어 있다고 가정한다. cert-manager는 `cert-manager` namespace에 설치되어 있다.
|
|
|
|
---
|
|
|
|
## 좋은 예시 1: cert-manager ClusterIssuer (staging + prod) + DNS-01 wildcard
|
|
|
|
```yaml
|
|
apiVersion: cert-manager.io/v1
|
|
kind: ClusterIssuer
|
|
metadata:
|
|
name: letsencrypt-staging
|
|
spec:
|
|
acme:
|
|
server: https://acme-staging-v02.api.letsencrypt.org/directory
|
|
email: platform@example.com
|
|
privateKeySecretRef:
|
|
name: letsencrypt-staging-account-key
|
|
solvers:
|
|
- http01:
|
|
ingress:
|
|
ingressClassName: traefik
|
|
---
|
|
apiVersion: cert-manager.io/v1
|
|
kind: ClusterIssuer
|
|
metadata:
|
|
name: letsencrypt-prod
|
|
spec:
|
|
acme:
|
|
server: https://acme-v02.api.letsencrypt.org/directory
|
|
email: platform@example.com
|
|
privateKeySecretRef:
|
|
name: letsencrypt-prod-account-key
|
|
solvers:
|
|
- http01:
|
|
ingress:
|
|
ingressClassName: traefik
|
|
selector:
|
|
dnsZones:
|
|
- example.com
|
|
---
|
|
apiVersion: cert-manager.io/v1
|
|
kind: ClusterIssuer
|
|
metadata:
|
|
name: letsencrypt-prod-dns
|
|
spec:
|
|
acme:
|
|
server: https://acme-v02.api.letsencrypt.org/directory
|
|
email: platform@example.com
|
|
privateKeySecretRef:
|
|
name: letsencrypt-prod-dns-account-key
|
|
solvers:
|
|
- dns01:
|
|
route53:
|
|
region: us-east-1
|
|
hostedZoneID: Z2FDTNDATAQYW2
|
|
selector:
|
|
dnsZones:
|
|
- example.com
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- Staging issuer로 먼저 발급 테스트(LE rate limit 절약). 검증 후 prod로 교체.
|
|
- HTTP-01 solver는 `ingressClassName: traefik`으로 challenge Ingress가 정확히 Traefik만 수락.
|
|
- DNS-01 solver는 wildcard(`*.example.com`) 발급에 필수. Route53 hosted zone ID 고정.
|
|
|
|
---
|
|
|
|
## 좋은 예시 2: Certificate CRD로 TLS Secret 자동 생성 + Ingress 재사용
|
|
|
|
```yaml
|
|
apiVersion: cert-manager.io/v1
|
|
kind: Certificate
|
|
metadata:
|
|
name: auth-example-com
|
|
namespace: auth-prod
|
|
spec:
|
|
secretName: auth-example-com-tls
|
|
secretTemplate:
|
|
annotations:
|
|
reflector.v1.k8s.emberstack.com/reflection-allowed: "false"
|
|
labels:
|
|
app.kubernetes.io/part-of: identity-platform
|
|
duration: 2160h # 90d
|
|
renewBefore: 360h # 15d
|
|
privateKey:
|
|
algorithm: ECDSA
|
|
size: 256
|
|
rotationPolicy: Always
|
|
usages:
|
|
- server auth
|
|
- digital signature
|
|
- key encipherment
|
|
dnsNames:
|
|
- auth.example.com
|
|
issuerRef:
|
|
kind: ClusterIssuer
|
|
name: letsencrypt-prod
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- cert-manager가 `auth-example-com-tls`라는 `kubernetes.io/tls` Secret을 자동 생성·회전(15일 전).
|
|
- ECDSA P-256 + 회전 정책으로 key lifecycle 관리.
|
|
- `usages` 명시로 SAN certificate의 Extended Key Usage 제어.
|
|
|
|
---
|
|
|
|
## 좋은 예시 3: Traefik Middleware(HSTS + HTTPS redirect) + TLSOption
|
|
|
|
```yaml
|
|
apiVersion: traefik.io/v1alpha1
|
|
kind: Middleware
|
|
metadata:
|
|
name: https-redirect
|
|
namespace: ingress-traefik
|
|
spec:
|
|
redirectScheme:
|
|
scheme: https
|
|
permanent: true
|
|
---
|
|
apiVersion: traefik.io/v1alpha1
|
|
kind: Middleware
|
|
metadata:
|
|
name: security-headers
|
|
namespace: ingress-traefik
|
|
spec:
|
|
headers:
|
|
stsSeconds: 31536000
|
|
stsIncludeSubdomains: true
|
|
stsPreload: true
|
|
forceSTSHeader: true
|
|
contentTypeNosniff: true
|
|
browserXssFilter: true
|
|
referrerPolicy: strict-origin-when-cross-origin
|
|
frameDeny: true
|
|
---
|
|
apiVersion: traefik.io/v1alpha1
|
|
kind: TLSOption
|
|
metadata:
|
|
name: modern-tls
|
|
namespace: ingress-traefik
|
|
spec:
|
|
minVersion: VersionTLS12
|
|
sniStrict: true
|
|
cipherSuites:
|
|
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
|
|
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
|
|
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
|
|
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
|
|
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
|
|
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
|
|
curvePreferences:
|
|
- CurveP521
|
|
- CurveP384
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- HSTS preload 조건(`max-age>=31536000` + `includeSubDomains` + `preload`)을 모두 만족.
|
|
- TLS 1.2+ 강제, 취약 cipher 제거. `sniStrict: true`로 SNI 없는 클라이언트 차단.
|
|
- 재사용 가능한 platform middleware — 각 namespace Ingress가 참조.
|
|
|
|
---
|
|
|
|
## 좋은 예시 4: auth-server Service + Ingress (TLS, HSTS, HTTPS redirect)
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: auth-server
|
|
namespace: auth-prod
|
|
labels:
|
|
app.kubernetes.io/name: auth-server
|
|
app.kubernetes.io/component: api
|
|
app.kubernetes.io/part-of: identity-platform
|
|
spec:
|
|
type: ClusterIP
|
|
selector:
|
|
app.kubernetes.io/name: auth-server
|
|
app.kubernetes.io/instance: auth-server
|
|
ports:
|
|
- name: http
|
|
port: 80
|
|
targetPort: http
|
|
protocol: TCP
|
|
appProtocol: http
|
|
- name: metrics
|
|
port: 9090
|
|
targetPort: metrics
|
|
protocol: TCP
|
|
appProtocol: http
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
name: auth-server
|
|
namespace: auth-prod
|
|
labels:
|
|
app.kubernetes.io/name: auth-server
|
|
app.kubernetes.io/part-of: identity-platform
|
|
annotations:
|
|
cert-manager.io/cluster-issuer: letsencrypt-prod
|
|
traefik.ingress.kubernetes.io/router.entrypoints: websecure
|
|
traefik.ingress.kubernetes.io/router.tls: "true"
|
|
traefik.ingress.kubernetes.io/router.tls.options: ingress-traefik-modern-tls@kubernetescrd
|
|
traefik.ingress.kubernetes.io/router.middlewares: ingress-traefik-https-redirect@kubernetescrd,ingress-traefik-security-headers@kubernetescrd
|
|
spec:
|
|
ingressClassName: traefik
|
|
tls:
|
|
- hosts:
|
|
- auth.example.com
|
|
secretName: auth-example-com-tls
|
|
rules:
|
|
- host: auth.example.com
|
|
http:
|
|
paths:
|
|
- path: /
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: auth-server
|
|
port:
|
|
name: http
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- `ingressClassName: traefik` 필드 사용, deprecated annotation 미사용.
|
|
- `cert-manager.io/cluster-issuer` annotation으로 TLS Secret(`auth-example-com-tls`)이 자동 발급.
|
|
- Traefik middleware 체인으로 HSTS + HTTPS redirect + TLSOption 적용.
|
|
- Service는 named port `http`, `metrics` 분리. Ingress는 `http`만 라우팅, metrics는 NetworkPolicy로 Prometheus만 허용.
|
|
|
|
---
|
|
|
|
## 나쁜 예시 1: deprecated ingress.class annotation + TLS 누락
|
|
|
|
```yaml
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
name: auth-server
|
|
namespace: auth-prod
|
|
annotations:
|
|
kubernetes.io/ingress.class: traefik
|
|
spec:
|
|
rules:
|
|
- host: auth.example.com
|
|
http:
|
|
paths:
|
|
- path: /
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: auth-server
|
|
port:
|
|
number: 80
|
|
```
|
|
|
|
**문제:**
|
|
|
|
- `kubernetes.io/ingress.class` annotation은 1.22부터 deprecated. 일부 컨트롤러는 무시한다.
|
|
- `spec.tls` 없음 → 평문 HTTP로 노출. 인증 시스템에는 특히 부적절.
|
|
- HSTS/HTTPS redirect 미적용.
|
|
- Service 포트를 `number: 80`으로 hard-code. named port drift에 취약.
|
|
|
|
---
|
|
|
|
## 좋은 예시 5: Keycloak은 `/realms/`, `/resources/`, `/.well-known/`만 공개
|
|
|
|
```yaml
|
|
apiVersion: cert-manager.io/v1
|
|
kind: Certificate
|
|
metadata:
|
|
name: sso-example-com
|
|
namespace: keycloak
|
|
spec:
|
|
secretName: sso-example-com-tls
|
|
duration: 2160h
|
|
renewBefore: 360h
|
|
privateKey:
|
|
algorithm: ECDSA
|
|
size: 256
|
|
dnsNames:
|
|
- sso.example.com
|
|
issuerRef:
|
|
kind: ClusterIssuer
|
|
name: letsencrypt-prod
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: keycloak
|
|
namespace: keycloak
|
|
labels:
|
|
app.kubernetes.io/name: keycloak
|
|
spec:
|
|
type: ClusterIP
|
|
selector:
|
|
app.kubernetes.io/name: keycloak
|
|
ports:
|
|
- name: http
|
|
port: 8080
|
|
targetPort: http
|
|
protocol: TCP
|
|
appProtocol: http
|
|
- name: management
|
|
port: 9000
|
|
targetPort: management
|
|
protocol: TCP
|
|
appProtocol: http
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
name: keycloak
|
|
namespace: keycloak
|
|
annotations:
|
|
cert-manager.io/cluster-issuer: letsencrypt-prod
|
|
traefik.ingress.kubernetes.io/router.entrypoints: websecure
|
|
traefik.ingress.kubernetes.io/router.tls: "true"
|
|
traefik.ingress.kubernetes.io/router.tls.options: ingress-traefik-modern-tls@kubernetescrd
|
|
traefik.ingress.kubernetes.io/router.middlewares: ingress-traefik-https-redirect@kubernetescrd,ingress-traefik-security-headers@kubernetescrd
|
|
spec:
|
|
ingressClassName: traefik
|
|
tls:
|
|
- hosts:
|
|
- sso.example.com
|
|
secretName: sso-example-com-tls
|
|
rules:
|
|
- host: sso.example.com
|
|
http:
|
|
paths:
|
|
- path: /realms/
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: keycloak
|
|
port:
|
|
name: http
|
|
- path: /resources/
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: keycloak
|
|
port:
|
|
name: http
|
|
- path: /.well-known/
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: keycloak
|
|
port:
|
|
name: http
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- Keycloak 공식 권장 공개 경로만 노출.
|
|
- `/admin/`, `/metrics`, `/health`는 Ingress에 없음 → 외부에서 접근 불가.
|
|
- `management`(9000) 포트는 Service에만 존재하고 Ingress에는 없음.
|
|
|
|
---
|
|
|
|
## 나쁜 예시 2: host 없는 defaultBackend + admin 노출
|
|
|
|
```yaml
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
name: catch-all
|
|
namespace: keycloak
|
|
spec:
|
|
ingressClassName: traefik
|
|
defaultBackend:
|
|
service:
|
|
name: keycloak
|
|
port:
|
|
number: 8080
|
|
rules:
|
|
- http:
|
|
paths:
|
|
- path: /admin
|
|
pathType: Prefix
|
|
backend:
|
|
service:
|
|
name: keycloak
|
|
port:
|
|
number: 9000
|
|
```
|
|
|
|
**문제:**
|
|
|
|
- `defaultBackend`가 모든 host의 unmatched 요청을 Keycloak으로 포워딩 → 다른 앱 공격면 확대.
|
|
- `/admin`을 관리 포트 9000으로 프록시 → Keycloak 공식 권고 위반, 관리 콘솔 외부 노출.
|
|
- TLS/HSTS/redirect 미적용.
|
|
|
|
---
|
|
|
|
## 좋은 예시 6: Traefik IngressRoute + Middleware(TLS 1.2, rate-limit, BasicAuth)
|
|
|
|
```yaml
|
|
apiVersion: traefik.io/v1alpha1
|
|
kind: Middleware
|
|
metadata:
|
|
name: rate-limit
|
|
namespace: auth-prod
|
|
spec:
|
|
rateLimit:
|
|
average: 100
|
|
burst: 200
|
|
period: 1s
|
|
---
|
|
apiVersion: traefik.io/v1alpha1
|
|
kind: IngressRoute
|
|
metadata:
|
|
name: auth-server
|
|
namespace: auth-prod
|
|
spec:
|
|
entryPoints:
|
|
- websecure
|
|
routes:
|
|
- kind: Rule
|
|
match: Host(`auth.example.com`) && PathPrefix(`/api/v1`)
|
|
services:
|
|
- kind: Service
|
|
name: auth-server
|
|
port: http
|
|
scheme: http
|
|
passHostHeader: true
|
|
middlewares:
|
|
- name: https-redirect
|
|
namespace: ingress-traefik
|
|
- name: security-headers
|
|
namespace: ingress-traefik
|
|
- name: rate-limit
|
|
namespace: auth-prod
|
|
tls:
|
|
secretName: auth-example-com-tls
|
|
options:
|
|
name: modern-tls
|
|
namespace: ingress-traefik
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- Traefik CRD 네이티브. match 표현이 강력(Host + PathPrefix 조합, Header match 가능).
|
|
- Middleware 체인(HSTS + redirect + rate-limit)을 순서대로 지정.
|
|
- `TLSOption`을 Route마다 override 가능(특정 host만 mTLS 요구 등).
|
|
|
|
---
|
|
|
|
## 좋은 예시 7: Vault/DB는 Ingress 없이 ClusterIP
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: vault
|
|
namespace: vault
|
|
labels:
|
|
app.kubernetes.io/name: vault
|
|
spec:
|
|
type: ClusterIP
|
|
selector:
|
|
app.kubernetes.io/name: vault
|
|
ports:
|
|
- name: https
|
|
port: 8200
|
|
targetPort: https
|
|
protocol: TCP
|
|
appProtocol: https
|
|
- name: cluster
|
|
port: 8201
|
|
targetPort: cluster
|
|
protocol: TCP
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: identity-postgres
|
|
namespace: data-prod
|
|
labels:
|
|
app.kubernetes.io/name: postgres
|
|
app.kubernetes.io/instance: identity-postgres
|
|
spec:
|
|
type: ClusterIP
|
|
clusterIP: None
|
|
selector:
|
|
app.kubernetes.io/name: postgres
|
|
app.kubernetes.io/instance: identity-postgres
|
|
ports:
|
|
- name: postgres
|
|
port: 5432
|
|
targetPort: postgres
|
|
protocol: TCP
|
|
appProtocol: postgresql
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- Vault/Postgres 둘 다 Ingress 없음 → 외부 L7 공격면 0.
|
|
- Postgres는 headless(`clusterIP: None`) → StatefulSet Pod에 직접 DNS.
|
|
- named port `https`/`postgres` 사용.
|
|
|
|
---
|
|
|
|
## 나쁜 예시 3: 운영 DB를 NodePort로 공개
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: identity-postgres
|
|
namespace: data-prod
|
|
spec:
|
|
type: NodePort
|
|
ports:
|
|
- port: 5432
|
|
targetPort: 5432
|
|
nodePort: 30032
|
|
```
|
|
|
|
**문제:**
|
|
|
|
- 모든 노드의 30032 포트가 외부에서 접근 가능 → DB가 인터넷에 노출될 수 있음.
|
|
- TLS/mTLS/NetworkPolicy 어디서도 통제 불가.
|
|
- `services.nodeports: 0` quota를 걸어 namespace 단에서 차단해야 한다.
|
|
|
|
---
|
|
|
|
## 좋은 예시 8: K3s Traefik HelmChartConfig override
|
|
|
|
```yaml
|
|
apiVersion: helm.cattle.io/v1
|
|
kind: HelmChartConfig
|
|
metadata:
|
|
name: traefik
|
|
namespace: kube-system
|
|
spec:
|
|
valuesContent: |-
|
|
deployment:
|
|
replicas: 3
|
|
service:
|
|
spec:
|
|
externalTrafficPolicy: Local
|
|
ports:
|
|
web:
|
|
redirectTo:
|
|
port: websecure
|
|
priority: 10
|
|
websecure:
|
|
tls:
|
|
enabled: true
|
|
ingressClass:
|
|
enabled: true
|
|
isDefaultClass: true
|
|
additionalArguments:
|
|
- "--providers.kubernetesingress.ingressclass=traefik"
|
|
- "--metrics.prometheus=true"
|
|
- "--entrypoints.websecure.http.tls.options=modern-tls@kubernetescrd"
|
|
resources:
|
|
requests:
|
|
cpu: 200m
|
|
memory: 256Mi
|
|
limits:
|
|
cpu: "1"
|
|
memory: 512Mi
|
|
```
|
|
|
|
**왜 좋은가:**
|
|
|
|
- K3s packaged Traefik manifest는 건드리지 않고, override만 선언적으로 관리.
|
|
- replica 3, `externalTrafficPolicy: Local`로 source IP 보존.
|
|
- 기본 websecure에 `modern-tls` TLSOption을 묶어 platform 전역 TLS 정책 통일.
|