refactor: 구조 변경

This commit is contained in:
donghyeon-ka
2026-08-28 17:24:26 +09:00
parent a6f6c663e0
commit b8626946b1
192 changed files with 2251 additions and 206 deletions
+33
View File
@@ -0,0 +1,33 @@
# GitOps Desired State
Kubernetes API 안에서 지속적으로 reconcile할 desired state를 관리합니다.
GitOps controller를 사용하지 않는 초기 단계에도 `kubectl kustomize`로 같은
entrypoint를 렌더할 수 있습니다.
```text
platform ─┐
policies ─┼──▶ clusters/<...> ◀── GitOps controller root
tenants ─┤
apps ─┘
```
- `clusters`: 클러스터별 최종 조립점
- `platform`: cluster-wide addon과 controller
- `policies`: cluster-wide admission과 거버넌스 규칙
- `tenants`: 구체적인 namespace/RBAC/quota/NetworkPolicy
- `apps`: application 배포 정의
Catalog 디렉터리를 controller root로 직접 지정하지 않습니다. cluster entrypoint가
필요한 base/overlay를 선택하고 의존 순서를 명시합니다.
## 기본 규칙
- `base`는 환경을 모르며 재사용 가능한 기본값만 가집니다.
- `overlays`는 차이만 patch하고 전체 manifest를 복사하지 않습니다.
- CRD/controller가 필요한 리소스는 controller 이후에 reconcile합니다.
- resource namespace, ownership label과 버전을 명시합니다.
- raw `Secret` 또는 실제 비밀값을 커밋하지 않습니다.
- 원격 base/chart를 참조할 때 immutable version 또는 digest를 사용합니다.
Flux/Argo CD 고유 리소스와 sync ordering은 선택한 controller를 기록한 ADR에
문서화합니다.
+18
View File
@@ -0,0 +1,18 @@
# Application Deployment Catalog
애플리케이션의 source code가 아니라 Kubernetes 배포 정의를 둡니다. app 팀이
별도 source/deploy 저장소를 소유하면 이곳에는 immutable artifact를 참조하는
GitOps 리소스만 둘 수 있습니다.
```text
apps/
└── example-api/
├── base/
└── overlays/
├── dev/
└── prod/
```
base는 환경을 모르고, overlay에는 replica/resource/config처럼 필요한 차이만
둡니다. image는 mutable tag 대신 조직 정책에 따른 고정 tag 또는 digest를
사용합니다.
+22
View File
@@ -0,0 +1,22 @@
# __REPLACE_ME_APPLICATION_NAME__
## 소유자
Team: __REPLACE_ME_OWNER__
repository와 on-call 정보를 적습니다.
## 배포 계약
- Namespace:
- Image/artifact source:
- Ports/protocol:
- Dependency:
- SLO/alerts:
## 구성
- `base`: 공통 Kubernetes 배포 정의
- `overlays`: 환경별 replica, resource, config 차이
비밀은 ExternalSecret 같은 참조 또는 승인된 암호화 형식으로만 추가합니다.
+4
View File
@@ -0,0 +1,4 @@
# Base
환경을 모르는 application의 공통 manifest를 둡니다. namespace 자체의 소유권이
tenant catalog에 있다면 이곳에서 중복 생성하지 않습니다.
@@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: []
+6
View File
@@ -0,0 +1,6 @@
# Overlays
필요한 환경에만 overlay를 추가합니다. 각 overlay는 `../../base`를 참조하고
환경별 patch만 포함합니다.
비밀값, 임시 debug 설정과 수동 hotfix 결과를 overlay에 커밋하지 않습니다.
@@ -0,0 +1,8 @@
# Project Auth system
Project Auth bounded context가 소유하는 PostgreSQL, Keycloak과 client sync의
환경 중립 base입니다. `dev-k3s` namespace, host, Vault annotation과
NetworkPolicy는
`gitops/clusters/dev-k3s/overlays/systems/auth-system`에서 결합합니다.
Owner: Project Auth system
@@ -0,0 +1,22 @@
{
"realm": "project-auth",
"enabled": true,
"displayName": "Project Auth",
"sslRequired": "NONE",
"registrationAllowed": false,
"loginWithEmailAllowed": true,
"duplicateEmailsAllowed": false,
"resetPasswordAllowed": true,
"clients": [
{
"clientId": "project-auth-server",
"name": "project-auth-server",
"enabled": true,
"protocol": "openid-connect",
"publicClient": false,
"standardFlowEnabled": true,
"directAccessGrantsEnabled": false,
"serviceAccountsEnabled": false
}
]
}
@@ -0,0 +1,19 @@
#!/bin/sh
set -eu
psql \
-v ON_ERROR_STOP=1 \
--set=auth_db_user="${AUTH_DB_USER}" \
--set=auth_db_password="${AUTH_DB_PASSWORD}" \
--set=auth_db_name="${AUTH_DB_NAME}" \
--set=keycloak_db_user="${KEYCLOAK_DB_USER}" \
--set=keycloak_db_password="${KEYCLOAK_DB_PASSWORD}" \
--set=keycloak_db_name="${KEYCLOAK_DB_NAME}" \
--username "${POSTGRES_USER}" \
--dbname "${POSTGRES_DB}" <<-'EOSQL'
CREATE USER :"auth_db_user" WITH PASSWORD :'auth_db_password';
CREATE DATABASE :"auth_db_name" OWNER :"auth_db_user";
CREATE USER :"keycloak_db_user" WITH PASSWORD :'keycloak_db_password';
CREATE DATABASE :"keycloak_db_name" OWNER :"keycloak_db_user";
EOSQL
@@ -0,0 +1,97 @@
apiVersion: batch/v1
kind: Job
metadata:
name: keycloak-client-sync
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation,HookSucceeded
argocd.argoproj.io/sync-wave: "4"
spec:
activeDeadlineSeconds: 600
backoffLimit: 1
template:
metadata:
labels:
app: keycloak-client-sync
spec:
serviceAccountName: keycloak-client-sync
automountServiceAccountToken: false
restartPolicy: Never
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: keycloak-client-sync
image: quay.io/keycloak/keycloak:26.5.5@sha256:a7b0cb7a43a1235a61872883414d3f1d9a3ceac9df6e5907bd12202778a6265c
command:
- /bin/sh
- -c
- |
set -eu
ready=false
for _ in $(seq 1 60); do
if /opt/keycloak/bin/kcadm.sh config credentials \
--server http://keycloak \
--realm master \
--user "$KC_BOOTSTRAP_ADMIN_USERNAME" \
--password "$KC_BOOTSTRAP_ADMIN_PASSWORD" >/dev/null 2>&1; then
ready=true
break
fi
sleep 5
done
test "$ready" = true
CLIENT_UUID=$(/opt/keycloak/bin/kcadm.sh get clients \
-r project-auth \
-q clientId="$KEYCLOAK_CLIENT_ID" | sed -n 's/.*"id" : "\([^"]*\)".*/\1/p' | head -n 1)
test -n "$CLIENT_UUID"
/opt/keycloak/bin/kcadm.sh update "clients/${CLIENT_UUID}" \
-r project-auth \
-s "secret=$KEYCLOAK_CLIENT_SECRET" \
-s "baseUrl=$AUTH_SERVER_BASE_URL" \
-s 'redirectUris=["'"$AUTH_SERVER_BASE_URL"'/login/oauth2/code/keycloak-google","'"$AUTH_SERVER_BASE_URL"'/login/oauth2/code/keycloak-github"]' \
-s 'webOrigins=["'"$AUTH_SERVER_BASE_URL"'"]'
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
env:
- name: KC_BOOTSTRAP_ADMIN_USERNAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_BOOTSTRAP_ADMIN_USERNAME
- name: KC_BOOTSTRAP_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-bootstrap-admin
key: KEYCLOAK_BOOTSTRAP_ADMIN_PASSWORD
- name: KEYCLOAK_CLIENT_ID
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_CLIENT_ID
- name: KEYCLOAK_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: keycloak-client-auth-server
key: KEYCLOAK_CLIENT_SECRET
- name: AUTH_SERVER_BASE_URL
valueFrom:
configMapKeyRef:
name: auth-system-config
key: AUTH_SERVER_BASE_URL
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: keycloak-client-sync
automountServiceAccountToken: false
@@ -0,0 +1,104 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: keycloak
annotations:
argocd.argoproj.io/sync-wave: "3"
spec:
replicas: 1
selector:
matchLabels:
app: keycloak
template:
metadata:
labels:
app: keycloak
spec:
serviceAccountName: keycloak
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: keycloak
image: quay.io/keycloak/keycloak:26.5.5@sha256:a7b0cb7a43a1235a61872883414d3f1d9a3ceac9df6e5907bd12202778a6265c
args:
- start-dev
- --import-realm
- --http-port=8080
ports:
- containerPort: 8080
name: http
- containerPort: 9000
name: management
env:
- name: KC_DB
value: postgres
- name: KC_DB_URL
value: jdbc:postgresql://postgres:5432/keycloak
- name: KC_DB_USERNAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_DB_USER
- name: KC_DB_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-keycloak-credentials
key: KEYCLOAK_DB_PASSWORD
- name: KC_HEALTH_ENABLED
value: "true"
- name: KC_BOOTSTRAP_ADMIN_USERNAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_BOOTSTRAP_ADMIN_USERNAME
- name: KC_BOOTSTRAP_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: keycloak-bootstrap-admin
key: KEYCLOAK_BOOTSTRAP_ADMIN_PASSWORD
volumeMounts:
- name: keycloak-realm-import
mountPath: /opt/keycloak/data/import/project-auth-realm.json
subPath: project-auth-realm.json
readinessProbe:
httpGet:
path: /health/ready
port: management
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 6
livenessProbe:
httpGet:
path: /health/live
port: management
initialDelaySeconds: 60
periodSeconds: 15
timeoutSeconds: 3
failureThreshold: 6
startupProbe:
httpGet:
path: /health/ready
port: management
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 30
resources:
requests:
cpu: 250m
memory: 768Mi
limits:
cpu: 1000m
memory: 1536Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
volumes:
- name: keycloak-realm-import
configMap:
name: keycloak-realm-import
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Service
metadata:
name: keycloak
spec:
selector:
app: keycloak
ports:
- name: http
port: 80
targetPort: 8080
type: ClusterIP
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: keycloak
automountServiceAccountToken: false
@@ -0,0 +1,19 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- postgres-serviceaccount.yaml
- keycloak-serviceaccount.yaml
- keycloak-client-sync-serviceaccount.yaml
- postgres-service.yaml
- postgres-statefulset.yaml
- keycloak-service.yaml
- keycloak-deployment.yaml
- keycloak-client-sync-job.yaml
configMapGenerator:
- name: postgres-init-script
files:
- files/postgres/01-init-project-auth-databases.sh
- name: keycloak-realm-import
files:
- files/keycloak/project-auth-realm.json
@@ -0,0 +1,13 @@
apiVersion: v1
kind: Service
metadata:
name: postgres
spec:
clusterIP: None
selector:
app: postgres
ports:
- name: postgres
port: 5432
targetPort: 5432
type: ClusterIP
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: postgres
automountServiceAccountToken: false
@@ -0,0 +1,131 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres
annotations:
argocd.argoproj.io/sync-wave: "2"
spec:
serviceName: postgres
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
serviceAccountName: postgres
securityContext:
fsGroup: 999
seccompProfile:
type: RuntimeDefault
containers:
- name: postgres
image: postgres:16-alpine@sha256:57c72fd2a128e416c7fcc499958864df5301e940bca0a56f58fddf30ffc07777
ports:
- containerPort: 5432
name: postgres
env:
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: auth-system-config
key: POSTGRES_SUPERUSER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-superuser-credentials
key: POSTGRES_SUPERUSER_PASSWORD
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: auth-system-config
key: POSTGRES_DEFAULT_DB
- name: AUTH_DB_NAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: AUTH_DB_NAME
- name: AUTH_DB_USER
valueFrom:
configMapKeyRef:
name: auth-system-config
key: AUTH_DB_USER
- name: AUTH_DB_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-auth-server-credentials
key: AUTH_DB_PASSWORD
- name: KEYCLOAK_DB_NAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_DB_NAME
- name: KEYCLOAK_DB_USER
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_DB_USER
- name: KEYCLOAK_DB_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-keycloak-credentials
key: KEYCLOAK_DB_PASSWORD
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
- name: postgres-init-script
mountPath: /docker-entrypoint-initdb.d/01-init-project-auth-databases.sh
subPath: 01-init-project-auth-databases.sh
- name: postgres-run
mountPath: /var/run/postgresql
readinessProbe:
exec:
command:
- sh
- -c
- pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 3
livenessProbe:
exec:
command:
- sh
- -c
- pg_isready -U "$POSTGRES_USER" -d "$POSTGRES_DB"
initialDelaySeconds: 20
periodSeconds: 15
timeoutSeconds: 3
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 1000m
memory: 1024Mi
volumes:
- name: postgres-init-script
configMap:
name: postgres-init-script
defaultMode: 0555
- name: postgres-run
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: postgres-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
@@ -0,0 +1,8 @@
# API server
별도 source repository에서 빌드되는 API server의 환경 중립 Kubernetes
base입니다. Image digest, dev configuration, ingress와 pull credential
reference는 `gitops/clusters/dev-k3s/overlays/workloads/api-server`
소유합니다.
Owner: Project API workload
@@ -0,0 +1,71 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-server
annotations:
argocd.argoproj.io/sync-wave: "10"
spec:
replicas: 1
selector:
matchLabels:
app: api-server
template:
metadata:
labels:
app: api-server
spec:
serviceAccountName: api-server
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: api-server
image: ghcr.io/donghyeonka/project-api-server
imagePullPolicy: Always
ports:
- containerPort: 8082
name: http
envFrom:
- configMapRef:
name: api-server-config
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 1000m
memory: 512Mi
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: http
initialDelaySeconds: 15
timeoutSeconds: 3
failureThreshold: 3
periodSeconds: 10
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: http
initialDelaySeconds: 20
timeoutSeconds: 3
failureThreshold: 5
periodSeconds: 15
startupProbe:
httpGet:
path: /actuator/health/liveness
port: http
initialDelaySeconds: 5
timeoutSeconds: 3
periodSeconds: 10
failureThreshold: 12
@@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- service.yaml
- deployment.yaml
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Service
metadata:
name: api-server
spec:
selector:
app: api-server
ports:
- name: http
port: 80
targetPort: 8082
type: ClusterIP
@@ -0,0 +1,7 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: api-server
automountServiceAccountToken: false
imagePullSecrets:
- name: ghcr-regcred
@@ -0,0 +1,7 @@
# Auth server
별도 source repository에서 빌드되는 Auth server의 환경 중립 Kubernetes
base입니다. Image digest, dev configuration, ingress와 Vault annotation은
`gitops/clusters/dev-k3s/overlays/workloads/auth-server`가 소유합니다.
Owner: Project Auth workload
@@ -0,0 +1,7 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: auth-db-migration
automountServiceAccountToken: false
imagePullSecrets:
- name: ghcr-regcred
@@ -0,0 +1,70 @@
apiVersion: batch/v1
kind: Job
metadata:
name: auth-db-migration
annotations:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation,HookSucceeded
argocd.argoproj.io/sync-wave: "5"
spec:
activeDeadlineSeconds: 600
backoffLimit: 1
template:
metadata:
labels:
app: auth-db-migration
spec:
serviceAccountName: auth-db-migration
automountServiceAccountToken: false
restartPolicy: Never
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: auth-db-migration
image: ghcr.io/donghyeonka/project-auth-server
imagePullPolicy: IfNotPresent
command:
- java
- -jar
- /app/migration.jar
env:
- name: SPRING_MAIN_WEB_APPLICATION_TYPE
value: none
- name: APP_PERSISTENCE_MIGRATION_RUN_ON_STARTUP
value: "true"
- name: APP_PERSISTENCE_MIGRATION_LOCATION
value: classpath:db/migration
- name: APP_DATASOURCE_URL
valueFrom:
configMapKeyRef:
name: auth-server-config
key: APP_DATASOURCE_URL
- name: APP_DATASOURCE_USERNAME
valueFrom:
secretKeyRef:
name: postgres-auth-server-credentials
key: APP_DATASOURCE_USERNAME
- name: APP_DATASOURCE_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-auth-server-credentials
key: APP_DATASOURCE_PASSWORD
- name: APP_DATASOURCE_DRIVER_CLASS_NAME
value: org.postgresql.Driver
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
@@ -0,0 +1,78 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-server
annotations:
argocd.argoproj.io/sync-wave: "10"
spec:
replicas: 1
selector:
matchLabels:
app: auth-server
template:
metadata:
labels:
app: auth-server
spec:
serviceAccountName: auth-server
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: auth-server
image: ghcr.io/donghyeonka/project-auth-server
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: http
envFrom:
- configMapRef:
name: auth-server-config
- secretRef:
name: postgres-auth-server-credentials
- secretRef:
name: keycloak-client-auth-server
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: false
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
cpu: 1000m
memory: 1024Mi
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: http
initialDelaySeconds: 20
timeoutSeconds: 3
failureThreshold: 3
periodSeconds: 10
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: http
initialDelaySeconds: 30
timeoutSeconds: 3
failureThreshold: 5
periodSeconds: 15
startupProbe:
httpGet:
path: /actuator/health/liveness
port: http
initialDelaySeconds: 10
timeoutSeconds: 3
periodSeconds: 10
failureThreshold: 12
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- serviceaccount.yaml
- auth-db-migration-serviceaccount.yaml
- service.yaml
- db-migration-job.yaml
- deployment.yaml
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Service
metadata:
name: auth-server
spec:
selector:
app: auth-server
ports:
- name: http
port: 80
targetPort: 8080
type: ClusterIP
@@ -0,0 +1,7 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: auth-server
automountServiceAccountToken: false
imagePullSecrets:
- name: ghcr-regcred
+30
View File
@@ -0,0 +1,30 @@
# Cluster Entrypoints
클러스터별 최종 desired state 진입점입니다. GitOps controller의 root는 이
디렉터리 아래 **하나의 실제 cluster leaf**만 가리킵니다.
소규모:
```text
clusters/
└── dev/
└── main/
└── kustomization.yaml
```
다중 환경·리전:
```text
clusters/
├── dev/ap-northeast-2/dev-a/
├── staging/ap-northeast-2/staging-a/
└── prod/
├── ap-northeast-2/prod-a/
└── ap-southeast-1/prod-b/
```
cluster leaf에는 catalog 구현을 복사하지 않고 선택 목록과 cluster 고유 patch만
둡니다. 계정 또는 조직 경계가 필요하면 경로 segment를 추가할 수 있지만 자동화가
고정 depth에 의존하지 않도록 합니다.
`_template`을 실제 경로로 복사해 시작합니다.
+31
View File
@@ -0,0 +1,31 @@
# __REPLACE_ME_CLUSTER_NAME__
## 대상
- Environment: __REPLACE_ME_ENVIRONMENT__
- Region: __REPLACE_ME_REGION__
- Cluster: __REPLACE_ME_CLUSTER_NAME__
- Owner: __REPLACE_ME_OWNER__
## 구성
`kustomization.yaml``resources`에 필요한 platform, policy, tenant와 app의
base 또는 overlay를 추가합니다. 예:
```yaml
resources:
- ../../../platform/core/base
- ../../../policies/baseline/base
- ../../../apps/example-api/overlays/prod
```
위 예시는 `clusters/dev/main` 경로를 기준으로 합니다. 실제 상대 경로는 cluster
leaf 깊이에 맞게 조정합니다.
## 규칙
- 이 디렉터리가 해당 클러스터의 유일한 root입니다.
- 공통 manifest를 복사하지 않습니다.
- 클러스터 고유 차이만 local patch로 둡니다.
- dependency/sync 순서와 장애 시 reconcile 중지 절차를 문서화합니다.
- `_template` 자체를 GitOps controller에 연결하지 않습니다.
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Add reusable platform, policy, tenant and app paths after copying this template.
resources: []
+35
View File
@@ -0,0 +1,35 @@
# dev-k3s cluster profile
`dev-k3s`는 이 reference lab이 현재 지원하는 유일한 cluster profile입니다.
ApplicationSet의 `server`는 in-cluster API
`https://kubernetes.default.svc`를 사용합니다.
`kustomization.yaml`은 bootstrap root Application이 읽는 유일한 cluster
entrypoint이며 permission-scoped Argo CD control plane을 조립합니다.
ApplicationSet inventory는 아래 cluster-specific overlay를 child
Application source로 선택합니다.
- `overlays/platform/vault`: dev Vault namespace, NetworkPolicy와 single-node profile
- `overlays/systems/auth-system`: `auth-system-dev` namespace, internal/public host,
Vault injection path와 system NetworkPolicy
- `overlays/workloads/*`: dev namespace, image reference, ingress, pull
SealedSecret과 workload NetworkPolicy
현재 Vault NetworkPolicy의 Kubernetes API CIDR와 node address는
`dev-k3s`에 종속됩니다. 다른 클러스터에 그대로 복사하지 말고 해당
클러스터의 service/node network를 확인해야 합니다.
두 번째 클러스터를 추가할 때는 다음 순서를 사용합니다.
1. 실제 차이가 있는 overlay만 `gitops/clusters/<cluster>/overlays`
추가합니다.
2. Argo CD cluster credential을 Git 밖에서 등록합니다.
3. 각 AppProject destination에 정확한 API server/namespace를 추가합니다.
4. 권한별 ApplicationSet inventory에 `autoSync: "false"` element를
추가합니다.
5. Render, live diff와 외부 prerequisite를 검증한 단계별 PR로 gate를
엽니다.
Production은 이 profile의 이름 변경이나 복사로 만들지 않습니다. TLS,
availability, secret authority, storage/backup, approval과 recovery
contract를 먼저 별도 설계해야 합니다.
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../platform/control-plane/argocd
@@ -0,0 +1,9 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: vault
resources:
- ../../../../../platform/shared-services/vault/base
- namespace.yaml
- networkpolicy.yaml
@@ -0,0 +1,13 @@
apiVersion: v1
kind: Namespace
metadata:
name: vault
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
labels:
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/enforce-version: latest
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/audit-version: latest
@@ -0,0 +1,118 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: vault-server-default-deny
spec:
podSelector:
matchLabels:
app: vault
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: vault-server-allow-dns-egress
spec:
podSelector:
matchLabels:
app: vault
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: vault-server-allow-postgres-egress
spec:
podSelector:
matchLabels:
app: vault
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: auth-system-dev
podSelector:
matchLabels:
app: postgres
ports:
- protocol: TCP
port: 5432
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: vault-server-allow-kubernetes-api-egress
spec:
podSelector:
matchLabels:
app: vault
policyTypes:
- Egress
egress:
- to:
- ipBlock:
cidr: 10.43.0.1/32
ports:
- protocol: TCP
port: 443
- to:
- ipBlock:
cidr: 10.208.141.98/32
ports:
- protocol: TCP
port: 6443
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: vault-server-allow-ingress-from-workloads
spec:
podSelector:
matchLabels:
app: vault
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: auth-dev
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: auth-system-dev
ports:
- protocol: TCP
port: 8200
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: vault-agent-injector-webhook-ingress
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: vault-agent-injector
policyTypes:
- Ingress
ingress:
- ports:
- protocol: TCP
port: 8080
@@ -0,0 +1,9 @@
POSTGRES_SUPERUSER=postgres
POSTGRES_DEFAULT_DB=postgres
AUTH_DB_NAME=project_auth
AUTH_DB_USER=project_auth
KEYCLOAK_DB_NAME=keycloak
KEYCLOAK_DB_USER=keycloak
KEYCLOAK_BOOTSTRAP_ADMIN_USERNAME=admin
KEYCLOAK_CLIENT_ID=project-auth-server
AUTH_SERVER_BASE_URL=http://auth-public.auth-dev.svc.cluster.local
@@ -0,0 +1,86 @@
apiVersion: batch/v1
kind: Job
metadata:
name: keycloak-client-sync
spec:
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-service-account-token-volume-name: vault-token
vault.hashicorp.com/agent-inject-perms-keycloak-sync-env: "0644"
vault.hashicorp.com/agent-inject-secret-keycloak-sync-env: kv/data/dev/systems/auth-system/keycloak/bootstrap-admin
vault.hashicorp.com/agent-inject-template-keycloak-sync-env: |
{{ with secret "kv/data/dev/systems/auth-system/keycloak/bootstrap-admin" }}
export KC_BOOTSTRAP_ADMIN_PASSWORD={{ printf "%q" .Data.data.KC_BOOTSTRAP_ADMIN_PASSWORD }}
{{ end }}
{{ with secret "kv/data/dev/workloads/auth-server/keycloak-client" }}
export KEYCLOAK_CLIENT_SECRET={{ printf "%q" .Data.data.KEYCLOAK_CLIENT_SECRET }}
{{ end }}
vault.hashicorp.com/agent-pre-populate-only: "true"
vault.hashicorp.com/role: keycloak-client-sync-dev
spec:
automountServiceAccountToken: false
volumes:
- name: vault-token
projected:
defaultMode: 0444
sources:
- serviceAccountToken:
audience: vault
expirationSeconds: 3600
path: token
containers:
- name: keycloak-client-sync
command:
- /bin/sh
- -ec
args:
- |
. /vault/secrets/keycloak-sync-env
set -eu
ready=false
for _ in $(seq 1 60); do
if /opt/keycloak/bin/kcadm.sh config credentials \
--server http://keycloak \
--realm master \
--user "$KC_BOOTSTRAP_ADMIN_USERNAME" \
--password "$KC_BOOTSTRAP_ADMIN_PASSWORD" >/dev/null 2>&1; then
ready=true
break
fi
sleep 5
done
test "$ready" = true
CLIENT_UUID=$(/opt/keycloak/bin/kcadm.sh get clients \
-r project-auth \
-q clientId="$KEYCLOAK_CLIENT_ID" | sed -n 's/.*"id" : "\([^"]*\)".*/\1/p' | head -n 1)
test -n "$CLIENT_UUID"
/opt/keycloak/bin/kcadm.sh update "clients/${CLIENT_UUID}" \
-r project-auth \
-s "secret=$KEYCLOAK_CLIENT_SECRET" \
-s "baseUrl=$AUTH_SERVER_BASE_URL" \
-s 'redirectUris=["'"$AUTH_SERVER_BASE_URL"'/login/oauth2/code/keycloak-google","'"$AUTH_SERVER_BASE_URL"'/login/oauth2/code/keycloak-github"]' \
-s 'webOrigins=["'"$AUTH_SERVER_BASE_URL"'"]'
env:
- $patch: replace
- name: KC_BOOTSTRAP_ADMIN_USERNAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_BOOTSTRAP_ADMIN_USERNAME
- name: KEYCLOAK_CLIENT_ID
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_CLIENT_ID
- name: AUTH_SERVER_BASE_URL
valueFrom:
configMapKeyRef:
name: auth-system-config
key: AUTH_SERVER_BASE_URL
@@ -0,0 +1,19 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: keycloak
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
ingressClassName: traefik
rules:
- host: keycloak-public.auth-system-dev.svc.cluster.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: keycloak
port:
number: 80
@@ -0,0 +1,16 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: keycloak
spec:
template:
spec:
containers:
- name: keycloak
env:
- name: KC_HOSTNAME
value: keycloak-public.auth-system-dev.svc.cluster.local
- name: KC_HOSTNAME_STRICT
value: "false"
- name: KC_PROXY_HEADERS
value: xforwarded
@@ -0,0 +1,59 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: keycloak
spec:
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-service-account-token-volume-name: vault-token
vault.hashicorp.com/agent-inject-perms-keycloak-env: "0644"
vault.hashicorp.com/agent-inject-secret-keycloak-env: kv/data/dev/systems/auth-system/postgres/keycloak
vault.hashicorp.com/agent-inject-template-keycloak-env: |
{{ with secret "kv/data/dev/systems/auth-system/postgres/keycloak" }}
export KC_DB_PASSWORD={{ printf "%q" .Data.data.KEYCLOAK_DB_PASSWORD }}
{{ end }}
{{ with secret "kv/data/dev/systems/auth-system/keycloak/bootstrap-admin" }}
export KC_BOOTSTRAP_ADMIN_PASSWORD={{ printf "%q" .Data.data.KC_BOOTSTRAP_ADMIN_PASSWORD }}
{{ end }}
vault.hashicorp.com/role: keycloak-dev
spec:
automountServiceAccountToken: false
volumes:
- name: vault-token
projected:
defaultMode: 0444
sources:
- serviceAccountToken:
audience: vault
expirationSeconds: 3600
path: token
containers:
- name: keycloak
command:
- /bin/sh
- -ec
args:
- |
. /vault/secrets/keycloak-env
exec /opt/keycloak/bin/kc.sh start-dev --import-realm --http-port=8080
env:
- $patch: replace
- name: KC_DB
value: postgres
- name: KC_DB_URL
value: jdbc:postgresql://postgres:5432/keycloak
- name: KC_DB_USERNAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_DB_USER
- name: KC_HEALTH_ENABLED
value: "true"
- name: KC_BOOTSTRAP_ADMIN_USERNAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_BOOTSTRAP_ADMIN_USERNAME
@@ -0,0 +1,26 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: auth-system-dev
resources:
- ../../../../../apps/systems/auth-system/base
- namespace.yaml
- keycloak-ingress.yaml
- public-access.yaml
- networkpolicy.yaml
generatorOptions:
annotations:
argocd.argoproj.io/sync-wave: "0"
configMapGenerator:
- name: auth-system-config
envs:
- config.env
patches:
- path: postgres.vault-patch.yaml
- path: keycloak.vault-patch.yaml
- path: keycloak-client-sync.vault-patch.yaml
- path: keycloak.public-url-patch.yaml
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Namespace
metadata:
name: auth-system-dev
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
labels:
vault-injection: enabled
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/enforce-version: latest
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/audit-version: latest
@@ -0,0 +1,161 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-system-default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-system-allow-dns-egress
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-system-allow-vault-egress
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: vault
podSelector:
matchLabels:
app: vault
ports:
- protocol: TCP
port: 8200
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-system-allow-keycloak-egress-to-postgres
spec:
podSelector:
matchLabels:
app: keycloak
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
app: postgres
ports:
- protocol: TCP
port: 5432
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-system-allow-keycloak-client-sync-to-keycloak
spec:
podSelector:
matchLabels:
app: keycloak-client-sync
policyTypes:
- Egress
egress:
- to:
- podSelector:
matchLabels:
app: keycloak
ports:
- protocol: TCP
port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-system-allow-keycloak-ingress
spec:
podSelector:
matchLabels:
app: keycloak
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 8080
- from:
- podSelector:
matchLabels:
app: keycloak-client-sync
ports:
- protocol: TCP
port: 8080
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-system-allow-postgres-ingress
spec:
podSelector:
matchLabels:
app: postgres
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: auth-dev
podSelector:
matchExpressions:
- key: app
operator: In
values:
- auth-server
- auth-db-migration
ports:
- protocol: TCP
port: 5432
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: vault
podSelector:
matchLabels:
app: vault
ports:
- protocol: TCP
port: 5432
- from:
- podSelector:
matchLabels:
app: keycloak
ports:
- protocol: TCP
port: 5432
@@ -0,0 +1,79 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres
spec:
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-service-account-token-volume-name: vault-token
vault.hashicorp.com/agent-inject-perms-postgres-env: "0644"
vault.hashicorp.com/agent-inject-secret-postgres-env: kv/data/dev/systems/auth-system/postgres/superuser
vault.hashicorp.com/agent-inject-template-postgres-env: |
{{ with secret "kv/data/dev/systems/auth-system/postgres/superuser" }}
export POSTGRES_PASSWORD={{ printf "%q" .Data.data.POSTGRES_SUPERUSER_PASSWORD }}
{{ end }}
{{ with secret "kv/data/dev/systems/auth-system/postgres/auth-server" }}
export AUTH_DB_PASSWORD={{ printf "%q" .Data.data.AUTH_DB_PASSWORD }}
{{ end }}
{{ with secret "kv/data/dev/systems/auth-system/postgres/keycloak" }}
export KEYCLOAK_DB_PASSWORD={{ printf "%q" .Data.data.KEYCLOAK_DB_PASSWORD }}
{{ end }}
vault.hashicorp.com/role: postgres-dev
spec:
automountServiceAccountToken: false
volumes:
- name: vault-token
projected:
defaultMode: 0444
sources:
- serviceAccountToken:
audience: vault
expirationSeconds: 3600
path: token
containers:
- name: postgres
command:
- /bin/sh
- -ec
args:
- |
. /vault/secrets/postgres-env
exec docker-entrypoint.sh postgres
env:
- $patch: replace
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: auth-system-config
key: POSTGRES_SUPERUSER
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: auth-system-config
key: POSTGRES_DEFAULT_DB
- name: AUTH_DB_NAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: AUTH_DB_NAME
- name: AUTH_DB_USER
valueFrom:
configMapKeyRef:
name: auth-system-config
key: AUTH_DB_USER
- name: KEYCLOAK_DB_NAME
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_DB_NAME
- name: KEYCLOAK_DB_USER
valueFrom:
configMapKeyRef:
name: auth-system-config
key: KEYCLOAK_DB_USER
@@ -0,0 +1,11 @@
apiVersion: v1
kind: Service
metadata:
name: keycloak-public
spec:
type: ExternalName
externalName: traefik.kube-system.svc.cluster.local
ports:
- name: http
port: 80
targetPort: 80
@@ -0,0 +1,2 @@
APP_SERVER_PORT=8082
APP_SECURITY_OAUTH2_RESOURCESERVER_JWT_ISSUER_URI=http://auth-public.auth-dev.svc.cluster.local
@@ -0,0 +1,14 @@
---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: ghcr-regcred
namespace: api-dev
spec:
encryptedData:
.dockerconfigjson: AgC7O6nq0aej7pxKXYyJy01HomK4s4jqkrwJ2iiVqjWrv4LjMqYY/8XiPnv7eyOAhBlpnSjULZ0pNPnQjUh6TUkcfbV0VOXjha6qkFIRZXLTlySvooS4FJH3lGjYBlli5bDfVw2KZTy9zR3qcpclOVJo5eEGJ+3rrl6NCUV7CzXfhpMsdATXtDRK1reSlpS2qWgLYJ8Dt0mdaz9yt1baGC3PwjeohnbmeCxRDi8HWOgWTXj0NqBtnpHnzQz5dox5NduQQCDh4ZrP7XK4CgeC3OIuTy1BokpTD5xfN3e0apiiZcUKQCN2I4NJ+B/JqgxceLxVXl3kYswlH+MjkzuejB3OPcGJsfQAs3gJUPB460mzT2S21xm4GloqL73utz+JuyE4pmCuPbFzgaJFNEamtAWtwZRiUwpjBG0HHLg2JPuLc07ljzfSiO9IqOA2zFmgYTX1dLdhrgSzIO6jaWFvctKTqavPxTRHNjBG9as+9gbQ7caH7vcV4k7q7XyEWpeEv9rCyVWcuRvZ7QNkHU1ss44uimr5I43JkEnsmacFRanxIlB80t357iGNiGNGCJFgfiVnh1PqmPlCvEtA42iTt+++2aU+x1SmiS4LYEs4YowVmanm0t1dBWSOMC7y6kM91W/fGPEo5i0lWbdvDKFuNS8RAWu4mXTgLjgbWz9a5PMcdioh0JMxV78FkDkU1jte/hFeIMOmA/xwrzIXVu7JTJK5a+6A9Dmhjpuiunv1nhDsxg1okPjiJ7W4T2pc0JpU+7XjhsXipE0FCwiNxwsh0GvI5MFGmipQwKbcSJuQJrXqAbTZZyDpa+0rUyc9YhYxAgxKf8AzxffMgDKm68MuS8KVdvIhC0dKUOccRnmrSYN8GMJSlZE065MOjY5FGsVfPRPHMYLhQnkNQPXSDB34UXl97mrUIz6mR3mnZar0PKqm9d5vZ19ujIwV7ZQEar8Sf6ZbT8n25hLUmq6JkGzsRiHo42S2JoVl1QlHgzQbv6i/JPsjzQcKfVhtAIXiyMIMvsbYLCjX9df6yVU3e0pCBVl3sGSMbOf4/QVrJtCqog==
template:
metadata:
name: ghcr-regcred
namespace: api-dev
type: kubernetes.io/dockerconfigjson
@@ -0,0 +1,20 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-server
annotations:
argocd.argoproj.io/sync-wave: "20"
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
ingressClassName: traefik
rules:
- host: api-public.api-dev.svc.cluster.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-server
port:
number: 80
@@ -0,0 +1,26 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: api-dev
resources:
- ../../../../../apps/workloads/api-server/base
- namespace.yaml
- ingress.yaml
- public-access.yaml
- networkpolicy.yaml
- ghcr-regcred.sealedsecret.yaml
generatorOptions:
annotations:
argocd.argoproj.io/sync-wave: "0"
configMapGenerator:
- name: api-server-config
envs:
- config.env
images:
- name: ghcr.io/donghyeonka/project-api-server
newName: ghcr.io/donghyeonka/project-api-server
newTag: fd097c9
@@ -0,0 +1,13 @@
apiVersion: v1
kind: Namespace
metadata:
name: api-dev
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
labels:
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/enforce-version: latest
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/audit-version: latest
@@ -0,0 +1,75 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-dev-default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-dev-allow-dns-egress
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-dev-allow-public-egress
spec:
podSelector:
matchLabels:
app: api-server
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 8000
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-dev-allow-ingress-from-traefik
spec:
podSelector:
matchLabels:
app: api-server
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 8082
@@ -0,0 +1,11 @@
apiVersion: v1
kind: Service
metadata:
name: api-public
spec:
type: ExternalName
externalName: traefik.kube-system.svc.cluster.local
ports:
- name: http
port: 80
targetPort: 80
@@ -0,0 +1,21 @@
SPRING_PROFILES_ACTIVE=dev
SERVER_FORWARD_HEADERS_STRATEGY=framework
APP_DOCS_TITLE=Project Auth Server API
APP_DOCS_DESCRIPTION=dev auth-server OpenAPI
APP_DOCS_VERSION=v1
APP_DATASOURCE_URL=jdbc:postgresql://postgres.auth-system-dev.svc.cluster.local:5432/project_auth
APP_PERSISTENCE_MIGRATION_RUN_ON_STARTUP=false
APP_SECURITY_OAUTH2_KEYCLOAK_ISSUER_URI=http://keycloak-public.auth-system-dev.svc.cluster.local/realms/project-auth
APP_SECURITY_OAUTH2_KEYCLOAK_CLIENT_ID=project-auth-server
APP_SECURITY_OAUTH2_GOOGLE_REGISTRATION_ID=keycloak-google
APP_SECURITY_OAUTH2_GOOGLE_IDP_HINT=google
APP_SECURITY_OAUTH2_GITHUB_REGISTRATION_ID=keycloak-github
APP_SECURITY_OAUTH2_GITHUB_IDP_HINT=github
APP_SECURITY_JWT_ISSUER=http://auth-public.auth-dev.svc.cluster.local
APP_SECURITY_JWT_ACTIVE_KEY_ID=dev-vault-rsa-1
APP_SECURITY_JWT_GENERATE_KEY_PAIR_ON_STARTUP=false
APP_SECURITY_JWT_ACCESS_TOKEN_EXPIRATION=PT30M
APP_SECURITY_JWT_VAULT_ENABLED=true
APP_SECURITY_JWT_VAULT_ADDRESS=http://vault.vault.svc.cluster.local:8200
APP_SECURITY_JWT_VAULT_MOUNT_PATH=transit
APP_SECURITY_JWT_VAULT_TRANSIT_KEY_NAME=project-auth-jwt
@@ -0,0 +1,55 @@
apiVersion: batch/v1
kind: Job
metadata:
name: auth-db-migration
spec:
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-service-account-token-volume-name: vault-token
vault.hashicorp.com/agent-inject-secret-migration-env: database/creds/auth-db-migration-dev
vault.hashicorp.com/agent-inject-template-migration-env: |
{{- with secret "database/creds/auth-db-migration-dev" -}}
export APP_DATASOURCE_USERNAME={{ printf "%q" .Data.username }}
export APP_DATASOURCE_PASSWORD={{ printf "%q" .Data.password }}
{{- end }}
vault.hashicorp.com/agent-pre-populate-only: "true"
vault.hashicorp.com/agent-run-as-group: "10001"
vault.hashicorp.com/agent-run-as-user: "10001"
vault.hashicorp.com/role: auth-db-migration-dev
spec:
automountServiceAccountToken: false
volumes:
- name: vault-token
projected:
defaultMode: 0444
sources:
- serviceAccountToken:
audience: vault
expirationSeconds: 3600
path: token
containers:
- name: auth-db-migration
command:
- /bin/sh
- -ec
args:
- |
. /vault/secrets/migration-env
exec java -jar /app/migration.jar
env:
- $patch: replace
- name: SPRING_MAIN_WEB_APPLICATION_TYPE
value: none
- name: APP_PERSISTENCE_MIGRATION_RUN_ON_STARTUP
value: "true"
- name: APP_PERSISTENCE_MIGRATION_LOCATION
value: classpath:db/migration
- name: APP_DATASOURCE_URL
valueFrom:
configMapKeyRef:
name: auth-server-config
key: APP_DATASOURCE_URL
- name: APP_DATASOURCE_DRIVER_CLASS_NAME
value: org.postgresql.Driver
@@ -0,0 +1,49 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-server
spec:
template:
metadata:
annotations:
vault.hashicorp.com/agent-cache-enable: "true"
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-service-account-token-volume-name: vault-token
vault.hashicorp.com/agent-inject-secret-runtime-env: kv/data/dev/systems/auth-system/postgres/auth-server
vault.hashicorp.com/agent-inject-template-runtime-env: |
{{ with secret "kv/data/dev/systems/auth-system/postgres/auth-server" }}
export APP_DATASOURCE_USERNAME={{ printf "%q" .Data.data.APP_DATASOURCE_USERNAME }}
export APP_DATASOURCE_PASSWORD={{ printf "%q" .Data.data.APP_DATASOURCE_PASSWORD }}
{{ end }}
{{ with secret "kv/data/dev/workloads/auth-server/keycloak-client" }}
export APP_SECURITY_OAUTH2_KEYCLOAK_CLIENT_SECRET={{ printf "%q" .Data.data.APP_SECURITY_OAUTH2_KEYCLOAK_CLIENT_SECRET }}
{{ end }}
vault.hashicorp.com/agent-inject-token: "true"
vault.hashicorp.com/agent-run-as-group: "10001"
vault.hashicorp.com/agent-run-as-user: "10001"
vault.hashicorp.com/role: auth-server-dev
spec:
automountServiceAccountToken: false
volumes:
- name: vault-token
projected:
defaultMode: 0444
sources:
- serviceAccountToken:
audience: vault
expirationSeconds: 3600
path: token
containers:
- name: auth-server
command:
- /bin/sh
- -ec
args:
- |
. /vault/secrets/runtime-env
export APP_SECURITY_JWT_VAULT_TOKEN="$(cat /vault/secrets/token)"
exec java -jar /app/application.jar
envFrom:
- configMapRef:
name: auth-server-config
@@ -0,0 +1,14 @@
---
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: ghcr-regcred
namespace: auth-dev
spec:
encryptedData:
.dockerconfigjson: AgAIKHR/abtMIrkphJPG63jmOCXIMSBdDMdMUYjx37FRV9SpCA/j3UTHhhZlDSK5gXK/p9FGsR1mBjME+imHEa7/rqBMzfSoJjRQCln3TjD1GT+XJ9J8/qS9AQl+GG9kcJfVMTfGslFbQc1993uURdb/CDupe8XsJosMfzvV5h+SpSdytU2Y8CrhZSDNtz7/2Y+x0LQNfyafjXCxsc1kS5qt3WzPFD/zyk70+e2oq868ETBaij2dHEAQXH+XVz2ilqyzLE633X3/Wf/ADRMJ1i7XWaMir01mSXaB97U2+OPaSRJDf1TAfikn6D4BydNsTgI2UQ839edrgr9EXz1Sr4oYdpWjOpNmtH6OjfiiRj2r0LMvNKR/C1ruROfBcEV7y3kXpKRM5kwG+WM1/1B0ziodu0LZJl23QAgfNnTJrYlJJXJNSFNapPMWEcq91TMeauSIyRgbtLUZEemmlTVybt2SV86Yr+w9SlNAsBJGQaW34Ljfmu7X9H0JwTDkphWUotSPmmKj2GFdpVaNwL0Xvi6OoytiWu0rJWVkGar3mnyLPrkVLb5AlOZYPKxfPhZJfwZxPXBqgWj+6kHGs1SpH6v55wVIz1MdMoGcVqcjlOiMNFd2leOyejYQvRBXKRW3wgzgB6J0OdL1S8z1cOfblO4egx9C6RsaxDC//VDK36aAYvdc55Hsk2GC4iZKsug6EAdXWiiJjdM9yfAcGcv7LemQneA3OslJTdDTpWUdbREiIDXBX4PjEQz0G4oI5YDNYvS8NxUOWZ1ly0BH+uHbcm69tnD9ATSVvjvtKlLiP7/BCYZAguCzNEYkmf1neTtHYlVsoPmIuTkifaf9qRPU831MYj8wTfz56YVT1+aSbfQBK4XP49+bIf5AVo9K5JuEQETqMhZ0G05VjmlwUrgtQ60oZLu8dbvFyt/0VxfWBGcCi+AW+nmro3c14XmA4k1d09xtuoa+8QEObtOTEow3+tyqL46FlJgl7UjX8dd4p7gUAF/iADw8a0M32kIN0NHJ9o2EsFe85rn9qBoDLGrogDXRXJ4iCxE0Zgf7bw6tFA==
template:
metadata:
name: ghcr-regcred
namespace: auth-dev
type: kubernetes.io/dockerconfigjson
@@ -0,0 +1,20 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: auth-server
annotations:
argocd.argoproj.io/sync-wave: "20"
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
ingressClassName: traefik
rules:
- host: auth-public.auth-dev.svc.cluster.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: auth-server
port:
number: 80
@@ -0,0 +1,30 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: auth-dev
resources:
- ../../../../../apps/workloads/auth-server/base
- namespace.yaml
- ingress.yaml
- public-access.yaml
- networkpolicy.yaml
- ghcr-regcred.sealedsecret.yaml
patches:
- path: deployment.vault-patch.yaml
- path: db-migration-job.vault-patch.yaml
generatorOptions:
annotations:
argocd.argoproj.io/sync-wave: "0"
configMapGenerator:
- name: auth-server-config
envs:
- config.env
images:
- name: ghcr.io/donghyeonka/project-auth-server
newName: ghcr.io/donghyeonka/project-auth-server
newTag: 1f47f2c
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Namespace
metadata:
name: auth-dev
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
labels:
vault-injection: enabled
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/enforce-version: latest
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/audit-version: latest
@@ -0,0 +1,105 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-dev-default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-dev-allow-dns-egress
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-dev-allow-auth-system-and-vault-egress
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: auth-system-dev
podSelector:
matchLabels:
app: postgres
ports:
- protocol: TCP
port: 5432
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: vault
podSelector:
matchLabels:
app: vault
ports:
- protocol: TCP
port: 8200
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-dev-allow-public-egress
spec:
podSelector:
matchLabels:
app: auth-server
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 8000
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: auth-dev-allow-ingress-from-traefik
spec:
podSelector:
matchLabels:
app: auth-server
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
app.kubernetes.io/name: traefik
ports:
- protocol: TCP
port: 8080
@@ -0,0 +1,11 @@
apiVersion: v1
kind: Service
metadata:
name: auth-public
spec:
type: ExternalName
externalName: traefik.kube-system.svc.cluster.local
ports:
- name: http
port: 80
targetPort: 80
+23
View File
@@ -0,0 +1,23 @@
# Platform Catalog
클러스터 전체에서 사용하는 addon과 controller의 재사용 구성을 둡니다.
예:
```text
platform/
├── ingress/
├── certificates/
├── external-dns/
├── external-secrets/
├── storage/
├── autoscaling/
└── observability/
```
실제로 사용하는 항목만 만듭니다. 각 component는 자신의 CRD, controller,
configuration과 values/patch를 응집해서 관리합니다. component 간 숨은 의존을
만들지 말고 cluster root 또는 선택한 GitOps controller의 ordering 기능으로
순서를 표현합니다.
새 component는 `_template`을 복사해 시작합니다.
+21
View File
@@ -0,0 +1,21 @@
# __REPLACE_ME_PLATFORM_COMPONENT_NAME__
## 책임
이 component가 소유하는 CRD, controller와 configuration을 적습니다.
## 의존성
선행 component, namespace, identity와 최소 Kubernetes version을 적습니다.
## 구성
- `base`: 모든 대상에서 공유하는 기본값
- `overlays`: 환경/규모별 차이가 실제로 있을 때만 추가
Helm을 사용하면 chart source/version, values와 release 리소스를 이 component
안에 함께 둡니다. 원격 version은 고정합니다.
## 운영
upgrade 순서, health check, rollback과 uninstall 영향을 runbook으로 연결합니다.
+7
View File
@@ -0,0 +1,7 @@
# Base
환경과 클러스터를 모르는 재사용 가능한 기본 manifest만 둡니다.
- 실제 domain, account ID, credential을 하드코딩하지 않습니다.
- resource request/limit와 security context의 안전한 기본값을 둡니다.
- 환경 차이는 `overlays`에서 patch합니다.
@@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: []
@@ -0,0 +1,7 @@
# Overlays
공통 base와 다른 값이 있을 때만 `<environment-or-profile>/` overlay를 만듭니다.
overlay는 base를 참조하고 patch만 포함해야 합니다.
`dev`, `prod` 전체 manifest 복사보다 replica, resource, retention처럼 실제로
달라지는 항목만 표현합니다.
@@ -0,0 +1,8 @@
# Argo CD control plane
Permission-scoped AppProject와 list-generated ApplicationSet inventory를
소유합니다. Bootstrap root는 이 경로를 직접 source로 사용하지 않고
`gitops/clusters/dev-k3s`를 통해 조립합니다.
새 inventory 항목은 `autoSync: "false"`로 시작하며 project, repository와
revision trust boundary를 template하지 않습니다.
@@ -0,0 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- platform-addons.yaml
- platform-services.yaml
- systems.yaml
- workloads.yaml
@@ -0,0 +1,122 @@
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: platform-addons
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-5"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
spec:
goTemplate: true
goTemplateOptions:
- missingkey=error
generators:
- list:
elements:
- component: sealed-secrets
cluster: dev-k3s
server: https://kubernetes.default.svc
namespace: kube-system
repoURL: https://bitnami.github.io/sealed-secrets
chart: sealed-secrets
# renovate: datasource=helm depName=sealed-secrets registryUrl=https://bitnami.github.io/sealed-secrets
revision: 2.17.9
autoSync: "true"
helmValues: |
fullnameOverride: sealed-secrets-controller
keyrenewperiod: 720h
image:
# renovate: datasource=docker depName=bitnami/sealed-secrets-controller
repository: bitnami/sealed-secrets-controller
tag: "0.33.1@sha256:e7fad65c2d2f47e48d9ca17408ed56961bfa6a6dd74ccd4a1a214664156534bc"
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 250m
memory: 256Mi
- component: vault-agent-injector
cluster: dev-k3s
server: https://kubernetes.default.svc
namespace: vault
repoURL: https://helm.releases.hashicorp.com
chart: vault
# renovate: datasource=helm depName=vault registryUrl=https://helm.releases.hashicorp.com
revision: 0.32.0
autoSync: "false"
helmValues: |
global:
externalVaultAddr: http://vault.vault.svc.cluster.local:8200
tlsDisable: true
server:
enabled: false
injector:
enabled: true
authPath: auth/kubernetes
image:
# renovate: datasource=docker depName=hashicorp/vault-k8s
repository: hashicorp/vault-k8s
tag: "1.7.2@sha256:ae3d307658b72a1cf35dab9bdf92c995d45cdc7183af0516857714b5bd0ba84d"
webhook:
failurePolicy: Fail
namespaceSelector:
matchLabels:
vault-injection: enabled
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
agentImage:
# renovate: datasource=docker depName=hashicorp/vault
repository: hashicorp/vault
tag: "1.18.5@sha256:750bb37c1638fa194ab37053a81618c61bb0491ddec6fccac87c07a8e6cd8166"
template:
metadata:
name: "{{ .component }}-{{ .cluster }}"
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
project-gitops.io/auto-sync-gate: "{{ .autoSync }}"
labels:
project-gitops.io/cluster: "{{ .cluster }}"
project-gitops.io/owner: platform
spec:
project: platform-addons
source:
repoURL: "{{ .repoURL }}"
chart: "{{ .chart }}"
targetRevision: "{{ .revision }}"
helm:
values: "{{ .helmValues }}"
destination:
server: "{{ .server }}"
namespace: "{{ .namespace }}"
syncPolicy:
syncOptions:
- CreateNamespace=true
- Prune=confirm
- PruneLast=true
- FailOnSharedResource=true
retry:
limit: 5
refresh: true
backoff:
duration: 5s
factor: 2
maxDuration: 3m
revisionHistoryLimit: 5
templatePatch: |
{{- if eq .autoSync "true" }}
spec:
syncPolicy:
automated:
enabled: true
prune: true
selfHeal: true
{{- end }}
syncPolicy:
applicationsSync: create-update
preserveResourcesOnDeletion: true
@@ -0,0 +1,64 @@
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: platform-services
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-5"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
spec:
goTemplate: true
goTemplateOptions:
- missingkey=error
generators:
- list:
elements:
- component: vault
cluster: dev-k3s
server: https://kubernetes.default.svc
namespace: vault
path: gitops/clusters/dev-k3s/overlays/platform/vault
autoSync: "true"
template:
metadata:
name: "{{ .component }}-{{ .cluster }}"
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
project-gitops.io/auto-sync-gate: "{{ .autoSync }}"
labels:
project-gitops.io/cluster: "{{ .cluster }}"
project-gitops.io/owner: platform
spec:
project: platform-services
source:
repoURL: https://git.learn.hyeonworks.com/donghyeon.kang/project-gitops
targetRevision: main
path: "{{ .path }}"
destination:
server: "{{ .server }}"
namespace: "{{ .namespace }}"
syncPolicy:
syncOptions:
- CreateNamespace=true
- PruneLast=true
- FailOnSharedResource=true
retry:
limit: 5
refresh: true
backoff:
duration: 5s
factor: 2
maxDuration: 3m
revisionHistoryLimit: 5
templatePatch: |
{{- if eq .autoSync "true" }}
spec:
syncPolicy:
automated:
enabled: true
prune: true
selfHeal: true
{{- end }}
syncPolicy:
applicationsSync: create-update
preserveResourcesOnDeletion: true
@@ -0,0 +1,64 @@
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: systems
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-5"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
spec:
goTemplate: true
goTemplateOptions:
- missingkey=error
generators:
- list:
elements:
- component: auth-system
cluster: dev-k3s
server: https://kubernetes.default.svc
namespace: auth-system-dev
path: gitops/clusters/dev-k3s/overlays/systems/auth-system
autoSync: "false"
template:
metadata:
name: "{{ .component }}-{{ .cluster }}"
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
project-gitops.io/auto-sync-gate: "{{ .autoSync }}"
labels:
project-gitops.io/cluster: "{{ .cluster }}"
project-gitops.io/owner: systems
spec:
project: systems
source:
repoURL: https://git.learn.hyeonworks.com/donghyeon.kang/project-gitops
targetRevision: main
path: "{{ .path }}"
destination:
server: "{{ .server }}"
namespace: "{{ .namespace }}"
syncPolicy:
syncOptions:
- CreateNamespace=true
- PruneLast=true
- FailOnSharedResource=true
retry:
limit: 5
refresh: true
backoff:
duration: 5s
factor: 2
maxDuration: 3m
revisionHistoryLimit: 5
templatePatch: |
{{- if eq .autoSync "true" }}
spec:
syncPolicy:
automated:
enabled: true
prune: true
selfHeal: true
{{- end }}
syncPolicy:
applicationsSync: create-update
preserveResourcesOnDeletion: true
@@ -0,0 +1,70 @@
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: workloads
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-5"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
spec:
goTemplate: true
goTemplateOptions:
- missingkey=error
generators:
- list:
elements:
- component: auth-server
cluster: dev-k3s
server: https://kubernetes.default.svc
namespace: auth-dev
path: gitops/clusters/dev-k3s/overlays/workloads/auth-server
autoSync: "false"
- component: api-server
cluster: dev-k3s
server: https://kubernetes.default.svc
namespace: api-dev
path: gitops/clusters/dev-k3s/overlays/workloads/api-server
autoSync: "false"
template:
metadata:
name: "{{ .component }}-{{ .cluster }}"
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
project-gitops.io/auto-sync-gate: "{{ .autoSync }}"
labels:
project-gitops.io/cluster: "{{ .cluster }}"
project-gitops.io/owner: workloads
spec:
project: workloads
source:
repoURL: https://git.learn.hyeonworks.com/donghyeon.kang/project-gitops
targetRevision: main
path: "{{ .path }}"
destination:
server: "{{ .server }}"
namespace: "{{ .namespace }}"
syncPolicy:
syncOptions:
- CreateNamespace=true
- PruneLast=true
- FailOnSharedResource=true
retry:
limit: 5
refresh: true
backoff:
duration: 5s
factor: 2
maxDuration: 3m
revisionHistoryLimit: 5
templatePatch: |
{{- if eq .autoSync "true" }}
spec:
syncPolicy:
automated:
enabled: true
prune: true
selfHeal: true
{{- end }}
syncPolicy:
applicationsSync: create-update
preserveResourcesOnDeletion: true
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- projects
- application-sets
@@ -0,0 +1,8 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- platform-addons.yaml
- platform-services.yaml
- systems.yaml
- workloads.yaml
@@ -0,0 +1,50 @@
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: platform-addons
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-10"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
description: Cluster-scoped controllers installed from approved Helm repositories
sourceRepos:
- https://bitnami.github.io/sealed-secrets
- https://helm.releases.hashicorp.com
destinations:
- namespace: kube-system
server: https://kubernetes.default.svc
- namespace: vault
server: https://kubernetes.default.svc
clusterResourceWhitelist:
- group: ""
kind: Namespace
- group: apiextensions.k8s.io
kind: CustomResourceDefinition
- group: rbac.authorization.k8s.io
kind: ClusterRole
- group: rbac.authorization.k8s.io
kind: ClusterRoleBinding
- group: admissionregistration.k8s.io
kind: MutatingWebhookConfiguration
namespaceResourceWhitelist:
- group: ""
kind: ConfigMap
- group: ""
kind: Secret
- group: ""
kind: Service
- group: ""
kind: ServiceAccount
- group: apps
kind: Deployment
- group: policy
kind: PodDisruptionBudget
- group: rbac.authorization.k8s.io
kind: Role
- group: rbac.authorization.k8s.io
kind: RoleBinding
orphanedResources:
warn: true
@@ -0,0 +1,37 @@
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: platform-services
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-10"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
description: Shared services owned by this standalone reference platform
sourceRepos:
- https://git.learn.hyeonworks.com/donghyeon.kang/project-gitops
destinations:
- namespace: vault
server: https://kubernetes.default.svc
clusterResourceWhitelist:
- group: ""
kind: Namespace
- group: rbac.authorization.k8s.io
kind: ClusterRoleBinding
namespaceResourceWhitelist:
- group: ""
kind: ConfigMap
- group: ""
kind: PersistentVolumeClaim
- group: ""
kind: Service
- group: ""
kind: ServiceAccount
- group: apps
kind: Deployment
- group: networking.k8s.io
kind: NetworkPolicy
orphanedResources:
warn: true
@@ -0,0 +1,39 @@
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: systems
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-10"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
description: Backing systems dedicated to an example product boundary
sourceRepos:
- https://git.learn.hyeonworks.com/donghyeon.kang/project-gitops
destinations:
- namespace: auth-system-dev
server: https://kubernetes.default.svc
clusterResourceWhitelist:
- group: ""
kind: Namespace
namespaceResourceWhitelist:
- group: ""
kind: ConfigMap
- group: ""
kind: Service
- group: ""
kind: ServiceAccount
- group: apps
kind: Deployment
- group: apps
kind: StatefulSet
- group: batch
kind: Job
- group: networking.k8s.io
kind: Ingress
- group: networking.k8s.io
kind: NetworkPolicy
orphanedResources:
warn: true
@@ -0,0 +1,41 @@
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: workloads
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-10"
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
description: First-party application workloads
sourceRepos:
- https://git.learn.hyeonworks.com/donghyeon.kang/project-gitops
destinations:
- namespace: auth-dev
server: https://kubernetes.default.svc
- namespace: api-dev
server: https://kubernetes.default.svc
clusterResourceWhitelist:
- group: ""
kind: Namespace
namespaceResourceWhitelist:
- group: ""
kind: ConfigMap
- group: ""
kind: Service
- group: ""
kind: ServiceAccount
- group: bitnami.com
kind: SealedSecret
- group: apps
kind: Deployment
- group: batch
kind: Job
- group: networking.k8s.io
kind: Ingress
- group: networking.k8s.io
kind: NetworkPolicy
orphanedResources:
warn: true
@@ -0,0 +1,8 @@
# Vault shared service
여러 system이 소비하는 dev Vault service의 Kubernetes base입니다. Namespace,
single-node profile과 NetworkPolicy는
`gitops/clusters/dev-k3s/overlays/platform/vault`에서 결합합니다.
Vault API object는 이 경로가 아니라 `infrastructure/live/dev-k3s`의 세
Terraform state가 소유합니다.
@@ -0,0 +1,13 @@
ui = true
disable_mlock = true
listener "tcp" {
address = "0.0.0.0:8200"
cluster_address = "0.0.0.0:8201"
tls_disable = 1
}
storage "raft" {
path = "/vault/data"
node_id = "vault-0"
}
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- vault-serviceaccount.yaml
- vault-auth-delegator.clusterrolebinding.yaml
- vault-pvc.yaml
- vault-service.yaml
- vault-deployment.yaml
configMapGenerator:
- name: vault-config
files:
- files/vault/vault.hcl
@@ -0,0 +1,12 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: vault-server-auth-delegator
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
subjects:
- kind: ServiceAccount
name: vault-server
namespace: vault
@@ -0,0 +1,91 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: vault
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: vault
template:
metadata:
labels:
app: vault
spec:
serviceAccountName: vault-server
containers:
- name: vault
image: hashicorp/vault:1.18.5@sha256:750bb37c1638fa194ab37053a81618c61bb0491ddec6fccac87c07a8e6cd8166
command:
- /bin/sh
- -ec
env:
- name: VAULT_ADDR
value: http://127.0.0.1:8200
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: VAULT_API_ADDR
value: http://$(POD_IP):8200
- name: VAULT_CLUSTER_ADDR
value: http://$(POD_IP):8201
args:
- |
cp /vault/config/vault.hcl /tmp/vault.hcl
exec vault server -config=/tmp/vault.hcl
ports:
- containerPort: 8200
name: http
- containerPort: 8201
name: cluster
volumeMounts:
- name: vault-config
mountPath: /vault/config
readOnly: true
- name: vault-data
mountPath: /vault/data
readinessProbe:
exec:
command:
- sh
- -c
- vault status -address=http://127.0.0.1:8200 >/dev/null 2>&1
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
livenessProbe:
exec:
command:
- sh
- -c
- vault status -address=http://127.0.0.1:8200 >/dev/null 2>&1; code=$?; [ "$code" -eq 0 ] || [ "$code" -eq 2 ]
initialDelaySeconds: 20
periodSeconds: 15
timeoutSeconds: 5
startupProbe:
exec:
command:
- sh
- -c
- vault status -address=http://127.0.0.1:8200 >/dev/null 2>&1; code=$?; [ "$code" -eq 0 ] || [ "$code" -eq 2 ]
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 30
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
volumes:
- name: vault-config
configMap:
name: vault-config
defaultMode: 0555
- name: vault-data
persistentVolumeClaim:
claimName: vault-data
@@ -0,0 +1,12 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: vault-data
annotations:
argocd.argoproj.io/sync-options: Prune=confirm,Delete=confirm
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: vault
spec:
selector:
app: vault
ports:
- name: http
port: 8200
targetPort: 8200
- name: cluster
port: 8201
targetPort: 8201
type: ClusterIP
@@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault-server
+18
View File
@@ -0,0 +1,18 @@
# Policy Catalog
cluster-wide admission 규칙과 거버넌스 정책을 둡니다. Kyverno, Gatekeeper 등
정책 엔진은 하나를 선택하고 ADR로 기록합니다. 구체적인 Namespace,
Role/RoleBinding, ResourceQuota와 NetworkPolicy 인스턴스는 `tenants`
소유하며 이곳에서 중복 생성하지 않습니다.
권장 분류:
- baseline workload security
- allowed registries와 image 검증
- resource request/limit
- namespace/RBAC 생성 규칙 검증
- network isolation 적용 여부 검증
- 정책 예외와 만료 조건
처음에는 audit 모드와 테스트로 영향 범위를 확인한 뒤 enforcement를 적용합니다.
예외에는 owner, 사유와 만료일을 반드시 기록합니다.
+17
View File
@@ -0,0 +1,17 @@
# __REPLACE_ME_POLICY_SET_NAME__
## 목적과 범위
보호하는 대상, 위협과 제외 범위를 적습니다.
## 적용 단계
audit 결과, enforcement 전환 조건과 rollback을 적습니다.
## 예외
예외 schema, 승인자와 만료 정책을 적습니다.
## 테스트
허용/거부 fixture와 선택한 policy engine의 test 명령을 추가합니다.
@@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: []
+16
View File
@@ -0,0 +1,16 @@
# Tenant Catalog
여러 팀 또는 workload 경계를 운영할 때 사용하는 선택 영역입니다.
가능한 리소스:
- Namespace와 ownership label
- Role/RoleBinding
- ResourceQuota와 LimitRange
- 기본 NetworkPolicy
- secret manager/service account 연결
작은 단일 팀 구성에서는 이 계층을 생략하고 platform 또는 app 소유권에 맞게
namespace를 관리할 수 있습니다. tenant와 app이 같은 namespace를 중복 생성하지
않도록 한쪽만 소유합니다. `policies`는 이런 namespace-scoped 리소스 자체가
아니라 조직 규칙을 검증하는 admission 정책만 소유합니다.
+19
View File
@@ -0,0 +1,19 @@
# __REPLACE_ME_TENANT_NAME__
## 소유자와 범위
Team: __REPLACE_ME_OWNER__
namespace, cluster 범위와 연락처를 적습니다.
## 권한
최소 RBAC와 workload identity contract를 적습니다.
## Guardrail
quota, limit, network와 승인된 policy exception ID/문서 참조를 적습니다.
## 온보딩/오프보딩
생성, 권한 회수, 데이터 보존과 namespace 삭제 절차를 적습니다.
@@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: []