Add platform infrastructure configuration

2026-08-28 17:35:41 +09:00
parent fa76531e5b
commit 16c337bcc9
302 changed files with 83259 additions and 1 deletions
@@ -0,0 +1,81 @@
+# MinIO AIStor ObjectStore
+
+상태(2026-07-23):
+
+- baseline ObjectStore와 인증 S3 스모크: 실제 적용 완료
+- admin-oidc profile과 비공개 Console Ingress: 선언형 구현·dry-run 완료, live 적용 대기
+
+공식 Chart를 수정하지 않고 Kustomize가 Helm을 렌더한다.
+
+## 빌드 루트
+
+- services/minio-aistor: 현재 live baseline 진입점
+- services/minio-aistor/base/deployment: 공통 Helm baseline과 values
+- services/minio-aistor/profiles/admin-oidc: baseline에 OIDC, Console Ingress와
+  추가 NetworkPolicy를 결합한 진입점
+
+root와 profile이 같은 형제 baseline을 참조하므로 profile이 조상 root를 다시
+포함하는 Kustomize 순환이 없다.
+
+## 고정 원본
+
+| 항목 | 값 |
+|---|---|
+| Chart 저장소 | https://helm.min.io/ |
+| Chart | aistor-objectstore |
+| 버전 | 1.0.16 |
+| archive SHA-256 | 50ffa6a4e014cdc48566b237593baab41d039f103cdc50c1b1ac173b8d8bf71e |
+
+검증기가 만드는 임시 Chart cache는
+base/deployment/.helm/charts/aistor-objectstore-1.0.16 아래에만 존재하고 종료 시
+제거한다.
+
+## baseline 계약
+
+- namespace: object-storage
+- ObjectStore: minio-aistor
+- pool/server/volume: 각각 1
+- 저장소: aistor-local-xfs-retain, 900Gi RWO
+- S3와 Console Service: ClusterIP
+- baseline Ingress, NodePort, LoadBalancer: 없음
+- pvcProtection과 Retain Local PV
+- 기존 root 계정: 비상용으로 유지
+
+Standalone이므로 erasure-code 이중화와 노드 failover는 없다.
+
+## Secret 계약
+
+baseline:
+
+- object-storage/aistor-root-configuration: config.env
+
+admin-oidc profile:
+
+- object-storage/aistor-keycloak-oidc: client-id, client-secret
+
+평문, Base64, checksum을 Git과 문서에 기록하지 않는다.
+
+## 렌더와 적용
+
+baseline 정적 렌더:
+
+    kubectl kustomize --enable-helm services/minio-aistor
+
+admin profile 전체 정적 검증:
+
+    PLATFORM_HELM_BIN=/home/donghyeon/.local/bin/helm \
+      bash scripts/validate/render-admin-services.sh
+
+admin profile 실제 적용은 private DNS, admin 인증서와 Keycloak client/group 준비
+뒤 apply-admin-services.sh가 수행한다. S3 API에는 Ingress를 만들지 않는다.
+
+현재 baseline은 Initialized/green이고 data0-minio-aistor-pool-0-0 PVC가
+aistor-data-local-pv에 Bound다. S3 회귀 검사는 다음과 같다.
+
+    bash scripts/validate/aistor-s3-smoke.sh --execute
+
+## 공식 문서
+
+- https://docs.min.io/aistor/reference/kubernetes/object-store-helm-chart/
+- https://docs.min.io/aistor/reference/kubernetes/pvc-protection/
+- https://docs.min.io/aistor/administration/iam/identity/keycloak-identity/
@@ -0,0 +1,7 @@
+# 기본 확장 지점
+
+공통 ObjectStore Helm baseline은 base/deployment에 있다. 저장소, resource,
+ClusterIP와 root 비상 계정 경계만 이 계층이 소유한다.
+
+OIDC 환경 변수와 Console Ingress는 profiles/admin-oidc에서만 추가한다.
+Chart 템플릿을 저장소에 복제하거나 직접 수정하지 않는다.
@@ -0,0 +1,6 @@
+# AIStor ObjectStore baseline
+
+AIStor ObjectStore의 공통 Helm baseline입니다. 저장소, root 비상 계정,
+리소스 제한과 ClusterIP 경계만 정의합니다.
+
+Console Ingress와 Keycloak OIDC는 profiles/admin-oidc에서만 추가합니다.
@@ -0,0 +1,17 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: object-storage
+
+helmGlobals:
+  chartHome: .helm/charts
+
+helmCharts:
+  - name: aistor-objectstore
+    repo: https://helm.min.io/
+    version: 1.0.16
+    releaseName: minio-aistor
+    namespace: object-storage
+    valuesFile: values/home.yaml
+    includeCRDs: false
+    skipTests: true
@@ -0,0 +1,67 @@
+secrets:
+  name: aistor-root-configuration
+  existingSecret: true
+  disableRoot: false
+
+objectStore:
+  name: minio-aistor
+
+  configuration:
+    name: aistor-root-configuration
+
+  pools:
+    - name: pool-0
+      servers: 1
+      volumesPerServer: 1
+      storageNamePrefix: data
+      size: 900Gi
+      storageClassName: aistor-local-xfs-retain
+      nodeSelector:
+        kubernetes.io/hostname: donghyeon-system-product-name
+      resources:
+        requests:
+          cpu: 500m
+          memory: 2Gi
+        limits:
+          cpu: "3"
+          memory: 8Gi
+      securityContext:
+        runAsUser: 1000
+        runAsGroup: 1000
+        runAsNonRoot: true
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+      containerSecurityContext:
+        runAsUser: 1000
+        runAsGroup: 1000
+        runAsNonRoot: true
+        allowPrivilegeEscalation: false
+        capabilities:
+          drop:
+            - ALL
+        seccompProfile:
+          type: RuntimeDefault
+
+  mountPath: /export
+  subPath: /data
+
+  certificates:
+    disableAutoCert: true
+
+  services:
+    minio:
+      serviceType: ClusterIP
+    console:
+      serviceType: ClusterIP
+
+  pvcProtection: true
+  prometheusOperator: false
+
+  sideCars:
+    resources:
+      requests:
+        cpu: 100m
+        memory: 128Mi
+      limits:
+        cpu: 500m
+        memory: 512Mi
@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - base/deployment
@@ -0,0 +1,8 @@
+# 홈 환경 Overlay 예약 지점
+
+현재 홈 환경의 공통 Helm values는
+../../base/deployment/values/home.yaml에 있다. 실제 배포 진입점은 저장소 root와
+profiles/admin-oidc다.
+
+이 디렉터리는 이후 홈 환경 전용 patch가 필요할 때만 사용하며 현재 활성
+Kustomization 리소스는 없다.
@@ -0,0 +1,16 @@
+# AIStor Console OIDC 프로필
+
+기존 ObjectStore baseline은 그대로 두고 다음 항목만 추가합니다.
+
+- storage-admin.learn.hyeonworks.com Console Ingress
+- Keycloak 표준 OIDC 환경 변수와 policy claim
+- Traefik의 Console 접근과 Keycloak HTTPS egress
+
+S3 API Service에는 Ingress를 만들지 않습니다. aistor-keycloak-oidc
+Secret의 client-id, client-secret 값은 적용 스크립트가 만들며 Git에
+저장하지 않습니다. 기존 root 계정은 비상용으로 유지합니다.
+
+`MINIO_IDENTITY_OPENID_VENDOR=keycloak`은 사용하지 않습니다. 이 옵션은
+AIStor가 Keycloak Admin API로 사용자를 추가 검증할 때 admin URL, realm과
+service-account 권한을 요구합니다. 이 프로필은 Keycloak이 발급한 `policy`
+claim을 표준 OIDC로 소비하므로 불필요한 Admin API 권한을 부여하지 않습니다.
@@ -0,0 +1,24 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: minio-aistor-console
+  namespace: object-storage
+  labels:
+    app.kubernetes.io/name: minio-aistor
+    app.kubernetes.io/component: admin-console
+    app.kubernetes.io/part-of: platform
+  annotations:
+    traefik.ingress.kubernetes.io/router.entrypoints: web
+spec:
+  ingressClassName: traefik
+  rules:
+    - host: storage-admin.learn.hyeonworks.com
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: minio-aistor-console
+                port:
+                  number: 9090
@@ -0,0 +1,15 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - ../../base/deployment
+  - console-ingress.yaml
+  - network-policies.yaml
+
+patches:
+  - path: objectstore-oidc-patch.yaml
+    target:
+      group: aistor.min.io
+      version: v1
+      kind: ObjectStore
+      name: minio-aistor
@@ -0,0 +1,101 @@
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: object-storage-allow-traefik-console
+  namespace: object-storage
+  labels:
+    app.kubernetes.io/part-of: platform
+    app.kubernetes.io/component: admin-console
+spec:
+  podSelector:
+    matchLabels:
+      aistor.min.io/objectStore: minio-aistor
+  policyTypes:
+    - Ingress
+  ingress:
+    - from:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: kube-system
+          podSelector:
+            matchLabels:
+              app.kubernetes.io/name: traefik
+      ports:
+        - protocol: TCP
+          port: 9090
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: object-storage-allow-host-nginx-keycloak
+  namespace: object-storage
+  labels:
+    app.kubernetes.io/part-of: platform
+    app.kubernetes.io/component: oidc-client
+spec:
+  podSelector:
+    matchLabels:
+      aistor.min.io/objectStore: minio-aistor
+  policyTypes:
+    - Egress
+  egress:
+    - to:
+        - ipBlock:
+            cidr: 192.168.0.107/32
+      ports:
+        - protocol: TCP
+          port: 443
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: object-storage-allow-blackbox-console-health
+  namespace: object-storage
+  labels:
+    app.kubernetes.io/part-of: platform
+    app.kubernetes.io/component: admin-console
+spec:
+  podSelector:
+    matchLabels:
+      aistor.min.io/objectStore: minio-aistor
+  policyTypes:
+    - Ingress
+  ingress:
+    - from:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: observability
+          podSelector:
+            matchLabels:
+              app.kubernetes.io/name: prometheus-blackbox-exporter
+              app.kubernetes.io/instance: blackbox-exporter
+      ports:
+        - protocol: TCP
+          port: 9090
+---
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: object-storage-allow-prometheus-metrics
+  namespace: object-storage
+  labels:
+    app.kubernetes.io/part-of: platform
+    app.kubernetes.io/component: object-storage-metrics
+spec:
+  podSelector:
+    matchLabels:
+      aistor.min.io/objectStore: minio-aistor
+  policyTypes:
+    - Ingress
+  ingress:
+    - from:
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: observability
+          podSelector:
+            matchLabels:
+              app.kubernetes.io/name: prometheus
+              app.kubernetes.io/instance: observability-core-kube-pr-prometheus
+      ports:
+        - protocol: TCP
+          port: 9000
@@ -0,0 +1,32 @@
+apiVersion: aistor.min.io/v1
+kind: ObjectStore
+metadata:
+  name: minio-aistor
+spec:
+  env:
+    - name: MINIO_PROMETHEUS_AUTH_TYPE
+      value: public
+    - name: MINIO_BROWSER_REDIRECT_URL
+      value: https://storage-admin.learn.hyeonworks.com
+    - name: MINIO_IDENTITY_OPENID_CONFIG_URL
+      value: https://id.learn.hyeonworks.com/realms/hyeonworks/.well-known/openid-configuration
+    - name: MINIO_IDENTITY_OPENID_CLIENT_ID
+      valueFrom:
+        secretKeyRef:
+          name: aistor-keycloak-oidc
+          key: client-id
+    - name: MINIO_IDENTITY_OPENID_CLIENT_SECRET
+      valueFrom:
+        secretKeyRef:
+          name: aistor-keycloak-oidc
+          key: client-secret
+    - name: MINIO_IDENTITY_OPENID_SCOPES
+      value: openid,profile,email
+    - name: MINIO_IDENTITY_OPENID_DISPLAY_NAME
+      value: Hyeonworks ID
+    - name: MINIO_IDENTITY_OPENID_CLAIM_NAME
+      value: policy
+    - name: MINIO_IDENTITY_OPENID_REDIRECT_URI_DYNAMIC
+      value: "on"
+    - name: MINIO_BROWSER_SESSION_DURATION
+      value: 8h
@@ -0,0 +1,67 @@
+secrets:
+  name: aistor-root-configuration
+  existingSecret: true
+  disableRoot: false
+
+objectStore:
+  name: minio-aistor
+
+  configuration:
+    name: aistor-root-configuration
+
+  pools:
+    - name: pool-0
+      servers: 1
+      volumesPerServer: 1
+      storageNamePrefix: data
+      size: 900Gi
+      storageClassName: aistor-local-xfs-retain
+      nodeSelector:
+        kubernetes.io/hostname: donghyeon-system-product-name
+      resources:
+        requests:
+          cpu: 500m
+          memory: 2Gi
+        limits:
+          cpu: "3"
+          memory: 8Gi
+      securityContext:
+        runAsUser: 1000
+        runAsGroup: 1000
+        runAsNonRoot: true
+        fsGroup: 1000
+        fsGroupChangePolicy: OnRootMismatch
+      containerSecurityContext:
+        runAsUser: 1000
+        runAsGroup: 1000
+        runAsNonRoot: true
+        allowPrivilegeEscalation: false
+        capabilities:
+          drop:
+            - ALL
+        seccompProfile:
+          type: RuntimeDefault
+
+  mountPath: /export
+  subPath: /data
+
+  certificates:
+    disableAutoCert: true
+
+  services:
+    minio:
+      serviceType: ClusterIP
+    console:
+      serviceType: ClusterIP
+
+  pvcProtection: true
+  prometheusOperator: false
+
+  sideCars:
+    resources:
+      requests:
+        cpu: 100m
+        memory: 128Mi
+      limits:
+        cpu: 500m
+        memory: 512Mi