1. 项目背景与核心挑战在传统企业IT架构向云原生转型的过程中Kubernetesk8s已成为事实上的容器编排标准。然而在实际生产环境中由于安全合规要求大量企业需要在内网隔离环境下部署云原生应用套件。本次要解决的正是这样一个典型场景在完全离线的k8s集群中部署DolphinScheduler工作流调度系统和SeaTunnel数据集成工具。这个组合方案特别适合需要处理以下需求的企业存在严格网络安全隔离要求的金融、政务等行业需要自动化调度异构数据源ETL流程的场景希望利用声明式配置管理整个数据流水线的团队关键难点提示离线环境意味着所有依赖镜像、charts包、系统组件都需要预先下载并完成内部仓库的搭建任何遗漏都可能导致部署过程中断。2. 离线环境准备工作2.1 基础设施规划建议建议采用以下节点规格作为基准可根据实际负载调整节点类型CPU内存磁盘数量Master4核16G100G3Worker8核32G200G至少2网络方面需要确保所有节点间网络延迟2ms节点间带宽≥1Gbps提前规划好Pod CIDR和Service CIDR2.2 离线资源打包清单需要预先准备的离线资源包括基础组件包k8s各节点所需系统依赖conntrack、socat等容器运行时推荐containerd 1.6Helm 3.10二进制文件镜像仓库方案# 搭建Harbor私有仓库示例 helm repo add harbor https://helm.goharbor.io helm fetch harbor/harbor --version 1.10.0应用镜像清单DolphinScheduler 3.2.1全套镜像含web、api、worker等SeaTunnel 2.3.2引擎镜像相关中间件镜像ZooKeeper、MySQL等Helm charts包helm repo add dolphinscheduler https://dolphinscheduler.apache.org helm pull dolphinscheduler/dolphinscheduler --version 3.2.13. 核心组件部署实战3.1 DolphinScheduler部署要点3.1.1 数据库初始化建议使用独立的MySQL实例非k8s内部署CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8mb4; CREATE USER ds_user% IDENTIFIED BY StrongPass123; GRANT ALL PRIVILEGES ON dolphinscheduler.* TO ds_user%;3.1.2 Helm定制化配置关键values.yaml配置项postgresql: enabled: false # 禁用内置PG externalDatabase: type: mysql host: mysql.internal port: 3306 username: ds_user password: StrongPass123 registry: type: harbor url: harbor.internal repository: library/dolphinscheduler部署命令helm install dolphinscheduler ./dolphinscheduler-3.2.1.tgz \ -n ds --create-namespace \ -f values.yaml3.2 SeaTunnel集成方案3.2.1 引擎部署模式选择推荐使用Spark on k8s模式spark: master: k8s://https://kubernetes.default.svc deployMode: cluster image: harbor.internal/library/seatunnel-spark:2.3.23.2.2 任务配置示例典型JDBC源到HDFS的作业配置env { execution.parallelism 3 } source { JdbcSource { url jdbc:mysql://mysql.internal:3306/source_db username etl_user password EtlPass456 query SELECT * FROM orders WHERE update_time ${last_update} } } transform { Sql { query SELECT user_id, SUM(amount) AS total FROM orders GROUP BY user_id } } sink { Hdfs { path hdfs://namenode:8020/data/orders_agg file_format parquet } }4. 生产环境调优指南4.1 性能关键参数DolphinScheduler工作节点配置建议worker: resources: limits: cpu: 2 memory: 4Gi requests: cpu: 1 memory: 2Gi env: - name: TASK_EXECUTE_THREADS value: 10 # 根据CPU核数调整4.2 高可用保障措施为关键组件配置Pod反亲和性affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app.kubernetes.io/component operator: In values: [master-server] topologyKey: kubernetes.io/hostname配置合理的健康检查livenessProbe: httpGet: path: /actuator/health port: 12345 initialDelaySeconds: 30 periodSeconds: 105. 典型问题排查手册5.1 镜像拉取失败现象Pod状态为ImagePullBackOff 解决方案确认harbor证书已添加到各个节点mkdir -p /etc/docker/certs.d/harbor.internal cp harbor-ca.crt /etc/docker/certs.d/harbor.internal/ca.crt检查secret配置是否正确kubectl create secret docker-registry harbor-secret \ --docker-serverharbor.internal \ --docker-usernameadmin \ --docker-passwordHarbor12345 \ -n ds5.2 权限相关问题现象SeaTunnel任务报Permission denied 处理方法为Spark Driver配置合适的安全上下文securityContext: runAsUser: 1000 fsGroup: 1000在HDFS端配置ACLhdfs dfs -setfacl -R -m user:spark:rwx /data6. 监控与运维建议6.1 指标采集方案推荐使用Prometheus Operator采集指标DolphinScheduler暴露的指标端点metrics: enabled: true port: 12345对应的ServiceMonitor配置apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: labels: release: prometheus name: dolphinscheduler-monitor spec: endpoints: - port: metrics selector: matchLabels: app.kubernetes.io/instance: dolphinscheduler6.2 日志收集策略建议采用Filebeat ELK方案filebeatConfig: filebeat.yml: | filebeat.inputs: - type: container paths: - /var/log/containers/*.log output.elasticsearch: hosts: [elasticsearch.internal:9200]
)
