一、环境信息组件版本OS银河麒麟 V10 (ky10), kernel 4.19.90-89.32.v2401K8sv1.32.13Docker26.0.0cri-dockerdv0.3.16CNIv1.6.0二、前置准备所有节点# 关闭防火墙和 SELinuxsystemctl stop firewalldsystemctl disable firewalld setenforce0sed-is/SELINUXenforcing/SELINUXdisabled/g/etc/selinux/config# 关闭 swapswapoff-ased-is/.*swap.*/#//etc/fstab# 配置 hostscat/etc/hostsEOF 192.168.25.10 master1 192.168.25.11 worker1 192.168.25.12 worker2 EOF# 配置内核参数cat/etc/sysctl.d/k8s.confEOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 EOFsysctl--system# 加载模块modprobe br_netfilterechobr_netfilter/etc/modules-load.d/br_netfilter.conf三、安装 Docker所有节点yuminstall-ydocker-ce docker-ce-cli containerd.io systemctlenable--nowdocker四、安装 cri-dockerd所有节点cd/usr/local/bin# 下载wgethttps://github.com/Mirantis/cri-dockerd/releases/download/v0.3.16/cri-dockerd-0.3.16.amd64.tgztar-xzfcri-dockerd-0.3.16.amd64.tgzmvcri-dockerd/cri-dockerd /usr/local/bin/cri-dockerdchmodx /usr/local/bin/cri-dockerd# 创建 systemd 服务cat/etc/systemd/system/cri-docker.serviceEOF [Unit] DescriptionCRI Interface for Docker Application Container Engine Afternetwork.target docker.service Requiresdocker.service [Service] Typesimple ExecStart/usr/local/bin/cri-dockerd --container-runtime-endpoint unix:///run/cri-dockerd.sock --network-plugincni --cni-bin-dir/opt/cni/bin --cni-conf-dir/etc/cni/net.d --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.10 ExecReload/bin/kill -s HUP $MAINPID Restartalways RestartSec2 [Install] WantedBymulti-user.target EOFsystemctl daemon-reload systemctlenable--nowcri-docker.service五、安装 K8s 二进制所有节点cd/usr/local/bin# 下载wgethttps://dl.k8s.io/v1.32.13/bin/linux/amd64/kubeletwgethttps://dl.k8s.io/v1.32.13/bin/linux/amd64/kubeadmwgethttps://dl.k8s.io/v1.32.13/bin/linux/amd64/kubectlchmodx kubelet kubeadm kubectl# CNI 插件mkdir-p/opt/cni/bincd/opt/cni/binwgethttps://github.com/containernetworking/plugins/releases/download/v1.6.0/cni-plugins-linux-amd64-v1.6.0.tgztar-xzfcni-plugins-linux-amd64-v1.6.0.tgzrm-fcni-plugins-linux-amd64-v1.6.0.tgz六、配置 kubelet所有节点# 创建 kubelet 服务cat/etc/systemd/system/kubelet.serviceEOF [Unit] DescriptionKubernetes Kubelet Documentationhttps://kubernetes.io/docs Afterdocker.service cri-docker.service Requiresdocker.service cri-docker.service [Service] EnvironmentFile/var/lib/kubelet/kubeadm-flags.env ExecStart/usr/local/bin/kubelet --config/var/lib/kubelet/config.yaml $KUBELET_KUBEADM_ARGS Restartalways StartLimitInterval0 RestartSec10 [Install] WantedBymulti-user.target EOFsystemctl daemon-reload七、Master 节点初始化kubeadm init\--apiserver-advertise-address192.168.25.10\--apiserver-bind-port6443\--kubernetes-versionv1.32.13\--pod-network-cidr10.244.0.0/16\--service-cidr10.96.0.0/12\--image-repositoryregistry.aliyuncs.com/google_containers\--cri-socketunix:///run/cri-dockerd.sock初始化后修复 kubelet 配置# 修复 containerRuntimeEndpointsed-is|containerRuntimeEndpoint: |containerRuntimeEndpoint: unix:///run/cri-dockerd.sock|/var/lib/kubelet/config.yaml# 确保 kubeadm-flags.env 包含 kubeconfigechoKUBELET_KUBEADM_ARGS--kubeconfig/etc/kubernetes/kubelet.conf --container-runtime-endpointunix:///run/cri-dockerd.sock --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.10/var/lib/kubelet/kubeadm-flags.env systemctl restart kubelet配置 kubectlmkdir-p$HOME/.kubecp-i/etc/kubernetes/admin.conf$HOME/.kube/configchown$(id-u):$(id-g)$HOME/.kube/config安装网络插件Calico# 下载 Calicowgethttps://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml# 修改镜像为国内可访问的源sed-is|docker.io/calico/|docker.m.daocloud.io/calico/|gcalico.yaml# 在所有节点手动拉取镜像dockerpull docker.m.daocloud.io/calico/cni:v3.27.0dockerpull docker.m.daocloud.io/calico/node:v3.27.0dockerpull docker.m.daocloud.io/calico/kube-controllers:v3.27.0# 打标签dockertag docker.m.daocloud.io/calico/cni:v3.27.0 docker.io/calico-cni:v3.27.0dockertag docker.m.daocloud.io/calico/node:v3.27.0 docker.io/calico-node:v3.27.0dockertag docker.m.daocloud.io/calico/kube-controllers:v3.27.0 docker.io/calico-kube-controllers:v3.27.0# 应用kubectl apply-fcalico.yaml八、Worker 节点加入生成 join 命令在 master1 上kubeadm token create --print-join-commandWorker 节点执行kubeadmjoin192.168.25.10:6443--tokentoken\--discovery-token-ca-cert-hash sha256:hash\--cri-socketunix:///run/cri-dockerd.sockWorker 节点修复 kubelet 配置# 修复 containerRuntimeEndpointsed-is|containerRuntimeEndpoint: |containerRuntimeEndpoint: unix:///run/cri-dockerd.sock|/var/lib/kubelet/config.yaml# 添加 bootstrap 配置到 kubeadm-flags.envcat/var/lib/kubelet/kubeadm-flags.envEOF KUBELET_KUBEADM_ARGS--bootstrap-kubeconfig/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig/etc/kubernetes/kubelet.conf --container-runtime-endpointunix:///run/cri-dockerd.sock --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.10 EOFsystemctl restart kubelet九、验证命令# 查看节点kubectl get nodes# 查看系统 Podkubectl get pods-nkube-system# 查看服务状态systemctl statusdockersystemctl status cri-docker.service systemctl status kubelet kubectl create deployment nginx--imagenginx kubectl expose deployment nginx--port80--typeNodePort kubectl get svc nginxdockerpull docker.m.daocloud.io/library/nginx:latestdockertag docker.m.daocloud.io/library/nginx:latest nginx kubectl get svc nginx kubectl get pods-owidecurlhttp://192.168.25.10:node-port十、遇到的问题及解决方案问题1cri-dockerd RPM 包依赖失败现象错误依赖检测失败 container-selinux 2:2.74 containerd.io 1.2.2-3 libc.so.6(GLIBC_2.32)(64bit) libc.so.6(GLIBC_2.34)(64bit)原因Fedora 35 的 RPM 包与银河麒麟 V10 的 glibc 版本不兼容。解决使用通用二进制安装 cri-dockerd不用 RPM 包。问题2kubelet 连接 containerd 而不是 cri-dockerd现象dial unix /run/containerd/containerd.sock: connect: no such file or directory原因kubeadm init/join 生成的 kubelet 配置中containerRuntimeEndpoint为空kubelet 默认连接 containerd。解决每次 kubeadm init/join 后修复 config.yamlsed-is|containerRuntimeEndpoint: |containerRuntimeEndpoint: unix:///run/cri-dockerd.sock|/var/lib/kubelet/config.yaml问题3kubelet 启动失败 - webhook authentication现象failed to run Kubelet: no client provided, cannot use webhook authentication原因kubeadm 生成的 config.yaml 缺少 kubeconfig 配置kubelet 无法连接 API server。解决在 kubeadm-flags.env 中指定 kubeconfigechoKUBELET_KUBEADM_ARGS--kubeconfig/etc/kubernetes/kubelet.conf --container-runtime-endpointunix:///run/cri-dockerd.sock/var/lib/kubelet/kubeadm-flags.env问题4worker 节点 join 时端口被占用现象[ERROR Port-10250]: Port 10250 is in use原因之前失败的 kubelet 进程还在运行。解决pkill-9kubelet# 或者systemctl stop kubelet问题5cgroup driver 不匹配核心问题现象Error response from daemon: cgroup-parent for systemd cgroup should be a valid slice named as xxx.slice原因kubelet config 中cgroupDriver: systemd但 cri-dockerd 0.3.16 与 Docker 的 systemd cgroup 驱动有兼容性问题cri-dockerd 传递给 Docker 的 cgroup-parent 格式不正确解决改用 cgroupfs 驱动# 1. 删除 Docker 的 systemd 配置rm-f/etc/docker/daemon.json# 2. 重启 Dockersystemctl restartdocker# 3. 确认是 cgroupfsdockerinfo|grep-iCgroup Driver# 应显示Cgroup Driver: cgroupfs# 4. 修改 kubelet configsed-is|cgroupDriver: systemd|cgroupDriver: cgroupfs|/var/lib/kubelet/config.yaml# 5. 重启服务systemctl restart cri-docker.service systemctl restart kubelet问题6cri-dockerd 不支持 --cgroup-parent 参数现象unknown flag: --cgroup-parent原因cri-dockerd 0.3.16 版本不支持--cgroup-parent启动参数。解决从 service 文件中删除该参数cat/etc/systemd/system/cri-docker.serviceEOF [Unit] DescriptionCRI Interface for Docker Application Container Engine Afternetwork.target docker.service Requiresdocker.service [Service] Typesimple ExecStart/usr/local/bin/cri-dockerd --container-runtime-endpoint unix:///run/cri-dockerd.sock --network-plugincni --cni-bin-dir/opt/cni/bin --cni-conf-dir/etc/cni/net.d --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.10 ExecReload/bin/kill -s HUP $MAINPID Restartalways RestartSec2 [Install] WantedBymulti-user.target EOFsystemctl daemon-reload systemctl restart cri-docker.service问题7Calico 镜像拉取失败现象Failed to pull image docker.io/calico-cni:v3.27.0: ... request canceled while waiting for connection原因国内网络访问 docker.io 超时或被墙。解决使用国内镜像代理如 daoCloud# 修改 calico.yamlsed-is|docker.io/calico/|docker.m.daocloud.io/calico/|gcalico.yaml# 或手动拉取并打标签dockerpull docker.m.daocloud.io/calico/cni:v3.27.0dockertag docker.m.daocloud.io/calico/cni:v3.27.0 docker.io/calico-cni:v3.27.0问题8Calico 和 Flannel 冲突现象Calico Pod 反复重启网络不通。原因之前安装了 Flannel又安装 Calico两个 CNI 插件冲突。解决删除 Flannel只保留 Calicokubectl delete-fkube-flannel.yml kubectl apply-fcalico.yaml十一、关键总结关键点说明二进制安装银河麒麟 V10 建议用二进制安装 K8s 组件避免 RPM 依赖问题kubeadm 会覆盖配置每次 init/join 后必须修复 containerRuntimeEndpointcgroup 驱动cri-dockerd 0.3.16 麒麟 V10 建议用 cgroupfs不要用 systemd镜像源国内环境提前准备镜像或使用代理daoCloud、阿里云等service 文件cri-dockerd 0.3.16 不支持--cgroup-parent参数