本文共 5079 字，大约阅读时间需要 16 分钟。

参考文档

架构

注意事项

1. 我并不打算使用kubeadm-ha中提供的nginx来代理master-API访问地址

概述

1. debian9.8 + docker 18.06 + kubernetes 1.14.0
2. 三台主节点+两台node节点
3. 负载使用keepalived，或域名解析（内网路由器设置，client机器的hosts解析）
4. 准备测试并对比各种监控软件

节点列表

IP	主机名或域名	作用
192.168.134.131	master-vip	keepalived-VIP
192.168.134.132	master1	master节点1
192.168.134.133	master2	master节点2
192.168.134.134	master3	master节点3
192.168.134.135	node1	node节点1
192.168.134.136	node2	node节点2

master节点准备

环境准备1(基础环境+docker)

1. 防火墙，swap，selinux修改

echo -e "net.bridge.bridge-nf-call-ip6tables =1\nnet.bridge.bridge-nf-call-iptables =1\nnet.ipv4.ip_forward = 1" >> /etc/sysctl.conf;sysctl -p;swapoff -a;sed -ri "/swap/s@(.*)@#/&@g" /etc/fstab;echo "SELINUX=disabled" > /etc/selinux/config

2. 时间同步

apt -y install ntpdateecho "  */5  *  *  *  *   /usr/sbin/ntpdate  ntp.sjtu.edu.cn" >> /var/spool/cron/crontabs/root

3. 安装环境配置

apt update && apt-get -y  install apt-transport-https  ca-certificates  curl  gnupg2 software-properties-common && curl -fsSL https://download.docker.com/linux/debian/gpg |  apt-key add - && add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian  $(lsb_release -cs)  stable" && apt-get update

4. 查看版本，安装docker

apt-cache madison docker-ceapt install docker-ce=18.06.2~ce~3-0~debian

5. docker镜像加速

echo '{"registry-mirrors": ["http://*******.daocloud.io"]}' >> /etc/docker/daemon.jsonsystemctl restart docker.servicesystemctl enable docker

环境准备2(kube组件+image下载)

#install kubectl kubelet kubeadm apt-get update && \apt-get install -y apt-transport-https curl && \curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg |apt-key add -  && \echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main"  >>/etc/apt/sources.list.d/kubernetes.list  && \apt update #查看版本，安装kubelet,kubeadm,kubectlapt-cache madison kubeadmapt-get install -y kubelet=1.14.0-00  kubeadm=1.14.0-00 kubectl=1.14.0-00

下载需要的images

1. 列出master需要的image

kubeadm --kubernetes-version=v1.14.0 config images list

#如果没有dokcer镜像加速，需要使用下列脚本下载后重命名tagimages=(    kube-apiserver:v1.14.0    kube-controller-manager:v1.14.0    kube-scheduler:v1.14.0    kube-proxy:v1.14.0    pause:3.1    etcd:3.3.10    coredns:1.3.1)for imageName in ${images[@]} ; do    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName        docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageNamedone

配置kubelet

docker info | grep -i cgroupecho 'KUBELET_EXTRA_ARGS="--cgroup-driver=cgroupfs --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause:3.1"' >/etc/default/kubelet

kubeadm-ha相关准备

master节点之间ssh互相免密登陆(包括自己)

#开启节点的root用户免密登陆功能，并互相无密登陆vim /etc/ssh/sshd_configPermitRootLogin yesPermitEmptyPasswords yes#无密登陆太简单，不写

编辑各节点的/etc/hosts文件

#添加下列行192.168.134.131         master-vip192.168.134.132         master1192.168.134.133         master2192.168.134.134         master3

kubeadm-ha相关下载&配置(仅在master1上执行)

下载 kubeadm-ha组件

git clone https://github.com/cookeem/kubeadm-ha

下载keepalived

apt install -y keepalived

#!/bin/bash#以下为修改的值export K8SHA_VIP=192.168.134.131export K8SHA_IP1=192.168.134.132export K8SHA_IP2=192.168.134.133export K8SHA_IP3=192.168.134.134export K8SHA_VHOST=master-vipexport K8SHA_HOST1=master1export K8SHA_HOST2=master2export K8SHA_HOST3=master3export K8SHA_NETINF1=ens33export K8SHA_NETINF2=ens33export K8SHA_NETINF3=ens33export K8SHA_KEEPALIVED_AUTH=412f7dc3bfed32194d1600c483e10ad1d#K8SHA_CALICO_REACHABLE_IP值为某一个节点的ip即可export K8SHA_CALICO_REACHABLE_IP=192.168.134.133export K8SHA_CIDR=10.96.0.0

执行create-config.sh会自动生成keepalived、calico对应的配置

bash -x  ./create-config.sh

先start master1节点的keepalived服务，确定master1中VIP已生效，再启动master2和master3中的keepalived服务

systemctl start keepalivedsystemctl enable keepalived

kubeadm init&calico安装&集群搭建

kubeadm init 初始化 master1节点

在master1节点中执行

#保存输出，后续添加节点使用kubeadm init --config=/root/kubeadm-ha/kubeadm-config.yaml  --experimental-upload-certs  --ignore-preflight-errors=all  &> /root/kubeadm-init-output-1

配置master1节点的kubectl的配置

#对应命令在init的输出中可见mkdir -p $HOME/.kubecp -i /etc/kubernetes/admin.conf $HOME/.kube/configchown $(id -u):$(id -g) $HOME/.kube/config

pull calico镜像，安装calico

calico 镜像每个master节点都需要pull

docker pull calico/cni:v3.6.0docker pull calico/node:v3.6.0docker pull calico/kube-controllers:v3.6.0

master1节点上安装calico

kubectl apply -f /root/kubeadm-ha/calico/calico.yaml

将另外两个master节点加入集群

确认coredns-pod状态为running

kubectl get pods --all-namespaces

添加其余master节点到集群

#对应命令在init的输出中可见#You can now join any number of the control-plane node running the following command on each as root:  kubeadm join 192.168.134.131:6443 --token px8o5t.cb3duj0uza8i7jrv \    --discovery-token-ca-cert-hash sha256:6c314cc87bdec72d7bae102678af98f8fa33cad06d36c0395ad588f2b816d630 \    --experimental-control-plane --certificate-key 8d1c74bf831e1579d7c1b4fd1d2bfc0e83dbebaeb53e1b39e240e9b3afad9e33    --ignore-preflight-errors=all

确认master多主集群状态正常

执行get node命令，输出正常即可

kubectl get node

#输出信息示例NAME      STATUS   ROLES    AGE    VERSIONmaster1   Ready    master   149m   v1.14.0master2   Ready    master   97m    v1.14.0master3   Ready    master   95m    v1.14.0

转载地址：http://nwili.baihongyu.com/