Kubernetes Kubeadm 安装

规划

IP 地址 主机名 角色 说明
192.168.1.39 k8s-master-lb, k8s-master-lb.jz-sz.com 负载均衡 虚拟 IP
192.168.1.40 k8s-master01, k8s-master01.jz-sz.com master master 节点
192.168.1.41 k8s-master02, k8s-master02.jz-sz.com master master 节点
192.168.1.42 k8s-master03, k8s-master03.jz-sz.com master master 节点
192.168.1.46 k8s-node01, k8s-node01.jz-sz.com node node 节点
192.168.1.47 k8s-node02, k8s-node02.jz-sz.com node node 节点
192.168.1.48 k8s-node03, k8s-node03.jz-sz.com node node 节点
192.168.1.49 k8s-node04, k8s-node04.jz-sz.com node node 节点

调整系统参数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
cat > /etc/sysctl.d/k8s.conf << EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory = 1
vm.panic_on_oom = 0
fs.inotify.max_user_watches = 89100
fs.file-max=52706963
fs.nr_open = 52706963
net.netfilter.nf_conntrack_max=2310720


net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_orphans = 327680
net.ipv4.tcp_orphans_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_timestamps = 0
net.core.somaxconn = 16384
EOF
# 立即生效
sysctl --system

安装 docker

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
yum install -y yum-utils
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache fast
yum list docker-ce --showduplicates
yum install -y docker-ce-18.09.9-3.el7 docker-ce-cli-18.09.9-3.el7
systemctl enable --now docker
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "insecure-registries": ["192.168.1.138:5000"],
  "registry-mirrors": [
    "https://3laho3y3.mirror.aliyuncs.com"
   ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
systemctl restart docker
# 测试
docker run hello-world

安装基础组件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
vim /etc/yum.repos.d/kubernetes.repo
cat << 'EOF' > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enable=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
              https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF


yum list kubeadm --showduplicates

版本太新，阿里云镜像不支持

1
2
yum install -y kubelet-1.16.6-0 kubeadm-1.16.6-0 kubectl-1.16.6-0
systemctl enable --now kubelet

检查初始化需要的镜像

1
2
3
4
5
6
7
8
kubeadm config images list
k8s.gcr.io/kube-apiserver:v1.16.6
k8s.gcr.io/kube-controller-manager:v1.16.6
k8s.gcr.io/kube-scheduler:v1.16.6
k8s.gcr.io/kube-proxy:v1.16.6
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2

使用阿里云并标记

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/{镜像名称}:{版本}
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/{镜像名称}:{版本} k8s.gcr.io/{镜像名称}:{版本}
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.16.6
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.16.6
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.16.6
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.16.6
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.3.15-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.2


docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.16.6 k8s.gcr.io/kube-apiserver:v1.16.6
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.16.6 k8s.gcr.io/kube-controller-manager:v1.16.6
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.16.6 k8s.gcr.io/kube-scheduler:v1.16.6
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.16.6 k8s.gcr.io/kube-proxy:v1.16.6
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1 k8s.gcr.io/pause:3.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.3.15-0 k8s.gcr.io/etcd:3.3.15-0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.2 k8s.gcr.io/coredns:1.6.2


docker pull quay.mirrors.ustc.edu.cn/coreos/flannel:v0.12.0-amd64
docker tag quay.mirrors.ustc.edu.cn/coreos/flannel:v0.12.0-amd64 quay.io/coreos/flannel:v0.12.0-amd64

初始化 master

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# 添加地址，方便后续高可用，阿里云或其它云请直接使用slb
ip addr add 192.168.122.200/32 dev eth0
# 可选，生成初始化yaml
kubeadm config print init-defaults > init-defaults.yaml
kubeadm init --kubernetes-version=1.16.6 \
--pod-network-cidr 10.244.0.0/16 \
--service-cidr 172.21.0.0/20 \
--apiserver-advertise-address=0.0.0.0 \
--control-plane-endpoint "192.168.122.200:6443" \
--upload-certs \
--ignore-preflight-errors=swap
# 失败重置
kubeadm reset
rm -rf $HOME/.kube/config
# 普通用户使用 kubectl
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 也可以使用环境变量
scp k8s-master:/etc/kubernetes/admin.conf /etc/kubernetes/
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source .bash_profile

配置网络，使用 flannel

1
2
3
4
5
6
7
8
9
10
11
12
13
# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
wget https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml
# 替换为对应网段
net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
docker pull quay.mirrors.ustc.edu.cn/coreos/flannel:v0.12.0-amd64
docker tag quay.mirrors.ustc.edu.cn/coreos/flannel:v0.12.0-amd64 quay.io/coreos/flannel:v0.12.0-amd64
kubectl apply -f kube-flannel.yml

查看集群状态

1
2
3
kubectl get nodes
# 检查集群配置
kubectl -n kube-system get cm kubeadm-config -o yaml

node 拉取镜像

1
2
3
4
5
6
7
8
9
10
11
12
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.16.6
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.2


docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.16.6 k8s.gcr.io/kube-proxy:v1.16.6
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1 k8s.gcr.io/pause:3.1
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.2 k8s.gcr.io/coredns:1.6.2


docker pull quay.mirrors.ustc.edu.cn/coreos/flannel:v0.12.0-amd64
docker tag quay.mirrors.ustc.edu.cn/coreos/flannel:v0.12.0-amd64 quay.io/coreos/flannel:v0.12.0-amd64

加入集群

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# 添加 master
kubeadm init phase upload-certs --upload-certs
kubeadm token create --print-join-command
# 合并
kubeadm join 192.168.122.252:6443 --token qmihb0.2j6ikktkew8ch3vz     --discovery-token-ca-cert-hash sha256:db586028c918052bfb1b657472a4bee3d114de93b809b5ff8b53b3b7bded665a \
--control-plane --certificate-key 0b84889916022953e6777da8babe9b643131d104a1d7f91db1a6ae7ddc60d18b
# 添加 node
kubeadm token create --print-join-command
kubeadm join 192.168.122.252:6443 --token uopfdg.haazo24wvd8qxxxd \
    --discovery-token-ca-cert-hash sha256:db586028c918052bfb1b657472a4bee3d114de93b809b5ff8b53b3b7bded665a
# 删除节点
kubectl drain NODE_ID --delete-local-data --force --ignore-daemonsets
kubectl delete node NODE_ID
kubeadm reset
1
2
3
4
5
6
7
#  查看集群版本信息
kubectl version --short=true
kubectl cluster-info
# 查看所有pod 状态
kubectl get pods --namespace=kube-system
# 查看 pod 状态
kubectl describe pod kub-proxy-t64ab --namespace=kube-system

安装 helm

1
2
3
helm version
docker pull registry.cn-hangzhou.aliyuncs.com/kubernetes-helm/tiller:1.6.2
docker tag quay.mirrors.ustc.edu.cn/coreos/flannel:v0.12.0-amd64 quay.io/coreos/flannel:v0.12.0-amd64

遇到问题

1
2
3
4
5
6
7
8
docker pull quay.mirrors.ustc.edu.cn quay 镜像错误：net/http: TLS handshake timeout ，将https://quay.mirrors.ustc.edu.cn" 加入/etc/docker/daemon.json register-mirrors
cgdriver 不一致导致不能启动问题
cat /etc/docker/daemon.json
cat /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS=--cgroup-driver=systemd
重启 pod
kubectl get pod {podname} -n {namespace} -o yaml | kubectl replace --force -f -
kubectl delete pod -n {namespace} {podname}s
运维部署
#Kubernetes #K8S #容器
Kubernetes Kubeadm 安装
http://relengxing.tech/2021/06/06/kubernetes kubeadm 安装/
作者
relengxing
发布于
2021年6月6日
许可协议