logo头像
Snippet 博客主题

230_k8s-install

Minikube安装单节点kubernetes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
#安装依赖
wget https://download.docker.com/linux/centos/7/x86_64/edge/Packages/containerd.io-1.2.6-3.3.el7.x86_64.rpm
yum -y install containerd.io-1.2.6-3.3.el7.x86_64.rpm

#安装docker-ce
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
systemctl start docker && systemctl enable docker

#切换阿里的registry镜像源
vi /etc/docker/daemon.json
{
"registry-mirrors": ["https://zhfojaep.mirror.aliyuncs.com"]
}
systemctl daemon-reload
systemctl restart docker

#下载kubectl
wget "https://storage.googleapis.com/kubernetes-release/release/v1.17.3/bin/linux/amd64/kubectl" -O "/usr/local/bin/kubectl"
chmod +x /usr/local/bin/kubectl

#下载minikube
wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 -O "/usr/local/bin/minikube"
chmod +x /usr/local/bin/minikube

#启动
minikube start --vm-driver=none --registry-mirror=https://registry.docker-cn.com --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers

#开机自启
systemctl enable docker
systemctl enable kubelet

#启用ingress
minikube addons enable ingress

#启用dashboard
minikube dashboard

kubeadm安装kubernetes集群

1、简介

本部分内容转载自 http://www.mydlq.club/article/4/#wow23

1)环境

软件 版本
CentOS 7.6
Docker 18.06.1-ce
Kubernetes 1.18.2
Kuberadm 1.18.2
Kuberlet 1.18.2
Kuberctl 1.18.2

2)集群架构

集群架构

2、前期准备(所有机器)

1)机器分配

ip 主机名 CPU & 内存 角色
172.16.0.100 icql-vip 1C & 1G vip
172.16.0.101 icql-master1 2C & 2G master
172.16.0.102 icql-master2 2C & 2G master
172.16.0.103 icql-master3 2C & 2G master
172.16.0.201 icql-node1 2C & 2G node
172.16.0.202 icql-node2 2C & 2G node

2)机器端口

  • master节点
协议 端口 使用者
TCP 6443 k8s api
TCP 2379-2380 etcd server
TCP 10250 kubelet api
TCP 10251 kube-scheduler
TCP 10252 kube-controller-manager
  • node节点
协议 端口 使用者
TCP 10250 kubelet api
TCP 30000-32767 NodePort Services

3)环境设置

  • 修改主机名

    1
    2
    # 具体参考机器分配
    hostnamectl set-hostname icql-vip
  • 主机名称解析

    1
    2
    3
    4
    5
    6
    7
    8
    vi /etc/hosts

    172.16.0.100 vip.k8s.icql.work icql-vip
    172.16.0.101 master1.k8s.icql.work icql-master1
    172.16.0.102 master2.k8s.icql.work icql-master2
    172.16.0.103 master3.k8s.icql.work icql-master3
    172.16.0.201 node1.k8s.icql.work icql-node1
    172.16.0.202 node2.k8s.icql.work icql-node2
  • 主机时间同步

    1
    systemctl start chronyd.service && systemctl enable chronyd.service
  • 关闭防火墙

    1
    systemctl stop firewalld && systemctl disable firewalld
  • 关闭并禁用SELinux

    1
    2
    3
    4
    5
    6
    # 关闭selinux
    setenforce 0
    # 编辑/etc/sysconfig selinux 文件,以彻底禁用 SELinux
    sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config
    # 查看selinux状态
    getenforce
  • 禁用 Swap 设备

    1
    swapoff -a && sed -ri 's/.*swap.*/#&/' /etc/fstab
  • 允许路由转发

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    vi /etc/sysctl.d/k8s.conf

    net.ipv4.ip_forward = 1
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1

    # 挂载br_netfilter
    modprobe br_netfilter

    # 使配置生效
    sysctl --system
  • 调整资源限制

    1
    2
    3
    4
    5
    6
    7
    8
    9
    vi /etc/security/limits.conf

    # 删除 End of file 行后的所有东西,加入以下内容
    * soft nofile 65536
    * hard nofile 65536
    * soft nproc 65536
    * hard nproc 65536
    * soft memlock unlimited
    * hard memlock unlimited

3、master高可用方案:Keepalived + haproxy(所有master机器)

待补充

4、安装docker(所有机器)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# 获取docker-ce.repo
wget http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O "/etc/yum.repos.d/docker-ce.repo"

# 更新yum源并安装
yum update && yum install \
containerd.io-1.2.10 \
docker-ce-19.03.4 \
docker-ce-cli-19.03.4

# 启动docker
systemctl start docker

# docker推荐设置,切换阿里的registry镜像源
vi /etc/docker/daemon.json

{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": ["https://zhfojaep.mirror.aliyuncs.com"]
}

# 启动docker
systemctl start docker && systemctl enable docker

# 设置镜像存储目录
vi /lib/systemd/system/docker.service

#找到 ExecStart 这行后面加上存储目录,例如这里是 --graph /data/docker
ExecStart=/usr/bin/dockerd --graph /data/docker

# 重启docker
systemctl daemon-reload && systemctl restart docker

# 确认iptables,查看开始的几行是否有ACCEPT
iptables -nvL

5、安装kubeadm/kubelet/kubectl(所有机器)

软件包 作用
kubeadm 用来初始化集群的指令
kubelet 在集群中的每个节点上用来启动 pod 和 container 等
kubectl 用来与集群通信的命令行工具
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 设置k8s源
vi /etc/yum.repos.d/kubernetes.repo

[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

# 安装kubeadm,kubelet和kubectl,指定版本
yum install -y kubelet-1.18.2 kubeadm-1.18.2 kubectl-1.18.2

# 自启动kubelet
systemctl enable kubelet

6、集群初始化(第一个master机器)

1
2
3
4
5
6
7
8
9
10
# 初始化
kubeadm init \
--apiserver-advertise-address=172.18.123.163 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.18.2 \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16

# 授权kubectl工具
mkdir -p $HOME/.kube && cp -i /etc/kubernetes/admin.conf $HOME/.kube/config && chown $(id -u):$(id -g) $HOME/.kube/config

7、集群安装网络插件flannel(第一个master机器)

1)获取镜像,如果失败请搜索查询解决办法

1
2
# 获取Pod网络插件镜像
docker pull quay.io/coreos/flannel:v0.12.0-amd64

2)部署

https://raw.githubusercontent.com/coreos/flannel/v0.12.0/Documentation/kube-flannel.yml

修改yaml文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp.flannel.unprivileged
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
privileged: false
volumes:
- configMap
- secret
- emptyDir
- hostPath
allowedHostPaths:
- pathPrefix: "/etc/cni/net.d"
- pathPrefix: "/etc/kube-flannel"
- pathPrefix: "/run/flannel"
readOnlyRootFilesystem: false
# Users and groups
runAsUser:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
fsGroup:
rule: RunAsAny
# Privilege Escalation
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
# Capabilities
allowedCapabilities: ['NET_ADMIN']
defaultAddCapabilities: []
requiredDropCapabilities: []
# Host namespaces
hostPID: false
hostIPC: false
hostNetwork: true
hostPorts:
- min: 0
max: 65535
# SELinux
seLinux:
# SELinux is unused in CaaSP
rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: "system:flannel"
rules:
- apiGroups: ['extensions']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: "system:flannel"
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds-amd64
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: beta.kubernetes.io/os
operator: In
values:
- linux
- key: beta.kubernetes.io/arch
operator: In
values:
- amd64
hostNetwork: true
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.12.0-amd64
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.12.0-amd64
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg

8、加入集群(其他机器)

1
2
3
4
5
6
7
8
9
# 加入集群
kubeadm join 172.18.123.163:6443 --token ygmmsu.8mpvlippqezbkyln \
--discovery-token-ca-cert-hash sha256:feff014c4302ea55feca17c259d207559d6d42f7087e02df17cb58a236ea50dc

# 节点角色
kubectl label node icql-node1 node-role.kubernetes.io/node=

# 设置master节点可调度pod,如有需要
kubectl taint node icql-master node-role.kubernetes.io/master-

9、其他

微信打赏

赞赏是不耍流氓的鼓励