kubeadmin安装集群
标签（空格分隔）：kubernetes架构系列
kubeadmin安装集群一：的重要更新二：的安装2.1：高可用Kubernetes集群规划2.2yum的更新配置（所有节点全部安装）
一：的重要更新1、Kubectldebug设置一个临时容器2、Sidecar3、Volume：更改目录权限，fsGroup4、ConfigMap和SecretK8S官网：：二：的安装2.1：高可用Kubernetes集群规划
k8s的高可用的架构图
所有节点配置hosts，修改/etc/hosts如下：cat/etc/2.2yum的更新配置（所有节点全部安装）
curl-o/etc//;EOF/etc//[kubernetes]name=Kubernetesbaseurl='//d'-e'//d'/etc//
必备工具安装:yuminstallwgetjqpsmiscvimnet-toolstelnetyum-utilsdevice-mapper-persistent-datalvm2git-y
所有节点关闭防火墙、selinux、dnsmasq、swap。服务器配置如下：systemctldisable--nowfirewalldsystemctldisable--nowdnsmasqsystemctldisable--nowNetworkManagersetenforce0sed-i'sSELINUX=disabledSELINUX=enforcingg'/etc/selinux/config
关闭swap分区(全部节点)=0sed-ri'/^[^@'/etc/fstab
安装ntpdaterpm-ivh。时间同步配置如下：ln-sf/usr/share/zoneinfo/Asia/Shanghai/etc/localtimeecho'Asia/Shanghai'/etc/加入到crontab*/5****所有节点配置limit：ulimit-SHn65535vim/etc/security/末尾添加如下内容*softnofile655360*hardnofile131072*softnproc655350*hardnproc655350*softmemlockunlimited*hardmemlockunlimitedMaster01节点免密钥登录其他节点：;/id_$i;done所有节点升级系统并重启：yumupdate-yreboot
下载安装源码文件：cd/root/;gitclone
CentOS7安装yum源如下：curl-o/etc//;EOF/etc//[kubernetes]name=Kubernetesbaseurl='//d'-e'//d'/etc//
CentOS8安装源如下：curl-o/etc//;EOF/etc//[kubernetes]name=Kubernetesbaseurl='//d'-e'//d'/etc//
所有节点升级系统并重启，此处升级没有升级内核，下节会单独升级内核：yuminstallwgetjqpsmiscvimnet-toolstelnetyum-utilsdevice-mapper-persistent-datalvm2-yyumupdate-y--exclude=kernel*rebootyum--disablerepo="*"--enablerepo="elrepo-kernel"listavailableLoadedplugins:fastestmirrorLoadingmirrorspeedsfromcachedhostfile*elrepo-kernel:|2.9kB00:00:00elrepo-kernel/primary_db|1.9MB00:00:00_644.4.229-1._644.4.229-1._644.4.229-1._644.4.229-1._644.4.229-1._644.4.229-1._645.7.7-1._645.7.7-1._645.7.7-1._645.7.7-1._645.7.7-1._645.7.7-1._645.7.7-1._645.7.7-1.安装最新版：yum--enablerepo=elrepo-kernelinstallkernel-mlkernel-ml-devel–y安装完成后reboot更改内核顺序：grub2-set-default0grub2-mkconfig-o/etc/="user_=1"--update-kernel="$(grubby--default-kernel)"reboot开机后查看内核[appadmin@k8s-node01~]$_64wget：_64_64
温馨提示：由于新版kubelet建议使用systemd，所以可以把docker的CgroupDriver改成systemdcat/etc/docker/{"exec-opts":["=systemd"]}EOF
启动dockerservicedockerstartchkconfigdockeron
安装k8s组件：_64--showduplicates|sort-r所有节点安装最新版本kubeadm：yuminstallkubeadm-y所有节点安装指定版本k8s组件：_64_64_64所有节点设置开机自启动Docker：systemctldaemon-reloadsystemctlenable--nowdocker默认配置的pause镜像使用仓库，国内可能无法访问，所以这里配置Kubelet使用阿里云的pause镜像：DOCKER_CGROUPS=$(dockerinfo|grep'Cgroup'|cut-d''-f4)cat/etc/sysconfig/kubeletEOFKUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS--pod-infra-container-image=/google_containers/pause-amd64:3.1"EOF
--
设置Kubelet开机自启动：systemctldaemon-reloadsystemctlenable--nowkubelet
1.1.4高可用组件安装所有Master节点通过yum安装HAProxy和KeepAlived：yuminstallkeepalivedhaproxy-y
所有Master节点配置HAProxy（详细配置参考HAProxy文档，所有Master节点的HAProxy配置相同）：[root@k8s-master01etc]vim/etc/haproxy/timeoutconnect5000timeoutclient50000timeoutserver50000timeouthttp-request15stimeouthttp-keep-alive15sfrontmonitor-inbind*:33305modehttpoptionhttplogmonitor-uri/monitorlistenstatsbind*:8006modehttpstatsenablestatshide-versionstatsuri/statsstatsrefresh30sstatsrealmHaproxy\Statisticsstatsauthadmin::16443:16443modetcpoptiontcplogtcp-requestinspect-delay5sdefault_backk8s-masterbackk8s-mastermodetcpoptiontcplogoptiontcp-checkbalanceroundrobindefault-serverinter10sdowninter5sr:6443:6443:6443check----三台机器的配置是一样的：@:/etc/haproxy/@:/etc/haproxy/
Master01节点的配置：[root@k8s-master01etc]vim/etc/keepalived/!ConfigurationFileforkeepalivedglobal_defs{router_idLVS_DEVEL}vrrp_scriptchk_apiserver{script"/etc/keepalived/check_"interval2weight-5fall3rise2}vrrp_instanceVI_1{stateMASTERinterfaceens33mcast_src__router_id51priority100advert_int2authentication{auth_typePASSauth_passK8SHA_KA_AUTH}virtual_ipaddress{192.168.100.200}track_script{chk_apiserver}}Master02节点的配置：!ConfigurationFileforkeepalivedglobal_defs{router_idLVS_DEVEL}vrrp_scriptchk_apiserver{script"/etc/keepalived/check_"interval2weight-5fall3rise2}vrrp_instanceVI_1{stateBACKUPinterfaceens33mcast_src__router_id51priority101advert_int2authentication{auth_typePASSauth_passK8SHA_KA_AUTH}virtual_ipaddress{192.168.100.200}track_script{chk_apiserver}}Master03节点的配置：!ConfigurationFileforkeepalivedglobal_defs{router_idLVS_DEVEL}vrrp_scriptchk_apiserver{script"/etc/keepalived/check_"interval2weight-5fall3rise2}vrrp_instanceVI_1{stateBACKUPinterfaceens33mcast_src__router_id51priority102advert_int2authentication{auth_typePASSauth_passK8SHA_KA_AUTH}virtual_ipaddress{192.168.100.200}track_script{chk_apiserver}}注意上述的健康检查是关闭的，集群建立完成后再开启：track_script{chk_apiserver}配置KeepAlived健康检查文件：[root@k8s-master01keepalived]!/bin/basherr=0forkin$(seq15)docheck_code=$(pgrepkube-apiserver)if[[$check_code==""]];thenerr=$(expr$err+1)sleep5continueelseerr=0breakfidoneif[[$err!="0"]];thenecho"systemctlstopkeepalived"/usr/bin/systemctlstopkeepalivedexit1elseexit0fi
启动haproxy和keepalived(所有master启动)[root@k8s-master01keepalived]systemctlenable--nowkeepalived
集群初始化：
各Master节点的配置文件如下：Master01：/daocloud-----apiVersion:/v1beta2bootstrapTokens:-groups:-system:bootstrappers:kubeadm:default-node-tokentoken:7:24h0m0susages:-signing-authenticationkind:InitConfigurationlocalAPIpoint:advertiseAddress:192.168.100.11bindPort:6443nodeRegistration:criSocket:/var/run/::-effect:NoSchedulekey:/master---apiServer:certSANs:-192.168.100.200timeoutForControlPlane:4m0sapiVersion:/v1beta2certificatesDir:/etc/kubernetes/pkiclusterName:kubernetescontrolPlanepoint:192.168.100.200:16443controllerManager:{}dns:type:CoreDNSetcd:local:dataDir:/var/lib/etcdimageRepository:/google_containerskind:ClusterConfigurationkubernetesVersion::dnsDomain::172.168.100.0/16serviceSubnet:10.96.0.0/12scheduler:{}----更新kubeadm文件所有Master节点提前下载镜像，可以节省初始化时间：(master节点)kubeadmconfigimagespull--config/root/所有节点设置开机自启动kubeletsystemctlenable--nowkubelet
Master01节点初始化，初始化以后会在/etc/kubernetes目录下生成对应的证书和配置文件，之后其他Master节点加入Master01即可：kubeadminit--config/root/不用配置文件初始化：kubeadminit--control-plane-point"LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"--upload-certs
初始化失败报错errorexecutionphaseupload-config/kubelet:ErrorwritingCrisocketinformationforthecontrol-planenode:timedoutwaitingfortheconditionToseethestacktraceofthiserrorexecutewith--v=5orhigher
解决方法：所有主机停掉kubeletservicekubeletstop执行命令：swapoff-akubeadmresetsystemctldaemon-reloadsystemctlrestartkubeletiptables-Fiptables-tnat-Fiptables-tmangle-Fiptables-Xipvsadm--clear
再次初始化：kubeadminit--config/root/
YourKubernetescontrol-planehasinitializedsuccessfully!Tostartusingyourcluster,youneedtorunthefollowingasaregularuser:mkdir-p$HOME/.kubesudocp-i/etc/kubernetes/$HOME/.kube/configsudochown$(id-u):$(id-g)$HOME/.kube/configAlternatively,ifyouaretherootuser,youcanrun:exportKUBECONFIG=/etc/kubernetes/"kubectlapply-f[podnetwork].yaml"withoneoftheoptionslistedat:\--discovery-token-ca-cert-hashsha256:7263545b1a028e6217ff4e55712bf24422e6d9aeba54e76daabfc8a824ffcd99\--control-plane--certificate-keyc0b3b67c42f4fe9ae2832d86f80df35ee2e7b32f945906fabe60e4ae1f4ba18fPleasenotethatthecertificate-keygivesaccesstoclustersensitivedata,keepitsecret!Asasafeguard,uploaded-certswillbedeletedintwohours;Ifnecessary,youcanuse"kubeadminitphaseupload-certs--upload-certs"asroot::16443--\--discovery-token-ca-cert-hashsha256:7263545b1a028e6217ff4e55712bf24422e6d9aeba54e76daabfc8a824ffcd99
mkdir-p$HOME/.kubesudocp-i/etc/kubernetes/$HOME/.kube/configsudochown$(id-u):$(id-g)$HOME/.kube/config
所有Master节点配置环境变量，用于访问Kubernetes集群：catEOF/root/.bashrcexportKUBECONFIG=/etc/kubernetes//root/.bashrc查看节点状态：[root@k8s-master01~]etcd_points:"http://ETCD_IP:ETCD_PORT"g'_CA=`cat/etc/kubernetes/pki/etcd/|base64|tr-d'\n'`ETCD_CERT=`cat/etc/kubernetes/pki/etcd/|base64|tr-d'\n'`ETCD_KEY=`cat/etc/kubernetes/pki/etcd/|base64|tr-d'\n'`sed-i"s@etcd-cert:null@etcd-cert:${ETCD_CERT}@g;s@etcd_ca:""g;setcd_cert:"/calico-secrets/etcd-cert"etcd_key:""g'_SUBNET=`cat/etc/kubernetes/manifests/|grepcluster-cidr=|awk-F='{print$NF}'`sed-i's@value:"192.168.0.0/16"@value:'"${POD_SUBNET}"'@g'配置metricservercd/root/k8s-ha-install/:v1kind:ServiceAccountmetadata:labels:k8s-app:metrics-servername:metrics-servernamespace:kube-system---apiVersion:/v1kind:ClusterRolemetadata:labels:k8s-app:/aggregate-to-admin:"true"/aggregate-to-edit:"true"/aggregate-to-view:"true"name:system:aggregated-metrics-readerrules:-apiGroups:-:-pods-nodesverbs:-get-list-watch---apiVersion:/v1kind:ClusterRolemetadata:labels:k8s-app:metrics-servername:system:metrics-serverrules:-apiGroups:-""resources:-pods-nodes-nodes/stats-namespaces-configmapsverbs:-get-list-watch---apiVersion:/v1kind:RoleBindingmetadata:labels:k8s-app:metrics-servername:metrics-server-auth-readernamespace:kube-systemroleRef:apiGroup::Rolename:extension-apiserver-authentication-readersubjects:-kind:ServiceAccountname:metrics-servernamespace:kube-system---apiVersion:/v1kind:ClusterRoleBindingmetadata:labels:k8s-app:metrics-servername:metrics-server:system:auth-delegatorroleRef:apiGroup::ClusterRolename:system:auth-delegatorsubjects:-kind:ServiceAccountname:metrics-servernamespace:kube-system---apiVersion:/v1kind:ClusterRoleBindingmetadata:labels:k8s-app:metrics-servername:system:metrics-serverroleRef:apiGroup::ClusterRolename:system:metrics-serversubjects:-kind:ServiceAccountname:metrics-servernamespace:kube-system---apiVersion:v1kind:Servicemetadata:labels:k8s-app:metrics-servername:metrics-servernamespace:kube-systemspec:ports:-name:httpsport:443protocol:TCPtargetPort:httpsselector:k8s-app:metrics-server---apiVersion:apps/v1kind:Deploymentmetadata:labels:k8s-app:metrics-servername:metrics-servernamespace:kube-systemspec:selector:matchLabels:k8s-app:metrics-serverstrategy:rollingUpdate:maxUnavailable:0template:metadata:labels:k8s-app:metrics-serverspec:containers:-args:---cert-dir=/tmp---secure-port=4443---metric-resolution=30s---kubelet-insecure-tls---kubelet-preferred-address-types=InternalIP,ExternalIP,=X-Remote-User---requestheader-group-headers=X-Remote-Group---requestheader-extra-headers-prefix=X-Remote-Extra-image:/dotbalo/metrics-server::IfNotPresentlivenessProbe:failureThreshold:3httpGet:path:/livezport:httpsscheme:HTTPSperiodSeconds:10name:metrics-serverports:-containerPort:4443name:httpsprotocol:TCPreadinessProbe:failureThreshold:3httpGet:path:/readyzport:httpsscheme:HTTPSperiodSeconds:10securityContext:readOnlyRootFilesystem:truerunAsNonRoot:truerunAsUser:1000volumeMounts:-mountPath:/tmpname:tmp-dir-name:ca-sslmountPath:/etc/kubernetes/pkinodeSelector:/os:linuxpriorityClassName:system-cluster-criticalserviceAccountName:metrics-servervolumes:-emptyDir:{}name:tmp-dir-name:ca-sslhostPath:path:/etc/kubernetes/pki---apiVersion:/v1kind:APIServicemetadata:labels:k8s-app:metrics-servername::group::100insecureSkipTLSVerify:trueservice:name:metrics-servernamespace:kube-systemversion:v1beta1versionPriority:100----安装dashbaordcd/root/k8s-ha-install/
kubectleditsvckubernetes-dashboard-nkubernetes-dashboard改一下svc的类型：type:Cluster-IP改为：type:NodePort
kubectlgetsvc-nkubernetes-dashboard-owide
kubectl-nkube-systemdescribesecret$(kubectl-nkube-systemgetsecret|grepadmin-user|awk'{print$1}')集群测试：
部署kuborad在node节点上面下载镜像：dockerpulleipwork/kuboard:latestkubectlapply-f|grepkuboard
获取Token#如果您参考提供的文档安装Kuberenetes，可在第一个Master节点上执行此命令echo$(kubectl-nkube-systemgetsecret$(kubectl-nkube-systemgetsecret|grepkuboard-user|awk'{print$1}')-ogo-template='{{.}}'|base64-d)卸载：kuborad-v2kubectldelete-f
安装kuboard-v3在node节点上面下载镜像：dockerpulleipwork/kuboard:v3dockerpulleipwork/etcd-host:3.4.16-1mkdir/datachmod777-R/data配置镜像下载策略wget(共有两处)---
访问Kuboard在浏览器中打开链接http://your-node-ip-address:30080输入初始用户名和密码，并登录用户名：admin密码：Kuboard123
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
[43]
[44]
[45]
[46]
[47]
[48]
[49]
[50]
[51]
[52]
[53]
