Rancher高可用集群部署

一、Rancher安装

1、Rancher HA架构

2、对应节点分别配置主机名

hostnamectl set-hostname rancher-master-1
hostnamectl set-hostname rancher-master-2
hostnamectl set-hostname rancher-master-3
hostnamectl set-hostname rancher-worker-1
hostnamectl set-hostname rancher-worker-2
hostnamectl set-hostname rancher-worker-3

3、分别配置每台主机的hosts(/etc/hosts),添加host_ip $hostname到/etc/hosts文件中。

172.16.0.101 rancher-master-1
172.16.0.102 rancher-master-2
172.16.0.103 rancher-master-3
172.16.0.104 rancher-worker-1
172.16.0.104 rancher-worker-2
172.16.0.104 rancher-worker-3

4、关闭防火墙

systemctl stop firewalld.service && systemctl disable firewalld.service

5、关闭SeLinux

sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

6、内核性能调优：

cat >> /etc/sysctl.conf<<EOF
net.ipv4.ip_forward=1
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
vm.swappiness=0
vm.max_map_count=655360
EOF
 sysctl --system

7、禁用swap

swapoff -a && sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

8、安装docker、配置容器目录、镜像加速地址等。

https://www.rancher.cn/docs/rancher/v2.x/cn/install-prepare/basic-environment-configuration/

9、添加一个新的用户以创建rke集群：

groupadd docker
useradd rancher -G docker
echo "gl2Hfc^JEF" | passwd --stdin rancher

10、从172.16.0.101配置免密登录到其他节点

su - rancher
ssh-keygen
ssh-copy-id rancher@172.16.0.101
ssh-copy-id rancher@172.16.0.102
ssh-copy-id rancher@172.16.0.103
ssh-copy-id rancher@172.16.0.104
ssh-copy-id rancher@172.16.0.105
ssh-copy-id rancher@172.16.0.106

11、在172.16.0.101安装rke、kubectl、helm

su root
sudo wget https://www.cnrancher.com/download/rke/v0.2.2-rke_linux-amd64
sudo wget https://www.cnrancher.com/download/kubernetes/linux-amd64-v1.13.6-kubectl
sudo wget https://www.cnrancher.com/download/helm/helm-v2.14.0-linux-amd64.tar.gz
sudo cp v0.2.2-rke_linux-amd64 /usr/bin/rke
sudo chmod +x /usr/bin/rke
sudo cp linux-amd64-v1.13.6-kubectl /usr/bin/kubectl
sudo chmod +x /usr/bin/kubectl
sudo tar -xvf helm-v2.14.0-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/bin/helm
sudo mv linux-amd64/tiller /usr/bin/tiller
sudo rm -rf  linux-amd64/

12、在172.16.0.101创建rancher集群配置文件:

su - rancher
cat > rancher-cluster.yml << EOF
nodes:
  - address: 172.16.0.101 
    user: rancher
    role: [controlplane,etcd]
  - address: 172.16.0.102
    user: rancher
    role: [controlplane,etcd]
  - address: 172.16.0.103
    user: rancher
    role: [controlplane,etcd]
  - address: 172.16.0.104
    user: rancher
    role: [worker]
  - address: 172.16.0.105
    user: rancher
    role: [worker]
  - address: 172.16.0.106
    user: rancher
    role: [worker]
services:
  etcd:
    snapshot: true
    creation: 6h
    retention: 24h
EOF

13、在172.16.0.101上启动rke集群

rke up --config ./rancher-cluster.yml

14、增删节点、修改12中的配置文件，执行13中的命令
15、在172.16.0.101配置环境变量：

su - root
echo export KUBECONFIG=/home/rancher/kube_config_rancher-cluster.yml  >>  /etc/profile
source /etc/profile

16、在172.16.0.101配置kubectl命令补全

echo "source <(kubectl completion bash)" >> ~/.bashrc
source ~/.bashrc
su - rancher
echo "source <(kubectl completion bash)" >> ~/.bashrc
source ~/.bashrc

17、在172.16.0.101配置Helm客户端访问权限

(1)、kubectl -n kube-system create serviceaccount tiller
(2)、kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller

18、在172.16.0.101安装Helm客户端

helm init --service-account tiller  --tiller-image registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.14.0 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts

18、在172.16.0.101添加Chart仓库地址

helm repo add rancher-stable https://releases.rancher.com/server-charts/stable

19、使用权威CA机构颁发的证书安装rancher-server
（1）、创建secret

kubectl create namespace cattle-system
kubectl -n cattle-system \
create secret tls tls-rancher-ingress \
--cert=./cartechfin.com.pem \
--key=./cartechfin.com.key

（2）、在172.16.0.101上使用helm安装rancher-server，域名为financial-k8s.coolops.cn

helm install rancher-stable/rancher \
--name rancher \
--namespace cattle-system \
--set hostname=financial-k8s.coolops.cn \
--set ingress.tls.source=secret

20、配置负载均衡，使用四层负载将ingress节点80、443端口配置到负载均衡，并将域名financial-k8s.coolops.cn解析到负载均衡外网IP。

二、其他设置

1、Ingress-NginX传递自定义header：

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
data:
  enable-underscores-in-headers: "true"

2、配置集群内部使用内网域名访问（非必须）

（1）kubectl -n cattle-system patch  daemonsets cattle-node-agent --patch '{
    "spec": {
        "template": {
            "spec": {
                "hostAliases": [
                    {
                        "hostnames":
                        [
                            "financial-k8s.coolops.cn"
                        ],
                            "ip": "172.16.0.107"
                    }
                ]
            }
        }
    }
}'
（2） 、kubectl -n cattle-system patch  deployments cattle-cluster-agent --patch '{
    "spec": {
        "template": {
            "spec": {
                "hostAliases": [
                    {
                        "hostnames":
                        [
                            "financial-k8s.coolops.cn"
                        ],
                            "ip": "172.16.0.107"
                    }
                ]
            }
        }
    }
}'