image.png

VXLAN

image.png

Overlay、Underlay

etcd

Mutil-host networking with etcd

setup etcd cluster

在docker-node1上

  1. ubuntu@docker-node1:~$ wget https://github.com/coreos/etcd/releases/download/v3.0.12/etcd-v3.0.12-linux-amd64.tar.gz
  2. ubuntu@docker-node1:~$ tar zxvf etcd-v3.0.12-linux-amd64.tar.gz
  3. ubuntu@docker-node1:~$ cd etcd-v3.0.12-linux-amd64
  4. ubuntu@docker-node1:~$ nohup ./etcd --name docker-node1 --initial-advertise-peer-urls http://192.168.205.10:2380 \
  5. --listen-peer-urls http://192.168.205.10:2380 \
  6. --listen-client-urls http://192.168.205.10:2379,http://127.0.0.1:2379 \
  7. --advertise-client-urls http://192.168.205.10:2379 \
  8. --initial-cluster-token etcd-cluster \
  9. --initial-cluster docker-node1=http://192.168.205.10:2380,docker-node2=http://192.168.205.11:2380 \
  10. --initial-cluster-state new&

在docker-node2上

  1. ubuntu@docker-node2:~$ wget https://github.com/coreos/etcd/releases/download/v3.0.12/etcd-v3.0.12-linux-amd64.tar.gz
  2. ubuntu@docker-node2:~$ tar zxvf etcd-v3.0.12-linux-amd64.tar.gz
  3. ubuntu@docker-node2:~$ cd etcd-v3.0.12-linux-amd64/
  4. ubuntu@docker-node2:~$ nohup ./etcd --name docker-node2 --initial-advertise-peer-urls http://192.168.205.11:2380 \
  5. --listen-peer-urls http://192.168.205.11:2380 \
  6. --listen-client-urls http://192.168.205.11:2379,http://127.0.0.1:2379 \
  7. --advertise-client-urls http://192.168.205.11:2379 \
  8. --initial-cluster-token etcd-cluster \
  9. --initial-cluster docker-node1=http://192.168.205.10:2380,docker-node2=http://192.168.205.11:2380 \
  10. --initial-cluster-state new&

检查cluster状态

  1. ubuntu@docker-node2:~/etcd-v3.0.12-linux-amd64$ ./etcdctl cluster-health
  2. member 21eca106efe4caee is healthy: got healthy result from http://192.168.205.10:2379
  3. member 8614974c83d1cc6d is healthy: got healthy result from http://192.168.205.11:2379
  4. cluster is healthy

重启docker服务

在docker-node1上

  1. $ sudo service docker stop
  2. $ sudo /usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --cluster-store=etcd://192.168.205.10:2379 --cluster-advertise=192.168.205.10:2375&

在docker-node2上

  1. $ sudo service docker stop
  2. $ sudo /usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --cluster-store=etcd://192.168.205.11:2379 --cluster-advertise=192.168.205.11:2375&

创建overlay network

在docker-node1上创建一个demo的overlay network

  1. ubuntu@docker-node1:~$ sudo docker network ls
  2. NETWORK ID          NAME                DRIVER              SCOPE
  3. 0e7bef3f143a        bridge              bridge              local
  4. a5c7daf62325        host                host                local
  5. 3198cae88ab4        none                null                local
  6. ubuntu@docker-node1:~$ sudo docker network create -d overlay demo
  7. 3d430f3338a2c3496e9edeccc880f0a7affa06522b4249497ef6c4cd6571eaa9
  8. ubuntu@docker-node1:~$ sudo docker network ls
  9. NETWORK ID          NAME                DRIVER              SCOPE
  10. 0e7bef3f143a        bridge              bridge              local
  11. 3d430f3338a2        demo                overlay             global
  12. a5c7daf62325        host                host                local
  13. 3198cae88ab4        none                null                local
  14. ubuntu@docker-node1:~$ sudo docker network inspect demo
  15. [
  16.     {
  17.         "Name": "demo",
  18.         "Id": "3d430f3338a2c3496e9edeccc880f0a7affa06522b4249497ef6c4cd6571eaa9",
  19.         "Scope": "global",
  20.         "Driver": "overlay",
  21.         "EnableIPv6": false,
  22.         "IPAM": {
  23.             "Driver": "default",
  24.             "Options": {},
  25.             "Config": [
  26.                 {
  27.                     "Subnet": "10.0.0.0/24",
  28.                     "Gateway": "10.0.0.1/24"
  29.                 }
  30.             ]
  31.         },
  32.         "Internal": false,
  33.         "Containers": {},
  34.         "Options": {},
  35.         "Labels": {}
  36.     }
  37. ]

我们会看到在node2上，这个demo的overlay network会被同步创建

  1. ubuntu@docker-node2:~$ sudo docker network ls
  2. NETWORK ID          NAME                DRIVER              SCOPE
  3. c9947d4c3669        bridge              bridge              local
  4. 3d430f3338a2        demo                overlay             global
  5. fa5168034de1        host                host                local
  6. c2ca34abec2a        none                null                local

通过查看etcd的key-value, 我们获取到，这个demo的network是通过etcd从node1同步到node2的

  1. ubuntu@docker-node2:~/etcd-v3.0.12-linux-amd64$ ./etcdctl ls /docker
  2. /docker/network
  3. /docker/nodes
  4. ubuntu@docker-node2:~/etcd-v3.0.12-linux-amd64$ ./etcdctl ls /docker/nodes
  5. /docker/nodes/192.168.205.11:2375
  6. /docker/nodes/192.168.205.10:2375
  7. ubuntu@docker-node2:~/etcd-v3.0.12-linux-amd64$ ./etcdctl ls /docker/network/v1.0/network
  8. /docker/network/v1.0/network/3d430f3338a2c3496e9edeccc880f0a7affa06522b4249497ef6c4cd6571eaa9
  9. ubuntu@docker-node2:~/etcd-v3.0.12-linux-amd64$ ./etcdctl get /docker/network/v1.0/network/3d430f3338a2c3496e9edeccc880f0a7affa06522b4249497ef6c4cd6571eaa9 | jq .
  10. {
  11.   "addrSpace": "GlobalDefault",
  12.   "enableIPv6": false,
  13.   "generic": {
  14.     "com.docker.network.enable_ipv6": false,
  15.     "com.docker.network.generic": {}
  16.   },
  17.   "id": "3d430f3338a2c3496e9edeccc880f0a7affa06522b4249497ef6c4cd6571eaa9",
  18.   "inDelete": false,
  19.   "ingress": false,
  20.   "internal": false,
  21.   "ipamOptions": {},
  22.   "ipamType": "default",
  23.   "ipamV4Config": "[{\"PreferredPool\":\"\",\"SubPool\":\"\",\"Gateway\":\"\",\"AuxAddresses\":null}]",
  24.   "ipamV4Info": "[{\"IPAMData\":\"{\\\"AddressSpace\\\":\\\"GlobalDefault\\\",\\\"Gateway\\\":\\\"10.0.0.1/24\\\",\\\"Pool\\\":\\\"10.0.0.0/24\\\"}\",\"PoolID\":\"GlobalDefault/10.0.0.0/24\"}]",
  25.   "labels": {},
  26.   "name": "demo",
  27.   "networkType": "overlay",
  28.   "persist": true,
  29.   "postIPv6": false,
  30.   "scope": "global"
  31. }