How to build an in-house Docker environment step by step?
royalwzy发表于 2016-10-24 08:32:02
主要内容:
1.环境准备;
2.安装并启动Docker;
3.部署私有Registry服务器;
4.安装并配置Rancher;
5.创建一个多容器的应用;
1.环境准备:
1.1 软件版本介绍:
1.OS:CentOS7.2/3.10.0-327.el7.x86_64(软件);
2.docker:1.11.2(软件);
3.rancher/server:v1.1.2(容器);
4.rancher/agent:v1.0.2(容器);
5.rancher/agent-instance:v0.8.3(容器);
6.registry:2.5.0(容器);
7.mysql:5.7.13(容器);
1.2 服务器准备:
1.Rancher服务器:192.168.10.160/rancher.htsec.com;用来安装Docker服务,Register服务,Rancher服务,Rancher代理和MySQL服务;
2.Container1服务器:192.168.10.161/container1.htsec.com;用来安装Docker服务,Rancher代理和创建容器;
3.Container2服务器:192.168.10.162/container2.htsec.com;用来安装Docker服务,Rancher代理和创建容器;
1.3 服务器配置:
# 分别设置hostname:
hostnamectl set-hostname rancher.htsec.com
hostnamectl set-hostname container1.htsec.com
hostnamectl set-hostname container2.htsec.com
# 分别关闭防火墙和网络管理服务:
systemctl stop firewalld.service
systemctl disable firewalld.service
systemctl stop NetworkManager.service
systemctl disable NetworkManager.service
# 分别配置/etc/hosts:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.10.160 rancher.htsec.com rancher
192.168.10.160 docker.htsec.com docker
192.168.10.160 registry.htsec.com registry
192.168.10.160 mysql.htsec.com mysql
192.168.10.161 container1.htsec.com container1
192.168.10.162 container2.htsec.com container2
1.4 下载Docker相关包(在可以连公网的机器下载,上传到内网服务器):
wget -S -c -r -np -L https://yum.dockerproject.org/repo/main/centos/7/;
mkdir -p /tools/docker/centos7
mv yum.dockerproject.org/repo/main/centos/7/* /tools/docker/centos7/
1.5 添加/etc/yum.repos.d/docker.repo文件,配置yum源;
[dockerrepo]
name=Docker Repository
baseurl=file:///tools/docker/centos7
enabled=1
gpgcheck=0
1.6 挂载操作系统光盘,添加/etc/yum.repos.d/centos.repo文件,配置yum源:
mkdir -p /tools/centos72
mount -o loop /dev/sr0 /tools/centos72
[centosrepo]
name=CentOS7 Repository
baseurl=file:///tools/centos72/
enabled=1
gpgcheck=0
1.7 生成YUM缓存:
yum clean all
yum makecache
2.分别安装并启动Docker:
2.1 必要条件:Docker必须安装在64位操作系统上,而且内核版本必须大于3.10;(CentOS7的内核版本最小是3.10,使用uname -r命令查看)
2.2 使用yum安装:
yum update -y
yum install -y docker-engine
2.3 如果直接使用rpm安装的话,需要先解决包依赖;
yum install -y libcgroup libtool-ltdl policycoreutils-python
rpm -ivh docker-engine-selinux-1.11.2-1.el7.centos.noarch.rpm
rpm -ivh docker-engine-1.11.2-1.el7.centos.x86_64.rpm
2.4 启动docker;
service docker start 或者 systemctl start docker.service
2.5 配置开机启动;
chkconfig docker on 或者 systemctl enable docker.service
3.部署私有Registry服务器:
3.1 加载registry容器(所以需要的容器都需要在其它环境下载,然后加载):
[root@rancher ~]# docker load < /tools/images/registry2.5.tar
[root@rancher ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry latest c6c14b3960bd 5 days ago 33.28 MB
3.2 启动一个Registry,并把数据映射到本地卷:
[root@rancher ~]# mkdir -p /var/lib/registry
[root@rancher ~]# docker run -d -p 5000:5000 --restart=always --name registry -v /var/lib/registry:/var/lib/registry registry:latest
3.3 加载其它镜像:
[root@rancher ~]# docker load < /tools/images/rancherserver_v1.1.2.tar
[root@rancher ~]# docker load < /tools/images/rancheragent_v1.0.2.tar
[root@rancher ~]# docker load < /tools/images/rancheragent-instance_v0.8.3.tar
[root@rancher ~]# docker load < /tools/images/mysql5.7.13.tar
[root@rancher ~]# docker load < /tools/images/wordpress.tar
[root@rancher ~]# docker load < /tools/images/oraclelinux6.8.tar
[root@rancher ~]# docker load < /tools/images/hello-world.tar
[root@rancher ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
wordpress latest 106a375e769a 46 hours ago 420.5 MB
registry latest c6c14b3960bd 5 days ago 33.28 MB
rancher/server latest ffe9c46b500a 11 days ago 842 MB
oraclelinux 6.8 175adfa05e40 13 days ago 223.1 MB
hello-world latest c54a2cc56cbb 4 weeks ago 1.848 kB
rancher/agent-instance v0.8.3 b6b013f2aa85 7 weeks ago 330.9 MB
rancher/agent v1.0.2 860ed2b2e8e3 7 weeks ago 454.3 MB
mysql latest 1195b21c3a45 7 weeks ago 380.2 MB
3.4 对镜像打标签,指向私有Registry:
[root@rancher ~]# docker tag registry:latest registry.htsec.com:5000/registry:2.5
[root@rancher ~]# docker tag rancher/server:latest registry.htsec.com:5000/rancher/server:v1.1.2
[root@rancher ~]# docker tag rancher/agent:v1.0.2 registry.htsec.com:5000/rancher/agent:v1.0.2
[root@rancher ~]# docker tag rancher/agent-instance:v0.8.3 registry.htsec.com:5000/rancher/agent-instance:v0.8.3
[root@rancher ~]# docker tag mysql:latest registry.htsec.com:5000/mysql:5.7.13
[root@rancher ~]# docker tag wordpress:latest registry.htsec.com:5000/wordpress:4.5.3
[root@rancher ~]# docker tag oraclelinux:6.8 registry.htsec.com:5000/oraclelinux:6.8
[root@rancher ~]# docker tag hello-world:latest registry.htsec.com:5000/hello-world:latest
3.5 把镜像推到私有Registry中:
[root@rancher ~]# docker push registry.htsec.com:5000/registry:2.5
[root@rancher ~]# docker push registry.htsec.com:5000/rancher/server:v1.1.2
[root@rancher ~]# docker push registry.htsec.com:5000/rancher/agent:v1.0.2
[root@rancher ~]# docker push registry.htsec.com:5000/rancher/agent-instance:v0.8.3
[root@rancher ~]# docker push registry.htsec.com:5000/mysql:5.7.13
[root@rancher ~]# docker push registry.htsec.com:5000/wordpress:4.5.3
[root@rancher ~]# docker push registry.htsec.com:5000/oraclelinux:6.8
[root@rancher ~]# docker push registry.htsec.com:5000/hello-world:latest
3.6 从私有Registry中拉取镜像:
[root@rancher ~]# docker pull registry.htsec.com:5000/registry:2.5
[root@rancher ~]# docker pull registry.htsec.com:5000/rancher/server:v1.1.2
[root@rancher ~]# docker pull registry.htsec.com:5000/rancher/agent:v1.0.2
[root@rancher ~]# docker pull registry.htsec.com:5000/rancher/agent-instance:v0.8.3
[root@rancher ~]# docker pull registry.htsec.com:5000/mysql:5.7.13
[root@rancher ~]# docker pull registry.htsec.com:5000/wordpress:4.5.3
[root@rancher ~]# docker pull registry.htsec.com:5000/oraclelinux:6.8
[root@rancher ~]# docker pull registry.htsec.com:5000/hello-world:latest
3.7 将本地的镜像删掉(untag):
[root@rancher ~]# docker rmi registry:latest
[root@rancher ~]# docker rmi rancher/server:latest
[root@rancher ~]# docker rmi rancher/agent:v1.0.2
[root@rancher ~]# docker rmi rancher/agent-instance:v0.8.3
[root@rancher ~]# docker rmi mysql:latest
[root@rancher ~]# docker rmi wordpress:latest
[root@rancher ~]# docker rmi oraclelinux:6.8
[root@rancher ~]# docker rmi hello-world:latest
[root@rancher ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.htsec.com:5000/wordpress 4.5.3 106a375e769a 47 hours ago 420.5 MB
registry.htsec.com:5000/registry 2.5 c6c14b3960bd 5 days ago 33.28 MB
registry.htsec.com:5000/rancher/server v1.1.2 ffe9c46b500a 11 days ago 842 MB
registry.htsec.com:5000/oraclelinux 6.8 175adfa05e40 2 weeks ago 223.1 MB
registry.htsec.com:5000/hello-world latest c54a2cc56cbb 4 weeks ago 1.848 kB
registry.htsec.com:5000/rancher/agent-instance v0.8.3 b6b013f2aa85 7 weeks ago 330.9 MB
registry.htsec.com:5000/rancher/agent v1.0.2 860ed2b2e8e3 7 weeks ago 454.3 MB
registry.htsec.com:5000/mysql 5.7.13 1195b21c3a45 7 weeks ago 380.2 MB
3.8 在私有Registry中检索镜像,不能使用docker search命令,需要通过使用v2版本的API:
[root@rancher ~]# docker search registry.htsec.com:5000/rancher/server
Error response from daemon: Unexpected status code 404
[root@rancher ~]# curl http://registry.htsec.com:5000/v2/_catalog
{"repositories":["hello-world","mysql","oraclelinux","rancher/agent","rancher/agent-instance","rancher/server","registry","wordpress"]}
[root@rancher ~]# curl http://registry.htsec.com:5000/v2/rancher/server/tags/list
{"name":"rancher/server","tags":["v1.1.2"]}
4.安装并配置Rancher:
4.1 启动并配置MySQL容器:
# 启动MySQL容器;
[root@rancher ~]# mkdir -p /var/lib/mysql
[root@rancher ~]# docker run -d --restart=always --name mysql -v /var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=pwd registry.htsec.com:5000/mysql:5.7.13
# 配置MySQL容器;
[root@rancher ~]# docker exec -it mysql /bin/bash
root@c94ee0fabd33:/# mysql -uroot -p
mysql> CREATE DATABASE IF NOT EXISTS cattle COLLATE = 'utf8_general_ci' CHARACTER SET = 'utf8';
mysql> GRANT ALL ON cattle.* TO 'cattle'@'%' IDENTIFIED BY 'cattle';
mysql> GRANT ALL ON cattle.* TO 'cattle'@'localhost' IDENTIFIED BY 'cattle';
mysql> flush privileges;
mysql> exit
root@c94ee0fabd33:/# exit
4.2 启动rancher/server容器(Rancher的三个组件[rancher/server,rancher/agent,rancher/agent-instance]的版本是有关联的):
[root@rancher ~]# docker run -d --restart=always -p 8080:8080 --name rancher-server --link mysql:mysql \
-e CATTLE_BOOTSTRAP_REQUIRED_IMAGE=registry.htsec.com:5000/rancher/agent:v1.0.2 \
-e CATTLE_AGENT_INSTANCE_IMAGE=registry.htsec.com:5000/rancher/agent-instance:v0.8.3 \
registry.htsec.com:5000/rancher/server:v1.1.2
# 打开浏览器测试;
http://192.168.10.160:8080
4.3 添加hosts:
# 在管理服务器上运行;
[root@rancher ~]# sudo docker run -e CATTLE_AGENT_IP="192.168.10.160" -e CATTLE_HOST_LABELS='role=administrator&id=010505' -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher registry.htsec.com:5000/rancher/agent:v1.0.2 http://192.168.10.160:8080/v1/scripts/2D061F6830FEC5A215F3:1470247200000:kz7K8gpchOMXD8poxh3kglKEBsc
# 在容器服务器上运行;
[root@container1 ~]# sudo docker run -e CATTLE_AGENT_IP="192.168.10.161" -e CATTLE_HOST_LABELS='role=owner' -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher registry.htsec.com:5000/rancher/agent:v1.0.2 http://192.168.10.160:8080/v1/scripts/2D061F6830FEC5A215F3:1470247200000:kz7K8gpchOMXD8poxh3kglKEBsc
[root@container2 ~]# sudo docker run -e CATTLE_AGENT_IP="192.168.10.162" -e CATTLE_HOST_LABELS='role=owner' -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher registry.htsec.com:5000/rancher/agent:v1.0.2 http://192.168.10.160:8080/v1/scripts/2D061F6830FEC5A215F3:1470247200000:kz7K8gpchOMXD8poxh3kglKEBsc
4.4 查看拓扑图:
5.创建一个多容器的应用:
5.1 创建一个Stacks:名叫WordPress;
5.2 添加一个Service(wp-mysql),由一个[registry.htsec.com:5000/mysql:5.7.13]镜像创建的容器组成,需要做端口转换(因为客户端可能在其它的主机上),并指定MYSQL_ROOT_PASSWORD环境变量;
5.2 添加一个Service(wp-app),由两个[registry.htsec.com:5000/wordpress:4.5.3]镜像创建的容器组成,链接到wp-mysql服务,并指定服务别名;
5.3 添加一个Service(wp-lb),由一个负载均衡器容器组成,映射80端口到8000端口,并链接到wp-app服务上;
5.4 访问负载均衡器的8000端口:http://192.168.10.162:8000/;
问题:[root@rancher ~]# docker push registry.htsec.com:5000/registry:2.5
The push refers to a repository [registry.htsec.com:5000/registry]
Get https://registry.htsec.com:5000/v1/_ping: tls: oversized record received with length 20527
解决办法:
1.添加文件:
[root@rancher ~]# vi /etc/sysconfig/docker
DOCKER_OPTS="--insecure-registry registry.htsec.com:5000"
2.修改文件:
[root@rancher ~]# vi /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network.target docker.socket
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
EnvironmentFile=-/etc/sysconfig/docker
ExecStart=/usr/bin/docker daemon $DOCKER_OPTS -H fd://
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
[Install]
WantedBy=multi-user.target
3.重启Docker:
[root@rancher ~]# systemctl daemon-reload
[root@rancher ~]# systemctl stop docker.service
[root@rancher ~]# systemctl start docker.service
问题:WARNING: IPv4 forwarding is disabled. Networking will not work.
解决办法:
1.在/usr/lib/sysctl.d/00-system.conf配置文件中添加代码;
net.ipv4.ip_forward=1
2.重启network服务;
systemctl restart network
3.查看是否修改成功;
sysctl net.ipv4.ip_forward