上一篇博文我介绍了如何快速搭建Salt环境, 以及利用相关pillar, state, grains模块进行基本的客户端部署操作.
本篇文档我们会详细介绍如何利用Salt来批量部署安装Nginx, 并自动化配置Nginx的相关属性.
Salt环境部署详见: http://www.showerlee.com/archives/1472
安装环境:
System: Centos 6.3
Salt master: salt-master.example.com
Salt minion: salt-client01.example.com
一. 主控端配置
1. 配置master基本参数
# vi /etc/salt/master
添加:
nodegroups:
webgroup1: 'salt-client01.example.com'
file_roots:
base:
- /srv/salt
pillar_roots:
base:
- /srv/pillar
2. 动态配置客户端系统连接数
使用python脚本编写grains_module, 实现动态配置被控主机连接数(CLI可用"limit -n"查询该值), 以便随后的Nginx配置文件中的worker_rlimit_nofile, worker_connections可动态调用脚本中max_open_file的参数值.
Tip: "ulimit -n" 是用来查询当前linux系统的最大打开文件数, 默认为1024
也可用"ulimit -a"来查看其他相关参数
# ulimit -a
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 7819 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
暂时修改当前session的参数值
# ulimit -n 2048
永久修改需修改该参数值
# vi /etc/security/limits.conf
添加:
root soft nofile 2048 root hard nofile 2048
# vi /etc/pam.d/common-session
添加:
session required pam_limits.so
重启生效.
脚本具体配置如下:
# mkdir -p /srv/salt/_grains
# vi /srv/salt/_grains/nginx_config.py
#!/usr/bin/env python
import os,sys,commands
# return Nginx config grains value
def NginxGrains():
grains = {}
max_open_file=65536
try:
getulimit=commands.getstatusoutput('source /etc/profile && ulimit -n')
except Exception,e:
pass
if getulimit[0]==0:
max_open_file=int(getulimit[1])
grains['max_open_file'] = max_open_file
return grains
Tip: 该脚本会同步到远程后执行, 脚本实际就是获取并返回当前主机的最大打开数值, 最终返回值会赋予字典 grains['max_open_file']
同步grains模块:
# salt '*' saltutil.sync_all
刷新模块(让minion编译模块)
# salt '*' sys.reload_modules
验证max_open_file key的value
# salt '*' grains.item max_open_file
Tip: 这里笔者测试更改客户端最大文件打开值时发现了一个问题, 无论客户端如何更改这个值, 在验证key value时终会显示最早的系统初始值1024, 翻阅了大量文档, 发现minion端会将所有服务端的推送保存在(/var/cache/salt/minion), 这里删除这个缓存目录并重启salt-minion, 让其生成新的缓存目录, 从新同步grains模块, 新的vaule就会生效.
# salt '*' cmd.run 'rm -rf /var/cache/salt/minion && /etc/init.d/salt-minion restart'
3. 配置pillar
1). 定义入口sls
# vi /srv/pillar/top.sls
base: webgroup1: - match: nodegroup - webserver1
2). 定义webserver1 sls
# vi /srv/pillar/webserver1.sls
nginx:
root: /www
3). 查看配置结果
# salt '*' pillar.data nginx
salt-client01.example.com:
----------
nginx:
----------
root:
/www
4. 配置state
1). 定义入口sls
# vi /srv/salt/top.sls
base: '*': - nginx
2). 定义nginx包及服务状态管理配置sls
# vi /srv/salt/nginx.sls
nginx:
pkg:
- installed
file.managed:
- source: salt://nginx/nginx.conf
- name: /etc/nginx/nginx.conf
- user: root
- group: root
- mode: 644
- template: jinja
service.running:
- enable: True
- reload: True
- watch:
- file: /etc/nginx/nginx.conf
- pkg: nginx
Tip: "salt://nginx/nginx.conf"为配置模板文件位置
"-enable: True" 等价于 "chkconfig nginx on"
"-reload True" 等价于 "service nginx reload", 若不加则默认执行"service nginx restart"
"-wotch -file:" 检查 /etc/nginx/nginx.conf是否发生变化
"-watch -pkg" 确保nginx安装成功.
3). 定义Nginx配置文件（引用jinja模板）
# vi /srv/salt/nginx/nginx.conf
# For more information on configuration, see:
user nginx;
worker_processes {{ grains['num_cpus'] }};
{% if grains['num_cpus'] == 2 %}
worker_cpu_affinity 01 10;
{% elif grains['num_cpus'] == 4 %}
worker_cpu_affinity 1000 0100 0010 0001;
{% elif grains['num_cpus'] >= 8 %}
worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000;
{% else %}
worker_cpu_affinity 1000 0100 0010 0001;
{% endif %}
worker_rlimit_nofile {{ grains['max_open_file'] }};
error_log /var/log/nginx/error.log;
#error_log /var/log/nginx/error.log notice;
#error_log /var/log/nginx/error.log info;
pid /var/run/nginx.pid;
events {
worker_connections {{ grains['max_open_file'] }};
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
# Load config files from the /etc/nginx/conf.d directory
# The default server is in conf.d/default.conf
#include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
server_name _;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root {{ pillar['nginx']['root'] }};
index index.html index.htm;
}
error_page 404 /404.html;
location = /404.html {
root /usr/share/nginx/html;
}
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
Tip:
4). 执行最终state配置, 将master的所有nginx配置部署到客户端
# salt '*' state.highstate
salt-client01.example.com:
----------
ID: nginx
Function: pkg.installed
Result: True
Comment: The following packages were installed/updated: nginx
Started: 15:51:50.117109
Duration: 311992.9 ms
Changes:
----------
GeoIP:
----------
new:
1.6.5-1.el6
old:
GeoIP-GeoLite-data:
----------
new:
2015.04-2.el6
old:
GeoIP-GeoLite-data-extra:
----------
new:
2015.04-2.el6
old:
gd:
----------
new:
2.0.35-11.el6
old:
geoipupdate:
----------
new:
2.2.1-2.el6
old:
libXpm:
----------
new:
3.5.10-2.el6
old:
nginx:
----------
new:
1.0.15-12.el6
old:
nginx-filesystem:
----------
new:
1.0.15-12.el6
old:
----------
ID: nginx
Function: file.managed
Name: /etc/nginx/nginx.conf
Result: True
Comment: File /etc/nginx/nginx.conf updated
Started: 15:57:02.118033
Duration: 50.538 ms
Changes:
----------
diff:
---
+++
@@ -1,42 +1,68 @@
-# For more information on configuration, see:
-# * Official English Documentation: http://nginx.org/en/docs/
-# * Official Russian Documentation: http://nginx.org/ru/docs/
-
-user nginx;
-worker_processes 1;
-
-error_log /var/log/nginx/error.log;
-#error_log /var/log/nginx/error.log notice;
-#error_log /var/log/nginx/error.log info;
-
-pid /var/run/nginx.pid;
-
-
-events {
- worker_connections 1024;
-}
-
-
-http {
- include /etc/nginx/mime.types;
- default_type application/octet-stream;
-
- log_format main '$remote_addr - $remote_user [$time_local] "$request" '
- '$status $body_bytes_sent "$http_referer" '
- '"$http_user_agent" "$http_x_forwarded_for"';
-
- access_log /var/log/nginx/access.log main;
-
- sendfile on;
- #tcp_nopush on;
-
- #keepalive_timeout 0;
- keepalive_timeout 65;
-
- #gzip on;
-
- # Load config files from the /etc/nginx/conf.d directory
- # The default server is in conf.d/default.conf
- include /etc/nginx/conf.d/*.conf;
-
-}
+# For more information on configuration, see:
+user nginx;
+worker_processes 1;
+
+worker_cpu_affinity 1000 0100 0010 0001;
+
+worker_rlimit_nofile 4096;
+
+error_log /var/log/nginx/error.log;
+#error_log /var/log/nginx/error.log notice;
+#error_log /var/log/nginx/error.log info;
+
+pid /var/run/nginx.pid;
+
+events {
+ worker_connections 4096;
+}
+
+
+http {
+ include /etc/nginx/mime.types;
+ default_type application/octet-stream;
+
+ log_format main '$remote_addr - $remote_user [$time_local] "$request" '
+ '$status $body_bytes_sent "$http_referer" '
+ '"$http_user_agent" "$http_x_forwarded_for"';
+
+ access_log /var/log/nginx/access.log main;
+
+ sendfile on;
+ #tcp_nopush on;
+
+ #keepalive_timeout 0;
+ keepalive_timeout 65;
+
+ #gzip on;
+
+ # Load config files from the /etc/nginx/conf.d directory
+ # The default server is in conf.d/default.conf
+ #include /etc/nginx/conf.d/*.conf;
+ server {
+ listen 80 default_server;
+ server_name _;
+
+ #charset koi8-r;
+
+ #access_log logs/host.access.log main;
+
+ location / {
+ root /www;
+ index index.html index.htm;
+ }
+
+ error_page 404 /404.html;
+ location = /404.html {
+ root /usr/share/nginx/html;
+ }
+
+ # redirect server error pages to the static page /50x.html
+ #
+ error_page 500 502 503 504 /50x.html;
+ location = /50x.html {
+ root /usr/share/nginx/html;
+ }
+
+ }
+
+}
----------
ID: nginx
Function: service.running
Result: True
Comment: Service nginx has been enabled, and is running
Started: 15:57:02.172052
Duration: 358.26 ms
Changes:
----------
nginx:
True
Summary
------------
Succeeded: 3 (changed=3)
Failed: 0
------------
Total states run: 3
从返回信息我们可以查看Nginx的配置文件参数是否正确调用, 以及最终是否部署成功.
最终/srv目录下的树状结构
# cd /srv && tree .
.
|-- pillar
| |-- top.sls
| `-- webserver1.sls
`-- salt
|-- _grains
| `-- nginx_config.py
|-- nginx
| `-- nginx.conf
|-- nginx.sls
`-- top.sls
4 directories, 6 files
至此, 一个模拟生产环境的WEB服务配置集中管理部署平台已经搭建并测试完成, 大家可以拓展思路, 利用该平台扩展到其他应用业务当中.