IT博客汇 | [原]mongodb 在单机上搭建分片集群详细实战过程

[原]mongodb 在单机上搭建分片集群详细实战过程

mchdba发表于 2016-07-11 21:53:41

<->本地搭建

一般线上mongodb服务器都好多台，可以自由搭建mongodb分片集群的，但是不能随意测试，如果想自己随意测试新功能怎么办呢？可以在本地搭建，在一台服务器mongodb1（hch_test_dbm1_121_62）上大家mongodb分片集群，下面分片是示意图(https://docs.mongodb.com/manual/_images/sharded-cluster-production-architecture.png)：

OK，接下来就准备开始搭建mongodb分片集群，从安装mongodb开始，然后逐渐部署config servers、mongos servers、shard servers一步步来开始准备。

1，下载安装：

https://www.mongodb.org/dr/fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.2.3.tgz/download，我这里下载的是稳定版本：mongodb-linux-x86_64-3.0.3.tgz

下载后解压缩

tar -xvf mongodb-linux-x86_64-3.0.3.tgz -C/usr/local/

mkdir /data

mkdir/usr/local/mongodb-linux-x86_64-3.0.3/log/

mkdir /data/mongodb

chown -R mongodb.mongodb /data/mongodb

chown -R mongodb.mongodb/usr/local/mongodb-linux-x86_64-3.0.3

chown -R mongodb:mongodb/usr/local/mongodb-linux-x86_64-3.0.3 -R

chown -R mongodb:mongodb /data/mongodb -R

2、在mongodb1上准备好3个config servers

mkdir /data/mongodb/config20001 -p

mkdir /data/mongodb/config20002 -p

mkdir /data/mongodb/config20003 -p

mkdir /data/mongodb/logs/ -p

启动config servers 进程：

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--configsvr --dbpath /data/mongodb/config20001 --port 20001 --logpath/data/mongodb/logs/configsvr_20001.log --logappend --fork

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--configsvr --dbpath /data/mongodb/config20002 --port 20002 --logpath/data/mongodb/logs/configsvr_20002.log --logappend --fork

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--configsvr --dbpath /data/mongodb/config20003 --port 20003 --logpath/data/mongodb/logs/configsvr_20003.log --logappend --fork

3、启动mongos路由服务

在mongodb1上启动mongos服务器（可以启动一个也可以启动多个）:

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongos--configdb mongodb1:20001,mongodb1:20002,mongodb1:20003 --port 40000 --chunkSize 64 --logpath/data/mongodb/logs/mongos40004.log --logappend --fork

4、开始准备shard servers

# 第一个shard1分片副本集

# 创建目录

mkdir /data/mongodb/shard37017

mkdir /data/mongodb/shard37027

mkdir /data/mongodb/shard37037

# 开始启动shard servers进程

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard1 --port 37017 --dbpath /data/mongodb/shard37017--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37017.log --logappend--fork

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard1 --port 37027 --dbpath /data/mongodb/shard37027--oplogSize 2048 --logpath /data/mongodb/logs/shard_m1s1_37027.log --logappend--fork

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard1 --port 37037 --dbpath /data/mongodb/shard37037--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37037.log --logappend--fork

# 第二个shard2分片副本集

# 创建目录

mkdir /data/mongodb/shard37018

mkdir /data/mongodb/shard37028

mkdir /data/mongodb/shard37038

# 启动shard severs进程

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard2 --port 37018 --dbpath /data/mongodb/shard37018--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37018.log --logappend--fork

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard2 --port 37028 --dbpath /data/mongodb/shard37028--oplogSize 2048 --logpath /data/mongodb/logs/shard_m1s1_37028.log --logappend--fork

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard2 --port 37038 --dbpath /data/mongodb/shard37038--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37038.log --logappend--fork

5、开始设置分片副本集shard1

# 设置第一个分片副本集，必须使用admin数据库，登录命令如下

/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongomongodb1:37017/admin

#定义副本集命令

> config = { _id:"shard1",members:[

{_id:0,host:"mongodb1:37017",priority:1},

{_id:1,host:"mongodb1:37027",priority:2},

{_id:2,host:"mongodb1:37037",arbiterOnly:true}

]

};

# 初始化副本集命令

> rs.initiate(config);

# 执行过程如下：

[mongodb@dnadb_1 mongodb]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27017/admin

MongoDB shell version: 3.0.3

connecting to: localhost:27017/admin

Server has startup warnings:

2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten]

2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.

2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'

2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten]

2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.

2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'

2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten]

> config = { _id:"shard1", members:[

... {_id:0,host:"mongodb1:37017",priority:1},

... {_id:1,host:"mongodb1:37027",priority:2},

... {_id:2,host:"mongodb1:37037",arbiterOnly:true}

... ]

... };

{

"_id" : "shard1",

"members" : [

{

"_id" : 0,

"host" : "mongodb1:37017",

"priority" : 1

{

"_id" : 1,

"host" : "mongodb1:37027",

"priority" : 2

{

"_id" : 2,

"host" : "mongodb1:37037",

"arbiterOnly" : true

}

]

}

> rs.initiate(config);

{ "ok" : 1 }

shard1:OTHER>

6、开始设置分片副本集shard2

# 开始设置第二个分片副本集

> config = { _id:"shard2",members:[

{_id:0,host:"mongodb1:37018",priority:1},

{_id:1,host:"mongodb1:37028",priority:2},

{_id:2,host:"mongodb1:37038",arbiterOnly:true}

]

};

# 初始化副本集

> rs.initiate(config);

执行过程如下：

[mongodb@dnadb_1 mongodb]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27018/admin

MongoDB shell version: 3.0.3

connecting to: localhost:27018/admin

Server has startup warnings:

2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten]

2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.

2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'

2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten]

2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.

2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'

2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten]

> config = { _id:"shard2", members:[

... {_id:0,host:"mongodb1:37018",priority:1},

... {_id:1,host:"mongodb1:37028",priority:2},

... {_id:2,host:"mongodb1:37038",arbiterOnly:true}

... ]

... };

{

"_id" : "shard2",

"members" : [

{

"_id" : 0,

"host" : "mongodb1:37018",

"priority" : 1

{

"_id" : 1,

"host" : "mongodb1:37028",

"priority" : 2

{

"_id" : 2,

"host" : "mongodb1:37038",

"arbiterOnly" : true

}

]

}

> rs.initiate(config);

{ "ok" : 1 }

shard2:OTHER>

7、检查下副本状态：

[mongodb@hch_test_dbm1_121_62 ~]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongomongodb1:37028/admin

MongoDB shell version: 3.0.3

connecting to: mongodb1:37028/admin

Server has startup warnings:

2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten]

2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten] ** WARNING: You are runningon a NUMA machine.

2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten] ** We suggest launching mongod like thisto avoid performance problems:

2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten] ** numactl --interleave=all mongod[other options]

2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten]

2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** WARNING:/sys/kernel/mm/transparent_hugepage/enabled is 'always'.

2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'

2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten]

2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** WARNING:/sys/kernel/mm/transparent_hugepage/defrag is 'always'.

2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'

2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten]

shard2:PRIMARY> rs.status();

{

"set": "shard2",

"date": ISODate("2016-07-09T11:35:36.680Z"),

"myState": 1,

"members": [

{

"_id": 0,

"name": "mongodb1:37018",

"health": 1,

"state": 2,

"stateStr": "SECONDARY",

"uptime": 62,

"optime": Timestamp(1468064073, 1),

"optimeDate": ISODate("2016-07-09T11:34:33Z"),

"lastHeartbeat": ISODate("2016-07-09T11:35:35.736Z"),

"lastHeartbeatRecv": ISODate("2016-07-09T11:35:35.739Z"),

"pingMs": 0,

"configVersion": 1

{

"_id": 1,

"name": "mongodb1:37028",

"health": 1,

"state": 1,

"stateStr": "PRIMARY",

"uptime": 1598,

"optime": Timestamp(1468064073, 1),

"optimeDate": ISODate("2016-07-09T11:34:33Z"),

"electionTime": Timestamp(1468064110, 1),

"electionDate": ISODate("2016-07-09T11:35:10Z"),

"configVersion": 1,

"self": true

{

"_id": 2,

"name": "mongodb1:37038",

"health": 1,

"state": 7,

"stateStr": "ARBITER",

"uptime": 62,

"lastHeartbeat": ISODate("2016-07-09T11:35:35.747Z"),

"lastHeartbeatRecv": ISODate("2016-07-09T11:35:35.745Z"),

"pingMs": 0,

"configVersion": 1

}

"ok": 1

}

shard2:PRIMARY>

8、设置自动分片生效

目前搭建了mongodb配置服务器、路由服务器，各个分片服务器，不过应用程序连接到 mongos 路由服务器并不能使用分片机制，还需要在程序里设置分片配置，让分片生效。

# 串联路由服务器与分配副本集命令

db.runCommand( { addshard :"shard1/mongodb1:37017,mongodb1:37027,mongodb1:37037"});

db.runCommand( { addshard :"shard2/mongodb1:37018,mongodb1:37028,mongodb1:37038"});

# 执行过程

[mongodb@db_m1_slave_1 logs]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongolocalhost:40000/admin

MongoDB shell version: 3.0.3

connecting to: localhost:30000/admin

mongos> db.runCommand( { addshard :"shard1/mongodb1:37017,mongodb1:37027,mongodb1:37037"});

{ "shardAdded" :"shard1", "ok" : 1 }

mongos> db.runCommand( { addshard :"shard2/mongodb1:37018,mongodb1:37028,mongodb1:37038"});

{ "shardAdded" :"shard2", "ok" : 1 }

mongos>

9、查看分片服务器配置

db.runCommand({listshards : 1 });

命令输出结果

mongos> db.runCommand({listshards : 1});

{

"shards": [

{

"_id": "shard1",

"host": "shard1/mongodb1:37017,mongodb1:37027"

{

"_id": "shard2",

"host": "shard2/mongodb1:37018,mongodb1:37028"

}

"ok": 1

}

mongos>

PS：因为37037、37038是每个分片副本集的仲裁节点，所以在上面结果没有列出来。

查看均衡器，默认是打开的：

mongos> sh.getBalancerState();

true

mongos>

10、如何让录入的数据自动分片

目前配置服务、路由服务、分片服务、副本集服务都已经串联起来了，但我们的目的是希望插入数据，数据能够自动分片，那么还需要做如下工作：

连接在mongos上，准备让指定的数据库、指定的集合分片生效。

# 指定分片生效

db.runCommand( { enablesharding:"app"});

db.runCommand( { enablesharding:"im"});

db.runCommand( { enablesharding:"parking"});

db.runCommand( { enablesharding:"pv"});

db.runCommand( { enablesharding:"report"});

db.runCommand( { enablesharding:"screen"});

db.runCommand( { enablesharding:"search"});

db.runCommand( { enablesharding :"traffice"});

db.runCommand( { enablesharding:"wifi"});

执行过程如下：

mongos> db.runCommand( { enablesharding:"app"});

});

db.runCommand( { enablesharding:"search"});

db.runCommand( { enablesharding:"traffice"});

db.runCommand( { enablesharding:"wifi"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"im"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"parking"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"pv"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"report"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"screen"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"search"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"traffice"});

{ "ok" : 1 }

mongos> db.runCommand( { enablesharding:"wifi"});

{ "ok" : 1 }

mongos>

# 指定数据库里需要分片的集合和片键

db.runCommand( { shardcollection :"uba.table1",key : {id: 1} } )

我们设置uba的 table1 表需要分片，根据 id 自动分片到 shard1 ，shard2，shard3 上面去。要这样设置是因为不是所有 mongodb 的数据库和表都需要分片！

--1 app

mongos> use bg;

switched to db bg

mongos> db.card.save({x:1});

WriteResult({ "nInserted" : 1 })

mongos>

sh.enableBalancing("bg");

db.runCommand({enablesharding:"app"});

db.runCommand( { shardcollection :"app.download",key : {_id: 1} } );

执行过程如下：

mongos>db.runCommand({enablesharding:"app"});

{"ok" : 0, "errmsg" : "already enabled" }

mongos>use admin

switchedto db admin

mongos>db.runCommand({enablesharding:"app"});

{"ok" : 0, "errmsg" : "already enabled" }

mongos>db.runCommand( { shardcollection : "app.download",key : {_id: 1} } );

{"collectionsharded" : "app.download", "ok" : 1 }

mongos>

11、配置片键

需要进入admin数据库里面去执行配置命令：

use admin

db.runCommand({"shardcollection":"201.customer_user_mst",key:{"name":1}})

db.runCommand({"shardcollection":"201.user_attr_mst",key:{"name":1}})

db.runCommand({"shardcollection":"pv.pv1",key:{"_id":1}})

db.pv1.insert({ "_id" :ObjectId("4b8ed00a1d42d47b3afa3c41"), "x" : 999,"note" : "in db2" });

[原]mongodb 在单机上搭建分片集群 详细实战过程

<->本地搭建

1，下载安装：

2、在mongodb1上准备好3个config servers

3、启动mongos路由服务

4、开始准备shard servers

5、开始设置分片副本集shard1

6、开始设置分片副本集shard2

7、检查下副本状态：

8、设置自动分片生效

9、查看分片服务器配置

10、如何让录入的数据自动分片

11、配置片键

[原]mongodb 在单机上搭建分片集群详细实战过程