mongoDB cluster

introduce

Solve data fragmentation and prevent data loss
Production environment needs to be deployed by scraping shards + replica sets

Component

• route: provide entry, do not store data
• configserver: stores metadata information. It is recommended to use replica set
• shardserver: data storage service, store real data, maybe use replica set

Dependency relationship

• When inserting data, you need to know which shardsrv partition to insert data from configsrv
• When users get data, they need to know which shardsrv segment the data is stored in from configsrv

Cluster building

• Use the same mongodb binary
• Modify the corresponding configuration to build a shard cluster

Cluster resource planning

Configserver: 280172801828019 three port building
route: 270172701827019 three port building
Shardserver: 29017290182901929020 four port building

configsvr configuration

configuration file

• 28017280182801928020 just modify the port and path in the configuration

systemLog:
  destination: file
  logAppend: true
  path: /data/mongodb/28017/mongodb.log
storage:
  dbPath: /data/mongodb/28017/
  journal:
    enabled: true
processManagement:
  fork: true
net:
  port: 28017
  bindIp: 0.0.0.0
replication:
  replSetName: copySet
sharding:
  clusterRole: configsvr

Startup service

[root@centos7-node1 ~]# /usr/local/mongodb/bin/mongod -f /data/mongodb/28017/mongodb.conf 
[root@centos7-node2 ~]# /usr/local/mongodb/bin/mongod -f /data/mongodb/28018/mongodb.conf
[root@centos7-node3 ~]# /usr/local/mongodb/bin/mongod -f /data/mongodb/28019/mongodb.conf

Configure replica sets

[root@centos7-node1 ~]# /usr/local/mongodb/bin/mongo localhost:28017
>config={ _id:"copySet",
  configsvr: true,
   members: [
     {_id:0,host:"192.168.56.11:28017"},
     {_id:1,host:"192.168.56.12:28018"},
     {_id:2,host:"192.168.56.13:28019"}
 ]
}
> rs.initiate(config)
> rs.ststus()

router configuration

router is the entrance of mongodb and does not store any data

configuration file

• Modify the three ports of 270172701827019

systemLog:
  destination: file
  logAppend: true
  path: /data/mongodb/27017/mongodb.log
processManagement:
  fork: true
net:
  port: 27017
  bindIp: 0.0.0.0
sharding:
  configDB: copySet/192.168.56.11:28017,192.168.56.12:28018,192.168.56.13:28019

Startup service

mongos start

[root@centos7-node1 ~]# /usr/local/mongodb/bin/mongos -f /data/mongodb/27017/mongodb.conf
[root@centos7-node2 ~]# /usr/local/mongodb/bin/mongos -f /data/mongodb/27018/mongodb.conf 
[root@centos7-node3 ~]# /usr/local/mongodb/bin/mongos -f /data/mongodb/27019/mongodb.conf 
[root@centos7-node2 ~]# /usr/local/mongodb/bin/mongo 127.0.0.1:27018    #Test login

Configuration of shardsvr

Data role

configuration file

29017290182901929020 four ports can be built
Data role
29017, 29018 data role data1
29019, 29020 data role data2

systemLog:
  destination: file
  logAppend: true
  path: /data/mongodb/29017/mongodb.log
storage:
  dbPath: /data/mongodb/29017/
  journal:
    enabled: true
processManagement:
  fork: true
net:
  port: 29017
  bindIp: 0.0.0.0
replication:
  replSetName: data1
sharding:
  clusterRole: shardsvr

systemLog:
  destination: file
  logAppend: true
  path: /data/mongodb/29019/mongodb.log
storage:
  dbPath: /data/mongodb/29019/
  journal:
    enabled: true
processManagement:
  fork: true
net:
  port: 29019
  bindIp: 0.0.0.0
replication:
  replSetName: data2
sharding:
  clusterRole: shardsvr

Service startup

/usr/local/mongodb/bin/mongod -f /data/mongodb/29017/mongodb.conf
/usr/local/mongodb/bin/mongod -f /data/mongodb/29018/mongodb.conf
/usr/local/mongodb/bin/mongod -f /data/mongodb/29019/mongodb.conf
/usr/local/mongodb/bin/mongod -f /data/mongodb/29020/mongodb.conf
29017#### To configure
> config = {
        "_id" : "data1",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "192.168.56.11:29017"
                },
                {
                        "_id" : 1,
                        "host" : "192.168.56.12:29018"
                }
        ]
}
> rs.initiate(config) 
29019#### To configure
{
        "_id" : "data2",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "192.168.56.13:29019"
                },
                {
                        "_id" : 1,
                        "host" : "192.168.56.14:29020"
                }
        ]
}
> rs.initiate(config)

The use of mongodb sharding

Add data fragmentation to cluster

• Pay attention to the operation on the router cluster node

[root@centos7-node1 ~]# /usr/local/mongodb/bin/mongo 127.0.0.1:27017
mongos> sh.status()      #State query
mongos> sh.addShard("data1/192.168.56.11:29017,192.168.56.12:29018")    #Add data fragment 1
mongos> sh.addShard("data2/192.168.56.13:29019,192.168.56.14:29020")   #Add data fragment 2

• Data operations are all on router

mongos> use test    
mongos> for(i=1;i<1000;i++){
... db.myuser.insert({name:"myuser"+i,age:i})
... }
mongos> sh.status()     #You can see which data segment the data is stored in
mongos> db.dropDatabase()    #Delete data

• For the collection of a database, hash partition storage is used, and the partition storage will assign two data roles to the same collection
  • This allocates data to two data slices

mongos> use admin
mongos> db.runCommand({enablesharding:"test"})
mongos> db.runCommand({shardcollection:"test.myuser",key:{_id:"hashed"}})
mongos> use test    #Write data
mongos> for(i=1;i<=1000;i++){
... db.myuser.insert({name:"myuser"+i,age:i})
... }
WriteResult({ "nInserted" : 1 })
mongos> db.myuser.count()
1000

Be careful

• If the data partition is hung up, the data will be lost
• To open the sharding of the database
• If one router and configsvr is hung up, it will have no impact on the cluster