Copy Set of MongoDB Learning

Posted by haixiao on Wed, 01 Dec 2021 23:18:55 +0100

Copy Set Principle

The role of replication sets

In order to achieve high availability of mongodb services, there is a master-slave relationship between replica sets

  • Quickly copy data to other nodes when data is written
  • Automatically select a new node to replace the primary node when it fails

With the above features, if we know redis, we can quickly associate Sentinel mode

In addition to the above two major features of data synchronization and fault node substitution, replication sets have the following roles

  • Data distribution: Distribute data to different areas to reduce latency in reading

    For example, the company has data centers in Beijing, Shanghai and Guangzhou, but only Beijing is a writable center, which can distribute data in different places to improve the reading efficiency of users in Shanghai and Guangzhou.

  • Read-Write Separation
  • Disaster tolerance in different places: Switch to different places quickly when the data center fails.

The structure of the replica set

The structure of the replica set is similar to that of other middleware.

A typical replica set consists of three or more nodes with votes:

  • Primary node: There is only one, responsible for writing and reading data, and voting during elections.
  • From Node: There are at least two responsible for data reading and voting.

Principle of data replication

  • When the data (primary node) is modified (added or deleted), its operations on the data are recorded in an oplog file.
  • Consistent with the data on the primary node, the slave node continuously obtains new oplog s from the primary node by opening a tailable cursor on the primary node and reproduces these operations on its own node.

Principle of primary node election

  1. Two or two nodes with voting rights send heartbeats to each other.
  2. Nodes are disconnected when five heartbeats are not received.
  3. If the primary node is disconnected, a new primary node is selected from the secondary node, but the disconnected secondary node will not participate in the election.
  4. Elections are based on the RAFT consistency algorithm, and the necessary condition for a successful election is that most voting nodes survive.
  5. A replica set can have up to 50 nodes, but a maximum of 7 nodes with voting rights.
  6. Nodes selected as primary nodes must be able to establish connections with most nodes, have newer oplog s, have higher priority (if any in the configuration)

Build a replica set

Here's how to create a mongodb replica set under linux

Create three directories

mkdir -p /data/mongodb/db{1,2,3}

configuration file

#Node 1
systemLog:
 destination: file
 path: /data/mongodb/db1/mongod.log #Log Path
 logAppend: true
storage:
 dbPath: /data/mongodb/db1 #data file
net:
 bindIp: 0.0.0.0    #Listen on all network cards for external network access
 port: 28017
replication:    #Copy set, if not a single node
 replSetName: rs0 #Copy Set Name
processManagement:
 fork: true #Make the process a stand-alone background process
#Node 2
systemLog:
 destination: file
 path: /data/mongodb/db2/mongod.log #Log Path
 logAppend: true
storage:
 dbPath: /data/mongodb/db2 #data file
net:
 bindIp: 0.0.0.0    #Listen on all network cards for external network access
 port: 28018
replication:    #Copy set, if not a single node
 replSetName: rs0 #Copy Set Name
processManagement:
 fork: true #Make the process a stand-alone background process
#Node 3
systemLog:
 destination: file
 path: /data/mongodb/db3/mongod.log #Log Path
 logAppend: true
storage:
 dbPath: /data/mongodb/db3 #data file
net:
 bindIp: 0.0.0.0    #Listen on all network cards for external network access
 port: 28019
replication:    #Copy set, if not a single node
 replSetName: rs0 #Copy Set Name
processManagement:
 fork: true #Make the process a stand-alone background process

Start process

mongod -f db1/mongod.conf
mongod -f db2/mongod.conf
mongod -f db3/mongod.conf

Configure replica sets

  1. Enter the primary node

    mongo --port 28017

  2. Set Primary Node

    rs.initiate()

    {
            "info2" : "no configuration specified. Using a default configuration for the set",
            "me" : "supman:28017",
            "ok" : 1
    }
  3. View node status

    rs.status()

    {
            "set" : "rs0",
            "date" : ISODate("2021-11-30T15:33:30.616Z"),
            "myState" : 1,
            "term" : NumberLong(1),
            "syncSourceHost" : "",
            "syncSourceId" : -1,
            "heartbeatIntervalMillis" : NumberLong(2000),
            "majorityVoteCount" : 1,
            "writeMajorityCount" : 1,
            "votingMembersCount" : 1,
            "writableVotingMembersCount" : 1,
            "optimes" : {
                    "lastCommittedOpTime" : {
                            "ts" : Timestamp(1638286404, 1),
                            "t" : NumberLong(1)
                    },
                    "lastCommittedWallTime" : ISODate("2021-11-30T15:33:24.656Z"),
                    "readConcernMajorityOpTime" : {
                            "ts" : Timestamp(1638286404, 1),
                            "t" : NumberLong(1)
                    },
                    "readConcernMajorityWallTime" : ISODate("2021-11-30T15:33:24.656Z"),
                    "appliedOpTime" : {
                            "ts" : Timestamp(1638286404, 1),
                            "t" : NumberLong(1)
                    },
                    "durableOpTime" : {
                            "ts" : Timestamp(1638286404, 1),
                            "t" : NumberLong(1)
                    },
                    "lastAppliedWallTime" : ISODate("2021-11-30T15:33:24.656Z"),
                    "lastDurableWallTime" : ISODate("2021-11-30T15:33:24.656Z")
            },
            "lastStableRecoveryTimestamp" : Timestamp(1638286364, 1),
            "electionCandidateMetrics" : {
                    "lastElectionReason" : "electionTimeout",
                    "lastElectionDate" : ISODate("2021-11-30T15:27:54.614Z"),
                    "electionTerm" : NumberLong(1),
                    "lastCommittedOpTimeAtElection" : {
                            "ts" : Timestamp(0, 0),
                            "t" : NumberLong(-1)
                    },
                    "lastSeenOpTimeAtElection" : {
                            "ts" : Timestamp(1638286074, 1),
                            "t" : NumberLong(-1)
                    },
                    "numVotesNeeded" : 1,
                    "priorityAtElection" : 1,
                    "electionTimeoutMillis" : NumberLong(10000),
                    "newTermStartDate" : ISODate("2021-11-30T15:27:54.637Z"),
                    "wMajorityWriteAvailabilityDate" : ISODate("2021-11-30T15:27:54.660Z")
            },
            "members" : [
                    {
                            "_id" : 0,
                            "name" : "supman:28017",
                            "health" : 1,
                            "state" : 1,
                            "stateStr" : "PRIMARY",
                            "uptime" : 1227,
                            "optime" : {
                                    "ts" : Timestamp(1638286404, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2021-11-30T15:33:24Z"),
                            "syncSourceHost" : "",
                            "syncSourceId" : -1,
                            "infoMessage" : "",
                            "electionTime" : Timestamp(1638286074, 2),
                            "electionDate" : ISODate("2021-11-30T15:27:54Z"),
                            "configVersion" : 1,
                            "configTerm" : 1,
                            "self" : true,
                            "lastHeartbeatMessage" : ""
                    }
            ],
            "ok" : 1,
            "$clusterTime" : {
                    "clusterTime" : Timestamp(1638286404, 1),
                    "signature" : {
                            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                            "keyId" : NumberLong(0)
                    }
            },
            "operationTime" : Timestamp(1638286404, 1)
    }

    Find the members.name field in the status information

    "name" : "supman:28017"
  4. Add slave node

    rs.add("supman:28018")

    {
            "ok" : 1,
            "$clusterTime" : {
                    "clusterTime" : Timestamp(1638286626, 1),
                    "signature" : {
                            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                            "keyId" : NumberLong(0)
                    }
            },
            "operationTime" : Timestamp(1638286626, 1)
    }

    rs.add("supman:28019")

    {
            "ok" : 1,
            "$clusterTime" : {
                    "clusterTime" : Timestamp(1638286660, 1),
                    "signature" : {
                            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                            "keyId" : NumberLong(0)
                    }
            },
            "operationTime" : Timestamp(1638286660, 1)
    }
  5. View status again

    {
            "set" : "rs0",
            "date" : ISODate("2021-11-30T15:44:29.903Z"),
            "myState" : 1,
            "term" : NumberLong(1),
            "syncSourceHost" : "",
            "syncSourceId" : -1,
            "heartbeatIntervalMillis" : NumberLong(2000),
            "majorityVoteCount" : 2,
            "writeMajorityCount" : 2,
            "votingMembersCount" : 3,
            "writableVotingMembersCount" : 3,
            "optimes" : {
                    "lastCommittedOpTime" : {
                            "ts" : Timestamp(1638287064, 1),
                            "t" : NumberLong(1)
                    },
                    "lastCommittedWallTime" : ISODate("2021-11-30T15:44:24.673Z"),
                    "readConcernMajorityOpTime" : {
                            "ts" : Timestamp(1638287064, 1),
                            "t" : NumberLong(1)
                    },
                    "readConcernMajorityWallTime" : ISODate("2021-11-30T15:44:24.673Z"),
                    "appliedOpTime" : {
                            "ts" : Timestamp(1638287064, 1),
                            "t" : NumberLong(1)
                    },
                    "durableOpTime" : {
                            "ts" : Timestamp(1638287064, 1),
                            "t" : NumberLong(1)
                    },
                    "lastAppliedWallTime" : ISODate("2021-11-30T15:44:24.673Z"),
                    "lastDurableWallTime" : ISODate("2021-11-30T15:44:24.673Z")
            },
            "lastStableRecoveryTimestamp" : Timestamp(1638287034, 1),
            "electionCandidateMetrics" : {
                    "lastElectionReason" : "electionTimeout",
                    "lastElectionDate" : ISODate("2021-11-30T15:27:54.614Z"),
                    "electionTerm" : NumberLong(1),
                    "lastCommittedOpTimeAtElection" : {
                            "ts" : Timestamp(0, 0),
                            "t" : NumberLong(-1)
                    },
                    "lastSeenOpTimeAtElection" : {
                            "ts" : Timestamp(1638286074, 1),
                            "t" : NumberLong(-1)
                    },
                    "numVotesNeeded" : 1,
                    "priorityAtElection" : 1,
                    "electionTimeoutMillis" : NumberLong(10000),
                    "newTermStartDate" : ISODate("2021-11-30T15:27:54.637Z"),
                    "wMajorityWriteAvailabilityDate" : ISODate("2021-11-30T15:27:54.660Z")
            },
            "members" : [
                    {
                            "_id" : 0,
                            "name" : "supman:28017",
                            "health" : 1,
                            "state" : 1,
                            "stateStr" : "PRIMARY",
                            "uptime" : 1886,
                            "optime" : {
                                    "ts" : Timestamp(1638287064, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2021-11-30T15:44:24Z"),
                            "syncSourceHost" : "",
                            "syncSourceId" : -1,
                            "infoMessage" : "",
                            "electionTime" : Timestamp(1638286074, 2),
                            "electionDate" : ISODate("2021-11-30T15:27:54Z"),
                            "configVersion" : 3,
                            "configTerm" : 1,
                            "self" : true,
                            "lastHeartbeatMessage" : ""
                    },
                    {
                            "_id" : 1,
                            "name" : "supman:28018",
                            "health" : 1,
                            "state" : 2,
                            "stateStr" : "SECONDARY",
                            "uptime" : 442,
                            "optime" : {
                                    "ts" : Timestamp(1638287064, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDurable" : {
                                    "ts" : Timestamp(1638287064, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2021-11-30T15:44:24Z"),
                            "optimeDurableDate" : ISODate("2021-11-30T15:44:24Z"),
                            "lastHeartbeat" : ISODate("2021-11-30T15:44:28.545Z"),
                            "lastHeartbeatRecv" : ISODate("2021-11-30T15:44:28.566Z"),
                            "pingMs" : NumberLong(0),
                            "lastHeartbeatMessage" : "",
                            "syncSourceHost" : "supman:28017",
                            "syncSourceId" : 0,
                            "infoMessage" : "",
                            "configVersion" : 3,
                            "configTerm" : 1
                    },
                    {
                            "_id" : 2,
                            "name" : "supman:28019",
                            "health" : 1,
                            "state" : 2,
                            "stateStr" : "SECONDARY",
                            "uptime" : 409,
                            "optime" : {
                                    "ts" : Timestamp(1638287064, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDurable" : {
                                    "ts" : Timestamp(1638287064, 1),
                                    "t" : NumberLong(1)
                            },
                            "optimeDate" : ISODate("2021-11-30T15:44:24Z"),
                            "optimeDurableDate" : ISODate("2021-11-30T15:44:24Z"),
                            "lastHeartbeat" : ISODate("2021-11-30T15:44:28.565Z"),
                            "lastHeartbeatRecv" : ISODate("2021-11-30T15:44:29.316Z"),
                            "pingMs" : NumberLong(0),
                            "lastHeartbeatMessage" : "",
                            "syncSourceHost" : "supman:28018",
                            "syncSourceId" : 1,
                            "infoMessage" : "",
                            "configVersion" : 3,
                            "configTerm" : 1
                    }
            ],
            "ok" : 1,
            "$clusterTime" : {
                    "clusterTime" : Timestamp(1638287064, 1),
                    "signature" : {
                            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                            "keyId" : NumberLong(0)
                    }
            },
            "operationTime" : Timestamp(1638287064, 1)
    }
  6. Configure Readable from Node

    Enter From Node

    mongo --port 28018 mongo --port 28019

    Set Readable From Node

    rs.slaveOk()

  7. Finally, add a user to the primary node, create a database and collection, and view the data from the node after inserting the data

Topics: Java Database JSON MongoDB nosql