Introduction to backup and recovery of mongodb

Posted by raku on Sun, 05 Jan 2020 22:00:48 +0100

1. Introduction to installation environment:

Installation environment for demo mongo:
Binary installation package mongoDB3.6.16
Grant mongodb the Super Administrator account:

mongo --host 127.0.0.1 --port 6068
db.createUser({user: 'root', pwd:'TdLLQ6689', roles:[{role: 'root', db: 'admin'}]});
use admin
db.auth("root","TdLLQ6689")

Database building table simulation data:

use dbtest001
db.chenji.insert({"name":"Floret","grade":"second grade","Gender":"male","hobby":"Study"})
use dbtest002
db.xiangmu.insert({"name":"Floret","grade":"second grade","Gender":"male","hobby":"Study"})
db.mumu.insert({"name":"Floret","grade":"second grade","Gender":"male","hobby":"Study"})

Tip: This demo backup and recovery mainly uses super account root for demo

2. Introduction to the mongodump backup command:

2.1 Syntax and parameter descriptions:

mongodump -h dbhost -u xxx -p xxx -d dbname -o dbdirectory
Parameter description:
-h Indicates the IP of the database host
--port indicates the port of the database
-u indicates the user name of the database
-p indicates the password for the database
-d indicates the name of the database
-c indicates the name of the collection
-o Specifies the name of the file to be exported
-q specifies the filter conditions for the exported data
--Name of authenticationDatabase validation data
--Compress on gzip backup
--oplog use oplog for taking a point-in-time snapshot

Create a backup directory of the presentation in advance:
mkdir full #Full backup
mkdir single_db #single library backup
Table backup in mkdir single_db_col #library
mkdir single_db.gzip #library backup compression
Table backup compression in mkdir single_db_col.gzip #library

2.2 Full backup:

[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u root -p TdLLQ6689  -o /root/full
2020-01-04T18:13:18.271+0800    writing admin.system.users to 
2020-01-04T18:13:18.272+0800    done dumping admin.system.users (7 documents)
2020-01-04T18:13:18.272+0800    writing admin.system.version to 
2020-01-04T18:13:18.273+0800    done dumping admin.system.version (2 documents)
2020-01-04T18:13:18.273+0800    writing dbtest001.chenji to 
2020-01-04T18:13:18.273+0800    writing dbtest002.mumu to 
2020-01-04T18:13:18.273+0800    writing dbtest002.xiangmu to 
2020-01-04T18:13:18.274+0800    done dumping dbtest002.mumu (1 document)
2020-01-04T18:13:18.274+0800    done dumping dbtest001.chenji (2 documents)
2020-01-04T18:13:18.290+0800    done dumping dbtest002.xiangmu (2 documents)
[root@localhost ~]# cd full/
[root@localhost full]# ls
admin  dbtest001  dbtest002

2.3 Single library backup:

Backup with super account root failed:

[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u root -p TdLLQ6689 -d dbtest001 -o /root/single_db
2020-01-04T18:22:20.087+0800    Failed: error connecting to db server: server returned error on SASL authentication step: Authentication failed.

The solutions are as follows:

First method: add--authenticationDatabase admin

[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin  -d dbtest001 -o /root/single_db
2020-01-04T18:37:46.926+0800    writing dbtest001.chenji to 
2020-01-04T18:37:46.927+0800    done dumping dbtest001.chenji (2 documents)
[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin  -d dbtest002 -o /root/single_db
2020-01-04T18:37:54.253+0800    writing dbtest002.xiangmu to 
2020-01-04T18:37:54.253+0800    writing dbtest002.mumu to 
2020-01-04T18:37:54.254+0800    done dumping dbtest002.xiangmu (2 documents)
2020-01-04T18:37:54.254+0800    done dumping dbtest002.mumu (1 document)

[root@localhost ~]# tree /root/single_db
/root/single_db
├── dbtest001
│   ├── chenji.bson
│   └── chenji.metadata.json
└── dbtest002
    ├── mumu.bson
    ├── mumu.metadata.json
    ├── xiangmu.bson
    └── xiangmu.metadata.json

2 directories, 6 files

Second method: backup for separate library authorization read and write permissions

[root@localhost ~]# mongo --host 127.0.0.1 --port 6068
MongoDB shell version v3.6.16
connecting to: mongodb://127.0.0.1:6068/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("93f83e64-71ca-4b1d-b599-508b7fba8722") }
MongoDB server version: 3.6.16
> use dbtest001
>db.auth("root","TdLLQ6689")

> use dbtest001
>db.createUser({user: 'backupuser', pwd:'TdLLQ6689', roles:[{role: 'readWrite', db: 'dbtest001'}]});
> use dbtest002
>db.createUser({user: 'backupuser', pwd:'TdLLQ6689', roles:[{role: 'readWrite', db: 'dbtest002'}]});
[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u backupuser -p TdLLQ6689  -d dbtest001 -o /root/single_db
2020-01-04T18:24:15.243+0800    writing dbtest001.chenji to 
2020-01-04T18:24:15.244+0800    done dumping dbtest001.chenji (2 documents)
[root@localhost ~]# ll /root/single_db/dbtest001/chenji.*
-rw-r--r-- 1 root root 192 1 April 18:24 /root/single_db/dbtest001/chenji.bson
-rw-r--r-- 1 root root 130 1 April 18:24 /root/single_db/dbtest001/chenji.metadata.json
[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u backupuser -p TdLLQ6689  -d dbtest002 -o /root/single_db
2020-01-04T18:28:39.523+0800    writing dbtest002.xiangmu to 
2020-01-04T18:28:39.523+0800    writing dbtest002.mumu to 
2020-01-04T18:28:39.524+0800    done dumping dbtest002.xiangmu (2 documents)
2020-01-04T18:28:39.524+0800    done dumping dbtest002.mumu (1 document)
[root@localhost ~]# cd  /root/single_db
[root@localhost single_db]# ls
dbtest001  dbtest002

2.4 Backup forms specified in the single library:

[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin  -d dbtest002 -c xiangmu -o /root/single_db_col
2020-01-04T18:34:30.260+0800    writing dbtest002.xiangmu to 
2020-01-04T18:34:30.261+0800    done dumping dbtest002.xiangmu (2 documents)
[root@localhost ~]# ll /root/single_db_col/dbtest002/
//Total usage 8
-rw-r--r-- 1 root root 192 1 April 18:34 xiangmu.bson
-rw-r--r-- 1 root root 131 1 April 18:34 xiangmu.metadata.json

2.5 Library table backup compression:

2.5.1 library backup compression:

 [root@localhost ~]# mongodump -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin  -d dbtest002 -o /root/single_db --gzip
2020-01-04T18:44:28.558+0800    writing dbtest002.xiangmu to 
2020-01-04T18:44:28.559+0800    writing dbtest002.mumu to 
2020-01-04T18:44:28.560+0800    done dumping dbtest002.xiangmu (2 documents)
2020-01-04T18:44:28.561+0800    done dumping dbtest002.mumu (1 document)
[root@localhost ~]# tree single_db
single_db
└── dbtest002
    ├── mumu.bson.gz
    ├── mumu.metadata.json.gz
    ├── xiangmu.bson.gz
    └── xiangmu.metadata.json.gz

1 directory, 4 files

Table backup compression in Library 2.5.2:

[root@localhost ~]# mongodump -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin  -d dbtest001 -o /root/single_db_col.gzip  --gzip
2020-01-04T18:41:08.459+0800    writing dbtest001.chenji to 
2020-01-04T18:41:08.460+0800    done dumping dbtest001.chenji (2 documents)
[root@localhost dbtest001]# ll /root/single_db_col.gzip/dbtest001
//Total usage 8
-rw-r--r-- 1 root root 124 1 April 18:41 chenji.bson.gz
-rw-r--r-- 1 root root 132 1 April 18:41 chenji.metadata.json.gz

3. Introduction of mongorestore Restore Backup Data Command

3.1 Introduction to syntax and parameters:

mongorestore -h <hostname><:port> -d dbname <path>

--host <:port>, -h <:port>:mongoDB server address, default:localhost:27017
-h Indicates the IP of the database host
-u indicates the user name of the database
-p indicates the password for the database
-d indicates the name of the database## database to use when restoring from a BSON file
-c indicates the name of the collection##collection to use when restoring from a BSON file
-o Specifies the name of the file to be exported
-q specifies the filter conditions for the exported data
--Name of authenticationDatabase validation data
--Compress on gzip backup
--oplog use oplog for taking a point-in-time snapshot
--Drop out the previous collection when drop restores

3.2 Restore to library using full backup:

Tip: Restoring directly to online libraries using full backup will prompt for key conflicts, restoring with the parameter--drop drops out the previous collection

[root@localhost ~]# mongorestore -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin   ./full/
2020-01-04T18:55:43.409+0800    preparing collections to restore from
2020-01-04T18:55:43.414+0800    reading metadata for dbtest001.chenji from full/dbtest001/chenji.metadata.json
2020-01-04T18:55:43.414+0800    restoring dbtest001.chenji from full/dbtest001/chenji.bson
2020-01-04T18:55:43.415+0800    reading metadata for dbtest002.xiangmu from full/dbtest002/xiangmu.metadata.json
2020-01-04T18:55:43.415+0800    reading metadata for dbtest002.mumu from full/dbtest002/mumu.metadata.json
2020-01-04T18:55:43.415+0800    restoring dbtest002.xiangmu from full/dbtest002/xiangmu.bson
2020-01-04T18:55:43.416+0800    restoring dbtest002.mumu from full/dbtest002/mumu.bson
2020-01-04T18:55:43.418+0800    error: multiple errors in bulk operation:
  - E11000 duplicate key error collection: dbtest002.xiangmu index: _id_ dup key: { : ObjectId('5e0f16191083b09e85237cb2') }
  - E11000 duplicate key error collection: dbtest002.xiangmu index: _id_ dup key: { : ObjectId('5e0f161d1083b09e85237cb3') }

2020-01-04T18:55:43.418+0800    no indexes to restore
2020-01-04T18:55:43.418+0800    finished restoring dbtest002.xiangmu (2 documents)
2020-01-04T18:55:43.436+0800    error: multiple errors in bulk operation:
  - E11000 duplicate key error collection: dbtest001.chenji index: _id_ dup key: { : ObjectId('5e0f150150fe973d9e9051f4') }
  - E11000 duplicate key error collection: dbtest001.chenji index: _id_ dup key: { : ObjectId('5e0f150750fe973d9e9051f5') }

2020-01-04T18:55:43.436+0800    no indexes to restore
2020-01-04T18:55:43.436+0800    finished restoring dbtest001.chenji (2 documents)
2020-01-04T18:55:43.436+0800    error: E11000 duplicate key error collection: dbtest002.mumu index: _id_ dup key: { : ObjectId('5e0f162d1083b09e85237cb4') }
2020-01-04T18:55:43.436+0800    no indexes to restore
2020-01-04T18:55:43.436+0800    finished restoring dbtest002.mumu (1 document)
2020-01-04T18:55:43.436+0800    restoring users from full/admin/system.users.bson
2020-01-04T18:55:43.447+0800    done

Delete the database and restore it:

> use dbtest001
switched to db dbtest001
> db.dropDatabase()
{ "dropped" : "dbtest001", "ok" : 1 }
> use dbtest002
switched to db dbtest002
> db.dropDatabase()
{ "dropped" : "dbtest002", "ok" : 1 }

[root@localhost ~]# mongorestore -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin  ./full/
2020-01-04T18:59:29.633+0800    preparing collections to restore from
2020-01-04T18:59:29.639+0800    reading metadata for dbtest001.chenji from full/dbtest001/chenji.metadata.json
2020-01-04T18:59:29.639+0800    reading metadata for dbtest002.mumu from full/dbtest002/mumu.metadata.json
2020-01-04T18:59:29.640+0800    reading metadata for dbtest002.xiangmu from full/dbtest002/xiangmu.metadata.json
2020-01-04T18:59:29.646+0800    restoring dbtest002.xiangmu from full/dbtest002/xiangmu.bson
2020-01-04T18:59:29.648+0800    no indexes to restore
2020-01-04T18:59:29.648+0800    finished restoring dbtest002.xiangmu (2 documents)
2020-01-04T18:59:29.664+0800    restoring dbtest001.chenji from full/dbtest001/chenji.bson
2020-01-04T18:59:29.677+0800    restoring dbtest002.mumu from full/dbtest002/mumu.bson
2020-01-04T18:59:29.690+0800    no indexes to restore
2020-01-04T18:59:29.690+0800    finished restoring dbtest001.chenji (2 documents)
2020-01-04T18:59:29.690+0800    no indexes to restore
2020-01-04T18:59:29.690+0800    finished restoring dbtest002.mumu (1 document)
2020-01-04T18:59:29.690+0800    restoring users from full/admin/system.users.bson
2020-01-04T18:59:29.701+0800    done

Add parameter--drop for recovery:

[root@localhost ~]# mongorestore -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin --drop ./full/
2020-01-04T19:03:04.854+0800    preparing collections to restore from
2020-01-04T19:03:04.859+0800    reading metadata for dbtest001.chenji from full/dbtest001/chenji.metadata.json
2020-01-04T19:03:04.864+0800    restoring dbtest001.chenji from full/dbtest001/chenji.bson
2020-01-04T19:03:04.866+0800    no indexes to restore
2020-01-04T19:03:04.866+0800    finished restoring dbtest001.chenji (2 documents)
2020-01-04T19:03:04.879+0800    reading metadata for dbtest002.xiangmu from full/dbtest002/xiangmu.metadata.json
2020-01-04T19:03:04.880+0800    reading metadata for dbtest002.mumu from full/dbtest002/mumu.metadata.json
2020-01-04T19:03:04.884+0800    restoring dbtest002.xiangmu from full/dbtest002/xiangmu.bson
2020-01-04T19:03:04.898+0800    restoring dbtest002.mumu from full/dbtest002/mumu.bson
2020-01-04T19:03:04.899+0800    no indexes to restore
2020-01-04T19:03:04.899+0800    finished restoring dbtest002.xiangmu (2 documents)
2020-01-04T19:03:04.915+0800    no indexes to restore
2020-01-04T19:03:04.915+0800    finished restoring dbtest002.mumu (1 document)
2020-01-04T19:03:04.915+0800    restoring users from full/admin/system.users.bson
2020-01-04T19:03:04.928+0800    done

3.3 Restore the backed up library table files to the specified library tables:

-d indicates the name of the database and the backup database file must be in BSON format 

[root@localhost ~]# mongorestore -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin -d dbtest002 ./full/dbtest002/
2020-01-05T07:36:24.293+0800    the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-01-05T07:36:24.293+0800    building a list of collections to restore from full/dbtest002 dir
2020-01-05T07:36:24.294+0800    reading metadata for dbtest002.xiangmu from full/dbtest002/xiangmu.metadata.json
2020-01-05T07:36:24.295+0800    reading metadata for dbtest002.mumu from full/dbtest002/mumu.metadata.json
2020-01-05T07:36:24.299+0800    restoring dbtest002.mumu from full/dbtest002/mumu.bson
2020-01-05T07:36:24.303+0800    no indexes to restore
2020-01-05T07:36:24.303+0800    finished restoring dbtest002.mumu (1 document)
2020-01-05T07:36:24.319+0800    restoring dbtest002.xiangmu from full/dbtest002/xiangmu.bson
2020-01-05T07:36:24.321+0800    no indexes to restore
2020-01-05T07:36:24.321+0800    finished restoring dbtest002.xiangmu (2 documents)
2020-01-05T07:36:24.321+0800    done

-c indicates the name of the collection and the backup database must have a collection data file in BSON format 

[root@localhost ~]# mongorestore -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin -d dbtest002 -c mumu  ./full/dbtest002/mumu.bson 
2020-01-05T07:45:53.367+0800    checking for collection data in full/dbtest002/mumu.bson
2020-01-05T07:45:53.368+0800    reading metadata for dbtest002.mumu from full/dbtest002/mumu.metadata.json
2020-01-05T07:45:53.373+0800    restoring dbtest002.mumu from full/dbtest002/mumu.bson
2020-01-05T07:45:53.436+0800    no indexes to restore
2020-01-05T07:45:53.436+0800    finished restoring dbtest002.mumu (1 document)
2020-01-05T07:45:53.436+0800    done
[root@localhost ~]# 
[root@localhost ~]# mongorestore -h 127.0.0.1:6068 -u root -p TdLLQ6689 --authenticationDatabase admin -d dbtest002 -c mumu  ./full/dbtest002/xiangmu.bson 
2020-01-05T07:46:35.517+0800    checking for collection data in full/dbtest002/xiangmu.bson
2020-01-05T07:46:35.519+0800    reading metadata for dbtest002.mumu from full/dbtest002/xiangmu.metadata.json
2020-01-05T07:46:35.519+0800    restoring dbtest002.mumu from full/dbtest002/xiangmu.bson
2020-01-05T07:46:35.582+0800    no indexes to restore
2020-01-05T07:46:35.582+0800    finished restoring dbtest002.mumu (2 documents)
2020-01-05T07:46:35.582+0800    done

Topics: Linux JSON Database MongoDB snapshot

Hot Topics

©2024 Programmer Think