Barman is an open-source excellent PG database backup tool based on Python, which integrates backup and recovery. It can make remote / local backup for single / multiple PG databases to enhance the security of business data and provide reliable help for DBA s to recover databases.
Barman advantages
- It supports the backup of multiple versions of PG. it is known that it supports all versions of PG 8.2 and later
- Full backup of database and real-time backup of WAL can effectively meet the demand of RPO ≈ 0
- It supports incremental backup of database file level, which depends on rysnc and hard link
- After PG 9.2, the backup mode of concurrent (consistent) mode is supported to avoid data inconsistency in data files under exclusive mode.
- Provide WAL transmission backup in stream replication mode, reduce the delay of WAL backup, ensure WAL redundant backup, and avoid the loss of WAL on database nodes that cannot be restored.
- Support compressed transmission and bandwidth limitation
- Provide reliable monitoring information. Used to collect reports on Barman backup and service status.
- The backup set directory that can be centrally managed makes it easy for users to obtain corresponding information such as backup data files and WAL files.
- Customized backup set retention policies to meet business needs.
- Starting from version 2.2, it supports parallel backup and recovery to accelerate the process of backup and recovery.
- Support local and remote recovery
Barman deployment
Barman installation
At present, barman is integrated into the yum source of PG. you can obtain the installation package through the yum source provided on the official website of PG for installation:
https://www.postgresql.org/download/linux/redhat/
Take PG version 11 as an example
yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
You can get the information of various versions of barman in yum source
yum provides barman
yum install -y barman
Barman configuration
Take stream replication as an example (this method does not rely on SSH communication between Barman and PostgreSQL):
Target database:
Users needed to create barman
psql=# create user barman password '<encrypt_password>' superuser; psql=# create user streaming_barman password '<encrypt_password>' superuser;
Data parameter and access rule configuration
Postgresql.conf
# The WAL level setting shall at least ensure the level above archive (v9.5 is replica) wal_level = replica # Enable Archive Mode archive_mode = on # If you use stream replication to transfer WAL to barman, archive_command is not required archive_command = ' ' # Maximum number of WAL sender processes, total > wal archive + streaming / logical replication max_wal_senders = 5 # Keep at least the number of WAL files to prevent the WAL from being removed prematurely by the service node wal_keep_segments = 64
Pg_hba.conf
# Access control allows barman users to access the database with the privileges of superuser and replication respectively host all barman 0.0.0.0/0 md5 host replication streaming_barman 0.0.0.0/0 md5
You can verify the success of stream replication by using the following command
psql "dbname=postgres port=5432 replication=database user=streaming_barman password=<encrypt_password>" -c "IDENTIFY_SYSTEM;"
The following information can be obtained normally:
systemid | timeline | xlogpos | dbname ---------------------------------+-----------+-----------------+---------- 6722702146844265873 | 3 | A/A4001AE8 | postgres (1 row)
Barman service configuration
Global configuration information
[barman] # Directory where barman backup sets are stored barman_home = barman barman_user = barman # Location of barman service log output log_file = var/log/barman/barman.log # Level of service log output log_level = INFO # For PG9 It is recommended to use consistent backup after 6 backup_options = concurrent_backup # Turn on backup compression compression = gzip minimum_redundancy = 0 # Backup set, data in backup set, retention policy of WAL file retention_policy = RECOVERY WINDOW OF 1 WEEK retention_policy_mode = auto wal_retention_policy = main # Specify the directory where the backup target configuration database is located configuration_files_directory = etc/barman.conf.d check_timeout = 5
Individual (target database backup) configuration information
[<target_database>] description = "<target_database> master database(Streaming Only)" # Barman connects characters through conninfo conninfo = host=<target_database> port=5432 user=barman dbname=postgres password=<encrypt_password> # Establish the connection string of stream replication and the archiving method of WAL stream replication streaming_conninfo = host=target_database port=5432 user=streaming_barman dbname=postgres password=<encrypt_password> # Enable archiving in streaming copy mode streaming_archiver = on # Control the streaming of WAL through slots slot_name = streaming_barman # Recommended backup method for database backup_method = postgres backup_options = concurrent_backup # The location where WAL transferred through streaming copy is stored streaming_wals_directory=/barman/<target_database>/streaming minimum_redundancy = 2 retention_policy = RECOVERY WINDOW OF 1 DAYS # Specify the installation location of the Barman server client path_prefix=/usr/pgsql-10/bin
Backup target library
Verification before backup
#Verify the status of the backup service configuration and ensure that all statuses are OK before starting the backup
barman check <target_database>
Server <target_database>:
PostgreSQL: OK
is_superuser: OK
PostgreSQL streaming: OK
wal_level: OK
replication slot: OK
directories: OK
retention policy settings: OK
backup maximum age: OK (no last_backup_maximum_age provided)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: OK (have 3 backups, expected at least 3)
pg_basebackup: OK
pg_basebackup compatible: OK
pg_basebackup supports tablespaces mapping: OK
archive_mode: OK
archive_command: OK
continuous archiving: OK
pg_receivexlog: OK
pg_receivexlog compatible: OK
receive-wal running: OK
archiver errors: OK
Initiate backup
$ barman backup <target_database> # Join crontab crontab -e 0 23 * * * usr/bin/barman backup <target_database> > barman/<target_database>/base/barman.log 2>&1
Restore target library
It is best to keep the data file directory of the target database before recovery to avoid failure of recovery.
Based on full recovery
$ barman recover <target_database> 20191127T220002 ${PGDATA}
PITR based recovery
$ barman recover --target-time "2019-11-27 15:00:00" <target_database> 20191127T220002 ${PGDATA}
Remote recovery target machine
$ barman recover --remote-ssh-command "ssh postgres@<target_host>" --target-time "2019-11-27 15:00:00" <target_database> 20191127T220002 ${PGDATA}
Daily maintenance order
#List all backup service information (- - minimal parameter only displays the service name, not the description) $ barman list-server #Lists all configuration information for a single backup service $ barman show-server <target_database> #Lists the backup set information for a single backup service $ barman list-backup <target_database> # Viewing information about a single backup set $ barman show-backup <target_database> <backup_id> Check backup service configuration status $ barman check <target_database> Check the real-time information of the backup service $ barman status <target_database> with Json The format lists the details of all backup services and backup sets $ barman diagnose
For more information, see barman --help
Troubleshooting
PostgreSQL: Failed
barman is not configured with a user with superuser permission to access the PG server
psql=# create user barman password '<encrypt_password>' superuser;
perhaps
$ createuser -s -p 5432 -Upostgres -d postgres barman
continuous archiving: Failed
Check whether the target WAL archive is normal
receive-wal running: Failed
You can try to refresh the reception status of the WAL by rebuilding the replication slot
barman receive-wal --stop <target_database> barman receive-wal --reset <target_database> barman receive-wal --create-slot <target_database> barman switch-wal <target_database>
WAL archive: FAILED (please make sure WAL shipping is setup)
Forcibly switch the WAL and archive it to the Barman server
$ barman switch-wal --force --archive <target_database> The WAL file 000000020000001400000096 has been closed on server '<target_database>' Waiting for the WAL file 000000020000001400000096 from server '<target_database>' (max: 30 seconds) Processing xlog segments from streaming for fcden1pgs01_master 000000020000001400000096
reference resources:
Official documents of the second quadrant http://docs.pgbarman.org/release/2.11/index.html