Introduction to Barman backup scheme

Posted by snakebit on Mon, 28 Feb 2022 15:40:54 +0100

Barman is an open-source excellent PG database backup tool based on Python, which integrates backup and recovery. It can make remote / local backup for single / multiple PG databases to enhance the security of business data and provide reliable help for DBA s to recover databases.

Barman advantages

  1. It supports the backup of multiple versions of PG. it is known that it supports all versions of PG 8.2 and later
  2. Full backup of database and real-time backup of WAL can effectively meet the demand of RPO ≈ 0
  3. It supports incremental backup of database file level, which depends on rysnc and hard link
  4. After PG 9.2, the backup mode of concurrent (consistent) mode is supported to avoid data inconsistency in data files under exclusive mode.
  5. Provide WAL transmission backup in stream replication mode, reduce the delay of WAL backup, ensure WAL redundant backup, and avoid the loss of WAL on database nodes that cannot be restored.
  6. Support compressed transmission and bandwidth limitation
  7. Provide reliable monitoring information. Used to collect reports on Barman backup and service status.
  8. The backup set directory that can be centrally managed makes it easy for users to obtain corresponding information such as backup data files and WAL files.
  9. Customized backup set retention policies to meet business needs.
  10. Starting from version 2.2, it supports parallel backup and recovery to accelerate the process of backup and recovery.
  11. Support local and remote recovery

Barman deployment

Barman installation
At present, barman is integrated into the yum source of PG. you can obtain the installation package through the yum source provided on the official website of PG for installation:
Take PG version 11 as an example
yum install -y
You can get the information of various versions of barman in yum source
yum provides barman
yum install -y barman

Barman configuration

Take stream replication as an example (this method does not rely on SSH communication between Barman and PostgreSQL):

Target database:
Users needed to create barman

psql=# create user barman password '<encrypt_password>' superuser;
psql=# create user streaming_barman password '<encrypt_password>' superuser;

Data parameter and access rule configuration

# The WAL level setting shall at least ensure the level above archive (v9.5 is replica)
wal_level = replica
# Enable Archive Mode
archive_mode = on
# If you use stream replication to transfer WAL to barman, archive_command is not required
archive_command = ' '
# Maximum number of WAL sender processes, total > wal archive + streaming / logical replication
max_wal_senders = 5
# Keep at least the number of WAL files to prevent the WAL from being removed prematurely by the service node
wal_keep_segments = 64


           # Access control allows barman users to access the database with the privileges of superuser and replication respectively
host    all    barman                 md5
host    replication     streaming_barman               md5

You can verify the success of stream replication by using the following command

psql "dbname=postgres port=5432 replication=database user=streaming_barman password=<encrypt_password>" -c "IDENTIFY_SYSTEM;"

The following information can be obtained normally:

   systemid               | timeline |     xlogpos    |  dbname  
 6722702146844265873 |        3    | A/A4001AE8 | postgres
(1 row)

Barman service configuration
Global configuration information

# Directory where barman backup sets are stored
barman_home = barman
barman_user = barman
# Location of barman service log output
log_file = var/log/barman/barman.log
# Level of service log output
log_level = INFO
# For PG9 It is recommended to use consistent backup after 6
backup_options = concurrent_backup
# Turn on backup compression
compression = gzip
minimum_redundancy = 0
# Backup set, data in backup set, retention policy of WAL file
retention_policy = RECOVERY WINDOW OF 1 WEEK
retention_policy_mode = auto
wal_retention_policy = main
# Specify the directory where the backup target configuration database is located
configuration_files_directory = etc/barman.conf.d
check_timeout = 5

Individual (target database backup) configuration information

description =  "<target_database> master database(Streaming Only)"
# Barman connects characters through conninfo
conninfo = host=<target_database> port=5432 user=barman dbname=postgres password=<encrypt_password>
# Establish the connection string of stream replication and the archiving method of WAL stream replication
streaming_conninfo = host=target_database port=5432 user=streaming_barman dbname=postgres password=<encrypt_password>
# Enable archiving in streaming copy mode
streaming_archiver = on
# Control the streaming of WAL through slots
slot_name = streaming_barman
# Recommended backup method for database
backup_method = postgres
backup_options = concurrent_backup
# The location where WAL transferred through streaming copy is stored
minimum_redundancy = 2
retention_policy = RECOVERY WINDOW OF 1 DAYS
# Specify the installation location of the Barman server client

Backup target library
Verification before backup

#Verify the status of the backup service configuration and ensure that all statuses are OK before starting the backup
           barman check <target_database>
        Server <target_database>:
       PostgreSQL: OK
        is_superuser: OK
        PostgreSQL streaming: OK
        wal_level: OK
        replication slot: OK
        directories: OK
        retention policy settings: OK
        backup maximum age: OK (no last_backup_maximum_age provided)
        compression settings: OK
        failed backups: OK (there are 0 failed backups)
        minimum redundancy requirements: OK (have 3 backups, expected at least 3)
        pg_basebackup: OK
        pg_basebackup compatible: OK
        pg_basebackup supports tablespaces mapping: OK
        archive_mode: OK
        archive_command: OK
        continuous archiving: OK
        pg_receivexlog: OK
        pg_receivexlog compatible: OK
        receive-wal running: OK
        archiver errors: OK

Initiate backup

$ barman backup <target_database>
# Join crontab
crontab -e
0 23 * * * usr/bin/barman backup <target_database> > barman/<target_database>/base/barman.log 2>&1

Restore target library
It is best to keep the data file directory of the target database before recovery to avoid failure of recovery.

Based on full recovery

$ barman recover <target_database> 20191127T220002 ${PGDATA}

PITR based recovery

$ barman recover --target-time "2019-11-27 15:00:00" <target_database> 20191127T220002 ${PGDATA}

Remote recovery target machine

$ barman recover --remote-ssh-command "ssh postgres@<target_host>" --target-time "2019-11-27 15:00:00" <target_database> 20191127T220002 ${PGDATA}

Daily maintenance order

#List all backup service information (- - minimal parameter only displays the service name, not the description)
$ barman list-server
#Lists all configuration information for a single backup service
$ barman show-server <target_database>
#Lists the backup set information for a single backup service
$ barman list-backup <target_database>
# Viewing information about a single backup set
$ barman show-backup <target_database> <backup_id>
Check backup service configuration status
$ barman check <target_database>
Check the real-time information of the backup service
$ barman status <target_database>
with Json The format lists the details of all backup services and backup sets
$ barman diagnose

For more information, see barman --help


PostgreSQL: Failed
barman is not configured with a user with superuser permission to access the PG server

psql=# create user barman password '<encrypt_password>' superuser;


$ createuser -s -p 5432 -Upostgres -d postgres barman

continuous archiving: Failed
Check whether the target WAL archive is normal

receive-wal running: Failed
You can try to refresh the reception status of the WAL by rebuilding the replication slot

barman receive-wal --stop <target_database>
barman receive-wal --reset <target_database>
barman receive-wal --create-slot <target_database>
barman switch-wal <target_database>

WAL archive: FAILED (please make sure WAL shipping is setup)
Forcibly switch the WAL and archive it to the Barman server

$ barman switch-wal --force --archive <target_database>
The WAL file 000000020000001400000096 has been closed on server '<target_database>'
Waiting for the WAL file 000000020000001400000096 from server '<target_database>' (max: 30 seconds)
Processing xlog segments from streaming for fcden1pgs01_master

reference resources:

Official documents of the second quadrant

Topics: Database PostgreSQL