[practical tutorial] deploy Redis applications on instances equipped with persistent memory

Posted by gershon on Sat, 29 Jan 2022 17:00:29 +0100

Introduction: instances equipped with persistent memory (such as re7p, r7p and re6p) provide a large CPU memory ratio. Redis applications running on such instances can greatly reduce the cost of single GiB memory. Taking some operating systems as examples, this paper introduces how to quickly deploy redis applications on such instances.

Instances equipped with persistent memory (such as re7p, r7p and re6p) provide a large CPU memory ratio. Redis applications running on such instances can greatly reduce the cost of single GiB memory. Taking some operating systems as examples, this paper introduces how to quickly deploy redis applications on such instances.

1, Background information

The steps for rapid deployment of Redis applications in this article are applicable to specific instance specifications and image versions. The requirements are as follows:

Example specification:

re7p: ecs.re7p.large,ecs.re7p.xlarge,ecs.re7p.2xlarge,ecs.re7p.16large,ecs.re7p.32xlarge
r7p: ecs.r7p.large,ecs.r7p.xlarge,ecs.r7p.2xlarge,ecs.r7p.16large,ecs.r7p.32xlarge
re6p: ecs.re6p-redis.large,ecs.re6p-redis.xlarge,ecs.re6p-redis.2xlarge,ecs.re6p-redis.4xlarge,ecs.re6p-redis.13xlarge

Description if ECS is required re6p-redis. 4XLARGE, you can click the link to submit the work order and get relevant help.

Mirror image:

Alibaba Cloud Linux 2
CentOS 7.6 and later
Ubuntu 18.10 and later
SUSE Linux 12 SP4 and later

2, Deploying Redis applications in Alibaba Cloud Linux 2

Alibaba Cloud Linux 2 has carried out special tuning for Redis applications. Compared with the community version operating system, the overall performance of Redis applications has been improved by more than 20%.

Alibaba Cloud Linux 2 has built-in yum source for Redis 6.0.5 and Redis 3.2.12. Execute yum install command to deploy Redis 6.0.5 and Redis 3.2.12. You can also manually deploy Redis applications of other versions. For specific operations, see deploying Redis applications in CentOS and deploying Redis applications in Ubuntu.

The configuration of relevant resources in this step is as follows:

Example specification: ECs re6p-redis. 2xlarge
Image: Alibaba Cloud Linux 2.1903 LTS 64 bit
Purchase persistent memory instances. For details, see creating an instance using the wizard. Note the following configuration:
Example: click the memory type classification under x86 computing architecture, and select ECs re6p-redis. Instance specification of 2xlarge.
Image: select Alibaba Cloud Linux 2.1903 LTS 64 bit.
Login instance. Please refer to the overview of connection mode for specific operation.
Deploy Redis 6.0.5 or Redis 3.2.12 as needed.
Redis 6.0.5 deployment:

yum install -y alinux-release-experimentals && \
yum install -y redis-6.0.5

Redis 3.2.12 deployment:

yum install -y alinux-release-experimentals && \
yum install -y redis-3.2.12

Start Redis application (configure the normal memory and persistent memory capacity used by default). Example commands are as follows:
The recommended ratio of ordinary memory to persistent memory is 1:4.

export MEMKIND_DAX_KMEM_NODES=1 && \
redis-server /etc/redis.conf --port 8369 --memory-alloc-policy ratio --dram-pmem-ratio 1 4 --hashtable-on-dram yes --daemonize yes --logfile /tmp/redis_8369.log --protected-mode no --bind 0.0.0.0

You can also customize the ratio of ordinary memory to persistent memory, and reserve some ordinary memory to run other applications. For example, the ratio is 1:16 and 34 GiB of memory is allocated (including 2 GiB of ordinary memory and 32 GiB of persistent memory).

export MEMKIND_DAX_KMEM_NODES=1 && \
redis-server /etc/redis.conf --port 8369 --memory-alloc-policy ratio --dram-pmem-ratio 1 16 --maxmemory 34G

3, Deploying Redis applications in CentOS

The configuration of relevant resources and software in this step is as follows:

Example specification: ECs re6p-redis. 2xlarge
Image: CentOS 7.6
Redis: Redis 4.0.14
memkind: memkind 1.10.1-rc2

Explain that this step involves starting from GitHub( https://github.com/ )When downloading resources from relevant addresses, please ensure that the download is successful before subsequent operations. If the download fails, please repeat the relevant commands until the download succeeds.

Purchase persistent memory instances. For details, see creating an instance using the wizard. Note the following configuration:
Example: click the memory type classification under x86 computing architecture, and select ECs re6p-redis. Instance specification of 2xlarge.
Mirroring: select CentOS 7.6 64 bit.
Login instance. Please refer to the overview of connection mode for specific operation.

Prepare the compilation environment.

export MEMKIND_DAX_KMEM_NODES=1 && \
yum -y install numactl-devel.x86_64 && \
yum -y groupinstall 'Development Tools'
prepare Redis 4.0.14 Source code.
wget https://github.com/redis-io/redis/archive/4.0.14.tar.gz && \
wget https://github.com/redis/redis/compare/4.0.14...tieredmemdb:4.0.14-devel.diff -O redis_4.0.14_diff_tieredmemdb.patch && \
tar xzvf 4.0.14.tar.gz && \
cd redis-4.0.14 && \
git apply --ignore-whitespace ../redis_4.0.14_diff_tieredmemdb.patch

Note patch is used to enable persistent memory. Different Redis versions use different patches. For more information, see downloading a patch that enables persistent memory.

Prepare memkind source code. Memkind is a memory management tool used to allocate and manage persistent memory.
Download memkind source code.

wget https://github.com/memkind/memkind/archive/v1.10.1-rc2.tar.gz && \
tar xzvf v1.10.1-rc2.tar.gz && \
mv memkind-1.10.1-rc2/* ./deps/memkind

Optional: adjust makefile.

Note you can run ldd --version to check the glibc version first. If the glibc version is equal to or higher than 2.17, you can skip the following operations and directly start compiling Redis.

cd ./deps/memkind && \
wget https://github.com/memKeyDB/memKeyDB/wiki/files/0001-Use-secure_getenv-when-possible.patch && \
git apply --ignore-whitespace 0001-Use-secure_getenv-when-possible.patch && \
cd /root/redis-4.0.14

Note: if the patch download fails, there is no need to include CD when executing the command again/ deps/memkind && \.

Compile and install Redis.

make clean && \
make distclean && \
make MALLOC=memkind -j 4 && \
make install

redis-server /root/redis-4.0.14/redis.conf --port 8369 --memory-alloc-policy ratio --dram-pmem-ratio 1 4 --hashtable-on-dram yes --daemonize yes --logfile /tmp/redis_8369.log --protected-mode no --bind 0.0.0.0

redis-server /root/redis-4.0.14/redis.conf --port 8369 --memory-alloc-policy ratio --dram-pmem-ratio 1 16 --maxmemory 34G

4, Deploy Redis application in Ubuntu

The configuration of relevant resources and software in this step is as follows:

Example specification: ECs re6p-redis. 2xlarge
Image: Ubuntu 20.04
Redis: Redis 6.2.5
memkind: memkind 1.10.1-rc2

Purchase persistent memory instances. For details, see creating an instance using the wizard. Note the following configuration:
Example: click the memory type classification under x86 computing architecture, and select ECs re6p-redis. Instance specification of 2xlarge.
Image: choose Ubuntu 20.04 64 bit.
Login instance. Please refer to the overview of connection mode for specific operation.
Prepare the compilation environment.

export MEMKIND_DAX_KMEM_NODES=1 && \
apt update && \
apt -y install git && \
apt install -y libnuma-dev && \
apt install -y numactl

Prepare Redis 6.2.5 source code.

wget https://download.redis.io/releases/redis-6.2.5.tar.gz && \
wget https://github.com/redis/redis/compare/6.2.5...tieredmemdb:6.2.5-devel.diff -O redis_6.2.5_diff_tieredmemdb.patch && \
tar xzf redis-6.2.5.tar.gz && \
cd redis-6.2.5 && \
git apply --ignore-whitespace ../redis_6.2.5_diff_tieredmemdb.patch

Note patch is used to enable persistent memory. Different Redis versions use different patches. For more information, see downloading a patch that enables persistent memory.

Prepare memkind source code. Memkind is a memory management tool used to allocate and manage persistent memory.
Download memkind source code.

wget https://github.com/memkind/memkind/archive/v1.10.1-rc2.tar.gz && \
tar xzvf v1.10.1-rc2.tar.gz && \
mv memkind-1.10.1-rc2/* ./deps/memkind/

Adjust the makefile.

Note you can run ldd --version to check the glibc version first. If the glibc version is equal to or higher than 2.17, you can skip the following operations and directly start compiling Redis.

cd ./deps/memkind && \
wget --no-check-certificate https://github.com/memKeyDB/memKeyDB/wiki/files/0001-Use-secure_getenv-when-possible.patch && \
git apply --ignore-whitespace 0001-Use-secure_getenv-when-possible.patch && \
cd /root/redis-6.2.5

Note: if the patch download fails, there is no need to include CD when executing the command again/ deps/memkind && \.

Compile and install Redis.

make clean && \
make distclean && \
make MALLOC=memkind -j 4 && \
make install

redis-server /root/redis-6.2.5/redis.conf --port 8369 --memory-alloc-policy ratio --dram-pmem-ratio 1 4 --hashtable-on-dram yes --daemonize yes --logfile /tmp/redis_8369.log --protected-mode no --bind 0.0.0.0

redis-server /root/redis-6.2.5/redis.conf --port 8369 --memory-alloc-policy ratio --dram-pmem-ratio 1 16 --maxmemory 34G

5, Download the patch that enables persistent memory

Replace the download address in the example command and the corresponding version number in the file name. For example, the command to download the patch applicable to Redis 6.2.5 is as follows:

wget https://github.com/redis/redis/compare/6.2.5...tieredmemdb:6.2.5-devel.diff -O redis_6.2.5_diff_tieredmemdb.patch

The currently supported patch download addresses are as follows:

Redis 6.0

Redis 5.0

Redis 4.0

Redis 3.0

https://github.com/redis/redi...

Original link
This article is the original content of Alibaba cloud and cannot be reproduced without permission.

Topics: Database Redis

Programmer Think