redis pipeline mass storage

Posted by contex on Sat, 09 Nov 2019 17:33:12 +0100

pipeline is a way for redis to submit in batch, that is, to establish a connection for multiple command operations and send them to redis for execution. In terms of performance, it will be much better than a single circular submission.

Performance comparison

Circular single submission

The code is as follows:

    public void testOne() {
        long begin = System.currentTimeMillis();
        for (int i = 0; i < 10000; i++) {
            redisTemplate.boundValueOps(i + "a").set(i);
        }
        long end = System.currentTimeMillis();
        System.out.println("Time spent connecting each time:" + (end - begin) + "Millisecond");
    }

Time consuming:

Connection time each time: 2954 MS

One commit using pipeline

Code

public void testPipeline() {
        long begin = System.currentTimeMillis();
        redisTemplate.executePipelined(new RedisCallback<Long>() {
            public Long doInRedis(RedisConnection connection) throws DataAccessException {
                connection.openPipeline();
                for (int i = 0; i < 10000; i++) {
                    byte[] key = ("pipeline" + i ).getBytes();
                    byte[] value = ("pipeline" + i ).getBytes();
                    connection.set(key, value);
                }
                //connection.closePipeline();
                return null;
            }
        });
        long end = System.currentTimeMillis();
        System.out.println("One connection time:" + (end - begin) + "Millisecond");
    }

Time consuming:

One connection time: 87 MS

conclusion

The performance improvement is obvious.
In addition, if you use the pipeline of jedis directly, you will find it is faster.

Attention points

Byte type parameter

Note that after using pipeline, we need to use the native redis interface in the callback method, most of which are byte type parameters. We need to convert to get byte stream, and then store or read.

How to read the accessed content

When using redisTemplate to read the key accessed through pipeline, there will be a Null problem. This is caused by the different methods of key value serialization. To avoid this situation, we can set the redisTemplate as follows

        redisTemplate.setKeySerializer(new StringRedisSerializer());
        redisTemplate.setValueSerializer(new StringRedisSerializer());

You can go through it later
redisTemplate.boundValueOps("pipeline0").get()
It's easy to read the content stored in the pipeline.

Topics: Redis Jedis