Using redis+protoStuff to implement database caching mechanism in JAVA

Posted by DrJonesAC2 on Thu, 04 Jul 2019 21:22:30 +0200

  1. Introduction to Redis
       Redis is a key-value storage system. Similar to Memcached, it supports relatively more value types of storage, including string (string), list (linked list), set (set) and Zset (ordered set). These data types support push/pop, add/remove, intersection, Union and difference sets, and richer operations, all of which are atomic. On this basis, redis supports sorting in various ways. Like memcached, data is cached in memory to ensure efficiency. The difference is that redis periodically writes the updated data to disk or writes the modification operation to the additional record file, and on this basis, master-slave synchronization is realized. 
    Redis is a high performance key-value database. The emergence of redis largely compensates for the insufficiency of keyvalue storage such as memcached, which can play a very good complementary role to relational databases in some occasions. It provides Python, Ruby, Erlang, PHP client, easy to use.
    Redis currently does not have a specific data type to store objects, but we can convert objects into binary files through object serialization for storage.
    JDK itself provides a serialization method for objects, that is, to implement serialization interface Serializable, and to use ObjectOutputStream to output binary streams, but this method has low performance and poor cross-linguistic.
    In fact, xml is also a serialization technology. The advantages of xml are good readability and expansibility, but its structure is relatively redundant, its parsing is complex and its performance is not high.
    We can also define the string format ourselves, which is more efficient for simple data formats, but has poor scalability and can not deal with complex objects.
    Google Protocol Buffer, also referred to as protobuf, is an efficient structured data storage format developed by Google. This serialization protocol is very portable and efficient, but protobuf requires us to write schema by ourselves, which is more troublesome.
    For protobuf, open source China has compiled a serialization format of dynamically generated schema. Its performance is not much different from protobuf, but it is more convenient to write schema without us. Here we use redis+protostuff to realize simple database caching.
    Jedis is Redis's official Java client Redis development kit. Here we use Jedis to connect to the Redis database.
    Jedis Download protostuff Download
package org.seckill.dao.cache;

import io.protostuff.LinkedBuffer;
import io.protostuff.ProtobufIOUtil;
import io.protostuff.runtime.RuntimeSchema;
import org.seckill.entity.Seckill;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Repository;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;

/**
 * Created by Administrator on 2017/4/29.
 * Seckill-->Objects for serialization
 */
@Repository
public class RedisDao {
    private final JedisPool jedisPool;
    private final Logger logger = LoggerFactory.getLogger(this.getClass());
    //Dynamic Generation of Seckill's schema
    private RuntimeSchema<Seckill> schema = RuntimeSchema.createFrom(Seckill.class);

    public RedisDao(String ip, int port) {
        //Constructing Method to Establish Jedis Connection Pool
        jedisPool = new JedisPool(ip, port);
    }

    public Seckill getSeckill(long seckillId) {
        //redis operation logic
        try {
            //Get the Jedis connection
            Jedis jedis = jedisPool.getResource();
            try {
                String key = "seckill:" + seckillId;
                //redis does not implement internal serialization
                //Get - > byte [] - > deserialization - > Object (Seckill)
                //Using custom serialization protostuff - > POJO object
                //Getting serialized byte arrays from Jedis
                byte[] bytes = jedis.get(key.getBytes());
                if (bytes != null) {
                    //Declare an empty object
                    Seckill seckill = schema.newMessage();
                    //Convert the byte array obtained in Redis into a Seckill object and assign it to seckill
                    ProtobufIOUtil.mergeFrom(bytes, seckill, schema);
                    //seckill is deserialized
                    return seckill;
                }
            } finally {
                //Close the connection
                jedis.close();
            }
        } catch (Exception e) {
            logger.error(e.getMessage(), e);
        }
        return null;
    }

    public String putSeckill(Seckill seckill) {
        //Set Object (seckill) --> serialization - > byte []
        try {
            Jedis jedis = jedisPool.getResource();
            try {
                String key = "seckill:" + seckill.getSeckillId();
                //Converting seckill objects into byte arrays through dynamically generated schema s
                //LinkedBuffer for caching larger objects
                byte[] bytes = ProtobufIOUtil.toByteArray(seckill, schema,
                        LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE));
                //Timeout cache
                //Redis stores data that can be permanent or time-bound
                //For database caching, the timeout mechanism is generally used to ensure the integrity of data caching and database data.
                int timeout = 60*60;
                String result = jedis.setex(key.getBytes(),timeout,bytes);
                return  result;
            } finally {
                jedis.close();
            }
        } catch (Exception e) {
            logger.error(e.getMessage(), e);
        }
        return null;
    }
}

Ideas:
  1. When looking up data, first look it up in the Redis cache.
  2. If you can't find it, turn to the database.
      2.1 Put the data found in the Redis cache
  3. Return the object found

Topics: Redis Jedis Database xml