In actual development, we often use cache. Its core idea is to record the results of process data reuse. When a program needs to perform complex and resource consuming operations, we generally save the running result in the cache, and read it out of the cache the next time we need the result. Caching applies to data that does not change often, or even never change. Changing data is not suitable for caching. For example, GPS data of aircraft flight should not be cached, otherwise you will get wrong data.

1, Cache type

There are three types of cache:

1. In memory cache: in-process cache. The cache terminates when the process terminates.
2. Persistent in-process cache: backup the cache outside the process memory. The backup location may be in the file, in the database, or in other locations. If the process restarts, the cache is not lost.
3. Distributed cache: multiple machines share the cache. If one server saves a cache entry, other servers can also use it.

Tip: in this article, we only talk about in-process caching.

2, Realize

Next, we implement in-process caching step by step by caching the avatar. In earlier versions of. NET, we implemented caching in a simple way, as shown in the following code:

public class NaiveCache<TItem>
{
    Dictionary<object, TItem> _cache = new Dictionary<object, TItem>();
    public TItem GetOrCreate(object key, Func<TItem> createItem)
    {
        if (!_cache.ContainsKey(key))
        {
            _cache[key] = createItem();
        }
        return _cache[key];
    }
}

The method of using it is as follows:

var _avatarCache = new NaiveCache<byte[]>();
var myAvatar = _avatarCache.GetOrCreate(userId, () => _database.GetAvatar(userId));

When obtaining the user's Avatar, only the first request will really request the database. After the request is sent to the database, the avatar data will be saved in the process memory, and all subsequent requests for avatars will be extracted from memory, thus saving time and resources. But for many reasons, this solution is not the best. First, it is not thread safe. Exceptions may occur when multiple threads are used. In addition, the cached data will remain in memory forever. Once the memory is cleaned up for various reasons, the data saved in memory will be lost. The disadvantages of this solution are summarized below:

1. The cache occupies a large amount of memory, resulting in insufficient memory, exception and crash;
2. High memory consumption will lead to memory pressure, and the workload of garbage collector will exceed its due level, damaging performance;
3. If the data changes, the cache needs to be refreshed

In order to solve the above problems, the cache framework must have an expulsion strategy to delete items from the cache according to the algorithm logic. Common expulsion policies are as follows:

1. Expiration policy: delete items from the cache after a specified time;
2. If an item is not accessed within the specified time period, the sliding expiration policy removes the item from the cache. For example, we set the expiration time to 1 minute. As long as we use the item every 30 seconds, it will remain in the cache. But if you don't use it for more than a minute, it will be deleted.
3. Size limit policy: limit the size of cache memory.

Now we can improve our code according to the above strategies, and we can use the solutions provided by Microsoft. Microsoft has two solutions, providing two NuGet packages for caching. Microsoft recommends using Microsoft.Extensions.Caching.Memory because it can be integrated with Asp.NET Core and can be easily injected into Asp.NET Core. The sample code for using Microsoft.Extensions.Caching.Memory is as follows:

public class SimpleMemoryCache<TItem>
{
    private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions());
    public TItem GetOrCreate(object key, Func<TItem> createItem)
    {
        TItem cacheEntry;
        if (!_cache.TryGetValue(key, out cacheEntry))
        {
            cacheEntry = createItem();
            _cache.Set(key, cacheEntry);
        }
        return cacheEntry;
    }
}

The method of using it is as follows:

var _avatarCache = new SimpleMemoryCache<byte[]>();
var myAvatar = _avatarCache.GetOrCreate(userId, () => _database.GetAvatar(userId));

First, this is a thread safe implementation that can be safely called from multiple threads at once. Secondly, MemoryCache allows all expulsion policies to be added. The following example is IMemoryCache with eviction policy:

public class MemoryCacheWithPolicy<TItem>
{
    private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions()
    {
        SizeLimit = 1024
    });
    public TItem GetOrCreate(object key, Func<TItem> createItem)
    {
        TItem cacheEntry;
        if (!_cache.TryGetValue(key, out cacheEntry))
        {
            cacheEntry = createItem();
            var cacheEntryOptions = new MemoryCacheEntryOptions()
             .SetSize(1)
                .SetPriority(CacheItemPriority.High)
                .SetSlidingExpiration(TimeSpan.FromSeconds(2))
                .SetAbsoluteExpiration(TimeSpan.FromSeconds(10));
            _cache.Set(key, cacheEntry, cacheEntryOptions);
        }
        return cacheEntry;
    }
}

1. SizeLimit is added to MemoryCacheOptions. This adds a cache size based policy to our cache container. Mixed village size has no units. We need to set the size on each cache entry;
2. We can use. SetPriority() to set the level of cache to be deleted when the size limit is reached. The levels are Low, Normal, High and NeverRemove;
3. SetSlidingExpiration(TimeSpan.FromSeconds(2)) sets the sliding expiration time to two seconds. If an item is not accessed within two seconds, it will be deleted;
4. Set absolute expiration (timespan. Fromseconds (10)) sets the absolute expiration time to 10 seconds, and the item will be deleted within 10 seconds.

Do you think this realization will be all right? In fact, he still has problems:

1. Although the cache size limit can be set, the cache does not actually monitor GC pressure.
2. When multiple threads request the same project at the same time, the request will not wait for the first to complete, and the project will be created multiple times. For example, if the avatar is being cached, it takes 5 seconds to get the avatar from the database. In 3 seconds after the first request, another request will get the avatar. It will check whether the Avatar has been cached. At this time, if the avatar is not cached, it will also start accessing the database.

Let's solve the two problems mentioned above: First, with regard to GC stress, we can use a variety of techniques and heuristics to monitor GC stress. The second problem is relatively easy to solve. It can be realized by using a MemoryCache:

public class WaitToFinishMemoryCache<TItem>
{
    private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions());
    private ConcurrentDictionary<object, SemaphoreSlim> _locks = new ConcurrentDictionary<object, SemaphoreSlim>();

    public async Task<TItem> GetOrCreate(object key, Func<Task<TItem>> createItem)
    {
        TItem cacheEntry;

        if (!_cache.TryGetValue(key, out cacheEntry))
        {
            SemaphoreSlim mylock = _locks.GetOrAdd(key, k => new SemaphoreSlim(1, 1));

            await mylock.WaitAsync();
            try
            {
                if (!_cache.TryGetValue(key, out cacheEntry))
                {
                    cacheEntry = await createItem();
                    _cache.Set(key, cacheEntry);
                }
            }
            finally
            {
                mylock.Release();
            }
        }
        return cacheEntry;
    }
}

Usage:

var _avatarCache = new WaitToFinishMemoryCache<byte[]>();
var myAvatar = await _avatarCache.GetOrCreate(userId, async () => await _database.GetAvatar(userId));

This implementation locks the creation of the project, which is key specific. If we are waiting to get Zhang San's Avatar, we can still get the cache of Li Si's Avatar on another thread_ Locks stores all locks. Because regular locks are not applicable to async and await, we need to use SemaphoreSlim. The above implementation has some overhead and can only be used if:

1. When the creation time of the project has some cost;
2. When a project takes a long time to create;
3. When you must ensure that each key creates an item.

TIP: caching is a very powerful pattern, but it is also dangerous and has its own complexity. Too much cache will lead to GC pressure, and too little cache will lead to performance problems.