In-Memory Caching .NET Core Concepts

Introduction

Before we define caching, let's try to give some ideas on the expectation when discussing caching.

To most back-end developers caching is popular due to its performance improvement and scalability of the application. Just imagine a web application that is highly responsive and with a great user experience while being used by thousands of multiple concurrent users with no network timeouts. You might be thinking, is that possible? The answer is yes; this is where the caching mechanisms come into the picture.

What's Caching?

We can define caching as the storage of frequently used data in a temporary location for quick access in the future. As a result, it will drastically improve your application's performance and avoid unnecessary database hits. So try to imagine what would happen once cached.

Note: Caching implementation depends on the situation or the problem; it is also highly recommended always to have a fallback mechanism in case your caching fails.

Types of Cache

Within the context of ASP.NET, it supports in-memory caching and distributed caching.

Let's try to see define each type.

In Memory Caching

In-Memory caching is mainly used in lightweight applications. Moreover, it practically stores data on the application server's memory. The only disadvantage is when the server restarts or crashes for some reason, data will be lost.

Distributed Caching

It mainly stores data using an external service that multiple application servers can share. Moreover, it can drastically improve the performance and scalability of an application if using a cloud service to host the application. Furthermore, we can use 3rd party tools like Redis, NCache, SQL Server Caching, etc.

Implementing an In-Memory Cache

I. Add the Microsoft.Extensions.Caching.Memory

Within your project, we install the Microsoft.Extensions.Caching.Memory package.

You can edit your .csproj file and add the sample below.

<ItemGroup>
      <PackageReference 
          Include="Microsoft.Extensions.Caching.Memory" Version="6.0.1"/>
</ItemGroup>

Or we can install it via .NET CLI.

dotnet add package Microsoft.Extensions.Caching.Memory --version 6.0.1

II. Add the AddMemoryCache method to the Service Collection.

In the Program.cs file, you can add the AddMemoryCache method; see the sample code below.

var builder = WebApplication.CreateBuilder(args);

var services = builder.Services;

//Add the IMemoryCache
services.AddMemoryCache();

Once it is ready, IMemoryCache it can now be called inside a controller.

III. Create  a Dummy Product Repository Data

Let's create dummy data where a specific repository will return a product list of information. Nothing fancy about the code, but it does the job of generating a dummy product list.

namespace InMemoryCachingWebAPI;

public interface IProductRepository<T>
{
    List<T> GetProducts();
}

public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
}

public class ProductRepository : IProductRepository<Product>
{
    public List<Product> GetProducts()
    {
        return new List<Product>
        {
            new Product { Id = 1, Name = "ABC"},
            new Product { Id = 2, Name = "DEF"},
            new Product { Id = 3, Name = "FGH"},
            new Product { Id = 4, Name = "IJK"},
            new Product { Id = 5, Name = "LMN"},
        };
    }
}

Another step is setup a singleton on IProductRepository before we create an endpoint.

services.AddSingleton<IProductRepository<Product>, ProductRepository>();

IV. Create an Endpoint That Depends on IMemoryCache

Before we see the code of the endpoint, let's show the services needed to be registered because we'll need them at runtime.

//Add the IMemoryCache
services.AddLogging();
services.AddMemoryCache();
services.AddSingleton<IProductRepository<Product>, ProductRepository>();

Now, let's see the endpoint code.

app.MapGet("/products", ([FromServices]IMemoryCache memoryCache, 
    [FromServices]IProductRepository<Product> productRepository,
    ILoggerFactory loggerFactory) =>
{
}

From the code sample, we have shown that we have created an endpoint where IMemoryCache , IProductRepository<Product> and ILoggerFactory are registered services where these parameters are bound to their specific instances, which is resolved from this request /products.

V. Explain Code Inside the Endpoint

Let's try to see how we can add something to the cache first, then let's build the entire code later.

app.MapGet("/products", ([FromServices]IMemoryCache memoryCache, 
    [FromServices]IProductRepository<Product> productRepository,
    ILoggerFactory loggerFactory) =>
{
    const string productListKey = "productList";
    
    var products = productRepository.GetProducts();

    var cacheEntryOptions = new MemoryCacheEntryOptions
    {
         SlidingExpiration = TimeSpan.FromSeconds(60),
         AbsoluteExpiration = DateTime.Now.AddSeconds(3600),
         Priority = CacheItemPriority.Normal,
         Size = 1024
    };
    /*method equivalents
            1. Sliding Expiration
            SlidingExpiration = TimeSpan.FromSeconds(60),
            cacheEntryOptions.SetSlidingExpiration(TimeSpan.FromSeconds(60));
            2. AbsoluteExpiration 
            AbsoluteExpiration = DateTime.Now.AddSeconds(3600),
            cacheEntryOptions.SetAbsoluteExpiration(TimeSpan.FromSeconds(60));
            3. Priority 
            cacheEntryOptions.SetPriority(CacheItemPriority.Normal);    
     */

    memoryCache.Set(productListKey, products, cacheEntryOptions);
    }
});

MemoryCacheEntryOptions

From the code above, you'll see the MemoryCacheEntryOptions which is the object responsible for defining the options needed inside the IMemoryCache entry.

Sliding Expiration

SlidingExpiration - it helps the cache determine how long a specific entry can be inactive before removing it. The only issue with it  (SlidingExpiration) is that it won't expire if we keep accessing the cache. In other words, it resets or extends the life span, but if it has not been used during a time, it will expire. See the method equivalent; use the code sample below.

cacheEntryOptions.SetSlidingExpiration(TimeSpan.FromSeconds(60));

Absolute Expiration

AbsoluteExpiration - this one helps us by ensuring that the cache entry expires irrespective of whether it is still active. See the method equivalent; use the code sample below.

cacheEntryOptions.SetAbsoluteExpiration(TimeSpan.FromSeconds(3600));

Note: A good caching technique used by many outstanding backend developers is the combination of SlidingExpiration and AbsoluteExpiration.

Priority

Priority - This helps us to set the priority of the cached object. The Normal is the default, but of course, changing to other states, such as Low, Normal, High and NeverRemove is up to our current needs.

  public enum CacheItemPriority
  {
    Low,
    Normal,
    High,
    NeverRemove,
  }

See the method equivalent when setting the Priority. It is the cacheEntryOptions.SetPriority(CacheItemPriority.Normal); method.

Complete Sample Code of the /product EndPoint

app.MapGet("/products", ([FromServices]IMemoryCache memoryCache, 
    [FromServices]IProductRepository<Product> productRepository,
    ILoggerFactory loggerFactory) =>
{
    const string productListKey = "productList";
    var logger = loggerFactory.CreateLogger("Caching Start");
    List<Product> products = new();
    
    logger.LogInformation("Fetching the product sample.");

    if (memoryCache.TryGetValue(productListKey, out products))
    {
        logger.LogInformation("Product list found in cache");
    }
    else
    {
        logger.LogInformation("Product list not found in cache. Fetching data.");
        
        products = productRepository.GetProducts();

        var cacheEntryOptions = new MemoryCacheEntryOptions
        {
            SlidingExpiration = TimeSpan.FromSeconds(60),
            AbsoluteExpiration = DateTime.Now.AddSeconds(3600),
            Priority = CacheItemPriority.Normal,
            Size = 1024
        };
        
        /*method equivalents
            1. Sliding Expiration
            SlidingExpiration = TimeSpan.FromSeconds(60),
            cacheEntryOptions.SetSlidingExpiration(TimeSpan.FromSeconds(60));
            2. AbsoluteExpiration 
            AbsoluteExpiration = DateTime.Now.AddSeconds(3600),
            cacheEntryOptions.SetAbsoluteExpiration(TimeSpan.FromSeconds(60));
            3. Priority 
            cacheEntryOptions.SetPriority(CacheItemPriority.Normal);    
        */
        
        memoryCache.Set(productListKey, products, cacheEntryOptions);
    }

    return products;
});

See the output below.

TryGetValue and Get

Both TryGetValue and Get returns an entry from the cache, but they behave and are used differently. Let's see them below.

TryGetValue - this method is used to check whether a particular cache entry exists. It will return false if null, otherwise true. Moreover, once the return value isn't null, the return value will be assigned to the out variable.

Just like from our example, if the TryGetValue saw that productList does exist or was previously set, it will be assigned to the products variable.

 const string productListKey = "productList";
 var logger = loggerFactory.CreateLogger("Caching Start");
 List<Product> products = new();
    
 if (memoryCache.TryGetValue(productListKey, out products))
 {
        logger.LogInformation("Product list found in cache");
 }

Get - this method is straightforward to implement, pass the key, and it will return null when it wasn't set previously; otherwise, the expected type. Although it is more elegant to use the TryGetValue.

Set

Set - this method creates the specified entry in the cache. Moreover, it also overrides an existing cache entry.

Let's see an example below.

app.MapGet("/products/2",   ([FromServices]IMemoryCache memoryCache, 
                            [FromServices]IProductRepository<Product> productRepository,
                            ILoggerFactory loggerFactory ) =>
{
    var logger = loggerFactory.CreateLogger("Showing Caching Set Method Start");
    
    //Create a new cache entry
    memoryCache.Set("12345", "Hello my value 1");
    logger.LogInformation($"'12345' value is  = {memoryCache.Get("12345")}");
    
    //Let's override the cache entry 
    memoryCache.Set("12345", "Hello my value 2");
    logger.LogInformation($"'12345' value is  = {memoryCache.Get("12345")}");

});

For the output logs:

info: Showing Caching Set Method Start[0]
      '12345' value is  = Hello my value 1
info: Showing Caching Set Method Start[0]
      '12345' value is  = Hello my value 2

Size Limit

Size - when this is set, it tells the cache instance that we have defined a unit of cache entries that the cache can hold. Moreover, new incoming cache entries will be ignored once this is set and the cache entry hits its limits.

Let's try to show an example of limiting the size of a cache entry.

app.MapGet("/products/1",([FromServices]IMemoryCache memoryCache, 
                                        [FromServices]IProductRepository<Product> productRepository,
                                        ILoggerFactory loggerFactory ) =>
{
    var logger = loggerFactory.CreateLogger("Showing Caching Size Start");
    
    //Let's define the size limit of the cache
    var memoryCacheOptions = new MemoryCacheOptions
    {
        SizeLimit = 4
    };

    var cache = new MemoryCache(memoryCacheOptions);

    //Let's define the size of each cache entry
    var sizeOptions = new MemoryCacheEntryOptions().SetSize(2);

    //let's just get cache instance size at runtime so we can monitor
    var cacheSizeInstance = cache.GetType()
        .GetProperty("Size", BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.GetField);

    for (var i = 1; i < 3; i++)
    {
        //adding HelloWorld{i}
        logger.LogInformation(
            $"Adding HelloWorld{i}. Cache Count: {cache.Count}, Cache Size: {cacheSizeInstance?.GetValue(cache) ?? 0}");
    
        cache.Set($"HelloWorld{i}", "I love C#", sizeOptions);
    
        logger.LogInformation($"'HelloWorld{i}' value is  = {cache.Get($"HelloWorld{i}")}");
    
        logger.LogInformation(
            $"Added HelloWorld{i}. Cache Count: {cache.Count}, Cache Size: {cacheSizeInstance?.GetValue(cache) ?? 0}");
        //added  HelloWorld{i}
    }
    
    //let's try to add another cache entry after having a size of 4
  
    logger.LogInformation(
        $"Current Cache. Cache Count: {cache.Count}, Cache Size: {cacheSizeInstance?.GetValue(cache) ?? 0}");
    
    cache.Set("HelloWorld3", "I love JetBrains",sizeOptions);

    logger.LogInformation($"'HelloWorld3' value is  = {cache.Get($"HelloWorld3") ?? "NULL"}");
    
    logger.LogInformation(
        $"Cache Count: {cache.Count}, Cache Size: {cacheSizeInstance?.GetValue(cache) ?? 0}");
    
});

From the example above, we have seen that to set the size limit; we need to create the MemoryCacheOptions.

//Let's define the size limit of the cache
var memoryCacheOptions = new MemoryCacheOptions
{
	SizeLimit = 4
};

Now, once the cache object is ready, we need also to set the size of the cache entry.

//Let's define the size of each cache entry
var sizeOptions = new MemoryCacheEntryOptions().SetSize(2);

Every time a new entry enters the cache, it will accumulate its size until it reaches its size limit. Once it reaches the size limit, the cache doesn't allow making a new entry.

That's why when it (HelloWorld2)enters the cache, and it already reached the size limit. That's when we tried to add a new cache, which HelloWorld3 is no longer added in this case.

Let's see the output logs:

info: Showing Caching Size Start[0]
      Adding HelloWorld1. Cache Count: 0, Cache Size: 0
info: Showing Caching Size Start[0]
      'HelloWorld1' value is  = I love C#
info: Showing Caching Size Start[0]
      Added HelloWorld1. Cache Count: 1, Cache Size: 2
info: Showing Caching Size Start[0]
      Adding HelloWorld2. Cache Count: 1, Cache Size: 2
info: Showing Caching Size Start[0]
      'HelloWorld2' value is  = I love C#
info: Showing Caching Size Start[0]
      Added HelloWorld2. Cache Count: 2, Cache Size: 4
info: Showing Caching Size Start[0]
      Current Cache. Cache Count: 2, Cache Size: 4
info: Showing Caching Size Start[0]
      'HelloWorld3' value is  = NULL
info: Showing Caching Size Start[0]
      Cache Count: 2, Cache Size: 4

Conclusion

This article taught us caching basics by implementing In-Memory Caching in ASP.NET Core applications using minimal API. Moreover, we have looked at the concepts of the following:

• MemoryCacheEntryOptions
• Sliding Expiration
• Absolute Expiration
• Priority
• TryGetValue
• Get
• Set
• MemoryCacheEntryOptions

References:

• https://www.c-sharpcorner.com/article/implement-in-memory-cache-in-the-net-core-api/
• https://code-maze.com/aspnetcore-in-memory-caching/
• https://binaryintellect.net/articles/a7d9edfd-1f86-45f8-a668-64cc86d8e248.aspx