Caching and Caching Architectural Patterns in Microservice

Dec 30, 2022

In this post:

Learn about Caching
Learn about Caching Architectural Patterns in Microservice

Caching

First of all, let’s learn about Caching.

Caching is the process of storing copies of files or data in a cache, or temporary storage location, so that they can be accessed more quickly.

Nowadays, caching is used everywhere, from the front-end to the back-end, and this can be either to improve performance, reduce time to load, or decrease downtime. This diagram illustrates where we cache data in a typical architecture

In this post, we will limit the scope to learning about caching in microservice architecture. As this illustrates, we could place the cache inside each service or as a completely separate cache server (Distributed cache), let’s summarize all the options we have and describe Caching Architectural Patterns.

Caching Architectural Patterns

1. Embedded Cache

This is the simplest possible caching pattern. In this diagram, the flow is as below:

Request comes from other services to the Load Balancer
The Load Balancer forwards the request to one of the Application services
The service receives the request and does some business that needs data from a long-lasting business operation. First, it checks to see if the data has already been executed and cached.
1. If yes, get the cached value.
2. If not, then perform the long-lasting business operation, store the result in the cache, and back to work.

The long-lasting business operation can be anything worth caching, such as a computation, querying a database, or calling other services.

When an instance uses embedded cache, it usually means that it is owned by that instance and cannot be accessed to read or write by another instance.

Now we move on to a small example with Java. As caching logic is very simple, we can implement it either using built-in data structures or some caching libraries, below we use Guava, but firstly, we need to setup a simple Java project with Spring boot, Spring JPA and Spring web.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
  <groupId>com.h2database</groupId>
  <artifactId>h2</artifactId>
</dependency>

Next we import Guava dependency.

<dependency>
  <groupId>com.google.guava</groupId>
  <artifactId>guava</artifactId>
  <version>31.0.1-jre</version>
</dependency>

First of all, we define an interface named ICache to represent our cache.

public interface ICache<K, V> {
    public V get(K pKey);
    public default V getIfPresent(K key) {
        return this.get(key);
    }
    public void set(K pKey, V pValue);
}

Then we use Guava Cache Loader to implement ICache.

@Configuration
public class EmbeddedCacheCfg {

    @Bean
    public ICache<String, String> embeddedCacheTemplate() {
        return new ICache<String, String>() {
            LoadingCache<String, String> simpleCaching = CacheBuilder.newBuilder()
                    .build(new CacheLoader<String, String>() {
                        @Override
                        public String load(final String pKey) throws Exception {
                            // do something, but in this case we return null
                            return null;
                        }
                    });

            @Override
            public String get(String pKey) {
                try {
                    return simpleCaching.get(pKey);
                } catch (ExecutionException e) {
                    e.printStackTrace();
                }
                return null;
            }

            @Override
            public void set(String pKey, String pValue) {
                simpleCaching.put(pKey, pValue);
            }

            @Override
            public String getIfPresent(String key) {
                return simpleCaching.getIfPresent(key);
            }

        };
    }
}

Next, we use this Caching implementation in our Rest Controller.

@RestController
@RequestMapping("/api/auth")
public class UserController {
    UserRepository userRepository;
    ICache<String, String> simpleCaching;

    @Autowired
    public UserController(UserRepository userRepository,
            @Qualifier("embeddedCacheTemplate") ICache<String, String> simpleCaching) {
        this.userRepository = userRepository;
        this.simpleCaching = simpleCaching;
    }

    @PostMapping("/login")
    public ResponseEntity<LoginResp> login(@RequestBody LoginReq pLoginReq) {
        LoginResp lvLoginResp = new LoginResp();

        List<UserEntity> lvListUser = userRepository.findByUsername(pLoginReq.getUsername());
        if (lvListUser != null && lvListUser.size() > 0) {
            if (pLoginReq.getPassword().equals(lvListUser.get(0).getPassword())) {
                lvLoginResp.setSuccess(true);
                final String lvToken = "token123";
                lvLoginResp.setToken(lvToken);
                this.simpleCaching.set(pLoginReq.getUsername(), lvToken);
            }
        }
        else {
            lvLoginResp.setSuccess(false);
        }
        return new ResponseEntity<>(lvLoginResp, lvLoginResp.isSuccess() ? HttpStatus.OK : HttpStatus.UNAUTHORIZED);
    }

    @PostMapping("/is-logged")
    public ResponseEntity<LoginResp> isLogged(@RequestBody LoginReq pLoginReq) {
        final String lvCachedData = this.simpleCaching.getIfPresent(pLoginReq.getUsername());
        LoginResp lvLoginResp = new LoginResp();

        if (lvCachedData != null) {
            lvLoginResp.setSuccess(true);
            lvLoginResp.setToken(lvCachedData);
        } else {
            lvLoginResp.setSuccess(false);
        }
        return new ResponseEntity<>(lvLoginResp, lvLoginResp.isSuccess() ? HttpStatus.OK : HttpStatus.UNAUTHORIZED);
    }
}

This is a simple example, we have 2 endpoints, /login for user login and /is-logged for checking whether they logged in or not by using cached data that is stored on the /login handler.

Let’s run this project

mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Dserver.port=8080"

Send /login request

curl --request POST \
  --url http://127.0.0.1:8081/api/auth/login \
  --header 'Content-Type: application/json' \
  --data '{
  "username":"supervisor1",
  "password":"123456"
}'

{"token":"token123","success":true}

Send /is-logged request

curl --request POST \
  --url http://127.0.0.1:8081/api/auth/is-logged \
  --header 'Content-Type: application/json' \
  --data '{
  "username":"supervisor1"
}'

{"token":"token123","success":true}

Now we stop the application, restart it, and try to call /is-logged request.

curl --request POST \
  --url http://127.0.0.1:8081/api/auth/is-logged \
  --header 'Content-Type: application/json' \
  --data '{
  "username":"supervisor1"
}'

{"token":null,"success":null}

As a result of this response, cached data was cleared when our application stopped.

In some cases, when we need to scale up this application, we run other applications.

mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Dserver.port=8081"

Then, we use NGINX as Load Balancer, and send /login request to Load Balancer, it’s no problem, but when we send /is-logged, somethime we receive {"token":null,"success":null}, sometime we receive {"token":"token123","success":true}, it was caused by NGINX using a round-robin strategy to distribute our requests to applications, and the important reason is that our cached data isn't shared with each application; for example, when we send /login, NGINX delivers it to server 8080, and when we send /is-logged, NGINX delivers it to server 8081, and we can't read them on server 8081.

2. Embedded distributed Cache

As mentioned above, the disadvantage of embedded cache is that it is not shared across multiple services. In the world of Microservice architecture, where we work with many services and each service can be replicated more than once, a sharing cache is of great importance. With embedded distributed cache, the cache still belongs to the instance, but whenever a new cache entry is written in the cache, the Caching Library takes care of distributing it to other members. When data is read from the cache, it can be found in other instances where the application is running.

The Caching Library mentioned here is Hazelcast, let's enter an example.

First, add Hazelcast dependency

<dependency>
  <groupId>com.hazelcast</groupId>
  <artifactId>hazelcast</artifactId>
  <version>4.1</version>
</dependency>

Because we're using Spring Boot, this dependency has an AutoConfiguration class that can help us reduce the time it takes to initialize a Hazelcast instance, but we'll disable it for ease of following, put this sentence in application.properties

spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.hazelcast.HazelcastAutoConfiguration

And then create new Bean named distributedCacheTemplate to implement ICache with Hazelcast.


@Configuration
@Lazy
public class DistributedCacheCfg {

    @Bean
    public ICache<String, String> distributedCacheTemplate() {
        return new ICache<String, String>() {
            Config lvHzConfig = new Config();
            {
                lvHzConfig.setClusterName("hazelcast");
            }
            HazelcastInstance lvHzInstance = Hazelcast.newHazelcastInstance(lvHzConfig);
            IMap<String, String> simpleCaching = lvHzInstance.getMap("distributed-map");

            @Override
            public String get(String pKey) {
                return simpleCaching.get(pKey);
            }

            @Override
            public void set(String pKey, String pValue) {
                simpleCaching.put(pKey, pValue);
            }
        };
    }
}

During the application startup lifecycle, once we call Hazelcast.newHazelcastInstance(), this will start a new Hazelcast node in the currently running JVM. Because of Hazelcast auto-discovery capabilities, without further configuration, nodes will discover each other and form a cluster.

Now start applications and test cases like those in section Embedded Cache. The results show that it fixed a cache sharing issue.

Using an embedded cache (both distributed and non-distributed) is simple because it does not require any additional configuration or deployment. Furthermore, because the cache runs in the same JVM as the application, data transfer is always low-latency. However, once all applications crash or shut down, all cached data is cleared, let's proceed to the next pattern.

3. Client-Server Cache

While in embedded cache, data belongs to the instance, in client-server cache, data belongs to the Cache server, and our application becomes a Cache client.

This architecture looks similar to the classic database architecture. We have a central server, and applications connect to that server. If we were to compare Client-server pattern with Embedded Cache, there are two main differences:

The first one is that the Cache server is a separate unit in our architecture, which means that we can manage it separately (such as scale up/down, backups, …)
The second is that the application uses a cache client library to communicate with the cache server, and that means we’re no longer limited to JVM-based languages. There is a well-defined protocol, and the programming language of the server part can be different from the client part.

In this section, we’ll use Redis as Cache server (Hazelcast can be setup as Cache server as well)

Because this pattern has two parts: server and client, therefore we need to start Redis server, for easily, we’ll use docker to setup.

version: "3.8"
services:
  redis-stack:
    image: redis/redis-stack:latest
    ports:
      - 6379:6379
      - 8001:8001
    restart: unless-stopped
    networks:
      - redis-nw
    volumes:
      - redis-data/:/data

networks:
  redis-nw:
    driver: bridge
volumes:
  redis-data:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: ./redisdt

Next, we add Redis dependency

<dependency>
  <groupId>biz.paluch.redis</groupId>
  <artifactId>lettuce</artifactId>
  <version>3.2.Final</version>
</dependency>

We can use "spring-data-redis" instead of this, but we need to setup from scratch for ease of following. Next, let’s create a new Bean named serverCacheTemplate to implement ICache with Redis.

@Configuration
@Lazy
public class ServerCacheCfg {
    @Value("${redis.uri}")
	String uri;

    @Bean
    public ICache<String, String> serverCacheTemplate() {
        return new ICache<String, String>() {
            RedisClient redisClient = new RedisClient(RedisURI.create(uri));
            RedisConnection<String, String> connection = redisClient.connect();

            @Override
            public String get(String pKey) {
                return connection.get(pKey);
            }

            @Override
            public void set(String pKey, String pValue) {
                connection.set(pKey, pValue);                
            }
        };
    }
}

With “redis.uri” is defined in application.properties

redis.uri=redis://localhost:6379

Run Redis server and our application, and test cases like those in sections above.

All source available at GitHub

Pros & Cons

Embedded Cache (Distributed and Non-Distributed)

Pros

Simple configuration and deployment
Low-latency data access
No seperate Ops effort needed

Cons

Not flexible management (scaling, backup)
Limited to JVM-based applications
Data collocated with applications

Client-Server Cache

Pros

Data seperate from applications
Seperate management (easy to scaling, backup)
Programming language agnostic

Cons

Seperate Ops effort
Higher latency
Server network requires adjusment

TanPham@Blog

Discussion about this post