Java High-Performance Local Cache Practices

By Yang Xian (Linjing)

Java cache technology can be divided into remote cache and local cache. Common schemes for remote cache include Redis and MemCache. Representative technologies for local cache mainly include HashMap, Guava Cache, Caffeine, and Encahche. The remote cache will be discussed in depth in the following article. This article only introduces local cache and highlights high-performance local cache. This article will introduce common local cache technology (for general understanding) and then introduce the best-performance cache. How good is its performance? What equips it with good performance? Finally, several practical examples are used to show the application of high-performance local cache in daily work.

1. An Introduction to Java Local Cache Technology

1.1 HashMap

The underlying mode of Map is used to place the objects that need to be cached in memory.

Advantages: Easy, no need to introduce third-party packages, and suitable for simple scenarios
Disadvantages: There is no cache elimination policy, and customized development costs are high.

public class LRUCache extends LinkedHashMap {
   /**
     * The read-write lock can be re-entered to ensure concurrent read-write security.
     */
    private ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
    private Lock readLock = readWriteLock.readLock();
    private Lock writeLock = readWriteLock.writeLock();
   /**
     * Cache Size Limit
     */
    private int maxSize;
   public LRUCache(int maxSize) {
        super(maxSize + 1, 1.0f, true);
        this.maxSize = maxSize;
    }
  @Override
    public Object get(Object key) {
        readLock.lock();
        try {
            return super.get(key);
        } finally {
            readLock.unlock();
        }
    }
    @Override
    public Object put(Object key, Object value) {
        writeLock.lock();
        try {
            return super.put(key, value);
        } finally {
            writeLock.unlock();
        }
    }
    @Override
    protected boolean removeEldestEntry(Map.Entry eldest) {
        return this.size() > maxSize;
    }
}

1.2 Guava Cache

Guava Cache is a caching technology open-sourced by Google based on the LRU replacement algorithm. However, Guava Cache has been replaced by Caffeine, which will be introduced next. Therefore, we will not show the sample code here. Interested readers can visit the Guava Cache homepage.

Advantages: Supports maximum capacity limit, two expired deletion policies (insertion time and access time), and simple statistical functions
Disadvantages: Both springboot2 and spring5 have dropped support for Guava Cache.

1.3 Caffeine

Caffeine uses W-TinyLFU (combining the advantages of LUR and LFU) open-source cache technology. The cache performance is close to the theoretical best, which is an enhanced version of Guava Cache.

public class CaffeineCacheTest {

    public static void main(String[] args) throws Exception {
        //Create guava cache
        Cache<String, String> loadingCache = Caffeine.newBuilder()
                // The initial capacity of the cache
                .initialCapacity(5)
                // The maximum number of the cache
                .maximumSize(10)
                // Specify that the write cache expires in n seconds.
                .expireAfterWrite(17, TimeUnit.SECONDS)
                // Specify that the read-write cache expires in n seconds. It is rarely used in practice and is similar to expireAfterWrite.
                //.expireAfterAccess(17, TimeUnit.SECONDS)
                .build();
        String key = "key";
        // Write data to the cache.
        loadingCache.put(key, "v");
      // Obtain the value of the value. If the key does not exist, obtain the value and then return.
        String value = loadingCache.get(key, CaffeineCacheTest::getValueFromDB);
   // Delete the key.
        loadingCache.invalidate(key);
    }
    private static String getValueFromDB(String key) {
        return "v";
    }
}

1.4 Encache

Ehcache is a fast and lean Java in-process caching framework. It is Hibernate's default cacheprovider.

Advantages: Supports multiple cache elimination algorithms, including LFU, LRU, and FIFO. The cache supports in-heap cache, out-of-heap cache, and disk cache. Supports multiple cluster solutions to solve data sharing problems.
Disadvantages: Poorer performance than Caffeine

public class EncacheTest {
    public static void main(String[] args) throws Exception {
        // Declare a cacheBuilder.
        CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder()
                .withCache("encacheInstance", CacheConfigurationBuilder
                        // Declare an in-heap cache with a capacity of 20.
                        .newCacheConfigurationBuilder(String.class,String.class, ResourcePoolsBuilder.heap(20)))
                .build(true);
        // Obtain the cache instance.
        Cache<String,String> myCache =  cacheManager.getCache("encacheInstance", String.class, String.class);
        // Write cache.
        myCache.put("key","v");
        // Read cache.
        String value = myCache.get("key");
        // Remove cache.
        cacheManager.removeCache("myCache");
        cacheManager.close();
    }
}

Based on Caffeine's official website, Caffeine has advantages over several other solutions in terms of performance and functionality. Next, I will introduce Caffeine's performance and implementation principles.

2. High-Performance Cache Caffeine

2.1 Cache Type

2.1.1 Cache

Cache<Key, Graph> cache = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .maximumSize(10_000)
    .build();
// Find a cache element and return null if it is not found.
Graph graph = cache.getIfPresent(key);
// Find the cache. If the cache does not exist, a cache element is generated. If the cache cannot be generated, null is returned.
graph = cache.get(key, k -> createExpensiveGraph(key));
// Add or update a cache element.
cache.put(key, graph);
// Remove a cache element.
cache.invalidate(key);

The Cache interface provides the ability to explicitly search and find, update, and remove cache elements. If the cached element cannot be generated or an exception occurs during the generation, the cache.get may return the null.

2.1.2 Loading Cache

LoadingCache<Key, Graph> cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));
// Find the cache. If the cache does not exist, a cache element is generated. If the cache cannot be generated, null is returned.
Graph graph = cache.get(key);
// Search the cache in batches. If the cache does not exist, a cache element is generated.
Map<Key, Graph> graphs = cache.getAll(keys);

A LoadingCache is a cache implementation after the CacheLoader capability is attached to the cache.

If the cache does not exist, CacheLoader.load is used to generate the corresponding cache element.

2.1.3 Async Cache

AsyncCache<Key, Graph> cache = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .maximumSize(10_000)
    .buildAsync();
// Find a cache element and return null if it is not found.
CompletableFuture<Graph> graph = cache.getIfPresent(key);
// Find the cache element. If it does not exist, it is generated asynchronously.
graph = cache.get(key, k -> createExpensiveGraph(key));
// Add or update a cache element.
cache.put(key, graph);
// Remove a cache element.
cache.synchronous().invalidate(key);

AsyncCache is the asynchronous form of cache. It provides the ability for an executor to generate cache elements and return CompletableFuture. The default thread pool implementation is ForkJoinPool.com monPool(). You can also customize your thread pool selection by overwriting and implementing Caffeine.executor(Executor) methods.

2.1.4 Async Loading Cache

AsyncLoadingCache<Key, Graph> cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .expireAfterWrite(10, TimeUnit.MINUTES)
    // You can choose to de-asynchronously encapsulate a synchronous operation to generate cache elements
    .buildAsync(key -> createExpensiveGraph(key));
    // You also can choose to build an asynchronous cache element operation and return a future.
    .buildAsync((key, executor) -> createExpensiveGraphAsync(key, executor));
// Find the cache element. If it does not exist, it will be generated asynchronously.
CompletableFuture<Graph> graph = cache.get(key);
// Find cache elements in batches. If they do not exist, they will be generated asynchronously.
CompletableFuture<Map<Key, Graph>> graphs = cache.getAll(keys);

AsyncLoadingCache is the asynchronous form of LoadingCache that provides asynchronous load generation of cache elements.

2.2 Eviction Strategy

Capacity-Based

// Perform eviction based on the number of elements in the cache.
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumSize(10_000)
    .build(key -> createExpensiveGraph(key));
// Perform eviction based on the weight of elements in the cache.
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumWeight(10_000)
    .weigher((Key key, Graph graph) -> graph.vertices().size())
    .build(key -> createExpensiveGraph(key));

Time-Based

// Based on the eviction policy with a fixed expiration time.
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfterAccess(5, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));
// Based on different expired eviction policies.
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfter(new Expiry<Key, Graph>() {
      public long expireAfterCreate(Key key, Graph graph, long currentTime) {
        // Use wall clock time, rather than nanotime, if from an external resource
        long seconds = graph.creationDate().plusHours(5)
            .minus(System.currentTimeMillis(), MILLIS)
            .toEpochSecond();
        return TimeUnit.SECONDS.toNanos(seconds);
      }
      public long expireAfterUpdate(Key key, Graph graph, 
          long currentTime, long currentDuration) {
        return currentDuration;
      }
      public long expireAfterRead(Key key, Graph graph,
          long currentTime, long currentDuration) {
        return currentDuration;
      }
    })
    .build(key -> createExpensiveGraph(key));

Reference-Based

// Evict when the key and cache elements no longer have other strong references.
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .weakKeys()
    .weakValues()
    .build(key -> createExpensiveGraph(key));
// Evict when GC is performed.
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .softValues()
    .build(key -> createExpensiveGraph(key));

2.3 Refresh Mechanism

LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumSize(10_000)
    .refreshAfterWrite(1, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

The refresh policy can only be used in LoadingCache. Unlike eviction, if you query the cache element during refresh, its old value will still be returned. The new value will not be returned until the refresh of the element is completed.

2.4 Statistics

Cache<Key, Graph> graphs = Caffeine.newBuilder()
    .maximumSize(10_000)
    .recordStats()
    .build();

The data collection function can be turned on using the Caffeine.recordStats() method. The Cache.stats() method will return a CacheStats object that will contain some statistical indicators, such as:

hitRate(): The hit rate of the query cache
evictionCount(): The number of caches evicted
averageLoadPenalty(): The average time of loading the new value

With the RESTful controller provided by SpringBoot, you can easily query the cache usage.

3. Caffeine's Practice in SpringBoot

According to its official document, Caffeine is a high-performance cache library based on Java8. In Spring5 (SpringBoot2.x), Guava is abandoned. It uses Caffeine with better performance as the default cache scheme.

SpringBoot uses Caffeine in two ways:

Method 1: It directly introduces Caffeine dependencies and then uses Caffeine functions to implement the cache.
Method 2: It introduces Caffeine and Spring Cache dependencies and uses the SpringCache annotation method to implement the cache.

The following are the two methods:

Method 1: Use Caffeine Dependency

First, introduce maven-related dependencies:

<dependency>  
  <groupId>com.github.ben-manes.caffeine</groupId>  
    <artifactId>caffeine</artifactId>  
</dependency>

Second, set the configuration options for the cache:

@Configuration
public class CacheConfig {
    @Bean
    public Cache<String, Object> caffeineCache() {
        return Caffeine.newBuilder()
                // Set a fixed time to expire after the last write or access.
                .expireAfterWrite(60, TimeUnit.SECONDS)
                // The initial cache size
                .initialCapacity(100)
                // The maximum of cached entries
                .maximumSize(1000)
                .build();
    }
}

Finally, add a cache function to the service:

@Slf4j
@Service
public class UserInfoServiceImpl {
    /**
     * Simulate the database to store data.
     */
    private HashMap<Integer, UserInfo> userInfoMap = new HashMap<>();
    @Autowired
    Cache<String, Object> caffeineCache;
    public void addUserInfo(UserInfo userInfo) {
        userInfoMap.put(userInfo.getId(), userInfo);
        // Add to the cache.
        caffeineCache.put(String.valueOf(userInfo.getId()),userInfo);
    }
    public UserInfo getByName(Integer id) {
        // Read from the cache first.
        caffeineCache.getIfPresent(id);
        UserInfo userInfo = (UserInfo) caffeineCache.asMap().get(String.valueOf(id));
        if (userInfo != null){
            return userInfo;
        }
        // If it does not exist in the cache, search from the library.
        userInfo = userInfoMap.get(id);
        // If the user information is not empty, add it to the cache.
        if (userInfo != null){
            caffeineCache.put(String.valueOf(userInfo.getId()),userInfo);
        }
        return userInfo;
    }
 public UserInfo updateUserInfo(UserInfo userInfo) {
        if (!userInfoMap.containsKey(userInfo.getId())) {
            return null;
        }
        // Take the old value.
        UserInfo oldUserInfo = userInfoMap.get(userInfo.getId());
        // Replace the content.
        if (!StringUtils.isEmpty(oldUserInfo.getAge())) {
            oldUserInfo.setAge(userInfo.getAge());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getName())) {
            oldUserInfo.setName(userInfo.getName());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getSex())) {
            oldUserInfo.setSex(userInfo.getSex());
        }
        // Store the new object and update the information of the old object.
        userInfoMap.put(oldUserInfo.getId(), oldUserInfo);
        // Replace the value in the cache.
        caffeineCache.put(String.valueOf(oldUserInfo.getId()),oldUserInfo);
        return oldUserInfo;
    }
  @Override
    public void deleteById(Integer id) {
        userInfoMap.remove(id);
        // Delete from the cache.
        caffeineCache.asMap().remove(String.valueOf(id));
    }
}

Method 2: Use Spring Cache Annotations

First, introduce maven-related dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
</dependency>

Second, configure the cache management class:

@Configuration  
public class CacheConfig {  
  
    /**  
     * Configure the cache manager. 
     *  
     * @return Cache Manager 
     */  
    @Bean("caffeineCacheManager")  
    public CacheManager cacheManager() {  
        CaffeineCacheManager cacheManager = new CaffeineCacheManager();  
        cacheManager.setCaffeine(Caffeine.newBuilder()  
                // Set a fixed time to expire after the last write or access. 
                .expireAfterAccess(60, TimeUnit.SECONDS)  
                // The initial cache size  
                .initialCapacity(100)  
                // The maximum number of cached entries 
                .maximumSize(1000));  
        return cacheManager;  
    }  
  
}

Finally, add a cache function to the service:

@Slf4j
@Service
@CacheConfig(cacheNames = "caffeineCacheManager")
public class UserInfoServiceImpl {
  /**
     * Simulate the database to store data.
     */
    private HashMap<Integer, UserInfo> userInfoMap = new HashMap<>();
  @CachePut(key = "#userInfo.id")
    public void addUserInfo(UserInfo userInfo) {
        userInfoMap.put(userInfo.getId(), userInfo);
    }
  @Cacheable(key = "#id")
    public UserInfo getByName(Integer id) {
        return userInfoMap.get(id);
    }
  @CachePut(key = "#userInfo.id")
    public UserInfo updateUserInfo(UserInfo userInfo) {
        if (!userInfoMap.containsKey(userInfo.getId())) {
            return null;
        }
        // Take the old value.
        UserInfo oldUserInfo = userInfoMap.get(userInfo.getId());
        // Replace the content.
        if (!StringUtils.isEmpty(oldUserInfo.getAge())) {
            oldUserInfo.setAge(userInfo.getAge());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getName())) {
            oldUserInfo.setName(userInfo.getName());
        }
        if (!StringUtils.isEmpty(oldUserInfo.getSex())) {
            oldUserInfo.setSex(userInfo.getSex());
        }
        // Store the new object and update the information of the old object.
        userInfoMap.put(oldUserInfo.getId(), oldUserInfo);
        // Return information about the new object.
        return oldUserInfo;
    }
 @CacheEvict(key = "#id")
    public void deleteById(Integer id) {
        userInfoMap.remove(id);
    }
}

4. Caffeine's Application in Reactor

The combination of Caffeine and Reactor is used through CacheMono and CacheFlux. Caffeine stores a Flux or Mono as a result of the cache. First, define Caffeine's cache:

final Cache<String, String> caffeineCache = Caffeine.newBuilder()
      .expireAfterWrite(Duration.ofSeconds(30))
      .recordStats()
      .build();

CacheMono

final Mono<String> cachedMonoCaffeine = CacheMono
      .lookup(
          k -> Mono.justOrEmpty(caffeineCache.getIfPresent(k)).map(Signal::next),
          key
      )
      .onCacheMissResume(this.handleCacheMiss(key))
      .andWriteWith((k, sig) -> Mono.fromRunnable(() ->
          caffeineCache.put(k, Objects.requireNonNull(sig.get()))
      ));

The lookup method queries whether the cache exists. If it does not exist, a Mono is regenerated through the onCacheMissResume, and the results are stored in the cache through the andWriteWith method.

CacheFlux

final Flux<Integer> cachedFluxCaffeine = CacheFlux
      .lookup(
          k -> {
            final List<Integer> cached = caffeineCache.getIfPresent(k);
 
            if (cached == null) {
              return Mono.empty();
            }
 
            return Mono.just(cached)
                .flatMapMany(Flux::fromIterable)
                .map(Signal::next)
                .collectList();
          },
          key
      )
      .onCacheMissResume(this.handleCacheMiss(key))
      .andWriteWith((k, sig) -> Mono.fromRunnable(() ->
          caffeineCache.put(
              k,
              sig.stream()
                  .filter(signal -> signal.getType() == SignalType.ON_NEXT)
                  .map(Signal::get)
                  .collect(Collectors.toList())
          )
      ));

The usage of CacheFlux is similar.

Community

Java High-Performance Local Cache Practices

1. An Introduction to Java Local Cache Technology

1.1 HashMap

1.2 Guava Cache

1.3 Caffeine

1.4 Encache

2. High-Performance Cache Caffeine

2.1 Cache Type

2.1.1 Cache

2.1.2 Loading Cache

2.1.3 Async Cache

2.1.4 Async Loading Cache

2.2 Eviction Strategy

2.3 Refresh Mechanism

2.4 Statistics

3. Caffeine's Practice in SpringBoot

Method 1: Use Caffeine Dependency

Method 2: Use Spring Cache Annotations

4. Caffeine's Application in Reactor

CacheMono

CacheFlux

References

Read previous post:

Read next post:

Alibaba Cloud Community

You may also like

Comments

Alibaba Cloud Community

Related Products

Best Practices

Web Hosting Solution

Web Hosting

Elastic High Performance Computing Solution