009-spring cache-自己定制缓存接入。CacheManager和Cache

一、概述

现状：目前缓存框架底层使用redis，但是进行了统一包装，相当于对外一个新缓存框架，提供了redis基础功能

问题：缓存混乱乱，由程序员自己set，get。清理不彻底。线上出问题。

需求：项目需要使用spring cache统一对service进行缓存处理。团队统一的缓存管理，以及同外部一致的缓存策略方案

1.1、实现自定义缓存方案

　　参看缓存实现图：006-spring cache-缓存实现-01-SimpleCacheConfiguration、RedisCacheConfiguration

　　参看redis 实现架构

　　　　

　　SpringCache本质上是一个对缓存使用的抽象，将存储的具体实现方案从缓存执行动作及流程中提取出来。缓存流程中面向的两个抽象接口是CacheManager、Cache。其中Cache提供了缓存操作的读取/写入/移除等方法，本着面向抽象编程的原则，内部将缓存对象统一封装成ValueWrapper。Cache接口代码如下：

public interface Cache {    
    String getName();  //缓存的名字    
    Object getNativeCache(); //得到底层使用的缓存，如Ehcache    
    ValueWrapper get(Object key); //根据key得到一个ValueWrapper，然后调用其get方法获取值    
    <T> T get(Object key, Class<T> type);//根据key，和value的类型直接获取value    
    void put(Object key, Object value);//往缓存放数据    
    void evict(Object key);//从缓存中移除key对应的缓存    
    void clear(); //清空缓存    
    
    interface ValueWrapper { //缓存值的Wrapper    
        Object get(); //得到真实的value    
    }    
}

由于在应用中可能定义多个Cache，因此提供了CacheManager抽象，用于缓存的管理，接口代码如下：

public interface CacheManager {    
    Cache getCache(String name); //根据Cache名字获取Cache     
    Collection<String> getCacheNames(); //得到所有Cache的名字    
}

　　任何实现了这两个接口的缓存方案都可以直接配置进SpringCache使用。其自带的SimpleCacheManager、ConcurrentMapCache是如此；使用ehcache作为存储实现的EhCacheCacheManager、EhCacheCache也是如此。可以自己实现CacheManager与Cache并将其集成进来。

二、自定义缓存实现

2.1、自定义缓存实现-基础版本【需要自己手动写缓存空间】

1、Cache接口，增删改差

　　为了方便展示，自定义缓存实现方案只实现最简单的功能，cache内部使用ConcurrentHashMap做为存储方案，使用默认实现SimpleValueWrapper，MyCache代码如下：

package com.aaa.test.config;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cache.Cache;
import org.springframework.cache.support.SimpleValueWrapper;
import org.springframework.context.annotation.Configuration;

import java.util.Map;
import java.util.concurrent.Callable;
import java.util.concurrent.ConcurrentHashMap;

@Configuration
public class MyCache implements Cache {
    final static Logger logger = LoggerFactory.getLogger(MyCache.class);

    String name;
    Map<Object, Object> store = new ConcurrentHashMap<Object, Object>();

    public MyCache() {
    }

    public MyCache(String name) {
        this.name = name;
    }

    @Override
    public String getName() {
        return this.name;
    }
    public void setName(String name){
        this.name = name;
    }


    @Override
    public Object getNativeCache() {
        return store;
    }


    @Override
    public ValueWrapper get(Object key) {
        ValueWrapper result = null;
        Object thevalue = store.get(key);
        if(thevalue!=null) {
            logger.info("["+name+"]got cache, key:"+key);
            result = new SimpleValueWrapper(thevalue);
        }else{
            logger.info("["+name+"]missing cache, key:"+key);
        }
        return result;
    }


    @SuppressWarnings("unchecked")
    @Override
    public <T> T get(Object key, Class<T> type) {
        ValueWrapper vw = get(key);
        if(vw==null){
            return null;
        }
        return (T)vw.get();
    }


    @SuppressWarnings("unchecked")
    @Override
    public <T> T get(Object key, Callable<T> valueLoader) {
        ValueWrapper vw = get(key);
        if(vw==null){
            return null;
        }
        return (T)vw.get();
    }


    @Override
    public void put(Object key, Object value) {
        store.put(key, value);
    }


    @Override
    public Cache.ValueWrapper putIfAbsent(Object key, Object value) {
        Object existing = this.store.putIfAbsent(key, value);
        return (existing != null ? new SimpleValueWrapper(existing) : null);
    }


    @Override
    public void evict(Object key) {
        store.remove(key);
    }


    @Override
    public void clear() {
        store.clear();
    }
}

2、MyCacheManager实现-缓存管理器

public class MyCacheManager extends AbstractCacheManager {
    private Collection<? extends MyCache> caches;

    public void setCaches(Collection<? extends MyCache> caches) {
        this.caches = caches;
    }

    @Override
    protected Collection<? extends MyCache> loadCaches() {
        return this.caches;
    }
}

3、注入使用，配置缓存空间

@Bean(name="myCacheManager")
    public CacheManager myCacheManager(){
        MyCacheManager myCacheManager = new MyCacheManager();
        List<MyCache> caches = new ArrayList<MyCache>();
        MyCache mycache = new MyCache("mycache");
        MyCache mycache2 = new MyCache("mycache2");
        caches.add(mycache);
        caches.add(mycache2);
        myCacheManager.setCaches(caches);
        return myCacheManager;
    }

2.2、 自定义缓存实现-基础版本【优化命名，使用注解自动化添加】

　　通过上述实现，发现缓存管理器需要手工注入缓存空间。

　　解决思路：在项目启动，类初始化或者调用方法处，获取创建命名空间。此处使用后置，调用方法前创建命名空间。

知识点、

　　使用aop拦截，获取自定义注解上的参数值

　　实现Ordered，已提前缓存注解拦截

1、将上述第三步，修改

@Configuration
@EnableCaching
public class SimpleCacheManagerConfiguration {
    @Bean
    public CacheManager cacheManager() {
        MyCacheManager cacheManager = new MyCacheManager();
        List<MyCache> caches = new ArrayList<MyCache>();
//        caches.add(new MyCache("cache"));
//        cacheManager.setCaches(caches);
        return cacheManager;
    }
}

2、添加 Aspect

package com.aaa.test.aspect;

import com.aaa.test.config.MyCache;
import com.aaa.test.config.MyCacheManager;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.CacheConfig;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.CachePut;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.core.Ordered;
import org.springframework.stereotype.Component;

import java.lang.annotation.Annotation;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;

@Aspect
@Component
public class CacheAspect implements Ordered {
    @Autowired
    private CacheManager cacheManager;

    @Before("@annotation(org.springframework.cache.annotation.Cacheable)")
    public void beforeCacheable(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, Cacheable.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CachePut)")
    public void beforeCachePut(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CachePut.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CacheEvict)")
    public void beforeCacheEvict(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CacheEvict.class);
    }

    private void setCacheNames(MethodSignature ms, Class cls) {
        Annotation[] annotationsClass = ms.getDeclaringType().getAnnotationsByType(cls);
        Annotation[] annotationsMethod = ms.getMethod().getAnnotationsByType(cls);
        Set<Annotation> set=new HashSet<>();
        set.addAll(Arrays.asList(annotationsClass));
        set.addAll(Arrays.asList(annotationsMethod));

        Set<MyCache> setCache = new HashSet<>();
        Collection<? extends MyCache> caches = ((MyCacheManager) cacheManager).getCaches();
        setCache.addAll(caches);
        for (Annotation item : set) {
            if (item instanceof CacheConfig) {
                CacheConfig config = (CacheConfig) item;
                for (String s : config.cacheNames()) {
                    setCache.add(new MyCache(s));
                }
            } else if (item instanceof Cacheable) {
                Cacheable config = (Cacheable) item;
                String[] strings = config.cacheNames();
                String[] values = config.value();
                Set<String> nameSet =new HashSet<>();
                nameSet.addAll(Arrays.asList(strings));
                nameSet.addAll(Arrays.asList(values));
                for (String s : nameSet) {
                    setCache.add(new MyCache(s));
                }
            }
        }
        ((MyCacheManager) cacheManager).setCaches(setCache);
        ((MyCacheManager) cacheManager).initializeCaches();
    }

    // 优先执行
    @Override
    public int getOrder() {
        return -999;
    }

}

2.3、实现类Redis缓存

　　参看上图，进行重新划分

　　 

 　　1、原生、继承实现复用；

　　2、当前的缓存体系，即被包装的缓存；

　　3、编码开发的部分，要重新复写一下Cache、CacheManager；

      4、忽略，springboot redis 配置类

开发步骤 

1、目前配置的有如下两个：

maven-archetypes-webapp 默认模式，常用

maven-archetypes-webapp-cache 使用redis缓存，没被包装的，需要编写service 代码添加spring cache注解

现状：公司统一包装了缓存，如redis等，此时直接没法使用，需要自定义。但是还想使用spring cache。

创建基础项目

　　使用：maven-archetypes-webapp-cache 构建基础项目

mvn archetype:generate \
-DgroupId=com.aaa.test -DartifactId=test-custom-cache-demo -Dversion=1.0.0-SNAPSHOT \
-DarchetypeGroupId=com.github.bjlhx15 -DarchetypeArtifactId=maven-archetypes-webapp-cache -DarchetypeVersion=0.0.2 \
-X -DarchetypeCatalog=local -DinteractiveMode=false

 

2、定义缓存

|--customcache
    |--CacheAspect
    |--CustomRedisCache
    |--CustomRedisCacheConfiguration
    |--CustomRedisCacheManager

 

　　如名称为：CustomRedisCache继承AbstractValueAdaptingCache，缓存实现

public class CustomRedisCache extends AbstractValueAdaptingCache {

    private final String name;
    private final CustomRedisDbService cacheService;
    private final RedisCacheConfiguration cacheConfig;
    private final ConversionService conversionService;


    public CustomRedisCache(String name, JimClientService cacheService) {
        super(true);
        this.name = name;
        this.cacheService = cacheService;
        this.cacheConfig = RedisCacheConfiguration.defaultCacheConfig();
        this.conversionService = cacheConfig.getConversionService();
    }

    //获根据key取缓存,如果返回null，则要读取持久层
    @Override
    protected Object lookup(Object key) {
        byte[] value = null;
        try {
            value = cacheService.hGet(name.getBytes("UTF-8"), createAndConvertCacheKey(key));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        if (value == null) {
            return null;
        }
        return deserializeCacheValue(value);
//        return value;
    }

    //获取 缓存 空间名
    @Override
    public String getName() {
        return name;
    }

    @Override
    public Object getNativeCache() {
        return this.cacheService;
    }

    @Override
    public synchronized <T> T get(Object key, Callable<T> valueLoader) {
        ValueWrapper result = get(key);

        if (result != null) {
//            JSON.parseObject(result,T);
            return (T) result.get();
        }

        T value = valueFromLoader(key, valueLoader);
        put(key, value);
        return value;
    }

    //从持久层读取value，然后存入缓存。允许value = null
    @Override
    public void put(Object key, Object value) {
        Object cacheValue = value;
        if (!isAllowNullValues() && cacheValue == null) {
            throw new IllegalArgumentException(String.format(
                    "Cache '%s' does not allow 'null' values. Avoid storing null via '@Cacheable(unless=\"#result == null\")' or configure RedisCache to allow 'null' via RedisCacheConfiguration.",
                    name));
        }
        try {
            cacheService.hSet(name.getBytes("UTF-8"), createAndConvertCacheKey(key), serializeCacheValue(value));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
    }

    //如果传入key对应的value已经存在，就返回存在的value，不进行替换。如果不存在，就添加key和value，返回null.
    @Override
    public ValueWrapper putIfAbsent(Object key, Object value) {
        Object cacheValue = value;

        if (!isAllowNullValues() && cacheValue == null) {
            return get(key);
        }

        Boolean result = null;
        try {
            result = cacheService.hSet(name.getBytes("UTF-8"), createAndConvertCacheKey(key), serializeCacheValue(cacheValue));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }


        if (result == null || result == false) {
            return null;
        }
        return get(key);
        //return new SimpleValueWrapper(fromStoreValue(deserializeCacheValue(result)));
    }

    @Override
    public void evict(Object key) {
        try {
            cacheService.hDel(name.getBytes("UTF-8"),createAndConvertCacheKey(key));
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
    }

    protected String convertKey(Object key) {

        TypeDescriptor source = TypeDescriptor.forObject(key);
        if (conversionService.canConvert(source, TypeDescriptor.valueOf(String.class))) {
            return conversionService.convert(key, String.class);
        }

        Method toString = ReflectionUtils.findMethod(key.getClass(), "toString");

        if (toString != null && !Object.class.equals(toString.getDeclaringClass())) {
            return key.toString();
        }

        throw new IllegalStateException(
                String.format("Cannot convert %s to String. Register a Converter or override toString().", source));
    }
    @Override
    public void clear() {
        cacheService.deleteRedisByKey(name);
    }

    protected String createCacheKey(Object key) {

        String convertedKey = convertKey(key);

        if (!cacheConfig.usePrefix()) {
            return convertedKey;
        }

        return prefixCacheKey(convertedKey);
    }
    protected byte[] serializeCacheKey(String cacheKey) {
        return ByteUtils.getBytes(cacheConfig.getKeySerializationPair().write(cacheKey));
    }

    private byte[] createAndConvertCacheKey(Object key) {
        return serializeCacheKey(createCacheKey(key));
    }

    @Nullable
    protected Object deserializeCacheValue(byte[] value) {
//        return null;//
         return cacheConfig.getValueSerializationPair().read(ByteBuffer.wrap(value));
    }

    protected byte[] serializeCacheValue(Object value) {
        return ByteUtils.getBytes(cacheConfig.getValueSerializationPair().write(value));
    }

    private static <T> T valueFromLoader(Object key, Callable<T> valueLoader) {
        try {
            return valueLoader.call();
        } catch (Exception e) {
            throw new ValueRetrievalException(key, valueLoader, e);
        }
    }

    private String prefixCacheKey(String key) {

        // allow contextual cache names by computing the key prefix on every call.
        return cacheConfig.getKeyPrefixFor(name) + key;
    }
}

　　缓存管理器 CustomRedisCacheManager

public class CustomRedisCacheManager extends AbstractCacheManager {
    private Collection<? extends CustomRedisCache> caches;

    /**
     * Specify the collection of Cache instances to use for this CacheManager.
     */
    public void setCaches(Collection<? extends CustomRedisCache> caches) {
        this.caches = caches;
    }

    @Override
    protected Collection<? extends CustomRedisCache> loadCaches() {
        return this.caches;
    }

    public Collection<? extends CustomRedisCache> getCaches() {
        return caches;
    }
}

　　配置类

@Configuration
public class CustomRedisCacheConfiguration {

    @Autowired
    private CustomRedisService cacheService;

    @Bean
    public CacheManager cacheManager() {
        CustomRedisCacheManager cacheManager = new CustomRedisCacheManager();
        List<CustomRedisCache> caches = new ArrayList<CustomRedisCache>();
//        caches.add(new JimDbCache("AccountBalance", cacheService));
        cacheManager.setCaches(caches);

        return cacheManager;
    }
}

　　其实此时这样就可以使用了，但是每次增加命名空间，上述注释的地方

　　此处使用注解，管理命名空间

@Aspect
@Component
public class CacheAspect implements Ordered {
    @Autowired
    private CacheManager cacheManager;
    @Autowired
    private CustomRedisDbService cacheService;

    @Before("@annotation(org.springframework.cache.annotation.Cacheable)")
    public void beforeCacheable(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, Cacheable.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CachePut)")
    public void beforeCachePut(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CachePut.class);
    }

    @Before("@annotation(org.springframework.cache.annotation.CacheEvict)")
    public void beforeCacheEvict(JoinPoint pjp) throws Throwable {
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        setCacheNames(ms, CacheConfig.class);
        setCacheNames(ms, CacheEvict.class);
    }

    private void setCacheNames(MethodSignature ms, Class cls) {
        Annotation[] annotationsClass = ms.getDeclaringType().getAnnotationsByType(cls);
        Annotation[] annotationsMethod = ms.getMethod().getAnnotationsByType(cls);
        Set<Annotation> set=new HashSet<>();
        set.addAll(Arrays.asList(annotationsClass));
        set.addAll(Arrays.asList(annotationsMethod));

        Set<CustomRedisCache> setCache = new HashSet<>();
        Collection<? extends CustomRedisCache> caches = ((CustomRedisCacheManager) cacheManager).getCaches();
        setCache.addAll(caches);
        for (Annotation item : set) {
            if (item instanceof CacheConfig) {
                CacheConfig config = (CacheConfig) item;
                for (String s : config.cacheNames()) {
                    setCache.add(new CustomRedisCache(s,cacheService));
                }
            } else if (item instanceof Cacheable) {
                Cacheable config = (Cacheable) item;
                String[] strings = config.cacheNames();
                String[] values = config.value();
                Set<String> nameSet =new HashSet<>();
                nameSet.addAll(Arrays.asList(strings));
                nameSet.addAll(Arrays.asList(values));
                for (String s : nameSet) {
                    setCache.add(new CustomRedisCache(s,cacheService));
                }
            }
        }
        ((CustomRedisCacheManager) cacheManager).setCaches(setCache);
        ((CustomRedisCacheManager) cacheManager).initializeCaches();
    }

    // 优先执行
    @Override
    public int getOrder() {
        return -999;
    }

} 

实现说明：

1、CustomRedisCache，redis实现中使用的是string类型，这里使用的是set类型

　　主要原因：删除这里如果使用String类型，删除使用的是pattern模式，找到每个集群每个点，使用模式删除

　　String优点：1、使用Stringkey，不同key分布在不同节点，更加负载化，降低缓存雪崩概率，使用Set的话，key在一个节点。

参看：006-spring cache-缓存原理-SimpleCacheConfiguration、RedisCacheConfiguration原理实现

2、缓存空间加载方式

　　上述使用注解拦截器，加载缓存空间名

　　redis使用→RedisCacheConfiguration→CacheProperties，内使用：private List<String> cacheNames = new ArrayList<>();