RedisTemplate源码分析以及Redis常见问题

1. RedisTemplate 默认配置下底层实现

使用jedis（spring-boot 1.x）或者lettuce(spring-boot 2.x)操作redis的

spring-boot 1.5.7

spring-data-redis 1.8.7

配置文件
# redis
spring.redis.host=172.168.32.145
spring.redis.password=
spring.redis.port=6379
spring.redis.database=7

单元测试代码，获取1000次字符类型的redis数值，以下代码为并行执行代码

@Test
public void getStringValue() throws Exception {
    final String key = "cloud_search_using_record";
    for (int i = 0; i < 1000; i++) {
        new Thread(()->{
            stringRedisTemplate.opsForValue().get(key);
        }).start();
    }
    Thread.sleep(1000000);
}

DefaultValueOperations.class , 25行

public V get(Object key) {
	return this.execute(new AbstractOperations<K, V>.ValueDeserializingRedisCallback(key) {
		protected byte[] inRedis(byte[] rawKey, RedisConnection connection) {
			return connection.get(rawKey);
		}
	}, true);
}

AbstractOperations.class , 56行

<T> T execute(RedisCallback<T> callback, boolean b) {
	return this.template.execute(callback, b);
}

RedisTemplate.class，126行

conn = RedisConnectionUtils.getConnection(factory);

RedisConnectionUtils.class， 61行

RedisConnection conn = factory.getConnection();

debug调试 JedisConnectionFactory.class 247行，进行断点查看jedis对象内存地址，当串行执行时，一般的会出现每次都是一样的，并行执行时jedis 地址是不断在一定范围变化的。

public RedisConnection getConnection() {
    if (this.cluster != null) {
        return this.getClusterConnection();
    } else {
        Jedis jedis = this.fetchJedisConnector();
        JedisConnection connection = this.usePool ? new JedisConnection(jedis, this.pool, this.dbIndex, this.clientName) : new JedisConnection(jedis, (Pool)null, this.dbIndex, this.clientName);
        connection.setConvertPipelineAndTxResults(this.convertPipelineAndTxResults);
        return this.postProcessConnection(connection);
    }
}

protected Jedis fetchJedisConnector() {
	try {
		if (this.usePool && this.pool != null) {
			return (Jedis)this.pool.getResource();
		} else {
			Jedis jedis = new Jedis(this.getShardInfo());
			jedis.connect();
			this.potentiallySetClientName(jedis);
			return jedis;
		}
	} catch (Exception var2) {
		throw new RedisConnectionFailureException("Cannot get Jedis connection", var2);
	}
}

获取到的jedis是封装好的RedisConnection对象，在RedisTemplate.class类execute方法中的action.doInRedis(connToExpose); 进行操作Redis并返回结果。

Spring-boot 1.x版本，默认使用Jedis 连接池，也可以主动不使用连接池，设置JedisConnectionFactory工厂类属性是否使用连接池为false，jedisConnectionFactory.setUsePool(false);

spring-boot 2.3.8.RELEASE

spring-data-redis 2.3.6.RELEASE

2.x版本默认优先使用lettuce，如果非要使用jedis 则必须去掉lettuce依赖和RedisAutoConfiguration

配置文件
# redis
spring.redis.host=172.168.32.145
spring.redis.password=
spring.redis.port=6379
spring.redis.database=7

debug调试 RedisTemplate.class，145行。进行断点查看((LettuceConnectionFactory.SharedConnection)((LettuceConnectionFactory)factory).connection).connection对象内存地址，无论串行还行并行，地址都一样。

conn = RedisConnectionUtils.getConnection(factory);

这是因为 shareNativeConnection默认值是true，即默认情况下都使用的同一个本地连接。

当设置本地共享连接为false时，在此处可以看到连接每次都不一样。

另外LettuceConnection 从LettuceConnectionFactory创建时，并没有立即创建连接，而是只创建了LettuceConnection 对象，在使用该对象操作Redis时会检查是否已经创建共享连接，没有的话进行创建。

详细过程如下：

RedisTemplate.class，153行

public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean pipeline) {
        ...
        // 如果是redisTemplate 则直接返回连接本身，如果是stringRedisTemplate则返回DefaultStringRedisConnection，连接RedisConnection对象引用给属性delegate
		RedisConnection connToUse = this.preProcessConnection(conn, existingConnection);
		...
        // 进行callback执行，入参为RedisConnection/DefaultStringRedisConnection     
		T result = action.doInRedis(connToExpose);
		...
}

DefaultStringRedisConnection.class 242行

public byte[] get(byte[] key) {
	return (byte[])this.convertAndReturn(this.delegate.get(key), this.identityConverter);
}

DefaultedRedisConnection.class 224行

default byte[] get(byte[] key) {
	return this.stringCommands().get(key);
}

LettuceStringCommands 31行

public byte[] get(byte[] key) {
	Assert.notNull(key, "Key must not be null!");

	try {
		...
            // 执行获取连接
			return (byte[])this.getConnection().get(key);
		}
	} catch (Exception var3) {
		throw this.convertLettuceAccessException(var3);
	}
}

LettuceConnection.class 664行代码 getConnection()

protected RedisClusterCommands<byte[], byte[]> getConnection() {
	if (this.isQueueing()) {
		return this.getDedicatedConnection();
	} else {
		if (this.asyncSharedConn != null) {
			if (this.asyncSharedConn instanceof StatefulRedisConnection) {
				return ((StatefulRedisConnection)this.asyncSharedConn).sync();
			}

			if (this.asyncSharedConn instanceof StatefulRedisClusterConnection) {
				return ((StatefulRedisClusterConnection)this.asyncSharedConn).sync();
			}
		}
		// 获取专用(非共享)连接
		return this.getDedicatedConnection();
	}
}

LettuceConnection.class 686行代码 getDedicatedConnection()

RedisClusterCommands<byte[], byte[]> getDedicatedConnection() {
	StatefulConnection<byte[], byte[]> connection = this.getOrCreateDedicatedConnection();
	if (connection instanceof StatefulRedisConnection) {
		return ((StatefulRedisConnection)connection).sync();
	} else if (connection instanceof StatefulRedisClusterConnection) {
		return ((StatefulRedisClusterConnection)connection).sync();
	} else {
		throw new IllegalStateException(String.format("%s is not a supported connection type.", connection.getClass().getName()));
	}
}

LettuceConnection.class 708行代码 getOrCreateDedicatedConnection()

private StatefulConnection<byte[], byte[]> getOrCreateDedicatedConnection() {
	if (this.asyncDedicatedConn == null) {
        // 获取连接
		this.asyncDedicatedConn = this.doGetAsyncDedicatedConnection();
	}

	return this.asyncDedicatedConn;
}

获取连接核心代码

//LettuceConnection.doGetAsyncDedicatedConnection()
StatefulConnection connection = this.connectionProvider.getConnection(StatefulConnection.class);
//LettucePoolingConnectionProvider.getConnection()
GenericObjectPool pool = (GenericObjectPool)this.pools.computeIfAbsent(connectionType, (poolType) -> {
	return ConnectionPoolSupport.createGenericObjectPool(() -> {
		return this.connectionProvider.getConnection(connectionType);
	}, this.poolConfig, false);
});
...
StatefulConnection<?, ?> connection = (StatefulConnection)pool.borrowObject();
...

2. 关于连接关闭

RedisTemplate.class，操作完redis 后finally里调用releaseConnection

public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean pipeline) {
    Assert.isTrue(this.initialized, "template not initialized; call afterPropertiesSet() before using it");
    Assert.notNull(action, "Callback object must not be null");
    RedisConnectionFactory factory = this.getConnectionFactory();
    RedisConnection conn = null;
...
} finally {
    // 执行完成之后释放连接：事务检查以及connection.close();
    RedisConnectionUtils.releaseConnection(conn, factory);
}

3. 关于jedis有一点官网描述需要注意

using Jedis in a multithreaded environment

You shouldn't use the same instance from different threads because you'll have strange errors. And sometimes creating lots of Jedis instances is not good enough because it means lots of sockets and connections, which leads to strange errors as well. A single Jedis instance is not threadsafe! To avoid these problems, you should use JedisPool, which is a threadsafe pool of network connections. You can use the pool to reliably create several Jedis instances, given you return the Jedis instance to the pool when done. This way you can overcome those strange errors and achieve great performance.

To use it, init a pool:

JedisPool pool = new JedisPool(new JedisPoolConfig(), "localhost");

Jedis继承BinaryJedis，而BinaryJedis成员变量Client，Client-->BinaryClient-->Connection

Connection的connect方法中可以看到读写流是成员变量也并没有做并发限制，所以现成不安全的。

this.outputStream = new RedisOutputStream(this.socket.getOutputStream());
this.inputStream = new RedisInputStream(this.socket.getInputStream());

所以一般的，使用Jedis 时必须时线程池(连接池)形式，每个线程独立使用一个Jedis 连接对象。

4. spring-boot 2.x 使用Lettuce优先Jedis官网解释

Spring 优先使用Lettuce，一个是因为Jedis线程不安全，另一个连接池是以每个实例的物理连接为代价的，这会增加Redis连接的数量。而Lettuce 基于Netty实现，支持多线程共享连接,不用担心并发线程数影响到Redis连接数。

5. RedisTemplate 使用Pipeline示例

final String key = "cloud_search_using_record";
RedisSerializer<String> serializer = redisTemplate.getStringSerializer();
List<String> value = redisTemplate.executePipelined(new RedisCallback<String>() {
	@Override
	public String doInRedis(RedisConnection connection) throws DataAccessException {
		// get
		connection.get(key.getBytes());
		// set
//                connection.set(serializer.serialize(key), serializer.serialize(value), Expiration.seconds(seconds), RedisStringCommands.SetOption.UPSERT);
		return null;
	}
}, serializer);
Assert.assertNotNull(value);
Assert.assertEquals(1, value.size());
System.out.println(value.get(0));

6. 常见错误原因以及排查

1.Connection reset 错误

服务端终止异常连接导致

原因是Tcp握手过程中，服务器返回了RST标志位，但是客户端还在进行输入输出数据，抛出此错误

RST表示复位，用来异常的关闭连接，在TCP的设计中它是不可或缺的

2.SocketTimeoutException: connect timed out错误

网络波动原因

服务器连接达到最大限制，应当检查代码中是否有连接未关闭

3.Could not get a resource from the pool 错误，连接池无空闲连接

Timeout waiting for idle object 等待获取连接超时

连接池大小调整
从连接池获取连连接等待超时调整
检查连接有无归还连接池或者关闭

4.当配置获取连接等待时间为-1，发生操作redis阻塞情况

检查是否关闭连接或者归还连接，这是形成无限等待了

5.Unexpected end of stream错误

服务端超时配置过短，导致客户端在用连接失效

posted @ 2022-02-09 17:17 衰草寒烟阅读(1595) 评论(0) 编辑收藏举报

刷新页面返回顶部

衰草寒烟