记录一次N久之前,碰到的一次Thrift OOM问题
突然看到这篇关于Thrift OOM的文章,突然想起了,自己曾经经受的一个项目也是相同的问题,当时也是找了很久的问题,写在博文里,以备查阅
一言以蔽之:使用HTTP请求thrift接口会导致反序列化异常,反序列化的数据会异常大,默认读取的最大值是Long.MAX_VALUE, 所以要限制属性值maxReadBufferBytes, 别坚持了你的服务器是不会有1045576TB
public abstract class AbstractNonblockingServer extends TServer {
protected final Logger LOGGER = LoggerFactory.getLogger(getClass().getName());
public static abstract class AbstractNonblockingServerArgs<T extends AbstractNonblockingServerArgs<T>> extends AbstractServerArgs<T> {
//这里可以看到最大读取值的大小
public long maxReadBufferBytes = Long.MAX_VALUE;
public AbstractNonblockingServerArgs(TNonblockingServerTransport transport) {
super(transport);
transportFactory(new TFramedTransport.Factory());
}
}
.....
服务端的示例
public void run() {
TThreadedSelectorServer server = null;
try {
TProcessor tprocessor = new DataCombineService.Processor < DataCombineService.Iface > (
new DataCombineServiceImpl(msg_piper_class, msg_piper_minidle, msg_piper_maxtotal, msg_piper_password,
msg_piper_names, msg_piper_addresses, business_hll_key, business_time_granularity,
business_time_interval, business_his_day, list_length, thread_exe_max_data_length, keys_size, redisProxy,
yamlPath, default_db_type));
TNonblockingServerTransport serverTransport = new TNonblockingServerSocket(thrift_server_port);
TThreadedSelectorServer.Args args = new TThreadedSelectorServer.Args(serverTransport);
args.processor(tprocessor);
// 设置协议工厂,高效率的、密集的二进制编码格式进行数据传输协议
args.protocolFactory(new TCompactProtocol.Factory());
// 设置传输工厂,使用非阻塞方式,按块的大小进行传输,类似于Java中的NIO
args.transportFactory(new TFramedTransport.Factory());
// 设置处理器工厂,只返回一个单例实例
args.processorFactory(new TProcessorFactory(tprocessor));
// 多个线程,主要负责客户端的IO处理
args.selectorThreads(selectorThreads);
//设置最大读取缓存大小 非常重要
args.maxReadBufferBytes = maxReadBufferBytes;
// 工作线程池
/*ExecutorService pool = Executors.newFixedThreadPool(executorServicePoolSize);
args.executorService(pool);*/
//工作线程池的大小为零workerThreads(0),相当于不设置工作线程池
args.workerThreads(0);
server = new TThreadedSelectorServer(args);
logger.info("Hello Thrift TThreadedSelectorServer Start......" + thrift_server_port);
server.serve(); // 启动服务
} catch (Exception e) {
logger.error("启动thrift服务端异常,异常原因:" + e.getMessage(), e);
}
}
看到的文章 https://blog.csdn.net/linzhiqiang0316/article/details/80807292