case:kafka消费者重新平衡问题

kafka消费者重新平衡问题
使用java的API消费kafka数据时，由于kafka集群稳定性或网络问题，commit提交超时，

expm=Commit cannot be completed since the group hasalready rebalanced and assigned the partitions to another member. 
This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms,
which typically implies that the poll loop is spending too much time message processing. 
You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.;
expc=org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. 
This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, 
which typically implies that the poll loop is spending too much time message processing. 
You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.;exps=

在提交offset的时候，已经过了时间（太长时间没有提交offset了，心跳时间过了），发生了再平衡
，重新分配了消费者，所有你提交offset失败了。让你增加间隔时间就是poll处理的时间太长，
超过了设置的时间，然后心跳断了，发生平衡，重新分配消费者的问题
max.poll.interval.ms

 

通过调参数，如每次poll消息的数据量max.poll.records，调大max.poll.interval.ms，session.timeout.ms

降低数据量，从而降低处理时间，增加心跳检查时间（但并没有解决我这边的问题，这里只能解决消费客户端的问题，由于kafka本身的稳定性和网络的问题无法从这里解决）
最终只能try catch，使得线程不会因为异常停止，等待消费者重新平衡后继续消费。

kafka消费者平衡策略
https://mp.weixin.qq.com/s/VDun_8TUhsHYN_YDnxQTBg

参考链接：

https://www.orchome.com/588
https://juejin.im/post/6844903713916583944
https://stackoom.com/question/3pgaQ/%E5%8D%A1%E5%A4%AB%E5%8D%A1%E4%B8%8D%E6%96%AD%E9%87%8D%E6%96%B0%E5%B9%B3%E8%A1%A1%E6%B6%88%E8%B4%B9%E8%80%85