本文主要是介绍Offset commit with offsets failed,希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!
报错现象
2021/05/31-14:17:26.488 [] [JdqConsumerThread_unique_device_identification_mapper_yf_0] ERROR org.apache.kafka.clients.consumer.internals.ConsumerCoordinator- [Consumer clientId=4670fce6, groupId=hisotry1617153935172] Offset commit with offsets {unique_device_identification_mapper_yf-15=OffsetAndMetadata{offset=555905135, leaderEpoch=null, metadata=''}, unique_device_identification_mapper_yf-14=OffsetAndMetadata{offset=554744491, leaderEpoch=null, metadata=''}} failedorg.apache.kafka.clients.consumer.CommitFailedException: Offset commit cannot be completed since the consumer is not part of an active group for auto partition assignment; it is likely that the consumer was kicked out of the group.at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.sendOffsetCommitRequest(ConsumerCoordinator.java:1093)at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.doCommitOffsetsAsync(ConsumerCoordinator.java:913)at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsAsync(ConsumerCoordinator.java:880)at org.apache.kafka.clients.consumer.KafkaConsumer.commitAsync(KafkaConsumer.java:1584)at org.apache.kafka.clients.consumer.KafkaConsumer.commitAsync(KafkaConsumer.java:1552)at org.apache.kafka.clients.consumer.KafkaConsumer.commitAsync(KafkaConsumer.java:1529)
原因
由于取出的一批消息数量太大,consumer在session.timeout.ms时间之内没有消费完成引起kafka rebalance
解决方法
调大sessionTimeout,调小maxPollRecord
session.timeout.ms改为300000,即5分钟(原先配置为10000 即10s)
max.poll.records改为5000(原先配置为10000)
参考文档:https://blog.csdn.net/a_sunny_a/article/details/112270638
这篇关于Offset commit with offsets failed的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!