Forked from marwei/how_to_reset_kafka_consumer_group_offset.md
Created
September 15, 2022 09:53
-
-
Save ram-pi/5072e13ad4293a7d0957b446716ee37b to your computer and use it in GitHub Desktop.
Revisions
-
marwei created this gist
Nov 9, 2017 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,32 @@ Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli `kafka-consumer-groups` command. 1. List the topics to which the group is subscribed ```bash kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe ``` Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions. 2. Reset the consumer offset for a topic (preview) ```bash kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --topic <topic_name> --reset-offsets --to-earliest ``` This will print the expected result of the reset, but not actually run it. 3. Reset the consumer offset for a topic (execute) ```bash kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --topic <topic_name> --reset-offsets --to-earliest --execute ``` This will execute the reset and reset the consumer group offset for the specified topic back to 0. 4. Repeat 1 to check if the reset is successful ### Note * The consumer group must have no running instance when performing the reset. Otherwise the reset will be rejected. * There are many other resetting options, run `kafka-consumer-groups` for details * --shift-by <positive_or_negative_integer> * --to-current * --to-latest * --to-offset <offset_integer> * --to-datetime <datetime_string> * --by-duration <duration_string> * The command also provides an option to reset offsets for all topics the consumer group subscribes to: `--all-topics`