Skip to content

Instantly share code, notes, and snippets.

View leoprince's full-sized avatar

Leo Prince leoprince

1 follower · 9 following
View GitHub Profile
@leoprince
leoprince / how_to_reset_kafka_consumer_group_offset.md
Created June 2, 2020 07:51 — forked from marwei/how_to_reset_kafka_consumer_group_offset.md
How to Reset Kafka Consumer Group Offset

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

  1. List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

  1. Reset the consumer offset for a topic (preview)



  


  




    


          
    

      

        
        

          

              @leoprince
          

          

            
                leoprince
                / benchmark-commands.txt
            
            

              Created
              July 22, 2016 17:26
                — forked from jkreps/benchmark-commands.txt
            

              
                Kafka Benchmark Commands
              
          

        

      

        

  

    
    


        



  


  

        

          
          Producer
        

        

          
          

        

        

          
          Setup
        

        

          
          bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test-rep-one --partitions 6 --replication-factor 1
        

        

          
          bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test --partitions 6 --replication-factor 3
        

        

          
          

        

        

          
          Single thread, no replication
        

        

          
          

        

        

          
          bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 -1 acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864 batch.size=8196