Skip to content

Instantly share code, notes, and snippets.

@saimon789
saimon789 / benchmark-commands.txt
Created December 20, 2024 14:59 — forked from jkreps/benchmark-commands.txt
Kafka Benchmark Commands
Producer
Setup
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test-rep-one --partitions 6 --replication-factor 1
bin/kafka-topics.sh --zookeeper esv4-hcl197.grid.linkedin.com:2181 --create --topic test --partitions 6 --replication-factor 3
Single thread, no replication
bin/kafka-run-class.sh org.apache.kafka.clients.tools.ProducerPerformance test7 50000000 100 -1 acks=1 bootstrap.servers=esv4-hcl198.grid.linkedin.com:9092 buffer.memory=67108864 batch.size=8196
@saimon789
saimon789 / Makefile
Created July 7, 2024 14:30 — forked from mpneuried/Makefile
Simple Makefile to build, run, tag and publish a docker containier to AWS-ECR
# import config.
# You can change the default config with `make cnf="config_special.env" build`
cnf ?= config.env
include $(cnf)
export $(shell sed 's/=.*//' $(cnf))
# import deploy config
# You can change the default deploy config with `make cnf="deploy_special.env" release`
dpl ?= deploy.env
include $(dpl)
@saimon789
saimon789 / docker-registry-delete-image.txt
Created March 26, 2024 13:54 — forked from DavidPesticcio/docker-registry-delete-image.txt
remove images from docker registry v2.4
How to delete an image tag/image from docker registry v2.4
1) Re/Start registry with delete option enabled
Edit the config.yml
storage:
delete:
enabled: true

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

  1. List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

  1. Reset the consumer offset for a topic (preview)