Skip to content

Instantly share code, notes, and snippets.

@ozlmulg
ozlmulg / kafka-cheat-sheet.md
Created July 22, 2020 19:35 — forked from filipefigcorreia/kafka-cheat-sheet.md
Apache Kafka Cheat Sheet

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --delete-config retention.ms

@ozlmulg
ozlmulg / SQL-Insert-Statements.sql.groovy
Created April 5, 2019 06:54 — forked from pasali/SQL-Insert-Statements.sql.groovy
SQL Bulk insert export script for DataGrip
SEP = ", "
QUOTE = "\'"
NEWLINE = System.getProperty("line.separator")
KEYWORDS_LOWERCASE = com.intellij.database.util.DbSqlUtil.areKeywordsLowerCase(PROJECT)
KW_INSERT_INTO = KEYWORDS_LOWERCASE ? "insert into " : "INSERT INTO "
KW_VALUES = KEYWORDS_LOWERCASE ? "values" : "VALUES"
KW_NULL = KEYWORDS_LOWERCASE ? "null" : "NULL"
OUT.append(KW_INSERT_INTO)

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

  1. List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

  1. Reset the consumer offset for a topic (preview)
@ozlmulg
ozlmulg / keybase.md
Created November 14, 2017 06:42
keybase.md

Keybase proof

I hereby claim:

  • I am ozlmulg on github.
  • I am ozlmulg (https://keybase.io/ozlmulg) on keybase.
  • I have a public key whose fingerprint is F7B0 9C3C 2AA0 DA2F E3A5 C4FF 5D86 1491 3143 F6F5

To claim this, I am signing this object:

Keybase proof

I hereby claim:

  • I am ozlmulg on github.
  • I am ozlmulg (https://keybase.io/ozlmulg) on keybase.
  • I have a public key ASBBF7C9nVYpEAXFuQ5tvixbZpcKd3oJRiz-r7KhOSC94go

To claim this, I am signing this object:

import com.fasterxml.jackson.annotation.JsonPropertyOrder;
import com.fasterxml.jackson.databind.ObjectWriter;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;