Ken Chen chenziliang

Benchmark commands

Producer

Setup

bin/kafka-topics.sh \
  --zookeeper zookeeper.example.com:2181 \
  --create \

Effective Modern CMake

Getting Started

For a brief user-level introduction to CMake, watch C++ Weekly, Episode 78, Intro to CMake by Jason Turner. LLVM’s CMake Primer provides a good high-level introduction to the CMake syntax. Go read it now.

After that, watch Mathieu Ropert’s CppCon 2017 talk Using Modern CMake Patterns to Enforce a Good Modular Design (slides). It provides a thorough explanation of what modern CMake is and why it is so much better than “old school” CMake. The modular design ideas in this talk are based on the book [Large-Scale C++ Software Design](https://www.amazon.de/Large-Scale-Soft

Building Kafka from the Hardware - up

Higher Message Retention ? - Increase disk size
Higher Message Throughput ? - Increase network capacity
Higher Producer Performance ? - Increase Disk I/O speed
Higher Consumer Performance ? - Increase Memory

Critical Configurations (Consumer)

queued.min.messages
fetch.wait.max.ms
socket.blocking.max.ms

Moved to git repository: https://github.com/denji/golang-tls

Generate private key (.key)

# Key considerations for algorithm "RSA" ≥ 2048-bit
openssl genrsa -out server.key 2048

# Key considerations for algorithm "ECDSA" ≥ secp384r1
# List ECDSA the supported curves (openssl ecparam -list_curves)

	CREATE TABLE states_raw(
	d date,
	uid UInt64,
	first_name String,
	last_name String,
	modification_timestamp_mcs DateTime64(3) default now64(3)
	) ENGINE = Null;

	CREATE TABLE final_states_by_month(
	d date,

	#
	# Building V8 for alpine is a real pain. We have to compile from source, because it has to be
	# linked against musl, and we also have to recompile some of the build tools as the official
	# build workflow tends to assume glibc by including vendored tools that link against it.
	#
	# The general strategy is this:
	#
	# 1. Build GN for alpine (this is a build dependency)
	# 2. Use depot_tools to fetch the V8 source and dependencies (needs glibc)
	# 3. Build V8 for alpine

	https://stackoverflow.com/questions/64369622/using-sample-in-clickhouse-seems-to-read-all-rows-and-more-bytes-is-this-expect

	CREATE TABLE table_one
	( timestamp UInt64,
	transaction_id UInt64,
	banner_id UInt16,
	value UInt32
	)
	ENGINE = MergeTree()
	PARTITION BY toYYYYMMDD(toDateTime(timestamp))

	sudo yum install numactl-devel
	wget https://fast.dpdk.org/rel/dpdk-17.11.2.tar.xz
	cd dpdk-stable-17.11.2/
	make config T=x86_64-native-linuxapp-gcc
	make
	sudo modprobe uio
	sudo insmod /home/ec2-user/dpdk-stable-17.11.2/build/kmod/igb_uio.ko
	sudo yum install pciutils
	sudo vi /etc/sysctl.conf vm.nr_hugepages = 1024
	sudo sysctl -p

	#include <errno.h>
	#include <string.h>

	#include <iostream>
	#include <sys/types.h> /* See NOTES */
	#include <sys/socket.h>

	#include <sys/types.h>
	#include <sys/stat.h>
	#include <fcntl.h>

	package main

	import (
	"crypto/tls"
	"crypto/x509"
	"flag"
	"io"
	"io/ioutil"
	"log"
	"os"