mysugar

set http or socks proxy environment variables

# set http proxy
export http_proxy=http://PROXYHOST:PROXYPORT

# set http proxy with user and password
export http_proxy=http://USERNAME:PASSWORD@PROXYHOST:PROXYPORT

# set http proxy with user and password (with special characters)

Docker compose Kafka, Zookeeper and Kafka manager

Gist inspired by this one intending to be an updated version of it : https://gist.github.com/17twenty/d619f922ab209f750630824f7c6836e3

Install

Create the environment variable that contains our host name (IP address) :

$ export EXPOSED_HOSTNAME=$(ip addr | grep 'state UP' -A2 | tail -n1 | awk '{print $2}' | cut -f1  -d'/')

Create the folders :

Opening and closing an SSH tunnel in a shell script the smart way

I recently had the following problem:

From an unattended shell script (called by Jenkins), run a command-line tool that accesses the MySQL database on another host.
That tool doesn't know that the database is on another host, plus the MySQL port on that host is firewalled and not accessible from other machines.

We didn't want to open the MySQL port to the network, but it's possible to SSH from the Jenkins machine to the MySQL machine. So, basically you would do something like

ssh -L 3306:localhost:3306 remotehost

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs

	#!/bin/bash

	# brew install jq
	# brew install qrencode

	config_file=v2rayx*.json
	rss_file=rss.txt

	cat $config_file \| \
	jq --raw-output '

	knowledge dump on container runtimes

	KataContainers
	- image coupled with kernel
	- light vm layer
	- can run in nested virturalization environments if hardware supports and you can enable it in bios (ex. only bare metal EC2 instances, limits many cloud providers)
	- slower startup time
	- OCI compliant
	- previously known as ClearContainers by Intel

	FROM golang:alpine as builder
	WORKDIR /app
	#the following 2 steps are optional if your image does not already have the certificate
	# package installed, golang:alpine now seems to have it. But a more base image could be missing it.
	#RUN apk update && apk upgrade && apk add --no-cache ca-certificates
	#RUN update-ca-certificates
	ADD main.go /app/main.go
	RUN CGO_ENABLED=0 GOOS=linux go build -a -ldflags="-s -w" -installsuffix cgo -o app .

	FROM scratch

	import org.apache.spark.sql.expressions.MutableAggregationBuffer
	import org.apache.spark.sql.expressions.UserDefinedAggregateFunction
	import org.apache.spark.sql.Row
	import org.apache.spark.sql.types._
	import org.apache.spark.mllib.linalg.SparseVector
	import org.apache.spark.mllib.linalg.Vector
	import org.apache.spark.mllib.linalg.Vectors
	import org.apache.spark.mllib.linalg.VectorUDT

	// MLG: this is highly unoptimized, but likely good enough for now

	events {
	worker_connections 1024;
	}

	http {
	default_type text/html;
	access_log /dev/stdout;
	sendfile on;
	keepalive_timeout 65;

	Latency Comparison Numbers (~2012)
	----------------------------------
	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns 3 us
	Send 1K bytes over 1 Gbps network 10,000 ns 10 us
	Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD