nickthorpe

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname

This is taken from here. http://stackoverflow.com/questions/67699/how-do-i-clone-all-remote-branches-with-git/72156#72156 Want this as a gist because I always come back to this posting.

First, clone a remote git repository and cd into it:

$ git clone git://example.com/myproject
$ cd myproject

Next, look at the local branches in your repository:

	#!/usr/bin/python

	# This little script creates users in an airflow instance so it can be open to the public.
	# It gets the password in plain text so be careful where you run it.
	# You can properly invoke the script as follows:
	# ./add_user.py <username> <[email protected]> <secretpassword>

	import airflow, sys
	from airflow import models, settings
	from airflow.contrib.auth.backends.password_auth import PasswordUser

	# execute query inline
	hive -e 'select count(1) from member' \| sed 's/[\t]/,/g' > test.csv

	# execute query in file
	hive -f test.hql \| sed 's/[\t]/,/g' > test.csv

	GRANT USAGE on schema analyzer_django to read_full;
	GRANT SELECT ON ALL TABLES IN SCHEMA public to read_full;
	GRANT CONNECT ON DATABASE nlp to read_full;
	GRANT SELECT ON ALL SEQUENCES IN SCHEMA public to read_full;

	sudo adduser [user-name]

	# type in password to give the user

	# run command to find what groups you're in
	groups

	# add user to the corresponding groups
	sudo gpasswd -a [user-name] [group-name]

	from multiprocessing import Pool
	from functools import partial
	import numpy as np

	def parallelize(data, func, num_of_processes=8):
	data_split = np.array_split(data, num_of_processes)
	pool = Pool(num_of_processes)
	data = pd.concat(pool.map(func, data_split))
	pool.close()
	pool.join()

	# check if an environment variable is set
	if [ -z ${var+x} ]; then echo "var is unset"; else echo "var is set to '$var'"; fi

	# look at a csv
	function pretty_csv {
	perl -pe 's/((?<=,)\|(?<=^)),/ ,/g;' "$@" \| column -t -s, \| less -F -S -X -K
	}

	# look at a psv
	function pretty_psv {

	# change users password
	sudo passwd <username>

	# add permission directories for user
	sudo chown -R $USER:$USER /var/www