Wayne Huang waynehuang13

Effective Engineer - Notes

By Edmond Lau
Highly Recommended 👍
http://www.theeffectiveengineer.com/

What's an Effective Engineer?

They are the people who get things done. Effective Engineers produce results.

Adopt the Right Mindsets

How to recover lost Python source code if it's still resident in-memory

I screwed up using git ("git checkout --" on the wrong file) and managed to delete the code I had just written... but it was still running in a process in a docker container. Here's how I got it back, using https://pypi.python.org/pypi/pyrasite/ and https://pypi.python.org/pypi/uncompyle6

Attach a shell to the docker container

Install GDB (needed by pyrasite)

apt-get update && apt-get install gdb

Comprehensions in Python the Jedi way

Beautiful is better than ugly. Explicit is better than implicit.

-- The Zen of Python

I frequently deal with collections of things in the programs I write. Collections of droids, jedis, planets, lightsabers, starfighters, etc. When programming in Python, these collections of things are usually represented as lists, sets and dictionaries. Oftentimes, what I want to do with collections is to transform them in various ways. Comprehensions is a powerful syntax for doing just that. I use them extensively, and it's one of the things that keep me coming back to Python. Let me show you a few examples of the incredible usefulness of comprehensions.

All of the tasks presented in the examples can be accomplished with the extensive standard library available in Python. These solutions would arguably be more terse and efficient in some cases. I don't have anything against the standard library. To me there is a certain

	#!/bin/bash
	set -x

	USAGE() {
	cat << EOF
	Usage: ${0##*/} <-i ident.p12> [-p password] <-m profile.mobileprovision> [-a com.example.app] [-n NewName] [-I Info.plist]
	-i ident.p12 The signing identity file.
	-p password The password of signing identity file.
	-m profile.mobileprovision Signing provision profile
	-a com.example.app Override CFBundleIdentifier

	The basic idea here is to substantiate the claims made by this square post:
	http://corner.squareup.com/2011/06/postgresql-data-is-important.html

	In PostgreSQL, and MySQL (MyISAM and InnoDB) I create millions of rows and then add
	and remove columns and add and remove indexes. For columns without defaults this is
	basically free in PostgreSQL and O(n) in MySQL. For adding indexes its at best O(n)
	everywhere, but with PostgreSQL it claims not to do any locking that would otherwise
	prevent table interaction.

	Also, PostgreSQL has _awsome_ documentation (it has real examples!). I always get

	# LBs have 8 cores. One is used for haproxy, rest are used for nginx workers
	worker_processes 7;
	worker_rlimit_nofile 90000;

	pid <PID_FILE>;

	events {
	use epoll;
	multi_accept off;
	accept_mutex off;

	# gem 'activerecord', '4.1.1' # passes
	gem 'activerecord', '4.1.2.rc1' # fails with ActiveModel::MassAssignmentSecurity::Error: Can't mass-assign protected attributes for Membership: group_id
	gem 'protected_attributes', '1.0.7'
	require 'active_record'
	require 'minitest/autorun'
	require 'logger'
	require 'protected_attributes'

	# Ensure backward compatibility with Minitest 4
	Minitest::Test = MiniTest::Unit::TestCase unless defined?(Minitest::Test)

	# inspired by http://ariejan.net/2010/08/23/resque-how-to-requeue-failed-jobs

	# retry all failed Resque jobs except the ones that have already been retried
	# This is, for instance, useful if you have already retried some jobs via the web interface.
	Resque::Failure.count.times do \|i\|
	Resque::Failure.requeue(i) unless Resque::Failure.all(i, 1)['retried_at'].present?
	end

	# retry all :)
	Resque::Failure.count.times do \|i\|

	max_connections = 1500 # (change requires restart)
	shared_buffers = 12000MB # min 128kB, based on 80GB RAM DB
	temp_buffers = 8MB # min 800kB
	work_mem = 64MB # min 64kB
	maintenance_work_mem = 512MB # min 1MB

	wal_level = hot_standby # minimal, archive, or hot_standby
	checkpoint_segments = 64 # in logfile segments, min 1, 16MB each
	checkpoint_completion_target = 0.9 # checkpoint target duration, 0.0 - 1.0
	max_wal_senders = 6 # max number of walsender processes

	To setup a test environment

	- install vmware Fusion (trial)
	- install A machine (windows 2003 / 64 Bit /trial available)
	- install B machine (ESX) (trial download)
	- install on A, vsphere server (trial download/requires Win2003/Win2008)
	- install on A, vsphere client
	- import ESX host B into vsphere via vsphere client
	- create a user vmware (Local User)
	- put it in Administrator group