YuZou hopkings2008

Moved to git repository: https://github.com/denji/nginx-tuning

NGINX Tuning For Best Performance

For this configuration you can use web server you like, i decided, because i work mostly with it to use nginx.

Generally, properly configured nginx can handle up to 400K to 500K requests per second (clustered), most what i saw is 50K to 80K (non-clustered) requests per second and 30% CPU load, course, this was 2 x Intel Xeon with HyperThreading enabled, but it can work without problem on slower machines.

You must understand that this config is used in testing environment and not in production so you will need to find a way to implement most of those features best possible for your servers.

	#!/usr/bin/env python

	import ast
	import json
	import os
	import subprocess
	import argparse
	import sys

	FNULL = open(os.devnull, 'w')

	[global]
	ioengine=libaio
	invalidate=1
	ramp_time=30
	iodepth=1
	runtime=180
	time_based
	direct=1

	[write-sdc-4k-seq]

	Install s3fs on Mac OS X

	1 - Install Homebrew - http://brew.sh/
	ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

	2 - Use Homebrew to install s3fs + dependencies
	brew install s3fs

	3 - Do some custom stuff. I only used the first step from here -> https://gist.github.com/fukayatsu/3910097
	sudo /bin/cp -rfX /usr/local/Cellar/fuse4x-kext/0.9.2/Library/Extensions/fuse4x.kext /Library/Extensions

	Latency Comparison Numbers (~2012)
	----------------------------------
	L1 cache reference 0.5 ns
	Branch mispredict 5 ns
	L2 cache reference 7 ns 14x L1 cache
	Mutex lock/unlock 25 ns
	Main memory reference 100 ns 20x L2 cache, 200x L1 cache
	Compress 1K bytes with Zippy 3,000 ns 3 us
	Send 1K bytes over 1 Gbps network 10,000 ns 10 us
	Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD

	#!/bin/bash

	# Minimum TODOs on a per job basis:
	# 1. define name, application jar path, main class, queue and log4j-yarn.properties path
	# 2. remove properties not applicable to your Spark version (Spark 1.x vs. Spark 2.x)
	# 3. tweak num_executors, executor_memory (+ overhead), and backpressure settings

	# the two most important settings:
	num_executors=6
	executor_memory=3g