Skip to content

Instantly share code, notes, and snippets.

View silverwolfx10's full-sized avatar

Leandro Menezes silverwolfx10

View GitHub Profile
@silverwolfx10
silverwolfx10 / XML_breaker.py
Last active August 2, 2019 00:37 — forked from nicwolff/XML_breaker.py
Python script to break large XML files
import os
import sys
from xml.sax import parse
from xml.sax.saxutils import XMLGenerator
class CycleFile(object):
def __init__(self, filename, output):
self.basename, self.ext = os.path.splitext(filename)
self.output = output
@silverwolfx10
silverwolfx10 / xml_split.py
Created August 2, 2019 00:00 — forked from benallard/xml_split.py
Small python script to split huge XML files into parts. It takes one or two parameters. The first is always the huge XML file, and the second the size of the wished chunks in Kb (default to 1Mb) (0 spilt wherever possible) The generated files are called like the original one with an index between the filename and the extension like that: bigxml.…
#!/usr/bin/env python
import os
import xml.parsers.expat
from xml.sax.saxutils import escape
from optparse import OptionParser
from math import log10
# How much data we process at a time

Nginx Tuning For Best Performance

For this configuration you can use web server you like, i decided, because i work mostly with it to use nginx.

Generally, properly configured nginx can handle up to 400,000 to 500,000 requests per second (clustered), most what i saw is 50,000 to 80,000 (non-clustered) requests per second and 30% CPU load, course, this was 2xIntel Xeon with HT enabled, but it can work without problem on slower machines.

You must understand that this config is used in testing environment and not in production so you will need to find a way to implement most of those features best possible for your servers.

First, you will need to install nginx, my way to install nginx is compiling it from source, but for now we will use apt-get