Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save yofayed/db33c817f00805a07c4162edb257ce24 to your computer and use it in GitHub Desktop.

Select an option

Save yofayed/db33c817f00805a07c4162edb257ce24 to your computer and use it in GitHub Desktop.
Intercept and manipulate HTTPs traffic with Python and mitmproxy

Intercepts HTTPs Traffic with Python & mitmproxy

Introduction

Modern applications usually make use of back-end API servers to provide their services. With a non-transparent HTTPs proxy, which intercepts the communication between clients and servers (aka the man-in-the-middle scheme), you can easily manipulate both API requests and responses.

This manual helps you create your own proxy with Python and mitmproxy/libmproxy. Mitmproxy ships with both a standalone command-line tool (mitmproxy) and a Python library (libmproxy).

Requirements

  • Python, >= 2.7.3
    • install through brew install python on Mac OS X
  • mitmproxy
    1. git clone https://github.com/cortesi/mitmproxy.git
    2. cd mitmproxy
    3. sudo python setup.py install
  • netlib, version matching mitmproxy
    1. git clone https://github.com/cortesi/netlib.git
    2. cd netlib
    3. sudo python setup.py install
  • PyOpenSSL, >= 0.13
    1. install OpenSSL development package (through sudo apt-get install libssl-dev on Ubuntu)
    2. download pyOpenSSL-0.13.tar.gz from PyPi website
    3. tar xvf pyOpenSSL-0.13.tar.gz
    4. cd pyOpenSSL-0.13
    5. python setup.py build
    6. sudo python setup.py install
  • pyasn1, >= 0.1.2
    • pip install pyasn1

Note: In my experience, mitmproxy depends on the latest netlib and PyOpenSSL, which cannot be installed from Pip. You may download the source tarball and install them manually.

Generate SSL Private Key and Certificate

  1. openssl genrsa -out mitmproxy.key 2048
  2. openssl req -new -x509 -key mitmproxy.key -out mitmproxy.crt -days 3650 -subj /CN=MitmProxy
  3. cat mitmproxy.key mitmproxy.crt > mitmproxy.pem
  4. install mitmproxy.crt on you device (desktop browser, iPhone, Android, etc.)

Example of proxy.py

#!/usr/bin/env python
# -*- encoding: utf-8 -*-
from libmproxy import controller, proxy
import os, sys, re, datetime, json

class InterceptingMaster (controller.Master):
  def __init__ (self, server):
    controller.Master.__init__(self, server)

  def run (self):
    while True:
      try:
        controller.Master.run(self)
      except KeyboardInterrupt:
        print 'KeyboardInterrupt received. Shutting down'
        self.shutdown()
        sys.exit(0)
      except Exception:
        print 'Exception catched. Intercepting proxy restarted'
        pass

  def handle_request (self, msg):
    timestamp = datetime.datetime.today().strftime("%Y/%m/%d %H:%M:%S")
    client_ip = msg.client_conn.address[0]
    request_url = '%s://%s%s' % (msg.scheme, .msg.host, msg.path)
    print '[%s %s] %s %s' % (timestamp, client_ip, msg.method, request_url)

    msg.reply()

  def handle_response (self, msg):
    # simple substitution for https://example.org/api/users/:id.json
    if "example.org" in msg.request.host:
      regex = re.compile("/api/users/(\d+).json")
      match = regex.search(msg.request.path)
      if match and msg.content:
        c = msg.replace('"private_data_accessible":false', '"private_data_accessible":true')
        if c > 0: print '[Example.org] Private info of user #%s revealed' % (match.groups()[0])

    # JSON manipulation for https://example.com/api/v2
    if ("example.com" in msg.request.host) and ("action=user_profile" in msg.request.content):
      msg.decode() # need to decode the message first
      data = json.loads(msg.content) # parse JSON with decompressed content
      data["access_granted"] = true
      msg.content = json.dumps(data) # write back our changes
      print '[Example.com] Access granted of user profile #%s' % data["id"]

    # Response inspection for https://example.net
    if "example.net" in msg.request.host:
      data = msg.get_decoded_content() # read decompressed content without modifying msg
      print '[Example.net] Respones: %s' % data

    msg.reply()


def main (argv):
  config = proxy.ProxyConfig(
    cacert = os.path.expanduser("./mitmproxy.pem"),
  )
  server = proxy.ProxyServer(config, 8080)
  print 'Intercepting Proxy listening on 8080'
  m = InterceptingMaster(server)
  m.run()

if __name__ == '__main__':
  main(sys.argv)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment