Skip to content

Instantly share code, notes, and snippets.

@karlkvist
karlkvist / pipelines.py
Created October 19, 2017 10:14 — forked from tzermias/pipelines.py
Scrapy MySQL pipeline. Just a mirror to the asynchronous MySQL pipeline. Copy-paste it directly to pipelines.py. Database credentials are stored in settings.py. Based on http://snipplr.com/view/66986/
import MySQLdb.cursors
from twisted.enterprise import adbapi
from scrapy.xlib.pydispatch import dispatcher
from scrapy import signals
from scrapy.utils.project import get_project_settings
from scrapy import log
SETTINGS = get_project_settings()