Skip to content

Instantly share code, notes, and snippets.

@haranjackson

haranjackson/scrapy_lambda_layer.sh

Last active May 7, 2022 09:24
Show Gist options
  • Save haranjackson/81ab8dc3ae54bbe91c623d7c4f1e4f14 to your computer and use it in GitHub Desktop.
Save haranjackson/81ab8dc3ae54bbe91c623d7c4f1e4f14 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. haranjackson revised this gist Aug 15, 2019. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion scrapy_lambda_layer.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -9,7 +9,8 @@ cd build/scrapy
    zip -r ../../scrapy.zip python/
    cd ../..

    aws lambda publish-layer-version --layer-name Scrapy \
    aws lambda publish-layer-version \
    --layer-name Scrapy \
    --region $REGION \
    --description $VER \
    --zip-file fileb://scrapy.zip \
  2. haranjackson created this gist Aug 15, 2019.
    18 changes: 18 additions & 0 deletions scrapy_lambda_layer.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,18 @@
    REGION=eu-west-1
    VER=1.7.3
    RUNTIME=python3.7

    docker run -v $(pwd):/out -it lambci/lambda:build-$RUNTIME \
    pip install scrapy==$VER -t /out/build/scrapy/python

    cd build/scrapy
    zip -r ../../scrapy.zip python/
    cd ../..

    aws lambda publish-layer-version --layer-name Scrapy \
    --region $REGION \
    --description $VER \
    --zip-file fileb://scrapy.zip \
    --compatible-runtimes $RUNTIME

    rm -rf build *.zip