Skip to content

Instantly share code, notes, and snippets.

@Cuneytt

Cuneytt/s3_down.sh

Forked from nfelger/s3_down.sh
Created June 22, 2017 18:08
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Cuneytt/5e92eb2cddb6ee92fcfa850703a484b8 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Cuneytt/5e92eb2cddb6ee92fcfa850703a484b8 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @nfelger nfelger created this gist Apr 15, 2015.
    15 changes: 15 additions & 0 deletions s3_down.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,15 @@
    #!/bin/bash
    # Inspired by http://dl.getipaddr.net/ and http://curl.haxx.se/mail/archive-2014-10/0006.html
    file=path/to/file
    bucket=your-bucket
    contentType="application/octet-stream"
    dateValue=`date -R`
    resource="/${bucket}/${file}"
    s3Key=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ec2-app-role | grep 'AccessKeyId' | sed 's/.* "\([^"]*\).*/\1/'`
    s3Secret=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ec2-app-role | grep 'SecretAccessKey' | sed 's/.* "\([^"]*\).*/\1/'`
    token=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ec2-app-role | grep 'Token' | sed 's/.* "\([^"]*\).*/\1/'`
    stringToSign="GET\n\n${contentType}\n${dateValue}\nx-amz-security-token:${token}\n${resource}"
    signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
    curlOut=$(curl -s -w "%{speed_download}" -o /dev/null -X GET -H "Host: ${bucket}.s3.amazonaws.com" -H "Date: ${dateValue}" -H "Content-Type: ${contentType}" -H "Authorization: AWS ${s3Key}:${signature}" -H "x-amz-security-token: ${token}" https://${bucket}.s3.amazonaws.com/${file})
    # Output x-fer speed in MB/sec
    echo "scale=2;$curlOut/1048576" | bc -q
    16 changes: 16 additions & 0 deletions s3_up.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,16 @@
    #!/bin/bash
    # Inspired by http://dl.getipaddr.net/ and http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash
    file=file-to-upload
    bucket=your-bucket
    pathInBucket=upload-tests/`date +%s`
    contentType="application/octet-stream"
    dateValue=`date -R`
    resource="/${bucket}/${pathInBucket}"
    s3Key=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ec2-app-role | grep 'AccessKeyId' | sed 's/.* "\([^"]*\).*/\1/'`
    s3Secret=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ec2-app-role | grep 'SecretAccessKey' | sed 's/.* "\([^"]*\).*/\1/'`
    token=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ec2-app-role | grep 'Token' | sed 's/.* "\([^"]*\).*/\1/'`
    stringToSign="PUT\n\n${contentType}\n${dateValue}\nx-amz-security-token:${token}\n${resource}"
    signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
    curlOut=$(curl -s -w "%{speed_upload}" -o /dev/null -X PUT -T "${file}" -H "Host: ${bucket}.s3.amazonaws.com" -H "Date: ${dateValue}" -H "Content-Type: ${contentType}" -H "Authorization: AWS ${s3Key}:${signature}" -H "x-amz-security-token: ${token}" https://${bucket}.s3.amazonaws.com/${pathInBucket})
    # Output x-fer speed in MB/sec
    echo "scale=2;$curlOut/1048576" | bc -q