Skip to content

Instantly share code, notes, and snippets.

@emersonf

emersonf/s3etag.sh

Last active April 4, 2025 08:33
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save emersonf/7413337 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save emersonf/7413337 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. emersonf revised this gist Nov 12, 2013. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion s3etag.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -13,7 +13,7 @@ if [ ! -f "$file" ]; then
    fi

    partSizeInMb=$2
    fileSizeInMb=$(du -m $file | cut -f 1)
    fileSizeInMb=$(du -m "$file" | cut -f 1)
    parts=$((fileSizeInMb / partSizeInMb))
    if [[ $((fileSizeInMb % partSizeInMb)) -gt 0 ]]; then
    parts=$((parts + 1));
  2. emersonf created this gist Nov 11, 2013.
    31 changes: 31 additions & 0 deletions s3etag.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,31 @@
    #!/bin/bash

    if [ $# -ne 2 ]; then
    echo "Usage: $0 file partSizeInMb";
    exit 0;
    fi

    file=$1

    if [ ! -f "$file" ]; then
    echo "Error: $file not found."
    exit 1;
    fi

    partSizeInMb=$2
    fileSizeInMb=$(du -m $file | cut -f 1)
    parts=$((fileSizeInMb / partSizeInMb))
    if [[ $((fileSizeInMb % partSizeInMb)) -gt 0 ]]; then
    parts=$((parts + 1));
    fi

    checksumFile=$(mktemp -t s3md5)

    for (( part=0; part<$parts; part++ ))
    do
    skip=$((partSizeInMb * part))
    $(dd bs=1m count=$partSizeInMb skip=$skip if="$file" 2>/dev/null | md5 >>$checksumFile)
    done

    echo $(xxd -r -p $checksumFile | md5)-$parts
    rm $checksumFile