Last active
January 6, 2025 06:31
-
-
Save nemani/defdde356b6678352bcd4af69b7fe529 to your computer and use it in GitHub Desktop.
Download All Lambda Functions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Parallelly download all aws-lambda functions | |
| # Assumes you have ran `aws configure` and have output-mode as "text" | |
| # Works with "aws-cli/1.16.72 Python/3.6.7 Linux/4.15.0-42-generic botocore/1.12.62" | |
| download_code () { | |
| local OUTPUT=$1 | |
| aws lambda get-function --function-name $OUTPUT --query 'Code.Location' | xargs wget -O ./lambda_functions/$OUTPUT.zip | |
| } | |
| mkdir -p lambda_functions | |
| for run in $(aws lambda list-functions | cut -f 6 | xargs); | |
| do | |
| download_code "$run" & | |
| done | |
| echo "Completed Downloading all the Lamdba Functions!" | |
| # If you want to download only ones with specific prefix | |
| # https://github.com/sambhajis-gdb/download_all_lambda_function/blob/master/get_lambda_with_prefix.sh | |
| # Credits to https://stackoverflow.com/users/6908140/sambhaji-sawant for the |
It crashed my 64GB RAM mac, Twice :/
I can verify that the original version does that on Macs. @handeglc removed the & to not run in background. I did the same and also added a loop to capture all regions. It could be improved even more of course. Thanks @nemani for posting this gist!
# Downloads all aws-lambda functions to a subdirectory
# Assumes you have a role that has at least read access to lambda.
# Credits to https://gist.github.com/nemani/defdde356b6678352bcd4af69b7fe529
# trap ctrl-c and call ctrl_c()
trap ctrl_c INT
function ctrl_c() {
echo "** CTRL-C Detected - aborting..."
exit -1
}
download_region(){
REGION=$1
echo Searching region: $REGION
for code in $(aws lambda list-functions --region $REGION --query 'Functions[].FunctionName' --output text);
do
download_path=./lambda_functions/$REGION/$code
mkdir -p $download_path
aws lambda get-function --function-name $code --region $REGION --query 'Code.Location' | xargs wget --quiet -O $download_path/$code.zip
unzip -o -f -d $download_path $download_path/$code.zip
done
}
# search all regions
for region in $(aws ec2 describe-regions --query "Regions[].{Name:RegionName}" --output text | sort -r); do
download_region $region
done
echo "Completed Downloading all the Lamdba Functions!"
Thanks for sharing, the code didn't work as stated in some previous replies I believe because of the filtering criteria to get the lambda function name.
AWS CLI is returning a JSON format which can be better explored using jq
If the above code didn't work for you too, try my implementation of the fix here.
@abhay330 thanks man! Worked very well
very nice collaboration. I extended it in a fork https://gist.github.com/bf4/825009c3e49bf53259e8c18c4c38f190
#!/bin/bash
#
# Usage:
# ./download_all.sh
# ./download_all.sh download us-east-1 my-function
# ./download_all.sh help
#
# Downloads all aws-lambda functions to a subdirectory
# Assumes you have a role that has at least read access to lambda.
# Credits to https://gist.github.com/nemani/defdde356b6678352bcd4af69b7fe529
set -euo pipefail
IFS=$'\n\t'
# Influenced by https://dev.to/thiht/shell-scripts-matter
abort=0
show_help() {
abort=${abort:0}
sed -ne '/^#/!q;s/.\{1,2\}//;1d;p' < "$0" >&$((abort+1))
exit $abort
}
# shellcheck disable=SC2034
export AWS_PROFILE="${AWS_PROFILE:-your-profile-here}"
readonly DOWNLOAD_PATH_BASE="./lambda_functions"
# trap ctrl-c and call ctrl_c()
trap ctrl_c INT
ctrl_c() {
echo "** CTRL-C Detected - aborting..."
exit 1
}
get_function_names() {
local region; region="$1"
aws lambda list-functions --region "$region" --query 'Functions[].FunctionName' --output text
}
get_function() {
local region; region="$1"
local function_name; function_name="$2"
local download_path
download_path="${DOWNLOAD_PATH_BASE}/$region/$function_name"
mkdir -p "$download_path"
aws lambda get-function --function-name "$function_name" --region "$region" --query 'Code.Location' \
| xargs wget --quiet -O "$download_path/$function_name.zip"
unzip -o -f -d "$download_path" "$download_path/$function_name.zip"
}
download_region(){
local region; region="$1"
echo "searching region: $region"
for function_name in $(get_function_names "$region"); do
get_function "$region" "$function_name"
done
}
get_regions(){
aws ec2 describe-regions --query "Regions[].{Name:RegionName}" --output text | sort -r
}
main() {
# search all regions
for region in $(get_regions); do
download_region "$region"
done
echo "Completed Downloading all the Lamdba Functions!"
}
case "${1:-}" in
help|-h|--help)
show_help
;;
download)
shift
get_function "$1" "$2"
;;
*)
shift
main
;;
esac
The original script worked for me on a Mac M1, ...BUT it deleted all the Lambda functions on AWS after downloading them.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It crashed my 64GB RAM mac, Twice :/