Skip to content

Instantly share code, notes, and snippets.

@mikebway
Last active December 14, 2019 05:55
Show Gist options
  • Save mikebway/a969ae6dc13f45abb9ddb883e4cbccfe to your computer and use it in GitHub Desktop.
Save mikebway/a969ae6dc13f45abb9ddb883e4cbccfe to your computer and use it in GitHub Desktop.

Revisions

  1. mikebway revised this gist Dec 14, 2019. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion AWS-CLI-Crib-Sheet.md
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    # AWS CLI Crib Sheet

    I might go for several nmonths without accessing my personal AWS account configuration, then when I need to do
    I might go for several months without accessing my personal AWS account configuration, then when I need to do
    something I find that I have forgotten how to do even the most basic of things. I created this crib sheet to
    save me from having to learn it all from first principles every time (especially that MFA tempoarary access
    token part that bites me again and again).
  2. mikebway revised this gist Dec 14, 2019. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions AWS-CLI-Crib-Sheet.md
    Original file line number Diff line number Diff line change
    @@ -1,9 +1,9 @@
    # AWS CLI Crib Sheet

    I might go for several nmonths without accessing my presonal AWS account configuration, then when I need to do
    I might go for several nmonths without accessing my personal AWS account configuration, then when I need to do
    something I find that I have forgotten how to do even the most basic of things. I created this crib sheet to
    save me from having to learn it all again (especially that MFA tempoarary access token part that bites me
    every time).
    save me from having to learn it all from first principles every time (especially that MFA tempoarary access
    token part that bites me again and again).

    ## Check the AWS CLI Version

  3. mikebway revised this gist Dec 14, 2019. 1 changed file with 35 additions and 0 deletions.
    35 changes: 35 additions & 0 deletions AWS-CLI-Crib-Sheet.md
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,10 @@
    # AWS CLI Crib Sheet

    I might go for several nmonths without accessing my presonal AWS account configuration, then when I need to do
    something I find that I have forgotten how to do even the most basic of things. I created this crib sheet to
    save me from having to learn it all again (especially that MFA tempoarary access token part that bites me
    every time).

    ## Check the AWS CLI Version

    The latest available version can be found here: <https://github.com/aws/aws-cli/releases>
    @@ -22,6 +27,26 @@ Using the AWS reccomended "Bundled Installer" (see [Installing the AWS CLI versi
    sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
    ```

    ## Configure Your Access Keys

    Run the following and answer the prompts to provide your access key ID, secret key, default region,
    and preferred output format:

    ```bash
    aws configure
    ```

    This should create or update the `~/.aws/credentials` file.

    You can confirm that all is well with these credentals using the following command to display
    basic information about your user:

    ```bash
    aws sts get-caller-identity
    ```

    This command will work no matter how limited your access rights might be.

    ## Obtaining Temporary Credentials for MFA Authentication

    Do you keep getting `You are not authorized to perform this operation` errors for almost every
    @@ -39,6 +64,16 @@ for the solution.
    aws s3 ls
    ```

    **NOTE:** If you have MFA configured for your user and have obtained temporary access keys as described
    by the "[How do I use an MFA token to authenticate access to my AWS resources through the AWS CLI?](https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/)"
    knowledge base article, and have stored this in a non-default section of your `~/.aws/credentials` file,
    then you will need to provide the name of that non-default section for this and most other commands.
    Something like this:

    ```bash
    aws s3 ls --profile mfa
    ```

    ## List all files in an S3 bucket

    ```bash
  4. mikebway revised this gist Dec 14, 2019. 1 changed file with 11 additions and 0 deletions.
    11 changes: 11 additions & 0 deletions AWS-CLI-Crib-Sheet.md
    Original file line number Diff line number Diff line change
    @@ -22,6 +22,17 @@ Using the AWS reccomended "Bundled Installer" (see [Installing the AWS CLI versi
    sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
    ```

    ## Obtaining Temporary Credentials for MFA Authentication

    Do you keep getting `You are not authorized to perform this operation` errors for almost every
    AWS CLI command that you try even though you have double checked your `~/.aws/credentials`
    file has been configured?

    Do you have MFA (Multi-Factor Authentication) enabled for your user? If yes, that is probably your issue.

    See [How do I use an MFA token to authenticate access to my AWS resources through the AWS CLI?](https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/)
    for the solution.

    ## List all S3 buckets

    ```bash
  5. mikebway revised this gist Dec 14, 2019. 1 changed file with 98 additions and 5 deletions.
    103 changes: 98 additions & 5 deletions AWS-CLI-Crib-Sheet.md
    Original file line number Diff line number Diff line change
    @@ -10,15 +10,108 @@ Check the installed version thus:
    aws --version
    ```

    ## Upgrade to Latest CLI Version
    ## Install or Upgrade to Latest CLI Version

    Using the AWS reccomended "Bundled Installer" (see [Installing the AWS CLI version 1](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv1.html)):

    1. Download and unpack the ZIP file from <https://s3.amazonaws.com/aws-cli/awscli-bundle.zip>

    1. Run the installer

    ```bash
    sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
    ```

    ```bash
    sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
    ```

    ## List all S3 buckets

    ```bash
    aws s3 ls
    ```

    ## List all files in an S3 bucket

    ```bash
    aws s3 ls s3://bucket-name --recursive
    ```

    ## Delete files that match a pattern

    ```bash
    aws s3 rm s3://bucket-name --recursive --only-show-errors --exclude "*" --include "your-pattern-here/*"
    ```

    **NOTE:** `--dryrun` displays what would happen but does not do it

    ## Copy all files in current directory files to S3

    ```bash
    aws s3 cp . s3://bucket-name --recursive
    ```

    ## Copy all matching files between S3 buckets

    ```bash
    aws s3 cp s3://source-bucket s3://target-bucket --recursive --exclude "*" --include "*onlye-this-pattern*"
    ```

    ## Retrieve log entries for a given time period (up to a maximum response size of 1MB)

    ```bash
    aws logs filter-log-events --log-group-name PUT_LOG_GROUP_NAME_HERE --interleaved --start-time 1440616500000 --end-time 1440619200000
    ```

    ## JavaScript to Pull and Dump AWS Logs To A Local System

    ```javascript
    // Load the Amazon AWS API module
    var AWS = require('aws-sdk');

    // Set the default AWS region
    AWS.config.update({region: 'us-east-1'});

    // Get an instance of CloudWatchLogs
    var cwlogs = new AWS.CloudWatchLogs();

    // Define the parameters for the initial request
    var params = {
    "logGroupName": "PUT_LOG_GROUP_NAME_HERE",
    "startTime": 1440696300000,
    "endTime": 1440697800000,
    "limit": 500,
    "interleaved": true
    }

    // Declare function that repeatedly calls itself until all of the log data has been read
    var fetchLogs = function(params) {

    // Fetch a page of log data
    cwlogs.filterLogEvents(params, function(err, data) {
    if (err) {
    // An error occurred
    console.log(err, err.stack);
    } else {
    // Successful response: dump out the AWS log events as single line tab separeted data set
    data.events.forEach(function(event) {

    // Log the event minus any trailing carriage returns on the message
    console.log(event.ingestionTime + "\t" + event.message.replace(/[\r\n]+$/, ""));
    });

    // If there are more AWS log entries to process, recurse
    if (typeof data.nextToken !== 'undefined') {
    params.nextToken = data.nextToken;
    fetchLogs(params);
    }
    }
    });
    }

    // Kick of the recursive log fetch
    fetchLogs(params);
    ```

    ## List Metrics In Namespace

    ```bash
    aws cloudwatch list-metrics --namespace "PUT_NAMESPACE_NAME_HERE"
    ```
  6. mikebway created this gist Dec 14, 2019.
    24 changes: 24 additions & 0 deletions AWS-CLI-Crib-Sheet.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,24 @@
    # AWS CLI Crib Sheet

    ## Check the AWS CLI Version

    The latest available version can be found here: <https://github.com/aws/aws-cli/releases>

    Check the installed version thus:

    ```bash
    aws --version
    ```

    ## Upgrade to Latest CLI Version

    Using the AWS reccomended "Bundled Installer" (see [Installing the AWS CLI version 1](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv1.html)):

    1. Download and unpack the ZIP file from <https://s3.amazonaws.com/aws-cli/awscli-bundle.zip>

    1. Run the installer

    ```bash
    sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
    ```