Skip to content

Instantly share code, notes, and snippets.

@ethangardner

ethangardner/awk-commands.md

Last active June 28, 2023 19:04
Show Gist options
  • Save ethangardner/aba699e9fc1b948db975e3c9894c360f to your computer and use it in GitHub Desktop.
Save ethangardner/aba699e9fc1b948db975e3c9894c360f to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. ethangardner revised this gist Jan 30, 2019. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions awk-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -14,6 +14,9 @@ means that columns are delimited by spaces or colons.
    ### Find referrers matching a pattern
    `cat /var/log/httpd/access_log | awk -F'"' '$4~/(menshealth\.com|fitnessmagazine\.com|seriouseats\.com|giants\.com|soaphub\.com|tmz\.com|bleacherreport\.com)/ {print $1,$4}' | more`

    ### Find requests matching a pattern
    `cat /var/log/httpd/access_log | awk -F'"' '$2~/\/search\?q=/ {print}' | more`

    ## More links
    - https://coderwall.com/p/_zkogw/awk-to-explore-your-access-log-files
    - https://coderwall.com/p/ueazhw/parsing-logs-with-awk
  2. ethangardner revised this gist Jan 24, 2019. No changes.
  3. ethangardner revised this gist Jan 24, 2019. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions awk-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -11,6 +11,9 @@ means that columns are delimited by spaces or colons.
    ### Print entire log entry for requests between the hours of 18 and 19 and status code is 301
    `cat /var/log/httpd/access_log | awk -F'[: ]' '$6 >= 18 && $6 <= 19 && $13 == 301 { print }' | more`

    ### Find referrers matching a pattern
    `cat /var/log/httpd/access_log | awk -F'"' '$4~/(menshealth\.com|fitnessmagazine\.com|seriouseats\.com|giants\.com|soaphub\.com|tmz\.com|bleacherreport\.com)/ {print $1,$4}' | more`

    ## More links
    - https://coderwall.com/p/_zkogw/awk-to-explore-your-access-log-files
    - https://coderwall.com/p/ueazhw/parsing-logs-with-awk
  4. ethangardner revised this gist Nov 19, 2018. 1 changed file with 4 additions and 0 deletions.
    4 changes: 4 additions & 0 deletions awk-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -10,3 +10,7 @@ means that columns are delimited by spaces or colons.

    ### Print entire log entry for requests between the hours of 18 and 19 and status code is 301
    `cat /var/log/httpd/access_log | awk -F'[: ]' '$6 >= 18 && $6 <= 19 && $13 == 301 { print }' | more`

    ## More links
    - https://coderwall.com/p/_zkogw/awk-to-explore-your-access-log-files
    - https://coderwall.com/p/ueazhw/parsing-logs-with-awk
  5. ethangardner created this gist Nov 19, 2018.
    12 changes: 12 additions & 0 deletions awk-commands.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,12 @@
    $1 is the first column. awk separates columns by spaces by default. delimiters can be specified with `-F`. In the examples below `'[: ]'`
    means that columns are delimited by spaces or colons.

    ```
    99.56.8.181 10.0.1.239 - - [16/Nov/2018:20:45:59 +0000] "GET /app/themes/finecooking/dist/img/marketing-hero-cover.jpg HTTP/1.0" 200 38808 "https://www.finecooking.com/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:63.0) Gecko/20100101 Firefox/63.0"
    ```

    ### Print IP and user agent for requests between the hours of 18 and 19
    `cat /var/log/httpd/access_log | awk -F'[: ]' '$6 >= 18 && $6 <= 19 { print }' | awk -F\" '{print $1,$6}' | more`

    ### Print entire log entry for requests between the hours of 18 and 19 and status code is 301
    `cat /var/log/httpd/access_log | awk -F'[: ]' '$6 >= 18 && $6 <= 19 && $13 == 301 { print }' | more`