Skip to content

Instantly share code, notes, and snippets.

@infinityhacks

infinityhacks/python_athena.py

Forked from ankit-crossml/python_athena.py
Created June 1, 2021 07:20
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save infinityhacks/e87f661c1f1246aa26bce01e36f51cd7 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save infinityhacks/e87f661c1f1246aa26bce01e36f51cd7 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @ankit-crossml ankit-crossml created this gist Apr 14, 2020.
    32 changes: 32 additions & 0 deletions python_athena.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,32 @@
    import boto3
    import time
    import re

    client = boto3.client('athena')

    # run athen query
    response = client.start_query_execution(
    QueryString=athena_query,
    QueryExecutionContext={'Database': ATHENA_DATABASE_NAME},
    ResultConfiguration={
    'OutputLocation': ATHENA_RESULT_S3_BUCKET
    }
    )

    # wait for query results for max 20 seconds
    execution_id = response['QueryExecutionId']
    state = 'RUNNING'
    max_wait = 20
    while (max_wait > 0 and state in ['RUNNING', 'QUEUED']):
    max_wait = max_wait - 1
    response = client.get_query_execution(
    QueryExecutionId=execution_id)
    if 'QueryExecution' in response and \
    'Status' in response['QueryExecution'] and \
    'State' in response['QueryExecution']['Status']:
    state = response['QueryExecution']['Status']['State']
    if state == 'SUCCEEDED':
    s3_path = response['QueryExecution']['ResultConfiguration']['OutputLocation']
    filename = re.findall('.*\/(.*)', s3_path)[0]
    return filename
    time.sleep(1)