Skip to content

Instantly share code, notes, and snippets.

@haandol

haandol/how-to-use-aws-elasticsearch.md

Forked from simonw/how-to-use-aws-elasticsearch.md
Created April 15, 2021 07:47
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save haandol/3dc1dae433afa6ed0225fa2f58b6486c to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save haandol/3dc1dae433afa6ed0225fa2f58b6486c to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. Simon Willison revised this gist Mar 16, 2016. 1 changed file with 32 additions and 0 deletions.
    32 changes: 32 additions & 0 deletions how-to-use-aws-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,4 +6,36 @@ The bad news: it's stuck on Elasticsearch 1.5.2 and dynamic scripting (Groovy) i

    http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html

    Authentication: the safest option is to create a brand new IAM user (using the tool at https://console.aws.amazon.com/iam/home?region=us-east-1 ) with its own access key and secret key. Then when you create the Elasticsearch instance you can paste in the following IAM string:

    arn:aws:iam::YOUR_AWS_ACCOUNT_ID:user/YOUR_IAM_USERNAME

    You'll need to look up YOUR_AWS_ACCOUNT_ID - http://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html

    Having done all of the above, here's the secret recipe to getting Python to talk to your new Elasticsearch instance:

    import requests
    from requests_aws4auth import AWS4Auth

    endpoint = 'https://search-your-endpoint.us-east-1.es.amazonaws.com'
    auth=AWS4Auth(ACCESS_ID, ACCESS_SECRET, 'us-east-1', 'es')

    print requests.get(endpoint, auth=auth).json()
    print requests.get(endpoint + '/_aliases', auth=auth).json()
    # etc

    If you're using the official Python client for Elasticsearch, the recipe looks like this:

    from elasticsearch import Elasticsearch, RequestsHttpConnection
    from requests_aws4auth import AWS4Auth

    es = Elasticsearch(
    'default',
    hosts=['search-your-endpoint.us-east-1.es.amazonaws.com'],
    http_auth=AWS4Auth(ACCESS_ID, ACCESS_SECRET, 'us-east-1', 'es'),
    use_ssl=True,
    verify_certs=True,
    connection_class=RequestsHttpConnection
    )

    See also https://github.com/elastic/elasticsearch-py/issues/280
  2. Simon Willison created this gist Mar 16, 2016.
    9 changes: 9 additions & 0 deletions how-to-use-aws-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,9 @@
    # How to use Amazon AWS Elasticsearch

    The good news: you can get it running on the free tier (with a tiny instance).

    The bad news: it's stuck on Elasticsearch 1.5.2 and dynamic scripting (Groovy) is disabled.

    http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html