Skip to content

Instantly share code, notes, and snippets.

@rjhowe

rjhowe/cheatsheet-elasticsearch.md

Forked from ruanbekker/cheatsheet-elasticsearch.md
Last active January 10, 2020 16:02
Show Gist options
  • Save rjhowe/0fd914d3585e4562e0a9c699f4f0f52d to your computer and use it in GitHub Desktop.
Save rjhowe/0fd914d3585e4562e0a9c699f4f0f52d to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. rjhowe revised this gist Jan 10, 2020. 1 changed file with 76 additions and 76 deletions.
    152 changes: 76 additions & 76 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -31,7 +31,7 @@ My Elasticsearch cheatsheet with example usage via rest api (still a work-in-pro
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/docs.html
    - https://www.elastic.co/blog/managing-time-based-indices-efficiently
    - http://joelabrahamsson.com/elasticsearch-101/
    - https://joelabrahamsson.com/elasticsearch-101/
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html
    - https://chatbots.network/logstash-exclude-bots-from-result/

    @@ -43,7 +43,7 @@ Resource:
    - https://www.elastic.co/guide/en/elasticsearch/guide/current/_cluster_health.html

    ```
    $ curl -XGET http://elasticsearch:9200/_cluster/health?pretty
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cluster/health?pretty
    {
    "cluster_name" : "docker-cluster",
    "status" : "green",
    @@ -66,7 +66,7 @@ $ curl -XGET http://elasticsearch:9200/_cluster/health?pretty
    ## Cluster Health: Index Level:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_cluster/health?level=indices&pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_cluster/health?level=indices&pretty'
    {
    "cluster_name" : "swarm-elasticsearch",
    "status" : "red",
    @@ -101,7 +101,7 @@ $ curl -XGET 'http://elasticsearch:9200/_cluster/health?level=indices&pretty'
    ## Cluster Health: Shard Level:

    ```
    curl -XGET 'http://elasticsearch:9200/_cluster/health?level=shards&pretty'
    curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_cluster/health?level=shards&pretty'
    {
    "cluster_name" : "swarm-elasticsearch",
    "status" : "red",
    @@ -178,7 +178,7 @@ curl -XGET 'http://elasticsearch:9200/_cluster/health?level=shards&pretty'
    ## Nodes Overview:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/nodes?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/nodes?v
    ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
    10.0.2.28 21 92 2 0.55 0.45 0.38 mdi - ea1q921
    10.0.2.24 27 95 5 0.17 0.24 0.22 mdi - rNDYCtL
    @@ -190,7 +190,7 @@ ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master
    ## Who is Master:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/master?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/master?v
    id host ip node
    mstWlaoyTM69xhSt-_rZAA 10.0.2.18 10.0.2.18 mstWlao
    ```
    @@ -200,23 +200,23 @@ mstWlaoyTM69xhSt-_rZAA 10.0.2.18 10.0.2.18 mstWlao
    View all your indices in your cluster:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/indices?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/indices?v
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open ruan-test CrQZB2L4SaaYCkvYPx5vUA 5 1 38 0 131.9kb 78.6kb
    ```

    View one index:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/_cat/indices/index-name-2018.01.01?v'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/_cat/indices/index-name-2018.01.01?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open index-name-2018.01.01 Nk8SMQvRSIaNm854bc3Zjg 5 1 395552 0 755.6mb 377.8mb
    ```

    View a range of indices:

    ```
    $ curl -XGET 'https://http://127.0.0.1:9200/_cat/indices/index-name-2018.01*?v'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://https://127.0.0.1:9200/_cat/indices/index-name-2018.01*?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open index-name-2018.01.19 Vp1EBoeMQkS-a_upLzedhQ 5 1 1220 0 2.6mb 1.3mb
    green open index-name-2018.01.17 hSJMzFJIQrePifCfgb1rOA 5 1 2875 0 3.8mb 1.9mb
    @@ -225,7 +225,7 @@ green open index-name-2018.01.17 hSJMzFJIQrePifCfgb1rOA 5 1 2875
    View only the index name header:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/_cat/indices/*2018.03.*?v&h=index'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/_cat/indices/*2018.03.*?v&h=index'
    index
    index-name-2018.03.01
    index-name-2018.03.02
    @@ -234,15 +234,15 @@ index-name-2018.03.02
    ## How Many Documents in the ES Cluster (Across all Indices):

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/count?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/count?v
    epoch timestamp count
    1502288579 14:22:59 38
    ```

    ## Shards Info per Index:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/shards/ruan-test?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/shards/ruan-test?v
    index shard prirep state docs store ip node
    ruan-test 3 r STARTED 10 6.9kb 10.0.2.28 ea1q921
    ruan-test 3 p STARTED 10 6.9kb 10.0.2.24 rNDYCtL
    @@ -259,7 +259,7 @@ ruan-test 0 r STARTED 4 12.9kb 10.0.2.27 bDWFHuw
    ## Shard Allocation per Node:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/allocation?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/allocation?v
    shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
    4 60.6mb 15.7gb 29.9gb 45.7gb 34 10.0.2.24 10.0.2.24 rNDYCtL
    4 48.3kb 16.7gb 28.9gb 45.7gb 36 10.0.2.18 10.0.2.18 mstWlao
    @@ -275,7 +275,7 @@ shards disk.indices disk.used disk.avail disk.total disk.percent host ip
    This will move shards from the mentioned node

    ```
    $ curl -XPUT 'localhost:9200/_cluster/settings?pretty' -d'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT 'localhost:9200/_cluster/settings?pretty' -d'
    {
    "transient" : {
    "cluster.routing.allocation.exclude._ip" : "10.0.0.1"
    @@ -292,7 +292,7 @@ $ curl -XPUT 'localhost:9200/_cluster/settings?pretty' -d'
    At the moment one of the nodes were down, and up again:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cat/allocation?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://127.0.0.1:9200/_cat/allocation?v
    shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
    290 54.1mb 1gb 20mb 1gb 98 10.79.2.193 10.79.2.193 es01
    151 43.5mb 1gb 11.9gb 13gb 8 10.79.3.171 10.79.3.171 es02
    @@ -302,7 +302,7 @@ shards disk.indices disk.used disk.avail disk.total disk.percent host ip
    ## Recovery API:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cat/recovery?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://127.0.0.1:9200/_cat/recovery?v
    index shard time type stage source_host target_host repository snapshot files files_percent bytes bytes_percent total_files total_bytes translog translog_percent total_translog
    sysadmins-2017.06.19 0 1512 replica done 10.79.2.193 10.79.3.171 n/a n/a 31 100.0% 340020 100.0% 31 340020 0 100.0% 0
    sysadmins-2017.06.19 0 7739 store done 10.79.2.193 10.79.2.193 n/a n/a 0 100.0% 0 100.0% 31 340020 0 100.0% 0
    @@ -314,7 +314,7 @@ sysadmins-2017.06.19 1 613 replica done 10.79.3.171 10.79.2.193
    ## Pending Tasks:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cat/pending_tasks?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://127.0.0.1:9200/_cat/pending_tasks?v
    insertOrder timeInQueue priority source
    1736 1.8s URGENT shard-started ([sysadmins-2017.06.02][2], node[WR3y31g1TnuufpNyrJnQtg], [R], v[91], s[INITIALIZING], a[id=wVTDn4nFSKKxvi07cU0uCg], unassigned_info[[reason=CLUSTER_RECOVERED], at[2017-08-11T07:50:56.550Z]]), reason [after recovery (replica) from node [{es01}{6ND8sZ_rTqaL42VdlxyW7Q}{10.79.2.193}{10.79.2.193:9300}]]
    1737 1.3s URGENT shard-started ([sysadmins-2017.06.02][3], node[WR3y31g1TnuufpNyrJnQtg], [R], v[91], s[INITIALIZING], a[id=JmrtwtYURMyQF6LspeJXLg], unassigned_info[[reason=CLUSTER_RECOVERED], at[2017-08-11T07:50:56.550Z]]), reason [after recovery (replica) from node [{es01}{6ND8sZ_rTqaL42VdlxyW7Q}{10.79.2.193}{10.79.2.193:9300}]]
    @@ -323,7 +323,7 @@ insertOrder timeInQueue priority source
    ## Clear Cache:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cache/clear
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://127.0.0.1:9200/_cache/clear
    {"_shards":{"total":21,"successful":15,"failed":0}}
    ```

    @@ -351,14 +351,14 @@ PUT /_cluster/settings
    When you create an Index, 5 Primary Shards and 1 Replica Shard will assigned to the Index by Default.

    ```
    $ curl -XPUT http://elasticsearch:9200/my2ndindex
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/my2ndindex
    {"acknowledged":true,"shards_acknowledged":true}
    ```

    To verify the behavior:

    ```
    curl -XGET -u http://elasticsearch:9200/_cat/indices?v
    curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET -u https://localhost:9200/_cat/indices?v
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open my2ndindex V32G9IOoTF6uq0DuNUIAMA 5 1 0 0 1.2kb 650b
    green open ruan-test CrQZB2L4SaaYCkvYPx5vUA 5 1 38 0 131.9kb 78.6kb
    @@ -372,7 +372,7 @@ You can ONLY set the number primary shards on index creation.
    While having 5 prmary shards and 1 replica shard, let's have a look at it:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/shards/my2ndindex?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/shards/my2ndindex?v
    index shard prirep state docs store ip node
    my2ndindex 3 p STARTED 0 130b 10.0.2.22 ifgr6ym
    my2ndindex 3 r STARTED 0 130b 10.0.2.27 bDWFHuw
    @@ -391,14 +391,14 @@ my2ndindex 0 r STARTED 0 130b 10.0.2.24 rNDYCtL
    Let's change the replica shard number to 2, meaning each primary shard will have 2 replica shards:

    ```
    $ curl -XPUT http://elasticsearch:9200/my2ndindex/_settings -d '{"settings": {"index": {"number_of_replicas": 2}}}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/my2ndindex/_settings -d '{"settings": {"index": {"number_of_replicas": 2}}}'
    {"acknowledged":true}
    ```

    Let's have a look at the shard info after we have increased the replica shard number:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/shards/my2ndindex?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/shards/my2ndindex?v
    index shard prirep state docs store ip node
    my2ndindex 3 r STARTED 0 130b 10.0.2.28 ea1q921
    my2ndindex 3 p STARTED 0 130b 10.0.2.22 ifgr6ym
    @@ -422,13 +422,13 @@ my2ndindex 0 r STARTED 0 130b 10.0.2.24 rNDYCtL
    Create a Index with Default Settings:

    ```
    $ curl -XPUT -H 'Content-Type: application/json' 'http://127.0.0.1:9200/ruan-test-2018.03.12'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT -H 'Content-Type: application/json' 'https://127.0.0.1:9200/ruan-test-2018.03.12'
    ```

    View the settings of the created index:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.12/_settings?pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/ruan-test-2018.03.12/_settings?pretty'
    {
    "ruan-test-2018.03.12" : {
    "settings" : {
    @@ -450,13 +450,13 @@ $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.12/_settings?pretty'
    Remember that primary shard number can only be set on index creation. Change the settings of the index, let's update the index to: 2 replica shards, and the total_fields limit to: 2000

    ```
    $ curl -XPUT -H 'Content-Type: application/json' 'http://127.0.0.1:9200/ruan-test-2018.03.12/_settings' -d '{"number_of_replicas": 0, "index.mapping.total_fields.limit": 2000}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT -H 'Content-Type: application/json' 'https://127.0.0.1:9200/ruan-test-2018.03.12/_settings' -d '{"number_of_replicas": 0, "index.mapping.total_fields.limit": 2000}'
    ```

    View the changes:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.12/_settings?pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/ruan-test-2018.03.12/_settings?pretty'
    {
    "ruan-test-2018.03.12" : {
    "settings" : {
    @@ -484,13 +484,13 @@ Now, to set the settings on Index Creation:


    ```
    $ curl -XPUT -H 'Content-Type: application/json' 'http://127.0.0.1:9200/ruan-test-2018.03.13' -d '{"settings": {"number_of_replicas": 1, "number_of_shards": 2, "index.mapping.total_fields.limit": 2000}}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT -H 'Content-Type: application/json' 'https://127.0.0.1:9200/ruan-test-2018.03.13' -d '{"settings": {"number_of_replicas": 1, "number_of_shards": 2, "index.mapping.total_fields.limit": 2000}}'
    ```

    Verifying our settings:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.13/_settings?pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/ruan-test-2018.03.13/_settings?pretty'
    {
    "ruan-test-2018.03.13" : {
    "settings" : {
    @@ -517,7 +517,7 @@ $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.13/_settings?pretty'
    Viewing our indexes:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/_cat/indices/ruan-test-*?v'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/_cat/indices/ruan-test-*?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open ruan-test-2018.03.12 EwGz6y7XQkK0ZI08u8qdrQ 5 1 2 0 15.7kb 7.8kb
    green open ruan-test-2018.03.13 hEY8HrlRTFuiYLwKVDAraQ 2 1 0 0 932b 466b
    @@ -528,22 +528,22 @@ green open ruan-test-2018.03.13 hEY8HrlRTFuiYLwKVDAraQ 2 1 0
    Let's ingest one docuemnt into Elasticsearch, and in this case we will specify the document id as `1`

    ```
    $ curl -XPUT http://elasticsearch:9200/my2ndindex/docs/1 -d '{"identity": {"name": "ruan", "surname": "bekker"}}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/my2ndindex/docs/1 -d '{"identity": {"name": "ruan", "surname": "bekker"}}'
    {"_index":"my2ndindex","_type":"docs","_id":"1","_version":1,"result":"created","_shards":{"total":3,"successful":3,"failed":0},"created":true}
    ```

    View the index info:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_cat/indices/my*?v'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_cat/indices/my*?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open my2ndindex V32G9IOoTF6uq0DuNUIAMA 5 2 1 0 13kb 4.3kb
    ```

    View the Shard information on our Index:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/shards/my2ndindex?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/shards/my2ndindex?v
    index shard prirep state docs store ip node
    my2ndindex 3 r STARTED 1 3.9kb 10.0.2.28 ea1q921
    my2ndindex 3 p STARTED 1 3.9kb 10.0.2.22 ifgr6ym
    @@ -591,22 +591,22 @@ To get the status back to green:
    Increase the replica shards to `5`:

    ```
    $ curl -XPUT http://elasticsearch:9200/my2ndindex/_settings -d '{"settings": {"number_of_replicas": 5}}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/my2ndindex/_settings -d '{"settings": {"number_of_replicas": 5}}'
    {"acknowledged":true}
    ```

    Verify the Indices Overview:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_cat/indices/my*?v'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_cat/indices/my*?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    yellow open my2ndindex V32G9IOoTF6uq0DuNUIAMA 5 5 1 0 22.2kb 4.4kb
    ```

    We can see that we have a YELLOW status, for more info let's have a look at the shards overview:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/shards/my2ndindex?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/shards/my2ndindex?v
    index shard prirep state docs store ip node
    my2ndindex 3 r STARTED 1 3.9kb 10.0.2.28 ea1q921
    my2ndindex 3 p STARTED 1 3.9kb 10.0.2.22 ifgr6ym
    @@ -643,7 +643,7 @@ my2ndindex 0 r UNASSIGNED
    Also, when we look at the allocation api, we can see that we have 5 shards that is unassigned:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/allocation?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/allocation?v
    shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
    9 59.2kb 16.8gb 28.8gb 45.7gb 36 10.0.2.18 10.0.2.18 mstWlao
    10 61.2mb 16.8gb 28.8gb 45.7gb 36 10.0.2.27 10.0.2.27 bDWFHuw
    @@ -658,14 +658,14 @@ shards disk.indices disk.used disk.avail disk.total disk.percent host ip
    Let's create an index with 10 primary shards and a replica count of 2:

    ```
    $ curl -XPUT http://elasticsearch:9200/my3rdindex -d '{"settings": {"index": {"number_of_shards": 10, "number_of_replicas": 2}}}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/my3rdindex -d '{"settings": {"index": {"number_of_shards": 10, "number_of_replicas": 2}}}'
    {"acknowledged":true,"shards_acknowledged":true}/ #
    ```

    Verify:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_cat/indices/my*?v'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_cat/indices/my*?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open my3rdindex ljovpse0RzCB5INxUBLBYg 10 2 0 0 2.4kb 650b
    green open my2ndindex V32G9IOoTF6uq0DuNUIAMA 5 2 1 0 13.3kb 4.4kb
    @@ -674,7 +674,7 @@ green open my2ndindex V32G9IOoTF6uq0DuNUIAMA 5 2 1 0
    View the shard info on our index:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/shards/my3rdindex?v
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/_cat/shards/my3rdindex?v
    index shard prirep state docs store ip node
    my3rdindex 8 r STARTED 0 130b 10.0.2.28 ea1q921
    my3rdindex 8 p STARTED 0 130b 10.0.2.22 ifgr6ym
    @@ -711,14 +711,14 @@ my3rdindex 0 r STARTED 0 130b 10.0.2.24 rNDYCtL
    Take note, with the configuration as above your index that you created will have 30 shards in your cluster:

    ```
    $ curl -s -XGET 'http://elasticsearch:9200/_cat/shards/my3rdindex?v' | grep -v 'node' | wc -l
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -s -XGET 'https://localhost:9200/_cat/shards/my3rdindex?v' | grep -v 'node' | wc -l
    30
    ```

    Number of Primary Shards per Node:

    ```
    $ curl -s -XGET 'http://elasticsearch:9200/_cat/shards/my3rdindex?v' | grep 'p STARTED' | awk '{print $7}' | sort | uniq -c
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -s -XGET 'https://localhost:9200/_cat/shards/my3rdindex?v' | grep 'p STARTED' | awk '{print $7}' | sort | uniq -c
    2 10.0.2.18
    3 10.0.2.22
    1 10.0.2.24
    @@ -746,15 +746,15 @@ When you do a `PUT` request, you need to specify the `id` of the document:
    Let's ingest a simple document with a random string as the document id:

    ```
    $ curl -XPUT http://elasticsearch:9200/people/users/abcd -d '{"name", "james", "age": 28}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/people/users/abcd -d '{"name", "james", "age": 28}'
    {"_index":"people","_type":"users","_id":"abcd","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"created":true}
    ```

    If we have to repeat the same request with the same `id`, the docuement will be overwritten, ES will create a new document if
    the `id` is not present.

    ```
    $ curl -XPUT http://elasticsearch:9200/people/users/abcd -d '{"name": "james", "age": 28}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/people/users/abcd -d '{"name": "james", "age": 28}'
    {"_index":"people","_type":"users","_id":"abcd","_version":2,"result":"updated","_shards":{"total":2,"successful":2,"failed":0},"created":false}
    ```

    @@ -763,7 +763,7 @@ $ curl -XPUT http://elasticsearch:9200/people/users/abcd -d '{"name": "james", "
    When you do a `POST` request, the service will automatically assign a `id` for your docuemt:

    ```
    $ curl -XPOST http://elasticsearch:9200/people/users/ -d '{"name": "susan", "age: 30}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST https://localhost:9200/people/users/ -d '{"name": "susan", "age: 30}'
    {"_index":"people","_type":"users","_id":"AV3H_9q6AH1phg1wCfDW","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"created":true}
    ```

    @@ -781,7 +781,7 @@ Our Sample Data: `info.json`:
    Ingest using the Bulk Api:

    ```
    curl -XPOST 'http://elasticsearch:9200/info/_bulk?pretty' --data-binary @info.json
    curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST 'https://localhost:9200/info/_bulk?pretty' --data-binary @info.json
    ```

    # Mapping
    @@ -794,7 +794,7 @@ curl -XPOST 'http://elasticsearch:9200/info/_bulk?pretty' --data-binary @info.js
    Check if a field exisists in your mapping:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/index-name-2018.03.01/_mapping/docs/field/company?pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/index-name-2018.03.01/_mapping/docs/field/company?pretty'
    {
    "index-name-2018.03.01" : {
    "mappings" : {
    @@ -825,21 +825,21 @@ $ curl -XGET 'http://127.0.0.1:9200/index-name-2018.03.01/_mapping/docs/field/co
    ## Close Index:

    ```
    $ curl -XPOST http://elasticsearch:9200/people/_close
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST https://localhost:9200/people/_close
    {"acknowledged":true}
    ```

    Trying to ingest while the index is closed:

    ```
    $ curl -XPOST http://elasticsearch:9200/people/users/ -d '{"name": "susan", "age": 30}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST https://localhost:9200/people/users/ -d '{"name": "susan", "age": 30}'
    {"error":{"root_cause":[{"type":"index_closed_exception","reason":"closed","index_uuid":"Yt31-EAwTOa-a6duElYRsQ","index":"people"}],"type":"index_closed_exception","reason":"closed","index_uuid":"Yt31-EAwTOa-a6duElYRsQ","index":"people"},"status":403}
    ```

    ## Open Index:

    ```
    $ curl -XPOST http://elasticsearch:9200/people/_open
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST https://localhost:9200/people/_open
    ```

    # Searching
    @@ -849,7 +849,7 @@ $ curl -XPOST http://elasticsearch:9200/people/_open
    We can get the document by passing the document `id`:

    ```
    $ curl -XGET http://elasticsearch:9200/people/users/abcd?pretty
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/people/users/abcd?pretty
    {
    "_index" : "people",
    "_type" : "users",
    @@ -866,7 +866,7 @@ $ curl -XGET http://elasticsearch:9200/people/users/abcd?pretty
    ## Determine which Shard a Document Reside on:

    ```
    $ curl -XGET 'http://elasticsearch:9200/people/users/_search?q=age:28&explain&pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/people/users/_search?q=age:28&explain&pretty'
    {
    "took" : 73,
    "timed_out" : false,
    @@ -919,7 +919,7 @@ $ curl -XGET 'http://elasticsearch:9200/people/users/_search?q=age:28&explain&pr
    Lets do a search on our index:

    ```
    $ curl -XGET http://elasticsearch:9200/people/_search?pretty
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://localhost:9200/people/_search?pretty
    {
    "took" : 29,
    "timed_out" : false,
    @@ -960,7 +960,7 @@ $ curl -XGET http://elasticsearch:9200/people/_search?pretty
    By default the Search API returns 10 items, which can be changed using `size`

    ```
    curl -XGET 'http://elasticsearch:9200/shakespeare/_search?size=3&pretty'
    curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/shakespeare/_search?size=3&pretty'
    {
    "took" : 25,
    "timed_out" : false,
    @@ -1025,7 +1025,7 @@ curl -XGET 'http://elasticsearch:9200/shakespeare/_search?size=3&pretty'
    Query our index for people with the age of 28:

    ```
    curl -XGET 'http://elasticsearch:9200/people/_search?q=age:30&pretty'
    curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/people/_search?q=age:30&pretty'
    {
    "took" : 25,
    "timed_out" : false,
    @@ -1056,7 +1056,7 @@ curl -XGET 'http://elasticsearch:9200/people/_search?q=age:30&pretty'
    #### Query by Term and limit results by 2:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    {
    "query": {
    "term": {
    @@ -1071,7 +1071,7 @@ $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    #### Query by Match:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET https://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    {
    "query": {
    "match": {
    @@ -1088,7 +1088,7 @@ $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    - Check if field exists in index:

    ```
    $ curl http://127.0.0.1:9200/test4/_search?pretty -d '
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key https://127.0.0.1:9200/test4/_search?pretty -d '
    {
    "query": {
    "bool": {
    @@ -1249,16 +1249,16 @@ Sort Per Field:
    Ingest a couple of example documents:

    ```
    $ curl -XPUT http://elasticsearch:9200/products/items/1 -d '{"product": "chocolate", "price": [20, 4]}'
    $ curl -XPUT http://elasticsearch:9200/products/items/2 -d '{"product": "apples", "price": [28, 6]}'
    $ curl -XPUT http://elasticsearch:9200/products/items/3 -d '{"product": "bananas", "price": [28, 22, 23, 20]}'
    $ curl -XPUT http://elasticsearch:9200/products/items/4 -d '{"product": "chips", "price": [14, 24, 22, 12]}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/products/items/1 -d '{"product": "chocolate", "price": [20, 4]}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/products/items/2 -d '{"product": "apples", "price": [28, 6]}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/products/items/3 -d '{"product": "bananas", "price": [28, 22, 23, 20]}'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT https://localhost:9200/products/items/4 -d '{"product": "chips", "price": [14, 24, 22, 12]}'
    ```

    Run a Sort Query on the term `bananas`, and show the `average` price. We can also use `min, max, avg, sum`:

    ```
    $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST https://localhost:9200/products/_search?pretty -d '
    {
    "query" : {
    "term" : {
    @@ -1311,7 +1311,7 @@ $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '
    Running the same, but wanting to see the sum of all the prices:

    ```
    $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST https://localhost:9200/products/_search?pretty -d '
    {
    "query" : {
    "term" : {
    @@ -1371,15 +1371,15 @@ References:
    ## Delete Index:

    ```
    $ curl -XDELETE http://elasticsearch:9200/myindex
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XDELETE https://localhost:9200/myindex
    ```

    ## Delete Documents on Query:

    We would like to delete all documents that has `"os_name": "Windows 10"`

    ```
    curl -XPOST 'http://elasticsearch:9200/weblogs/_delete_by_query?pretty' -d '
    curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST 'https://localhost:9200/weblogs/_delete_by_query?pretty' -d '
    {
    "query": {
    "match": {
    @@ -1410,7 +1410,7 @@ curl -XPOST 'http://elasticsearch:9200/weblogs/_delete_by_query?pretty' -d '
    If routing is provided, then the routing is copied to the scroll query, limiting the process to the shards that match that routing value:

    ```
    $ curl -XPOST 'http://elasticsearch:9200/people/_delete_by_query?routing=1
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST 'https://localhost:9200/people/_delete_by_query?routing=1
    {
    "query": {
    "range" : {
    @@ -1425,7 +1425,7 @@ $ curl -XPOST 'http://elasticsearch:9200/people/_delete_by_query?routing=1
    By default _delete_by_query uses scroll batches of 1000. You can change the batch size with the scroll_size URL parameter:

    ```
    $ curl -XPOST 'http://elasticsearch:9200/weblogs/_delete_by_query?scroll_size=5000
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST 'https://localhost:9200/weblogs/_delete_by_query?scroll_size=5000
    {
    "query": {
    "term": {
    @@ -1438,7 +1438,7 @@ $ curl -XPOST 'http://elasticsearch:9200/weblogs/_delete_by_query?scroll_size=50
    ## Delete Stats:

    ```
    $ curl -XGET 'elasticsearch:9200/_tasks?detailed=true&actions=*/delete/byquery&pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'localhost:9200/_tasks?detailed=true&actions=*/delete/byquery&pretty'
    {
    "nodes" : {
    "s5A2CoRWrwKf512z6NEscF" : {
    @@ -1484,7 +1484,7 @@ Setup the [S3 Snapshot Repository](https://sysadmins.co.za/aws-elasticsearch-reg
    List the Snapshot Repositories:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/_cat/repositories?v'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://127.0.0.1:9200/_cat/repositories?v'
    id type
    foo-bacups s3
    bar-backups s3
    @@ -1493,7 +1493,7 @@ bar-backups s3
    View the Snapshot Repository:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/bar-backups?pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_snapshot/bar-backups?pretty'
    {
    "bar-backups" : {
    "type" : "s3",
    @@ -1511,8 +1511,8 @@ $ curl -XGET 'http://elasticsearch:9200/_snapshot/bar-backups?pretty'
    Create a Snapshot named `mysnapshot_ruan-test-2018-05-24_1` of the index: `ruan-test-2018-05-24` and return the exit when the snapshot is done:

    ```
    $ curl -XPUT -H 'Content-Type: application/json' \
    'http://elasticsearch:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1?wait_for_completion=true&pretty=true' -d '
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPUT -H 'Content-Type: application/json' \
    'https://localhost:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1?wait_for_completion=true&pretty=true' -d '
    {
    "indices": "ruan-test-2018-05-24",
    "ignore_unavailable": true,
    @@ -1546,7 +1546,7 @@ $ curl -XPUT -H 'Content-Type: application/json' \
    Verify the Snapshot:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_cat/snapshots/bar-backups?v&s=id'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_cat/snapshots/bar-backups?v&s=id'
    id status start_epoch start_time end_epoch end_time duration indices successful_shards failed_shards total_shards
    mysnapshot_ruan-test-2018-05-24_1 SUCCESS 1527254411 06:20:11 1527254411 06:20:11 389ms 1 5 0 5
    ```
    @@ -1556,7 +1556,7 @@ mysnapshot_ruan-test-2018-05-24_1 SUCCESS 1527254411 06:20:11 1527254411
    Get the Metadata of the Snapshot:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1?pretty'
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XGET 'https://localhost:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1?pretty'
    {
    "snapshots" : [ {
    "snapshot" : "mysnapshot_ruan-test-2018-05-24_1",
    @@ -1591,7 +1591,7 @@ $ aws s3 --profile es ls s3://my-es-snapshot-bucket/ | grep VRTF2942QCeqyEaMxPgb
    Execute the Restore:

    ```
    $ curl -XPOST -H 'Content-Type: application/json' 'http://elasticsearch:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1/_restore -d '
    $ curl -sk curl --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key -XPOST -H 'Content-Type: application/json' 'https://localhost:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1/_restore -d '
    {
    "indices": "ruan-test-2018-05-24",
    "ignore_unavailable": true,
  2. @ruanbekker ruanbekker revised this gist Jul 13, 2019. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    (still a work-in-progress)
    My Elasticsearch cheatsheet with example usage via rest api (still a work-in-progress)

    # Shortlinks:

    @@ -18,6 +18,7 @@
    - [Query](#query)
    - [Query by Match](#query-by-match)
    - [Query with Bool](#query-with-bool)
    - [Other Examples with Query](#other-examples-of-query)
    - [Sort](#sort)
    - [Aggregate]()
    - [Delete](#delete)
  3. @ruanbekker ruanbekker revised this gist Jul 13, 2019. 1 changed file with 111 additions and 0 deletions.
    111 changes: 111 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1127,6 +1127,117 @@ $ curl http://127.0.0.1:9200/test4/_search?pretty -d '
    }
    ```

    #### Other Examples of Query:

    Match:

    ```
    {
    "query": {
    "match": {
    "title": "something"
    }
    }
    }
    ```

    Multi match with boost on title:

    ```
    # ^ boosts the score 4 times on title
    {
    "query": {
    "multi_match": {
    "query": "something",
    "fields": ["title^4", "plot"]
    }
    }
    }
    ```

    Match phrase:

    ```
    {
    "query": {
    "match_phrase": {
    "title": "somethings got to give"
    }
    }
    }
    ```

    Common terms:

    ```
    {
    "query": {
    "common": {
    "title": {
    "query": "the something word"
    }
    }
    }
    }
    ```

    Query string:

    ```
    {
    "query": {
    "query_string": {
    "query": "the something AND (gives OR gave)"
    }
    }
    }
    ```

    Simple query string:

    ```
    {
    "query": {
    "simple_query_string": {
    "query": "\"give got to\"~4 | *thing~2",
    "fields": ["title"]
    }
    }
    }
    ```

    More info on above:

    ```
    + -> Acts as the AND operator
    | -> Acts as the OR operator
    * -> Acts as a wildcard.
    "" -> Wraps several terms into a phrase.
    () -> Wraps a clause for precedence.
    ~n -> When used after a term (e.g. thign~3), sets fuzziness. When used after a phrase, sets slop. See Options.
    - -> Negates the term.
    ```

    Match all:

    ```
    {
    "query": {
    "match_all": {}
    }
    }
    ```

    Match none:

    ```
    {
    "query": {
    "match_none": {}
    }
    }
    ```

    - https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html

    ## Sort
  4. @ruanbekker ruanbekker revised this gist Jul 13, 2019. 1 changed file with 72 additions and 6 deletions.
    78 changes: 72 additions & 6 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1055,21 +1055,51 @@ curl -XGET 'http://elasticsearch:9200/people/_search?q=age:30&pretty'
    #### Query by Term and limit results by 2:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"term": {"title": "traefik"}}, "size": 2}'
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    {
    "query": {
    "term": {
    "title": "traefik"
    }
    },
    "size": 2
    }
    '
    ```

    #### Query by Match:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"match": {"title": "traefik"}}, "size": 10}'
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '
    {
    "query": {
    "match": {
    "title": "traefik"
    }
    },
    "size": 10
    }
    '
    ```

    #### Query with Bool:

    - Check if field exists in index:

    ```
    $ curl http://127.0.0.1:9200/test4/_search?pretty -d '{"query": {"bool": {"must": [{"exists": {"field": "name"}}]}}}}'
    $ curl http://127.0.0.1:9200/test4/_search?pretty -d '
    {
    "query": {
    "bool": {
    "must": [{
    "exists": {
    "field": "name"
    }
    }]
    }
    }
    }'
    {
    "took" : 7,
    "timed_out" : false,
    @@ -1116,7 +1146,21 @@ $ curl -XPUT http://elasticsearch:9200/products/items/4 -d '{"product": "chips",
    Run a Sort Query on the term `bananas`, and show the `average` price. We can also use `min, max, avg, sum`:

    ```
    $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '{"query" : {"term" : { "product" : "bananas" }}, "sort" : [{"price" : {"order" : "asc", "mode" : "avg"}}]}'
    $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '
    {
    "query" : {
    "term" : {
    "product" : "bananas"
    }
    },
    "sort" : [{
    "price" : {
    "order" : "asc",
    "mode" : "avg"
    }
    }]
    }'
    {
    "took" : 9,
    "timed_out" : false,
    @@ -1155,7 +1199,21 @@ $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '{"query" : {
    Running the same, but wanting to see the sum of all the prices:

    ```
    $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '{"query" : {"term" : { "product" : "bananas" }}, "sort" : [{"price" : {"order" : "asc", "mode" : "sum"}}]}'
    $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '
    {
    "query" : {
    "term" : {
    "product" : "bananas"
    }
    },
    "sort" : [{
    "price" : {
    "order" : "asc",
    "mode" : "sum"
    }
    }]
    }'
    {
    "took" : 34,
    "timed_out" : false,
    @@ -1209,7 +1267,15 @@ $ curl -XDELETE http://elasticsearch:9200/myindex
    We would like to delete all documents that has `"os_name": "Windows 10"`

    ```
    curl -XPOST 'http://elasticsearch:9200/weblogs/_delete_by_query?pretty' -d' {"query": {"match": {"os_name": "Windows 10"}}}'
    curl -XPOST 'http://elasticsearch:9200/weblogs/_delete_by_query?pretty' -d '
    {
    "query": {
    "match": {
    "os_name": "Windows 10"
    }
    }
    }'
    {
    "took" : 1217,
    "timed_out" : false,
  5. @ruanbekker ruanbekker revised this gist Jun 20, 2018. 1 changed file with 9 additions and 0 deletions.
    9 changes: 9 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -221,6 +221,15 @@ green open index-name-2018.01.19 Vp1EBoeMQkS-a_upLzedhQ 5 1 1220
    green open index-name-2018.01.17 hSJMzFJIQrePifCfgb1rOA 5 1 2875 0 3.8mb 1.9mb
    ```

    View only the index name header:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/_cat/indices/*2018.03.*?v&h=index'
    index
    index-name-2018.03.01
    index-name-2018.03.02
    ```

    ## How Many Documents in the ES Cluster (Across all Indices):

    ```
  6. @ruanbekker ruanbekker revised this gist Jun 20, 2018. 1 changed file with 16 additions and 7 deletions.
    23 changes: 16 additions & 7 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1294,12 +1294,21 @@ $ curl -XGET 'elasticsearch:9200/_tasks?detailed=true&actions=*/delete/byquery&p

    Setup the [S3 Snapshot Repository](https://sysadmins.co.za/aws-elasticsearch-register-s3-repository-for-snapshots-using-the-cli/?rbas_source=gist.github.com?rbas_sourcepage=cheatsheet-elasticsearch.md)

    Viw the Snapshot Repository:
    List the Snapshot Repositories:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/index-backups?pretty'
    $ curl -XGET 'http://127.0.0.1:9200/_cat/repositories?v'
    id type
    foo-bacups s3
    bar-backups s3
    ```

    View the Snapshot Repository:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/bar-backups?pretty'
    {
    "index-backups" : {
    "bar-backups" : {
    "type" : "s3",
    "settings" : {
    "bucket" : "my-es-snapshot-bucket",
    @@ -1316,7 +1325,7 @@ Create a Snapshot named `mysnapshot_ruan-test-2018-05-24_1` of the index: `ruan-

    ```
    $ curl -XPUT -H 'Content-Type: application/json' \
    'http://elasticsearch:9200/_snapshot/index-backups/mysnapshot_ruan-test-2018-05-24_1?wait_for_completion=true&pretty=true' -d '
    'http://elasticsearch:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1?wait_for_completion=true&pretty=true' -d '
    {
    "indices": "ruan-test-2018-05-24",
    "ignore_unavailable": true,
    @@ -1350,7 +1359,7 @@ $ curl -XPUT -H 'Content-Type: application/json' \
    Verify the Snapshot:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_cat/snapshots/index-backups?v&s=id'
    $ curl -XGET 'http://elasticsearch:9200/_cat/snapshots/bar-backups?v&s=id'
    id status start_epoch start_time end_epoch end_time duration indices successful_shards failed_shards total_shards
    mysnapshot_ruan-test-2018-05-24_1 SUCCESS 1527254411 06:20:11 1527254411 06:20:11 389ms 1 5 0 5
    ```
    @@ -1360,7 +1369,7 @@ mysnapshot_ruan-test-2018-05-24_1 SUCCESS 1527254411 06:20:11 1527254411
    Get the Metadata of the Snapshot:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/index-backups/mysnapshot_ruan-test-2018-05-24_1?pretty'
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1?pretty'
    {
    "snapshots" : [ {
    "snapshot" : "mysnapshot_ruan-test-2018-05-24_1",
    @@ -1395,7 +1404,7 @@ $ aws s3 --profile es ls s3://my-es-snapshot-bucket/ | grep VRTF2942QCeqyEaMxPgb
    Execute the Restore:

    ```
    $ curl -XPOST -H 'Content-Type: application/json' 'http://elasticsearch:9200/_snapshot/index-backups/mysnapshot_ruan-test-2018-05-24_1/_restore -d '
    $ curl -XPOST -H 'Content-Type: application/json' 'http://elasticsearch:9200/_snapshot/bar-backups/mysnapshot_ruan-test-2018-05-24_1/_restore -d '
    {
    "indices": "ruan-test-2018-05-24",
    "ignore_unavailable": true,
  7. @ruanbekker ruanbekker revised this gist Jun 16, 2018. 1 changed file with 19 additions and 0 deletions.
    19 changes: 19 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -8,6 +8,8 @@
    - [Nodes Overview](#nodes-overview)
    - [Indices Overview](#indices-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Settings]()
    - [Cluster Settings](#cluster-settings)
    - [Ingest](#ingest-documents-into-elasticsearch)
    - [Mapping](#mapping)
    - [Check Fields in Mappings](#check-fields-in-mappings)
    @@ -315,6 +317,23 @@ $ curl -XGET http://127.0.0.1:9200/_cache/clear
    {"_shards":{"total":21,"successful":15,"failed":0}}
    ```

    # Settings

    ## Cluster Settings

    Search Timeout:

    Global Search Timeout, that applies to all search queries across the entire cluster -> search.default_search_timeout:

    ```
    PUT /_cluster/settings
    {
    "persistent" : {
    "search.default_search_timeout" : "50"
    }
    }
    ```

    # Index Info (Shards, Replicas, Allocation):

    ## Create Index:
  8. @ruanbekker ruanbekker revised this gist May 29, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1273,7 +1273,7 @@ $ curl -XGET 'elasticsearch:9200/_tasks?detailed=true&actions=*/delete/byquery&p

    ## Elasticsearch S3 Snapshot Repo

    Setup the [S3 Snapshot Repository](https://sysadmins.co.za/aws-elasticsearch-register-s3-repository-for-snapshots-using-the-cli/?rbas_source=gist.github.com?rbas_sourcepage=cheachsheet-elasticsearch.md)
    Setup the [S3 Snapshot Repository](https://sysadmins.co.za/aws-elasticsearch-register-s3-repository-for-snapshots-using-the-cli/?rbas_source=gist.github.com?rbas_sourcepage=cheatsheet-elasticsearch.md)

    Viw the Snapshot Repository:

  9. @ruanbekker ruanbekker revised this gist May 29, 2018. 1 changed file with 132 additions and 1 deletion.
    133 changes: 132 additions & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -19,6 +19,10 @@
    - [Sort](#sort)
    - [Aggregate]()
    - [Delete](#delete)
    - [Snapshots](#snapshots)
    - [Create Snapshot Repository on S3](#elasticsearch-s3-snapshot-repo)
    - [Create a Snapshot](#elasticsearch-snapshots)
    - [Restore from a Snapshot](#elasticsearch-restore)

    # Resources
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
    @@ -1263,4 +1267,131 @@ $ curl -XGET 'elasticsearch:9200/_tasks?detailed=true&actions=*/delete/byquery&p
    }
    }
    }
    ```
    ```

    # Snapshots

    ## Elasticsearch S3 Snapshot Repo

    Setup the [S3 Snapshot Repository](https://sysadmins.co.za/aws-elasticsearch-register-s3-repository-for-snapshots-using-the-cli/?rbas_source=gist.github.com?rbas_sourcepage=cheachsheet-elasticsearch.md)

    Viw the Snapshot Repository:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/index-backups?pretty'
    {
    "index-backups" : {
    "type" : "s3",
    "settings" : {
    "bucket" : "my-es-snapshot-bucket",
    "region" : "eu-west-1",
    "role_arn" : "arn:aws:iam::0123456789012:role/elasticsearch-snapshot-role"
    }
    }
    }
    ```

    ## Elasticsearch Snapshots

    Create a Snapshot named `mysnapshot_ruan-test-2018-05-24_1` of the index: `ruan-test-2018-05-24` and return the exit when the snapshot is done:

    ```
    $ curl -XPUT -H 'Content-Type: application/json' \
    'http://elasticsearch:9200/_snapshot/index-backups/mysnapshot_ruan-test-2018-05-24_1?wait_for_completion=true&pretty=true' -d '
    {
    "indices": "ruan-test-2018-05-24",
    "ignore_unavailable": true,
    "include_global_state": false
    }
    '
    {
    "snapshot" : {
    "snapshot" : "mysnapshot_ruan-test-2018-05-24_1",
    "uuid" : "YRTE5922QCeqyEaMxPqb1A",
    "version_id" : 6000199,
    "version" : "6.0.1",
    "indices" : [ "ruan-test-2018-05-24" ],
    "state" : "SUCCESS",
    "start_time" : "2018-05-25T13:20:11.497Z",
    "start_time_in_millis" : 1527254411497,
    "end_time" : "2018-05-25T13:20:11.886Z",
    "end_time_in_millis" : 1527254411886,
    "duration_in_millis" : 389,
    "failures" : [ ],
    "shards" : {
    "total" : 5,
    "failed" : 0,
    "successful" : 5
    }
    }
    }
    ```

    Verify the Snapshot:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_cat/snapshots/index-backups?v&s=id'
    id status start_epoch start_time end_epoch end_time duration indices successful_shards failed_shards total_shards
    mysnapshot_ruan-test-2018-05-24_1 SUCCESS 1527254411 06:20:11 1527254411 06:20:11 389ms 1 5 0 5
    ```

    ## Elasticsearch Restore

    Get the Metadata of the Snapshot:

    ```
    $ curl -XGET 'http://elasticsearch:9200/_snapshot/index-backups/mysnapshot_ruan-test-2018-05-24_1?pretty'
    {
    "snapshots" : [ {
    "snapshot" : "mysnapshot_ruan-test-2018-05-24_1",
    "uuid" : "YRTE5922QCeqyEaMxPqb1A",
    "version_id" : 6000199,
    "version" : "6.0.1",
    "indices" : [ "ruan-test-2018-05-24" ],
    "state" : "SUCCESS",
    "start_time" : "2018-05-25T13:20:11.497Z",
    "start_time_in_millis" : 1527254411497,
    "end_time" : "2018-05-25T13:20:11.886Z",
    "end_time_in_millis" : 1527254411886,
    "duration_in_millis" : 389,
    "failures" : [ ],
    "shards" : {
    "total" : 5,
    "failed" : 0,
    "successful" : 5
    }
    } ]
    }
    ```

    Inspect the Snapshot on S3:

    ```
    $ aws s3 --profile es ls s3://my-es-snapshot-bucket/ | grep VRTF2942QCeqyEaMxPgb1B
    2018-05-25 15:20:12 90 meta-VRTF2942QCeqyEaMxPgb1B.dat
    2018-05-25 15:20:12 258 snap-VRTF2942QCeqyEaMxPgb1B.dat
    ```

    Execute the Restore:

    ```
    $ curl -XPOST -H 'Content-Type: application/json' 'http://elasticsearch:9200/_snapshot/index-backups/mysnapshot_ruan-test-2018-05-24_1/_restore -d '
    {
    "indices": "ruan-test-2018-05-24",
    "ignore_unavailable": true,
    "include_global_state": false,
    "rename_pattern": "index_(.+)",
    "rename_replacement": "restored_index_$1"
    }
    '
    ```

    or leave out the body for normal restore

    ## Elasticsearch Snapshot Resources:

    - https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html
    - https://www.youtube.com/watch?v=Otl-IcmbiDE
    - https://recology.info/2015/02/elasticsearch-backup-restore/
    - https://medium.com/@rcdexta/periodic-snapshotting-of-elasticsearch-indices-f6b6ca221a0c
  10. @ruanbekker ruanbekker revised this gist Mar 13, 2018. 1 changed file with 106 additions and 0 deletions.
    106 changes: 106 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -384,6 +384,112 @@ my2ndindex 0 p STARTED 0 130b 10.0.2.27 bDWFHuw
    my2ndindex 0 r STARTED 0 130b 10.0.2.24 rNDYCtL
    ```

    ## Create a Index:

    Create a Index with Default Settings:

    ```
    $ curl -XPUT -H 'Content-Type: application/json' 'http://127.0.0.1:9200/ruan-test-2018.03.12'
    ```

    View the settings of the created index:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.12/_settings?pretty'
    {
    "ruan-test-2018.03.12" : {
    "settings" : {
    "index" : {
    "creation_date" : "1520929659349",
    "number_of_shards" : "5",
    "number_of_replicas" : "1",
    "uuid" : "EwGz6y7XQkK0ZI08u8qdrQ",
    "version" : {
    "created" : "6000199"
    },
    "provided_name" : "ruan-test-2018.03.12"
    }
    }
    }
    }
    ```

    Remember that primary shard number can only be set on index creation. Change the settings of the index, let's update the index to: 2 replica shards, and the total_fields limit to: 2000

    ```
    $ curl -XPUT -H 'Content-Type: application/json' 'http://127.0.0.1:9200/ruan-test-2018.03.12/_settings' -d '{"number_of_replicas": 0, "index.mapping.total_fields.limit": 2000}'
    ```

    View the changes:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.12/_settings?pretty'
    {
    "ruan-test-2018.03.12" : {
    "settings" : {
    "index" : {
    "mapping" : {
    "total_fields" : {
    "limit" : "2000"
    }
    },
    "number_of_shards" : "5",
    "provided_name" : "ruan-test-2018.03.12",
    "creation_date" : "1520929659349",
    "number_of_replicas" : "0",
    "uuid" : "EwGz6y7XQkK0ZI08u8qdrQ",
    "version" : {
    "created" : "6000199"
    }
    }
    }
    }
    }
    ```

    Now, to set the settings on Index Creation:


    ```
    $ curl -XPUT -H 'Content-Type: application/json' 'http://127.0.0.1:9200/ruan-test-2018.03.13' -d '{"settings": {"number_of_replicas": 1, "number_of_shards": 2, "index.mapping.total_fields.limit": 2000}}'
    ```

    Verifying our settings:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/ruan-test-2018.03.13/_settings?pretty'
    {
    "ruan-test-2018.03.13" : {
    "settings" : {
    "index" : {
    "mapping" : {
    "total_fields" : {
    "limit" : "2000"
    }
    },
    "number_of_shards" : "2",
    "provided_name" : "ruan-test-2018.03.13",
    "creation_date" : "1520929638792",
    "number_of_replicas" : "1",
    "uuid" : "hEY8HrlRTFuiYLwKVDAraQ",
    "version" : {
    "created" : "6000199"
    }
    }
    }
    }
    }
    ```

    Viewing our indexes:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/_cat/indices/ruan-test-*?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open ruan-test-2018.03.12 EwGz6y7XQkK0ZI08u8qdrQ 5 1 2 0 15.7kb 7.8kb
    green open ruan-test-2018.03.13 hEY8HrlRTFuiYLwKVDAraQ 2 1 0 0 932b 466b
    ```

    ## Ingest Document into Elasticsearch:

    Let's ingest one docuemnt into Elasticsearch, and in this case we will specify the document id as `1`
  11. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 20 additions and 0 deletions.
    20 changes: 20 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,6 +6,7 @@
    - [Index Level](#cluster-health-index-level)
    - [Shard Level](#cluster-health-shard-level)
    - [Nodes Overview](#nodes-overview)
    - [Indices Overview](#indices-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)
    - [Mapping](#mapping)
    @@ -189,12 +190,31 @@ mstWlaoyTM69xhSt-_rZAA 10.0.2.18 10.0.2.18 mstWlao

    ## Indices Overview:

    View all your indices in your cluster:

    ```
    $ curl -XGET http://elasticsearch:9200/_cat/indices?v
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open ruan-test CrQZB2L4SaaYCkvYPx5vUA 5 1 38 0 131.9kb 78.6kb
    ```

    View one index:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/_cat/indices/index-name-2018.01.01?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open index-name-2018.01.01 Nk8SMQvRSIaNm854bc3Zjg 5 1 395552 0 755.6mb 377.8mb
    ```

    View a range of indices:

    ```
    $ curl -XGET 'https://http://127.0.0.1:9200/_cat/indices/index-name-2018.01*?v'
    health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
    green open index-name-2018.01.19 Vp1EBoeMQkS-a_upLzedhQ 5 1 1220 0 2.6mb 1.3mb
    green open index-name-2018.01.17 hSJMzFJIQrePifCfgb1rOA 5 1 2875 0 3.8mb 1.9mb
    ```

    ## How Many Documents in the ES Cluster (Across all Indices):

    ```
  12. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -13,6 +13,8 @@
    - [Close API](#open--close-api)
    - [Search](#searching)
    - [Query](#query)
    - [Query by Match](#query-by-match)
    - [Query with Bool](#query-with-bool)
    - [Sort](#sort)
    - [Aggregate]()
    - [Delete](#delete)
  13. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -892,19 +892,19 @@ curl -XGET 'http://elasticsearch:9200/people/_search?q=age:30&pretty'
    }
    ```

    Query by Term and limit results by 2:
    #### Query by Term and limit results by 2:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"term": {"title": "traefik"}}, "size": 2}'
    ```

    Query by Match:
    #### Query by Match:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"match": {"title": "traefik"}}, "size": 10}'
    ```

    Query with Bool:
    #### Query with Bool:

    - Check if field exists in index:

  14. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -9,6 +9,7 @@
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)
    - [Mapping](#mapping)
    - [Check Fields in Mappings](#check-fields-in-mappings)
    - [Close API](#open--close-api)
    - [Search](#searching)
    - [Query](#query)
  15. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 70 additions and 1 deletion.
    71 changes: 70 additions & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -8,7 +8,8 @@
    - [Nodes Overview](#nodes-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)
    - [Open/Close API](#open--close-api)
    - [Mapping](#mapping)
    - [Close API](#open--close-api)
    - [Search](#searching)
    - [Query](#query)
    - [Sort](#sort)
    @@ -621,6 +622,41 @@ Ingest using the Bulk Api:
    curl -XPOST 'http://elasticsearch:9200/info/_bulk?pretty' --data-binary @info.json
    ```

    # Mapping

    ## Create Mapping
    ## View Mappings

    ## Check Fields in Mappings:

    Check if a field exisists in your mapping:

    ```
    $ curl -XGET 'http://127.0.0.1:9200/index-name-2018.03.01/_mapping/docs/field/company?pretty'
    {
    "index-name-2018.03.01" : {
    "mappings" : {
    "docs" : {
    "company" : {
    "full_name" : "company",
    "mapping" : {
    "company" : {
    "type" : "text",
    "fields" : {
    "keyword" : {
    "type" : "keyword",
    "ignore_above" : 256
    }
    }
    }
    }
    }
    }
    }
    }
    }
    ```

    # Open / Close API:
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-open-close.html

    @@ -867,6 +903,39 @@ Query by Match:
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"match": {"title": "traefik"}}, "size": 10}'
    ```

    Query with Bool:

    - Check if field exists in index:

    ```
    $ curl http://127.0.0.1:9200/test4/_search?pretty -d '{"query": {"bool": {"must": [{"exists": {"field": "name"}}]}}}}'
    {
    "took" : 7,
    "timed_out" : false,
    "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
    },
    "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [
    {
    "_index" : "test4",
    "_type" : "docs",
    "_id" : "2",
    "_score" : 1.0,
    "_source" : {
    "id" : "2",
    "name" : "ruan"
    }
    }
    ]
    }
    }
    ```

    - https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html

    ## Sort
  16. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -4,6 +4,7 @@

    - [Cluster Health](#cluster-health)
    - [Index Level](#cluster-health-index-level)
    - [Shard Level](#cluster-health-shard-level)
    - [Nodes Overview](#nodes-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)
  17. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,7 +3,7 @@
    # Shortlinks:

    - [Cluster Health](#cluster-health)
    -- [Index Level](#cluster-health-index-level)
    - [Index Level](#cluster-health-index-level)
    - [Nodes Overview](#nodes-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)
  18. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,6 +3,7 @@
    # Shortlinks:

    - [Cluster Health](#cluster-health)
    -- [Index Level](#cluster-health-index-level)
    - [Nodes Overview](#nodes-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)
  19. @ruanbekker ruanbekker revised this gist Mar 12, 2018. 1 changed file with 8 additions and 0 deletions.
    8 changes: 8 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -14,6 +14,9 @@
    - [Delete](#delete)

    # Resources
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/docs.html
    - https://www.elastic.co/blog/managing-time-based-indices-efficiently
    - http://joelabrahamsson.com/elasticsearch-101/
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html
    - https://chatbots.network/logstash-exclude-bots-from-result/
    @@ -958,6 +961,11 @@ $ curl -XPOST http://elasticsearch:9200/products/_search?pretty -d '{"query" : {

    # Delete

    References:

    - [Delete API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete.html)
    - [Delete by Query](https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-delete-by-query.html)

    ## Delete Index:

    ```
  20. @ruanbekker ruanbekker revised this gist Mar 11, 2018. 1 changed file with 10 additions and 1 deletion.
    11 changes: 10 additions & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -698,7 +698,8 @@ $ curl -XGET 'http://elasticsearch:9200/people/users/_search?q=age:28&explain&pr
    },
    {
    "value" : 1.0,
    "description" : "queryNorm",
    "description" : "
    Norm",
    "details" : [ ]
    }
    ]
    @@ -855,6 +856,14 @@ Query by Term and limit results by 2:
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"term": {"title": "traefik"}}, "size": 2}'
    ```

    Query by Match:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"match": {"title": "traefik"}}, "size": 10}'
    ```

    - https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html

    ## Sort
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html

  21. @ruanbekker ruanbekker revised this gist Sep 23, 2017. 1 changed file with 6 additions and 0 deletions.
    6 changes: 6 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -849,6 +849,12 @@ curl -XGET 'http://elasticsearch:9200/people/_search?q=age:30&pretty'
    }
    ```

    Query by Term and limit results by 2:

    ```
    $ curl -XGET http://127.0.0.1:9200/scrape-sysadmins/_search?pretty -d '{"query": {"term": {"title": "traefik"}}, "size": 2}'
    ```

    ## Sort
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html

  22. @ruanbekker ruanbekker revised this gist Aug 24, 2017. 1 changed file with 7 additions and 0 deletions.
    7 changes: 7 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -275,6 +275,13 @@ insertOrder timeInQueue priority source
    1737 1.3s URGENT shard-started ([sysadmins-2017.06.02][3], node[WR3y31g1TnuufpNyrJnQtg], [R], v[91], s[INITIALIZING], a[id=JmrtwtYURMyQF6LspeJXLg], unassigned_info[[reason=CLUSTER_RECOVERED], at[2017-08-11T07:50:56.550Z]]), reason [after recovery (replica) from node [{es01}{6ND8sZ_rTqaL42VdlxyW7Q}{10.79.2.193}{10.79.2.193:9300}]]
    ```

    ## Clear Cache:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cache/clear
    {"_shards":{"total":21,"successful":15,"failed":0}}
    ```

    # Index Info (Shards, Replicas, Allocation):

    ## Create Index:
  23. @ruanbekker ruanbekker revised this gist Aug 14, 2017. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -239,6 +239,9 @@ $ curl -XPUT 'localhost:9200/_cluster/settings?pretty' -d'
    '
    ```

    ## Recovery Resources:
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-recovery.html

    ## Recovering from Node Failure:

    At the moment one of the nodes were down, and up again:
  24. @ruanbekker ruanbekker revised this gist Aug 12, 2017. 1 changed file with 33 additions and 0 deletions.
    33 changes: 33 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -239,6 +239,39 @@ $ curl -XPUT 'localhost:9200/_cluster/settings?pretty' -d'
    '
    ```

    ## Recovering from Node Failure:

    At the moment one of the nodes were down, and up again:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cat/allocation?v
    shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
    290 54.1mb 1gb 20mb 1gb 98 10.79.2.193 10.79.2.193 es01
    151 43.5mb 1gb 11.9gb 13gb 8 10.79.3.171 10.79.3.171 es02
    139 UNASSIGNED
    ```

    ## Recovery API:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cat/recovery?v
    index shard time type stage source_host target_host repository snapshot files files_percent bytes bytes_percent total_files total_bytes translog translog_percent total_translog
    sysadmins-2017.06.19 0 1512 replica done 10.79.2.193 10.79.3.171 n/a n/a 31 100.0% 340020 100.0% 31 340020 0 100.0% 0
    sysadmins-2017.06.19 0 7739 store done 10.79.2.193 10.79.2.193 n/a n/a 0 100.0% 0 100.0% 31 340020 0 100.0% 0
    sysadmins-2017.06.19 1 2592 relocation done 10.79.2.193 10.79.3.171 n/a n/a 13 100.0% 246229 100.0% 13 246229 0 100.0% 0
    sysadmins-2017.06.19 1 613 replica done 10.79.3.171 10.79.2.193 n/a n/a 0 0.0% 0 0.0% 0 0 0 100.0% 0
    ```

    ## Pending Tasks:

    ```
    $ curl -XGET http://127.0.0.1:9200/_cat/pending_tasks?v
    insertOrder timeInQueue priority source
    1736 1.8s URGENT shard-started ([sysadmins-2017.06.02][2], node[WR3y31g1TnuufpNyrJnQtg], [R], v[91], s[INITIALIZING], a[id=wVTDn4nFSKKxvi07cU0uCg], unassigned_info[[reason=CLUSTER_RECOVERED], at[2017-08-11T07:50:56.550Z]]), reason [after recovery (replica) from node [{es01}{6ND8sZ_rTqaL42VdlxyW7Q}{10.79.2.193}{10.79.2.193:9300}]]
    1737 1.3s URGENT shard-started ([sysadmins-2017.06.02][3], node[WR3y31g1TnuufpNyrJnQtg], [R], v[91], s[INITIALIZING], a[id=JmrtwtYURMyQF6LspeJXLg], unassigned_info[[reason=CLUSTER_RECOVERED], at[2017-08-11T07:50:56.550Z]]), reason [after recovery (replica) from node [{es01}{6ND8sZ_rTqaL42VdlxyW7Q}{10.79.2.193}{10.79.2.193:9300}]]
    ```

    # Index Info (Shards, Replicas, Allocation):

    ## Create Index:
  25. @ruanbekker ruanbekker revised this gist Aug 12, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -16,7 +16,7 @@
    # Resources
    - http://joelabrahamsson.com/elasticsearch-101/
    - https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html

    - https://chatbots.network/logstash-exclude-bots-from-result/

    # Overview

  26. @ruanbekker ruanbekker revised this gist Aug 12, 2017. 1 changed file with 6 additions and 5 deletions.
    11 changes: 6 additions & 5 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,11 +6,12 @@
    - [Nodes Overview](#nodes-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)
    - [](#open--close-api)
    - [](#searching)
    - [](#query)
    - [](#sort)
    - [](#delete)
    - [Open/Close API](#open--close-api)
    - [Search](#searching)
    - [Query](#query)
    - [Sort](#sort)
    - [Aggregate]()
    - [Delete](#delete)

    # Resources
    - http://joelabrahamsson.com/elasticsearch-101/
  27. @ruanbekker ruanbekker revised this gist Aug 12, 2017. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,7 +6,11 @@
    - [Nodes Overview](#nodes-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)

    - [](#open--close-api)
    - [](#searching)
    - [](#query)
    - [](#sort)
    - [](#delete)

    # Resources
    - http://joelabrahamsson.com/elasticsearch-101/
  28. @ruanbekker ruanbekker revised this gist Aug 12, 2017. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -4,6 +4,8 @@

    - [Cluster Health](#cluster-health)
    - [Nodes Overview](#nodes-overview)
    - [Cluster Maintenance](#cluster-maintenance)
    - [Ingest](#ingest-documents-into-elasticsearch)


    # Resources
  29. @ruanbekker ruanbekker revised this gist Aug 12, 2017. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,6 +3,8 @@
    # Shortlinks:

    - [Cluster Health](#cluster-health)
    - [Nodes Overview](#nodes-overview)


    # Resources
    - http://joelabrahamsson.com/elasticsearch-101/
  30. @ruanbekker ruanbekker revised this gist Aug 12, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion cheatsheet-elasticsearch.md
    View file
    Original file line number Diff line number Diff line change
    @@ -2,7 +2,7 @@

    # Shortlinks:

    - [#]()
    - [Cluster Health](#cluster-health)

    # Resources
    - http://joelabrahamsson.com/elasticsearch-101/