Skip to content

Instantly share code, notes, and snippets.

@tommydangerous

tommydangerous/mage_delta-lake_read_battle-history.py

Last active May 9, 2023 23:42
Show Gist options
  • Save tommydangerous/f86cf4a62bdc308de85bab20f4892094 to your computer and use it in GitHub Desktop.
Save tommydangerous/f86cf4a62bdc308de85bab20f4892094 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. tommydangerous revised this gist May 9, 2023. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions mage_delta-lake_read_battle-history.py
    View file
    Original file line number Diff line number Diff line change
    @@ -2,6 +2,8 @@


    dt = DeltaTable(
    # Change this to your unique URI from a previous step
    # if you’re using your own AWS credentials.
    's3://mage-demo-public/battle-history/1337',
    storage_options={
    'AWS_ACCESS_KEY_ID': '...',
  2. tommydangerous revised this gist May 9, 2023. 1 changed file with 0 additions and 1 deletion.
    1 change: 0 additions & 1 deletion mage_delta-lake_read_battle-history.py
    View file
    Original file line number Diff line number Diff line change
    @@ -4,7 +4,6 @@
    dt = DeltaTable(
    's3://mage-demo-public/battle-history/1337',
    storage_options={
    # Read-only credentials for Amazon S3
    'AWS_ACCESS_KEY_ID': '...',
    'AWS_SECRET_ACCESS_KEY': '...',
    },
  3. tommydangerous revised this gist May 9, 2023. 1 changed file with 2 additions and 5 deletions.
    7 changes: 2 additions & 5 deletions mage_delta-lake_read_battle-history.py
    View file
    Original file line number Diff line number Diff line change
    @@ -3,13 +3,10 @@

    dt = DeltaTable(
    's3://mage-demo-public/battle-history/1337',
    # If you created a Delta Table with your own personal storage options
    # from the previous step, change the storage options here
    # to match the ones you used previously.
    storage_options={
    # Read-only credentials for Amazon S3
    'AWS_ACCESS_KEY_ID': 'AKIAZ4SRK3YKRLA66XKN',
    'AWS_SECRET_ACCESS_KEY': '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',
    'AWS_ACCESS_KEY_ID': '...',
    'AWS_SECRET_ACCESS_KEY': '...',
    },
    )
    dt.to_pandas()
  4. tommydangerous created this gist May 9, 2023.
    15 changes: 15 additions & 0 deletions mage_delta-lake_read_battle-history.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,15 @@
    from deltalake import DeltaTable


    dt = DeltaTable(
    's3://mage-demo-public/battle-history/1337',
    # If you created a Delta Table with your own personal storage options
    # from the previous step, change the storage options here
    # to match the ones you used previously.
    storage_options={
    # Read-only credentials for Amazon S3
    'AWS_ACCESS_KEY_ID': 'AKIAZ4SRK3YKRLA66XKN',
    'AWS_SECRET_ACCESS_KEY': '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',
    },
    )
    dt.to_pandas()