Each of these commands will run an ad hoc http static server in your current (or specified) directory, available at http://localhost:8000. Use this power wisely.
$ python -m SimpleHTTPServer 8000
jornh
/ pandas_s3_streaming.py
Created
January 24, 2018 07:12
Streaming pandas DataFrame to/from S3 with on-the-fly processing and GZIP compression
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def s3_to_pandas(client, bucket, key, header=None): | |
| # get key using boto3 client | |
| obj = client.get_object(Bucket=bucket, Key=key) | |
| gz = gzip.GzipFile(fileobj=obj['Body']) | |
| # load stream directly to DF | |
| return pd.read_csv(gz, header=header, dtype=str) | |
| def s3_to_pandas_with_processing(client, bucket, key, header=None): |
jornh
/ web-servers.md
Created
November 1, 2016 13:46
— forked from willurd/web-servers.md
Big list of http static server one-liners