Skip to content

Instantly share code, notes, and snippets.

@statguy

statguy/write_read_pickle_example.py

Last active February 10, 2022 08:59
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save statguy/f92b8fc08211b232c4ae77ca8662a704 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save statguy/f92b8fc08211b232c4ae77ca8662a704 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. statguy revised this gist Feb 10, 2022. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions write_read_pickle_example.py
    View file
    Original file line number Diff line number Diff line change
    @@ -2,6 +2,7 @@
    import fsspec
    import pickle


    def write_pickle(data: Any, path: str) -> None:
    with fsspec.open(path, "wb") as file:
    pickle.dump(data, file)
  2. statguy created this gist Feb 10, 2022.
    19 changes: 19 additions & 0 deletions write_read_pickle_example.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,19 @@
    from typing import Any
    import fsspec
    import pickle

    def write_pickle(data: Any, path: str) -> None:
    with fsspec.open(path, "wb") as file:
    pickle.dump(data, file)

    def read_pickle(path: str) -> Any:
    with fsspec.open(path, "rb") as file:
    return pickle.load(file)


    data = {"a": 1}
    write_pickle(data, "s3://my_bucket/data.tmp")
    write_pickle(data, "file:///tmp/data.tmp")

    assert read_pickle("s3://my_bucket/data.tmp") == data
    assert read_pickle("file:///tmp/data.tmp") == data