Skip to content

Instantly share code, notes, and snippets.

@husanpy

husanpy/duckdb_bq_storage_api.py

Forked from ML-engineer/duckdb_bq_storage_api.py
Created March 17, 2024 16:53
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save husanpy/7b877f1edd072dc908bc1a02e255300c to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save husanpy/7b877f1edd072dc908bc1a02e255300c to your computer and use it in GitHub Desktop.
Download ZIP
Read BQ table to DuckDB directly from storage read api
Raw
duckdb_bq_storage_api.py
import duckdb
from google.cloud import bigquery
bqclient = bigquery.Client()
table = bigquery.TableReference.from_string(
"bigquery-public-data.utility_us.country_code_iso"
)
rows = bqclient.list_rows(table)
country_code_iso = rows.to_arrow(create_bqstorage_client=True)
cursor = duckdb.connect()
print(cursor.execute('SELECT * FROM country_code_iso').fetchall())
@husanpy
Copy link
Author

husanpy commented Mar 17, 2024

.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment