This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -- =================================================================================== | |
| -- Configuration Block: Adjust these parameters for your analysis | |
| -- =================================================================================== | |
| DECLARE reservation_admin_project_id STRING DEFAULT 'your-reservation-admin-project'; | |
| DECLARE region STRING DEFAULT 'us'; | |
| DECLARE start_timestamp TIMESTAMP DEFAULT TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY); | |
| DECLARE end_timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP(); | |
| -- Note: Adjust the cost based on your BigQuery Edition and commitment level. | |
| -- Example: Enterprise Edition Pay-as-you-go is ~$0.06 per slot-hour. | |
| DECLARE cost_per_slot_hour FLOAT64 DEFAULT 0.06; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| interactive_recipe_agent = LlmAgent( | |
| name="interactive_recipe_agent", | |
| model=config.worker_model, | |
| description=( | |
| "The primary assistant for creating baby food recipes. It collaborates" | |
| " with the user to generate a recipe and then gets it approved before" | |
| " finalization." | |
| ), | |
| instruction=f""" | |
| You are a friendly and helpful AI assistant for parents, named 'Tiny Tastes'. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from datetime import datetime, timedelta | |
| from airflow.models import DAG | |
| from airflow.operators.python import PythonOperator | |
| from airflow.utils.session import provide_session | |
| from alvin_integration.producers.airflow.pipeline.extractor import extract_dag_metadata | |
| seven_days_ago = datetime.combine( | |
| datetime.today() - timedelta(1), datetime.min.time() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| "facets": { | |
| "alvin": { | |
| "connection_id": "google_cloud_default", | |
| "execution": { | |
| "alvin_package_version": "0.15.7", | |
| "platform_id": "airflowprd", | |
| "duration": 41.69428, | |
| ... | |
| }, | |
| "job_id": "airflow_2141512512_fcf9812412412412412421412" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "eventType": "START", | |
| "eventTime": "2020-12-09T23:37:31.081Z", | |
| "run": { | |
| "runId": "3b452093-782c-4ef2-9c0c-aafe2aa6f34d", | |
| }, | |
| "job": { | |
| "namespace": "my-scheduler-namespace", | |
| "name": "myjob.mytask", | |
| "facets": {...} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def execute(self, context: 'Context'): | |
| # ... code suppresed for readability | |
| if isinstance(self.sql, str): | |
| job_id: Union[str, List[str]] = self.hook.run_query( | |
| sql=self.sql, | |
| destination_dataset_table=self.destination_dataset_table, | |
| write_disposition=self.write_disposition, | |
| allow_large_results=self.allow_large_results, | |
| flatten_results=self.flatten_results, | |
| udf_config=self.udf_config, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def execute(self, context: Any): | |
| hook = BigQueryHook( | |
| gcp_conn_id=self.gcp_conn_id, | |
| delegate_to=self.delegate_to, | |
| impersonation_chain=self.impersonation_chain, | |
| ) | |
| self.hook = hook | |
| job_id = self._job_id(context) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| handler = logging.StreamHandler() | |
| # Set debug level for the client | |
| es_logger.setLevel(logging.DEBUG) | |
| es_logger.addHandler(handler) | |
| es_logger.disabled = False | |
| # Set trace logger | |
| es_trace_logger = logging.getLogger('elasticsearch.trace') | |
| es_trace_logger.setLevel(logging.DEBUG) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| sql_objects: | |
| - name: 'functions' | |
| enabled: True |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SELECT f.SCHEMA_NAME AS SCHEMA_NAME, | |
| f.FUNCTION_NAME AS function_name, | |
| f.INPUT_PARAMETER_COUNT AS input_parameter_count, | |
| f.RETURN_VALUE_COUNT AS return_value_count, | |
| f.DEFINITION AS definition, | |
| f.IS_VALID AS is_valid, | |
| f.OWNER_NAME AS owner_name, | |
| f.CREATE_TIME AS create_time | |
| FROM SYS.FUNCTIONS f | |
| // ... Omitted to improve readability |
NewerOlder