Skip to content

Instantly share code, notes, and snippets.

@kaxil

kaxil/airflow_json_variables.py

Last active December 10, 2021 09:39
Show Gist options
  • Save kaxil/61f41dd87a69230d1a637dc3a1d2fa2c to your computer and use it in GitHub Desktop.
Save kaxil/61f41dd87a69230d1a637dc3a1d2fa2c to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. kaxil revised this gist Dec 25, 2018. 1 changed file with 8 additions and 1 deletion.
    9 changes: 8 additions & 1 deletion airflow_json_variables.py
    View file
    Original file line number Diff line number Diff line change
    @@ -11,4 +11,11 @@
    dag_config = Variable.get("dag1_config", deserialize_json=True)
    dag_config["var1"]
    dag_config["var2"]
    dag_config["var3"]
    dag_config["var3"]

    # You can directly use it Templated arguments {{ var.json.my_var.path }}
    bash_task = BashOperator(
    task_id="bash_task",
    bash_command='{{ var.json.dag1_config.var1 }} ',
    dag=dag,
    )
  2. kaxil created this gist Dec 25, 2018.
    14 changes: 14 additions & 0 deletions airflow_json_variables.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,14 @@
    from airflow.models import Variable

    # Common (Not-so-nice way)
    # 3 DB connections when the file is parsed
    var1 = Variable.get("var1")
    var2 = Variable.get("var2")
    var3 = Variable.get("var3")

    # Recommended Way
    # Just 1 Database call
    dag_config = Variable.get("dag1_config", deserialize_json=True)
    dag_config["var1"]
    dag_config["var2"]
    dag_config["var3"]