Skip to content

Instantly share code, notes, and snippets.

@vanerpool

vanerpool/pyspark_tips.py

Created November 1, 2018 10:10
Show Gist options
  • Save vanerpool/9be57e925f33ba37172be1ab2f825f19 to your computer and use it in GitHub Desktop.
Save vanerpool/9be57e925f33ba37172be1ab2f825f19 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @sirpantene sirpantene revised this gist Nov 1, 2018. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions pyspark_tips.py
    View file
    Original file line number Diff line number Diff line change
    @@ -1 +1,2 @@
    # insert overwrite table partition
    df.write.mode("overwrite").partitionBy("col1","col2","col3").insertInto("db_name.table_name")
  2. @sirpantene sirpantene created this gist Nov 1, 2018.
    1 change: 1 addition & 0 deletions pyspark_tips.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1 @@
    df.write.mode("overwrite").partitionBy("col1","col2","col3").insertInto("db_name.table_name")