Skip to content

Instantly share code, notes, and snippets.

@edwintyh

edwintyh/pyspark_rename.py

Created February 2, 2022 04:09
Show Gist options
  • Save edwintyh/5055bc30004b8a0337b8eec5279ef6b4 to your computer and use it in GitHub Desktop.
Save edwintyh/5055bc30004b8a0337b8eec5279ef6b4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
pyspark_rename.py
def rename(self, columns):
'''
Rename colum headers
Args:
columns(dict):
A dictionary where key are current column names and values are new column names
{'old_column_name1':'new_column_name1', 'old_column_name2':'new_column_name2'}
Returns:
pyspark.sql.DataFrame
'''
for old_name, new_name in columns.items():
self = self.withColumnRenamed(old_name, new_name)
return self
pyspark.sql.DataFrame.rename = rename
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment