Skip to content

Instantly share code, notes, and snippets.

@eqbalz

eqbalz/hive_data_summary.R

Last active February 8, 2017 19:03
Show Gist options
  • Save eqbalz/7f06b026a1e874daa58b3d29ebb347d6 to your computer and use it in GitHub Desktop.
Save eqbalz/7f06b026a1e874daa58b3d29ebb347d6 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
hive_data_summary.R
#RxHiveData only works with spark compute context
computeContext <- RxSpark(consoleOutput = TRUE, persistentRun = TRUE)
rxSetComputeContext(computeContext)
airColInfo <- list(
arrdelay = list(type = "integer"),
#crsdeptime = list(type = "numeric"),
dayofweek = list(
type = "factor",
levels = c(
"Monday",
"Tuesday",
"Wednesday",
"Thursday",
"Friday",
"Saturday",
"Sunday"
)
)
)
hive_data <- RxHiveData(
query = "select * from AirlineDemoSmallHive",
colInfo = airColInfo
)
rxSummary(~., hive_data)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment