Skip to content

Instantly share code, notes, and snippets.

@rafaelfelix

rafaelfelix/spark-on-k8s-sample-job-output.txt

Last active November 13, 2019 17:53
Show Gist options
  • Save rafaelfelix/9c14a1a061689a368e9677f129e765d2 to your computer and use it in GitHub Desktop.
Save rafaelfelix/9c14a1a061689a368e9677f129e765d2 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. rafaelfelix revised this gist Nov 13, 2019. 1 changed file with 4 additions and 2 deletions.
    6 changes: 4 additions & 2 deletions spark-on-k8s-sample-job-output.txt
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    Using properties file: null
    Parsed arguments:
    master k8s://https://192.168.99.108:8443
    master k8s://https://192.168.99.105:8443
    deployMode client
    executorMemory null
    executorCores null
    @@ -43,4 +43,6 @@ Spark config:
    (spark.submit.deployMode,client)
    (spark.master,k8s://https://192.168.99.108:8443)
    Classpath elements:
    file:/Users/rafael.correa/workspace/github/olx-global/spark-on-k8s/tmp/spark/examples/jars/spark-examples_2.11-2.4.3.jar
    file:/Users/rafael.correa/workspace/github/olx-global/spark-on-k8s/tmp/spark/examples/jars/spark-examples_2.11-2.4.3.jar

    Pi is roughly 3.147275736378682
  2. rafaelfelix created this gist Nov 13, 2019.
    46 changes: 46 additions & 0 deletions spark-on-k8s-sample-job-output.txt
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,46 @@
    Using properties file: null
    Parsed arguments:
    master k8s://https://192.168.99.108:8443
    deployMode client
    executorMemory null
    executorCores null
    totalExecutorCores null
    propertiesFile null
    driverMemory null
    driverCores null
    driverExtraClassPath null
    driverExtraLibraryPath null
    driverExtraJavaOptions null
    supervise false
    queue null
    numExecutors null
    files null
    pyFiles null
    archives null
    mainClass org.apache.spark.examples.SparkPi
    primaryResource file:/Users/rafael.correa/workspace/github/olx-global/spark-on-k8s/tmp/spark/examples/jars/spark-examples_2.11-2.4.3.jar
    name org.apache.spark.examples.SparkPi
    childArgs []
    jars null
    packages null
    packagesExclusions null
    repositories null
    verbose true

    Spark properties used, including those specified through
    --conf and those from the properties file null:
    (spark.kubernetes.container.image,10.99.82.129:5000/spark)


    Main class:
    org.apache.spark.examples.SparkPi
    Arguments:

    Spark config:
    (spark.jars,file:/Users/rafael.correa/workspace/github/olx-global/spark-on-k8s/tmp/spark/examples/jars/spark-examples_2.11-2.4.3.jar)
    (spark.app.name,org.apache.spark.examples.SparkPi)
    (spark.kubernetes.container.image,10.99.82.129:5000/spark)
    (spark.submit.deployMode,client)
    (spark.master,k8s://https://192.168.99.108:8443)
    Classpath elements:
    file:/Users/rafael.correa/workspace/github/olx-global/spark-on-k8s/tmp/spark/examples/jars/spark-examples_2.11-2.4.3.jar