Shaurya Agarwal shauryashaurya

Step 1: Basic PySpark Qualifier

Part 1: Multiple Choice Assessment (Basic PySpark Concepts)

Here are five multiple-choice questions to test fundamental PySpark knowledge:

Which of the following is NOT a valid Spark transformation?
- a) map
- b) filter

c) reduceByKey

58 bytes of CSS to look great nearly everywhere

When making this website, i wanted a simple, reasonable way to make it look good on most displays. Not counting any minimization techniques, the following 58 bytes worked well for me:

main {
  max-width: 38rem;
  padding: 2rem;
  margin: auto;
}

Hello  (<-- two spaces)
World

Hello
World

	<!doctype html>
	<html>
	<head>
	<title></title>
	<style>
	body {
	background: white;
	text-align: center;
	padding: 20px;
	font-family: Georgia, serif;

	from fabric.api import local
	import pip

	def freeze ():
	local("pip freeze > requirements.txt")
	local("git add requirements.txt")
	local("git commit -v")

	def upgrade ():
	for dist in pip.get_installed_distributions():

	/**
	* Find the longest class names in Spring.
	* Also find FactoryFactory classes.
	* a goof-off project by @thom_nic
	*/

	import java.util.jar.*

	defaultTasks 'longest', 'factoryfactory'

	# minimum number of moves that a knight needs to go from source to destination.

	# given that the chessboard looks like this:
	# -------------------------
	# \| 0\| 1\| 2\| 3\| 4\| 5\| 6\| 7\|
	# -------------------------
	# \| 8\| 9\|10\|11\|12\|13\|14\|15\|
	# -------------------------
	# \|16\|17\|18\|19\|20\|21\|22\|23\|
	# -------------------------

	body { counter-reset: h1counter h2counter h3counter h4counter h5counter h6counter; }

	h1 { counter-reset: h2counter; }
	h2 { counter-reset: h3counter; }
	h3 { counter-reset: h4counter; }
	h4 { counter-reset: h5counter; }
	h5 { counter-reset: h6counter; }
	h6 {}

	h2:before {

	# A simple cheat sheet of Spark Dataframe syntax
	# Current for Spark 1.6.1

	# import statements
	from pyspark.sql import SQLContext
	from pyspark.sql.types import *
	from pyspark.sql.functions import *

	#creating dataframes
	df = sqlContext.createDataFrame([(1, 4), (2, 5), (3, 6)], ["A", "B"]) # from manual data

	<html>
	<head><script src="BarnsleyFern.js"></script></head>
	<body onload="drawBarnsleyFern('canvas', 10000000,200,0.39)">
	<br /> <h3>// Exploring Barnsley Fern Fractals</h3>
	<h4>// shauryashaurya</h4>
	<canvas id="canvas" width="4080" height="4080" style="border: 0px inset;"></canvas>
	<h4>ref: https://mathworld.wolfram.com/BarnsleysFern.html</h4>
	</body>
	</html>