Skip to content

Instantly share code, notes, and snippets.

@richardxin
richardxin / AkkaStreamSparkIntegration.scala
Created December 2, 2017 01:14 — forked from lloydmeta/AkkaStreamSparkIntegration.scala
Example for how to connect Akka Stream and Spark Streaming by turning creating a Flow element that feeds into an InputDstream
import akka.actor._
import akka.stream.scaladsl.Flow
import org.apache.spark.streaming.dstream.ReceiverInputDStream
import org.apache.spark.streaming.receiver.ActorHelper
import akka.actor.{ ExtensionKey, Extension, ExtendedActorSystem }
import scala.reflect.ClassTag
object AkkaStreamSparkIntegration {
#!/bin/bash
GIT_REPO="https://github.com/richardxin/test"
if [ $# -ne 2 ]; then
echo "=== Missing arguments ===: \nUSAGE: deploy_note.sh <zeppelin_env> <note_name>"
exit 1
fi
# replace / with -
file_name="${2/\//-}.json"
#!/bin/bash
STATUS_URL="http://$1:8890/api/notebook/job/$2"
# curl -X GET http://ec2-34-215-13-164.us-west-2.compute.amazonaws.com:8890/api/notebook | jq '.body[] | select (.id == "2CX4Y3FAS") | .name'
while true
do
STATUS=`curl -X GET $STATUS_URL | jq -cr '.body[] | .status'`
echo $STATUS
#!/bin/bash
# example: sh run_note.sh ec2-34-215-13-164.us-west-2.compute.amazonaws.com 2CX4Y3FAS
# Usage: run_note.sh "<zeppelin_host>" "<note_id>"
# http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[noteId]
if [ $# -ne 2 ]; then
echo "=== Missing arguments ===: \nUSAGE: run_note.sh <zeppelin_host> <note_id>"
exit 1
fi
#!/bin/bash
# Usage: slackpost "<webhook_url>" "<channel>" "<username>" "<message>"
# also (echo $RANDOM; echo $RANDOM) |slackpost "<channel>" "<username>"
# https://gist.github.com/dopiaza/6449505
# ------------
webhook_url=$1
if [[ $webhook_url == "" ]]
then
@richardxin
richardxin / installing_cassandra.md
Created September 13, 2017 00:38 — forked from hkhamm/installing_cassandra.md
Installing Cassandra on Mac OS X

Installing Cassandra on Mac OS X

Install Homebrew

Homebrew is a great little package manager for OS X. If you haven't already, installing it is pretty easy:

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
@richardxin
richardxin / postgres-cheatsheet.md
Created June 19, 2017 18:47 — forked from Kartones/postgres-cheatsheet.md
PostgreSQL command line cheatsheet

PSQL

Magic words:

psql -U postgres

If run with -E flag, it will describe the underlaying queries of the \ commands (cool for learning!).

Most \d commands support additional param of __schema__.name__ and accept wildcards like *.*