Skip to content

Instantly share code, notes, and snippets.

View bartekdobija's full-sized avatar
:octocat:
Simplicity is the ultimate sophistication.

Bartek Dobija bartekdobija

:octocat:
Simplicity is the ultimate sophistication.
6 followers · 0 following
  • Earth
View GitHub Profile
@bartekdobija
bartekdobija / sketch-never-ending.md
Created June 2, 2018 17:07 — forked from Bhavdip/sketch-never-ending.md
Modify Sketch to never ending trial

###Sketch trial non stop

Open hosts files:

$ open /private/etc/hosts

Edit the file adding:

127.0.0.1 backend.bohemiancoding.com

127.0.0.1 bohemiancoding.sketch.analytics.s3-website-us-east-1.amazonaws.com

@bartekdobija
bartekdobija / gist:501ec7afd9bd2bb2f68ab853f1fbd3b4
Created May 11, 2017 11:51
flink source compilation
MAVEN_OPTS="-Xmx5g -XX:MaxPermSize=1024m" /usr/local/apache-maven-3.0.4/bin/mvn clean install -DskipTests -Pvendor-repos -Dhadoop.profile=2 -Dhadoop.version=2.6.0-cdh5.10.0 -Dhbase.version=1.2.0-cdh5.10.0
@bartekdobija
bartekdobija / Vagrantfile-SiLK
Last active November 6, 2015 17:25
# Basic OS configuration
$sysconfig = <<SCRIPT
# disable IPv6
echo "net.ipv6.conf.all.disable_ipv6=1" > /etc/sysctl.conf && sysctl -f /etc/sysctl.conf
# this should be a persistent config
ulimit -n 65536
ulimit -s 10240
@bartekdobija
bartekdobija / Vagrantfile-kafka
Last active October 22, 2015 15:05
Kafka Node Vagrantfile
# Basic OS configuration
$sysconfig = <<SCRIPT
# disable IPv6
echo "net.ipv6.conf.all.disable_ipv6=1" > /etc/sysctl.conf && sysctl -f /etc/sysctl.conf
# this should be a persistent config
ulimit -n 65536
ulimit -s 10240
@bartekdobija
bartekdobija / Vagrantfile
Last active October 30, 2015 14:52
Cloudera Vagrantfile
# Anaconda dependencies
$anaconda_deps = <<SCRIPT
ANACONDA_INSTALLER=https://3230d63b5fc54e62148e-c95ac804525aac4b6dba79b00b39d1d3.ssl.cf1.rackcdn.com/Anaconda-2.3.0-Linux-x86_64.sh
if [ ! -d "/usr/local/anaconda" ]; then
echo "Anaconda installation..." \
&& echo "downloading binaries" \
&& wget ${ANACONDA_INSTALLER} -q -P /tmp/ \
&& echo "running installer" \
@bartekdobija
bartekdobija / jupyterhub
Last active February 23, 2019 07:15
JupiterHub start script
# JupyterHub start / stop / restart script
#
# description: JupyterHub start / stop / restart script
# chkconfig: 2345 20 80
# processname: jupyterhub
# pidfile: /var/run/jupyterhub.pid
JUPYTERHUB_HOME=/usr/local/bin
JUPYTERHUB_CONFIG=/etc/ipython/jupyterhub_config.py
JUPYTERHUB_LOG=/var/log/jupyterhub.log
@bartekdobija
bartekdobija / spark-without-hadoop.sh
Last active August 31, 2015 13:33
#!/usr/bin/env bash
# In this case I have a Hadoop distro compiled from source:
# MAVEN_OPTS="-Xms512m -Xmx1024m" mvn package -Pdist,native -DskipTests -Dtar
# verified with:
# hadoop checknative -a
# with output:
# Native library checking:
# hadoop: true /usr/local/hadoop-2.6.0/lib/native/libhadoop.so.1.0.0
# zlib: true /lib64/libz.so.1
@bartekdobija
bartekdobija / hadoop_installer.sh
Last active August 29, 2015 14:08
Fast Hadoop Installer
#!/usr/bin/env bash
# This script will download, unpack and register Hadoop 2.5.1 in your home directory.
# No RPMs, no integration with the OS.
# The installer is created to automate and simplify the process of deployment of the local development Hadoop version.
# It is ideal for developers who begin their adventure with the cluster or for all you lazy programmers that make 90% of the community;)
# The simplest way to execute this script is to:
# bash -c "$(curl -fsSL https://goo.gl/x2VTbz)"
tmp_hadoop_location=/tmp/hadoop-2.5.1.tar.gz
@bartekdobija
bartekdobija / batch_tail.sh
Last active August 29, 2015 14:08
Batch tail data files
#!/usr/bin/env bash
function process() {
files=$(ls *.$1)
for file in $files; do
tail -n $2 $file > "$output_folder/$file"
done
}
function show_help() {
@bartekdobija
bartekdobija / log4j.properties
Created November 2, 2014 19:53
Apache Pig - development configuration
# ***** Set root logger level to DEBUG and its only appender to A.
log4j.logger.org.apache.pig=WARN, A
log4j.logger.org.apache.hadoop=ERROR, A
# ***** A is set to be a ConsoleAppender.
log4j.appender.A=org.apache.log4j.ConsoleAppender
# ***** A uses PatternLayout.
log4j.appender.A.layout=org.apache.log4j.PatternLayout
log4j.appender.A.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n