Skip to content

Instantly share code, notes, and snippets.

View linncy's full-sized avatar

Changyuan Lin linncy

19 followers · 11 following
View GitHub Profile
@linncy
linncy / gist:8ed3c176f34bd954138b9e18d3c5592e
Created May 6, 2022 21:20 — forked from corck/gist:d4c63e3908548963ce2e58b45d9976e2
Restart Networkmanager on connection reset, restart VPN after network manager restart
For monitoring Network Manager connection and restart it on a dropped connection (eth/wifi). Make sure to specify the right interface below (eth0, wlan0...)
#/etc/init/reconnect.conf
start on started network-manager
stop on runlevel [016]
script
while true; do
if ifconfig eth0 | grep -q "inet addr:"; then
# echo "all ok!"
@linncy
linncy / yarn_applications_kill.sh
Created March 19, 2022 17:55 — forked from y2k-shubham/yarn_applications_kill.sh
Kill all running YARN applications
`yarn application --list | awk '{print $1}' | grep application | xargs yarn application -kill`
@linncy
linncy / ** How to Implement Movie Recommendation System-P5-Spark **.md
Created February 18, 2022 04:38 — forked from chliwei199/** How to Implement Movie Recommendation System-P5-Spark **.md

** How to Implement Movie Recommendation System-P5-Spark **

What is Spark?

Apache Spark is an open-source, distributed processing system used for big data workloads. It utilizes in-memory caching and optimized query execution for fast queries against data of any size. Simply put, Spark is a fast and general engine for large-scale data processing.

  • Resilient Distributed Dataset (RDD): RDD was the primary user-facing API in Spark since its inception. At the core, an RDD is an immutable distributed collection of elements of your data, partitioned across nodes in your cluster that can be operated in parallel with a low-level API that offers transformations and actions.

Code example:

A. Spark docker-compose

@linncy
linncy / gist:8705ab5277082230a4eca8412eea7d39
Last active December 6, 2020 22:30 — forked from ryogesh/gist:37ee2273c5fe0d38d8128feb78d7812a
Ubuntu 18.x: jupyterhub init.d script
#!/bin/bash
### BEGIN INIT INFO
# Provides: jupyterhub
# Required-Start: $local_fs $syslog
# Required-Stop: $local_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Jupyterhub service
# Description: Place the script as /etc/init.d/jupyterhub
#