Skip to content

Instantly share code, notes, and snippets.

@ernesthan
ernesthan / cli.py
Last active January 22, 2024 07:29
Kedro Clean command
#Reference: https://github.com/kedro-org/kedro/issues/635
from pathlib import Path
from kedro.framework.startup import _is_project, bootstrap_project
from kedro.framework.context import KedroContext
from kedro.framework.project import settings, find_pipelines
from kedro.framework.hooks import _create_hook_manager
from kedro.config import ConfigLoader
@project_group.command()
@ernesthan
ernesthan / hooks.py
Last active January 3, 2024 04:23
Hooks for kedro to print the hook arguments and observe hook behaviour.
import logging
from kedro.framework.hooks import hook_impl
## Hooks specs: https://docs.kedro.org/en/stable/kedro.framework.hooks.specs.html
## There are five types of hooks: (1) Kedro Context (2) DataCatalog (3) Pipeline (4) Node (5) Dataset
class PrintEverythingHooks:
_logger = None
def __init__(self, log_file_path='PrintEverything.log') -> None:
@ernesthan
ernesthan / zsh-keyboard-shortcuts.md
Created September 20, 2023 02:42 — forked from mkfares/zsh-keyboard-shortucts.md
Common zsh Keyboard Shortcuts on macOS Catalina

Common zsh Keyboard Shortcuts on macOS

Navigation

CTRL + A : Move the cursor to the beginning of the line
CTRL + E : Move the cursor to the end of the line
OPTION + Left Arrow : Move the cursor one word backward
OPTION + Right arrow : Move the cursor one word forward
Left Arrow : Move the cursor one character backward
Right Arrow : Move the cursor one character forward

@ernesthan
ernesthan / mac_pyodbc_issue.py
Created August 12, 2023 02:47
Apple Silicon has an issue with pyodbc, reinstalling with no-binary option will work
pip uninstall pyodbc -y
pip install --no-binary :all: pyodbc
python -c "import pyodbc"
@ernesthan
ernesthan / api_utils.py
Created July 12, 2023 14:35 — forked from Causb1A/api_utils.py
escape hatch
# Base Library Packages
import _thread
# Third Party Packages
from pynput import keyboard
from functools import wraps
import sys
def escape_hatch(
@ernesthan
ernesthan / tqdm.py
Created February 2, 2023 18:16
Customise TQDM bar
# https://github.com/tqdm/tqdm
# Search "bar_format"
from tqdm import tqdm
for i in tqdm(range(1_000_000_000), total=1_000_000_000, bar_format = "{desc}: {percentage:.2f}%|{bar}| {n:,}/{total:,} [{elapsed}<{remaining}"):
pass
@ernesthan
ernesthan / wordle_first_guess.py
Created January 13, 2022 08:11
Wordle First Guess
#### Best first guess for Wordle
# Wordle: https://www.powerlanguage.co.uk/wordle/
# Script by: @ernesthan
# Created on: 13th Jan 2022
#### If not already installed, run the following lines in interactive python
# import nltk
# nltk.download('wordnet')
# nltk.download('omw-1.4')
@ernesthan
ernesthan / BrewInstallPsql.sh
Created September 6, 2021 00:36
Standard Brew Install (eg: Postgres)
# Update and check Brew
brew update
brew doctor
# Install Postgres
brew install postgresql
# Start Postgres
brew services start postgresql
@ernesthan
ernesthan / load_multiple_csv.py
Created March 24, 2021 07:09
Load multiple CSV (small) into a single Pandas DF
import os
import pandas as pd
def load_file(filedir):
filenames = os.listdir(filedir)
filenames.sort()
filepaths = [os.path.join(filedir, filename) for filename in filenames if filename.endswith('.csv')]
combined_csv = pd.concat([pd.read_csv(path) for path in filepaths])
return combined_csv
@ernesthan
ernesthan / postgres-cheatsheet.md
Created December 17, 2020 07:49 — forked from Kartones/postgres-cheatsheet.md
PostgreSQL command line cheatsheet

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h or --help depending on your psql version):

  • -E: will describe the underlaying queries of the \ commands (cool for learning!)
  • -l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)