Skip to content

Instantly share code, notes, and snippets.

View darrenangle's full-sized avatar
🏋️

darren angle darrenangle

🏋️
32 followers · 119 following
View GitHub Profile
@TimDettmers
TimDettmers / gist:a96b0d948a97583f8ab0599fa888c35c
Created August 10, 2023 04:57
Hyperparameter grid search for LLaMA models on Alpaca dataset for QLoRA finetuning
This table contains data from multiple software versions. Some hyperparamter names are "NaN" meaning, they did not exist in that software version. The best 7B result is 40.08 MMLU.
================================================================================
Config: learning_rate: 0.005, adam_beta2: 0.999, lora_dropout: 0.0 , max_grad_norm: 1.0 , max_steps: 7320, lr_scheduler_type: <SchedulerType.COSINE: cosine>, weight_decay: 0.0 , base_model: /gscratch/zlab/llama/7B, quant_type: nf4 , gradient_accumulation_steps: 6 , per_device_train_batch_size: 2
acc mean (SE): 0.2290 (nan). 95% CI (nan, nan). Sample size: 1
================================================================================
Config: learning_rate: 0.0002, adam_beta2: 0.999, lora_dropout: 0.0 , max_grad_norm: 0.3 , max_steps: 9750, lr_scheduler_type: <SchedulerType.CONSTANT: constant>, weight_decay: 0.0 , base_model: NaN , quant_type: nf4 , gradient_accumulation_steps: 2 , per_device_train_batch_size: 8
acc mean (SE): 0.2290 (0.0
@lmcneel
lmcneel / remove-node-modules.md
Last active July 31, 2025 14:34
How to remove node_modules after they have been added to a repo

How to remove node_modules

Create a .gitignore file

  1. Check for an existing .gitignore file in the project directory
ls -a
@ivanistheone
ivanistheone / .cleanup.sh
Last active January 30, 2025 13:54
A shell script I use to (mostly) automatically cleanup my ~/Desktop
#!/bin/bash
MYHOME=$HOME
MYDESKTOP=$HOME/Desktop
echo " _ "
echo " | | "
echo " ___ | | ___ __ _ _ __ _ _ _ __ "
echo " / __|| |/ _ \/ _ | '_ \| | | | '_ \ "
echo " | (__ | | __/ (_| | | | | |_| | |_) |"
echo " \___||_|\___|\__,_|_| |_|\__,_| .__/ "