Skip to content

Instantly share code, notes, and snippets.

@shawwn

shawwn/example.sh

Created March 6, 2023 05:17
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save shawwn/65a2ae80c8c0369a8cb8ff24d88d4f80 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save shawwn/65a2ae80c8c0369a8cb8ff24d88d4f80 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. shawwn created this gist Mar 6, 2023.
    7 changes: 7 additions & 0 deletions example.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,7 @@
    mp=1; size=7B; # to run 7B
    mp=8; size=65B; # to run 65B
    for seed in $(randint 1000000)
    do
    export TARGET_FOLDER=~/ml/data/llama/LLaMA
    time python3 -m torch.distributed.run --nproc_per_node $mp example.py --ckpt_dir $TARGET_FOLDER/$size --tokenizer_path $TARGET_FOLDER/tokenizer.model --seed $seed --max_seq_len 2048 --max_gen_len 2048 --count 0 | tee -a ${size}_startrek.txt
    done