Skip to content

Instantly share code, notes, and snippets.

@jhwohlgemuth

jhwohlgemuth/llama2-mac-gpu.sh

Forked from adrienbrault/llama2-mac-gpu.sh
Created July 22, 2023 22:54
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jhwohlgemuth/54eabe2440756e03baa10e6a9aba3d06 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jhwohlgemuth/54eabe2440756e03baa10e6a9aba3d06 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @adrienbrault adrienbrault revised this gist Jul 20, 2023. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion llama2-mac-gpu.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -21,4 +21,4 @@ echo "Prompt: " \
    --temp 0.7 \
    --repeat_penalty 1.1 \
    -n -1 \
    -p "### Instruction: ${PROMPT} \n### Response:"
    -p "[INST] ${PROMPT} [/INST]"
  2. @adrienbrault adrienbrault revised this gist Jul 19, 2023. No changes.
  3. @adrienbrault adrienbrault revised this gist Jul 19, 2023. No changes.
  4. @adrienbrault adrienbrault created this gist Jul 19, 2023.
    24 changes: 24 additions & 0 deletions llama2-mac-gpu.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,24 @@
    # Clone llama.cpp
    git clone https://github.com/ggerganov/llama.cpp.git
    cd llama.cpp

    # Build it
    LLAMA_METAL=1 make

    # Download model
    export MODEL=llama-2-13b-chat.ggmlv3.q4_0.bin
    wget "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/${MODEL}"

    # Run
    echo "Prompt: " \
    && read PROMPT \
    && ./main \
    -t 8 \
    -ngl 1 \
    -m ${MODEL} \
    --color \
    -c 2048 \
    --temp 0.7 \
    --repeat_penalty 1.1 \
    -n -1 \
    -p "### Instruction: ${PROMPT} \n### Response:"