Skip to content

Instantly share code, notes, and snippets.

@petebytes

petebytes/increase_model_context.sh

Last active March 11, 2025 22:26
Show Gist options
  • Save petebytes/acc35506cc296f6476811a944a9b2f20 to your computer and use it in GitHub Desktop.
Save petebytes/acc35506cc296f6476811a944a9b2f20 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. petebytes revised this gist Mar 11, 2025. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions increase_model_context.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,7 @@
    #!/bin/bash

    # developed and tested on a macbook
    # original instructions discovered at <https://www.youtube.com/@technovangelist>

    # Check if correct number of arguments provided
    if [ $# -ne 2 ]; then
  2. petebytes created this gist Mar 11, 2025.
    34 changes: 34 additions & 0 deletions increase_model_context.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,34 @@
    #!/bin/bash

    # developed and tested on a macbook

    # Check if correct number of arguments provided
    if [ $# -ne 2 ]; then
    echo "Usage: $0 <model_id> <context_size>"
    echo "Example: $0 llama2:latest 8192"
    exit 1
    fi

    MODEL_ID=$1
    CONTEXT_SIZE=$2

    # Create custom models directory if it doesn't exist
    MODELS_DIR="$HOME/ollama_custom_models"
    mkdir -p "$MODELS_DIR"

    # Create a clean model name by replacing special characters
    CLEAN_MODEL_ID=$(echo "$MODEL_ID" | tr ':' '-')
    NEW_MODEL_NAME="${CLEAN_MODEL_ID}-ctx${CONTEXT_SIZE}"
    MODELFILE="$MODELS_DIR/Modelfile-${NEW_MODEL_NAME}"

    # Create Modelfile with increased context size
    echo "FROM $MODEL_ID" > "$MODELFILE"
    echo "PARAMETER num_ctx $CONTEXT_SIZE" >> "$MODELFILE"

    echo "Creating new model $NEW_MODEL_NAME with context size $CONTEXT_SIZE..."
    ollama create "$NEW_MODEL_NAME" -f "$MODELFILE"

    echo "Showing model details..."
    ollama show "$NEW_MODEL_NAME"

    echo "Done! You can now use the model with: ollama run $NEW_MODEL_NAME"