Skip to content

Instantly share code, notes, and snippets.

@arun-gupta

arun-gupta/readme.md

Last active September 18, 2024 17:21
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save arun-gupta/5f02b5a57030ba8f975a4c328178ffe8 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save arun-gupta/5f02b5a57030ba8f975a4c328178ffe8 to your computer and use it in GitHub Desktop.
Download ZIP
More OPEA Examples using Docker Compose
Raw
readme.md

CodeTrans using OPEA on AWS and Docker Compose

  • Spin up Ubuntu 24.04 VM and install Docker following the instructions at https://gist.github.com/arun-gupta/7e9f080feff664fbab878b26d13d83d7
  • Replace HuggingFace API token and private IP address of the host below and copy the contents in a file named .env: 
    export host_ip="172.31.50.223"
export HUGGINGFACEHUB_API_TOKEN="Your_Huggingface_API_Token"
export LLM_MODEL_ID="meta-llama/CodeLlama-7b-hf"
export TGI_LLM_ENDPOINT="http://${host_ip}:8028"
export MEGA_SERVICE_HOST_IP=${host_ip}
export LLM_SERVICE_HOST_IP=${host_ip}
export BACKEND_SERVICE_ENDPOINT="http://${host_ip}:7778/v1/codegen"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment