Skip to content

Instantly share code, notes, and snippets.

@em-winterschon

em-winterschon/ollama-python-api.pseudo.code

Created October 16, 2024 22:46
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save em-winterschon/ed56c31daa7eb2ce5f6b312f8fde7047 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save em-winterschon/ed56c31daa7eb2ce5f6b312f8fde7047 to your computer and use it in GitHub Desktop.
Download ZIP
pseudo-code function for ollama api caller
Raw
ollama-python-api.pseudo.code
@decorator(auth-validate connection via url+token+scope)
@decorator(request list of enabled models -> session::var)
@decorator(limit multi-put calls in parallel to config.item)
function manyModelMultiPUT:
input: longtext for prompt content
input: dict of models to exec request
iterates over model quantity count, generating an api call per each
response handler + collect responses
returns:
json nested dict
primary api call syntax, api total metrics
model name, prompt response, exec metrics
..
total timing values etc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment