Skip to content

Instantly share code, notes, and snippets.

@ShawonAshraf

ShawonAshraf/main.py

Created April 11, 2024 20:11
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save ShawonAshraf/628640f1b39b427e354476f650735d97 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save ShawonAshraf/628640f1b39b427e354476f650735d97 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. Shawon Ashraf created this gist Apr 11, 2024.
    32 changes: 32 additions & 0 deletions main.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,32 @@
    import openai
    from pydantic import BaseModel, Field
    import json

    class ResponseStructure(BaseModel):
    bookname: str = Field(..., title="Book Name")
    author: str = Field(..., title="Author")
    reading_list_name: str = Field(..., title="Reading List Name")


    client = openai.Client(
    api_key="ollama",
    base_url="http://gandalf:8000/v1"
    )


    messages = [
    {"role": "system", "content": "You are a helpful assistant designed to output JSON."},
    {"role": "user", "content": "Add the book '1984' by 'George Orwell' to the 'Currently Reading' list."},
    ]

    chat_response = client.chat.completions.create(
    model="solidrust/Hermes-2-Pro-Mistral-7B-AWQ",
    messages=messages,# type: ignore
    response_format={"type": "json_object", "schema": ResponseStructure.model_json_schema()}, # type: ignore
    temperature=0.0,
    )

    # json_obj = json.loads(chat_response.choices[0].message.content) # type: ignore
    # print(json_obj)

    print(chat_response.choices[0].message.content)