Ryuukae’s gists

Ryuukae / makeollamamodelfile

Created January 28, 2024 21:26

Saved from https://unmesh.dev/post/ollama_custom_model/

	# Modelfile for creating an API security assistant
	# Run `ollama create api-secexpert -f ./Modelfile` and then `ollama run api-secexpert` and enter a topic

	FROM codellama
	PARAMETER temperature 1

	SYSTEM """
	You are a senior API developer expert, acting as an assistant.
	You offer help with API security topics such as: Secure Coding practices,
	API security, API endpoint security, OWASP API Top 10.

Ryuukae / ollamamakemodelfile.txt

Created January 28, 2024 16:53

Saved from https://unmesh.dev/post/ollama_custom_model/

	# Modelfile for creating an API security assistant
	# Run `ollama create api-secexpert -f ./Modelfile` and then `ollama run api-secexpert` and enter a topic

	FROM codellama
	PARAMETER temperature 1

	SYSTEM """
	You are a senior API developer expert, acting as an assistant.
	You offer help with API security topics such as: Secure Coding practices,
	API security, API endpoint security, OWASP API Top 10.

Ryuukae / Llama_cpp_python_Text_Completion.py

Created January 21, 2024 23:01

Saved from https://www.phind.com/search?cache=zhowsvc9xopklexus6xc6ww8

	from llama_cpp import Llama
	llm = Llama(model_path="C:/models/TheBloke/CodeLlama-7B-Instruct-GGUF/llama-2-7b-chat.Q5_K_M.gguf",
	n_gpu_layers=35,
	)
	output = llm(
	"Q: Name the planets in the solar system? A: ", # Prompt
	max_tokens=4096, # Generate up to 32 tokens
	stop=["Q:", "\n"], # Stop generating just before the model would generate a new question
	echo=True # Echo the prompt back in the output
	) # Generate a completion

Ryuukae / GPT4All_Inference.py

Created January 21, 2024 18:46

Saved from https://www.phind.com/search?cache=t1skkx5el2izp4ipl34apu6t

	from gpt4all import GPT4All

	# Instantiate the GPT4All model
	model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")

	# Use the model to generate text
	output = model.generate("The capital of France is ", max_tokens=3)

	# Print the generated text
	print(output)

Ryuukae / Langchain_GPT4ALL_Inference

Created January 21, 2024 18:43

	from langchain.llms import GPT4All
	from langchain import PromptTemplate, LLMChain

	# create a prompt template where it contains some initial instructions
	# here we say our LLM to think step by step and give the answer

	template = """
	Let's think step by step of the question: {question}
	Based on all the thought the final answer becomes:
	"""

Ryuukae Ryuukae