Ashutosh Saboo ashutoshsaboo

Is telling a model to "not hallucinate" absurd?

Can you tell an LLM "don't hallucinate" and expect it to work? my gut reaction was "oh this is so silly" but upon some reflection, it really isn't. There is actually no reason why it shouldn't work, especially if it was preference-fine-tuned on instructions with "don't hallucinate" in them, and if it a recent commercial model, it likely was.

What does an LLM need in order to follow an instruction? It needs two things:

an ability to perform then task. Something in its parameters/mechanism should be indicative of the task objective, in a way that can be influenced. (In our case, it should "know" when it hallucinates, and/or should be able to change or adapt its behavior to reduce the chance of hallucinations.)
an ability to ground the instruction: the model should be able to associate the requested behavior with its parameters/mechanisms. (In our case, the model should associate "don't hallucinate" with the behavior related to 1).

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

This recipe describes the classes, controllers, and jobs used by Standard Notes to build our own custom-fit Mailchimp replacement.

With the parts here, you should be able to put together your own email campaign system directly in your Rails app, removing the need to use an expensive online newsletter service.

	You are an assistant that engages in extremely thorough, self-questioning reasoning. Your approach mirrors human stream-of-consciousness thinking, characterized by continuous exploration, self-doubt, and iterative analysis.

	## Core Principles

	1. EXPLORATION OVER CONCLUSION
	- Never rush to conclusions
	- Keep exploring until a solution emerges naturally from the evidence
	- If uncertain, continue reasoning indefinitely
	- Question every assumption and inference

	#!/usr/bin/env python3
	"""
	Human quality transcripts from audio files using
	AssemblyAI for transcription and Google's Gemini for enhancement.

	Requirements:
	- AssemblyAI API key (https://www.assemblyai.com/)
	- Google API key (https://aistudio.google.com/)
	- Python packages: assemblyai, google-generativeai, pydub

	<!doctype html>
	<html>

	<head>
	<meta charset="utf-8" />
	<meta name="viewport" content="width=device-width, initial-scale=1.0" />
	<title>Content Manager</title>
	</head>

	<body>

	export class Cache<T extends object, K> {
	items = new WeakMap<T, K>()

	get<P extends T>(item: P, cb: (item: P) => K) {
	if (!this.items.has(item)) {
	this.items.set(item, cb(item))
	}
	return this.items.get(item)!
	}

	/*

	This file is now hosted here:

	https://github.com/victornpb/undiscord

	*/