freeman alexguo

4 Steps in Running LLaMA-7B on a M1 MacBook

The large language models usability

The problem with large language models is that you can’t run these locally on your laptop. Thanks to Georgi Gerganov and his llama.cpp project, it is now possible to run Meta’s LLaMA on a single computer without a dedicated GPU.

Running LLaMA

There are multiple steps involved in running LLaMA locally on a M1 Mac after downloading the model weights.

install emscripten

$ brew install emscripten

and setup by following the instruction after the installation.

get the location of emscripten header files

Build libcurl for android NDK

Libcurl requires openssl and zlib to be fully operationnal

Step 1 : cross compile zlib
Step 2 : cross compile openssl
Step 3 : cross compile curl with zlib/openssl external link

Prerequisites :

	<artifacts_info>
	The assistant can create and reference artifacts during conversations. Artifacts are for substantial, self-contained content that users might modify or reuse, displayed in a separate UI window for clarity.

	# Good artifacts are...
	- Substantial content (>15 lines)
	- Content that the user is likely to modify, iterate on, or take ownership of
	- Self-contained, complex content that can be understood on its own, without context from the conversation
	- Content intended for eventual use outside the conversation (e.g., reports, emails, presentations)
	- Content likely to be referenced or reused multiple times

	from __future__ import absolute_import, division, print_function

	import argparse
	import glob
	import logging
	import os
	import random

	import numpy as np
	import torch

	import SwiftUI
	import Combine

	class MyDatabase: BindableObject {
	let didChange = PassthroughSubject<MyDatabase, Never>()

	var contacts: [Contact] = [
	Contact(id: 1, name: "Anna"), Contact(id: 2, name: "Beto"),
	Contact(id: 3, name: "Jack"), Contact(id: 4, name: "Sam")
	] {

	#!/bin/sh

	# cross & static compile shadowsocks-libev

	PCRE_VER=8.41
	PCRE_FILE="http://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-$PCRE_VER.tar.gz"

	MBEDTLS_VER=2.6.0
	MBEDTLS_FILE="https://tls.mbed.org/download/mbedtls-$MBEDTLS_VER-gpl.tgz"

	<?xml version="1.0" encoding="utf-8"?>
	<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

	<plist version="1.0">
	<dict>
	<key>KeepAlive</key>
	<dict>
	<key>NetworkState</key>
	<true/>
	</dict>