Created
October 21, 2025 12:04
-
-
Save do-me/374c741d1ecd1d84fdcac3473a50b25e to your computer and use it in GitHub Desktop.
Benchmark for Mac M3 Max 128Gb and mlx-community/gemma-3-270m-it-4bit with mlx-lm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from mlx_lm import batch_generate, load | |
| model, tokenizer = load("mlx-community/gemma-3-270m-it-4bit") | |
| # load a pandas df here, df has a text column | |
| import pandas as pd | |
| df = pd.read_parquet("2000_benchmark_texts_BAAI.parquet") | |
| # Apply the chat template and encode to tokens | |
| prompts = [i + "--------\nSummarize this article in one sentence" for i in df.text.to_list()] | |
| prompts = [ | |
| tokenizer.apply_chat_template( | |
| [{"role": "user", "content": p}], | |
| add_generation_prompt=True, | |
| ) | |
| for p in prompts[:2000] | |
| ] | |
| result = batch_generate(model, tokenizer, prompts, verbose=False) |
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Results
1 mlx-community/gemma-3-270m-it-4bit
The more the better! Only marginal gains after 2000 if at all.
Graph
Just copy & paste this code in Jupyter Lite
Graph Code
Test Data
2000 newspaper full texts from BAAI's high quality news dataset: https://huggingface.co/datasets/BAAI/IndustryCorpus2
Sorry the file upload here in this gist seems to be broken; I cannot upload any kind of file (tried csv, csv.gz, 7z, zip, parquet...). Please just reach out to me.