Ed Lee edlee123

Quick Guide: Running FLUX Schnell Model on Habana

Step 1: Pull and run the Habana PyTorch Docker image with the necessary configurations

docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all --ipc=host --cap-add=sys_nice --ulimit memlock=-1:-1 --security-opt seccomp=unconfined \
-v /home/ubuntu/workspace:/workspace \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vault.habana.ai/gaudi-docker/1.18.0/ubuntu24.04/habanalabs/pytorch-installer-2.4.0:latest /bin/bash

	# 1. Install Ansible
	# sudo apt update
	# sudo apt install software-properties-common
	# sudo add-apt-repository --yes --update ppa:ansible/ansible
	# sudo apt install ansible -y

	# 2.
	# sudo ansible-playbook docker_install.yml

	---

	from flask import Flask, request, jsonify
	import requests
	import random

	app = Flask(__name__)

	endpoints = {
	'model_provider/model_1': 'http://0.0.0.0:8001',
	'model_provider/model_2': 'http://0.0.0.0:8002',
	'model_provider/model_3': 'http://0.0.0.0:8003',