Zheyue Tan tanzeyy

🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., *.bin or *.safetensors).
🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.

	execute_cmd_until_success() {
	local CMD="$1"
	local MAX_RETRIES="${2:-3}"q
	local RETRY_COUNT=0

	while true; do
	eval $CMD
	if [ $? -eq 0 ]; then
	echo "Command executed successfully!"
	break

	# [ohmyzsh]
	# * https://ohmyz.sh/
	# * https://github.com/zsh-users/zsh-syntax-highlighting
	# * https://github.com/zsh-users/zsh-autosuggestions

	# sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

	git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
	git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions

	# Byte-compiled / optimized / DLL files
	__pycache__/
	*.py[cod]
	*$py.class

	# C extensions
	*.so

	# Distribution / packaging
	.Python