Skip to content

Instantly share code, notes, and snippets.

@awerlang

awerlang/README

Last active June 24, 2024 15:50
Show Gist options
  • Save awerlang/b792a3f908206a90ad58ba559c5400bb to your computer and use it in GitHub Desktop.
Save awerlang/b792a3f908206a90ad58ba559c5400bb to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. awerlang revised this gist Feb 22, 2024. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions README
    View file
    Original file line number Diff line number Diff line change
    @@ -31,6 +31,6 @@ Refer to instructions:
    Known issues
    ------------

    * Visual Studio Code repository follows a different directory structure which which is incompatible with this tool
    * Only supports HTTP/HTTPS/FTP protocols for repositories
    * Cannot be passed an individual .rpm URL directly (use aria2c instead)
    * Cannot be passed an individual .rpm URL directly (use aria2c instead)
    * Minor: Some third-party repositories (e.g. Visual Studio Code) follow a different directory structure which results in those packages being skipped
  2. awerlang revised this gist Jul 3, 2022. 1 changed file with 4 additions and 5 deletions.
    9 changes: 4 additions & 5 deletions zypper-download
    View file
    Original file line number Diff line number Diff line change
    @@ -28,7 +28,7 @@
    ## RUN_ZYPPER = no
    ##
    ### LICENSE
    ## zypper-download v0.3.2
    ## zypper-download v0.3.3
    ##
    ## Copyright (C) 2020-2022 André Werlang
    ##
    @@ -120,7 +120,7 @@ package_dir() {
    run() {
    call_zypper "${SUB_COMMAND[@]}" \
    | create_download_spec \
    | tee -a ./zypper-download.log \
    | tee ./zypper-download.log \
    | download_all

    check_rpm || { error "Package signature failed"; exit 1; }
    @@ -186,16 +186,15 @@ create_download_spec() {
    arch=$1; getline;
    repo=trim($0); getline;
    host=uri[repo];
    suffix=index(host, "//download.opensuse.org") != 0 ? ".meta4" : "";
    print host "/" arch "/" name "-" version "." arch ".rpm" suffix;
    print host "/" arch "/" name "-" version "." arch ".rpm";
    print " dir=" PKG_CACHE_DIR "/" alias[repo] "/" arch;
    }
    ' -
    }

    download_all() {
    aria2c --input-file=- \
    --max-connection-per-server=10 \
    --max-connection-per-server=6 \
    --split=16 \
    --min-split-size=1M \
    --check-integrity=true \
  3. awerlang revised this gist Jun 23, 2022. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions zypper-download
    View file
    Original file line number Diff line number Diff line change
    @@ -28,7 +28,7 @@
    ## RUN_ZYPPER = no
    ##
    ### LICENSE
    ## zypper-download v0.3.1
    ## zypper-download v0.3.2
    ##
    ## Copyright (C) 2020-2022 André Werlang
    ##
    @@ -154,7 +154,7 @@ create_download_spec() {
    BEGIN {
    print "BEGIN {";
    }
    /^[0-9]/ {
    /:\/\// {
    alias=trim($2);
    name=trim($3);
    uri=trim($NF);
  4. awerlang revised this gist Jun 10, 2022. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions README
    View file
    Original file line number Diff line number Diff line change
    @@ -32,3 +32,5 @@ Known issues
    ------------

    * Visual Studio Code repository follows a different directory structure which which is incompatible with this tool
    * Only supports HTTP/HTTPS/FTP protocols for repositories
    * Cannot be passed an individual .rpm URL directly (use aria2c instead)
  5. awerlang revised this gist Jan 15, 2022. 2 changed files with 4 additions and 3 deletions.
    2 changes: 1 addition & 1 deletion aria2.conf
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    check-integrity=true
    console-log-level=warn
    console-log-level=error
    file-allocation=falloc
    max-concurrent-downloads=40
    5 changes: 3 additions & 2 deletions zypper-download
    View file
    Original file line number Diff line number Diff line change
    @@ -28,9 +28,9 @@
    ## RUN_ZYPPER = no
    ##
    ### LICENSE
    ## zypper-download v0.3
    ## zypper-download v0.3.1
    ##
    ## Copyright (C) 2020-2021 André Werlang
    ## Copyright (C) 2020-2022 André Werlang
    ##
    ## License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
    ## This is free software: you are free to change and redistribute it.
    @@ -120,6 +120,7 @@ package_dir() {
    run() {
    call_zypper "${SUB_COMMAND[@]}" \
    | create_download_spec \
    | tee -a ./zypper-download.log \
    | download_all

    check_rpm || { error "Package signature failed"; exit 1; }
  6. awerlang revised this gist Jan 15, 2022. 1 changed file with 11 additions and 0 deletions.
    11 changes: 11 additions & 0 deletions README
    View file
    Original file line number Diff line number Diff line change
    @@ -15,9 +15,20 @@ Make sure the program is executable:

    chmod +x $HOME/bin/zypper-download

    This tool leverages aria2 to be awesome, so make sure it is installed (use zypper one last time):

    zypper in aria2

    Usage
    -----

    It works best with repositories set to an official download redirector e.g. http://download.opensuse.org, since it relies on generated metalinks. So update your repositories URLs as needed.

    Refer to instructions:

    $ zypper-download -h

    Known issues
    ------------

    * Visual Studio Code repository follows a different directory structure which which is incompatible with this tool
  7. awerlang revised this gist Jan 15, 2022. 1 changed file with 23 additions and 0 deletions.
    23 changes: 23 additions & 0 deletions README
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,23 @@
    zypper-download
    ===============

    Downloads packages using any amount of available openSUSE mirrors.

    Installation
    ------------

    Copy both files to the following locations:

    $HOME/.config/aria2/aria2.conf
    $HOME/bin/zypper-download

    Make sure the program is executable:

    chmod +x $HOME/bin/zypper-download

    Usage
    -----

    Refer to instructions:

    $ zypper-download -h
  8. awerlang renamed this gist Jan 15, 2022. 1 changed file with 0 additions and 0 deletions.
    0 zypper-download.sh → zypper-download
    View file
    File renamed without changes.
  9. awerlang revised this gist Jan 15, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -28,7 +28,7 @@
    ## RUN_ZYPPER = no
    ##
    ### LICENSE
    ## zypper-download v0.2
    ## zypper-download v0.3
    ##
    ## Copyright (C) 2020-2021 André Werlang
    ##
  10. awerlang revised this gist Jan 15, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -30,7 +30,7 @@
    ### LICENSE
    ## zypper-download v0.2
    ##
    ## Copyright (C) 2020 André Werlang
    ## Copyright (C) 2020-2021 André Werlang
    ##
    ## License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
    ## This is free software: you are free to change and redistribute it.
  11. awerlang revised this gist Dec 4, 2021. 1 changed file with 6 additions and 3 deletions.
    9 changes: 6 additions & 3 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -40,6 +40,7 @@

    set -o errexit -o nounset -o noclobber -o pipefail
    shopt -s nullglob
    export LC_ALL=C

    main() {
    parse_opts "$@"
    @@ -139,7 +140,7 @@ call_zypper() {
    --non-interactive \
    "$command" --details \
    --dry-run \
    "${command_args[@]}" || true
    "${command_args[@]}"
    }

    create_download_spec() {
    @@ -174,7 +175,7 @@ create_download_spec() {
    pkg=0;
    next;
    }
    / packages? (is|are) going to (be installed|be upgraded|be downgraded|change architecture):$/ {
    / packages? (is|are) going to (be installed|be reinstalled|be upgraded|be downgraded|change architecture):$/ {
    pkg=1;
    next;
    }
    @@ -183,7 +184,9 @@ create_download_spec() {
    version=$NF; getline;
    arch=$1; getline;
    repo=trim($0); getline;
    print uri[repo] "/" arch "/" name "-" version "." arch ".rpm";
    host=uri[repo];
    suffix=index(host, "//download.opensuse.org") != 0 ? ".meta4" : "";
    print host "/" arch "/" name "-" version "." arch ".rpm" suffix;
    print " dir=" PKG_CACHE_DIR "/" alias[repo] "/" arch;
    }
    ' -
  12. awerlang revised this gist Aug 31, 2020. No changes.
  13. awerlang revised this gist Aug 31, 2020. 1 changed file with 12 additions and 9 deletions.
    21 changes: 12 additions & 9 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -5,7 +5,7 @@
    ## zypper-download - Downloads packages using any amount of available openSUSE mirrors.
    ##
    ### SYNOPSIS
    ## Usage: zypper-download.sh [options] command [command-args]...
    ## Usage: zypper-download [options] command [command-args]...
    ##
    ## Options:
    ##
    @@ -81,7 +81,8 @@ read_default_config() {
    }

    set_default_config() {
    local vars=$(read_default_config)
    local vars
    vars=$(read_default_config)
    eval "$vars"
    }

    @@ -123,7 +124,8 @@ run() {
    check_rpm || { error "Package signature failed"; exit 1; }

    if [[ "$RUN_ZYPPER" == yes ]]; then
    local dir=$(package_dir)
    local dir
    dir=$(package_dir)
    sudo zypper --pkg-cache-dir "$dir" "${SUB_COMMAND[@]}"
    fi
    }
    @@ -137,11 +139,12 @@ call_zypper() {
    --non-interactive \
    "$command" --details \
    --dry-run \
    "${command_args[@]}"
    "${command_args[@]}" || true
    }

    create_download_spec() {
    local repo_arrays=$(zypper repos --uri | awk -F "|" '
    local repo_arrays
    repo_arrays=$(zypper repos --uri | awk -F "|" '
    function trim(s) {
    gsub(/^ +| +$/, "", s);
    return s;
    @@ -171,7 +174,7 @@ create_download_spec() {
    pkg=0;
    next;
    }
    / packages? (is|are) going to be (installed|upgraded):$/ {
    / packages? (is|are) going to (be installed|be upgraded|be downgraded|change architecture):$/ {
    pkg=1;
    next;
    }
    @@ -198,11 +201,11 @@ download_all() {
    }

    check_rpm() {
    local dir=$(package_dir)
    local output
    local dir output
    dir=$(package_dir)
    output=$(find "$dir" -type f -name "*.rpm" -execdir rpm --checksig "{}" +)
    local result=$?
    if (( $result != 0 )); then
    if (( result != 0 )); then
    grep " NOT OK$" <<< "$output"
    fi
    return $result
  14. awerlang revised this gist Apr 10, 2020. 1 changed file with 26 additions and 20 deletions.
    46 changes: 26 additions & 20 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -41,18 +41,18 @@
    set -o errexit -o nounset -o noclobber -o pipefail
    shopt -s nullglob

    function main () {
    main() {
    parse_opts "$@"
    set_default_config

    run
    }

    function error () {
    error() {
    echo "$0: $*"
    } 1>&2

    function parse_opts () {
    parse_opts() {
    local OPTIND OPTARG flag
    while getopts hvczd: flag
    do
    @@ -76,59 +76,59 @@ function parse_opts () {
    readonly SUB_COMMAND=("$@")
    }

    function read_default_config () {
    read_default_config() {
    configuration | awk '/[[:alnum:]_ *= *.*]/{ print "[[ -v " $1 " ]] || readonly " $1 $2 "\x27" $3 "\x27" }'
    }

    function set_default_config () {
    set_default_config() {
    local vars=$(read_default_config)
    eval "$vars"
    }

    function subst_vars () {
    subst_vars() {
    export PKG_CACHE_DIR
    envsubst '$HOME $PKG_CACHE_DIR'
    }

    function about () {
    about() {
    set_default_config
    subst_vars
    }

    function usage () {
    usage() {
    sed '/^### SYNOPSIS$/,/^###/!d;//d;s/^## \{0,6\}//' "$0" | about
    }

    function version () {
    version() {
    sed '/^### LICENSE$/,/^###/!d;//d;s/^## \{0,6\}//' "$0" | about
    }

    function help () {
    help() {
    sed '/^##$/,/^####/!d;//d;s/^##.\{0,2\}//' "$0" | about
    }

    function configuration () {
    configuration() {
    sed '/^### CONFIGURATION$/,/^###/!d;//d;s/^## \{0,6\}//' "$0"
    }

    function package_dir () {
    package_dir() {
    echo "$PKG_CACHE_DIR" | subst_vars
    }

    function run () {
    run() {
    call_zypper "${SUB_COMMAND[@]}" \
    | create_download_spec \
    | download_all

    check_rpm
    check_rpm || { error "Package signature failed"; exit 1; }

    if [[ "$RUN_ZYPPER" == yes ]]; then
    local dir=$(package_dir)
    sudo zypper --pkg-cache-dir "$dir" "${SUB_COMMAND[@]}"
    fi
    }

    function call_zypper () {
    call_zypper() {
    local command="$1"
    local command_args=("${@:2}")

    @@ -140,7 +140,7 @@ function call_zypper () {
    "${command_args[@]}"
    }

    function create_download_spec () {
    create_download_spec() {
    local repo_arrays=$(zypper repos --uri | awk -F "|" '
    function trim(s) {
    gsub(/^ +| +$/, "", s);
    @@ -186,7 +186,7 @@ function create_download_spec () {
    ' -
    }

    function download_all () {
    download_all() {
    aria2c --input-file=- \
    --max-connection-per-server=10 \
    --split=16 \
    @@ -197,9 +197,15 @@ function download_all () {
    --continue=true
    }

    function check_rpm () {
    check_rpm() {
    local dir=$(package_dir)
    find "$dir" -type f -name "*.rpm" -print0 | xargs -0 rpm --checksig
    }
    local output
    output=$(find "$dir" -type f -name "*.rpm" -execdir rpm --checksig "{}" +)
    local result=$?
    if (( $result != 0 )); then
    grep " NOT OK$" <<< "$output"
    fi
    return $result
    } 1>&2

    main "$@"
  15. awerlang revised this gist Apr 10, 2020. 1 changed file with 18 additions and 4 deletions.
    22 changes: 18 additions & 4 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -24,12 +24,11 @@
    ## inr, install-new-recommends
    ##
    ### CONFIGURATION
    ##
    ## PKG_CACHE_DIR = ${HOME}/.cache/zypp/packages
    ## RUN_ZYPPER = no
    ##
    ### LICENSE
    ## zypper-download v0.1
    ## zypper-download v0.2
    ##
    ## Copyright (C) 2020 André Werlang
    ##
    @@ -112,13 +111,19 @@ function configuration () {
    sed '/^### CONFIGURATION$/,/^###/!d;//d;s/^## \{0,6\}//' "$0"
    }

    function package_dir () {
    echo "$PKG_CACHE_DIR" | subst_vars
    }

    function run () {
    call_zypper "${SUB_COMMAND[@]}" \
    | create_download_spec \
    | download_all

    check_rpm

    if [[ "$RUN_ZYPPER" == yes ]]; then
    local dir=$(echo "$PKG_CACHE_DIR" | subst_vars)
    local dir=$(package_dir)
    sudo zypper --pkg-cache-dir "$dir" "${SUB_COMMAND[@]}"
    fi
    }
    @@ -158,6 +163,10 @@ function create_download_spec () {
    )

    awk -v PKG_CACHE_DIR="$PKG_CACHE_DIR" "$repo_arrays"'
    function trim(s) {
    gsub(/^ +| +$/, "", s);
    return s;
    }
    NF==0 {
    pkg=0;
    next;
    @@ -170,7 +179,7 @@ function create_download_spec () {
    name=$1; getline;
    version=$NF; getline;
    arch=$1; getline;
    repo=$1; getline;
    repo=trim($0); getline;
    print uri[repo] "/" arch "/" name "-" version "." arch ".rpm";
    print " dir=" PKG_CACHE_DIR "/" alias[repo] "/" arch;
    }
    @@ -188,4 +197,9 @@ function download_all () {
    --continue=true
    }

    function check_rpm () {
    local dir=$(package_dir)
    find "$dir" -type f -name "*.rpm" -print0 | xargs -0 rpm --checksig
    }

    main "$@"
  16. awerlang revised this gist Apr 7, 2020. 1 changed file with 37 additions and 18 deletions.
    55 changes: 37 additions & 18 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -9,9 +9,13 @@
    ##
    ## Options:
    ##
    ## -d <dir> Package cache directory. Default: $PKG_CACHE_DIR
    ## -h Display this message and exit.
    ## -v Display version information and exit.
    ## -d <dir> Package cache directory. It can be an absolute path or relative to the
    ## current directory, or start with the environment variable ${HOME}.
    ## Default: $PKG_CACHE_DIR
    ## -z Run zypper once packages are ready in cache.
    ## -c Display default configuration.
    ## -h Display this message and exit.
    ## -v Display version information and exit.
    ##
    ## Commands:
    ##
    @@ -22,6 +26,7 @@
    ### CONFIGURATION
    ##
    ## PKG_CACHE_DIR = ${HOME}/.cache/zypp/packages
    ## RUN_ZYPPER = no
    ##
    ### LICENSE
    ## zypper-download v0.1
    @@ -37,11 +42,9 @@
    set -o errexit -o nounset -o noclobber -o pipefail
    shopt -s nullglob

    readonly ARGS="$@"

    function main () {
    parse_opts $ARGS
    eval $(read_default_config)
    parse_opts "$@"
    set_default_config

    run
    }
    @@ -52,13 +55,14 @@ function error () {

    function parse_opts () {
    local OPTIND OPTARG flag
    while getopts hvcd: flag
    while getopts hvczd: flag
    do
    case "$flag" in
    (h) help; exit 0;;
    (v) version; exit 0;;
    (c) configuration | about; exit 0;;
    (d) readonly PKG_CACHE_DIR="$OPTARG";;
    (z) readonly RUN_ZYPPER="yes";;
    (*) usage; exit 1;;
    esac
    done
    @@ -70,19 +74,28 @@ function parse_opts () {
    exit 1;
    fi

    readonly SUB_COMMAND="$@"
    readonly SUB_COMMAND=("$@")
    }

    function read_default_config () {
    configuration | awk '/[[:alnum:]_ *= *.*]/{ print "[[ -v " $1 " ]] || readonly " $1 $2 "\x27" $3 "\x27" }'
    }

    function about () {
    eval $(read_default_config)
    function set_default_config () {
    local vars=$(read_default_config)
    eval "$vars"
    }

    function subst_vars () {
    export PKG_CACHE_DIR
    envsubst '$HOME $PKG_CACHE_DIR'
    }

    function about () {
    set_default_config
    subst_vars
    }

    function usage () {
    sed '/^### SYNOPSIS$/,/^###/!d;//d;s/^## \{0,6\}//' "$0" | about
    }
    @@ -100,21 +113,26 @@ function configuration () {
    }

    function run () {
    call_zypper "$SUB_COMMAND" \
    call_zypper "${SUB_COMMAND[@]}" \
    | create_download_spec \
    | download_all

    if [[ "$RUN_ZYPPER" == yes ]]; then
    local dir=$(echo "$PKG_CACHE_DIR" | subst_vars)
    sudo zypper --pkg-cache-dir "$dir" "${SUB_COMMAND[@]}"
    fi
    }

    function call_zypper () {
    local command=$1
    local command_args="${@:2}"
    local command="$1"
    local command_args=("${@:2}")

    sudo zypper --quiet \
    --terse \
    --non-interactive \
    $command --details \
    --dry-run \
    $command_args
    "$command" --details \
    --dry-run \
    "${command_args[@]}"
    }

    function create_download_spec () {
    @@ -164,9 +182,10 @@ function download_all () {
    --max-connection-per-server=10 \
    --split=16 \
    --min-split-size=1M \
    --check-integrity=true \
    --allow-overwrite=true \
    --auto-file-renaming=false \
    --continue=true
    }

    main
    main "$@"
  17. awerlang revised this gist Apr 6, 2020. 1 changed file with 123 additions and 25 deletions.
    148 changes: 123 additions & 25 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,121 @@
    #!/bin/bash
    #####################################################################################################
    ##
    ### NAME
    ## zypper-download - Downloads packages using any amount of available openSUSE mirrors.
    ##
    ### SYNOPSIS
    ## Usage: zypper-download.sh [options] command [command-args]...
    ##
    ## Options:
    ##
    ## -d <dir> Package cache directory. Default: $PKG_CACHE_DIR
    ## -h Display this message and exit.
    ## -v Display version information and exit.
    ##
    ## Commands:
    ##
    ## dup, dist-upgrade
    ## in, install package-name...
    ## inr, install-new-recommends
    ##
    ### CONFIGURATION
    ##
    ## PKG_CACHE_DIR = ${HOME}/.cache/zypp/packages
    ##
    ### LICENSE
    ## zypper-download v0.1
    ##
    ## Copyright (C) 2020 André Werlang
    ##
    ## License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
    ## This is free software: you are free to change and redistribute it.
    ## There is NO WARRANTY, to the extent permitted by law.
    ##
    #####################################################################################################

    set -o errexit -o nounset -o noclobber -o pipefail
    shopt -s nullglob

    readonly ARGS="$@"

    function main () {
    parse_opts $ARGS
    eval $(read_default_config)

    run
    }

    function error () {
    echo "$0: $*"
    } 1>&2

    function parse_opts () {
    local OPTIND OPTARG flag
    while getopts hvcd: flag
    do
    case "$flag" in
    (h) help; exit 0;;
    (v) version; exit 0;;
    (c) configuration | about; exit 0;;
    (d) readonly PKG_CACHE_DIR="$OPTARG";;
    (*) usage; exit 1;;
    esac
    done
    shift $((OPTIND-1))

    if [ $# -eq 0 ]; then
    error "expected required argument -- command"
    usage;
    exit 1;
    fi

    readonly SUB_COMMAND="$@"
    }

    function read_default_config () {
    configuration | awk '/[[:alnum:]_ *= *.*]/{ print "[[ -v " $1 " ]] || readonly " $1 $2 "\x27" $3 "\x27" }'
    }

    function about () {
    eval $(read_default_config)
    export PKG_CACHE_DIR
    envsubst '$HOME $PKG_CACHE_DIR'
    }

    function usage () {
    sed '/^### SYNOPSIS$/,/^###/!d;//d;s/^## \{0,6\}//' "$0" | about
    }

    function version () {
    sed '/^### LICENSE$/,/^###/!d;//d;s/^## \{0,6\}//' "$0" | about
    }

    function help () {
    sed '/^##$/,/^####/!d;//d;s/^##.\{0,2\}//' "$0" | about
    }

    function configuration () {
    sed '/^### CONFIGURATION$/,/^###/!d;//d;s/^## \{0,6\}//' "$0"
    }

    function run () {
    call_zypper "$SUB_COMMAND" \
    | create_download_spec \
    | download_all
    }

    function call_zypper () {
    local command=$1
    local command_args="${@:2}"

    sudo zypper --quiet \
    --terse \
    --non-interactive \
    $command --details \
    --dry-run \
    $command_args
    }

    function create_download_spec () {
    local repo_arrays=$(zypper repos --uri | awk -F "|" '
    @@ -22,7 +139,7 @@ function create_download_spec () {
    '
    )

    awk "$repo_arrays"'
    awk -v PKG_CACHE_DIR="$PKG_CACHE_DIR" "$repo_arrays"'
    NF==0 {
    pkg=0;
    next;
    @@ -37,38 +154,19 @@ function create_download_spec () {
    arch=$1; getline;
    repo=$1; getline;
    print uri[repo] "/" arch "/" name "-" version "." arch ".rpm";
    print " dir=" alias[repo] "/" arch;
    print " dir=" PKG_CACHE_DIR "/" alias[repo] "/" arch;
    }
    ' -
    }

    function download_all () {
    local local_dir="$HOME/.cache/zypp/packages"
    mkdir -p $local_dir
    cd $local_dir
    aria2c --input-file=- \
    --max-connection-per-server=10 \
    --split=16 \
    --min-split-size=1M \
    --auto-file-renaming=false
    }

    function call_zypper () {
    local command=${1:-"dup"}
    local command_args="${@:2}"

    sudo zypper --quiet \
    --terse \
    --non-interactive \
    $command --details \
    --dry-run \
    $command_args
    }

    function main () {
    call_zypper $@ |
    create_download_spec |
    download_all
    --allow-overwrite=true \
    --auto-file-renaming=false \
    --continue=true
    }

    main $@
    main
  18. awerlang revised this gist Apr 4, 2020. 2 changed files with 2 additions and 2 deletions.
    1 change: 1 addition & 0 deletions aria2.conf
    View file
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,4 @@
    check-integrity=true
    console-log-level=warn
    file-allocation=falloc
    max-concurrent-downloads=40
    3 changes: 1 addition & 2 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -50,8 +50,7 @@ function download_all () {
    --max-connection-per-server=10 \
    --split=16 \
    --min-split-size=1M \
    --auto-file-renaming=false \
    --console-log-level=warn
    --auto-file-renaming=false
    }

    function call_zypper () {
  19. awerlang revised this gist Apr 4, 2020. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions aria2.conf
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,3 @@
    check-integrity=true
    file-allocation=falloc
    max-concurrent-downloads=40
  20. awerlang revised this gist Apr 4, 2020. 1 changed file with 0 additions and 1 deletion.
    1 change: 0 additions & 1 deletion zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -50,7 +50,6 @@ function download_all () {
    --max-connection-per-server=10 \
    --split=16 \
    --min-split-size=1M \
    --max-concurrent-downloads=30 \
    --auto-file-renaming=false \
    --console-log-level=warn
    }
  21. awerlang revised this gist Apr 4, 2020. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -27,7 +27,7 @@ function create_download_spec () {
    pkg=0;
    next;
    }
    / (NEW packages are going to be installed:|packages are going to be upgraded:)$/ {
    / packages? (is|are) going to be (installed|upgraded):$/ {
    pkg=1;
    next;
    }
    @@ -50,7 +50,7 @@ function download_all () {
    --max-connection-per-server=10 \
    --split=16 \
    --min-split-size=1M \
    --max-concurrent-downloads=20 \
    --max-concurrent-downloads=30 \
    --auto-file-renaming=false \
    --console-log-level=warn
    }
  22. awerlang revised this gist Apr 4, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -50,7 +50,7 @@ function download_all () {
    --max-connection-per-server=10 \
    --split=16 \
    --min-split-size=1M \
    --max-concurrent-downloads=50 \
    --max-concurrent-downloads=20 \
    --auto-file-renaming=false \
    --console-log-level=warn
    }
  23. awerlang revised this gist Apr 4, 2020. 1 changed file with 64 additions and 58 deletions.
    122 changes: 64 additions & 58 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -1,70 +1,76 @@
    #!/bin/bash

    MAX_PROC=6
    function create_download_spec () {
    local repo_arrays=$(zypper repos --uri | awk -F "|" '
    function trim(s) {
    gsub(/^ +| +$/, "", s);
    return s;
    }
    BEGIN {
    print "BEGIN {";
    }
    /^[0-9]/ {
    alias=trim($2);
    name=trim($3);
    uri=trim($NF);
    print " uri[\"" name "\"]=\"" uri "\";";
    print " alias[\"" name "\"]=\"" alias "\";";
    }
    END {
    print "}";
    }
    '
    )

    function repos_to_update () {
    zypper list-updates | grep '^v ' | awk -F '|' '{ print $2 }' | sort --unique | tr -d ' '
    awk "$repo_arrays"'
    NF==0 {
    pkg=0;
    next;
    }
    / (NEW packages are going to be installed:|packages are going to be upgraded:)$/ {
    pkg=1;
    next;
    }
    pkg==1 {
    name=$1; getline;
    version=$NF; getline;
    arch=$1; getline;
    repo=$1; getline;
    print uri[repo] "/" arch "/" name "-" version "." arch ".rpm";
    print " dir=" alias[repo] "/" arch;
    }
    ' -
    }

    function packages_from_repo () {
    local repo=$1

    zypper list-updates | grep " | $repo " | awk -F '|' '{ print $6, "#", $3, "-", $5, ".", $6, ".rpm" }' | tr -d ' '
    }

    function repo_uri () {
    local repo=$1

    zypper repos --uri | grep " | $repo " | awk -F '|' '{ print $7 }' | tr -d ' '
    }

    function repo_alias () {
    local repo=$1

    zypper repos | grep " | $repo " | awk -F '|' '{ print $2 }' | tr -d ' '
    }

    function download_package () {
    local alias=$1
    local uri=$2
    local line=$3
    IFS=# read arch package_name <<< "$line"

    local package_uri="$uri/$arch/$package_name"
    local local_dir="$HOME/.cache/zypp/packages/$alias/$arch"
    local local_path="$local_dir/$package_name"
    printf -v y %-30s "$repo"
    printf "Repository: $y Package: $package_name\n"
    if [ ! -f "$local_path" ]; then
    mkdir -p $local_dir
    curl --silent --fail -L -o $local_path $package_uri
    fi
    function download_all () {
    local local_dir="$HOME/.cache/zypp/packages"
    mkdir -p $local_dir
    cd $local_dir
    aria2c --input-file=- \
    --max-connection-per-server=10 \
    --split=16 \
    --min-split-size=1M \
    --max-concurrent-downloads=50 \
    --auto-file-renaming=false \
    --console-log-level=warn
    }

    function download_repo () {
    local repo=$1
    function call_zypper () {
    local command=${1:-"dup"}
    local command_args="${@:2}"

    local uri=$(repo_uri $repo)
    local alias=$(repo_alias $repo)
    local pkgs=$(packages_from_repo $repo)
    local max_proc=$MAX_PROC
    while IFS= read -r line; do
    if [ $max_proc -eq 0 ]; then
    wait -n
    ((max_proc++))
    fi
    download_package "$alias" "$uri" "$line" &
    ((max_proc--))
    done <<< "$pkgs"
    sudo zypper --quiet \
    --terse \
    --non-interactive \
    $command --details \
    --dry-run \
    $command_args
    }

    function download_all () {
    local repos=$(repos_to_update)
    while IFS= read -r line; do
    download_repo $line &
    done <<< "$repos"
    wait
    function main () {
    call_zypper $@ |
    create_download_spec |
    download_all
    }

    download_all
    #sudo cp -r ~/.cache/zypp/packages/* /var/cache/zypp/packages/
    main $@
  24. awerlang created this gist Apr 1, 2020.
    70 changes: 70 additions & 0 deletions zypper-download.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,70 @@
    #!/bin/bash

    MAX_PROC=6

    function repos_to_update () {
    zypper list-updates | grep '^v ' | awk -F '|' '{ print $2 }' | sort --unique | tr -d ' '
    }

    function packages_from_repo () {
    local repo=$1

    zypper list-updates | grep " | $repo " | awk -F '|' '{ print $6, "#", $3, "-", $5, ".", $6, ".rpm" }' | tr -d ' '
    }

    function repo_uri () {
    local repo=$1

    zypper repos --uri | grep " | $repo " | awk -F '|' '{ print $7 }' | tr -d ' '
    }

    function repo_alias () {
    local repo=$1

    zypper repos | grep " | $repo " | awk -F '|' '{ print $2 }' | tr -d ' '
    }

    function download_package () {
    local alias=$1
    local uri=$2
    local line=$3
    IFS=# read arch package_name <<< "$line"

    local package_uri="$uri/$arch/$package_name"
    local local_dir="$HOME/.cache/zypp/packages/$alias/$arch"
    local local_path="$local_dir/$package_name"
    printf -v y %-30s "$repo"
    printf "Repository: $y Package: $package_name\n"
    if [ ! -f "$local_path" ]; then
    mkdir -p $local_dir
    curl --silent --fail -L -o $local_path $package_uri
    fi
    }

    function download_repo () {
    local repo=$1

    local uri=$(repo_uri $repo)
    local alias=$(repo_alias $repo)
    local pkgs=$(packages_from_repo $repo)
    local max_proc=$MAX_PROC
    while IFS= read -r line; do
    if [ $max_proc -eq 0 ]; then
    wait -n
    ((max_proc++))
    fi
    download_package "$alias" "$uri" "$line" &
    ((max_proc--))
    done <<< "$pkgs"
    }

    function download_all () {
    local repos=$(repos_to_update)
    while IFS= read -r line; do
    download_repo $line &
    done <<< "$repos"
    wait
    }

    download_all
    #sudo cp -r ~/.cache/zypp/packages/* /var/cache/zypp/packages/