Skip to content

Instantly share code, notes, and snippets.

@Dedsec1
Created February 18, 2017 22:11
Show Gist options
  • Save Dedsec1/66a0eeb733cfbc738a329a4114109ab4 to your computer and use it in GitHub Desktop.
Save Dedsec1/66a0eeb733cfbc738a329a4114109ab4 to your computer and use it in GitHub Desktop.

Revisions

  1. Dedsec1 created this gist Feb 18, 2017.
    116 changes: 116 additions & 0 deletions dest.conf
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,116 @@
    # HB Rclone destination
    #
    # Rclone is a program that syncs local directories to various cloud
    # services. The Rclone project is at: http://rclone.org
    #
    # HashBackup supports shell destinations to interface to storage
    # systems that are not natively supported in HashBackup. If there is
    # native support for a destination, it should always be preferred over
    # a shell destination.
    #
    # rclone.py is an HB shell destination, written in Python, that runs
    # Rclone to get, send, and delete files on any storage provider
    # supported by Rclone. For example, Rclone supports Google Drive,
    # which is not natively supported in HashBackup. Using the rclone.py
    # shell destination, HashBackup gains the ability to use Google Drive
    # via Rclone.
    #
    # Steps to use Rclone with HashBackup:
    #
    # 1. download and install Rclone from http://rclone.org
    #
    # 2. run rclone config to create a .rclone.conf file
    #
    # 3. copy the rclone.py script from doc/dest.conf.examples to your
    # backup directory. If you will have multiple backups using this
    # rclone.py, you might want to copy it to a single directory all
    # backups can access.
    #
    # 4. read the README file and dest.conf.shell to see how destinations
    # work in general. Read this entire dest.conf.rclone file to see
    # how the rclone shell destination works.
    #
    # 5. create a dest.conf file in your HB backup directory. Below are
    # example destinations using Rclone.
    #
    # NOTE: The B2 destination below is just for testing. HB has
    # integrated B2 support that will work better than a shell
    # destination.
    #
    # 6. Before your first backup with any shell destination, do:
    #
    # $ hb dest -c backupdir setid <shell destname>
    #
    # If you skip this step, HB will display 3 errors on your first
    # backup about DESTID and will not start the destination.
    #
    # IMPORTANT: the destname keyword and --destname are HB values and
    # should match. The rclone remote name following --clonedir must
    # match the name used in rclone config.

    destname gdrive
    type shell
    debug 1
    run python /home/jim/hb/rclone.py --debug --args "-v " --destname hubic --backupdir /home/jim/hb --clonedir gdrive:test --command

    destname acd
    type shell
    workers 5
    run python /Users/jim/rclone.py --args "--bwlimit 400K" --destname acd --backupdir /Users/jim/hbrel/hb --clonedir acd:test --command

    destname hubic
    type shell
    debug 1
    run python /home/jim/hb/rclone.py --debug --args "-v " --destname hubic --backupdir /home/jim/hb --clonedir hubic:test --command

    # the debug keyword (shown beneath type shell) is optional. It causes
    # the HB shell destination to display debug info. Specifically, it
    # displays the command line used to start rclone.py script. debug 99
    # causes HB to abort with a traceback instead of doing error retries.
    #
    # shell destinations support the workers dest.conf keyword, to control
    # how many threads to use.
    #
    # The run keyword controls the rclone.py script. It must be all on 1
    # line and has several keywords. You can use run python2.7 ... to use
    # a different version of python to run this script
    #
    # The rclone.py script runs the rclone command without a pathname, so
    # the rclone executable must be in your current path. For cron, add
    # PATH=... to set the path cron uses to find rclone.

    # The options below apply only to the rclone.py shell destination.
    # Every shell script has its own set of options.
    #
    # --debug is optional and will cause rclone.py to display more about
    # what it is doing and what is happening. It displays the complete
    # Rclone command line before it is executed.
    #
    # --args is optional. Anything in the next quoted string will be
    # added to the Rclone command line. In the examples above, -v is
    # being added to the Rclone command so that Rclone displays detailed
    # output about its operations. See the Rclone option list for
    # examples of what you can put here.
    #
    # IMPORTANT GOTCHA! With --args there must be a space in the quoted
    # string or Python will complain.
    #
    # --destname is required. It tells the rclone.py script what your
    # destination name is, usually to display error messages.
    #
    # --backupdir is required. It is your HB -c backup directory.
    #
    # --clonedir is your Rclone remote destination directory. Different
    # types of storage may require different things with --clonedir.
    #
    # --command is required and must be at the end of the run line.
    #
    # When trying to debug problems, you will likely want to set debug 1,
    # add --debug to the "run" line, and add --args "-v " to get as much
    # output as possible. It's also a good idea to use workers 1 when
    # debugging, to avoid mixing multiple workers' output. Debug output
    # has a process ID in square brackets so you can follow the messages
    # for a specific worker.
    #
    # Some of the destinations Rclone supports can be quite slow. You may
    # need patience!