Skip to content

Instantly share code, notes, and snippets.

@05nelsonm

05nelsonm/Move Dojo bitcoind container data to host machine

Last active August 25, 2019 03:31
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save 05nelsonm/76876de732bc965808c2b363b51db5d1 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save 05nelsonm/76876de732bc965808c2b363b51db5d1 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. 05nelsonm revised this gist Aug 25, 2019. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion Move Dojo bitcoind container data to host machine
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    #### Move your bitcoind data from the Dojo's bitcoind docker container, to outside of the container on your host machine ####

    ## Written for use with Ubuntu Desktop 18.04 LTS ##
    ## Written for use with Ubuntu Desktop 18.04 LTS & Dojo v1.0.0 ##

    ## Start Dojo
    $ cd /path/to/docker/my-dojo/ && sudo ./dojo.sh start
  2. 05nelsonm revised this gist Jun 17, 2019. 1 changed file with 10 additions and 8 deletions.
    18 changes: 10 additions & 8 deletions Move Dojo bitcoind container data to host machine
    View file
    Original file line number Diff line number Diff line change
    @@ -44,11 +44,9 @@ $ touch bitcoin_data/test

    ## Under the section for `bitcoind` --> `volumes:` add:
    - ./bitcoin_data:/mnt/bitcoin_data

    ## Save and exit
    ctrl+x --> y --> return

    ## Your bitcoind section should now look something like the following ####

    #### Your bitcoind section should now look something like the following ####
    bitcoind:
    image: "samouraiwallet/dojo-bitcoind:1.0.0"
    container_name: bitcoind
    @@ -64,7 +62,7 @@ $ touch bitcoin_data/test
    - "9501"
    - "9502"
    volumes:
    - ./bitcoin_data:/mnt/bitcoin_data
    - ./bitcoin_data:/mnt/bitcoin_data
    - data-bitcoind:/home/bitcoin/.bitcoin
    depends_on:
    - db
    @@ -73,6 +71,10 @@ $ touch bitcoin_data/test
    dojonet:
    ipv4_address: 172.28.1.5


    ## Save and exit
    ctrl+x --> y --> return

    ## Start Dojo
    $ sudo ./dojo.sh start

    @@ -95,7 +97,7 @@ $ mv /home/bitcoin/.bitcoin/ /mnt/bitcoin_data/
    ## This takes a bit, and seemed like it locked up my computer but it did not. Be patient.
    ## Took my pc about 15 minutes on an SSD.

    ## Open another terminal and make sure the files are there, then exit that new terminal
    ## Open another terminal [ctrl + alt + t] and make sure the files are there
    $ cd /home/path/to/docker/my-dojo/
    $ du -h bitcoin_data

    @@ -104,7 +106,7 @@ $ du -h bitcoin_data
    ## Exit that new terminal you just opened
    $ exit

    ## Switching back to your previous terminal, exit the docker container
    ## Switching back to your previous terminal and exit the docker container
    $ exit

    ## Stop Dojo
    @@ -168,7 +170,7 @@ rpcthreads=20
    End bitcoin.conf file

    ## Note: if you've previously followed my guide for installing Electrs (Electrum Personal Server), as well as LND with
    ## Dojo v1.0.0, you will **NOT** have to change the configurations for both of those **IF** you use the above .conf file.
    ## Dojo v1.0.0, you will **NOT** have to change the configurations for either of those **IF** you use the above .conf file.
    ## If you do need to modify them to work with the standard bitcoindrpc port (default is 8332), those files can be found at:
    $ nano ~/.lnd/lnd.conf
    $ sudo nano /usr/share/applications/electrum-personal-server.desktop
  3. 05nelsonm revised this gist Jun 17, 2019. 1 changed file with 3 additions and 3 deletions.
    6 changes: 3 additions & 3 deletions Move Dojo bitcoind container data to host machine
    View file
    Original file line number Diff line number Diff line change
    @@ -43,7 +43,7 @@ $ touch bitcoin_data/test
    ## Modify the docker-compose.yaml file to bind mount a host directory to a bitcoind container directory

    ## Under the section for `bitcoind` --> `volumes:` add:
    - ./bitcoin_data:/mnt/bitcoin_data
    - ./bitcoin_data:/mnt/bitcoin_data

    ## Save and exit
    ctrl+x --> y --> return
    @@ -86,8 +86,8 @@ $ sudo docker exec -u root -it bitcoind /bin/bash
    $ cd /mnt/bitcoin_data
    $ ls -la

    ## You should see the `test` file we created earlier. You can remove it now.
    $ rm -rf test
    ## You should see the `test` file we created earlier. You can remove it now.
    $ rm -rf test

    ## Move the .bitcoin directory to your bind mounted directory
    $ mv /home/bitcoin/.bitcoin/ /mnt/bitcoin_data/
  4. 05nelsonm revised this gist Jun 17, 2019. 1 changed file with 0 additions and 1 deletion.
    1 change: 0 additions & 1 deletion Move Dojo bitcoind container data to host machine
    View file
    Original file line number Diff line number Diff line change
    @@ -147,7 +147,6 @@ proxy=127.0.0.1:9050

    # RPC information
    rpcport=28256
    rpcbind=bitcoind
    rpcuser=***your_dojo_rpc_username***
    rpcpassword=***your_dojo_rpc_password***

  5. 05nelsonm revised this gist Jun 17, 2019. 1 changed file with 2 additions and 4 deletions.
    6 changes: 2 additions & 4 deletions Move Dojo bitcoind container data to host machine
    View file
    Original file line number Diff line number Diff line change
    @@ -175,10 +175,8 @@ End bitcoin.conf file
    $ sudo nano /usr/share/applications/electrum-personal-server.desktop

    ########## Install Tor ############################################################################################################
    ## The Dojo's tor container has the control port bound to it's container's localhost, and at the moment editing the
    ## /etc/tor/torrc file yeilds poor results as it reverts back after a Dojo stop and start cycle. My workaround is to just
    ## install tor on the machine because... why not?

    ## Login to root user
    $ sudo -s

    ## *** Follow the guide at https://2019.www.torproject.org/docs/debian.html.en ***
    @@ -188,7 +186,7 @@ $ sudo -s
    ## Log out of root user
    $ exit

    ## Add user to debian-tor group
    ## Add your user to debian-tor group
    $ sudo adduser $USER debian-tor

    ## Verify that your username has been added to the debian-tor group (should be the last line of what prints out)
  6. 05nelsonm revised this gist Jun 17, 2019. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions Move Dojo bitcoind container data to host machine
    View file
    Original file line number Diff line number Diff line change
    @@ -160,9 +160,11 @@ disablewallet=1
    # ZeroMQ Notification Settings
    zmqpubrawtx=tcp://127.0.0.1:28333
    zmqpubrawblocks=tcp://127.0.0.1:28332
    zmqpubhashblock=tcp://127.0.0.1:9502

    # Additional Settings
    rpcthreads=20

    -----------------------------------------------------------------------------------------------------------------------------
    End bitcoin.conf file

  7. 05nelsonm created this gist Jun 17, 2019.
    220 changes: 220 additions & 0 deletions Move Dojo bitcoind container data to host machine
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,220 @@
    #### Move your bitcoind data from the Dojo's bitcoind docker container, to outside of the container on your host machine ####

    ## Written for use with Ubuntu Desktop 18.04 LTS ##

    ## Start Dojo
    $ cd /path/to/docker/my-dojo/ && sudo ./dojo.sh start

    ## Login to the bitcoind docker container as root
    $ sudo docker exec -u root -it bitcoind /bin/bash

    ## Update and install a text editor
    $ apt-get update && apt-get install nano

    ## Stopping bitcoind via 'sudo ./dojo.sh bitcoin-cli stop' shuts the container down, so I found it easiest to
    ## just add the following to bitcoin.conf and then stop the tor container to cut internet so the bitcoind
    ## container won't be able to download anything while moving the .bitcoin directory
    ## In [terminal Doc]
    $ nano /home/bitcoin/.bitcoin/bitcoin.conf

    ## Add to bitcoin.conf
    onlynet=onion
    dnsseed=0
    dns=0

    ## Save and exit
    ctrl+x --> y --> return

    ## Make a new directory in the bitcoind container to mount a host directory to
    $ mkdir /mnt/bitcoin_data

    ## Exit the bitcoind container
    $ exit

    ## Stop Dojo
    $ sudo ./dojo.sh stop

    ## Make a new directory in your /path/to/docker/my-dojo/ directory to bind to the bitcoind container
    ## and create a test file to make sure it mounted properly
    ## *** your current working directory should be /path/to/docker/my-dojo/ ***
    $ mkdir bitcoin_data
    $ touch bitcoin_data/test

    ## Modify the docker-compose.yaml file to bind mount a host directory to a bitcoind container directory

    ## Under the section for `bitcoind` --> `volumes:` add:
    - ./bitcoin_data:/mnt/bitcoin_data

    ## Save and exit
    ctrl+x --> y --> return

    ## Your bitcoind section should now look something like the following ####
    bitcoind:
    image: "samouraiwallet/dojo-bitcoind:1.0.0"
    container_name: bitcoind
    build:
    context: ./bitcoin
    env_file:
    - ./.env
    - ./conf/docker-bitcoind.conf
    restart: on-failure
    command: "/wait-for-it.sh tor:9050 --timeout=360 --strict -- /restart.sh"
    expose:
    - "28256"
    - "9501"
    - "9502"
    volumes:
    - ./bitcoin_data:/mnt/bitcoin_data
    - data-bitcoind:/home/bitcoin/.bitcoin
    depends_on:
    - db
    - tor
    networks:
    dojonet:
    ipv4_address: 172.28.1.5

    ## Start Dojo
    $ sudo ./dojo.sh start

    ## Pause the TOR container. This will stop bitcoind from downloading anything b/c it won't have an internet connection
    $ sudo docker container pause tor

    ## Login to bitcoind container as root
    $ sudo docker exec -u root -it bitcoind /bin/bash

    ## Ensure that things mounted properly
    $ cd /mnt/bitcoin_data
    $ ls -la

    ## You should see the `test` file we created earlier. You can remove it now.
    $ rm -rf test

    ## Move the .bitcoin directory to your bind mounted directory
    $ mv /home/bitcoin/.bitcoin/ /mnt/bitcoin_data/

    ## This takes a bit, and seemed like it locked up my computer but it did not. Be patient.
    ## Took my pc about 15 minutes on an SSD.

    ## Open another terminal and make sure the files are there, then exit that new terminal
    $ cd /home/path/to/docker/my-dojo/
    $ du -h bitcoin_data

    ## As of June 2019, my .bitcoin directory was 263G

    ## Exit that new terminal you just opened
    $ exit

    ## Switching back to your previous terminal, exit the docker container
    $ exit

    ## Stop Dojo
    $ sudo ./dojo.sh start

    ## Move files to your host system's home folder
    $ cd bitcoin_data
    $ sudo mv .bitcoin ~

    ## Change ownership of everything to your current user
    $ sudo chown $USER:$USER ~/.bitcoin/*
    $ sudo chown $USER:$USER ~/.bitcoin/blocks/*
    $ sudo chown $USER:$USER ~/.bitcoin/blocks/index/*
    $ sudo chown $USER:$USER ~/.bitcoin/chainstate/*
    $ sudo chown $USER:$USER ~/.bitcoin/indexes/*
    $ sudo chown $USER:$USER ~/.bitcoin/indexes/txindex/*
    $ sudo chown $USER:$USER ~/.bitcoin/wallets/*

    ##### If there are issues with changing ownership of the txindex/* files (Arguments being too long), try the following:
    $ sudo find ~/.bitcoin/indexes/txindex/ -type f -exec chown $USER:$USER {} \;

    ## Remove the `.lock` , `.pid` , and `.walletlock` files
    $ rm -rf ~/.bitcoin/.lock
    $ rm -rf ~/.bitcoin/bitcoind.pid
    $ rm -rf ~/wallets/.walletlock

    ## Modify bitcoin.conf (it might need to be modified again if installing the next Dojo release which can be installed ontop
    ## of an already existing bitcoind instance. Follow those directions when doing so for modifying bitcoin.conf as needed).
    $ nano ~/.bitcoin/bitcoin.conf

    Begin sample bitcoin.conf file
    -----------------------------------------------------------------------------------------------------------------------------
    # Bitcoin Configuration
    server=1
    listen=1
    bind=127.0.0.1

    # Tor proxy
    proxy=127.0.0.1:9050

    # RPC information
    rpcport=28256
    rpcbind=bitcoind
    rpcuser=***your_dojo_rpc_username***
    rpcpassword=***your_dojo_rpc_password***

    # Store transaction information for fully-spent txns
    txindex=1

    # No wallet
    disablewallet=1

    # ZeroMQ Notification Settings
    zmqpubrawtx=tcp://127.0.0.1:28333
    zmqpubrawblocks=tcp://127.0.0.1:28332

    # Additional Settings
    rpcthreads=20
    -----------------------------------------------------------------------------------------------------------------------------
    End bitcoin.conf file

    ## Note: if you've previously followed my guide for installing Electrs (Electrum Personal Server), as well as LND with
    ## Dojo v1.0.0, you will **NOT** have to change the configurations for both of those **IF** you use the above .conf file.
    ## If you do need to modify them to work with the standard bitcoindrpc port (default is 8332), those files can be found at:
    $ nano ~/.lnd/lnd.conf
    $ sudo nano /usr/share/applications/electrum-personal-server.desktop

    ########## Install Tor ############################################################################################################
    ## The Dojo's tor container has the control port bound to it's container's localhost, and at the moment editing the
    ## /etc/tor/torrc file yeilds poor results as it reverts back after a Dojo stop and start cycle. My workaround is to just
    ## install tor on the machine because... why not?

    $ sudo -s

    ## *** Follow the guide at https://2019.www.torproject.org/docs/debian.html.en ***

    ## Be sure to select the correct distribution of your Linux package

    ## Log out of root user
    $ exit

    ## Add user to debian-tor group
    $ sudo adduser $USER debian-tor

    ## Verify that your username has been added to the debian-tor group (should be the last line of what prints out)
    $ cat /etc/group

    ## Edit /etc/tor/torrc
    $ sudo nano /etc/tor/torrc

    ## Add to torrc
    SocksPort 9050
    Log notice stdout
    ControlPort 9051
    CookieAuthentication 1

    ## Save and exit
    ctrl+x --> y --> return

    ## Reboot the machine
    $ sudo shutdown -r now

    ######### Install Bitcoin Core ####################################################################################################
    ## Go to --> https://bitcoincore.org/en/download/
    ## Follow instructions to verify the download in the section just under the download selection (ABSOLUTELY DO NOT BE LAZY AND SKIP)
    ## Follow instructions to install by going to --> https://bitcoin.org/en/full-node#linux-instructions

    ## Boot it all up and make sure it works!
    $ bitcoind

    ## Now go get the next release of Dojo!

    ## Done