Skip to content

Instantly share code, notes, and snippets.

@figassis

figassis/README.md

Last active April 6, 2024 15:53
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save figassis/4054bb15d40d7516ab1898d65bac9b89 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save figassis/4054bb15d40d7516ab1898d65bac9b89 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. figassis revised this gist Oct 12, 2019. 1 changed file with 13 additions and 14 deletions.
    27 changes: 13 additions & 14 deletions docker-compose.yaml
    View file
    Original file line number Diff line number Diff line change
    @@ -1,21 +1,20 @@
    version: "2.1"
    services:
    http:
    image: tecnativa/tcp-proxy
    tunnel:
    image: "dweomer/stunnel"
    restart: always
    ports:
    - 80:80
    - 443:443
    - 53:53
    container_name: stunnel
    network_mode: host
    volumes:
    - ./certs/stunnel.pem:/etc/stunnel/stunnel.pem:ro
    - ./certs/stunnel.key:/etc/stunnel/stunnel.key:ro
    - ./stunnel.conf:/srv/stunnel/stunnel.conf.template:ro
    environment:
    LISTEN:
    :80
    :443
    :53
    TALK:
    host_ip:8080
    host_ip:8443
    host_ip:8053
    STUNNEL_SERVICE: nfs
    STUNNEL_ACCEPT: 2050
    STUNNEL_CONNECT: 127.0.0.1:2049
    STUNNEL2_ACCEPT: 111
    STUNNEL2_CONNECT: 127.0.0.1:8111
    logging:
    driver: "json-file"
    options:
  2. figassis revised this gist Oct 12, 2019. 1 changed file with 15 additions and 4 deletions.
    19 changes: 15 additions & 4 deletions README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -65,10 +65,21 @@ vboxmanage import ~/images/AWS-Appliance.ova --vsys 0 --vmname storage_gateway
    ```


    ## Create NAT Network, which as opposed to simple NAT, allows guest machines to communicate with each other
    ```bash
    VBoxManage natnetwork add --netname default --network "192.168.15.0/24" --enable --dhcp on
    VBoxManage natnetwork start --netname default
    ## Condigure vm to use NAT Networking
    ```bash
    vboxmanage modifyvm storage_gateway --nic1 nat
    vboxmanage modifyvm storage_gateway --natpf1 "nfs,tcp,,2049,,2049"
    vboxmanage modifyvm storage_gateway --natpf2 "nfsudp,udp,,2049,,2049"
    vboxmanage modifyvm storage_gateway --natpf3 "ntp,tcp,,8123,,123"

    Ports below may not be required
    vboxmanage modifyvm storage_gateway --natpf4 "dnstcp,tcp,,8053,,53"
    vboxmanage modifyvm storage_gateway --natpf5 "dnsudp,udp,,8053,,53"
    vboxmanage modifyvm storage_gateway --natpf6 "http,tcp,,8080,,80"
    vboxmanage modifyvm storage_gateway --natpf7 "https,tcp,,8443,,443"
    vboxmanage modifyvm storage_gateway --natpf1 "ssh_support,tcp,,2222,,22"
    vboxmanage modifyvm storage_gateway --natpf1 "nfs2,tcp,,8111,,111"
    vboxmanage modifyvm storage_gateway --natpf1 "nfs2udp,udp,,8111,,111"
    ```

    ## Forward required ports for AWS Storage Gateway.
  3. figassis revised this gist Sep 19, 2019. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -86,6 +86,8 @@ VBoxManage natnetwork modify --netname default --port-forward-4 "ssh_support:tcp
    VBoxManage natnetwork modify --netname default --port-forward-4 "ntp:udp:[]:8123:[192.168.15.2]:123"
    VBoxManage natnetwork modify --netname default --port-forward-4 "nfs:tcp:[]:2049:[192.168.15.2]:2049"
    VBoxManage natnetwork modify --netname default --port-forward-4 "nfsudp:udp:[]:2049:[192.168.15.2]:2049"
    VBoxManage natnetwork modify --netname default --port-forward-4 "nfs2:tcp:[]:8111:[192.168.15.2]:111"
    VBoxManage natnetwork modify --netname default --port-forward-4 "nfs2udp:udp:[]:8111:[192.168.15.2]:111"
    ```

    ## Configure and enable UFW. Make sure to allow port 22 for ssh
  4. figassis revised this gist Sep 16, 2019. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -149,5 +149,5 @@ After creating a share on the AWS Console, you can mount it with:
    sudo apt-get -y install nfs-common

    # Mount Share
    sudo mount -t nfs -o nolock,hard gateway_ip:/static.nellcorp.com MOUNTPATH
    sudo mount -t nfs -o nolock,hard gateway_ip:/bucket/prefix MOUNTPATH
    ```
  5. figassis revised this gist Sep 16, 2019. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -145,5 +145,9 @@ vboxmanage controlvm storage_gateway pause|resume|reset|poweroff|savestate
    ## Mount NFS share
    After creating a share on the AWS Console, you can mount it with:
    ```bash
    sudo mount -t nfs -o nolock,hard gateway_ip:/static.nellcorp.com /home/assis/test
    # Install NFS helper utils
    sudo apt-get -y install nfs-common

    # Mount Share
    sudo mount -t nfs -o nolock,hard gateway_ip:/static.nellcorp.com MOUNTPATH
    ```
  6. figassis revised this gist Sep 16, 2019. 1 changed file with 10 additions and 1 deletion.
    11 changes: 10 additions & 1 deletion README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -104,9 +104,12 @@ sudo ufw enable
    ## Configure Docker to forward ports
    docker-compose up -d

    ## Modify the vm to use the new nat network and start at boot
    ## Modify the vm to use the new nat network and start at boot, and also create and attach a 250 GB disk for caching
    ```bash
    vboxmanage modifyvm storage_gateway --nic1 natnetwork --nat-network1 default --autostart-enabled on
    vboxmanage createhd --filename ~/disks/cache.vdi --size 262144
    mkdir -p ~/disks
    vboxmanage storageattach storage_gateway --storagectl "SATA Controller" --port 1 --device 0 --type hdd --medium ~/disks/cache.vdi
    ```

    ## Optionally, check your VM info before starting it
    @@ -137,4 +140,10 @@ When you see the Appliance's login screen, login with user: admin / password: pa
    ## VM Controls
    ```bash
    vboxmanage controlvm storage_gateway pause|resume|reset|poweroff|savestate
    ```

    ## Mount NFS share
    After creating a share on the AWS Console, you can mount it with:
    ```bash
    sudo mount -t nfs -o nolock,hard gateway_ip:/static.nellcorp.com /home/assis/test
    ```
  7. figassis revised this gist Sep 16, 2019. 1 changed file with 16 additions and 3 deletions.
    19 changes: 16 additions & 3 deletions README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -3,7 +3,7 @@

    ## Install Required Linux Headers
    ```bash
    sudo apt-get -y install gcc make linux-headers-$(uname -r) dkms ufw
    sudo apt-get -y install gcc make linux-headers-$(uname -r) dkms ufw apt-transport-https ca-certificates curl software-properties-common
    ```

    ## Add VirtualBox Repository and key
    @@ -27,14 +27,24 @@ curl -O http://download.virtualbox.org/virtualbox/5.2.4/Oracle_VM_VirtualBox_Ext
    sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-5.2.4-119785.vbox-extpack
    ```

    ## Install Docker and Docker Compose
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
    sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
    sudo apt-get update && sudo apt-get -y install docker-ce
    sudo usermod -aG docker ${USER}
    sudo curl -L https://github.com/docker/compose/releases/download/1.17.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
    sudo chmod +x /usr/local/bin/docker-compose


    ## Configure vm autostart
    ### Place Autostart configuration files
    ```bash
    sudo mv virtualbox /etc/default/virtualbox
    sudo mv vbox.cfg /etc/vbox/vbox.cfg
    ```

    ### Lock down configurations and allow current user access (log out and back in again to refresh the permissions after this step)
    ### Lock down configurations and allow current user access
    Log out and back in again to refresh the permissions after this step. This will also allow you to use docker
    ```bash
    sudo chgrp vboxusers /etc/vbox
    sudo chmod 1775 /etc/vbox
    @@ -84,13 +94,16 @@ sudo ufw allow 22/tcp && sudo ufw allow 2222/tcp
    sudo ufw allow 80/tcp && sudo ufw allow 8080/tcp
    sudo ufw allow 443/tcp && sudo ufw allow 8443/tcp
    sudo ufw allow 53 && sudo ufw allow 8053
    sudo ufw allow 123/ucp && sudo ufw allow 8123/ucp
    sudo ufw allow 123/ucp && sudo ufw allow 8123/udp
    sudo ufw allow 2049 && sudo ufw allow 2049
    sudo ufw allow 111
    sudo ufw default deny
    sudo ufw enable
    ```

    ## Configure Docker to forward ports
    docker-compose up -d

    ## Modify the vm to use the new nat network and start at boot
    ```bash
    vboxmanage modifyvm storage_gateway --nic1 natnetwork --nat-network1 default --autostart-enabled on
  8. figassis revised this gist Sep 16, 2019. 1 changed file with 7 additions and 1 deletion.
    8 changes: 7 additions & 1 deletion docker-compose.yaml
    View file
    Original file line number Diff line number Diff line change
    @@ -2,6 +2,7 @@ version: "2.1"
    services:
    http:
    image: tecnativa/tcp-proxy
    restart: always
    ports:
    - 80:80
    - 443:443
    @@ -14,4 +15,9 @@ services:
    TALK:
    host_ip:8080
    host_ip:8443
    host_ip:8053
    host_ip:8053
    logging:
    driver: "json-file"
    options:
    max-size: "10M"
    max-file: "3"
  9. figassis revised this gist Sep 16, 2019. 1 changed file with 17 additions and 0 deletions.
    17 changes: 17 additions & 0 deletions docker-compose.yaml
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,17 @@
    version: "2.1"
    services:
    http:
    image: tecnativa/tcp-proxy
    ports:
    - 80:80
    - 443:443
    - 53:53
    environment:
    LISTEN:
    :80
    :443
    :53
    TALK:
    host_ip:8080
    host_ip:8443
    host_ip:8053
  10. figassis revised this gist Sep 16, 2019. 1 changed file with 127 additions and 0 deletions.
    127 changes: 127 additions & 0 deletions README.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,127 @@
    ## Update Ubuntu
    `sudo apt-get update && sudo apt-get dist-upgrade && sudo apt-get autoremove`

    ## Install Required Linux Headers
    ```bash
    sudo apt-get -y install gcc make linux-headers-$(uname -r) dkms ufw
    ```

    ## Add VirtualBox Repository and key
    ```bash
    wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
    wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
    ```

    ## Add VirtualBox repository
    ```bash
    sudo sh -c 'echo "deb http://download.virtualbox.org/virtualbox/debian $(lsb_release -sc) contrib" >> /etc/apt/sources.list'
    ```

    ## Install VirtualBox
    ```bash
    sudo apt-get update && apt-get -y install virtualbox-5.2
    ```
    ## Install VirtualBox Extension Pack
    ```bash
    curl -O http://download.virtualbox.org/virtualbox/5.2.4/Oracle_VM_VirtualBox_Extension_Pack-5.2.4-119785.vbox-extpack
    sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-5.2.4-119785.vbox-extpack
    ```

    ## Configure vm autostart
    ### Place Autostart configuration files
    ```bash
    sudo mv virtualbox /etc/default/virtualbox
    sudo mv vbox.cfg /etc/vbox/vbox.cfg
    ```

    ### Lock down configurations and allow current user access (log out and back in again to refresh the permissions after this step)
    ```bash
    sudo chgrp vboxusers /etc/vbox
    sudo chmod 1775 /etc/vbox
    sudo usermod -a -G vboxusers ${USER}

    VBoxManage setproperty autostartdbpath /etc/vbox
    sudo service vboxautostart-service restart
    ```

    ## Download Storage Gateway ESXi Image
    ```bash
    mkdir ~/images
    wget https://dslbbkfzjw91h.cloudfront.net/AWS-Appliance.zip && unzip AWS-Appliance.zip
    mv AWS-Appliance-2019-07-24-1563990364.ova ~/images/AWS-Appliance.ova

    ## .ova file might have a different name, use the correct name
    vboxmanage import ~/images/AWS-Appliance.ova --vsys 0 --vmname storage_gateway
    ```


    ## Create NAT Network, which as opposed to simple NAT, allows guest machines to communicate with each other
    ```bash
    VBoxManage natnetwork add --netname default --network "192.168.15.0/24" --enable --dhcp on
    VBoxManage natnetwork start --netname default
    ```

    ## Forward required ports for AWS Storage Gateway.
    Decide on a static IP to assign the appliance. 192.168.15.2 will do.
    Also, under a non root user, VirtualBox only forwards ports > 1024 (no privileged ports). Make sure
    all your ports are acceptable. Because we will need to connect to http port 80 on the host to activate
    the appliance, we will use a docker ha-proxy container to forward 80 to 8080.

    ```bash
    VBoxManage natnetwork modify --netname default --port-forward-4 "http:tcp:[]:8080:[192.168.15.2]:80"
    VBoxManage natnetwork modify --netname default --port-forward-4 "https:tcp:[]:8443:[192.168.15.2]:443"
    VBoxManage natnetwork modify --netname default --port-forward-4 "dnstcp:tcp:[]:8053:[192.168.15.2]:53"
    VBoxManage natnetwork modify --netname default --port-forward-4 "dnsudp:udp:[]:8053:[192.168.15.2]:53"
    VBoxManage natnetwork modify --netname default --port-forward-4 "ssh_support:tcp:[]:2222:[192.168.15.2]:22"
    VBoxManage natnetwork modify --netname default --port-forward-4 "ntp:udp:[]:8123:[192.168.15.2]:123"
    VBoxManage natnetwork modify --netname default --port-forward-4 "nfs:tcp:[]:2049:[192.168.15.2]:2049"
    VBoxManage natnetwork modify --netname default --port-forward-4 "nfsudp:udp:[]:2049:[192.168.15.2]:2049"
    ```

    ## Configure and enable UFW. Make sure to allow port 22 for ssh
    ```bash
    sudo ufw allow 22/tcp && sudo ufw allow 2222/tcp
    sudo ufw allow 80/tcp && sudo ufw allow 8080/tcp
    sudo ufw allow 443/tcp && sudo ufw allow 8443/tcp
    sudo ufw allow 53 && sudo ufw allow 8053
    sudo ufw allow 123/ucp && sudo ufw allow 8123/ucp
    sudo ufw allow 2049 && sudo ufw allow 2049
    sudo ufw allow 111
    sudo ufw default deny
    sudo ufw enable
    ```

    ## Modify the vm to use the new nat network and start at boot
    ```bash
    vboxmanage modifyvm storage_gateway --nic1 natnetwork --nat-network1 default --autostart-enabled on
    ```

    ## Optionally, check your VM info before starting it
    ```bash
    vboxmanage showvminfo storage_gateway
    ```

    ## Start the Appliance VM.
    ```bash
    vboxmanage startvm storage_gateway --type headless
    ```

    RDP server will be available on port 5961, which you can connect to using any RCP client.
    Likely, you've setup the VM on a linux server, so it has no GUI, and we've not allowed port 5961 on UFW.
    You will need to forward the port to your workstation via ssh tunnel, like so:

    ```bash
    ssh -i ssh_key_file -L 5961:127.0.0.1:5961 -fN remoteuser@remotehost
    ```

    You can now connect to the AWS appliance via an RDP client like Microsoft's Remote Desktop 10
    The client will ask for credentials, but the RDP connection is not secured, so type anything and connect
    When you see the Appliance's login screen, login with user: admin / password: password combination

    * Follow the AWS guide to setup your gateway: https://docs.amazonaws.cn/en_us/storagegateway/latest/userguide/manage-on-premises.html
    * After that is done, follow the getting started guide here: https://docs.aws.amazon.com/storagegateway/latest/userguide/GettingStarted.html

    ## VM Controls
    ```bash
    vboxmanage controlvm storage_gateway pause|resume|reset|poweroff|savestate
    ```
  11. figassis revised this gist Sep 16, 2019. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions vbox.cfg
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,2 @@
    # Allow users to autostart VMs
    default_policy = allow
  12. figassis created this gist Sep 16, 2019.
    3 changes: 3 additions & 0 deletions virtualbox
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,3 @@
    # virtualbox defaults file
    VBOXAUTOSTART_DB=/etc/vbox
    VBOXAUTOSTART_CONFIG=/etc/vbox/vbox.cfg