-
-
Save alekonko/c0c06a478aff5b0e8bf5b74265229dbb to your computer and use it in GitHub Desktop.
Revisions
-
PavelSusloparovNYT revised this gist
Jul 26, 2018 . 1 changed file with 82 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,82 @@ Create a new network in auto mode gcloud compute networks create mynetwork --subnet-mode=auto Create a new network in custom mode gcloud compute networks create privatenet --subnet-mode=custom Create a custom subnet gcloud compute networks subnets create privatesubnet --network=privatenet \ --region=us-central1 --range=10.0.0.0/24 --enable-private-ip-google-access Create instances in diffrent networks gcloud compute instances create default-us-vm --zone=us-central1-a --network=default gcloud compute instances create mynet-us-vm --zone=us-central1-a --network=mynetwork gcloud compute instances create mynet-eu-vm --zone=europe-west1-b --network=mynetwork gcloud compute instances create privatenet-bastion --zone=us-central1-c \ --subnet=privatesubnet --can-ip-forward gcloud compute instances create privatenet-us-vm --zone=us-central1-f \ --subnet=privatesubnet Allow/Ingress Rules gcloud beta compute firewall-rules create mynetwork-allow-icmp --network mynetwork \ --action ALLOW --direction INGRESS --rules icmp gcloud beta compute firewall-rules create mynetwork-allow-ssh --network mynetwork \ --action ALLOW --direction INGRESS --rules tcp:22 gcloud beta compute firewall-rules create mynetwork-allow-internal --network \ mynetwork --action ALLOW --direction INGRESS --rules all \ --source-ranges 10.128.0.0/9 gcloud beta compute firewall-rules list \ --filter="network:mynetwork" Deny/Egress Rules gcloud beta compute firewall-rules create privatenet-allow-icmp \ --network privatenet --action ALLOW --direction INGRESS --rules icmp gcloud beta compute firewall-rules create privatenet-allow-ssh \ --network privatenet --action ALLOW --direction INGRESS --rules tcp:22 gcloud beta compute firewall-rules create privatenet-allow-internal \ --network privatenet --action ALLOW --direction INGRESS --rules all \ --source-ranges 10.0.0.0/24 gcloud beta compute firewall-rules create mynetwork-deny-icmp \ --network mynetwork --action DENY --direction EGRESS --rules icmp \ --destination-ranges 10.132.0.2 --priority 500 gcloud beta compute firewall-rules list \ --filter="network:mynetwork AND name=mynetwork-deny-icmp" This rule was created with the direction EGRESS. Since ping is a bi-directional protocol, this will block ICMP as traffic leaves the VMs virtual NIC. If this rule were to be created with the INGRESS direction, the ICMP packets would be allowed to leave the VM's virtual NIC. ssh to mynet-us-vm ping mynet-eu-vm doesn't work anymore In this part of the lab you will convert the privatenet-bastion instance to a NAT gateway so privatenet-us-vm can talk to the Internet without having an IP assigned. Within privatenet, are 2 instances: privatenet-bastion and privatenet-us-vm. Both have a public IP, but you will remove the public IP from privatenet-us-vm. After the public IP is removed, you can SSH into privatenet-us-vm through private-net-bastion. You might want to use a NAT gateway either for additional filtering or if you want to egress from specific static IP addresses. In this case we just set it up without specific functionality. ssh privatenet-us-vm ping www.starwars.com curl --head www.starwars.com gsutil ls gs://gcp-next2017-security-bootcamp/README gsutil cat gs://gcp-next2017-security-bootcamp/README None of this command works Create a NAT gateway gcloud compute instances add-tags privatenet-us-vm --zone us-central1-f --tags nat-me gcloud compute routes create nat-route --network privatenet \ --destination-range 0.0.0.0/0 --next-hop-instance privatenet-bastion \ --next-hop-instance-zone us-central1-c --tags nat-me --priority 800 ssh to private-bashtion sudo sysctl -w net.ipv4.ip_forward=1 sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE ssh privatenet-us-vm Ping works ping www.starwars.com curl --head www.starwars.com curl --head www.starwars.com/blahz gsutil ls gs://gcp-next2017-security-bootcamp/README gsutil cat gs://gcp-next2017-security-bootcamp/README curl ifconfig.co NAT is working: privetnet-us-vm appears to the outside from privatenet-bastion IP address. -
Jul 26, 2018 . No changes.There are no files selected for viewing
-
Jul 26, 2018 . 1 changed file with 93 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,93 @@ Create custom network gcloud compute networks create taw-custom-network --subnet-mode custom Create subnet-us-central subnet gcloud compute networks subnets create subnet-us-central \ --network taw-custom-network \ --region us-central1 \ --range 10.0.0.0/16 Create subnet-europe-west subnet gcloud compute networks subnets create subnet-europe-west \ --network taw-custom-network \ --region europe-west1 \ --range 10.1.0.0/16 Create subnet-asia-east subnet gcloud compute networks subnets create subnet-asia-east \ --network taw-custom-network \ --region asia-east1 \ --range 10.2.0.0/16 List networks gcloud compute networks subnets list \ --network taw-custom-network Create a firewall rule. 0.0.0.0/0 - firewall open for any IP addresses from the internet gcloud compute firewall-rules create nw101-allow-http \ --allow tcp:80 --network taw-custom-network --source-ranges 0.0.0.0/0 \ --target-tags http ICMP firewall gcloud compute firewall-rules create "nw101-allow-icmp" --allow icmp --network "taw-custom-network" --target-tags rules Internal Communication firewall gcloud compute firewall-rules create "nw101-allow-internal" --allow tcp:0-65535,udp:0-65535,icmp --network "taw-custom-network" --source-ranges "10.0.0.0/16","10.2.0.0/16","10.1.0.0/16" SSH gcloud compute firewall-rules create "nw101-allow-ssh" --allow tcp:22 --network "taw-custom-network" --target-tags "ssh" RDP gcloud compute firewall-rules create "nw101-allow-rdp" --allow tcp:3389 --network "taw-custom-network" VMs creation gcloud compute instances create us-test-01 \ --subnet subnet-us-central \ --zone us-central1-a \ --tags ssh,http,rules gcloud compute instances create europe-test-01 \ --subnet subnet-europe-west \ --zone europe-west1-b \ --tags ssh,http,rules gcloud compute instances create asia-test-01 \ --subnet subnet-asia-east \ --zone asia-east1-a \ --tags ssh,http,rules Internal DNS hostName.c.PROJECT_ID.internal Example: SSH on the machine and run ping -c 3 asia-test-01.c.qwiklabs-gcp-e639f64b367ff562.internal Network perfomance test sudo apt-get update sudo apt-get -y install traceroute mtr tcpdump iperf whois host dnsutils siege Traceroute: traceroute www.icann.org increase max TTL traceroute -m 255 bad.horse Iperf us-test-01 iperf -s #run in server mode europe-test-01 iperf -c us-test-01 #run in client mode, connection to eu1-vm Create us-test-02 gcloud compute instances create us-test-02 \ --subnet subnet-us-central \ --zone us-central1-b \ --tags ssh,http iperf -s -u #iperf server side iperf -c europe-test-01 -u -b 2G #iperf client side - send 2 Gbits/s iperf -c us-test-01 -P 20 #parallel mode -
Jul 25, 2018 . 1 changed file with 70 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,70 @@ gcloud auth list gcloud config list project gcloud config set compute/zone us-central1-a gcloud config set compute/region us-central1 cat << EOF > startup.sh #! /bin/bash apt-get update apt-get install -y nginx service nginx start sed -i -- 's/nginx/Google Cloud Platform - '"\$HOSTNAME"'/' /var/www/html/index.nginx-debian.html EOF gcloud compute instance-templates create nginx-template \ --metadata-from-file startup-script=startup.sh gcloud compute target-pools create nginx-pool gcloud compute instance-groups managed create nginx-group \ --base-instance-name nginx \ --size 2 \ --template nginx-template \ --target-pool nginx-pool gcloud compute instances list gcloud compute firewall-rules create www-firewall --allow tcp:80 Create a network Load Balancer gcloud compute forwarding-rules create nginx-lb \ --region us-central1 \ --ports=80 \ --target-pool nginx-pool gcloud compute forwarding-rules list You can then visit the load balancer from the browser http://IP_ADDRESS/ where IP_ADDRESS is the address shown as the result of running the previous command. Create a HTTP Load Balancer Create healthchecks. gcloud compute http-health-checks create http-basic-check Define an HTTP service and map a port name to the relevant port for the instance group. gcloud compute instance-groups managed \ set-named-ports nginx-group \ --named-ports http:80 Create a backend service: gcloud compute backend-services create nginx-backend \ --protocol HTTP --http-health-checks http-basic-check --global Add the instance group into the backend service: gcloud compute backend-services add-backend nginx-backend \ --instance-group nginx-group \ --instance-group-zone us-central1-a \ --global Create a default URL map that directs all incoming requests to all your instances: gcloud compute url-maps create web-map \ --default-service nginx-backend Create a target HTTP proxy to route requests to your URL map: gcloud compute target-http-proxies create http-lb-proxy \ --url-map web-map Create a global forwarding rule to handle and route incoming requests. gcloud compute forwarding-rules create http-content-rule \ --global \ --target-http-proxy http-lb-proxy \ --ports 80 gcloud compute forwarding-rules list -
Jul 25, 2018 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,60 @@ https://cloud.google.com/storage/docs/encryption/ BUCKET_NAME=YOUR_NAME_enron_corpus gsutil mb gs://${BUCKET_NAME} gsutil cp gs://enron_corpus/allen-p/inbox/1. . tail 1. Enable API/Create keyring gcloud services enable cloudkms.googleapis.com KEYRING_NAME=test CRYPTOKEY_NAME=qwiklab gcloud kms keyrings create $KEYRING_NAME --location global gcloud kms keys create $CRYPTOKEY_NAME --location global \ --keyring $KEYRING_NAME \ --purpose encryption Encrypt 1 file PLAINTEXT=$(cat 1. | base64 -w0) curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:encrypt" \ -d "{\"plaintext\":\"$PLAINTEXT\"}" \ -H "Authorization:Bearer $(gcloud auth application-default print-access-token)"\ -H "Content-Type: application/json" curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:encrypt" \ -d "{\"plaintext\":\"$PLAINTEXT\"}" \ -H "Authorization:Bearer $(gcloud auth application-default print-access-token)"\ -H "Content-Type:application/json" \ | jq .ciphertext -r > 1.encrypted curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:decrypt" \ -d "{\"ciphertext\":\"$(cat 1.encrypted)\"}" \ -H "Authorization:Bearer $(gcloud auth application-default print-access-token)"\ -H "Content-Type:application/json" \ | jq .plaintext -r | base64 -d gsutil cp 1.encrypted gs://${BUCKET_NAME} IAM permissions USER_EMAIL=$(gcloud auth list --limit=1 2>/dev/null | grep '@' | awk '{print $2}') gcloud kms keyrings add-iam-policy-binding $KEYRING_NAME \ --location global \ --member user:$USER_EMAIL \ --role roles/cloudkms.admin gcloud kms keyrings add-iam-policy-binding $KEYRING_NAME \ --location global \ --member user:$USER_EMAIL \ --role roles/cloudkms.cryptoKeyEncrypterDecrypter Encrypt bulk gsutil -m cp -r gs://enron_corpus/allen-p . MYDIR=allen-p FILES=$(find $MYDIR -type f -not -name "*.encrypted") for file in $FILES; do PLAINTEXT=$(cat $file | base64 -w0) curl -v "https://cloudkms.googleapis.com/v1/projects/$DEVSHELL_PROJECT_ID/locations/global/keyRings/$KEYRING_NAME/cryptoKeys/$CRYPTOKEY_NAME:encrypt" \ -d "{\"plaintext\":\"$PLAINTEXT\"}" \ -H "Authorization:Bearer $(gcloud auth application-default print-access-token)" \ -H "Content-Type:application/json" \ | jq .ciphertext -r > $file.encrypted done gsutil -m cp allen-p/inbox/*.encrypted gs://${BUCKET_NAME}/allen-p/inbox