aimrbrto · August 20, 2018 19:29 · Apr 30, 2017 · Apr 30, 2017
diff --git a/GCLOUD-EXAMPLE.md b/GCLOUD-EXAMPLE.md
@@ -0,0 +1,57 @@
+# Postal on Kubernetes in Google Cloud Platform
+
+## Requirements
+
+1. Google Cloud Platform (GCP) account.
+1. A project in the account (we're using `postal-165921` in our example commands).
+
+## Infrastructure
+
+1. Set the defailt zone to use with our `gcloud` commands: `gcloud config set compute/zone us-central1-a`
+1. `gcloud config set container/cluster postal`
+1. Navigate to https://console.cloud.google.com/ and create a new project for Postal (our example one will be called Postal and has an ID of `postal-165921`).
+1. Navigate to the Container Engine section and create a new cluster called `postal`. The command we are using to create is `gcloud container --project "postal-165921" clusters create "postal" --zone "us-central1-a" --machine-type "n1-standard-1" --image-type "COS" --disk-size "100" --scopes "https://www.googleapis.com/auth/compute","https://www.googleapis.com/auth/devstorage.read_only","https://www.googleapis.com/auth/logging.write","https://www.googleapis.com/auth/servicecontrol","https://www.googleapis.com/auth/service.management.readonly","https://www.googleapis.com/auth/trace.append" --num-nodes "3" --network "default" --enable-cloud-logging --no-enable-cloud-monitoring`
+1. In a moment you should be able to run `gcloud container clusters get-credentials postal --zone us-central1-a --project postal-165921` to set up your local `kubectl` credentials.
+1. Create a MySQL 5.7 instance in the SQL console. Our example is called `postal`.
+1. In IAM and Admin create a service account. Role will be SQL Client and check to Furnish new private key (JSON type). Save the key somewhere locally.
+1. Create the proxy user with `gcloud beta sql users create postal cloudsqlproxy~% --instance=postal --password=p0st4l`
+1. Run `gcloud sql instances describe postal` to get the connectionName. Our example is `postal-165921:us-central1:postal`.
+1. `kubectl create secret generic cloudsql-instance-credentials --from-file=credentials.json=/Users/andy/Documents/Postal-d609301cc404.json`
+1. `kubectl create secret generic cloudsql-db-credentials --from-literal=username=postal --from-literal=password=p0st4l`
+1. Create a `grants.sql` file with the following contents:
+```
+CREATE DATABASE `postal` CHARSET utf8mb4 COLLATE utf8mb4_unicode_ci;
+GRANT ALL ON `postal`.* TO `postal`@`cloudsqlproxy~%`;
+GRANT ALL PRIVILEGES ON `postal-%` . * to `postal`@`cloudsqlproxy~%`;
+```
+1. Create a new storage bucket and upload the `grants.sql` file to it.
+1. Import this SQL file into the new `postal` database.
+1. Enable Google Cloud SQL API at https://console.developers.google.com/apis/api/sqladmin.googleapis.com/overview?project=postal-165921&duration=PT1H.
+1. Create two external IP addresses (`mx1` and `mx2`) for later use in SMTP load balancers.
+
+# Secrets
+
+1. Generate a SSL certificate using `openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=postaldemo/O=postaldemo"`.
+1. `kubectl create secret tls postal-tls --key tls.key --cert tls.crt`
+
+# Application
+
+1. Create your configuration map using `kubectl create configmap postal-config --from-file ~/.postal --dry-run -o yaml`. This assumes you already have the files from the `postal initialize-config` command living at `~/.postal`.
+
+# DNS
+
+We are using the domain `postaldemo.com` for our demo. We are going to host it in GCP.
+
+1. Create the zone: `gcloud dns managed-zones create postaldemo --dns-name postaldemo.com --description "Postal demonstration domain"`.
+1. Get the name servers to use at the registrar using `gcloud dns managed-zones describe postaldemo` and update your registrar name server records.
+1. Set static IPs for MX1 and MX2 to the instances in the SMTP server pool: `gcloud compute addresses create mx1 mx2 --addresses $(gcloud compute instances list --filter='name:postal-smtp' --format='value[terminator=","](networkInterfaces[0].accessConfigs[0].natIP)')`
+1. Create MX1: `gcloud dns record-sets transaction add --zone postaldemo --type A --name mx1.postaldemo.com. --ttl 300 $(gcloud compute addresses describe mx1 --format "value(address)")`.
+1. Create MX2: `gcloud dns record-sets transaction add --zone postaldemo --type A --name mx2.postaldemo.com. --ttl 300 $(gcloud compute addresses describe mx2 --format "value(address)")`.
+1. Create SMTP endpoint: `gcloud dns record-sets transaction add --zone postaldemo --type CNAME --name smtp.postaldemo.com. --ttl 300 $(gcloud compute addresses describe mx2 --format "value(address)") $(gcloud compute addresses describe mx1 --format "value(address)")`.
+
+## Future
+
+* Convert all the initial GCP stuff to Terraform.
+* Use https://github.com/kubernetes-incubator/external-dns instead of manual DNS.
+* Figure out if we can use a PersistentVolume in a job for read/write and then read-only in the pods (for assets). Currently using a gcePersistentDisk for this instead.
+* How to add 443 as Ingress controllers frontend ports.
diff --git a/assets-job.yaml b/assets-job.yaml
@@ -0,0 +1,31 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: postal-assets
+spec:
+  template:
+    metadata:
+      name: postal-assets
+      labels:
+        name: postal-assets
+    spec:
+      restartPolicy: Never
+      containers:
+        - image: andyshinn/postal-app
+          name: assets
+          command: ["bundle", "exec", "rake", "assets:precompile"]
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+            - mountPath: /usr/src/app/public/assets
+              name: assets
+              readOnly: false
+      restartPolicy: Never
+      volumes:
+        - name: config
+          configMap:
+            name: postal-config
+        - name: assets
+          gcePersistentDisk:
+            pdName: postal-assets
+            fsType: ext4
diff --git a/cron-deployment.yaml b/cron-deployment.yaml
@@ -0,0 +1,29 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: cron
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        postal.app: cron
+    spec:
+      containers:
+        - image: andyshinn/postal-app
+          name: cron
+          command: ["bundle", "exec", "rake", "postal:cron"]
+          ports:
+            - containerPort: 2525
+          resources: {}
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+      restartPolicy: Always
+      volumes:
+        - name: config
+          configMap:
+            name: postal-config
+status: {}
diff --git a/fast-deployment.yaml b/fast-deployment.yaml
@@ -0,0 +1,30 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: fast
+spec:
+  replicas: 2
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        postal.app: fast
+    spec:
+      containers:
+        - image: andyshinn/postal-app
+          name: fast
+          command: ["bundle", "exec", "rake", "postal:fast_server"]
+          ports:
+            - containerPort: 5010
+              protocol: TCP
+            - containerPort: 5011
+              protocol: TCP
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+      restartPolicy: Always
+      volumes:
+        - name: config
+          configMap:
+            name: postal-config
diff --git a/fast-service.yaml b/fast-service.yaml
@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    postal.app: fast
+  name: fast
+spec:
+  clusterIP: None
+  ports:
+  - name: headless
+    port: 55555
+    targetPort: 0
+  selector:
+    postal.app: fast
+status:
+  loadBalancer: {}
diff --git a/initialize-job.yaml b/initialize-job.yaml
@@ -0,0 +1,45 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: postal-initialize
+spec:
+  template:
+    metadata:
+      name: postal-initialize
+      labels:
+        name: postal-initialize
+    spec:
+      restartPolicy: Never
+      containers:
+        - image: gcr.io/cloudsql-docker/gce-proxy:1.09
+          name: cloudsql-proxy
+          command: ["/cloud_sql_proxy", "--dir=/cloudsql",
+                    "-instances=postal-165921:us-central1:postal=tcp:3306",
+                    "-credential_file=/secrets/cloudsql/credentials.json"]
+          volumeMounts:
+            - name: cloudsql-instance-credentials
+              mountPath: /secrets/cloudsql
+              readOnly: true
+            - name: ssl-certs
+              mountPath: /etc/ssl/certs
+            - name: cloudsql
+              mountPath: /cloudsql
+        - image: andyshinn/postal-app
+          name: initialize
+          command: ["bash", "-c", "sleep 10 && bundle exec rake db:schema:load db:seed"]
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+      restartPolicy: Never
+      volumes:
+        - name: config
+          configMap:
+            name: postal-config
+        - name: cloudsql-instance-credentials
+          secret:
+            secretName: cloudsql-instance-credentials
+        - name: ssl-certs
+          hostPath:
+            path: /etc/ssl/certs
+        - name: cloudsql
+          emptyDir:
diff --git a/mx-services.yaml b/mx-services.yaml
@@ -0,0 +1,27 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: mx1
+spec:
+  ports:
+    - port: 25
+      protocol: TCP
+      targetPort: 2525
+  selector:
+    postal.app: smtp
+  type: LoadBalancer
+  loadBalancerIP: 130.211.122.23
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: mx2
+spec:
+  ports:
+    - port: 25
+      protocol: TCP
+      targetPort: 2525
+  selector:
+    postal.app: smtp
+  type: LoadBalancer
+  loadBalancerIP: 104.198.75.93
diff --git a/rabbitmq-deployment.yaml b/rabbitmq-deployment.yaml
@@ -0,0 +1,56 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  creationTimestamp: null
+  name: rabbitmq
+spec:
+  replicas: 3
+  strategy: {}
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        postal.app: rabbitmq
+    spec:
+      containers:
+        - image: andyshinn/postal-rabbitmq
+          env:
+            - name: MY_POD_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
+            - name: MY_POD_IP
+              valueFrom:
+                fieldRef:
+                  fieldPath: status.podIP
+            - name: RABBITMQ_NODENAME
+              value: "rabbit@$(MY_POD_IP)"
+            - name: RABBITMQ_USE_LONGNAME
+              value: "true"
+            - name: AUTOCLUSTER_TYPE
+              value: k8s
+            - name: RABBITMQ_ERLANG_COOKIE
+              value: PHIOCHASOUQUIAXUFIETH
+            - name: RABBITMQ_DEFAULT_PASS
+              valueFrom:
+                secretKeyRef:
+                  name: postal-secrets
+                  key: rabbitpassword
+            - name: RABBITMQ_DEFAULT_USER
+              value: postal
+            - name: RABBITMQ_DEFAULT_VHOST
+              value: postal
+          name: rabbitmq
+          ports:
+            - name: management
+              containerPort: 15672
+            - name: amqp
+              containerPort: 5672
+          resources: {}
+          livenessProbe:
+            exec:
+              command: ["rabbitmqctl", "node_health_check"]
+            initialDelaySeconds: 15
+            periodSeconds: 15
+      restartPolicy: Always
+status: {}
diff --git a/rabbitmq-service.yaml b/rabbitmq-service.yaml
@@ -0,0 +1,19 @@
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    postal.app: rabbitmq
+  name: rabbitmq
+spec:
+  ports:
+  - name: "http"
+    port: 15672
+    targetPort: 15672
+  - name: "amqp"
+    port: 5672
+    targetPort: 5672
+  selector:
+    postal.app: rabbitmq
+status:
+  loadBalancer: {}
diff --git a/requeuer-deployment.yaml b/requeuer-deployment.yaml
@@ -0,0 +1,47 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: requeuer
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        postal.app: requeuer
+    spec:
+      containers:
+        - image: gcr.io/cloudsql-docker/gce-proxy:1.09
+          name: cloudsql-proxy
+          command: ["/cloud_sql_proxy", "--dir=/cloudsql",
+                    "-instances=postal-165921:us-central1:postal=tcp:3306",
+                    "-credential_file=/secrets/cloudsql/credentials.json"]
+          volumeMounts:
+            - name: cloudsql-instance-credentials
+              mountPath: /secrets/cloudsql
+              readOnly: true
+            - name: ssl-certs
+              mountPath: /etc/ssl/certs
+            - name: cloudsql
+              mountPath: /cloudsql
+        - image: andyshinn/postal-app
+          name: requeuer
+          command: ["bundle", "exec", "rake", "postal:requeuer"]
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+      restartPolicy: Always
+      volumes:
+        - name: config
+          configMap:
+            name: postal-config
+        - name: cloudsql-instance-credentials
+          secret:
+            secretName: cloudsql-instance-credentials
+        - name: ssl-certs
+          hostPath:
+            path: /etc/ssl/certs
+        - name: cloudsql
+          emptyDir:
+status: {}
diff --git a/smtp-deployment.yaml b/smtp-deployment.yaml
@@ -0,0 +1,34 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  creationTimestamp: null
+  name: smtp
+spec:
+  replicas: 2
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        postal.app: smtp
+    spec:
+      containers:
+        - image: andyshinn/postal-app
+          name: smtp
+          command: ["bundle", "exec", "rake", "postal:smtp_server"]
+          ports:
+            - containerPort: 2525
+              hostPort: 25
+          resources: {}
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+      restartPolicy: Always
+      volumes:
+        - name: config
+          configMap:
+            name: postal-config
+      nodeSelector:
+        cloud.google.com/gke-nodepool: smtp
+status: {}
diff --git a/smtp-service.yaml b/smtp-service.yaml
@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    postal.app: smtp
+  name: smtp
+spec:
+  ports:
+    - name: smtp
+      protocol: TCP
+      port: 25
+      targetPort: 2525
+  externalIPs:
+    - 130.211.185.93
+    - 104.154.77.75
+  selector:
+    postal.app: smtp
diff --git a/web-ingress.yaml b/web-ingress.yaml
@@ -0,0 +1,13 @@
+apiVersion: extensions/v1beta1
+kind: Ingress
+metadata:
+  name: web-external
+  annotations:
+    #kubernetes.io/ingress.global-static-ip-name: "test-ip"
+    kubernetes.io/ingress.class: "gce"
+spec:
+  tls:
+    - secretName: postal-secret
+  backend:
+    serviceName: web
+    servicePort: 80
diff --git a/web-service.yaml b/web-service.yaml
@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    postal.app: web
+  name: web
+spec:
+  ports:
+    - name: "web"
+      port: 80
+      targetPort: 5000
+  selector:
+    postal.app: web
+  type: NodePort
+status:
+  loadBalancer: {}
diff --git a/worker-deployment.yaml b/worker-deployment.yaml
@@ -0,0 +1,47 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: worker
+spec:
+  replicas: 2
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      labels:
+        postal.app: worker
+    spec:
+      containers:
+        - image: gcr.io/cloudsql-docker/gce-proxy:1.09
+          name: cloudsql-proxy
+          command: ["/cloud_sql_proxy", "--dir=/cloudsql",
+                    "-instances=postal-165921:us-central1:postal=tcp:3306",
+                    "-credential_file=/secrets/cloudsql/credentials.json"]
+          volumeMounts:
+            - name: cloudsql-instance-credentials
+              mountPath: /secrets/cloudsql
+              readOnly: true
+            - name: ssl-certs
+              mountPath: /etc/ssl/certs
+            - name: cloudsql
+              mountPath: /cloudsql
+        - image: andyshinn/postal-app
+          name: worker
+          command: ["bundle", "exec", "rake", "postal:worker"]
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+      restartPolicy: Always
+      volumes:
+        - name: config
+          configMap:
+            name: postal-config
+        - name: cloudsql-instance-credentials
+          secret:
+            secretName: cloudsql-instance-credentials
+        - name: ssl-certs
+          hostPath:
+            path: /etc/ssl/certs
+        - name: cloudsql
+          emptyDir:
+status: {}
diff --git a/web-deployment.yaml b/web-deployment.yaml
@@ -0,0 +1,77 @@
+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  creationTimestamp: null
+  name: web
+spec:
+  replicas: 2
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        postal.app: web
+    spec:
+      containers:
+        - image: gcr.io/cloudsql-docker/gce-proxy:1.09
+          name: cloudsql-proxy
+          command: ["/cloud_sql_proxy", "--dir=/cloudsql",
+                    "-instances=postal-165921:us-central1:postal=tcp:3306",
+                    "-credential_file=/secrets/cloudsql/credentials.json"]
+          volumeMounts:
+            - name: cloudsql-instance-credentials
+              mountPath: /secrets/cloudsql
+              readOnly: true
+            - name: ssl-certs
+              mountPath: /etc/ssl/certs
+            - name: cloudsql
+              mountPath: /cloudsql
+        - image: andyshinn/postal-app
+          name: web
+          command: ["bundle", "exec", "puma", "-C", "config/puma.rb"]
+          env:
+            - name: RAILS_SERVE_STATIC_FILES
+              value: "true"
+          ports:
+            - containerPort: 5000
+          resources: {}
+          livenessProbe:
+            httpGet:
+              path: /login
+              port: 5000
+            initialDelaySeconds: 15
+            periodSeconds: 15
+          readinessProbe:
+            httpGet:
+              path: /login
+              port: 5000
+            initialDelaySeconds: 15
+            periodSeconds: 15
+          volumeMounts:
+            - mountPath: /opt/postal/config
+              name: config
+            - mountPath: /usr/src/app/public/assets
+              name: assets
+              readOnly: true
+      restartPolicy: Always
+      nodeSelector:
+        cloud.google.com/gke-nodepool: default-pool
+      volumes:
+        - name: assets
+          gcePersistentDisk:
+            pdName: postal-assets
+            fsType: ext4
+            readOnly: true
+        - name: config
+          configMap:
+            name: postal-config
+        - name: cloudsql-instance-credentials
+          secret:
+            secretName: cloudsql-instance-credentials
+        - name: ssl-certs
+          hostPath:
+            path: /etc/ssl/certs
+        - name: cloudsql
+          emptyDir:
+status: {}
No results found