-
-
Save VaibhavPokale/910d3f0d7f58222fe8b64f9b0f99f2d3 to your computer and use it in GitHub Desktop.
AWS DevOps Engineer Professional Study Notes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| CI & CD: | |
| 2 core software development processes | |
| CI process of automating regular code commits followed by an automated build and test process designed to highlight intergration issues early. | |
| Additional tooling and functionality provided by Bamboo, CruiseControl, Jenkins, Go and TeamCity etc. | |
| workflow based | |
| CD takes the form of a workflow based process which accepts a tested software build payload from a CI server. Automates the deployment into a working QA, Pre-prod or Prod environment. | |
| AWS CodeDeploy and CodePipeline provide CI/CD services | |
| Elasticbeanstalk and CFN provide functionality which can be utilized by CI/CD servers. | |
| Deployment Types: | |
| 1. Single Target Deployment - small dev projects, legacy or non-HA infrastructure; outage occurs in case of failure, testing opportunity is limited. | |
| 2. All-at-Once Deployment - deployment happens on multiple targets, requires Orchestration tools, suitable for non critical apps in 5-10 range. | |
| 3. Minimum in-service Deployment - keeps min in-service targets and deploy in multiple stages, suitable for large environments, allow automated testing, no downtime | |
| 4. Rolling Deployments - x targets per stage, happens in multiple stages, after completion of stage 1, next stage begins, orchestration and health check required, can be least efficient if x is smaller, allow automated testing, no downtime if x is not large to impact application, can be paused, allowing multi-version testing. | |
| 5. Blue Green Deployment - Deploy to seperate Green environment, update the code on Green, extra cost due to duplicate env during deployment, Deployment is rapid, cutover and migration is clean(DNS Change), Rollback easy(DNS regression), can be fully automates using CFN etc. | |
| Binary, No Traffic Split, not used to feature test | |
| A/B Testing - distribution traffic between blue/green, allows gradual performance/stability/health analysis, allows new feature testing, rollback is quick, end goal of A/B testing is not migration, Uses Route 53 for DNS resolution, 2 records one pointing A, other pointing B, weighted/round robin. | |
| Bootstraping: process starts with base image (ISO/AMI) and via automation build on it to create more complex object. | |
| Cloud Init, CFN Init (complex bootstraping engine) | |
| AMI (Baking) based approach large number of AMIs even for a small system change. Ex: Quickstart | |
| Bootstraping - every option like OS, patches, dependencies, applications can be tweaked. Ex: Quick launch | |
| Immutable Architecture - Replace infra instead of upgrading or repairing faulty components, treat servers as unchangeable objects, don't diagnose and fix, throw away and re-create, Nothing bootstraped except AMI. | |
| Example: Pets vs Cattles | |
| Container/Docker: contains only application and dependencies, achieves higher density, and improved portability by removing per container Guest OS. | |
| Escape for dependency hell, Consistent progression from DEV->TEST->QA->PROD, Isolation, resource scheduling at micro level, Extreme code portability, Micro-services; Docker Image(Read Only), Docker container (hold everything required to run), Layers/UFS (combines layers using UFS), Dockerfile, Docker Daemon/Engine, Docker Client (Interface between user and Docker Engine), Docker Registry/Hub (Private/Public store for Docker images) | |
| CloudFormation: Building block service to provision infra within AWS, Elastic Beanstalk uses CFN, JSON format, Stack (CFN unit of grouping infra), Template (JSON doc giving instructions for CFN on how to act and what to create/update), Stack Policy (IAM style policy governs what/who can change, can be added via CLI or UI, updated but can not be removed) | |
| Create CFN template -> Add template to CFN -> Create CFNStack -> Resources (200 per template) -> Update template/Stack -> Delete Stack | |
| Template Anatomy: Parameters (pass variables into a template), Mappings (allow processing of hash's by CFN template), Resources, Outputs (results from template); Only Resources is mandatory. CFN can run scripts within instances, expand files within instances, stack id(unique). | |
| Use cases: | |
| 1. Template for manual deployment of bespoke infra. | |
| 2. create repeatable patterned environment | |
| 3. run automated testing for CI/CD environments | |
| 4. define an environment once, deployed | |
| 5. manage infra config using software development style versioning and testing concepts. | |
| IMPORTANT: Template should be designed to work 1 or 1000 app in one or more regions. | |
| CloudFormation Structure: | |
| CloudWatch: Metric gathering service, Monitoring/alerting service, A graphing service | |
| Metrics data will be available only for 2 weeks. Anything over 2 weeks should be stored outside. | |
| Namespaces - Ex: EBS, EC2, Check CloudWatch Developer Guide for all Namespaces. Enable detailed monitoring for additional metrics under Namespaces. | |
| Metrics can be aggregated across Autoscaling Services. | |
| Custom Metrics: | |
| Create an IAM role -> CloudWatch -> EC2 -> CloudWatch Full Access | |
| Create EC2 instance with CloudWatch Role -> SSH -> Install Python, Pip, Git and AWS CLI -> git clone https://github.com/ACloudGuru/resources; cd resources; | |
| cat time.sh; chmod a+x time.sh -> setup a cron job (crontab -e) */1 * * * * /home/ubuntu/resources/time.sh -> save -> CloudWatch Min granualrity 1 min -> | |
| Check the CloudWatch for new namespace with Unixtime metrics | |
| CloudWatch Alarms: | |
| Initiate actions (SNS notification) based on parameters specified against metrics available. Alarm period equal to metric frequency (5 min or 1 min). | |
| Alarm can't invoke actions unless state change. Alarms actions must be in same region as alarm. AWS resource wont send data under certain | |
| conditions (ex: EBS not attached to EC2 wont send data to CloudWatch) | |
| Alarm States: OK (metric match thresold) ALARM (metric outside thresold) INSUFFICIENT_DATA (metric not available/not enough) | |
| Limits: 5000 alarms per AWS account, create or update alarm using mon-put-metric-alarm, enable/disable using mon-[enable/disable]-alarm, | |
| Describe alarm using mon-describe-alarm. Alarm can be created before metrics created. | |
| date +%s -> Get Unix Time | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment