Skip to content

Instantly share code, notes, and snippets.

@FranklinHarry

FranklinHarry/deployment-tool-ansible-puppet-chef-salt.md

Forked from jaceklaskowski/deployment-tool-ansible-puppet-chef-salt.md
Created June 6, 2018 03:25
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save FranklinHarry/6be6c956a7505560df56a80b51ac7fca to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save FranklinHarry/6be6c956a7505560df56a80b51ac7fca to your computer and use it in GitHub Desktop.
Download ZIP
Choosing a deployment tool - ansible vs puppet vs chef vs salt
Raw
deployment-tool-ansible-puppet-chef-salt.md

Requirements

  • no upfront installation/agents on remote/slave machines - ssh should be enough
  • application components should use third-party software, e.g. HDFS, Spark's cluster, deployed separately
  • configuration templating
  • environment requires/asserts, i.e. we need a JVM in a given version before doing deployment
  • deployment process run from Jenkins

Solution

Ansible (perhaps with Docker + Vagrant)

FIXME: Why?

Candidates and management summary

Ansible, Chef, Puppet, Salt

  • config management tool
  • main difference between them: the language
  • Both Puppet and Chef are both languages that allow you to write scripts to quickly provision servers (including instances of Vagrant and/or Docker). You don't need to use Puppet or Chef to setup these services, but sometimes they can be a quick way to do so.

Docker and Vagrant are mentioned, too, for similar requirements.

  • approach - Containers vs Hypervisors (VMs)
    • V for Virtualization
    • C for Containerization
  • With Vagrant you might spin up a few CentOS boxes and install nginx on them, with Docker you might just spin up a few nginx instances without the overhead of the entire VM.
  • Vagrant gives a new VM to work within - all components inside are the same, regardless of developer's local machine setup
  • Docker is a two part shell/management layer for building and running virtual linux containers, based on lxc.
  • I always say that in Development/Test and Staging, you always have to test your application against the current production environment (including configuration) and whatever potential alternate production environment. Are you planning a security update? Do you want to update or switch your Java stack?
  • This is where Vagrant and Docker shine. I would expect Docker to help you speed up testing against multiple operating environments.
  • Vagrant abstracts the machine, while Docker abstracts the application.
    • When you need a throwaway machine, use Vagrant. If you want a throwaway application, use Docker.
    • Why would you need a throwaway machine for an application? When you want to do anything sophisticated with networking or hardware.
    • If you're not sure, use Vagrant.
  • In addition to the factors mentioned above, one significant factor is the host operating system. If you are on a Windows hosts, then it rules out docker at a native level.
  • Vagrant on Docker = Docker provider in Vagrant
    • https://docs.vagrantup.com/v2/docker/basics.html
    • Beyond the 'use both' solutions already given, which I have done on projects before and can agree is viable; there's an argument for docker in that 'Everyone's doing it' So you'd have a lot of support from a really great community. I've found vagrant community sparse at times.
  • Vagrant with a VM based backend like vSphere or VirtualBox vs. Fig with Docker.

Found on the net

Vocabulary

  • the tools = ansible, chef, puppet and salt
  • provisioning

What is difference between docker, puppet, chef and vagrant?:

  • the tools turn the configuration of an environment into source code
  • when the environment configuration becomes code it can then be managed from within a VCS such as git or SVN so that changes are attempted, shared, rolled forward and rolled back in a much more frictionless way than the traditional written specification documents or word-of-mouth configuration sharing (e.g. do this .... now try that ... no, OK then try this)
  • Gene Kim's book The Phoenix Project recommended
  • CFEngine mentioned - review, too?
  • configuration management tools
  • Puppet: This solution seems to appeal mostly to operations teams with little to no development background
    • a Puppet Master = a state server to track your infrastructure
    • While Puppet can be extended using the Ruby language, it is not terribly easy to do so
    • Puppet is difficult to pick up.
  • Chef: This solution resonates best with teams that, while not developers, are familiar with unit and integration testing, use of source control and other developer tooling.
    • highly mature and works at massive scale due to its adoption by Facebook which also has contributed
    • extensible using the Ruby language
    • Chef is very difficult to learn, though the exceptionally verbose output of a convergence run eases the identification and rectification of problems.
  • Ansible: This solution is by far the simplest of systems and appeals greatly to front line developers who often moonlight as their companies operations folks.
    • written in Python, so has a certain attraction to the Python community
    • If you are considering configuration management for the first time ever and need an easy win, Ansible is good place to start.
  • not familiar with Salt Stack and CF Engine.
  • Docker as a way to package code into consistent units of work.
    • These units of work can then be deployed to testing, QA and production environments with far greater ease.
    • Docker needs to only to express the configuration for a single process, the problem becomes far easier. The Dockerfile is, thus, a bit more than a simple bash script for configuring a process with its dependencies.
    • Docker also brings artifact management along with it via the public Docker Hub. This can be thought of as a correlate to public Github repositories. When a developer writes code and packages it with a Docker the can push this to the Docker hub to be shared as a binary artifact. The ramifications of this are quite extensive. This artifact can be tested for function, performance and security as a "black box" without needing to know what the contents of the "box" (container really) are.
    • Docker also brings artifact management along with it via the public Docker Hub. This can be thought of as a correlate to public Github repositories. When a developer writes code and packages it with a Docker the can push this to the Docker hub to be shared as a binary artifact. The ramifications of this are quite extensive. This artifact can be tested for function, performance and security as a "black box" without needing to know what the contents of the "box" (container really) are.
    • Docker is revolutionary in its scope because of the combined container, packaging and artifact management it seeks to employ.
    • still very weak in production due to difficulties in networking, identity management and data persistence = consider Docker, but be sure to use it in the real world with your eyes wide open.
  • The Configuration Management solutions all share a complexity that reflects the difficulty of configuring bare metal and virtual machines.
    • not only launching them, but also modifying the state of their configuration -- often with the release of new software.
  • Vagrant is a way to use Oracle's Virtual Box or VMWare Fusion on a developer workstation for the purpose of creating disposable and shareable development environments.
    • Vagrant does not compare to Puppet, Chef or Docker, Vagrant is meant to be used with them.
    • Vagrant takes the entire description of your development environment and couches it in a Ruby file. This means your development environment configuration is code and can be shared, rolled back and rolled forward with ease.
    • You, the developer, are free to try new and innovative things such as that new, awesome Java package or the latest version of PHP without worrying that a failure might take you days to set up or unravel.
    • Vagrant is a killer development application, and should be considered by nearly every development team.
    • Vagrant + Chef == Docker
  • First consider your situation in terms of the paradigm of production deployment (Containers vs. VMs). If you chose containers, then Docker/Boot2Docker is a good choice as it has a great deal of momentum right now and its ecosystem is beginning to grow.
  • Puppet and Chef are configuration management tools = keep all of your server configuration in a central place.

Tools at glance

Ansible

  • Seems the choice of many that gets me curious what others offer in the space.
  • Nie wymaga agenta na maszynach docelowych, ale wymaga na nich Pythona. Opisuje "stany docelowe", a nie czynności.
  • using SSH, with no agents to install on remote systems.

Chef

  • Chef uses a subset of Ruby, and server configurations are written as a series of steps. e.g. install apache, enable authn_ldap module, etc.

Puppet

  • Puppet uses a DSL (Ruby-based) that tries to be declarative, as in, ensure a machine is in a certain state, but not done in any particular order.

Vagrant

  • Vagrant is a tool for quickly spinning up virtual machines.
  • There's a configuration file (again in Ruby) that specifies what vm image to start with, what additional tasks to run, how the network should be configured, and so on. Vagrant has pretty good support for running configuration management software as well.
  • One place Vagrant really shines is setting up test environments. You're using Ubuntu Trusty with Apache 2.4 in production? Just spin up a few equivalent boxes locally with the same configuration. Using something like Beaker (another Puppetlabs project), you can even test your Puppet code itself!
  • Another use case for Vagrant is as a provisioning tool, since you're not technically limited to the default Virtualbox provider. You can use it with XEN, KVM, EC2, basically whatever. However, there are probably better tools for this use case. Razor and the Foreman come to mind.
  • Vagrant is a Virtual Machine which uses another service (such as VirtualBox or AWS) as its provider. You can launch many different types of Virtual Environments with Vagrant, but the most common is a Linux server.
  • Vagrant on the other hand is a wonderful tool for automatically provisioning multiple virtual machines each with their own configurations managed with puppet and/or chef. For its virtualisation it can use different providers. Originally the default provider was virtualbox, but it now supports many more, including vmware fusion and even amazon-ec2.
  • Interestingly, Vagrant has a Docker provider now, so you can use vagrant to manage your Docker builds and deployments.
  • Vagrant has similar challenges, as boxes can become out of date, and sometimes boxes can be hard to find and/or update. There are tools like packer and the older veewee to help you build so called 'base' boxes.
  • Vagrant uses virtualbox to spin up a virtual machine for you where you can set up your own environment and install everything you need on that machine mostly through some provisioning scripts. Its most common use case is to test your application on different environments and spin up test servers/machines where your teammates can ssh to and test their work on where everyone else can see and use.
  • Vagrant talks with virtualization systems (VMware, virtualbox, aws, even docker) to mainly create full virtual machines with their own IPs, running any OS and of course, all the applications than implies booting that VM.
  • We've tried Vagrant + Chef (solo and server) and it gets really complicated fast; the addition of berkshelf created huge problems for local installs on Windows (sometimes for Macs as well).
  • I recommend using the Vagrant solution if you are not planning to use Docker in production.

Docker

  • Docker is a tool for managing Linux containers. At first glance it seems similar to Vagrant, but the use case is actually quite different. With containers, you generally don't want an entire distribution installed, but rather just a single service.
  • Think of Docker as something a little closer to package management, rather than a virtual machine.
  • One really nice thing about Docker (and containers in general), is that you can get away with running more bleeding-edge sofware in production, since when you deploy a Docker container, you're deploying the entire stack, not just a few files. So you do an update on everything in a container? It goes through QA just like the rest of the code.
  • Docker is not a full-fledged Virtual Machine, but rather a container. Docker enables you to run instances of services/servers in a specific virtual environment. A good example of this would be running a Docker container with Ruby on Rails on Ubuntu Linux.
  • The great thing about Docker is that it is light-weight (because it relies on shared-kernel linux containers) and it is distribution agnostic. While the kernel between all instances is shared (but isolated from the host and each other), the user space for different instances can be based on different linux distributions.
  • Docker is yet limited in its flexibility - 'everything is an image', and you can create variant images, and whole stacks of images, where each one adds features to a previous one. Managing that can become a challenge.
  • Docker on the other hand uses images and containers to build your application as an image. An image is basically an instance of your application with all of its setup environment and requirements installed, however its not a machine.. A container is basically a process/service that runs on the background, it acts as a virtual machine that contains your images, but its not one, its just a service that runs on top of a machine. You can run many images in one container and you can run many containers on one machine.
  • If you have your docker image, you can run your application on any machine, all you need is to have docker installed. Docker uses dockerhub as a CDN where users can pull/push images from repositories.
  • Docker talks with the kernel of linux (and in a near future, windows) to create containers (think in a chroot that have its own network interfaces, users and process, isolated from the rest of the system ) to launch applications, in an opinionated way (the emphasis is in running single apps, declaring exposed network ports, and image based filesystems).
  • Docker will be more complicated to setup, because in the case of a multi-tier app, you'll want to have more than one container; this means images for each container type, the fig template, some setup/docs and a build system/procedure to tie them together. However, once you get the ball rolling, people will get updates fast, and will generally have an easier time with it. Caveat: win and mac folks will need to do some tinkering with boot2docker and such.
  • Docker did not come in the market to play a "versus" role for Vagrant. Docker does a good job, but the basic fact still lies that it needs an OS to run on. A VM would serve the purpose for getting docker run over it.
  • Docker up and destroy is faster than Vagrant, so just use one Vagrant VM and multiple Docker images over it.

Sources

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment