Docker containers are a form of "lightweight" virtualization They allow a process or process group to run in an environment with its own file system, somewhat
In addition to containers, Docker makes it easy to build and distribute images that wrap up an application with its complete runtime environment.
For more information, see What are containers and why do you need them? and What Do Containers Have to Do with DevOps, Anyway?.
The difference between the "lightweight" virtualization of containers and "heavyweight" virtualization of VMs boils down to that, for the former, the virtualization happens at the kernel level while for the latter it happens at the hypervisor level. In other words, all the containers on a machine share the same kernel, and code in the kernel isolates the containers from each other whereas each VM acts like separate hardware and has its own kernel.
Containers are much less resource intensive than VMs because they do not need to be allocated exclusive memory and file system space or have the overhead of running an entire operating system. This makes it possible to run many more containers on a machine than you would VMs. Containers start nearly as fast as regular processes (you don't have to wait for the OS to boot), and parts of the host's file system can be easily "mounted" into the container's file system without any additional overhead of network file system protocols.
On the other hand, isolation is less guaranteed. If not careful, you can oversubscribe a machine by running containers that need more resources than the machine has available (this can be mitigated by setting appropriate resource limits on containers). While containers security is an improvement over normal processes, the shared kernel means the attack surface is greater and there is more risk of leakage between containers than there is between VMs.
For more information, see Docker containers vs. virtual machines: What’s the difference? and DevOps Best Practices: Immutability
There are, broadly, two areas where containers fit into your devops workflow: for builds, and for deployment. They are often used
Synchronizing build environments: It can be difficult to keep build environments synchronized between developers and CI/CD servers, which can lead to unexpected build failures or changes in
behaviour. Docker images let you specify exactly the build tools, libraries, and other dependencies (including their versions) required without needing to install them on individual machines,and distribute those images easily. This way you can be sure that everyone is using exactly the same build environment.
Managing changes to build environments: Managing changes to build environments can also be
difficult,since you need to roll those out to all developers and build servers at the right time. This can be especially tricky when there are multiple branches of development some of which may need older or newer environments than each other. With Docker, you can specify a particular version of the build image along with the source code, which means a particular revision of the source code will always build in the right environment.
Isolating build environments: One CI/CD server may have to build multiple projects, which may have conflicting requirements for build tools, libraries, and other dependencies. By running each build in its own ephemeral container created from potentially different Docker images, you can be certain that these builds environments will not interfere with each other.
Runtime environment bundled with
application: The CD system builds a complete Docker image which bundles the application's environment with the application itself and then deploys the whole image as one "atomic" step. There is no chance for configuration management scripts to fail at deployment time, and no risk of the system configuration to be out of sync.
Preventing malicious changes: Container security is improved by using immutable SHA digests to identify Docker images, which means there is no way for a malicious actor to inject malware into your application or its environment.
Easily roll back to a previous version: All it takes to roll back is to deploy a previous version of the Docker image. There is no worrying about system configuration changes needing to be manually rolled back.
Zero downtime rollouts: In conjunction with container orchestration tools like Kubernetes, it is
easilyto roll out new image versions with zero downtime.
High availability and horizontal scaling: Container orchestration tools like Kubernetes make it easy to distribute the same image to containers on multiple servers, and add remove replicas at will or automatically.
Sharing a server between multiple applications: Multiple applications, or multiple versions of the same application (e.g. a dev and
qadeployment), can run on the same server even if they have conflicting dependencies,since their runtime environments are completely separate.
Isolating applications: When multiple applications are deployed to a server in containers, they are isolated from one another. Container security means each has its own file system, processes, and users there is less risk that they interfere with each other intentionally. When data does need to be shared between applications, parts of the host file system can be mounted into multiple containers, but this is something you have full control over.
For more information, see:
- Continuous Integration: An Overview
- Containers as the foundation for DevOps collaboration
- Docker and DevOps – Enabling DevOps Teams Through Containerization.
Containers can be integrated into your DevOps toolchain incrementally. Often it makes sense to start with the
- Docker Engine installed on build servers and/or application servers
- Access to a Docker Registry. This is where Docker images are stored and pulled. There are numerous services that provide registries, and it's also easy to run your own.
Many CI/CD systems now including built-in Docker support or easily enable it through plugins,
Determine your build environment requirements and write a
Dockerfilebased on an existing Docker image, which is the specification used to build an image for build containers. If you already use a configuration management tool, you can use it within the Dockerfile. Always specify precise versions of base images and installed packages so that image builds are consistent and upgrades are deliberate.
Build the image
andpush it to the Docker registry using
Dockerfilefor the application that is based on build image (specify the exact version of the base build image), builds the application, adds any required runtime dependencies
notin the build image, and tests the application. A multi-stage Dockerfiles can be used if you don't want the application deployment image to include all the build dependencies.
Modify CI build scripts to use build the application image and push it to the Docker registry. The image should be tagged with the build number, and possibly additional information such as the name of the branch.
If you are not yet ready to deploy with Docker, you can extract the build artifacts from the resulting Docker image.
It is best to also integrate building the build image itself into your devops automation tools.
This can be easier if your CD tool has support for Docker, but that is by no means necessary. We also recommend deploying to a container orchestration system such as Kubernetes in most cases.
Half the work has already been
If using Docker directly, now it's a matter of updating deployment scripts to
onthe application server with the image and tag that was pushed in the previous section (after stopping any existing container). Ideallyyour application accepts its configuration via environment variables, in which case you use the
argumentto specify those values depending on which stage is being deployed. If a configuration file areused, write it to the host file system and then use the
argumentto mount it to the correct path in the container.
If using a container orchestration system such as Kubernetes, you will typically have the deployment script connect to the orchestration API endpoint to trigger an image update (e.g.
kubectl set image
,or better yet a Helm chart).
Once deployed, tools such as Prometheus are well suited to
FP Complete has implemented this kind of DevOps workflow, and significantly more complex ones, for many clients and would love to count you among them! See our Devops Services page.