Docker

What is Docker? What problems it solves, and why you should learn it right now!
Docker is very useful for a lot of people, not only dev, even if you work with the IT even if you are an engineer, Docker is a must-know!
Other than how to install docker on Windows, OSX, and Linux, we are going to see the basic of containers (what’s in it, what is not in it running some example), then we are going to build some images, that are the building blocks of containers, then Networking for Docker, Storage stuff (Volumes).
Then we get into Docker compose that is a tool that comes with docker but really it’s designed for local dev and test!
Once you understand all the basics of those tools we are going to talk about orchestration, which allows you to run containers in multiple servers that act sort/like a single server, and we start with “Swarm” because it comes with Docker, it is very easy to understand and it works really great in a lot of scenarios, and you’ll learn about YAML files and how to configure multi-container solutions and distributed systems and microservices.
But eventually, you’ll want to jump into Kubernetes, which comes after Swarm, even if it is more complicated and does a lot more than Swarm.

Why Docker?

So one more time what is Docker? What is it about? Why it exists?
Docker is a huge shift, one of the biggest in the latest years, and eventually, you are going to shift too.
A little of history of major infrastructure shifts:

Mainframe to PC (1990)

In the 90s we shifted from a traditional mainframe architecture that had been around for many years to a PC distributed architecture where we’re pitting in Mac and Windows and Dos, and we changed our networks and putting in fiber Ethernet and TCP/IP etc..

Virtualisation (2000)

Not even 10 years later (00s) we were changing again, in the data center and in our servers, we were shifting to virtualization because our servers were too powerful, and we had a lot of idle time, and we needed to get better utilization, so we started creating a lot of OS’s inside a single machine, using virtualization with VMware, Hyper-V etc…

Cloud (2010)

It started in 2007/8 when Amazon released the AWS solution. That was the first real idea we had around easy, cheap, disposable compute power that we could spin up and shut down really quickly from a website. No one had done that before, that easy, on that level of scale, and it really shifted the landscape.

Host to Container (Serverless)

Then we have the next wave, and that’s where we’re at now which is containers. Now containers would also include server lists, or functions as a service (if you have heard of that), because all those things were made possible by containers and run inside containers to do their work.
Fundamentally, it’s the shift to containers as the next object of computing. You will see very soon, where it’s the default for us to all assume that you’re running your stuff in containers.

Migration pain

All those changes in the course of the years mean migration, learn new tools, learn new terminology and a lot of more work, we all know that migrating is the hard part.

Unlike previous shifts, Docker is focused on the migration experience, the docker tools were really created from the “get go” with the developer in mind, which means that they work just as well for dev as they do for sysops and operators.

What will Docker Do for Me?

Why you need docker? Why is the real benefit? It’s all about speed!
Like all the other major shifts, always about speed, about speed deployments, the speed of business and getting things done in a company for profit.
You will see big benefits in all the areas when you adopt various tools of the ecosystem.

If you are not working with containers now you are probably working with multiple types of applications: frontend, backend, worker, middle tier, you have a lot of different things that all need to work together to run your software, and those all will have their own dependencies, their own requirements, they may even run on different operating system or different clouds.
And then you have them running on developer machines, and in testing infrastructure, and then in production, and maybe it’s in a data center and also in the cloud. It’s complicated!

We have a lot of different ways of dealing with that, but the way that containers do it, now, is consistent across the board, it allows you to package the same way regardless of your operating system, it allows you to distribute the software regardless of the setup, it allows you to run and test the software the same way everywhere you’re running it: Mac, Windows, PC, cloud, data center, edge devicesit doesn’t matter.

Docker party

Docker started and still is, largely open-source, so every year on the month that Docker project was originally released (March) there is a party around the world, basically, all the various meetups around container technologies get together and celebrate docker releasing.

Docker Other tools

Docker is the essence that you need to understand first but once you move beyond that you’re probably going to want to check your other tools that you know, or what tools you might need to use to fill the gap in your toolset.
There is a great series of posters and diagrams that describe a lot of these things at https://landscape.cncf.io/

Installing Docker

There are different flavors of Docker At this moment, and we need to go through real quick, to understand which one you need for the course.
Let’s learn about the free Community Edition Docker, Docker CE, and the EE version, Enterprise Edition. We won’t actually cover much of that in this course, it’s a paid version and it’s usually for larger business, but we’ll touch on it every so often but we won’t be using it.

We are going also to know the difference between the Stable and Edge releases, and which one you should use.

Which one to use?

Let’s figure it out which one we need:
Docker is not just a simple point and clicks installation sometimes because it’s very big and contains a lot of features.

Your version matters: Since the evolution of containers is very fast and continuous, you need to make sure that you’re on the most recent version.

There are three major types of installs: Direct, Mac/win, Cloud

Direct installation

You just install it directly on a supported operating system, so that it runs on the kernel and that OS it’s supported, and until 2016 was mostly just Linux, and then we started seeing innovations where the Raspberry PI could use it, and we saw it on mainframes, and then windows Server 2016 supported it natively.

Mac and Windows 10

Installing for Mac/Win, is more of a not direct method, but more of a suite of tools including a GUI and seeings, preferences, to make it really easy for you to develop. Since is not supported natively, Docker has to start a Virtual Machine to run the containers in.

Cloud Installing

The cloud version is really what I would say are AWS, Azure, or Google versions, so Docker for AWS or Docker for Azure etc… And those versions come with features specific to that cloud vendor, for example for AWS side, it’ll come with the cloud formation template, and with persistent storage options for storing your databases.

Of course, we are going to start with the Win/Mac/Linux version but in the end we will start to create Swarms in the cloud, and we will see the options at that point on how to do that.

Edge vs Stable

Edge means Beta in the Docker world, it comes out every month, and it is only supported for a month until the next beta come out. You may not want to do that, when you’re downloading Docker by default, you will get the Stable version that comes out once a quarter, and is supported for the next 4 month, and then is not supported and you need to update it (or pay the EE version)

Installing on Windows (finally)

There are two types of containers now on Windows, which is really cool, we have the option for Linux containers, or windows containers. This course will talk about Linux containers because the windows one it’s new. But most of the concepts are the same, you are going to use them the same way, you are going to manage them the same way, it just matters that binaries that are running inside them, you know, SQL, EXE vs MySQL on Linux or something. So when I say container I mean Linux container, and I will specifically call for Windows containers.

The best version of Docker on Windows is, of course, Docker for Windows, that is actually an addition of Docker and it got a whole install that we’ll see a moment and it’s really great on Windows 10 (Pro, Enterprise Only)

If you don’t have Hyper-v enabled on your machine, when installing Docker it will be enabled and the system need a restart.

On some laptop and very rarely on PC you might need to enable virtualization on the BIOS of your machine, so if you have an error lookup on the internet and you probably find out that you need to go to the BIOS.

once you have installed Docker, and that it is running you can open PowerShell and do:

docker version 

And you should see it running. And if you open Hyper-v you will see the virtual machine running (on a MobyLinuxVM)

Share the harddrive

If Docker didn’t ask you to share a hard drive, or if your code is not in C:\ you need to go to setting->Shared drives and share the drive where you have your code.

Adjust the resources

Still, in setting on the Advanced tab, you can choose to adjust the resources dedicated to Docker, so you can choose how many cores you want to give to him and how much memory (RAM)

CLIENT – SERVER – ENGINE

When installing docker you can see the version of docker using

$ docker version
 Client:
  Version:           19.03.8-ce
  API version:       1.40
  Go version:        go1.14.1
  Git commit:        afacb8b7f0
  Built:             Thu Apr  2 00:04:36 2020
  OS/Arch:           linux/amd64
  Experimental:      false
 Server:
  Engine:
   Version:          19.03.8-ce
   API version:      1.40 (minimum version 1.12)
   Go version:       go1.14.1
   Git commit:       afacb8b7f0
   Built:            Thu Apr  2 00:04:16 2020
   OS/Arch:          linux/amd64
   Experimental:     false
  containerd:
   Version:          v1.3.4.m
   GitCommit:        d76c121f76a5fc8a462dc64594aea72fe18e1178.m
  runc:
   Version:          1.0.0-rc10
   GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
  docker-init:
   Version:          0.18.0
   GitCommit:        fec3683

As you can see this return us two parameters:
Client and Server!
The client is the CLI, the software that “talks” with the Server, while the Server (also called the engine) is the main component that makes docker run!
It’s a service (or a daemon in Linux).
Ideally you want them both at the same version, and of course at the latest version.

By the way, if the server part is not showing, it’s probably because you didn’t access it through superuser privileges, so you need to give authorizations to your user, or just use “sudo” when you need to work with docker.

Docker info

If you do:

$ docker info
 Client:
  Debug Mode: false
 Server:
  Containers: 0
   Running: 0
   Paused: 0
   Stopped: 0
  Images: 0
  Server Version: 19.03.8-ce
  Storage Driver: overlay2
   Backing Filesystem: 
   Supports d_type: true
   Native Overlay Diff: false
  Logging Driver: json-file
  Cgroup Driver: cgroupfs
  Plugins:
   Volume: local
   Network: bridge host ipvlan macvlan null overlay
   Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
  Swarm: inactive
  Runtimes: runc
  Default Runtime: runc
  Init Binary: docker-init
  containerd version: d76c121f76a5fc8a462dc64594aea72fe18e1178.m
  runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
  init version: fec3683
  Security Options:
   apparmor
   seccomp
    Profile: default
  Kernel Version: 5.3.18-1-MANJARO
  Operating System: Manjaro Linux
  OSType: linux
  Architecture: x86_64
  CPUs: 4
  Total Memory: 3.753GiB
  Name: dell-sp
  ID: ETOK:2XDZ:X52V:NHTR:DEWT:AZJ4:32CR:Z5NP:JVKU:QDPQ:GBN3:JXXB
  Docker Root Dir: /var/lib/docker
  Debug Mode: false
  Registry: https://index.docker.io/v1/
  Labels:
  Experimental: false
  Insecure Registries:
   127.0.0.0/8
  Live Restore Enabled: false

You will get a lot more information about your docker engine.
Infos about configuration and setup, our engine, the number of container running and

WHAT COMMAND?

If you want to see all (not really all but most) of the command you can use on docker, just write docker on your CLI and hit enter:

Usage:    docker [OPTIONS] COMMAND
 A self-sufficient runtime for containers
 Options:
       --config string      Location of client config files (default
                            "/home/simonep/.docker")
   -c, --context string     Name of the context to use to connect to the
                            daemon (overrides DOCKER_HOST env var and
                            default context set with "docker context use")
   -D, --debug              Enable debug mode
   -H, --host list          Daemon socket(s) to connect to
   -l, --log-level string   Set the logging level
                            ("debug"|"info"|"warn"|"error"|"fatal")
                            (default "info")
       --tls                Use TLS; implied by --tlsverify
       --tlscacert string   Trust certs signed only by this CA (default
                            "/home/simonep/.docker/ca.pem")
       --tlscert string     Path to TLS certificate file (default
                            "/home/simonep/.docker/cert.pem")
       --tlskey string      Path to TLS key file (default
                            "/home/simonep/.docker/key.pem")
       --tlsverify          Use TLS and verify the remote
   -v, --version            Print version information and quit
 Management Commands:
   builder     Manage builds
   config      Manage Docker configs
   container   Manage containers
   context     Manage contexts
   engine      Manage the docker engine
   image       Manage images
   network     Manage networks
   node        Manage Swarm nodes
   plugin      Manage plugins
   secret      Manage Docker secrets
   service     Manage services
   stack       Manage Docker stacks
   swarm       Manage Swarm
   system      Manage Docker
   trust       Manage trust on Docker images
   volume      Manage volumes
 Commands:
   attach      Attach local standard input, output, and error streams to a running container
   build       Build an image from a Dockerfile
   commit      Create a new image from a container's changes
   cp          Copy files/folders between a container and the local filesystem
   create      Create a new container
   diff        Inspect changes to files or directories on a container's filesystem
   events      Get real time events from the server
   exec        Run a command in a running container
   export      Export a container's filesystem as a tar archive
   history     Show the history of an image
   images      List images
   import      Import the contents from a tarball to create a filesystem image
   info        Display system-wide information
   inspect     Return low-level information on Docker objects
   kill        Kill one or more running containers
   load        Load an image from a tar archive or STDIN
   login       Log in to a Docker registry
   logout      Log out from a Docker registry
   logs        Fetch the logs of a container
   pause       Pause all processes within one or more containers
   port        List port mappings or a specific mapping for the container
   ps          List containers
   pull        Pull an image or a repository from a registry
   push        Push an image or a repository to a registry
   rename      Rename a container
   restart     Restart one or more containers
   rm          Remove one or more containers
   rmi         Remove one or more images
   run         Run a command in a new container
   save        Save one or more images to a tar archive (streamed to STDOUT by default)
   search      Search the Docker Hub for images
   start       Start one or more stopped containers
   stats       Display a live stream of container(s) resource usage statistics
   stop        Stop one or more running containers
   tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
   top         Display the running processes of a container
   unpause     Unpause all processes within one or more containers
   update      Update configuration of one or more containers
   version     Show the Docker version information
   wait        Block until one or more containers stop, then print their exit codes
 Run 'docker COMMAND --help' for more information on a command.

CONTAINERS

Containers are the fundamental building block of the Docker toolkit and that’s one of the first things we are going to learn, we are not going to build images or do any advanced stuff until once we’ve learned what containers are.

IMAGE VS CONTAINER

An image is the library, the binaries, the source code that makes up your application! This could be PHP, MySQL, PostgreSQL, Node, pretty much anything …

The container is a running instance of that image, so the actual running thing based on an image, you can have different running containers of the same image.

REGISTRIES

Ok, but where we get the images that we need? We use registries, like npm for node, or repositories for Linux.

the one for docker is hub.docker.com

BUILD THE FIRST CONTAINER

Let’s build our first container:

$ docker container run --publish 80:80 nginx

(don’t worry we are going to explain everything later)
This is the result:

Unable to find image 'nginx:latest' locally
 latest: Pulling from library/nginx
 afb6ec6fdc1c: Pull complete 
 b90c53a0b692: Pull complete 
 11fa52a0fdc0: Pull complete 
 Digest: sha256:30dfa439718a17baafefadf16c5e7c9d0a1cde97b4fd84f63b69e13513be7097
 Status: Downloaded newer image for nginx:latest

Now if you go to your web browser and type localhost you will be welcomed by nginx! So in no time, we have built an instance of Nginx (that could have been Apache too) and it is up and running.

So what just happend?
In when we pass the command, Docker is looking for an image (remember?) called Nginx and if he doesn’t find it on the local machine, it will look in the repository (hub.docker.com) and pull the latest version of that image, and it starts it! As a new process(run), in a new container!

The --public part of the code expose my local port 80 on my local machine, and sends all traffic from it to the executable (docker instance) inside that container on port 80..
So 80:80 means actually From 80(local) to 80(container).
Remember that containers are like micro computer with their Os and their network.

DETACH or -d

So, if we close the terminal (at least on Linux) the container will stop working, (also if you press ctrl+c.
If you want your container to run like a service, so that is not liked to a terminal you can use the --detach option while creating the command:

$ docker container run --publish 80:80 --detach nginx

And as response we have the container ID:

a66dfb13ee401d5b6bdb3f7226162b4653fc5aecd97dac88702c5837af601a18

Now also know that every time you create a new container, it will have a new ID!

LISTING CONTAINERS

So we know how to create differents containers, but now we want to know how to see what are the container created, to do so you can just do:

docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a66dfb13ee40 nginx "nginx -g 'daemon of…" About a minute ago Up About a minute 0.0.0.0:80->80/tcp agitated_joliot

The command docker container will return us some informations about the containers that are up and running.

STOPPING A CONTAINER

Now we need to know how to stop a container.
In order to stop a container you can just send:

docker container stop a66

What is a66? Is the beginning of the ID of the Container that we want to stop! You don’t need to write all the ID down, just what is needed to understand that is the correct and unique ID.

LIST ALL CONTAINERS AND NAMES

Now let’s try this:

docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a66dfb13ee40 nginx "nginx -g 'daemon of…" 9 minutes ago Exited (0) 2 minutes ago agitated_joliot
65edd8c7c563 nginx "nginx -g 'daemon of…" About an hour ago Exited (0) 10 minutes ago thirsty_shockley

using the option ls -a we list all the existing containers not just the one that is running. Im not sure if you can see well the result, but there is a name at the end of each line (thirsty_shockley and agitated_joliot) those are like ID, but are names.

Like ID because they need to be unique, and if we don’t specify a name for container, docker will give one for us, and he does that using an open source database of names, surnames, adjectives, or verbs, of notable hackers or scientists(just a funny story)

Let’s try to give a container a name:

docker container run --publish 80:80 --detach --name webhost nginx

We use the opion --name followed by the name we want to give and we can check the result with docker container ls -a

You can use the name to stop the container:

docker container stop webhost

And the container will stop. And if you want that container to run this time you use start

docker container start webhost

This way you don’t have to re-parameter the container but just call his name.

REMOVING CONTAINERS

All containers are actually “stored” in your computer, even if they are not running, if you want to delete a container you can simply do:

docker container rm container_name 

SEE CONTAINER LOG

Another useful use of the name, is when you want to see the log of a container.
If you detach your container, in order to see the log of your container you must send via CLI:

docker container logs containername

This will show us the log of that container.

WHAT HAPPENS IN DOCKER CONTAINER RUN?

Let’s focus now on what happens when you a docker container run.

First there is a misconception that Docker is just a running container and that is his main job, but there is actually a lot more that is happening in the background that does in addition to those containers executing commands.

We already know that when we do a run, docker will look for that image and if it doesn’t find it locally in the image cache, it looks on his repositories, then it downloads it and store it in the local image cache.

Then when the image is ready to go, it’s going to start up a new container BASED on that image.
It’s not going to make a copy of that image, it’s actually going to just start a new layer of changes, right on top of where that image left off, and it’s going to customize the networking, so it is going to give a specific virtual IP address that’s inside a Docker Virtual Network, and It’s actually open up the port that we specified.

If we didn’t specify that Publish command, it is not going to open any port at all, and since we did the 80:80 we are specifying to take the port 80 on the host and forward all the traffic to the port 80 of the container.

Then that container finally start using a command specified in the Dockerfile (we will see this later).

Those are also all the thing that we can change with the command that we run!

CONTAINER VS VIRTUAL MACHINE

When you look online, to understand better what a container is, people usually compare them to virtual machines, and yes there are some similarities, but there are so many things that are just not the same.

Containers are more just like a process that VMs, in fact if you do:

docker top container_name

You will see that that container has a PID (process ID) and you can find this process id also if you see and list all the process id in your machine (if you use Linux, if you use windows or mac docker create a virtual machine that run docker, but the process will be on that virtual machine) .
So as you can see, the PID is not hided in a virtual machine, because it is a process of the host machine.

INSIDE A CONTAINER!

USEFUL COMMANDS

Here’s a list of usefull comands to see what is going inside a container:

docker container top container_name_or_id  -process list in a container
docker container inspect container_name_or_id  -details of container config
docker container stats container_name_or_id  -performance stats for all container

SHELL OF THE CONTAINER

One of the most common things to do with a container is to pass command via the shell, do to so you may think that you need to run ssh to connect to the process, like you would do in a virtual machine env, but there are some tool that we can use directly with docker:

DOCKER CONTAINER RUN

The docker container run command let’s us actually do the trick:

docker container run -it 

The -it option is actually 2 separate otions, you can see that with --help!
The -t give us a pseudo-TTY (or a prompt) simulating a real terminal.
The -i keep STDIN open (the STDIN is the session).

Let’s see this at work:

docker container run -it --name proxy nginx bash
root@a490b73d82e9:/#

As you can see, we create a new docker container and we name it proxy, then we give the command bash, to indicate that we want to use the bash terminal

At the end of the run command, you can see that you can send commands and arguments, (if you use --help), an image has a default command in it that we can configure (later), but if you want to change what is run on startup, you can do right from the run line (bash).
And as you can see the user that we use inside the container is root!
And if you try to do a ls you can see that you have a bunch of folders you can access

From here so you can change config files, download packages from the internet, basically anything that you can do during your common administrative thing from a normal shell.

To get out of that shell, just type exit like you would do from a normal shell, but doing so, you also exited the container! Why? because we changed the command that run when we run the container! So automatically exiting the script will stop the container.

INSTALLING UBUNTU

You could if you need download install and run an Ubuntu image!
Of course the Ubuntu image from the hub.docker.com is way more lightweight than a normal distribution, but you could in theory, install al the missing component that you may need.

RESTART A CONTAINER WITH A CLI

We can use the docker container start!
With the difference that you don’t use -it but -ai

docker container start -ai proxy

and you can access the shell inside that container, bu what if you just want to access the shell of a already running container?

We can do that too using the exec command: This command create another process inside the container, so when you exit this process the container will not stop. Let’s try:

docker container exec -it mysql bash

And if we exit the process, this will not stop