Iteration 4: Orchestration. Docker Swarm

Learning objective

Deploying systems having multiple services deployed using Docker Swarm.

Deadline

October 12th at 23.59

Exercise 'swarm-quote'

Initialize your Docker installation to run in 'swarm' mode, and create a docker compose file that starts a SkyCave Daemon, and a QuoteService based upon 'henrikbaerbak/quote:msdo_1_0_1'. All other services can be the default test double variants.

Exercise 'swarm-db' (*) [M 60]

Now, to start your SkyCave backtier, you need to start two containers: the daemon and the Redis db; and you need to bind them together using a docker network. Manual procedures are not DevOps... We instead need infrastructure-as-code!

Initialize your Docker installation to run in 'swarm' mode, and create a docker compose file that allows starting, updating, and stopping a full SkyCave server side system, on a named network.

Requirements:

Your docker compose file must be named 'swarm-db.yml' and it must be in version 3.7+ format. Remember to add it to your 'skycave-image' image, so I can review it on the Crunch machine.
Your daemon must be started with a 'swarm-db.cpf' which is configured to connect to the Redis in the deployed stack, and other services like this:

Subscription: Real CaveStorage: Redis Quote: Real PlayerNameService: InMemory
We should be able to start your fully operational SkyCave backtier with daemon and db by issuing
```
docker stack deploy -c swarm-db.yml (stackname)
            
```
There should be only a single daemon running, exposed on the host port on port 7777, allowing connections to be made from a locally running 'cmd'.
The DB contents must survive stack removal and restarts, defined by a named volume.
The services must run on a network defined in the compose file.
The compose file must have "high analyzability", that is well structured, few but proper comments, and include preamble comments stating how to use it. In addition, a description of how the skycave container image refered in the compose file is created, is welcome.

Hand-in:

Submit your compose/stack file. Remember to include it in your skycave image as well.
Include screen shot(s) of shell(s) in which you
1. Deploy the stack
2. Inspect the status (stack ps)
3. Connect using a host based 'cmd' and dig a new room
4. Show the daemon log output associated with the issued commands.
5. Connect to the redis and demonstrate 'queries' to find the newly dug room in the redis datastructures.
6. Removes the stack, starts it again, and demonstrate that the dug room still exists.

Evaluation: I will evaluate along two learning goals. I will use the grade sheet to evaluate your submission. As compose files are pretty complex and involve quite a learning curve, I will then multiply the total by three to get your final score.

Learning Goal	Assessment parameters
Analyzability	The compose file is easy to analyze (read, understand, reason correctly about). The compose file contains a preamble outlining how to use it (stack deploy command, image build instruction, etc.)
Correctness	I will (as best possible) evaluate correctness of the composefile from your screenshots, the file contents, and potential execution. The screenshots must be detailed and comprehensive enough that I can convince myself that everything works correctly.

Exercise 'swarm-db-quote'

Create a compose file 'swarm-db-quote.yml' that starts a SkyCave daemon, a 'henrikbaerbak/quote:msdo_1_0_1' quote service, and a Redis, correctly configured.

Next, add Docker visualizer support.

Exercise 'update-swarm-db'

Do a) deploy swarm-db stack b) make some trivial modification of skycave daemon source code and resulting container c) update the swarm-db by redeploying. What happens?

Exercise 'real-swarm-db' [M 20]

Create a swarm consisting of at least two (virtual) machines, one with larger disk space allocated to host the db, and one node running the Docker visualizer.

Requirements:

Your docker swarm file must be named 'real-swarm-db.yml' and it must be in version 3+ format.
Assign the label 'database' to the machine with a large disk.
Specify that the Redis is constrained to be deployed on a node whose type is 'database'.

Hand-in:

Submit your compose/stack file. Remember to include it in your image as well.
Submit screen shot(s) that show the swarm nodes, the assignment of each container to the underlying nodes, your docker visualizer web page.

Evaluation: I will evaluate analyzability and correctness, with (0,4,7,10) points for each learning goal.

Learning Goal	Assessment parameters
Analyzability	The compose file is easy to analyze (read, understand, reason correctly about). The compose file contains a preamble outlining how to use it (build instruction, stack deploy command, etc.)
Correctness	I will (as best possible) evaluate correctness of the composefile from your screenshots, the file contents, and potential execution. The screenshots must be detailed and comprehensive enough that I can convince myself that everything works correctly.

Exercise 'operations-swarm' [M 20]

Production Checkpoint! Update your production environment, so your production server(s) are configured as a docker swarm and deploys the SkyCave system using a composefile. (No need for multiple VMs, it is just fine just to run swarm mode on your single cloud VM.)

Requirements:

None. I may run your Cmd using the 'operations.cpf' to connect to your production server, and Crunch keeps doing its test journey tests.

Hand-in: Submit screenshot(s) with shells on your production server that shows

The IP address of your server ('ifconfig eth0' or 'ifconfig ens32').
Output of 'docker stack ps' showing deployed services.
Evidence that Redis stores data on the production/host machine (volume mounts, directory contents, ...)

Evaluation: I will review your output and award 20 points if provided artefacts convince me that your production system is swarm based.. I may log into your production server using your Cmd, DIG a room, and ask you to find that room in the database at any time during the course.

Exercise 'portainer'

Play around with https://www.portainer.io/, a web ui for container management. Hm, it used to be 'free' but seems to have become 'free trial' now.