You should learn the basics of handling containers using the docker command line, as well as start to build your own Dockerfiles. Second, you should be able to publish images using Docker Hub.
September 28th at 23.59. That is, after this time, none of the exercises in this iteration (on this webpage) that are marked M can be handed in on BrightSpace and thus cannot be awarded any points.
Solve the PyCon 2018 Tutorial's exercise : 02_dockerfile_essentials
Create a Dockerfile that builds the Java/Spark 'Hello-Spark' web server, a small web server demo.
/* Build file hello-spark */
plugins {
id 'java'
}
repositories {
mavenCentral()
}
dependencies {
// Use the SparkJava web server framework
implementation group: 'com.sparkjava', name: 'spark-core',
version: '2.9.3'
}
task hello(type: JavaExec) {
group 'SWEA'
description 'Dependency management demonstration'
main = 'example.HelloSpark'
classpath = sourceSets.main.runtimeClasspath
}
package example;
import static spark.Spark.*;
public class HelloSpark {
public static void main(String[] args) {
get("/hello/:name",
(req, res) -> "Hello to you "+req.params(":name")+"");
}
}
Spark-java defaults to port 4567, so when run, the user should be able to see a greeting when browing to URL: http://localhost:4567/hello/(your name).
Create a docker image containing the full SkyCave system using the 'classic' process. That is, start bash in a container run from image 'henrikbaerbak/jdk11-gradle68' and then use copy commands to copy your SkyCave code into it (or use your version control system to clone your repository).
Once complete, commit the container to an image which you can then start as a new container, if the following shell command is issued on the container host:
docker run -d -p 7777:7777 --name mydaemon
[yourimagename] gradle daemon -Pcpf=[cpf file name]
Note: Do not spend too much time on this exercise. It is good learning experience BUT it is not the standard way to create images, spend your time on the next exercise instead.
Create a Dockerfile that contains your full SkyCave system.
Requirements:
docker run -d -p 7777:7777 --name mydaemon
[yourimagename] gradle daemon -Pcpf=[cpf file name]
docker run -d -p 7777:7777 --name mydaemon
[yourimagename]
will start a daemon using 'http.cpf'.
docker run -ti --rm [yourimagename] gradle test
jacocoRootReport
Explain what '--rm' does.
docker run -ti --net container:mydaemon [yourimagename] gradle cmd -Pcpf=[cpf file name] -Pid=[loginName] -Ppwd=[password]
Process Hint: Take small steps! Make a simple Dockerfile, build it and then use docker run -ti [imagename] bash to inspect that what ever you have put into the image is copied to its proper place. Once it is correct, take another small interation adding more 'stuff', and verify. Keep on until everything is in place. Use the ladder, avoid the vaulting pole!
Hint: There are quite a few ways to do that and you can find much inspiration from the Dockerfiles associated with public images on Docker hub. A common technique that I advice is to add the Dockerfile to the root folder of your code base and then use COPY commands to copy build scripts, source folders, etc., into the image; and use CMD to define the startup command. DO NOT create a Dockerfile that rely on accessing your own SCM repositories for checking out your source code - we do not have (and do not want to have) your repo credentials! Also DO NOT utilize .dockerignore files (the resulting image requires you to read and understand TWO source files that are doing opposite things!) I also advice not to use ENTRYPOINT as it hard-codes the image too much to a specific start command.
Hand-in: Submit your Dockerfile in BrightSpace.
Evaluation:
I will not evaluate correctness of your dockerfile; Crunch will exercise your resulting image thoroughly later.
I will use the grade sheet to evaluate your submission. The score counts towards your final grade.
| Learning Goal | Assessment parameters |
| Submission | Required artifacts (dockerfile) are all present. |
| Analyzability | The dockerfile is easy to analyze (read, understand, reason correctly about). It contains author information (LABEL). It contains comments detailing the docker command to execute it correctly. |
| Completeness | The resulting image will contain all required artifacts for execution and analysis (including all source code, resource files, CPF, etc.) The image contains the proper instructions for default execution, port assignment, etc. |
| Terseness | The resulting image does not contain derived artefacts from the host computer (aah, the subproject /build folders becomes cumbersome to avoid, so it is OK to go for the 'simple' copy which include them). |
| Selfcontained | The resulting image is built only from host's artefacts (no pulling from git or other remote resources), and does not rely on '.dockerignore'. |
Regarding the analyzability property; I have found it extremely useful to include usage comments into every Dockerfile I write, ala this header in the henrikbaerbak/jdk11-gradle68 image:
# Usage: docker build -t henrikbaerbak/jdk11-gradle68 -f (thisfile) .
# Test proper install
# docker run --rm -ti henrikbaerbak/jdk11-gradle68 bash
# and then 'gradle -v' in the running container...
Note 2: Yes - it is pretty non-standard to include source code etc., into the docker image, and you should of course never do that in a non-teaching context! Never-the-less this is how we will do it throughout the course in order for me to be able to retrieve your full work (i.e. your code!) in the rest of the course.
Docker stores a lot of meta information along with your image. Play around with 'docker image inspect (imagename)'. Sometimes the 'Created' tag comes in handy when you get lost in push and pulls and builds and why does it run the old code???
Observe that starting your docker daemon is slow in the present setup.
The issue is that Gradle's cache is cold in each new container, forcing gradle to download all SkyCave's dependencies (which are quite a few!). Gradle keeps all dependencies already downloaded in a special folder: /root/.gradle in our newly made containers.
One way to speed up the daemon significantly is to ensure that the cache is not cold. We can do that by mounting a folder of the host machine as a folder in the container.
Use docker's '-v' switch to mount /root/.gradle in the container to a folder on your host machine, so it is reused the next time you start a container. And, do not try to be smart and reuse your host machine's own gradle cache folder---explain why.
Create a 'multistage' Dockerfile, named 'Dockerfile-multistage', to contain the SkyCave daemon (only).
Requirements:
docker run -d -p 7777:7777 --name mydaemon
[yourimagename] java -jar /root/cave/daemon.jar http.cpf
docker run -d -p 7777:7777 --name mydaemon
[yourimagename]
then the daemon is started using the 'http.cpf'
configuration.
Hand-in: Submit your Dockerfile in BrightSpace.
Evaluation:
I will not evaluate this exercise until Crunch has access to your 'skycave-image' image.
I will use the grade sheet to evaluate your submission. The score counts towards your final grade.
| Learning Goal | Assessment parameters |
| Submission | Required artifacts (dockerfile) is present in the BS submission. The dockerfile will produce the proper image. |
| Analyzability | The dockerfile is easy to analyze (read, understand, reason correctly about). The stages are clearly separated. It contains author information (LABEL). It contains comments detailing the docker command to execute it correctly. |
| Completeness | The resulting image will contain all required artifacts for execution of the daemon. The image contains the proper instructions for default execution, port assignment, etc. |
| Terseness | The resulting image does not contain derived artefacts from the host computer, nor source code from the build phase. |
| Selfcontained | The resulting image is built only from host's artefacts (no pulling from git or other remote resources), and does not rely on '.dockerignore'. |
Hint: The SkyCave code already contains the gradle target to build the proper deployment unit: gradle :server:fatJar, which will produce a fat Jar file cave/server/build/libs/daemon.jar
Create a private Docker hub repository and push your SkyCave image to it. Give docker hub users 'henrikbaerbak' and all group members 'Collaborator' permission. (I promise not to push to your repo :).
Again: Other teams should NOT be able to pull your images! You may test this by a) 'docker logout' b) remove your image from local cache 'docker image rm [imagename]' and then c) 'docker image pull [imagename]'. This should issue an error, as the image is not public. Then 'docker login' as yourself and pull the image to get it back.
Detailed requirements:
Hint: You can tag images, but the convention is that tag 'latest' contains the tip revision of your development - the most reliable and production ready version. And I continously have to be able to pull your latest work.
Hand-in:
Evaluation: I will pull your image once you submit, and if I can do that but an anonymous Docker user cannot because the repository is private, you are awarded 5 points. Next Crunch records 10 points when it can pull the image, run all JUnit tests and see them all pass.
Learning Goal and Motivation: Most of our evaluations of whatever your group produces of SkyCave code take place in a staging environment. These are characterized by being shortlived and often different from a real production environment. However, software earns money in the production environment! The 'operations' exercise is the starting point for a series of exercises that requires you to put your environment in production at well-defined checkpoints to ensure that its deployment follows your progress in solving exercises.
Create a node with fixed IP (or assign a DNS name to it) on your favorite cloud computing IaaS provider, and install Docker version 1.19+. Deploy your SkyCave server on it, so clients can contact it at '(IP):(port)'.
Requirements:
docker run --rm -ti [yourimagename] gradle cmd -Pcpf=operations.cpf -Pid=[loginName] -Ppwd=[password]
Hand-in:
Evaluation: I will use your 'cmd' in your
image to contact your server, and if I can do that, you are
awarded 5 points.
At a later point in time, the 'operations' exercise will
hopefully be available to submit to Crunch, check with any
announcements. At that time, you get the last 15 points when
Crunch exercises your production daemon.
Note: Crunch or I may create rooms and put contents into your cave, and will do so in room positions with Z < 0, that is, in the 'cellar' of the (0,0,0) starting position, or deeper.