Iteration 6: Continuous Deployment. Exam Preparation

Learning objective:

To explore continuous deployment. To prepare for the forthcoming exam.

Deadline

October 19th at 23.59

Exercise 'operations-pipeline-cd' [M 40]

You have to design some kind of actual deployment step, which ensures that your production cave is updated. Either as a direct consequence of the pipeline, like a ssh command on the production server (proactive approach), OR, as part of a scheduled/regular update, like a cron job on the production server (reactive approach).

Requirements: A successfull run of the CI pipeline that ends in a new release of an update skycave image, should result in your production servers being updated.

Hand-in Submit a Word or PDF report containing:

Section 1: A short discussion and presentation of the approach taken: proactive in which the pipeline IaC script itself triggers a updating of services in the swarm; or reactive in which some recurring script/cron job on the swarm master initiates an update.
Section 2: Include the IaC scripts/code (pipeline file, cron jobs, scripts, whatever makes the clock tick.)
Section 3: A comprehensive set of screenshots that demonstrate the CD pipeline being successfully executed (show commit, pipeline processing, evidence that prod server is updated.)

Evaluation: I will award full 40 points for a report that concisely demonstrate your CD pipeline. Incorrect and/or ill explained aspects may subtract points.

Exercise 'split-repo' (*)

The final exam is individual! So create your own private bitbucket/gitlab account and your own private docker hub account. And get the latest source code, and ensure your development environment supports building, testing, and execution of SkyCave, building and running docker images, and supports compose files in swarm mode.

Exercise 'exam-command'

A feature of SkyCave that will be used at the exam, is the ability to dynamically add new commands to the 'cmd' by updating the server only. To do that, the SkyCave daemon utilizes the Command Pattern.

Requirements:

Review the Command pattern on wikipedia or some other design pattern resource.
Review the 'JumpCommand', 'HomeCommand', etc. in the cloud.cave.extension package. Note that Command classes must be in this package.

Create a PlusCommand so a running 'cmd' can make simple two integer additions, and provide output ala:

== Welcome to SkyCave, player Mikkel ==
Entering command loop, type "q" to quit, "h" for help.
> exec PlusCommand 64 32
The result is: 96.

> exec PlusCommand 999 777
The result is: 1776.

Exercise 'exam-command-http'

The Command you develop at the exam will be contacting a REST service, provided by me. Be sure you have a working knowledge of the UniRest HTTP client (or whatever you choose) framework, and can find code blocks to reuse from your quote service and subscription service implementations.

Requirements:

Create a QuoteCommand that contacts the official quote service, using raw HTTP client communication, and returns something ala:

== Welcome to SkyCave, player Mikkel ==
Entering command loop, type "q" to quit, "h" for help.
> exec QuoteCommand 34
To dare is to lose one's footing momentarily. Not to dare is to lose oneself.
By: Søren Kierkegaard

Exercise 'exam-docker'

Redo the 'docker-hello-spark' exercise from the grounds up.

You will at the exam be asked to create a Gradle based Java project with Java source code and build file that I provide, and asked to create a docker image, via a Dockerfile, as deployment unit.

Exercise 'exam-swarm'

Redo the 'swarm-quote' exercise from the grounds up.

You will at the exam be asked to create a swarm compose-file that includes SkyCave daemon and some new service(s).

Exercise 'threading'

Analyze SkyCave for potential treading issues. Our HTTP URI Tunnel ClientRequestHandler/ServerRequestHandler uses Spark-Java which again relies on Jetty, which handles incoming request in a thread pool, so high volume traffic poses a concurrency situation.

Exercise 'daemon-tls'

This is a purely optional exercise, quite implementation heavy, and SkyCave is not really coded for this (yet), so be prepared to 'hack' quite a bit. And, its outcome will not be of any importance for the rest of the course. It is mostly added as my own recipe for solving the issue that cmd-daemon communication is not encrypted, which is less-than-ideal.

The 'cmd' and 'daemon' use the FRDS.Broker pattern to communicate, defaulting to HTTP / URI tunneling. Actually, the Broker can be configured to run TLS, even though it is not stated in the ReadMe file. Thus - you can run cmd-daemon using a TLS connection.

The client side entails:

In 'StandardClientFactory' the 'createClien...()' sets the server's hostname and port by reading from the CPF and invoking crh.setServer(hostname, port);. The FRDS.Broker actually overloads this method (rather unelegantly, but still):
```
public void setServer(String hostname, int port, boolean useTLS)          
       
```
So the first step is to recode it and somehow determine if TLS should be used or not (adding a new property to the CPF and read that, would be the proper course).

Now, the client can talk HTTPS

As usual, the server is a bit more complex. The coding, however, is similar.

In 'StandardServerFactory' the 'createServerRequestHandler()' sets the port and invoker srh.setPortAndInvoker(port, objMgr.getInvoker());, and again you can instead use the overloaded method
```
public void setPortAndInvoker(int port, Invoker invoker, boolean useTLS) {
         
```
Again, a flag must be introduced to set the last parameter to true/false.
Then you need a certificate (self-signed or issued by a CA authority), convert it to a keyStore, and keep that on the server hosting 'daemon'.
And then use the Java system properties to tell the location and password of that keyStore
```
javax.net.ssl.keyStore
javax.net.ssl.keyStorePassword
           
```
The FRDS.Broker will use these to load the certificate.

The final piece is how to tell Gradle to pass these to an executing JVM. The issue is that Gradle spawns a new process, different from the one Gradle itself runs in, which means setting them must be done in the task itself ala this example

task server(type: JavaExec) {
  group 'Secure Web Server'
  description 'Run a HTTPS enabled server'

  classpath sourceSets.main.runtimeClasspath
  main = 'example.Hello'

  // This is the trick - use default system props
  systemProperty("javax.net.ssl.keyStore",
    "/home/csdev/certificate.keystore")
  systemProperty("javax.net.ssl.keyStorePassword",
    "changeit")
}

Alternative: Use Traefik or some other proxy service.

Morale: There is a reason I decided not to let the cmd-daemon communication be encrypted this time around.