How to create self-signed certificates

When we create a Web Server that should run over HTTPS we need some server certificate. If the Web Server will be exposed to internet we should buy a certificate signed by a well-known authority but if we are coding a Web Server for some internal or private use we can create our Server Certificate and sign it by ourselves.

All the certificates had to be signed by another certificate. This second certificate have to belong to a Certificate Authority (CA) in which all the clients that will receive the first certificate should trust.

But as I said before, If the clients that will use our Web Server are also our own clients we can specify that the client should trust in any CA Certificate we want.
So we can create our own CA Certificate to later signed any Server Certificate we want to use in a Web Server.

We are going to use openssl for creating the certificates.

CA Certificate

The first step is to create our own CA Certificate. To do that we should run the following commands in a terminal.

  • For generating the Private Key for the CA Certificate.
> openssl genrsa -des3 -out myOwnCA.key 2048

Passphrase: Whatever you want. For example ‘myca’.

  • For generating the CA Certificate (also know as Public Key) to later signed the Server Certificate.
> openssl req -x509 -new -nodes -key myOwnCA.key -sha256 -days 1024 -out myOwnCA.pem

Passphrase: What you put in the previous step.

When we create a certificate openssl asks us some information. We should complete at least Common Name. We can use the default values for the rest of the fields just entering a dot ‘.’

I usually completes 3 fields and for the sake of the example we can set:
1) Country Name as ‘US’
2) Organization Name as ‘My Organization’
3) Common Name with ‘MyOwnCA’

Server Certificate

Once we created the CA Certificate we have to proceed to create the Server Certificate. To do that we should run the following commands in a terminal.

  • For generating a Private Key for the Server Certificate. This step is similar to what we did for the CA Certificate.
> openssl genrsa -des3 -out server.key 2048

Passphrase: Whatever you want. For example: ‘server’.

  • For generating a Server Certificate Sign Request with the Server Private Key. Now, we don’t create the Server Certificate, we create a Server Certificate Sign Request. This request is the one we should send to some Certificate Authority for signing if we pay for that.
> openssl req -new -key server.key -out server.csr

Passphrase: What you put in the previous step.

Here again openssl asks us some information. The most important one is ‘Common Name’. We have to enter there the IP, or host name. For example: ‘localhost’. Remember that we can complete the rest of the fields or just entering a dot for the default values.

Once we have the Server Certificate Sign Request (server.csr) we should sign this request with the CA Certificate in order to get the Server Certificate Signed by our own CA.

For generating the Server Certificate we have to sign the Server Certificate Sign Request with the CA Certificate.

> openssl x509 -req -in server.csr -CA myOwnCA.pem -CAkey myOwnCA.key -CAcreateserial -out server.crt -days 500 -sha256

Passphrase: CA Certificate passphrase. Following the example it should be: ‘myca’.

Finally, we should create the PKCS12 file with the Server Private and Public Keys. To do that we should run the following commands in a terminal:

  • For joining the Server Private Key and the Server Certificate in the same file.
> cat server.key > server.pem
> cat server.crt >> server.pem
  • For creating the PKCS12 file.
> openssl pkcs12 -export -in server.pem -out server.pkcs12

Passphrase: Server passphrase. In our example: ‘server’.
Export Passphrase: Whatever you want. For example: ‘Server’ again.

Final comments:

Reach this point, we should set the server.pkcs12in the Server as the certificate to use and add the myOwnCA.pem certificate as a trusted one in the client you use. For example a Web Browser.

Reference (List of commands):

1. openssl genrsa -des3 -out myOwnCA.key 2048
2. openssl req -x509 -new -nodes -key myOwnCA.key -sha256 -days 1024 -out myOwnCA.pem
3. openssl genrsa -des3 -out server.key 2048
4. openssl req -new -key server.key -out server.csr
5. openssl x509 -req -in server.csr -CA myOwnCA.pem -CAkey myOwnCA.key -CAcreateserial -out server.crt -days 500 -sha256
6. cat server.key > server.pem
7. cat server.crt >> server.pem
8. openssl pkcs12 -export -in server.pem -out server.pkcs12

About ${packaging.type} error when sbt resolves dependencies

I have been using Swagger pages in Scala Maven project without any problem for a long time. Recently, I had to add a Swagger page to a Scala Sbt project and I faced with an unexpected problem. Besides I did the same as always I found myself struggling with some weird problem with the javax.ws.rs-api library used by Swagger.

This is the Sbt console when compiling:

[info] Updating {file:/home/null/code/me/packaging.type/packaging-type-workaround/}packaging-type-workaround...
[info] Resolving jline#jline;2.14.6 ...
[warn] [FAILED ] javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}: (0ms)
[warn] ==== local: tried
[warn] /home/null/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
[warn] ==== activator-launcher-local: tried
[warn] /home/null/.activator/repository/javax.ws.rs/javax.ws.rs-api/2.1.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
[warn] ==== activator-local: tried
[warn] /home/null/programs/activator-dist-1.3.12/repository/javax.ws.rs/javax.ws.rs-api/2.1.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.${packaging.type}
[warn] ==== typesafe-releases: tried
[warn] http://repo.typesafe.com/typesafe/releases/javax/ws/rs/javax.ws.rs-api/2.1.1/javax.ws.rs-api-2.1.1.${packaging.type}
[warn] ==== typesafe-ivy-releasez: tried
[warn] http://repo.typesafe.com/typesafe/ivy-releases/javax.ws.rs/javax.ws.rs-api/2.1.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: FAILED DOWNLOADS ::
[warn] :: ^ see resolution messages for details ^ ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run last *:update for the full output.
[error] (*:update) sbt.ResolveException: download failed: javax.ws.rs#javax.ws.rs-api;2.1.1!javax.ws.rs-api.${packaging.type}
[error] Total time: 4 s, completed Jan 14, 2019 4:08:00 PM

When Sbt tries to resolve the javax.ws.rs-api dependency a weird ${packaging.type} shows up instead of the expecting jar.

I tried to compile the same code using Maven and I everything worked as expected. So I started looking in the internet about the problem and I found that this is known problem with some dependency libraries in Sbt. Basically, Sbt doesn’t handle the ${packaging.type} variable. For more information see: https://github.com/sbt/sbt/issues/3618

There are several workarounds but the one I like is to create an Sbt AutoPlugin for setting the jar package type in the build settings.

This is the vert simple workaround code:

import sbt._

object PackagingTypeWorkaround extends AutoPlugin {
  override val buildSettings = {
    sys.props += "packaging.type" -> "jar"
    Nil
  }
}

You should place this code into a PackagingTypeWorkaround.scala file in the project folder.

After that you can compile your Sbt project without any problem.

> compile
[info] Compiling 1 Scala source to /home/null/code/me/packaging.type/packaging-type-workaround/target/scala-2.12/classes...
[info] 'compiler-interface' not yet compiled for Scala 2.12.8. Compiling...
[info] Compilation completed in 15.15 s
[success] Total time: 17 s, completed Jan 14, 2019 4:15:55 PM

Final Comments:

When you build your project using Jenkins for the very first time after adding the workaround it will fail. You don’t have to do anything else, just run the Jenkins job again.

 

NOTE: You can find the code for this project at: https://github.com/Scalents/packaging-type-workaround

How to create an Akka HTTP Server

The Akka HTTP modules implement a full server-side and client-side HTTP stack on top of akka-actor and akka-stream. It offers two different API with different levels of abstraction: a high-level one and a low-level one.

The high-level routing API of Akka HTTP provides a DSL to describe HTTP “routes” and how they should be handled. Each route is composed of one or more levels of Directive s that narrows down to handling one specific type of request.

The low-level Akka HTTP server API allows for handling connections or individual requests by accepting HttpRequestobjects and answering them by producing HttpResponse objects.

Adding the Akka-HTTP dependency

Akka HTTP is provided in a separate JAR file, to use it you should include the following dependency in your build.sbt file

"com.typesafe.akka" %% "akka-http" % "10.0.9"

Creating Server basic structure

First we add an Scala file to our project and create an object that extends from App and Directives.

import akka.http.scaladsl.server.Directives

object Server extends App with Directives {}

Then we need to add the ActorSystem and ActorMaterializer to that object.

import akka.http.scaladsl.server._
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer

object Server extends App with Directives {
  implicit val system = ActorSystem("actor-system")
  implicit val materializer: ActorMaterializer = ActorMaterializer()
}

Now we can get the server up by adding the following lines after the implicits.

val routes: Route = path("/test") { complete("test") }

The above line defines an example route /test which returns the text: test

Http().bindAndHandle(routes, "0.0.0.0", 8002)

This final line allows us to bind an IP and port to a set of routes.

Server basic Structure:

import akka.http.scaladsl.server._
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer

object Server extends App with Directives {
  implicit val system = ActorSystem("actor-system")
  implicit val materializer: ActorMaterializer = ActorMaterializer()

  val routes: Route = path("/test") { complete("test") } 
  Http().bindAndHandle(routes, "0.0.0.0", 8002)
}

Defining Routes

Now we are going to replace the example route /test with some more interesting ones.

All the routes we are going to define in our server are implemented using Akka-HTTP directives and have to be assigned to a Route type variable. This variable is the one which will be used as the first parameter of thebindAndHandle method.

The most easy way to add a new route is using the directive ‘path‘ along with a path as we saw in the example route above.

All the directives should finish with a complete method call.

val routes: Route =
  path("test") {
    complete("test")
  }

Inside the path we can use different directives depending on the HTTP method that we want to use. Here is an example using the POST verb.

val routes: Route =
  path("test") {
    post {
      complete("test")
    }
  }

If we want to add a new route we can concatenate two path directives just using the ~ symbol.

val routes: Route =
 path("test") {
   ...
 } ~
 path("test2") {
   ...
 }

Responding requests

Till now we saw how to return a text for an endpoint using the complete method and a text. But what about if we want to return JSON data? Well certainly we can do something like this:

val routes: Route = 
  path("test") { 
    post {
      complete("{\"my_key\":\"my_value\"}")
    }
}

However this JSON will be returned with a wrong content type.

In order to set the right content type we have to use the Akka HTTP low level objects HttpResponse and ResponseEntity

val routes: Route = 
  path("test") { 
    post {
      val resp: ResponseEntity = HttpEntity(ContentTypes.`application/json`,"{\"my_key\":\"my_value\"}")
      complete(HttpResponse(StatusCodes.OK, entity = resp)
    }
  }

Note: Besides HttpResponse object has a parameter to define headers the Content Type can not be defined in that parameter. It has to be defined in the HttpEntity.

Dealing with input data in URLs

If we want to get a number from an URL we should use the IntNumber object.

val routes: Route =
  path("test" / IntNumber) { id =>
    get {
      complete(s"get test - $id")
    } 

But if we want to get the content from one / to the next one (or the url end if there is no one more) we should use the Segment object.

val routes: Route =
  path("test" / Segment ) { data =>
    get {
      complete(s"get test - $data")
    }
  } 

Of course there is more options but IntNumber and Segment are the two more useful ones.

We can also combine more than one input data in the same route as follows:

val routes: Route =
  path("test3" / Segment / IntNumber) { (data, id) =>
    get {
      complete(s"get test3 - $data, $id")
    }
  }

Dealing with input data in request body

One common thing when we work with HTTP requests is to get information from the request body. This information has to be converted into an Scala data type in order to be able to work with it in our Akka HTTP Server.  To do that we could use the entity(as(..)) structure.

Example:

path("test") {
  post {
    entity(as[String]) { param =>
      ...
    }
  }
}

In this example param contains the request body as an String.

This can be accomplished due Akka HTTP includes marshallers for some basic data types. Default marshallers are provided for simple objects like String or ByteString.

Dealing with JSON

Most of the times the request body information comes as a JSON structure and deal with that it’s a little more complex. There is no default marshaller for your own JSON data of course, so you have to define it.

There are some JSON libraries which can help but the most common library for these cases is the spay-json library. And here we are going to use this library to define our own marshaller.

First of all we have to define a case class with all the parameters that we are going to receive in the request JSON body.

So, for example, if our JSON data is something like:

{
  names: ['jhon', 'freddy', 'kurt'],
  id: 10
}

We should define a case class like this:

case class TestAPIParams(names: List[String], id: Int)

But after defining the right case class we have to define the marshaller.

To do that, we have to define a trait extending from prayJsonSupport and DefaultJsonProtocol. Inside this trait we have to define and implicit using jsonFormatX where X is the amount of parameters we have. In our case we should define the trait as follows:

trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
  implicit val testAPIJsonFormat = jsonFormat2(TestAPIParams)
}

Once we have a marshaller for our JSON data we have to extend the object in which we are defining the route.

object Server extends App with Directives with JsonSupport {
  ...
}

And now we can use the entity(as(...)) structure with the our case class

path("test") {
  post {
    entity(as[TestAPIParams]) { params =>
      ... // We can use params.names and params.id
    }
  }
}

Final comments

It may seem a little difficult to develop a server over Akka HTTP from scratch and certainly there are complex things in the library. However following these steps it’s easy to start and have the basic server structure and functionality quite soon.  Then it’s only hard work and read the documentation. Luckily Akka HTTP has a very complete and clear official documentation, fully of examples.