Basic Category Theory for (Scala) Programmers (Part I)

“Aren’t you tired of just nodding along when your friends starts talking about morphisms? Do you feel left out when your coworkers discuss a coproduct endofunctor?

From the dark corners of mathematics to a programming language near you, category theory offers a compact but powerful set of tools to build and reason about programs. If you ever wondered what’s a category or a functor and why care, this series might be just what are you looking for.

But don’t wait! If you call now, you’ll get this explanation of dual categories!

Next time, you too can be the soul of the party and impress your friends with category theory!*”

*(results may vary)

Intro

Category theory is a branch of abstract math. Why it gets so much attention from (functional) programmers?

As it happens, modeling programs using category theory allows us to apply theoretical results directly to our code, explore new approaches to existing problems, and increase our confidence on the solutions. At first, category theory might seem impenetrable, but one can go far by learning the basic vocabulary

But let’s go to the beginning

What’s one of the most important technique for programming?

ABSTRACTION!

 

Removing unnecessary detail and keeping the essence is an extremely powerful tool for programming.

What if we dial it to eleven?

Let’s abstract over all the characteristics of the things we want to model, and just end with “things” (called objects) and the connections between them (called arrows, or if you want to get really fancy, morphisms).

Just things and the connections between them:

 

 

To make a category, we are going to require only two things: every object is connected with itself (identity) and if object A is connected with object B which in turn is connected with object C, we can consider that object A is connected with object C (composition)

If we formalize the definition:

A category Cat is structure consisting of:

Obj(Cat): collection of objects.

For each A,B ∈ Obj(Cat), there’s a set C(A,B) of morphisms from A to B

f:A→B means f ∈ C(A,B)

(In other words, for every pair of objects A and B, there’s a bunch of arrows connecting them… or not)

A composition operation between arrows:

if f:A→B and g:B→C, then g∘f:A→C

(I can make a “new” arrow connecting the end of f with the beginning of g )

For each object X, exists an identity arrow:

IdX:X→X

We’re going to have only two requirements (laws) for the identity and composition of a category:

Identity as unit

For any arrow f:A→B,

f∘IdA = f = IdB∘f

(f composed with identity on A is equal to f and is equal identity on B composed with f)

Composition is associative

f∘(g∘h) = (f∘g)∘h

(it doesn’t matter if I compose f and g first or g and h first, the resulting composition is the same)

Some mathematical examples of categories are:

Set:  the category where the objects are sets and the arrows are functions from one set to another

Pfn: the category of sets and partial functions

How is related to programming?

Yes, most examples in category theory are from math, but what about programming? If we consider the types of a program and the functions between those types, we can form a category: function composition will be our arrow composition and the identity function applied to each type will be our identity morphism. (with the caveat that we have to consider all our functions total and ignore non-termination [infinite loops, exceptions, etc],  also known as bottom ‘_|_’ ).

This is a toy program that takes a String, parses it as Int, divides by two, and gets the byte value of the result.

In Scala:

def toInt(s: String) = s.toInt

def divByTwo(i: Int): Float = i/2f

val program: String => Byte = (toInt _) andThen (divByTwo _) andThen (_.byteValue)

Scala provides an identity function,  so we know that identity[String] , identity[Int], identity[Float], and , identity[Byte] exists. Also, andThen acts as function composition in Scala

Now, is easy to see that (toInt _) andThen (divByTwo _) andThen (_.byteValue) == (toInt _) andThen ((divByTwo _) andThen (_.byteValue)) == ((toInt _) andThen (divByTwo _)) andThen (_.byteValue)

Since identity is defined as def identity[A](x: A): A = x , we can verify identity andThen f == f == f andThen identity is true for any f

So, if we squint and pretend the functions are total, we have:

Objects:  String, Int, Float, Byte

Arrows: toInt _divByTwo _ , _.byteValue

Id: identity

composition:  andThen

identity is neutral and andThen is associative.

So we can model our program with category theory, and take advantage of it.

That’s where the applicability of concepts like Functor, Monad, Natural transformations, etc.  come from.

In our next articles we’re going to expand on why that’s useful… stay tunned.

How to create an Akka HTTP Server

The Akka HTTP modules implement a full server-side and client-side HTTP stack on top of akka-actor and akka-stream. It offers two different API with different levels of abstraction: a high-level one and a low-level one.

The high-level routing API of Akka HTTP provides a DSL to describe HTTP “routes” and how they should be handled. Each route is composed of one or more levels of Directive s that narrows down to handling one specific type of request.

The low-level Akka HTTP server API allows for handling connections or individual requests by accepting HttpRequestobjects and answering them by producing HttpResponse objects.

Adding the Akka-HTTP dependency

Akka HTTP is provided in a separate JAR file, to use it you should include the following dependency in your build.sbt file

"com.typesafe.akka" %% "akka-http" % "10.0.9"

Creating Server basic structure

First we add an Scala file to our project and create an object that extends from App and Directives.

import akka.http.scaladsl.server.Directives

object Server extends App with Directives {}

Then we need to add the ActorSystem and ActorMaterializer to that object.

import akka.http.scaladsl.server._
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer

object Server extends App with Directives {
  implicit val system = ActorSystem("actor-system")
  implicit val materializer: ActorMaterializer = ActorMaterializer()
}

Now we can get the server up by adding the following lines after the implicits.

val routes: Route = path("/test") { complete("test") }

The above line defines an example route /test which returns the text: test

Http().bindAndHandle(routes, "0.0.0.0", 8002)

This final line allows us to bind an IP and port to a set of routes.

Server basic Structure:

import akka.http.scaladsl.server._
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer

object Server extends App with Directives {
  implicit val system = ActorSystem("actor-system")
  implicit val materializer: ActorMaterializer = ActorMaterializer()

  val routes: Route = path("/test") { complete("test") } 
  Http().bindAndHandle(routes, "0.0.0.0", 8002)
}

Defining Routes

Now we are going to replace the example route /test with some more interesting ones.

All the routes we are going to define in our server are implemented using Akka-HTTP directives and have to be assigned to a Route type variable. This variable is the one which will be used as the first parameter of thebindAndHandle method.

The most easy way to add a new route is using the directive ‘path‘ along with a path as we saw in the example route above.

All the directives should finish with a complete method call.

val routes: Route =
  path("test") {
    complete("test")
  }

Inside the path we can use different directives depending on the HTTP method that we want to use. Here is an example using the POST verb.

val routes: Route =
  path("test") {
    post {
      complete("test")
    }
  }

If we want to add a new route we can concatenate two path directives just using the ~ symbol.

val routes: Route =
 path("test") {
   ...
 } ~
 path("test2") {
   ...
 }

Responding requests

Till now we saw how to return a text for an endpoint using the complete method and a text. But what about if we want to return JSON data? Well certainly we can do something like this:

val routes: Route = 
  path("test") { 
    post {
      complete("{\"my_key\":\"my_value\"}")
    }
}

However this JSON will be returned with a wrong content type.

In order to set the right content type we have to use the Akka HTTP low level objects HttpResponse and ResponseEntity

val routes: Route = 
  path("test") { 
    post {
      val resp: ResponseEntity = HttpEntity(ContentTypes.`application/json`,"{\"my_key\":\"my_value\"}")
      complete(HttpResponse(StatusCodes.OK, entity = resp)
    }
  }

Note: Besides HttpResponse object has a parameter to define headers the Content Type can not be defined in that parameter. It has to be defined in the HttpEntity.

Dealing with input data in URLs

If we want to get a number from an URL we should use the IntNumber object.

val routes: Route =
  path("test" / IntNumber) { id =>
    get {
      complete(s"get test - $id")
    } 

But if we want to get the content from one / to the next one (or the url end if there is no one more) we should use the Segment object.

val routes: Route =
  path("test" / Segment ) { data =>
    get {
      complete(s"get test - $data")
    }
  } 

Of course there is more options but IntNumber and Segment are the two more useful ones.

We can also combine more than one input data in the same route as follows:

val routes: Route =
  path("test3" / Segment / IntNumber) { (data, id) =>
    get {
      complete(s"get test3 - $data, $id")
    }
  }

Dealing with input data in request body

One common thing when we work with HTTP requests is to get information from the request body. This information has to be converted into an Scala data type in order to be able to work with it in our Akka HTTP Server.  To do that we could use the entity(as(..)) structure.

Example:

path("test") {
  post {
    entity(as[String]) { param =>
      ...
    }
  }
}

In this example param contains the request body as an String.

This can be accomplished due Akka HTTP includes marshallers for some basic data types. Default marshallers are provided for simple objects like String or ByteString.

Dealing with JSON

Most of the times the request body information comes as a JSON structure and deal with that it’s a little more complex. There is no default marshaller for your own JSON data of course, so you have to define it.

There are some JSON libraries which can help but the most common library for these cases is the spay-json library. And here we are going to use this library to define our own marshaller.

First of all we have to define a case class with all the parameters that we are going to receive in the request JSON body.

So, for example, if our JSON data is something like:

{
  names: ['jhon', 'freddy', 'kurt'],
  id: 10
}

We should define a case class like this:

case class TestAPIParams(names: List[String], id: Int)

But after defining the right case class we have to define the marshaller.

To do that, we have to define a trait extending from prayJsonSupport and DefaultJsonProtocol. Inside this trait we have to define and implicit using jsonFormatX where X is the amount of parameters we have. In our case we should define the trait as follows:

trait JsonSupport extends SprayJsonSupport with DefaultJsonProtocol {
  implicit val testAPIJsonFormat = jsonFormat2(TestAPIParams)
}

Once we have a marshaller for our JSON data we have to extend the object in which we are defining the route.

object Server extends App with Directives with JsonSupport {
  ...
}

And now we can use the entity(as(...)) structure with the our case class

path("test") {
  post {
    entity(as[TestAPIParams]) { params =>
      ... // We can use params.names and params.id
    }
  }
}

Final comments

It may seem a little difficult to develop a server over Akka HTTP from scratch and certainly there are complex things in the library. However following these steps it’s easy to start and have the basic server structure and functionality quite soon.  Then it’s only hard work and read the documentation. Luckily Akka HTTP has a very complete and clear official documentation, fully of examples.

5 Tools for Managing Remote Teams

There are lot of new tools available to help you effectively manage the work of remote team members. Those tools will help you to improve communication, project management and development setups.

1 – Chat rooms: Slack/Gitter/flowdock

Slack, Gitter and Flowdock are effective communication tools that allow you to create channels to talk about specific topics. These are fantastic tools when you have workers all around the globe. It’s a good way of making communication less overkill and you can assign users to channels according to your needs. They also have voice/video calls for free.

2 – Cloud storage provider: Dropbox/Google Drive

Your team needs to access documents, and Dropbox and Google Drive are the best option to do it. They are simple apps. The user just need to drop a new file and assign permissions to others users that will be notify when changes are made. They also have multi edit functionalities that enable two or more users to edit the same document simultaneously.

3 – Issue Tracker: Jira

Issue trackers like Jira have a lot of plug-in that help your team to manage requirements and projects. This helps teams to organize work, document and prioritize tickets. Define releases and dates. Jira integrates well with code repositories like Git, so when a developer closes a task, the ticket will be automatically updated. Managers can create their own dashboards to see the status of each ticket, and you can use some Tempo plugins to load hours. This is a must in any remote development team.

4 – Software container platform: Docker

Docker.com says “Developers use Docker to eliminate “it works on my machine” problems when collaborating on code with co-workers. Operators use Docker to run and manage apps side-by-side in isolated containers to get better compute density. Enterprises use Docker to build agile software delivery pipelines to ship new features faster, more securely and with confidence for both Linux and Windows Server apps.”
Basically you can create docker containers that will include all the dependencies and products you need to build your software and distribute them to your developers in a very easy way. That will make developers life easier.

5 – Continuous Integration tools: Bamboo/Jenkins

This CI tools will help you to build your software and run all your test to detect issues sooner. With distributed teams you don’t want to have your repository broken. These apps will help you to maintain your repository in a good state, and they can build and publish your app directly to test environments. If we integrate this with JIRA and Git, a developer can push a fix to a bug in Git that will automatically move the ticket in JIRA to “Ready to Test” and Bamboo will build the app, run the tests, notify if something went wrong, and publish the app to QA. This is a great feature when you manage remote teams.

When working with remote team members you want to automate all your process to avoid blocking someone’s work. These apps help you to make that possible and to have everything on the cloud. Each member will know where to go to get the information and how processes work. I strongly recommend to use these apps no matters if your team is remote or not, because with these apps you can highly increase your team performance . Now, for a remote team, this should be a must.

New technologies: Big challenge for non-dynamic organizations?

Innovation should be thought as a process, not as a goal. The world is changing very quickly and organizations need to move fast, so it’s not about how to innovate but to create the space to let the innovation happen.

Current mid-range level managers are Baby Boomers or Gen X that have a big challenge to manage new generation developers.

There are very few old school developers willing to learn new languages, most of them just stay coding in the same language they started coding, so we can think that new technologies are going to be mastered by new developers, I mean, Millennial developers.

Millennials understand the world in a different way, not better or worse, just different. They are not affected by the technological changes around them, and they actually enjoy learning new technologies. This is why Millennials will master new languages naturally, and that’s why companies who want to innovate will need to know how to deal with them.

So, innovation translates to our capability to give Millennials the space to learn and build new stuff. Current managers will need to adapt themselves to work with them, and that means to accept remote working, informal communication, and setting up goals instead of requesting 8 hours per day, etc.

How is your company dealing with this? Is it dealing with it at all?

Some startups are starting to accept Remote Working as they know that they can find better skilled developers faster and for less money. They only care for what the developers deliver and not the time they spend doing it. There are a lot of issue tracker tools that help to organize and supervise work . There are a lot of instant messaging tools available to have real time discussions by topic that help everyone in the organization to be updated with the latest status. Millennials handle these tools naturally. X Gen developers are also incurring in this process as they understand that this impacts positively in their performance.

Nowadays we can know where our Uber is, how long it will take to arrive and how much our trip will cost. So who is going to waste time in a corner waiting for a cub? The same happens with the new ways of working. If you have the chance to hire developers remotely and do the job easily for less money, why are you not doing it?