Integrating Docker with Cloud Foundry

Docker and Cloud Foundry are two of my favorite topics. So I thought why not put them together and write a blog post on integration of Docker in Cloud Foundry.  Let's see how these two technologies work together. Both of them try to solve one problem. You have software and you want to ship and deploy that software. That's what both technologies suppose to do. But they are completely different. Cloud Foundry has all the stuff around it such as combining services, health monitoring, load balancing, scaling and so it is very complex. Docker on the other hand is pretty light weight, allows to create a container for your application, use it and ship it wherever you want to. So there are interesting use cases if you start using them together.

There are many solutions for deploying your docker containers in Cloud Foundry. In this post, I will be focusing on some of them. 

Docker and Cloud Foundry

                                1. Docker Buildpack
                                2. Docker Service Provider
                                3. Cloud Rocker
                                4. Diego
                                5. Lattice

1. Docker Buildpack

A buildpack creates the environment for an application to run. A docker buildpack uses all the benefits of the Cloud Foundry platform for Docker containers. Doug Davis of IBM had been capable of creating a Docker buildpack. 
You can get it from
This Docker buildpack allows me to just push my Docker container up to Cloud Foundry using the same cf CLI tool that we use with pivotal. You can run the command, 
                              cf push <name of Docker image>

So this approach is really cool. The problem with this approach is that it requires an external Docker host to be maintained some where outside of Cloud Foundry. So if you host your own Cloud Foundry system or Docker farm, you will have to face some troubles. And also it does not take a Dockerfile, it takes the URL of a Docker image from public Docker Registry. So it is limited in that way. The explanation at clearly tells that,
"This is just a proof-of-concept and therefore has some limitations but does show what the basic user experience might be like if Cloud Foundry supported Docker in a first class way."

There are some barriers with Health Manager as well. So if a docker container fails or stops, Cloud foundry is not able to detect it with hundred percent guarantee. Besides that, you get all the benefits from Cloud Foundry such as load balancing, log aggregation, authorization to Cloud Foundry system etc. But you don't get the total Cloud Foundry experience or Docker experience. Any way it is a very interesting project.

2. Docker Service Provider

This is a service provider to expose Docker containers as services. If we relate to this to an example like MySQL, MySQL service provider stays between Cloud Foundry installation and MySQL cluster and it is responsible for spinning up the database, configure authentication and so on. In case of Docker service provider, we can configure it in a way that if a certain application requires a Redis service, then it should spin up a Redis container. If it requires a Postgres database, it should spin containers accordingly. So I can tap in to the Docker Hub Registry and look at all those containers that are provided by different people to provide all those database services in order to make them accessible to my application. This is not that difficult to install.

This also has the limitation that this also requires an external Docker host which can be installed through Bosh. Bosh allows to get all this quickly up, run and orchestrated. Here is that corresponding repository.

3. Cloud Rocker

This is a project carried out by a company called 'Cloud Credo'. What they tried to do is to build docker images using Cloud Foundry buildpacks. The expectation is to provide rapid turnaround for local development. This is also an interesting project. 

This is how it works. You have your application and the Cloud Rocker installation running. Then you upload your buildpack to Rocker system, which is a VirtualBox machine. Application is deployed using that buildpack and output of that will be a docker container. So here it uses the buildpack system for Cloud Foundry and the output is a docker image which I can push it to wherever I want to. So that's really cool. This makes lot of sense if you want to leverage the buildpack system and create a docker container.

So if you don't want to use Cloud Foundry, but you want to use its buildpack system, this is the best choice for you. 

This uses couple of Cloud Foundry tools to run the buildpack. It does not run inside Warden, hence it runs inside a docker container. It uses your internal docker host and it spins up docker containers to deploy your applications into a droplet. 

4. Diego

Deigo is a rewrite of Cloud Foundry DEA (Droplet Execution Engine). DEA uses warden inside to put all things into containers. A language called 'Go' is used to rewrite this. Diego is special beacause it provides built-in-support for Docker containers. So now you are able to push your containers with Diego.

This project started in a small scale and now they have almost rearchitectured the Cloud Foundry system. It has changed how container placement works, how health management works, what information is stored in the Cloud Controller etc. It is abstracting from Linux containers and they are planning to use Windows containers as well.  You are able to push your Docker containers via Diego. Meanwhile you get all the benefits from Cloud Foundry. now supports Diego and you can also experience it locally in your machine by deploying Diego to BOSH-Lite. 

So now you have the option either to push up your application or container. Interesting point here is that you are unable to push your Dockerfile. But you are able to push up your Docker image. The reason for that is Docker files are not stable over the time. 

I am hoping to write a separate blog post on Diego very soon. So I will be ending this topic from here.

5. Lattice

Diego components can be used to run containers, run applications, compile your applications and buildpacks. So people started to think more and more on what we can do with this stand alone component. So they decided to patch that up in a way that you end up with a very light weight PaaS. And they named it as 'Lattice'. 

In short, Lattice is a standalone scheduler extracted from Diego for Docker images. Diego is the core of Lattice and they put some other components around it, manly the routing layer, log aggregation layer etc. With all these packed together, it allows you to push your Docker containers, scale it up and down, do load balancing, monitor health and log aggregation etc. The nice thing about this Lattice system is that it is super light weight. 

So if Cloud Foundry is too much for you, Lattice might be an interesting project for you. Lattice tries to be little simpler and easier to set up with less features around it.

Containers vs Buildpacks : What to choose?

Cloud Foundry allows you to deploy containers and applications through buildpacks. So which should you choose? 

As an application developer, should I develop a container for my application and push up the container or should I push up the application and trust on the buildpack?

Answer to this question is not that simple. It depends on many things. Given below is a general recommendation, but you can select your own preference.

Containers are better when,
                        - Developers require more control
                        - Developers know Operations/Docker
                        - Time to market is important

It seems that the choice of buildpacks or containers depends on the application developer, operator, environment, requirements etc. If we really want to get something really fast, Cloud Foundry recommends to build a docker container and push it, so that we can use that container for local testing as well. But the drawback that lies here is that the developers have to know about Docker and other operations. (eg: how to write a Dockerfile). Thus developers have to be interested in Docker and stay up to date. They say that if the developer needs to set up the environment instead of the operator, this approach would be perfect. 

Buildpacks are better when,
                        - Operations require more control
                        - Developers focus on application
                        - Low maintenance cost is important

Cloud Foundry suggests that buildpacks are suitable if you don't want to use Ubuntu, Red Hat, Fedora, CentOS or different java versions that need to be patched differently or else if you don't want to use different custom things etc. Hence the developer can focus more on application logic and business logic rather than running Dockerfiles. Any way, the buildpack approach also ends up in a container(droplet) eventually.

Low maintenance cost can be achieved with buildpacks because this approach does not end up with very heterogenic container cluster where every container is really different, then you have to come up with different ways when you need to patch something. But with buildpacks, we can patch up one buildpack so that we can deploy all applications and go ahead. I suppose this might be a reason for mentioning it as a low maintenance cost. 

I hope that this has been a nice post to help you clear out most of your doubts on integrating Cloud Foundry with Docker.

1 comment :

  1. nice write up. Top 10 Best Free Open Source DNS Servers.