A buildpack provides framework and run time support for applications. Buildpacks examine user-provided artifacts to determine what dependencies to download and how to configure applications to communicate with different services.
When we push an application, Cloud Foundry automatically detects which buildpack is required and installs it on the Droplet Execution Agent where the application needs to run.
There are three elements of a buildpack. They are detect, compile and release.
Detect
A script to detect condition to run buildpack. Runs this detect script against deployed application. OK if exit 0. To get an idea, let's look into the detect script of ruby buildpack.
#!/bin/bash
if [ -f "$1/Gemfile" ]; thenSCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" echo "ruby $(cat $SCRIPT_DIR/../VERSION)" exit 0 else echo "no" exit 1 fi
Detect can be written in ruby or bash.
There are some examples that detect itself is written in bash and then calls python script. cf-php buildpack is similar to that.
#!/bin/bash
BP=$(dirname $(dirname $0)) export PYTHONPATH=$BP/lib VERSION=`cat $BP/VERSION` python $BP/scripts/detect.py $1 $VERSION
Compile
This is the key script to setup language or framework runtime setup.
What this does is, it downloads execution environment sources and then compiles.
Because it takes time to do compile, many buildpacks download pre-compiled binaries.
Release
A script to return metadata required for execution in yaml format.
#!/usr/bin/env ruby require 'pathname' release = Pathname.new(ARGV.first).join("tmp/heroku-buildpack-release-step.yml") puts release.read if release.exist?
Staging
Staging refers to building the droplet, which is the executable bundle of application. At this time, user deploys the application, process the application by buildpack description and then create the droplet. Let me describe this graphically.
1. When user gives the command 'cf push', user uploads the application to Cloud Controller.
2. Cloud controller sends the 'staging.start' message to DEA to request staging activity.
3. DEA does staging and build the droplet.
4. Once droplet is built, it returns back to cloud controller.
The above diagram explains the complete scenario.
Admin buildpack
This is a buildpack which is directly installed in Cloud Foundry. Suppose that user did not specify a buildpack. Then Cloud Foundry selects the appropriate buildpack from the admin buildpack. Administrator can add using create-buildpack. Cloud Foundry service provider can add supported languages to the platform.
Staging.start is received by DEA.
Download application from Cloud Controller.
Execute admin buildpack detect based on priorities.
Run buildpack compile which matched.
Download Ruby binaries, setup binaries, execute 'bundle install', replace database.yml(in case of rails) etc.
Execute release and produce necessary information.
Droplet contents.
In case of a Ruby app, the droplet produced consists ruby executable created by the buildpack, uploaded application, files installed by bundle install and start command description.
System Buildpacks
This table describes Cloud Foundry system buildpacks. Each buildpack row lists supported languages and frameworks and includes a GitHub repo link. Specific buildpack names also link to additional documentation.
| Name | Other Supported Languages and Frameworks | GitHub Repo |
|---|---|---|
| Java |
Grails, Play, Spring, or any other JVM-based language or framework
| Java source |
| Ruby |
Ruby, Rack, Rails, or Sinatra
| Ruby source |
| Node.js |
Node or JavaScript
| Node.js source |
| Binary |
NA
| Binary source |
| Go |
NA
| Go source |
| PHP |
NA
| PHP source |
| Python |
NA
| Python source |
| Staticfile |
HTML, CSS, or JavaScript
| Staticfile source |
Other Buildpacks
If a certain application uses a language or framework that Cloud Foundry buildpacks do not support, we can try the following ways.
1. Customize an existing buildpack, or create your own custom buildpack.
For example, by default Cloud Foundry does not support Haskell. Using a buildpack for Haskell allows us to add support for it at the deployment stage. When we push an application written using Haskell, the required buildpack is automatically installed on the Cloud Foundry Droplet Execution Agent (DEA) where the application will run.
A common development practice for custom buildpacks is to fork existing buildpacks and sync subsequent patches from upstream.
3. Use a Heroku Third-Party Buildpack.
Heroku maintains a collection of officially supported buildpacks but there are also third-party buildpacks that enable us to use languages and frameworks beyond those that are officially supported by Heroku.
Custom buildpack
User can specify the buildpack at the time of deployment. This is called custom buildpack. When custom buildpack is specified, detect step is skipped, compile step is executed.
Binary buildpack
This is a special buildpack that I came across. This is a Cloud Foundry buildpack for running arbitrary binary web servers.
Unlike most other Cloud Foundry buildpacks, the binary buildpack must be specified in order for it to be used in staging your binary file. You can specify the buildpack your app should use with the -b option for cf push as follows.
cf push my_app -b https://github.com/cloudfoundry/binary-buildpack.git
There are two ways to provide Cloud Foundry with the shell command to execute your binary.
1. Procfile
Include in your app's root directory a Procfile that specifies a web task. Eg: web: ./app2. Command line
Alternatively, you can provide the start command when deploying your app with cf push by including a -c option:
cf push my_app -c './app' -b binary-buildpack
Container to run application
Warden
Warden is a container to isolate user application. User application runs inside Warden container. When the application stops, then the container gets destroyed.
Currently, in addition to run the application, staging process itself and also used to run component in Bosh-lite environment.
dea_next uses Warden Client to invoke Warden Server API. Communication between Warden is determined by Warden protocol. Warden Server is written in Ruby and and the container which Warden Server runs is written in C.
Why Warden(vs Docker)
- Docker only runs a single process, it does not conform Cloud Foundry architecture.
- Cannot control limit after the container creation.
- Cannot control disk and bandwidth limits.
The main purpose is ultimately the same in Docker and Cloud Foundry.
