Architecture

Ferry is one server program, ferryd, and one command-line client, ferry (the CLI). This page shows how the parts of ferryd work together.

For the code-level details, read DESIGN.md in the repository.

The partsCopy Anchor Link

The store has the desired state: which deploy is live for each service, how many instances it wants, and which datastores exist. Docker has the actual state: what runs now. The engine makes Docker agree with the store.

The API does not wait for long work. For a deploy, it puts the deploy in the queue and answers 202. The client then follows the log of the deploy as a stream.

The proxy never reads the database for a request. It reads the route table in memory, which gives the instances of each hostname. The engine updates this table after each deploy and each pass of the reconciler.

When ferryd starts, it reads its options and locks the data directory. Then it claims the Docker name prefix. Then it starts the engine, the proxy and the API, in this order.

What happens during a deployCopy Anchor Link

A deploy changes a source into instances behind the proxy. The source is a git commit, an uploaded folder, an image, or the image of an earlier deploy.

ferry deploy api --follow

The engine runs the deploy in the background, in these steps:

1. Queue. The deploy gets the status queued. Each service has one deploy worker, which runs the deploys of the service one at a time, in order. A new deploy cancels the older deploys of the same service that wait in the queue.
2. Build. The engine gets an image. For a git commit or an upload, it runs docker build. For a service that runs an image, it pulls the image. A restart or a rollback builds nothing: it uses the image of the live deploy or of an earlier deploy. The server builds 2 images at a time (--build-concurrency).
3. Snapshot. The engine fixes the environment, the port, the command, the disk, the image and the limits. It saves them with the deploy as the snapshot. Later instances of this deploy start from it too, for example after a crash. Thus a change of settings, limits or variables applies only at the next deploy or restart.
4. Start. The engine starts the new instances next to the old instances. This is a blue-green deploy deploy. A service with a disk uses recreate: the old instance stops first, and the service is not available for a short time.
5. Health check. Each new instance must pass a health check in 120 seconds or less (--health-check-timeout). The test is an HTTP request to the health check path. If the service has no path, the test is a TCP connection to its port. A background worker passes when its process still runs after 5 seconds.
6. Switch. The routes change to the new instances in one step, and the deploy becomes live. The engine waits 2 seconds for the requests in progress on the old instances. Then each old instance has 10 seconds to stop.
7. Clean up. The engine keeps the 5 newest images of the service (--keep-images) and deletes the older images. A rollback needs an image that still exists.

The deploy log shows each step. The lines that start with ==> come from Ferry:

09:38:58 ==> Starting deploy dep-3271cdfd1415400d9a5e (trigger: blueprint)
09:38:58 ==> Using image nginx:alpine (already present)
09:38:58 ==> Using port 80 (service setting)
09:38:58 ==> Limits: 512 MiB memory, 1 CPU per instance
09:38:58 ==> Starting 2 instance(s)
09:38:58 ==> Started instance 9c5087 (port 80, published on 127.0.0.1:63120)
09:38:58 ==> Started instance 77122e (port 80, published on 127.0.0.1:63121)
09:38:59 ==> Health check passed
09:38:59 ==> Routing traffic to the new instance(s)
09:38:59 ==> Your service is live 🎉

If a deploy failsCopy Anchor Link

A failed deploy never changes what is live. The engine removes the new instances, and the old instances continue to answer. A failure before the image is ready gives the status build_failed. A failure at the start or at the health check gives deploy_failed.

You can cancel a deploy with ferry cancel until the switch. After the switch, roll back to undo it. If ferryd stops during a deploy, the deploy fails with the error interrupted by server shutdown or interrupted by server restart. The live deploy stays live.

The reconcilerCopy Anchor Link

The reconciler compares the store with Docker and repairs the difference. Thus an app that crashed comes back with no action from you.

In each pass, the reconciler also writes the routes again from the instances that run and accept connections.

When it runsCopy Anchor Link

Two passes never run at the same time.

A deploy holds the service lock while it starts its instances. The reconciler does not wait for this lock. It skips a busy service until its next pass.

What it never doesCopy Anchor Link

The reconciler never changes your settings, variables or instance counts. It never deletes a volume. It never touches the containers of another name prefix.

The two watchersCopy Anchor Link

Two tasks run between the passes:

• The route watcher checks the host ports of the instances each second. When Docker restarts a container, the port can change, and the routes follow it.
• The OOM watcher writes a warning in the server log when the kernel kills a container because of its memory limit.

instance a1b2c3 of service 'web' ran out of memory (limit 512 MiB) — raise the service's memory limit

The networking modelCopy Anchor Link

Containers talk to each other on a private Docker network. The outside reaches the public services only through the proxy.

The private networkCopy Anchor Link

Each environment has one bridge network. The network of production has the name prefix of the server as its name: ferry by default. The network of another environment is <prefix>-env-<name>, for example ferry-env-staging. Each service container, datastore and job run joins the network of its environment only. On this network, the DNS alias of a container is its name. Docker does not connect two bridge networks, so a name of one environment does not resolve in another. Thus each environment can use the same names: http://api:8000 in staging is the api of staging.

Do not write these addresses by hand. Use references such as ${{service.api.internalUrl}} and ${{datastore.app-db.connectionString}}.

The portsCopy Anchor Link

Each container that listens publishes its port on 127.0.0.1, with a random host port. The proxy and the health checks connect to this address, never to the IP address of the container. There are three reasons for this:

• It works on Docker Desktop. On macOS and Windows, the host cannot reach the IP addresses of the containers.
• Nothing is open by accident. Only the host can reach these ports.
• No port conflict. The old and the new instances of a blue-green deploy use the same container port.

Background workers and cron jobs publish no port. Your app must listen on the port in $PORT, on 0.0.0.0. An app that listens on 127.0.0.1 in its container fails the health check.

The proxyCopy Anchor Link

The proxy finds the service from the Host header of each request.

The proxy sends the requests to the instances of a service in turn (round-robin). It supports HTTP/1.1, HTTP/2 and websockets. With --https-addr and --acme-email, it gets a certificate from Let’s Encrypt for each public hostname, and renews it. Local hostnames, such as *.localhost, stay on HTTP.

When the proxy answers by itself, the response has the x-ferry-error. Thus you know that the answer did not come from your app.

Where the data livesCopy Anchor Link

The files of ferryd are in its data directory. The option is --data-dir, and the default is ./ferry-data. Only one ferryd can use a data directory at a time.

Docker has the images, the containers, the private network and the volumes. Ferry finds its own resources by the name prefix (--name-prefix, default ferry). Each container also has the label ferry.instance, with the prefix as its value. Thus many Ferry servers can share one Docker daemon. See Multiple servers.