Building a container runtime from scratch in Go
This is Part 0 of the series Building a container runtime from scratch in Go.
This series of posts is the culmination of the last few months of exploration I’ve been doing to build a container runtime from scratch.
Things started off innocently enough; “Why do we need to run Docker on Linux?,” I was asked, “Isn’t the point of Docker is that it can run anywhere?”
“Our software doesn’t work in Docker running on Windows,” I replied, “…only Docker on Linux.”
Despite being an absolutely true-and-factual answer to the first part of the question, it didn’t feel satisfactory to me. It also completely failed to address the second part, or the broader question in the context of the discussion, the details of which I won’t bore you with.
Trying to find the answer to that question is what set me down the path of building a container runtime, to understand what they are, how they work, and why they (seemingly) don’t work well in non-Linux environments. The answer being blindingly obvious, once we have an understanding of the first two.
Where to start?
A quick Google search for “build a container runtime” returns a bunch of results.
Despite the enticing titles of articles, the short runtime of videos, and the credentials of the authors, it is (I would submit) not possible to build a container runtime in 100 lines of Go, or in 30 minutes, even if you’re authority on the subject. At least, not one I would consider complete. Though, those served as a good introduction to how container runtimes work at a high-level (which, in fairness, is exactly what they aim to do).
Ultimately, I came across the Open Container Initiative (OCI) Runtime Spec1, which “aims to specify the configuration, execution environment, and lifecycle of a container”. It’s also what is implemented by existing popular runtimes, such as:
- runc- “a CLI tool for spawning and running containers on Linux according to the OCI specification”
- youki - “an implementation of the OCI runtime-spec in Rust”
- crun - “A fast and lightweight fully featured OCI runtime and C library”
It’s worth noting that runc is the default container runtime used by Docker and Kubernetes, so this spec definitely seemed like the place to start.
Roadmap for this series
This series of posts documents what it takes to implement (from scratch) the OCI Runtime Spec in Go, and end up with a solution that works as a drop-in replacement for runc in Docker. We shall call our runtime anocir (“an-oh-cheer”) (Another OCI Runtime).
It’s probably going to take a couple of months to get everything typed up, so keep checking back if a section you’re interested in isn’t published yet. If you have a specific question about any of the sections (published or not) then feel free to reach out!
Listing of posts (click to collapse/expand)
- Building a container runtime from scratch in Go 👈️ You are here
- Building the CLI interface for a container runtime
- Reading a bundle config and saving a containers state
- Loading a container, getting its state, and deleting it
- Initialising a container and starting the user process 🔜 Coming soon!
- Executing container runtime lifecycle hooks
- Sending signals to a running container
- Setting up the OCI Runtime Spec test suite
- Using Linux syscalls in Go to configure container environments (overview)
- Wiring up a console to a container
- Isolating a container process using namespaces
- Managing container resources using cgroups & rlimits
- Setting up the root filesystem (rootfs) of a container
- Modifying runtime kernel parameters of a container
- Mounting masked and readonly paths of a container
- Propagating the rootfs mount of a container
- Setting the hostname and domain name of a container
- Setting capabilities and restricting privileges of a container
- Applying scheduling policies and I/O priority to a container
- Applying UID/GID and additional GIDs to a container process
- Writing a container process PID out to file
- Adjusting the OOM score of a container process
- Implementing the ‘features’ API for a container runtime
- Using an alternative container runtime with Docker and other tools
- Thoughts on next steps to improve our custom container runtime (CRI?)
Choice of tooling
Q. Why Go?
A. Why not? It’s like the lingua franca of container tooling development.* It’s also what I’ve been working with most recently…bonus.
Note: I have some retrospective thoughts on the suitability of Go for this project, which I’ll share in a follow up post.
A note on code style
The code is structured and written in a way that makes it easier to present in a series of individually useful blog posts, not to be perfectly ‘idiomatic’ Go.