gyptazy

DevOps

Developer

IT Consultant

gyptazy

DevOps

Developer

IT Consultant

Blog Post

HowTo: Kleene as a Container Management Platform for FreeBSD

HowTo: Kleene as a Container Management Platform for FreeBSD

Kleene is a container management (jail manager) platform designed specifically for FreeBSD, bringing familiar concepts from Docker while fully embracing FreeBSDs native tools and philosophy. Instead of reinventing the wheel, Kleene follows the KISS (Keep It Simple, Stupid) principle, leveraging FreeBSD’s built-in features to simplify application deployment, maintenance, and upgrades without unnecessary complexity or abstraction.

By using FreeBSD’s core technologies like jails, ZFS, and PF (Packet Filter), Kleene provides a streamlined way to build, run, and manage containerized applications while maintaining transparency and control. Key functionalities include:

  • Jailed Applications: Running lightweight, isolated environments using ZFS for storage efficiency.
  • Native Networking: Automatically setting up necessary network devices and configurations.
  • Firewall Integration: Managing connectivity securely with FreeBSD’s PF firewall.

Unlike some container solutions that introduce heavy layers of abstraction, Kleene stays true to FreeBSD’s “Power to Serve” motto, ensuring users retain full visibility and control over their system. If you’re familiar with Docker, many of the ideas will feel familiar, but Kleene is built exclusively with FreeBSD’s native tools where no further rocket science is involved.

For those new to containers, reading Docker’s high-level overview is a good starting point, though it’s important to note that Kleene has some fundamental differences from Linux-based containerization.

Kleene makes container management on FreeBSD simpler, more transparent, and more efficient and you as you can already see – it’s more a jail manager on steroids adapting the ideas of Docker.

Architecture

Kleene is built on a client-server model. Therefore, Kleene mainly consists of two main components: The client, called Klee (written in Python), sends instructions to the server. The server, Kleened (mostly written in the Elixir programming language using the underlying OTP-framework and the BEAM VM of Erlang), which handles all the tasks related to building and running containers. Both Klee and Kleened can operate on the same computer, or Klee can connect to a remote Kleened server. The communication is then handled by a REST API, using either UNIX sockets or a network interface.

However, we need to interact with some objects and therefore, we can also add a third component: The Kleene objects which can be images, container, network, volumes and other objects.

Requirements

Kleene has minimal requirements on FreeBSD, needing only FreeBSD 13.x or 14.x, along with the ZFS and PF firewall kernel modules. With these components enabled, Kleene can be used without additional dependencies or complex setup. In fact, it simply requires:

  • FreeBSD 13.x or 14.x.
  • ZFS kernel module
  • PF firewall kernel module

Installation: Kleened

Kleene components are not yet included in the official FreeBSD ports and packages. However, everything is already in place in dedicated repositories to make the usage and installation as easy as possible for users by simply installing it by pkg. In this HowTo we will run the kleened and the client tool klee on the same host.

FreeBSD 14.x

fetch https://github.com/kleene-project/kleened/releases/download/v0.1.0-rc.1/kleened-0.1.0rc1_FreeBSD14-amd64.pkg
pkg install kleened-0.1.0rc1_FreeBSD14-amd64.pkg

FreeBSD 13.x

fetch https://github.com/kleene-project/kleened/releases/download/v0.1.0-rc.1/kleened-0.1.0rc1_FreeBSD13-amd64.pkg
pkg install kleened-0.1.0rc1_FreeBSD13-amd64.pkg

Ports

git clone https://github.com/kleene-project/ports.git kleene-ports
cd kleene-ports/sysutils/kleene-daemon
make install

Post-Configure

After installation, you should first validate the generated config in /usr/local/etc/kleened/config.yaml where you can adjust some minor things, like the root ZFS dataset where Kleened stores images, containers and volumes, but also the pf firewall config template and some other things.

Afterwards, we need to do the typical service management related things, like enabling the service etc. Therefore, we can simply run the following commands to enable the service and to initialize the required kernel modules:

sysrc kleened_enable=YES
# Dry-run: service kleened dryinit (see below)
service kleened init

With the command service kleened dryinit you can also simply get an overview of things that need to be done to make kleened work properly on your system. As a result, you get the following overview:

root@gyptazy-kleene01:~ # service kleened dryinit
Verifying kernel module zfs is loaded...OK
Verifying kernel module pf is loaded...not loaded, trying to load..OK
Verifying kernel module pflog is loaded...not loaded, trying to load..OK
Verifying if 'zfs_load' is set to "YES" in /boot/loader.conf...OK
Verifying if 'zfs_enable' is set to "YES" in /etc/rc.conf...OK
Verifying if 'pf_load' is set to "YES" in /boot/loader.conf...not enabled, enabling...OK
Verifying if 'pf_enable' is set to "YES" in /etc/rc.conf...not enabled, enabling...OK
Verifying if 'pflog_enable' is set to "YES" in /etc/rc.conf...not enabled, enabling...OK
Verifying if zfs dataset zroot/kleene exists...does not exist, creating...OK
Verifying if zfs dataset zroot/kleene/image exists...does not exist, creating...OK
Verifying if zfs dataset zroot/kleene/container exists...does not exist, creating...OK
Verifying if zfs dataset zroot/kleene/volumes exists...does not exist, creating...OK
Verifying rctl is enabled...error!
Rctl does not seem to be enabled, so container resource limiting will not work.
Set kern.racct.enable=1 in /boot/loader.conf and reboot the system to enable rctl.

In all cases you, should make sure kern.racct.enable=1 is enabled in /boot/loader.conf.

Installation: Klee

After installing the Kleened service as the backend application, we also need to install our client management tool, called Klee. Klee can not only be used on FreeBSD based systems but also on other ones, such like Debian. In all ways, this is installed by pipx since the the application is not yet available in the regular Distribution’s software repositories. I’ll provide an example for the installation for this right here.

FreeBSD

The client utility can simply be installed on FreeBSD based systems by running the following commands:

pkg install py311-pipx
pipx install kleene-cli
pipx ensurepath

Debian

The client utility can simply be installed on Debian based systems by running the following commands:

apt-get install pipx
pipx install kleene-cli

Usage

We’ve finished setting up all the necessary tools for both the backend and frontend, allowing us to begin containerizing applications. In this case, we’ll be working on containerizing Nginx.

Prepare Container Host

First, we to prepare the container host node where kleened is running. Therefore, we create an initial base image including the whole userland content and create a new subnet (here: dev01-gyptazy-nginx01) where we will place the container.

klee image create fetch-auto
klee network create --subnet 10.11.12.0/24 dev01-gyptazy-nginx01

Build Container Image

Finally, we can create our container image.

mkdir -p /opt/kleene/nginx && cd $_
vi Dockerfile

We will fill the Dockerfile with the following content, which used FreeBSD 14.1 as our image, installs nginx and replaces the default index.html:

FROM FreeBSD-14.1-RELEASE:latest
RUN pkg install -y nginx
WORKDIR /app
COPY . .
RUN echo "hello from the container served by klenee" > /usr/local/www/nginx/index.html
CMD service nginx enable && service nginx start

Afterwards, we can build the container image and will simply name it nginx:

klee build -t nginx .

Example:

gyptazy-kleene01:/opt/kleene/nginx # klee build -t nginx .
Started to build image with ID 62b6c616e00c
Step 1/6 : FROM FreeBSD-14.1-RELEASE:latest
Step 2/6 : RUN pkg install -y nginx
Bootstrapping pkg from pkg+https://pkg.FreeBSD.org/FreeBSD:14:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[62b6c616e00c] Installing pkg-1.21.3...
[62b6c616e00c] Extracting pkg-1.21.3: 100%
Updating FreeBSD repository catalogue...
[62b6c616e00c] Fetching meta.conf: 100%    178 B   0.2kB/s    00:01    
[62b6c616e00c] Fetching data.pkg: 100%    7 MiB   7.5MB/s    00:01    
Processing entries: 100%
FreeBSD repository update completed. 35857 packages processed.
All repositories are up to date.
Updating database digests format: 100%
The following 2 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	nginx: 1.26.2_9,3
	pcre2: 10.43

Number of packages to be installed: 2

The process will require 9 MiB more space.
2 MiB to be downloaded.
[62b6c616e00c] [1/2] Fetching nginx-1.26.2_9,3.pkg: 100%  558 KiB 571.2kB/s    00:01    
[62b6c616e00c] [2/2] Fetching pcre2-10.43.pkg: 100%    1 MiB   1.5MB/s    00:01    
Checking integrity... done (0 conflicting)
[62b6c616e00c] [1/2] Installing pcre2-10.43...
[62b6c616e00c] [1/2] Extracting pcre2-10.43: 100%
[62b6c616e00c] [2/2] Installing nginx-1.26.2_9,3...
===> Creating groups
Using existing group 'www'
===> Creating users
Using existing user 'www'
[62b6c616e00c] [2/2] Extracting nginx-1.26.2_9,3: 100%
=====
Message from nginx-1.26.2_9,3:

--
Recent version of the NGINX introduces dynamic modules support.  In
FreeBSD ports tree this feature was enabled by default with the DSO
knob.  Several vendor's and third-party modules have been converted
to dynamic modules.  Unset the DSO knob builds an NGINX without
dynamic modules support.

To load a module at runtime, include the new `load_module'
directive in the main context, specifying the path to the shared
object file for the module, enclosed in quotation marks.  When you
reload the configuration or restart NGINX, the module is loaded in.
It is possible to specify a path relative to the source directory,
or a full path, please see
https://www.nginx.com/blog/dynamic-modules-nginx-1-9-11/ and
http://nginx.org/en/docs/ngx_core_module.html#load_module for
details.

Default path for the NGINX dynamic modules is

/usr/local/libexec/nginx.
--> Snapshot created: @52ec34e72917
Step 3/6 : WORKDIR /app
Step 4/6 : COPY . .
--> Snapshot created: @e53c104e059c
Step 5/6 : RUN echo "hello from the container served by klenee" > /usr/local/www/nginx/index.html
--> Snapshot created: @c18b953a6379
Step 6/6 : CMD service nginx enable && service nginx start 

image created
62b6c616e00c

Our image is now created and available under the name nginx.

Image Validation

We can simply validate and list the present images on our system by running klee image ls command:

gyptazy-kleene01:/opt/kleene/nginx # klee image ls
 ID             NAME                   TAG      CREATED        
───────────────────────────────────────────────────────────────
 62b6c616e00c   nginx                  latest   5 minutes ago  
 0d3c34719874   FreeBSD-14.1-RELEASE   latest   14 minutes ago

Looks good, let’s start the image!

Run an Image

Finally, we can start our newly created image. The easiest way to do so, is simply to use the run parameter. To make sure that we can access the container we assign it to our previously generated network. It’s up to you how you like to manage your networks but here, we simply attach it to the created one and detach the nginx image. Keep in mind, that you can also define fixed IPs by passing the args –ip or –ip6 which might become handy at some other cases.

# Example:
# klee run -n network -d nginx
klee run -n dev01-gyptazy-nginx01 -d nginx

The container is now executed and should be up and running.

Access the Container

To check which containers are running along with their IP addresses, use the jls command. This will display the JID, IP Address, Hostname, and Path of the running containers. Once you have the IP address of a container, you can test if it is serving content by using curl <container-ip>. If the container is properly configured and running, the response should display the content served by it, such as “hello from the container served by klenee.”.

gyptazy-kleene01:/opt/kleene/nginx # jls
   JID  IP Address      Hostname                      Path
     6  10.11.12.1      ed8d766138aa                  /zroot/kleene/container/ed8d766138aa

gyptazy-kleene01:/opt/kleene/nginx # curl 10.11.12.1
hello from the container served by klenee

Conclusion

The Kleene project brings a Docker-like experience to FreeBSD, making containerization more accessible while staying true to the system’s core strengths. By closely mirroring Docker’s syntax, Klee provides a familiar environment for Linux users who are accustomed to containerization but may not have deep knowledge of FreeBSD’s jails. At the same time, it fully leverages FreeBSD’s native security and efficiency, ensuring a robust and lightweight solution without compromising on performance.

For those transitioning from Linux, Kleene simplifies the learning curve, making it easier to adopt FreeBSD’s powerful jail system without immediately needing to master its intricacies. This lowers the barrier for newcomers while still retaining the flexibility and security benefits of jails. However, it’s important to recognize that Klee adds an additional management layer rather than being a direct alternative to native jail configurations. Unlike the recent Podman integration and Linuxlator improvements, which enhance Linux container compatibility, Klee is fundamentally a more advanced jails manager, specifically designed to streamline container workflows on FreeBSD.

Overall, the project looks promising, especially for users who appreciate a more user-friendly approach to managing jails. While it simplifies handling for beginners, it also introduces another abstraction layer that some experienced FreeBSD users may find unnecessary. Nonetheless, for those seeking a Docker-like experience on FreeBSD without sacrificing the system’s security and efficiency, Kleene offers an interesting and practical solution.

Taggs: