When talking about open-source virtualization platforms, Proxmox VE is probably one of the first solutions that comes to mind. It combines KVM virtual machines, LXC containers, storage, networking and clustering behind a convenient management interface.

But what if you want to build the same kind of environment on FreeBSD?

This is exactly where Sylve becomes interesting. Some more general information about Sylve can also be found in my blog post about Sylve in general. If you want to skip this, you can find a short summary, now:

Sylve is an open-source virtualization and management platform built specifically around FreeBSD. Instead of putting another abstraction layer on top of Linux and KVM, Sylve embraces technologies that are already native to the FreeBSD ecosystem: bhyve for virtual machines, FreeBSD Jails for lightweight operating system virtualization and ZFS for storage.

The result is something that increasingly feels like a FreeBSD-native alternative to platforms such as Proxmox VE.

bhyve as the Virtualization Foundation

At the core of Sylve is bhyve (and jails), the native FreeBSD hypervisor.

bhyve has been part of the FreeBSD base system for many years and provides hardware-assisted virtualization for running operating systems such as FreeBSD, Linux and Windows.

Technically, bhyve has been capable for a long time. What has traditionally been missing is the surrounding management layer.

Running a handful of bhyve commands manually is perfectly fine for testing. Managing dozens or hundreds of virtual machines this way is a completely different story.

This is where Sylve changes the experience.

Instead of treating bhyve as an individual command-line hypervisor, Sylve provides the management plane around it. Virtual machines can be created, configured, started, stopped and managed from a modern web interface.

CPU, memory, storage, networking, snapshots, consoles and additional hardware configuration can be handled centrally without having to maintain custom shell scripts around bhyve.

FreeBSD, bhyve and ZFS Fit Together Naturally

One of the biggest advantages of this approach is that the complete virtualization stack stays very close to FreeBSD itself.

Sylve uses ZFS as its storage foundation and integrates management of pools, datasets, volumes and snapshots directly into the platform.

This creates a very clean architecture:

FreeBSD
 |
 +-- ZFS
 |
 +-- bhyve
 | |
 | +-- Virtual Machines
 |
 +-- FreeBSD Jails
 | |
 | +-- Lightweight Containers
 |
 +-- Sylve
 |
 +-- Web UI
 +-- REST API
 +-- Networking
 +-- Storage Management
 +-- Clustering
 +-- Backups

Instead of replacing the operating system, Sylve acts as the management layer on top of it.

This is an important distinction.

You are still running FreeBSD. You can still use the operating system exactly as you normally would and benefit from technologies such as ZFS, PF, Jails and bhyve directly.

Sylve simply makes these technologies much easier to operate as a complete virtualization platform.

FreeBSD Jails as the Counterpart to LXC

Virtual machines are only one part of a modern virtualization environment.

On Proxmox VE, administrators can choose between KVM virtual machines and LXC containers depending on the workload.

Sylve follows a very similar concept.

Instead of LXC, the FreeBSD side naturally uses Jails.

Conceptually, this gives us a very familiar mapping:

Proxmox VE Sylve
--------------------------------
KVM VM -> bhyve VM
LXC Container -> FreeBSD Jail
ZFS -> ZFS
Web UI -> Web UI
REST API -> REST API
Cluster -> Cluster

FreeBSD Jails have existed for decades and provide lightweight operating system-level isolation without requiring a complete virtual machine.

For many services, a Jail can therefore be the direct equivalent of using an LXC container on Proxmox.

Sylve makes this especially convenient because bhyve VMs and Jails follow similar management concepts. Networking, storage, resource configuration and lifecycle operations can be managed through the same platform.

This makes it possible to decide per workload whether full hardware virtualization with bhyve or lightweight isolation with Jails makes more sense.

From a Web Interface to Infrastructure as Code

A good web interface is important, but for larger environments it is not enough.

Infrastructure needs to be reproducible.

Virtual machines should not only be created by clicking through a web interface. They should be deployable from Git repositories, CI pipelines and configuration management systems.

This is especially important to me because I have been heavily involved in extending the Ansible automation around Proxmox VE.

Several modules that I worked on for the Proxmox ecosystem made it possible to manage more and more parts of a Proxmox infrastructure directly through Ansible.

Naturally, I wanted the same experience for Sylve.

This resulted in the sylve-ansible project:

https://git.gyptazy.com/gyptazy/sylve-ansible

The project provides Ansible modules for managing Sylve environments through the Sylve API. The first modules focus on the two primary virtualization technologies available within Sylve: bhyve virtual machines and FreeBSD Jails.

The modules are written in a very similar way to the Proxmox Ansible modules, where I was also involved in the development of several modules. The goal was to keep the structure and usage familiar for administrators who already automate Proxmox environments with Ansible.

This does not only make the modules easier to use, but also simplifies their development and future extension. Instead of implementing authentication and API handling independently in every module, the Sylve Ansible modules rely on a common basic utility module for authentication and shared API functionality.

This common foundation makes it possible to add new modules much faster. New modules can focus primarily on the actual Sylve functionality they are intended to manage, while common functionality such as authentication can be reused across the collection.

Managing FreeBSD Jails with Ansible

FreeBSD Jails can be managed directly through the sylve_jail module. The module handles the lifecycle of a Jail and allows its most important resource and system settings to be defined declaratively within an Ansible playbook.

A basic example creates a FreeBSD 14.2 Jail on the zroot storage pool, assigns CPU and memory resources and immediately starts the Jail:

- name: Create and start a FreeBSD jail
 sylve_jail:
 api_url: https://sylve.example:8181
 validate_certs: True
 api_user: admin
 api_token: your-jwt-secret
 ctid: 201
 name: dns01
 state: started
 pool: zroot
 jail_type: freebsd
 base: 14.2-RELEASE
 hostname: dns01.example
 cores: 1
 memory: 1024
 resource_limits: true
 resolv_conf: "nameserver 1.1.1.1\n"

From an automation perspective, this is very similar to provisioning an LXC container on Proxmox VE. The playbook describes the desired guest configuration and the module communicates with the Sylve API to create and manage the corresponding Jail.

Managing bhyve Virtual Machines with Ansible

The same concept applies to bhyve virtual machines through the sylve_vm module.

A virtual machine including its CPU topology, memory, ZFS-backed storage, network interface, VNC configuration and boot behavior can be described directly in the playbook:

- name: Create and start a Bhyve VM
 sylve_vm:
 api_url: https://sylve.example:8181
 validate_certs: True
 api_user: admin
 api_token: your-jwt-secret
 vmid: 101
 name: web01
 state: started
 sockets: 1
 cores: 2
 memory: 268435456 # 256MB
 storage_pool: zroot
 storage_type: zvol
 disk_size: 21474836480 # 20GB
 disk_emulation: virtio-blk
 bridge: public
 network_model: virtio
 vnc_port: 5901
 vnc_resolution: 1024x768
 start_at_boot: true

This creates and starts a bhyve virtual machine with two CPU cores, 256 MB of memory and a 20 GB ZFS volume using VirtIO for both storage and networking.

From an Ansible perspective, the workflow should feel immediately familiar to anyone already using the Proxmox modules. Instead of building a completely different automation model around FreeBSD, the same declarative approach can be used to describe how a guest should look and which state it should have.

The shared utility layer also provides a foundation for extending the project with additional Sylve modules over time. Storage, networking and other platform functionality can follow the same module structure without having to reimplement common authentication and API functionality for every new module.

Moving Existing Proxmox Automation to Sylve

One of the design goals here is particularly interesting for environments that already use Ansible with Proxmox.

The Sylve modules intentionally follow concepts that will feel familiar when coming from Proxmox automation.

That means moving an existing automation workflow from Proxmox to Sylve does not necessarily require rewriting the entire infrastructure definition.

In many cases, the module itself can simply be replaced while much of the surrounding structure and many familiar attributes can remain conceptually the same.

Imagine an existing infrastructure repository containing definitions for CPU, memory, networking, storage and guest state.

Instead of creating an entirely different automation architecture for FreeBSD, the same inventory and playbook concepts can continue to be used.

Conceptually, a migration can therefore look like this:

Proxmox module
 |
 | replace virtualization backend
 v
Sylve module

inventory -> mostly unchanged
variables -> mostly unchanged
roles -> mostly unchanged
CI/CD -> unchanged
Ansible -> unchanged

KVM -> bhyve
LXC -> Jail

Of course, Proxmox VE and Sylve are different platforms and not every platform-specific option can have a one-to-one equivalent.

But keeping the automation model familiar significantly reduces the effort required to experiment with or migrate workloads to a FreeBSD-based virtualization environment.

Instead of replacing the entire automation stack, you primarily replace the virtualization backend.

Automating Jails Instead of LXC Containers

The same approach applies to containers.

If an existing Proxmox environment uses Ansible to deploy LXC containers, the equivalent workload on Sylve can be represented by a FreeBSD Jail.

The automation model remains very similar:

Proxmox:
Ansible
 |
 +-- KVM VM
 |
 +-- LXC Container


Sylve:
Ansible
 |
 +-- bhyve VM
 |
 +-- FreeBSD Jail

This is especially useful when infrastructure definitions are already separated from the virtualization implementation.

An application role does not necessarily need to care whether its target was originally provisioned as an LXC container or is now running inside a FreeBSD Jail.

The provisioning layer creates the guest and the remaining Ansible roles can continue configuring the workload.

The Sylve API Makes This Possible

An important part of this architecture is the Sylve REST API.

The web interface itself is not the only way to interact with the platform. Sylve exposes its orchestration functionality through its backend API, which makes integrations such as Ansible possible.

This is an important architectural decision.

A virtualization platform becomes significantly more useful when the web interface is only one consumer of the underlying API.

It allows environments to be managed from multiple directions:

 +----------------+
 | Web UI |
 +-------+--------+
 |
 |
+-------------+ +-----v------+ +-------------+
| Ansible +----->| Sylve API |<-----+ Other Tools |
+-------------+ +-----+------+ +-------------+
 |
 +------------+------------+
 | | |
 bhyve Jails ZFS

This turns Sylve from a convenient bhyve frontend into something much more interesting: an automation-friendly virtualization management plane for FreeBSD.

A FreeBSD-native Alternative

The goal should not necessarily be to recreate Proxmox VE feature by feature.

FreeBSD already has its own technologies and its own strengths.

bhyve provides the hypervisor.

Jails provide lightweight operating system virtualization.

ZFS provides an extremely powerful storage layer.

FreeBSD provides networking, security and the underlying operating system.

What has historically been missing is a modern management layer connecting these technologies together.

Sylve is increasingly filling exactly this gap, even still some things might be missing at the current time. For me, from an ISP point of view (for BoxyBSD), metrics of guests in Prometheus/Grafana were missing. This way, I crafted an exporter: bhyve-sylve-freebsd-prometheus-exporter.

And with an API and Ansible modules available, it becomes possible to use the platform not only interactively but also as part of fully automated infrastructure deployments.

Keeping the Same Automation Philosophy

For me, this is probably one of the most interesting aspects.

After working on Ansible modules around Proxmox, being able to apply the same automation philosophy to FreeBSD and Sylve makes switching between the ecosystems significantly easier.

An infrastructure repository should describe what should exist.

It should not require an entirely different philosophy just because the hypervisor underneath changed.

With Proxmox, this might mean:

Ansible -> Proxmox API -> KVM / LXC

With Sylve, the same concept becomes:

Ansible -> Sylve API -> bhyve / Jails

The hypervisor changes.

The container technology changes.

The operating system changes.

But the Infrastructure as Code workflow remains familiar.

Conclusion

bhyve has never really been the limiting factor for virtualization on FreeBSD.

The bigger challenge has always been everything around it.

A modern virtualization platform needs storage management, networking, guest lifecycle management, clustering, an API, automation and an interface that administrators actually want to use.

Sylve brings these components together while staying close to the native FreeBSD ecosystem.

With bhyve for virtual machines, FreeBSD Jails as an alternative to LXC containers, ZFS for storage and a REST API for external integrations, Sylve provides a very interesting foundation for building FreeBSD-based virtualization environments.

The addition of Ansible automation makes the platform even more interesting.

Especially for administrators already managing Proxmox environments with Ansible, the transition does not have to mean abandoning existing Infrastructure as Code concepts. The Sylve Ansible modules follow a familiar approach, allowing existing playbooks and infrastructure definitions to be adapted while retaining much of their structure and attributes.

At the same time, the common utility layer behind the modules provides a reusable foundation for authentication and shared API functionality. This makes the Ansible integration easier to maintain and allows support for additional parts of the Sylve platform to be added more quickly through new modules.

Instead of:

Linux + KVM + LXC + Proxmox

we can now build around:

FreeBSD + bhyve + Jails + Sylve

And thanks to Ansible, both can follow the same automation philosophy.

For the FreeBSD ecosystem, this is exactly the kind of tooling that can make bhyve and Jails much more accessible as a complete virtualization platform.