Running Linux-only binaries under FreeBSD

I've taken a real liking to FreeBSD lately, but my biggest initial annoyance was its lack of compatibility with Linux apps and tools I've grown accustomed to.

Despite also being a Unix-like operating system, it isn't able to run many Linux binaries and programs. Spotify is one example I ran into while testing out FreeBSD as a desktop OS. Due to certain DRM library dependencies (*cough cough*, Widevine CDM), it's not able to run natively. This would be a deal-breaker for me, but luckily, there's a quick and rather easy workaround, using a virtualization concept called Jails, and FreeBSD's Linux binary compatibility.

Jails are FreeBSD's approach to OS-level virtualization.  ¹ Each Jail operates independently with its own file system, network stack, and process tree, and is isolated from the host and from other Jails. With much less overhead than full-blown virtual machines, they're perfect for running a Linux environment efficiently.

FreeBSD's Linux binary compatibility feature is the magic wand here - it emulates a Linux environment within a Jail, allowing you to run many Linux apps seamlessly. The environment I'll be setting up here, based on Ubuntu, provides the necessary libraries and components required to run many Linux-only binaries (not just Spotify, although that's what my focus was on here). Not everything works perfectly, however, and certain binaries or programs may require modifying the basic setup I detail in this article.

First, a quick note: throughout this guide, lines starting with $ relate to our host BSD shell, while /$ denotes commands to be run within the Jail.

First things first, let's enable Linux binary compatibility. Loading the necessary kernel modules is the first step:

• linux: The basic Linux compat layer, necessary for running Linux binaries on FreeBSD
• linux64: Specifically for 64-bit compat
• fdescfs: Used by Linux programs to access file descriptors in a UNIX-like manner
• linprocfs: Emulates the Linux /proc filesystem, providing process and system information in a way Linux binaries will expect
• tmpfs: A temporary filesystem for creation of cache in RAM
• linsysfs: Emulates the Linux /sys filesystem, providing system and device information for Linux binaries, similar to linprocfs

This can be done manually at run-time but it's easy to persist this change across reboots by adding these all to /boot/loader.conf:

Our goal is to keep things (relatively) simple. We'll create a script to manage the Ubuntu Jail, under /usr/local/etc/rc.d/ubuntu. It'll handle loading necessary modules and setting up the Jail's environment.

If you want the benefits of ZFS (snapshots, integrity checks, deduplication, etc), now's a good time to create a ZFS dataset for the Ubuntu directory:

You could also use BTRFS - although it's much less battle-tested, it's what I use on my main Arch install, and I chose to use it here as well rather than ZFS.

For either option, snapshotting should be used as a quick backup before making any potentially-breaking changes:

Before diving into installing Ubuntu, we need to ensure the necessary directories are in place:

Now we enable (and start) the service with sysrc:

Using Debootstrap, we'll download and install the Ubuntu base systems by specifying the target's name, e.g., focal for 'Focal Fossa' (version 20.04 LTS).  ²

The output should look similar to the following:

Post-installation, we need to restart the service:

And replace the following shared lib with a new symbolic link:

There are a few small configuration tweaks we need to make. First, though, chroot into the Jail, which will confine the following commands to it:

Next, we can adjust the timezone to match our host system:

Fix the apt package manager:

And add some initial repositories to sources.list:

To exit the jail, use exit.

And voilà! FreeBSD is good-to-go now with our Ubuntu compatibility-layer jail, with its Ubuntu base system installed.

First, make sure you're chroot-ed into the Jail. apt can then be run as it would on a normal Ubuntu system:

If wanted, the Ubuntu Jail and these compatibility tools can be safely removed, mostly by following the reverse order. First, the automatic startup we set needs to be disabled. FreeBSD then needs to be rebooted, before finally deleting the Ubuntu system directory and/or destroying its volume.

With those updating/future removal tips in mind, we're almost done! The final step is to use Ubuntu's apt package manager as you would in a normal Ubuntu system to install Spotify. You may want to create a script or symlink for easier access without needing to navigate the command-line and chroot into the jail each time. That aside, most programs/binaries should work out-of-the-box. If some libraries are still missing, though, there's a section in FreeBSD's docs on installing additional libs manually.

Great success! 🥳

I hope this writeup helped out! A massive thanks to Micski and their amazing writeup on this same topic back in 2021 - their steps & scripts are what I based this guide on, and helped me get up-and-running with Spotify, Slack, Zoom, and other essentials, allowing me to daily-drive FreeBSD on my development machine.