I think one of the larger problems I've had when writing software has been doing proper testing CI on OpenBSD, which for me, obviously is a problem. Much of the time, I end up either just doing CI on Linux or not setting up CI at all.

There are a few popular CI options available:

• Travis CI (supports linux and macos)
• Appveyor (supports Windows, linux and macos)
• GitHub Actions (supports Windows, linux and macos)
• GitLab (officially supports Windows, linux, macos and FreeBSD)

Finally, I should mention that there is one platform that does absolutely solve my problem, definitely better than I will in this blog post. That's Sourcehut! They officially have machines for a variety of Linux platforms, FreeBSD, OpenBSD and 9front, and on a few different architectures as well!

I'm on the fence about sr.ht at the moment, and in general I'm pretty fond of GitLab, so I wanted to tackle the issue of having some degree of OpenBSD testing through GitLab.

NB: It's worth noting that the setup that I'm going to describe isn't particularly secure, and doesn't provide build isolation. CI is quite literally remote code execution as a service, so the setup I'm going to describe is not suitable for colocation with anything else.

Getting an OpenBSD VM

This isn't particularly related to the guide, but if you're looking for a hosting platform that supports OpenBSD, I've enjoyed Vultr quite a bit. I've had fairly good experiences with their support and had p good uptime with my VMs. They also offer cheap $2.50/mo, IPv6-only VMs (yes, GitLab and the GitLab runner have full IPv6 support, so this is an option).

I should also mention OpenBSD Amsterdam, which is run by OpenBSD fans and makes monthly donations to the OpenBSD project with the proceeds.

Building the runner

Building is simple. We're gonna deviate from the official instructions a bit, because we don't need the helper images or anything.

$ doas pkg_add bash gmake
$ git clone https://gitlab.com/gitlab-org/gitlab-runner
$ cd gitlab-runner
$ BUILD_PLATFORMS='-osarch openbsd/$(arch -s) gmake runner-bin

The runner binary will be in out/binaries/gitlab-runner-openbsd-$(arch -s)

Machine setup

I'm assuming you're starting with a completely fresh install.

First, install the following packages:

$ doas pkg_add bash git git-lfs curl jq

Despite GitLab's docs saying they support sh, they only support an sh that supports long options (--login), so we have to use bash here.

We're going to run the runner (and our builds) as a separate user, not as root. There's no great way to do proper isolation of arbitrary binaries on OpenBSD outside of vmd, which I don't believe supports nested virtualization. We could run this in a chroot, but unfortunately it'd be a pretty loaded chroot, since the deps for all of our builds would also live in the chroot.

$ doas useradd -s /sbin/nologin -m gitlab

If you have other users, I'd recommend making their home directories unreadable to others.

$ doas chmod -R o-rx /home/*

Resources

To reduce the impact of abuse, we're gonna try and restrict the usable resources a bit. We're going to modify /etc/login.conf.

The following values are specific for my use-case, and won't necessarily be applicable for your system.

gitlab:\
	:coredumpsize=0:\  # prevent coredumps
	:filesize=30M:\    # largest file can be 30M
	:maxproc=50:\      # max 50 processes
	:datasize=1G:\     # 1G heap per proc
	:stacksize=4M:\    # 4M stack per proc
	:openfiles=512:\   # 512 open files per process
	:tc=default:       # inherit other values from `default` class

Since I'm on Vultr, and they have a weird partition scheme where /home isn't its own partition, I'm going to set up filesystem quotas for gitlab.

# edit `fstab` to add the `quota` option to `/`
$ doas mount -o update,quota /
$ doas edquota
$ doas quotaon
$ doas -u gitlab quota

Network

We're also going to restrict network access for the gitlab user. We'll do this through adding the following lines to our /etc/pf.conf

table <github> file "/etc/pf/github"
table <allowed_hosts> const { gitlab.com crates.io static.crates.io }

block out log proto {tcp udp} user gitlab                     # by default block outgoing traffic and log to pflog0
block in proto {tcp udp} user gitlab                          # block any incoming traffic
pass out proto {tcp udp} to 127.0.0.1 port 53 user gitlab     # pass dns traffic
pass out proto tcp to <allowed_hosts> port 443 user gitlab
pass out proto tcp to <github> port 443 user gitlab

I'm primarily writing rust, so I just need the three hosts in the <allowed_hosts> table, as well as whatever IPs github is using that day. Part of the reason I log blocked outgoing calls, is that if a build fails due to a network issue, it makes it a lot easier to figure out what new rules you may have to add to your pf.conf to get builds flowing again.

To build the github table, I run the following script in an hourly cronjob:

#!/bin/sh

/usr/local/bin/curl -sSH "Accept: application/vnd.github.v3+json" https://api.github.com/meta |
	/usr/local/bin/jq --raw-output '.api+.git+.web|sort|join("\n")' > /etc/pf/github
/sbin/pfctl -T replace -t github -f /etc/pf/github

Finally, I also add the following to /etc/ssh/sshd_config to prevent this user from logging in via ssh.

DenyUsers gitlab

Installing the runner

To install, copy the binary into /usr/local/bin. I'd recommend keeping it out of the home directory, so it's harder for a rogue job to overwrite.

I drop the following config file in /etc/gitlab/config.toml

concurrent = 1
check_interval = 3
log_level = "info"

[session_server]
  session_timeout = 1800

[[runners]]
  name = "openbsd-6.8"
  url = "https://gitlab.com/"
  token = "<secret>"
  executor = "shell"
  shell = "bash"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

I also have an rc script to start it

#!/bin/ksh

daemon="/usr/local/bin/gitlab-runner-openbsd-amd64"
daemon_class="gitlab"
daemon_user="gitlab"
daemon_flags="run -c /etc/gitlab/config.toml"
daemon_timeout=60

. /etc/rc.d/rc.subr

pexp="${daemon} ${daemon_flags}"

rc_bg="YES"
rc_reload=NO

rc_pre() {
	# There are some interesting network timeouts
	# if I don't refresh the pf rules
	/sbin/pfctl -f /etc/pf.conf
}

rc_start() {
	${rcexec} "${daemon} ${daemon_flags}" >> /var/log/gitlab-runner.log 2>&1 &
}

rc_cmd $1

Following all of this, you can just follow the normal gitlab instructions.

Conclusion

Does this work?

Well, yeah.

The big problem is that I wouldn't necessarily call this secure by any means. At the end of the day, users submitting PRs to my project that use this runner are still able to run arbitrary code on my gitlab runner. All CI/CD is remote code execution as a service. Even with the steps taken above, there are still ripe opportunities for abuse.

I definitely don't feel comfortable integrating this machine with my personal infrastructure. I'm certainly not setting it up to forward mail to my mail servers, since that'd require a key on the box. For now, this is roughly the best I think I'll do.

What would be better?

Ephemeral VMs (also with similar settings) would be the ideal here. If vmd supported nested VMs, it'd be awesome to add vmd support to the gitlab runner and launch all of my jobs in individual VMs. Alas, as far as I'm aware, this would only be possible if I had a bare metal machine handy. VirtualBox on a non-OpenBSD platform could also be a fair option worth exploring.

For right now, this works for my case, is cheap enough, and this VM is essentially ephemeral to me and easily trashed if I need to replace it for any reason.

Feel free to reach out if you think that there are ways to improve on this setup, or if I got something super wrong.