Setting Up A Jenkins Server On NetBSD

I’ve been working on improving Nim’s BSD support for a little while now, first by adding continuous integration running on FreeBSD and then on OpenBSD. The next step was obvious: running CI on NetBSD. Unfortunately, I couldn’t find any CI services currently offering NetBSD as an option, so off I went down the rabbit hole of setting up a Jenkins install myself on NetBSD. In this post, I’ll run through the process of doing so, as it’s not as obvious as it may be on other platforms.

The first step is obvious: get NetBSD installed. I wanted to run it on a VPS rather than dedicating some local hardware to the cause, so my first step was actually finding a host that supported either installing NetBSD from a pre-built image or installing from an ISO. Fortunately, BuyVM KVM slices support custom ISOs and have a NetBSD 9.0 ISO readily available. I installed a clean copy of NetBSD 9.0 using the standard installation procedure.

During the install, I went for a full installation rather than a minimal install. I also took the chance to use some of the final optional steps such as setting up networking, setting up binary package installation, setting up pkgsrc, adding a user, etc. It’s worth doing these at install time just to save some effort at a later date.

Initial set up

There are a few initial set up steps I like to do on any system before we even get to installing Jenkins or anything else. These are mostly for convenience, and are entirely up to you.

The first steps should be ran in a root shell:

  • Install some standard packages that are useful:
    pkgin install sudo nano htop git fish mozilla-rootcerts
    
  • Set up Mozilla’s root cetificate bundle, which is used for SSL certificate verification:
    mozilla-rootcerts install
    
  • Configure sudo to allow members of the wheel group to execute commands:
    visudo
    # find the line `# %wheel ALL=(ALL) ALL` and un-comment it
    

Now that I had sudo set up I dropped back to my standard user’s shell, which was added to the wheel group during install, and ran some further steps:

  • Configure fish with some standard variables and functions:
    mkdir -p $HOME/.config/fish
    echo 'set fish_greeting ""
    
    set -gx EDITOR (which nano)
    set -gx VISUAL $EDITOR
    
    function sudo!
        set -l cmd $history[1]
        echo "sudo $cmd" 1>&2
        eval sudo $cmd
    end' > $HOME/.config/fish/config.fish
    
  • Set my shell to fish:
    chsh -s $(which fish)
    
  • Set up my SSH public key:
    mkdir -p $HOME/.ssh
    chmod 0700 $HOME/.ssh
    echo 'my-ssh-public-key' > $HOME/.ssh/authorized_keys
    chmod 0600 $HOME/.ssh/authorized_keys
    
  • Configure SSH to deny root login, disallow password authentication, disable X11 forwarding and a few other tweaks:
    sudo sed -i 's/^LoginGraceTime .*/LoginGraceTime 60/g' /etc/ssh/sshd_config
    sudo sed -i 's/^#PermitRootLogin prohibit-password/PermitRootLogin no/g' /etc/ssh/sshd_config
    sudo sed -i 's/^#MaxAuthTries 6/MaxAuthTries 3/g' /etc/ssh/sshd_config
    sudo sed -i 's/^#PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config
    sudo sed -i 's/^#ChallengeResponseAuthentication yes/ChallengeResponseAuthentication no/g' /etc/ssh/sshd_config
    sudo sed -i 's/^#X11Forwarding no/X11Forwarding no/g' /etc/ssh/sshd_config
    sudo sed -i 's/^#UseDNS no/UseDNS no/g' /etc/ssh/sshd_config
    sudo service sshd restart
    
  • Configure some basic firewall rules for NPF to block all incoming traffic except SSH, HTTP and HTTPS. Note that the network interface here may be different:
    sudo touch /etc/npf_blacklist
    echo '# NOTE: change the `wm0` to the name of your external network interface
    $public_if = ifaddrs(wm0)
    
    # create 2 tables - one for blacklisted IPs, the other for suspicious traffic
    table <blacklist> type ipset file "/etc/npf_blacklist"
    table <suspicious> type lpm
    
    # create a variable for the TCP services we wish to allow
    $tcp_services = { http, https }
    
    alg "icmp"
    
    procedure "log" {
        log: npflog0
    }
    
    procedure "normalize" {
        normalize: "random-id", "min-ttl" 64, "max-mss" 1432, "no-df"
    }
    
    group default {
        # pass everything on loopback
        pass final on lo0 all
    
        # block blacklisted IPs
        block in final from <blacklist>
    
        # block suspicious IPs
        block in final from <suspicious>
    
        # allow all outgoing
        pass stateful out final all
    
        # allow ICMP
        pass in final proto icmp all
    
        # allow and log ssh
        pass stateful in final proto tcp from any to $public_if port ssh apply "log"
    
        # allow incoming TCP
        pass stateful in final proto tcp to $public_if port $tcp_services apply "normalize"
    
        # reject everything else
        block return in final all apply "log"
    }' | sudo tee /etc/npf.conf
    echo 'npf=YES' | sudo tee -a /etc/rc.conf
    sudo /etc/rc.d/npf reload
    sudo /etc/rc.d/npf start
    

Install and configure Nginx

We’re going to use Nginx as a reverse proxy to Jenkins, so let’s install that. Luckily, there’s a relatively up to date binary package available:

sudo pkgin install nginx

Now let’s configure Nginx to start at boot:

# copy the RC script into place
sudo cp /usr/pkg/share/examples/rc.d/nginx /etc/rc.d/nginx
echo 'nginx=YES' | sudo tee -a /etc/rc.conf
echo '/var/log/nginx/access.log nginx:nginx 640 7 * 24 Z /var/run/nginx.pid SIGUSR1
/var/log/nginx/error.log nginx:nginx 640 7 * 24 Z /var/run/nginx.pid SIGUSR1' | sudo tee -a /etc/newsyslog.conf

Then let’s do some Nginx configuration to point to where Jenkins will eventually be running:

# take a backup of the existing configuration
sudo tar -czvf /usr/pkg/etc/nginx.tar.gz /usr/pkg/etc/nginx
sudo mkdir -p /usr/pkg/etc/nginx/conf.d
echo 'user nginx nginx;

worker_processes auto;

worker_rlimit_nofile 8192;

events {
    worker_connections 8000;
}

error_log /var/log/nginx/error.log warn;

pid /var/run/nginx.pid;

http {
    server_tokens off;

    include mime.types;

    default_type application/octet-stream;

    charset utf-8;

    charset_types
        text/css
        text/plain
        text/vnd.wap.wml
        text/javascript
        text/markdown
        text/calendar
        text/x-component
        text/vcard
        text/cache-manifest
        text/vtt
        application/json
        application/manifest+json;

    log_format main '\''$remote_addr - $remote_user [$time_local] "$request" '\''
                    '\''$status $body_bytes_sent "$http_referer" '\''
                    '\''"$http_user_agent" "$http_x_forwarded_for"'\'';

    access_log /var/log/nginx/access.log main;

    keepalive_timeout 20s;

    sendfile on;

    tcp_nopush on;

    gzip on;

    gzip_comp_level 5;

    gzip_min_length 256;

    gzip_proxied any;

    gzip_vary on;

    gzip_types
        application/atom+xml
        application/geo+json
        application/javascript
        application/x-javascript
        application/json
        application/ld+json
        application/manifest+json
        application/rdf+xml
        application/rss+xml
        application/vnd.ms-fontobject
        application/wasm
        application/x-web-app-manifest+json
        application/xhtml+xml
        application/xml
        font/eot
        font/otf
        font/ttf
        image/bmp
        image/svg+xml
        text/cache-manifest
        text/calendar
        text/css
        text/javascript
        text/markdown
        text/plain
        text/xml
        text/vcard
        text/vnd.rim.location.xloc
        text/vtt
        text/x-component
        text/x-cross-domain-policy;

    include conf.d/*.conf;
}' | sudo tee /usr/pkg/etc/nginx/nginx.conf
echo 'server {
    listen [::]:80 default_server;
    listen 80 default_server;

    server_name _;

    return 444;
}' | sudo tee /usr/pkg/etc/nginx/conf.d/no-default.conf
echo 'server {
    listen [::]:80;
    listen 80;

    # change this to your actual domain name
    server_name jenkins.euantorano.co.uk;

    location / {
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_pass http://127.0.0.1:8080;
        proxy_read_timeout 90s;

        # change this to your actual domain name
        proxy_redirect http://127.0.0.1:8080 http://jenkins.euantorano.co.uk;

        proxy_http_version 1.1;
        proxy_request_buffering off;

    }
}' | sudo tee /usr/pkg/etc/nginx/conf.d/jenkins.conf

And now let’s start nginx:

sudo /etc/rc.d/nginx start

Install Java

Jenkins uses Java, which we must install. We’ll use the binary package for OpenJDK 8, as OpenJDK 11 on NetBSD currently (at the time of writing) has a bug related to network connections.

sudo pkgin install openjdk8

Install Jenkins

Jenkins is available as a package on NetBSD, but unfortunately the version available is well out of date as of the time of writing (the package version 2.73, while the current version is version 2.259). As such, we’ll download the current version from the Jenkins website and use that instead:

sudo mkdir -p /usr/lib/jenkins
sudo mkdir -p /var/lib/jenkins
sudo mkdir -p /var/log/jenkins
sudo chown jenkin:jenkins /var/log/jenkins
sudo ftp -o /usr/lib/jenkins/jenkins.war https://get.jenkins.io/war/latest/jenkins.war
# create a group for jenkins to run under
sudo groupadd jenkins
# create a user for jenkins to run under
sudo useradd -s /sbin/nologin -g jenkins -m -d /home/jenkins jenkins
# Create an RC script for Jenkins
echo '#!/bin/sh

# PROVIDE: jenkins
# REQUIRE: DAEMON

. /etc/rc.subr

JENKINS_USER=jenkins
JENKINS_GROUP=jenkins

JENKINS_HOME=/home/jenkins
export JENKINS_HOME

HOME=$JENKINS_HOME

name="jenkins"
rcvar=$name
pidfile="/var/run/jenkins.pid"
start_cmd="jenkins_start"
stop_cmd="jenkins_stop"
restart_cmd="jenkins_restart"
status_cmd="jenkins_status"
version_cmd="jenkins_version"
extra_commands="status version"

jenkins_start()
{
    su -m jenkins:jenkins -c '\''nohup \
        /usr/pkg/java/openjdk8/bin/java -jar /usr/lib/jenkins/jenkins.war \
            --httpPort=8080 --httpListenAddress=127.0.0.1 \
        </dev/null >> /var/log/jenkins/jenkins.log 2>&1 &
        echo "$!"'\'' > $pidfile
    pid=$(cat "$pidfile")
    echo "Started Jenkins: $pid"
}

jenkins_stop()
{
    if [ -f "$pidfile" ]; then
        pid=$(cat "$pidfile")
        echo "Stopping Jenkins: $pid"
        kill -15 "$pid"
        rm "$pidfile"
    fi
}

jenkins_restart()
{
    jenkins_stop && jenkins_start
}

jenkins_status()
{
    if [ ! -f "$pidfile" ]; then
        echo "Jenkins is not running"
        exit 1
    fi

    pid=$(cat "$pidfile")
    kill -0 "$pid"
    if [ $? -eq 0 ]; then
        echo "Jenkins is running (PID $pid)"
        exit 0
    else
        echo "Jenkins is not running"
        rm "$pidfile"
        exit 1
    fi
}

jenkins_version()
{
    /usr/pkg/java/openjdk8/bin/java -jar /usr/lib/jenkins/jenkins.war --version
}


load_rc_config "$name"
run_rc_command "$1"' | sudo tee /etc/rc.d/jenkins
sudo chmod +x /etc/rc.d/jenkins
echo '/var/log/jenkins/jenkins.log jenkins:jenkins 640 7 * 24 Z /var/run/jenkins.pid SIGUSR1' | sudo tee -a /etc/newsyslog.conf

And now let’s start Jenkins:

sudo /etc/rc.d/jenkins start

Finishing up

Jenkins is now running behind Nginx as a reverse proxy, but there are still a few tasks to do, including running through the Jenkins install web UI and setitng up build jobs. It’s probably a good idea to set up HTTPS too. I’m afraid these are left to the reader for the time being!