Foswiki, Alpine Linux and Docker

Look mom no CPAN

Simplifying a Foswiki installation is a large ask. Multiple operating systems, extensions, Perl dependancies, libraries, web servers etc. How can we make it repeatable and easy to do. First step find a Linux distribution that provides ALL the required perl modules.

Recently I began looking into upgrading a legacy Foswiki installation. It began like any other, deploy a new Linux OS, begin installing the requirements and extensions and regret deploying it on RedHat Enterprise Linux.

After years using Foswiki on a Ubuntu, the tested robust and quality controlled RHEL was a reminder that some distributions are ill-suited to a fast changing free software project, like Foswiki, that has a multitude of dependencies. Perl modules that were available on Ubuntu as packages simply were not available if you wanted to avoid CPAN installs or Centos packages.

As I contemplated going back to Ubuntu a friend mentioned Docker. It wasn't something I knew much about but reminders of dependency hell meant it was time to look for alternatives. I got the link for a Docker container for Foswiki and never looked back.

Docker

Essentially Docker provides a way to create small packages (Docker containers) that include everything the application needs to run. Better still it includes a “recipe”, the Dockerfile, to recreate the container exactly the same way each time. In traditional installs you need to remember what packages, libraries, etc. that you have installed and keep your documentation up to date. With Docker, the Dockerfile is that documentation. Simply add what you need to the Dockerfile and rebuild the image.

The Docker container runs similar to a virtual machine but with less overhead. The image is small and in the traditions of Unix/Linux dedicated to a single purpose. Don't install more than you need for the application you need. Use multiple Docker containers built for a specific purpose to deliver the full functionality you need.

Alpine Linux

Alpine Linux is a small, lightweight Linux distribution that is well suited to creating Docker containers. In addition, it has a lot of available packages and a fairly simple process to create and add additional packages. Early on in my work to get a Foswiki container that included everything I wanted, I considered going back to Ubuntu as the base Docker container OS. I am glad I didn't. The comfort of Ubuntu aside, Alpine Linux allowed me a lot of flexibility and I was able to gradually work through the process of creating and adding missing dependencies.

Docker Containers, Foswiki and Alpine Linux aports

As mentioned, a Dockerfile is a recipe to build a container that is configured exactly as you need it. You start with a base Linux distro (alpine) and layer on the things you need. Need certain Alpine Linux packages modify the Dockerfile to add them. Want Foswiki installed, have the Docker file download, install it set the permissions. Want specific Foswiki extensions installed, have the Dockerfile use the Foswiki version it just installed to download and install the extension.

The docker-foswiki github repo was my starting point. It is pretty simple, install of the bare number of required packages to start up a nginx based Foswiki install. As it is, it is pretty amazing, run:

docker run -idt -p 80:80 michael34435/docker-foswiki

and a working Foswiki installation is running on port 80. Unlike traditional installs, Docker has some “strange” features. Changes are not persistent. That is, if you stop and restart a Docker container you lose any changes you made while it was running. Obviously, there are methods to create persistent storage for your data but to get the true value of Docker, the container should be interchangeable. Need patches, stop the old container and start a patched version.

My initial approach was to add the items I needed however I could. Need a perl module that has a Alpine Linux package get the Dockerfile to install it. Need a CPAN based perl module, add the available dependencies from Alpine Linux and install the module and dependencies via CPAN using the Dockerfile. Need to compile a module against openssl, install openssl-dev, gcc, make, etc. using the Dockerfile.

However, Docker container images get larger as you add different components in layers. What starts out small and lean quickly becomes a large container with that approach. There had to be a better way. Why install the development tools? How could I get the required Perl modules that were not pre-packaged on Alpine Linux without the hassle and extra size that extra steps in the Dockerfile causes. Figure out how to create the Alpine Linux packages and get them accepted in the Alpine Linux aports project of course.

Alpine Linux Aports Project

The Alpine Linux aports project provides the tools to create Alpine Linux packages. The process is documented here. The majority (all) of the missing packages to fully support Foswiki with native packages were Perl modules.

apkbuild-cpan

This is a tool delivered in the aports toolchain to ease the creation of packages for Perl modules. When I started looking though, that tool had not been updated for the changes required for the retirement of search.cpan.org and the transition from cpan.org to metacpan.org. The required changes were made and it makes the process of creating a package for a perl module very simple.

apkbuild-cpan create DB_File::Lock for example will reach out to metacpan, download the module, determine the dependencies, obtain the information and create a file db_file-lock/APKBUILD

APKBUILD

This is a file that tells the abuild utility how to download the Perl module, what the build dependencies and runtime dependencies are, defines variables for the description, license, etc. It also has the information needed to apply patches, functions to customize the package preparation, etc.

While most Perl modules are fairly simple to build, the APKBUILD file generally needs a few corrections. In most cases the version numbers of the dependencies need to be removed as Alpine Linux will install the latest version available for that version.

abuild -r

The abuild script with the -r flag will attempt to use the APKBUILD file in the current directory to build a package after installing the required dependencies (due to the -r).

Patching Perl Modules via abuild

Over the years some CPAN modules have lost their maintainers and issues in the modules caused by newer operating system versions, updated libraries, etc. have not been addressed. However, in most cases some helpful person has posted a patch to the issues log for the Perl module. abuild can use that patch located in the same directory as the APKBUILD module and apply it to the downloaded extracted source before the build begins.

abuild checksum

If you add patch files to the local directory of the APKBUILD file you need to update the APKBUILD file with the filename.patch as in:

source="https://cpan.metacpan.org/authors/id/D/DH/DHARRIS/DB_File-Lock-0.05.tar.gz \
         fix-bug-98224-test-fails.patch"

abuild checksum updates the APKBUILD file with the checksum for the referenced patch file. This allows people to trust that the patch is the one that was provided by the package maintainer.

Submitting the package to the aports project

The easiest way to do this is to use Alpine Linux VM to clone the aports github repo and install the package tools as documented here. Then change the directory to the aports repo and the testing directory. My basic process is:

1. cd ~/aports/testing
2. git checkout -b perl-module-name
3. apkbuild-cpan create Module::NAME
4. cd perl-module-name
5. vim APKBUILD (as required, remove depends version numbers, etc)
6. abuild -r
7. Troubleshoot the build as required and update the APKBUILD
8. git add APKBUILD
9. git commit -m “testing/perl-module-name: new aport”
10. git push --set-upstream origin perl-module-name
11. Submit a Pull Request to the aports repository
12. Wait patiently for it to do automated builds
13. Fix issues seen on the build servers (missing dependencies, bugs that need patching)
14. git add APKBUILD
15. git commit -m “testing/perl-module-name: new aport”
16. git rebase --interactive HEAD~2 (to squash two commits to one)
17. git push --force (you lose history squashing commits so your github repo is out of date)
18. Wait patiently for it to do automated builds
19. Wait patiently for a project member to review the build

Things I have learned in the process

1. The format of the commit message: “testing/perl-module-name: new aport” is not a suggestion.
2. One package one commit is not a suggestion (squash your commits)
3. A package that builds on your local computer may not build on the build server (dependencies missing)
4. License names need to be in spdx format
5. Large patches are frowned upon but may be accepted
6. Empty Variables in the APKBUILD file must be removed
7. An update to the package without a version increment requires pkgrel to be incremented
8. Perl Module dependencies are painful when building packages
9. Apparently packages with dependencies can be sent in the same Pull Request as long as each package in the PR has a single commit (did not find out till I was done)
10. The Alpine Linux aports folks are patient, accepting of new submissions and helpful but busy so return the favour and be patient and courteous.