Migrating Foswiki 1.1.9 on Ubuntu to Foswiki 2.1.6 on Docker

Migrating a Foswiki versions has never been easier

Current Installation

This is from where I started:

1. Ubuntu 14.0
2. Foswiki 1.1.9
3. LDAP Authentication
4. Solr
5. A whole bunch of Plugins from over the years

New Installation

And this is what I should become:

1. RedHat Hosted Docker Container running timlegge/docker-foswiki
2. NGINX Reverse Proxy to several other docker containers including docker-foswiki
3. SAML Authentication
4. Solr Docker Container hosted on the same RedHat Server

The Journey will save you time

The previous posting on Foswiki, Alpine Linux and Docker documents the journey to produce a Docker container for Foswiki that is as small as possible while including all the Perl Modules and Plugins that I needed (and most that you would need).

A seperate Blog entry will document the ins and outs of running Docker Containers with Foswiki but to start up a fully functioning Docker Container on your server install docker and run:

docker run --rm --name docker-foswiki \
      --network foswiki-net -d \
      -v foswiki_www:/var/www/foswiki:z \
      -v nginx_conf:/etc/nginx/conf.d:z  timlegge/docker-foswiki:v1.01

Migration

Foswiki 1.1.9 is old. So old that the migration to 2.1.6 will likely require an intermediate server as Perl changed quite a bit and more recent versions will not properly run the old Foswiki functions required to run the conversion. I set up a Ubuntu 16.04.5 server version to make the transfer easier:

Preparing your Intermediate Foswiki Conversion Server

You need the basic setup to run Foswiki. While the packages below appeared to work fine as a bare minimum to convert from 1.1.9 to 2.1.6 but you would do well to install the required Ubuntu packages. In particular you will need some of the required perl modules to reduce/avoid distracting error messages. In my case I needed the perl ldap libraries as my old Foswiki instalation used LDAP.

I also needed to create an empty data/.htpasswd file as it complained that it was missing.

For Ubuntu 16.04.6 the following was necessary:

apt-get install libcgi-pm-perl libjson-perl libcgi-session-perl \
        libcrypt-passwdmd5-perl libfile-copy-recursive-perl \
        libnet-ldap-perl libdb-file-lock-perl rcs
wget https://github.com/foswiki/distro/releases/download/FoswikiRelease02x01x06/Foswiki-2.1.6.tgz
cd /var/www/
tar xvfz ~/FoswikiRelease02x01x06/Foswiki-2.1.6.tgz
cd /var/www/Foswiki-2.1.6
touch data/.htpasswd

Backup Old Fowswiki version and copy to Conversion Server

Basically create a gzip or bzip2 tar file of your old Foswiki version and copy it to your intermediate conversion server. To make things easier, locate it under the same directory that you used on the old server:

My setup required:

cd /home/http
tar cvfj Foswiki-1.1.9.tar.bz2 Foswiki-1.1.9

Copy it to your conversion server and unpack it:

cd /home/http
tar xvfj ~/Foswiki-1.1.9.tar.bz2

Run the conversion to Foswiki-2.1.6

cd /var/www/Foswiki-2.1.6/tools
perl bulk_copy.pl --xweb System --xweb _default --xweb _empty --latest '*.WebStatistics' /home/http/Foswiki-1.1.9/bin /var/www/Foswiki-2.1.6/bin
cd /var/www
tar cvfj Foswiki-2.1.6.tar.bz2 Foswiki-2.1.6

Then copy all of the Foswiki-2.1.6.tar.bz2 over to your new docker server.

Getting your converted Foswiki into your Docker Container

The following assumes that you have started your docker container with the command above. Basically you need to copy it to your persistant volume for the foswiki container.

Docker has some real advantages. One of them is that you can shutdown your container and make a copy of your persistant volume that contains the Foswiki install. That allows you to then start up the container pointed to the copied volume. You can repeat this process as many times as is necessary to complete your conversion.

Shutdown and copy the container volume

docker stop docker-foswiki
cd /var/lib/docker/volumes
cp -r fowsiki_www foswiki_www-conv-test

Copy Converted Foswiki to test conversion volume

This is necessary to get a copy of the converted data into your container. Putting it in your /var/www/foswiki directory is a convenient way to do that and it is removed afterward.

cp ~/Foswiki-2.1.6.tar.bz2 /var/lib/docker/volumes/foswiki_www-conv-test/_data/

Start your container using the test conversion volume

The volume in this case is the copy. Your original gold configuration with no data is safe.

docker run --rm --name docker-foswiki  --network foswiki-net -d  -v foswiki_www-conv-test:/var/www/foswiki:z  -v nginx_conf:/etc/nginx/conf.d:z  timlegge/docker-foswiki:v1.01

Copy/Convert the Converted version to your Docker version

The timlegge/docker-foswiki uses a real directory (not a symbolic link) to hold the Foswiki install so you can extract the converted version to Foswiki-2.1.6 as it was backed up earlier.

docker exec -it docker-foswiki /bin/sh
cd /var/www
mv /var/www/foswiki/Foswiki-2.1.6.tar.bz2 ./
tar xvfj Foswiki-2.1.6.tar.bz2
cd /var/www/foswiki/tools
perl bulk_copy.pl --xweb System --xweb _default --xweb _empty --latest '*.WebStatistics' /var/www/Foswiki-2.1.6/bin /var/www/foswiki/bin
chown -r nginx:nginx /var/www/foswiki

Test your conversion

Testing is an art. Check that you are able to login, that the Webs look to have copied correctly, that the permissions look correct. Basically convince yourself that the data copied correctly and you are not missing any data.

Finalize your Conversion

Basically this is a repeat of the above but with your production copy of the Foswiki-1.1.9 and Docker volume:

Shutdown and backup the container volume

docker stop docker-foswiki

Copy Converted Foswiki to test conversion volume

cp ~/Foswiki-2.1.6.tar.bz2 /var/lib/docker/volumes/foswiki_www/

Start your container using the test conversion volume

docker run --rm --name docker-foswiki  --network foswiki-net -d  -v foswiki_www:/var/www/foswiki:z  -v nginx_conf:/etc/nginx/conf.d:z  timlegge/docker-foswiki:v1.01

Copy/Convert the Converted version to your Docker version

The timlegge/docker-foswiki uses a real directory (not a symbolic link) to hold the Foswiki install so you can extract the converted version to Foswiki-2.1.6 as it was backed up earlier.

docker exec -it docker-foswiki /bin/sh
cd /var/www
mv /var/www/foswiki/Foswiki-2.1.6.tar.bz2 ./
tar xvfj Foswiki-2.1.6.tar.bz2
cd /var/www/foswiki/tools
perl bulk_copy.pl --xweb System --xweb _default --xweb _empty --latest '*.WebStatistics' /var/www/Foswiki-2.1.6/bin /var/www/foswiki/bin
chown -r nginx:nginx /var/www/foswiki

Test your conversion

Check that you are able to login, that the Webs look to have copied correctly, that the permissions look correct. Basically convince yourself that the data copied correctly and you are not missing any data.