Blog - Foswiki Blog

Foswiki 2.1.9 is released

2024-12-18T00:00:00Z

Foswiki-2.1.9 is now available for download! We are delighted to announce the new release, which includes 57 significant bug fixes compared to the previous 2.1.8 version. This update addresses a range of important issues and enhances the overall stability and performance.

See the release notes for additional information.

Optimizing Storage Performance

This release focuses on performance optimization, building upon Foswiki's historical struggles with disk I/O bottlenecks. Traditionally, for each request, the engine would access multiple files to compile the necessary objects required to render a wiki page. This was particularly true for preference settings and access control rights, which often necessitated loading complex combinations of objects. To address this issue, extensive profiling has been conducted to identify areas where time can be shaved off. Notably, our analysis reveals that we have a performance window between 300ms and 1 second in which to make further optimizations, allowing us to fine-tune the engine's efficiency.

See Item15206 for more details.

Optimizing renaming/deleting webs

In large wikis renaming/deleting webs takes forever with the web-server's connection to the Foswiki backend timing out. This was caused by the system iterating over all of the wiki for all of the topics contained in the web to be renamed. In most cases this type of rewriting links isn't required or even desirable.

See Item15333 and Item15220 for more details.

Optimizing loading CSS, JavaScript and I18n files

A significant contributor to slow page loading times is the sheer volume of assets being loaded. Each wiki page may require a unique set of CSS and JavaScript files, which can be substantial in number. However, there are common core functionalities that need to be loaded on every page, while additional assets might be required only for specific cases. To address this, Foswiki now allows for the predefinition of this core set of assets, which can then be compiled into a single combined chunk. This approach reduces the number of parallel HTTP requests needed to load a page, resulting in improved performance.

See Item15229 for more details.

Get started with Foswiki today!

Download it immediately from various locations, as detailed on our download page. If you encounter any issues during installation, please submit a report through our task tracker for assistance. For further guidance, visit our System Requirements and Installation Guide pages.

When reporting problems or requesting support, kindly use our task tracker. You can also reach us online via IRC or Slack channels.

Tags: features, performance

Foswiki 2.1.8 is released

2023-08-06T00:00:00Z

We are very pleased to announce the availability of Foswiki 2.1.8. This release contains 61 fixes relative to 2.1.7, including 9 critical security related fixes.

Upgrading to Foswiki 2.1.8 is highly recommended.

Most notable are:

CVE-2023-33756: SpreadSheetPlugin's EVAL feature exposes information about paths and files on the server
CVE-2023-24698: Local file inclusion vulnerability in viewfile

But also:

directories in working directory are created as world writable 777 permissions
possible XSS attack in attachment comments
restricted allowed protocols to http and https, i.e. forbid file protocol for local file inclusion
prevent symlink attacks by defaulting to a secure location for temporary files
update to jquery-ui 1.13.2
backport patch to earlier jQuery versons to fix a potential XSS vulnerability
possible XSS vulnerability in topic title field

For more details read the release notes

You can download it from different locations immediately, see our download page for details. Please use our task tracker to report any issues. Or contact us on online via IRC or Slack.

For installation information, see the System Requirements and the Installation Guide.

Tags: security

Updating Docker-Foswiki to 2.1.7

2022-04-05T00:00:00Z

timlegge/docker-foswiki has been updated to Foswiki 2.1.7. What you need to know to upgrade your instance.

Introduction
Persistent Volumes
Upgrading
- Example Steps assuming your Docker Container can access the Internet
- Example Steps assuming your Docker Container cannot access the Internet

Introduction

With the first release of Foswiki since I started maintaining timlegge/docker-foswiki I wondered how painful the upgrade would be. I had vague visions of Migrating Foswiki 1.1.9 on Ubuntu to Foswiki 2.1.6 on Docker which was certainly an undertaking.

No, it was much simpler than that but the nature of Docker images means its a little different.

Persistent Volumes

In order for a docker instance to host something like Foswiki it needs persistent volumes. That is a place to hold the changing content. In timelegge/docker-foswiki the provided docker compose files esentially put the entire Foswiki website on a persistent volume. The way Foswiki automatically inter-mingles some of the data and code (or more properly automatically generates content sections under data) makes it difficult to do anything else.

So, simply pulling the latest Docker image for timlegge/docker-foswiki means you get all the updated alpine linux packages but Foswiki remains at the same version.

Upgrading

The steps are pretty minor and are essentially the exact came that are documented at here.

Download the upgrade package
Copy it to your Docker Image's persistent volume
Unzip/tar over the existing Foswiki Installation
Save the updated LocalSite.cfg
Restart your Docker Container

Example Steps assuming your Docker Container can access the Internet

docker exec -it docker-foswiki /bin/bash
cd /var/www/foswiki
wget https://github.com/foswiki/distro/releases/download/FoswikiRelease02x01x07/Foswiki-upgrade-2.1.7.tgz 
tar --strip-components=1 -zxf Foswiki-upgrade-2.1.7.tgz 
cd tools
./configure --save
rm Foswiki-upgrade-2.1.7.tgz
exit

Then you simply need to restart your docker image to cache the updated Perl Code

Example Steps assuming your Docker Container cannot access the Internet

wget https://github.com/foswiki/distro/releases/download/FoswikiRelease02x01x07/Foswiki-upgrade-2.1.7.tgz
cp Foswiki-upgrade-2.1.7.tgz /var/lib/docker/volumes/foswiki_foswiki_www/_data
docker exec -it docker-foswiki /bin/bash
cd /var/www/foswiki
tar --strip-components=1 -zxf Foswiki-upgrade-2.1.7.tgz 
cd tools
./configure --save
rm Foswiki-upgrade-2.1.7.tgz
exit

Tags: docker, installation, migration, update

Foswiki 2.1.7 is released

2022-03-28T00:00:00Z

We are very pleased to announce the availability of Foswiki 2.1.7. This release addresses significant security issues we discovered in all previous Foswiki releases.

Upgrade to Foswiki 2.1.7 is highly recommended.

This release comes with a total of 110 fixes and enhancements as well as 7 security fixes. For more details read the release notes

You can download it from different locations immediately, see our download page for details. Please use our task tracker to report any issues. Or contact us on online via IRC or Slack.

For installation information, see the System Requirements and the Installation Guide.

Tags: security

Foswiki, Docker and performance: Examples

2021-02-02T07:11:40Z

I have spent some time setting up an Apache Foswiki docker container to use as the basis for some course material. I wanted a container that started quickly and worked in the Katacoda course environment. I also wanted it to have good response times , which prompted me to try various "optimisations" that are available in Apache and Foswiki. Since it is easy to set up multiple environments in Katacoda/Docker, my curiosity got the better of me and I decided to compare and contrast. It sent me down a few rabbit holes and I thought the results might be of interest to others. So here is the write up.

The Katacoda scenario is available at Katacoda tutorials You can login with your GitHub userid and password. The scenario builds 5 different environments and measures their performance. The environments are:

Environment	Supports
CGI	The out of the box Foswiki install with the standard CGI configuration file. This illustrates the standard performance.
CGI-gz	The CGI environment, with support for the delivery of the pre-zipped `js` and `css` files. This will reduce the volume of data to be transported to render the page
PageOpt	The CGI environment with the `PageOptimizerPlugin` configured. This will reduce the number of files to be requested to render the page. One `js` and one `css` file
CGI-deflate	The CGI environment with the Apache `mod_deflate` enabled. This will reduce the volume of data to be transported to render the page
FCGI	The foswiki install with the Fast CGI configuration file and Apache `mod_fcgid` installed

In the first instance I measured page download times using a simple technique:

/usr/bin/time --quiet -f "%e" wget -pq --header="accept-encoding: gzip" --no-check-certificate --delete-after https://localhost/foswiki

This command will download all the components for the Main.WebHome page and print the time it took for the command to complete. My first results were:

Environment	from the `localhost` (ms)			from my home server (ms)
	first	second	third	first	avg (4)	std dev
CGI	600	570	630	1100	690	105
CGI-gz	640	570	580	700	685	82
PageOpt	630	640	600	700	700	105
CGI-deflate	700	660	650	870	687	58
Fast CGI	660	170	170	690	682	101

In the first instance, these numbers were rather perplexing. All the timings are with 2 standard deviations of one another. No different! Except for the first CGI measurement on my home server (which is also the first measurement against the platform) And the second and third Fast CGI measurement on the localhost. The 1100ms on my home server is likely caused by a DNS lookup that is cached for the subsequent experiments. And the change in the Fast CGI measurement between the first and second sample is caused by the startup of foswiki.fcgi, which is not repeated in the later retrievals.

But all other timings are around 650 +/- 200 ms with 95% confidence. Judging from the Fast CGI measurements on localhost, the time is mostly taken up in the transport of the various elements and the server delivers them faster to the network than the network can deliver them to the client in all configurations.

To look at this further, I tried several tools that measure web page download speed. I used Dynomapper's blog Top 15 tools for measuring website or application speed as a guide an got some interesting insights.

Pingdom gave me the first confirmation that the different configurations were actually working.

Environment	Grade	Size (kB)	Time (s)	Requests (count)	Comment
CGI	C 74	242.9	3.67	36	The baseline configuration
CGI-gz	C 74	125.2	3.39	36	Reduced download size for zipped .css and .js
PageOpt	C 76	239.3	3.83	29	Reduced requests due to `PageOptimizerPlugin`
CGI-deflate	C 74	242.8	3.51	36	No noticeable change. The Katacoda `nginx` proxy unzips the result. (Serverfault). Apache debug shows compression is done. Timing diagram shows that response from the server for jquery-2.2.4.js (85.6kB) takes 220ms ve 82ms in the CGI environment. Compressing on the fly creates overhead
Fast CGI	C 74	241.6	3.76	36	No noticeable change

Looking at the detailed timings shown in Pingdom, you will notice that most of the time is recorded as Wait time. Wait time is made up of a number of components:

Component	Abbreviation
Time to send the request from the browser to the server	`Tsend`
Time to retrieve the requested object from disk	`Tget`
Time to process the requested object for transmission	`Tprocess`
Time to return the requested object from the server to the browser	`Treturn`

Tsend and Treturn are defined by the network properties,I will assume that they are constant across elements for a single measurement. Tget is a server property and will be relatively constant across elements (ignoring disk transfer time, which will depend of element size). Tprocess will be zero for the static elements.

This is probably a naive model, but it gave me the idea that Tprocess should be zero for static elements (js, css, png, etc…). And Pingdom creates a JSON file with all the measurements for down load, the HTTP archive file (HAR). I took measurements in three different environments. The results were revealing.

Environment - Tinkerbell Apache CGI

The graph on the right shows the wait times for the objects retrieved to create the Main.WebHome page. All but one are static elements. And there is one clear outlier: The html whch has a wait time of almost 4 seconds! Tinkerbell is a 20 year old machine, with a very low power consumption that has been running in my basement non stop for all that time. It's not built for high powered processing and it illustrates the point that creating html needs cpu and memory. From the graph you can also see that the static elements wait varying times, but that large elements do not necessarily create a longer wait. The median wait time at Tinkerbell is 246 ms. And all elements are retrieved in approximately that time.

Environment - Katacoda Apache CGI

Next I did the same experiment in my Katacoda Apache CGI environment. It's a docker container in a docker container, so it will have some overheads. But it is likely to have a much better cpu and probably more memory too. The results are in the graph on the left. There is again a single outlier: the html that now has a wait time of 730 ms. So more iron improved the wait time for Main.Webhome four fold. At the same time, you can see that the there is some spread in the wait times of the static elements. To my surprise, the mediam wait time for static elements went up to 323 ms. I won't speculate why this is longer than for Tinkerbell. But it does suggest that the actual processing time for Main.Webhome can be as short and 730-323=407 ms. The rest is starting perl and getting Main/WebHome.txt from disk.

Environment - Katacoda Apache Fast CGI

If starting perl is a substantial overhead in retrieving the Main.WebHome page, then the wait time can be reduced further by configuring Apache and Foswiki with FCGI. The results in my Katacoda Apache FCGI environment are shown in the graph on the right. The median processing time for the static elements in this sample is 408 ms, demonstrating how unreliable my arguments here are. But wait time for Main.WebHome has disappeared in the noise! The actual number is: 229 ms.

So what does all this mean for the performance of Foswiki.

The result that surprised me most is the the wait times in the retrieval of the static elements. It should be noted that many of these elemnts are retrieved in parallel, but only after the the html has been evaluated in the browser. So it seems one should expect at least 200-400 ms delay in the retrieval of a page for the static elements. Caching these elements in the browser will help improve the overall performance. But the first encounter is sensitive to this wait time.
Use FastCGI. It will reduce the wait time for html generation. In the Katacoda Apache FCGI environment it takes 500 ms to start, which are a consistent overhead on each transaction if you use regular CGI.
If you want to improve performance beyond what Fast CGI gives you, use a server with more generous specifications. More cpu and more memory make a difference.
Pre-zipped elements provided with Foswiki may make a difference on slow networks, but they will have no effect on the wait time. And for most elements, the wait time is 99% of the total time to receive the element.
Apache mod_deflate is unlikely to make any improvement in the performance of your Foswiki site. The time taken to compress and decompress is added to the time it takes to retrieve the element from disk. It may help on a slow network, but you should check if it is worth it.
PageOptimizerPlugin. This plugin collects all static css and js in one css and one js file and caches the result. From the server side perspective there are now two files to serve instead of say 10. But since the the retrieval is parallel, you are still stuck with whatever wait time is associated with the two files. At least 200 ms. Sites like Pingdom and many others recommend reducing the number of static elements per page to retrieve, because the browser needs the css and js before it can render the page. So combining the elements together may improve the user experience.

Tags: development, installation, performance