Upgrading Matomo from 3.14.1 to 4.1.1

Matomo - the Open Source Analytics software and a popular alternative to Google Analytics - is becoming increasingly popular in Europe. Unlike the “Don’t be EVIL” counterpart aka GA, Matomo allows for a 100% data ownership and can easily be setup to comply with the GDPR.

Those of you who have been in the game longer will still know it by it’s by the name Piwik. It’s a pretty good, stable and powerful piece of open source software that I’d really recommend for everyone who’s operating a webserver of a certain scale.

Now, as Matomo is gaining popularity, a growing group of developers are adding new features and functionalities new versions and regular upgrades provide increased stability, bugs are fixed etc. Using the on-board web-based updates makes it rather easy (in most cases) to do a quick upgrade conveniently from your browser. But once your installation has grown to a certain size, say you’re hosting a Matomo server for several major high-traffic websites, the story’s slightly different. Mainly because of your database - you can expect the database to grow and be of a tangible size, especially after some years of usage.

https://matomo.org/changelog/

So this other night I was about to upgrade such a major Matomo installation from an outdated version 3.14.1 to the newest version 4.1.1.

How did that go, and what lessons did I learn?

My initial approach when it comes to upgrading critical aka production systems is pretty easy:

1. Run an incremental or full backup
2. RTFM - in this case https://matomo.org/docs/update/

So I started with a basic system check to make sure the current instance was running all well and without any error messages. Now, the important part when it comes to high-traffic, is the following:

the update process could take from a few minutes to a few hours, it is highly recommended that you run the database update process in the command line. You can trigger the Matomo update by calling in your shell

First we’ll start by disabling the Matomo visitor tracking as well as the web GUI aka maintenance mode. You can do that by simply adding:

  [Tracker]
record_statistics = 0 

and

  [General]
maintenance_mode = 1 

to your config/config.ini.php

Read more about that in the Matomo FAQ How do I momentarily disable Matomo User Interface and/or Matomo visitor Tracking? https://matomo.org/faq/how-to/faq_111/

In the next step, you’ll download the latest Matomo release and extract it on your server:

# go into the directory that contains your existing "matomo/" installation 
$ wget https://builds.matomo.org/matomo.zip # download latest version
$ unzip -o matomo.zip # extract it to "matomo/" directory
$ rm matomo.zip # delete the archive and read below the next step

Now the fun part begins… Quoting the Matomo upgrade guide:

For high traffic websites, where the update process could take from a few minutes to a few hours, it is highly recommended that you run the database update process in the command line. You can trigger the Matomo update by calling in your shell

$ ./path/to/matomo/console core:update

Depending on the size of your database, this might take a looooong time. On the server I upgraded, I took almost 90min to simple alter one single table. After 30min or so I started to get slightly nervous - did the process hang? Nah, ps auxf still showed a running process, and mysql was still spinning hard.

After 60min or so it started to feel weird, a quick size check of the table piwik_log_link_visit_action showed a size of 21GB. This Matomo production server is running on some pretty serious hardware with 80GB of RAM, so I did a quick search and found this:

https://forum.matomo.org/t/timeout-while-update-to-4-1-0/40084

Where a guy is reporting the same issue with that very table, and states he had to wait 30min. So assuming that “my installation was a lot bigger than his” - I decided to let the process run even if it’d take several hours to (hopefully) complete.

And really glad I did that - after 93min. the table was altered and the upgrade script happily continued going through the remaining tables.

At the end of the day - literally! - the upgrade finished gracefully.

Lesson learned:

Never halt an upgrade of a larger Matomo instance.

:wq