Cumulus MX Version 4 Beta - Additional Information

mcrossley · Post by **mcrossley** » Sun 17 Mar 2024 12:36 pm

A new major version of Cumulus is coming. This is an open beta for anybody to try, and together we can work out any issues and bugs with it before it takes over from version 3.0

The main change for this version of CMX is to move away from .Net Framework 4.8 to .NET 8: The latest Long Term Stable release from Microsoft.

By moving to .NET 8 we are back on a fully supported Microsoft release, and as Microsoft make version of .NET 8 available for Linux and MacOS it means we can also move away from Mono on these platforms and the issues that has been causing. Note there is no support for Cumulus MX on MacOS Catalina or later as Apple have enforced signing and notarization for this version of the OS.

That does mean that if you run Cumulus on a Linux platform there is a little preparation work installing .NET 8 to do before you can run CMX v4.

On Windows you may also need to install .NET8, though it may already be present.

There are some changes to the data storage (see below), logging, backup. Functionally though, version 4 operates and looks much the same as version 3.

Changes to the data storage of Cumulus MX

In Cumulus MX historically all data was written in ASCII in readable files. The logfiles were named with the short file names according to the locale (the language setting) the user had chosen, either in the setup of the account under which Cumulus MX runs or on the command line when starting Cumulus MX. The locale also determined the format of the separators in the logfiles (datetime separators, decimal separators and column separators were all locale dependent).

This increasingly led to issues and while creating this version 4 it was found that the numeric minus sign under certain locales was written to the as a non-standard minus (the minus symbol rather than a hypen). Storing those numbers - e.g. a negative temperature - in log files meant that Cumulus could not read them back. These "odd" negative numbers would eventually propagate to all systems reading the datafiles and up into the JavaScript datafiles and so on. This is more than undesirable as it would lead inevitably to problems. Therefore, it was decided to remove the locale dependency from the data storage.

Installing .NET 8 on Windows

First check if you already have .NET 8 installed. Start a command or PowerShell session, then enter the command:

Code: Select all

dotnet --info

If the command is not recognised, or it shows a version below 8.0.101, then you need to install .NET 8.

If you need to install .NET 8, start an Administrator command or PowerShell session, then enter the command:

Code: Select all

winget install Microsoft.DotNet.Runtime.8

(if winget is not installed, or out of date, please see this page for details on installing/updating it)

Installing .NET 8 on Linux

There is only a packaged version of .NET 8 for x64 distributions, all other versions - x86, Arm32, Arm64 etc - all require a manual or scripted install.
These instructions cover the typical arm/arm64 required for Raspberry Pi Debian based installations. For other distribution please see this page.

NOTE: .NET 8 is officially supported on the following distributions (though it may work on others): Alpine, CentOS, Debian, Fedora, openSUSE, SLES, Ubuntu.

First remove any existing .NET install:
Code: Select all
```
sudo rm -r /usr/share/dotnet
```

Next install the .NET dependencies (Check this page for OS other than Debian 11/12)

Debian 12 bookworm

Code: Select all

sudo apt-get update
sudo apt install libc6 libgcc-s1 libgssapi-krb5-2 libicu72 libssl1.1 libstdc++6 zlib1g

Debian 11 bullseye

Code: Select all

sudo apt-get update
sudo apt install libc6 libgcc-s1 libgssapi-krb5-2 libicu67 libssl1.1 libstdc++6 zlib1g

Now install .NET, this may be done from a bash script. Do NOT use -c Current as that has been deprecated

Code: Select all

wget https://dot.net/v1/dotnet-install.sh
chmod +x dotnet-install.sh
sudo ./dotnet-install.sh -c LTS --install-dir /usr/share/dotnet --runtime dotnet

The following is best done manually

Code: Select all

sudo echo 'export PATH=$PATH:/usr/share/dotnet' | sudo tee /etc/profile.d/dotnet.sh > /dev/null
sudo chmod +x /etc/profile.d/dotnet.sh
sudo ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet

Test the run time
Code: Select all
```
dotnet --info
```

If everything is satisfactory you can conclude dotnet version 8 has been installed correctly.

Converting Data from Cumulus MX v3 to v4

A conversion program has been made which must be run before you can use version 4.

All data files are now written/read as invariant (dot decimal, comma field separator, dd/MM/yy HH:mm), this applies to: day file, monthly log files, extra log files, custom log files.

Monthly log files now renamed to "[yyyyMM]log.txt" to remove localised month name - and are now sortable in the file system!

In the /CumulusMX folder you will find the migration utility: MigrateData3to4

A basic workflow to run the utility:

Start with a clean install v4
Copy your v3 Cumulus.ini and strings.ini files to root of v4
Copy your v3 /data and /Reports folders to the v4 install
Rename the copied /data folder to /datav3
From the command prompt, change the path to your v4 /CumulusMX folder
Run MigrateData3to4
Note: If you have any Custom Daily log files, you must supply these file names to the migration utility as parameters as they can have any free-format file name.
- Windows:
  Code: Select all
```
MigrateData3to4.exe [<CustomDailyLog1> <CustomeDailyLog2> ...]
```
- Linux:
  Code: Select all
```
dotnet MigrateData3to4.dll  [<CustomDailyLog1> <CustomeDailyLog2> ...]
```
Follow the prompts and look for any errors - the utility also outputs a log file to the /MXdiags folder
If you use PHP upload, check the upload.php script on your web server is up to date with the v4 zip file versions
Done!

If you do not manipulate or read the data files directly, there is nothing more you need to do. You can now startup Cumulus MX, you may need to update some scripts of yours to remove all references to mono

Installing as a Service on Linux

Cumulus MX can run as a service on Linux, it now has the ability create the service config file for you.

Caution: If you already have v3 running as a service the self-installer for v4 will overwrite your v3 service config file! Only use the self-installer if v4 is the only copy of MX running on the box, or you are prepared to manually adjust the config files.

To run the self-installer, run Cumulus MX v4 with elevated privileges and use the -install option...

Code: Select all

sudo dotnet CumulusMX.dll -install -user <username> [-group <groupname>] [-port port_num] [-lang <lang-code>]

Where:
<username> = the user you want MX to run under
<groupname> = the user group you want MX to use, if omitted it defaults to the username
<port_num> = the optional port number to use. The default is the normal 8998
<lang-code> - the optional locale to run MX under. The default is the system locale. Use a code like en-GB, nl-NL etc.

To make the service start when you server boots, use the command:

Code: Select all

systemctl enable cumulusmx

What else is New/Changed

There are lots of minor changes, but changes of note:

Credentials in cumulus.ini are now encrypted. The encryption is unique to your installation.
- You will find a file called "UniqueId.txt" in your CMX root folder, this contains an identifier unique to your installation.
- If you copy your v4 cumulus.ini to another installation, from now on you must also copy this file otherwise the decryption of the credentials on the target installation will fail.
The backups CMX makes at start-up and daily are now zipped
FTP logging has been rewritten making it easier to differentiate between messages from MX, real-time FTP, and interval FTP
There is a new folder /runtimes in the Cumulus distribution, this must be copied from the zip to your installation every time you update

As usual, full details of the changes are contained in the Updates.txt file.

Third Party Utilities

The change in the monthly log file names and in some locales, contents of both monthly files and the dayfile.txt, may affect third party utilities that read these files. Often these utilities have settings around date and decimal formats that may need to be changed. Others may need updates to cope with the changes.

Utilites that spring to mind include:

WxSim - the author is already aware and working on an update to accommodate v4

Reverting to Version 3

As there is no automated mechanism to convert v4 log files to v3, rolling back to version 3 will mean reverting to last time v3 ran and catching up. For this reason, I strongly advise that you run v4 in parallel with v3 for the duration of the beta test.

You can happily have Mono and .NET 8 installed together on the same machine, so if you have a station that only accepts a single connection another possibility is that you just run v4 periodically with v3 shut down for a while.

You can download the current beta version of Cumulus MX version 4 from here: https://github.com/cumulusmx/CumulusMX/ ... ta4017.zip