Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

First startup after upgrading to b3141 - error message noticed

From build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

Post Reply
freddie
Posts: 2434
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

First startup after upgrading to b3141 - error message noticed

Post by freddie »

Just upgraded to b3141 and started up with no problems - all seems to be working just fine. I did scan quickly through the MXDiags file after the restart and saw the following:

Code: Select all

2021-07-31 14:52:26.545 Reading history data from log files
2021-07-31 14:52:26.546 Loading last N hour data from data logs: 31/07/2021 14:44:00
2021-07-31 14:52:26.552 LoadRecent: Attempting to load 7 days of entries to recent data list
2021-07-31 14:52:26.553 LoadRecent: Error querying database for latest record - Object reference not set to an instance of an object  <-- ** THIS LINE **
2021-07-31 14:52:29.075 LoadRecent: Loaded 10081 new entries to recent database
2021-07-31 14:52:29.079 LoadLast3Hour: Attempting to load 3 hour data list
2021-07-31 14:52:29.101 LoadLast3Hour: Loaded 181 entries to last 3 hour data list
I'm sure it's nothing to worry about, but wanted to bring it to your attention.
Freddie
Image
User avatar
mcrossley
Posts: 12694
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: First startup after upgrading to b3141 - error message noticed

Post by mcrossley »

Nothing to worry about, 3.12 now uses a new temporary database to store recent data. On the first run this does not exist and gets created.
freddie
Posts: 2434
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: First startup after upgrading to b3141 - error message noticed

Post by freddie »

Fab, thanks for the detail. I was sure it was nothing to worry about as everything was running sweet as a nut :)
Freddie
Image
sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

Re: First startup after upgrading to b3141 - error message noticed

Post by sfws »

A technical question related to your persistent database mentioned in your Updates.txt, that gave the message that Freddie spotted:

A SQLite3 database uses the row number as the primary key in any database table,

When the "clocks go back", there will be an hour of duplicated minutes, will MX (from 3.12.0) now be able to retain all the entries because in some way it makes use of that primary key, or (as I suspect might be the way you have coded it) does MX use the datetime column as its key and therefore overwrite the entries for the past hour?

When the "clocks go forward", obviously we lose an hour of values for the clock times that do not exist, so 3.12.0 does not change the need to be careful with selection of input modification parameters when seeking values on the day involved.
freddie
Posts: 2434
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: First startup after upgrading to b3141 - error message noticed

Post by freddie »

If it was my design, then internally I would use a fixed time zone with no DST and convert on-the-fly when making the data available externally. Something I've been doing for years. When storing and manipulating timeline-based data such as meteorological readings then artificial adjustments such as DST have no place in the storage layer.
Freddie
Image
sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

Re: First startup after upgrading to b3141 - error message noticed

Post by sfws »

freddie wrote: Sun 01 Aug 2021 6:45 am If it was my design, then internally I would use a fixed time zone with no DST and convert on-the-fly when making the data available externally.
That has been suggested many times in this support forum over the years.

Of course, in response, Steve Loft's Cumulus 2 was coded to do just that, using UTC internally, and in log files, just converting to local time zone on outputs.
Steve apparently found it impossible to make it work, I don't recall him explaining why he struggled with the UTC to local time conversions. Perhaps it was because of the complications for rollover processing time or because his C# skills were limited?
Anyway, he abandoned that idea when his MX beta took some ideas from both Cumulus 1 and Cumulus 2.

I have an idea that Mark has, at times, considered making MX incompatible with the legacy by various internal standardisations (with external presentational conversions) to make coding easier. In my opinion MX has already diverged so much from the legacy, I don't think compatibility between them matters any more, especially as it is possible to write one-off conversion utilities.
freddie wrote: Sun 01 Aug 2021 6:45 am Something I've been doing for years. When storing and manipulating timeline-based data such as meteorological readings then artificial adjustments such as DST have no place in the storage layer.
I am sure I recall several other Cumulus users in the past, not just you, saying they had adopted a fixed time-zone, either for easier comparison with official data or because they have an international interest in their weather data.
User avatar
mcrossley
Posts: 12694
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: First startup after upgrading to b3141 - error message noticed

Post by mcrossley »

I think using UTC for all internal storage is a great idea, and I do not see any great reason why it would not work. But...

The issue is the effort involved in converting all the existing code. Then writing all the code required for converting all the various existing data files. Then all the support required hand holding people migrating and getting into problems.

The short answer - yes the new database still has all the issues with DST that the original code had.

Background...

Before this release MX kept recent data (used for the recent graphs etc) in in-memory structures. So if you restarted the graphs reverted to using whatever log interval you used until the in-memory storage buffered eventually held the 7 days or whatever you specified.

For this release I just moved that recent data from in-memory to an on-disk SQLite data base so it persists across runs of MX.
sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

Re: First startup after upgrading to b3141 - error message noticed

Post by sfws »

Steve Loft, for anyone interested, mentions this in viewtopic.php?p=100098#p100098.
Post Reply