Welcome to the Cumulus Support forum.

Latest Cumulus MX V4 release 4.3.3 (build 4070) - 04 January 2025

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

If you are posting a new Topic on an error or if you need help PLEASE read this first viewtopic.php?p=164080#p164080

Extra Sensors logging anomaly

From Cumulus MX version 3 build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since, and has recently released Cumulus MX version 4. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

Post Reply
User avatar
HansR
Posts: 6727
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Extra Sensors logging anomaly

Post by HansR »

I have several extra sensors, all on the same Ecowitt gateway as the main station. One of those is soil moisture, currently a value of 82% (easy to spot in the log). When I disable the sensor I expect the sensor to have a null value. Instead it keeps logging its last value (I can accept that) but it also logs a value some 24 positions ( :?: ) further. The last log section shows (with two 82 values the sensor is disconnected/disabled):

Code: Select all

07/01/25,17:25,2.2,18.6,13.4,,,,,,,,92,62,75,,,,,,,,1.0,11.2,9.1,,,,,,,,,,,,82,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,15.5,,,,,,,,,,,,,,,
07/01/25,17:30,2.1,18.5,13.4,,,,,,,,93,62,75,,,,,,,,1.1,11.1,9.1,,,,,,,,,,,,82,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,15.4,,,,,,,,,,,,,,,
07/01/25,17:35,2.0,18.5,13.4,,,,,,,,93,62,75,,,,,,,,1.0,11.1,9.1,,,,,,,,,,,,82,,,,,,,,,,,,,,,,,,,,,,,,82,,,,,,,,,,,,,,,,15.4,,,,,,,,,,,,,,,
07/01/25,17:40,1.9,18.7,13.4,,,,,,,,93,62,75,,,,,,,,0.9,11.3,9.1,,,,,,,,,,,,82,,,,,,,,,,,,,,,,,,,,,,,,82,,,,,,,,,,,,,,,,15.7,,,,,,,,,,,,,,,
07/01/25,17:45,2.0,19.0,13.4,,,,,,,,94,62,75,,,,,,,,1.1,11.6,9.1,,,,,,,,,,,,82,,,,,,,,,,,,,,,,,,,,,,,,82,,,,,,,,,,,,,,,,15.8,,,,,,,,,,,,,,,
What is happening here and what should the moisture value be? null or 82? I think null would be best: it makes it compulsory for the user to do something and it shows the sensor is out of order (either not connected, out of power, disabled or whatever).
Hans

https://meteo-wagenborgen.nl
CMX build 4039+ ● RPi 4B ● Linux 6.6.51+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
User avatar
HansR
Posts: 6727
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Re: Extra Sensors logging anomaly

Post by HansR »

And I can add that restarting without the sensor active works as expected (a null value) when activating the sensor I get only the 82 value on that position, not 25 positions further. The value is not displayed in the dashboard however. I'll add a logfile tomorrow after restart of CMX.
Hans

https://meteo-wagenborgen.nl
CMX build 4039+ ● RPi 4B ● Linux 6.6.51+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
User avatar
mcrossley
Posts: 14073
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Extra Sensors logging anomaly

Post by mcrossley »

A bug in the code, I'll fix it.
User avatar
HansR
Posts: 6727
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Re: Extra Sensors logging anomaly

Post by HansR »

mcrossley wrote: Wed 08 Jan 2025 10:36 am A bug in the code, I'll fix it.
OK. Thanks.
No need for a logfile I assume :D
Hans

https://meteo-wagenborgen.nl
CMX build 4039+ ● RPi 4B ● Linux 6.6.51+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
User avatar
mcrossley
Posts: 14073
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Extra Sensors logging anomaly

Post by mcrossley »

No - but looking at what is required, this will take a big chunk of dev time :(
User avatar
HansR
Posts: 6727
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Re: Extra Sensors logging anomaly

Post by HansR »

mcrossley wrote: Wed 08 Jan 2025 11:06 am No - but looking at what is required, this will take a big chunk of dev time :(
OK. With improvement at stake I am a patient man :D
Hans

https://meteo-wagenborgen.nl
CMX build 4039+ ● RPi 4B ● Linux 6.6.51+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
User avatar
mcrossley
Posts: 14073
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Extra Sensors logging anomaly

Post by mcrossley »

Digging into this some more, it turns out to be even more complex than a first look would indicate. It will add a considerable amount of extra processing to every data message received from the station :(

Unless AI can come up with a brilliant solution I haven't thought of! Framing the question to get any sensible output is the hard part!
User avatar
HansR
Posts: 6727
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Re: Extra Sensors logging anomaly

Post by HansR »

Interesting that this is so difficult. I did not expect that.
And aha.... AI is entering the CMX environment :D How and which tool? Do you put the software as input?
Hans

https://meteo-wagenborgen.nl
CMX build 4039+ ● RPi 4B ● Linux 6.6.51+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
User avatar
HansR
Posts: 6727
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Re: Extra Sensors logging anomaly

Post by HansR »

mcrossley wrote: Thu 09 Jan 2025 11:33 am Digging into this some more, it turns out to be even more complex than a first look would indicate. It will add a considerable amount of extra processing to every data message received from the station :(

Unless AI can come up with a brilliant solution I haven't thought of! Framing the question to get any sensible output is the hard part!
Extra processing for every message received... I thought that over.

Wouldn't it be more efficient to just check every 5 or 10 minutes (async) and array of available sensors for their presence (in the latetst communication with the station)) and those who failed get ignored as if not present after that. If I get a value for a failed sensor once or twice I could not care less. It is just that it has to be detected somewhere that a sensor is lost (and possibly raise a general alarm on a sensor lost indicating which sensor?). No need to do that every 4 seconds with every message of the station I think.

It's not AI but maybe it helps :?
Hans

https://meteo-wagenborgen.nl
CMX build 4039+ ● RPi 4B ● Linux 6.6.51+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
User avatar
mcrossley
Posts: 14073
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Extra Sensors logging anomaly

Post by mcrossley »

The problem with that is you then have to record the time the last data was received for each sensor.

The big complication is sensor data coming from extra sensors on a secondary station.

Example:

My primary station has Extra Temp #1, my secondary station has Extra Temp #2.

The primary station sends its data, there is data for extra temp #1, but nothing for extra temp #2. But we cannot set extra temp #2 to null, because it may be coming from the secondary station. The same is true in reverse for the secondary station.

So the boths stations must record if it previously had data for each sensor, and if it "disappears" or becomes invalid, only then set it to null.

Further complications, Davis for instance send a fixed data structure, with an entry for each sensor, each sensor contains data or not. Ecowitt only send the sensors that actually have data. Simple in the past, Ecowitt MX just parsed each sensor and recorded its value. Now it must check if every possible sensor is present, and if not, was it there in the past? This all equals lots more code for all the station types.
User avatar
HansR
Posts: 6727
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bookworm
Location: Wagenborgen (NL)
Contact:

Re: Extra Sensors logging anomaly

Post by HansR »

I see. Thnx for explaining. :|
Hans

https://meteo-wagenborgen.nl
CMX build 4039+ ● RPi 4B ● Linux 6.6.51+rpt-rpi-v8 aarch64 (bookworm) ● dotnet 8.0.1
BlueSky: https://bsky.app/profile/wagenborgenwx.bsky.social
Post Reply