Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.5 (build 3282) - 23 February 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Data spike (humidity)

From build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

Post Reply
Benji60
Posts: 67
Joined: Mon 11 Oct 2021 12:38 pm
Weather Station: HP1000 Froggit Pro
Operating System: Windows 10

Data spike (humidity)

Post by Benji60 »

Hello,

First real big storm for me since I got my station and Cumulus detected a humidity spike which means that since yesterday evening when the storm arrived, the humidity has been stuck at 64% instead of rising to 90% in less than 10 minutes. Except that the software remained on 64% until I restarted it this morning.
As I have set the data catching up with Ecowitt, is there a way to catch up the 10 hours of humidity?
I have a huge doubt, I can't find the place where you can remove the spikes suppression?

Translated with www.DeepL.com/Translator (free version)

Thanks

Translated with www.DeepL.com/Translator (free version)
User avatar
Gyvate
Posts: 295
Joined: Wed 16 Dec 2020 2:14 pm
Weather Station: GW1x00/WH2650WiFi/HP2553/GW2000
Operating System: Win 11 (PC/RPi), Raspbian 11,WSL
Location: Saarbrücken, Germany
Contact:

Re: Data spike (humidity)

Post by Gyvate »

Assuming your console has posted the proper humidity readings to the Ecowitt cloud.
If your CMX is configured for the Ecowitt cloud backfill (API key, App key, MAC address of the posting console), you can go back to the daily backup of either last midnight or day before last midnight. The files you will find in ..\CumulusMX\backups\daily
First save the files in the ..\CumulusMX\data directory which have the same name as those in the backups directory to some other place (or rename them).
Shut down CMX.
Copy the files from the .\backup directory into the .\data directory overwriting your before saved files.
Restart CMX. It should now download all the data since midnight of the day you chose the backup from.

If you have a one minute logging interval, you can fill the gaps from your saved files (maybe using an average humidity value between the 5 minute readings from the cloud) by copying the respective lines into the log file. (your extra sensor logfile data - provided you have any - you can take 1:1 from the saved files as no humidity reading should be there).

Work on a copy of May22log.txt to complete the work. Then shut down CMX again on the minute (2 seconds later). Copy your updated data into the actual May22log.txt file before the time CMX continued logging while you were working on the copy. Restart CMX as soon as possible, best still within the 30 seconds bfore the next full minute not to lose a record.

If you are anyway using a 5 minute interval, you are already set after CMX completed the backfill.
User avatar
mcrossley
Posts: 12641
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Data spike (humidity)

Post by mcrossley »

You will find your spike removal settings under "Calibration settings" - they default to 999 - i.e. "off".
User avatar
Gyvate
Posts: 295
Joined: Wed 16 Dec 2020 2:14 pm
Weather Station: GW1x00/WH2650WiFi/HP2553/GW2000
Operating System: Win 11 (PC/RPi), Raspbian 11,WSL
Location: Saarbrücken, Germany
Contact:

Re: Data spike (humidity)

Post by Gyvate »

as far as I read the OP's writing, his issue was mainly that the "spike", a real reading of 90% humidity, wasn't recorded by CMX but somehow got stuck at 64%.
So the opposite of spike removal - rather spike allowance within the set limits.
The spike "removal" won't help here. But still good to know for him where this option is located.
But I don't think that this option has to do with the CMX behaviour in that case.
freddie
Posts: 2421
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Data spike (humidity)

Post by freddie »

I read it as the spike detection/removal at work.
Cumulus detected a humidity spike which means that since yesterday evening when the storm arrived, the humidity has been stuck at 64% instead of rising to 90% in less than 10 minutes
So Cumulus detected the spike in humidity value and disallowed it as there was a spike setting in place. If the humidity remained at a high value then spike detection would have continued to be triggered, with Cumulus discarding the readings. This would result in the humidity being recorded at the "stuck" lower value until a restart was performed.

IMHO, of course :)
Freddie
Image
User avatar
Gyvate
Posts: 295
Joined: Wed 16 Dec 2020 2:14 pm
Weather Station: GW1x00/WH2650WiFi/HP2553/GW2000
Operating System: Win 11 (PC/RPi), Raspbian 11,WSL
Location: Saarbrücken, Germany
Contact:

Re: Data spike (humidity)

Post by Gyvate »

let's see what the OP reports. By the question it seems he didn't know where this is done. Then it should still be the the default = off - and then no suppression would have taken place and especially not over a long period of time (10 hours) ...
Benji60
Posts: 67
Joined: Mon 11 Oct 2021 12:38 pm
Weather Station: HP1000 Froggit Pro
Operating System: Windows 10

Re: Data spike (humidity)

Post by Benji60 »

Hello,
Thank you for the answers.
Yes indeed since the storm arrived the humidity has hardly gone below 85% and cumulus has sent me an email saying that a spike has been removed.
So I'll try the procedure to put back the right data and switch to off.

I'm going to have to manually modify my SQL database (for my charts) which cumulus also completes because of this 😅
https://raspigournayaronde.sytes.net/Gr ... phique.php
User avatar
Gyvate
Posts: 295
Joined: Wed 16 Dec 2020 2:14 pm
Weather Station: GW1x00/WH2650WiFi/HP2553/GW2000
Operating System: Win 11 (PC/RPi), Raspbian 11,WSL
Location: Saarbrücken, Germany
Contact:

Re: Data spike (humidity)

Post by Gyvate »

Benji60 wrote: Mon 16 May 2022 11:55 am ....
I'm going to have to manually modify my SQL database (for my charts) which cumulus also completes because of this 😅
https://raspigournayaronde.sytes.net/Gr ... phique.php
if you know SQL, you can also do this faster ;) , but as it's 600 records only (10 hours x 60 records/hour), it's not too much work doing it manually 8-)
Post Reply