Page 1 of 1

CumulusMX is omitting some 10-minute readings at random in the Jun19log.txt file

Posted: Fri 07 Jun 2019 8:47 am
by martinu
Since we moved house and I cleared the contents of the CumulusMX/data folder (stop Cum, delete folder contents, restart) to begin a new set of readings, I've noticed that some readings are missing at random from the Jun19log.txt file.

I log to the file at 10 minute intervals (this is also the interval that I've set on the weather station), and I've set Cumulus to generate a updated web page and a new data point on the graphs every minute. The graphs appear to have points every minute, even over times when an entry is missing from the log file. My first thought was that the base station (Davis Vantage Vue) was sometimes losing radio contact with the remote sensor unit, but the fact that there are valid data points on the graph seems to refute that.

For example, looking at the temperature graph in http://goosebears.co.uk/weather/trends.htm (pages uploaded manually by temporarily connecting my PC to a tethered mobile phone connection!), there are points every minute at around 0800-0830 today (7 Jun) although there is a missing point at 0810 in the log file.

Any suggestions. I never saw this symptom at our old house, before I cleared the data and before I changed the sensor-to-base unit distance. Same setup, same USB port on the logging computer.

The only thing that is non-standard is that at the moment the router has no WAN connection to the internet (we're waiting for broadband to be enabled) so all the FTP updates performed by Cumulus will be failing. Could that be significant in any way?

I'm running Cumulus v3.0.0 (3043) on a Raspberry Pi 3B+.

Re: CumulusMX is omitting some 10-minute readings at random in the Jun19log.txt file

Posted: Fri 07 Jun 2019 9:40 am
by mcrossley
Switch on debug logging, let it run until the problem occurs a couple of times, then zip up your MXDiags folder and post it here.

Re: CumulusMX is omitting some 10-minute readings at random in the Jun19log.txt file

Posted: Fri 07 Jun 2019 3:53 pm
by martinu
Here's the MXdiag.zip file. There was an instance at 16:20 on 7 Jun 2019, in the latest log file. Interestingly, there is no error message as such, just an absence of the normal "Writing log entry for 07/06/2019 16:30:52" / "Written log entry for 07/06/2019 16:30:52" lines every 10 minutes.

This is with Settings | Options | Debug logging ticked, but without restarting Cumulus; the help text "Changing setting here affects current session only" implies that the change affects the currently running Cumulus and that restarting Cumulus will turn debugging off again.

(Ignore all the "failed to write to realtime.txt" type errors. They are due to us not having broadband yet in our new house. It will be a while before we get proper internet: when the BT Openreach engineer turned up today he found that another engineer had "stolen" our line-pair back to the exchange yesterday, believing it was no longer being used because the previous occupants had ceased the line and we hadn't (by a mere couple of hours!) submitted our order for phone/broadband. BT have gone away to work out what to do next...)

Re: CumulusMX is omitting some 10-minute readings at random in the Jun19log.txt file

Posted: Sun 16 Jun 2019 6:40 pm
by mcrossley
Sorry for the late reply, could you collect the diags again after updating to the latest release please.

Re: CumulusMX is omitting some 10-minute readings at random in the Jun19log.txt file

Posted: Tue 18 Jun 2019 7:41 pm
by martinu
Thanks for the suggestion to upgrade to the latest version, V3.0.0 B3050. I hadn't noticed that there had been a few newer versions. I've upgraded now, but it's only been running a couple of hours. It may be a day or so until I see a rash of missing values. I'll report back either way - either "seems to be fixed" or "it's still happening".

It will be interesting to see if the newer version also cures a problem with the console site (localhost:8998) stopping responding after a few days or else displaying the page templates but no data values or graphs.

Odd that both of those problems only began when we moved house a few weeks ago, with no change to the weather station or the version/config of Cumulus. The only difference is the lack of an internet connection at present - latest from our ISP is that we shoudl be activated on 22 June. Could an inability to upload to a web site via FTP cause intermittent loss of 10-minute log file entries, even though they all show up in the graph?

Re: CumulusMX is omitting some 10-minute readings at random in the Jun19log.txt file

Posted: Wed 19 Jun 2019 2:35 pm
by mcrossley
martinu wrote: Tue 18 Jun 2019 7:41 pm Could an inability to upload to a web site via FTP cause intermittent loss of 10-minute log file entries, even though they all show up in the graph?
Without trawling through the code I guess it could be possible, though you'd expect the internet calls to timeout fairly quickly when you're disconnected.

Re: CumulusMX is omitting some 10-minute readings at random in the Jun19log.txt file

Posted: Fri 21 Jun 2019 9:46 am
by martinu
It would appear that upgrading to V3.0.0 B3050 has cured the problem.

It's been soak-testing (not a comment on the amount of rain the country has had recently!) for about 48 hours, and:

a) I've not seen any missed 10-minute-interval entries in Jun19log.txt since then
b) I've always been able to connect to <RasPi IP>:8998 from other computers on the network to access the console

This has been the case even while I had no internet connection (though that was enabled this morning), so I think the lack of ability to Cumulus upload pages by FTP was probably a red herring. Intriguing that the old version (B3043, I think) had been working flawlessly until we moved house, and that the missing log entries and intermittent loss of access to the port 8998 site only began after that. Weird, but it works now which is all that matters.

Thanks to MCrossley for suggesting upgrading. I'd checked the release notes and hadn't spotted anything that appeared to relate to these symptoms, so I made the fatal mistake of thinking "the old version used to work, so I don't need to upgrade" :-(