Page 1 of 1

[Implemented] release 3.10.1 *** Data input appears to have stopped

Posted: Wed 07 Apr 2021 5:54 pm
by sfws
When communication between a weather station console and the original cumulus software stopped:
-> it showed a flashing red fault light on the main screen
-> the software halted further processing, for example no longer updating any files, no longer uploading to web server
-> optionally (by a setting), the software could be asked to close and restart, which generally cured the problem

MX works differently:
=> it logs *** Data input appears to have stopped to the current diagnostics logging file within MXdiags folder (if you bother to look at it)
=> the software continues to run
=> whilst today.ini no longer gets updated with new data, it and the other logging files are still showing as being accessed/modified, although nothing is changing
=> all processing of any files specified in extra web files continues, again although no data is changing
=> unless you look at that diagnostic file, or scroll down on the dashboard page of the admin interface, one could assume all is still working as normal
=> there is no option that closes MX, despite the fact it is obviously not doing anything useful
=>,there is no option to see if an automatic restart would clear the problem

This post is asking if MX can be changed to be more like the original Cumulus functionality.
My life is not about sitting still tied to watching a screen. I find it annoying, MX will read my weather station perfectly for days. From time to time, I can suddenly discover Cumulus has not received any information from my weather station for perhaps days, despite a casual inspection making me believe MX is still running properly.

A few facts about my implementation.... I am still using a Chas Olsen Fine Offset console/transmitter (with various non-standard old Maplin sensors), and I run MX on a RPi. I am wondering if it is possible that, at apparently random times, the RPi is somehow struggling to keep the USB link open. My RPi is left on, without me monitoring it.
I have also tried running as a service, and can confirm that no message is in service log nor (when not as a service) put out to the terminal session when this problem happens.
I have not investigated for any system messages.

EDITS: Minor changes for clarity to text and marking topic as implemented, despite some posts in this thread not being related to my post here

Re: *** Data input appears to have stopped

Posted: Thu 08 Apr 2021 4:10 pm
by mcrossley
I'll add better handling of these errors to the ToDo list...

Re: *** Data input appears to have stopped

Posted: Fri 09 Apr 2021 12:46 pm
by sfws
mcrossley wrote: Thu 08 Apr 2021 4:10 pm I'll add better handling of these errors to the ToDo list...
Thank you, maybe it is worth saying that the weather station is not showing any problems, and when MX was restarted, it had no difficulty catching up all missed data.

Therefore, your to do list is to perhaps find a way to close connection with weather station and then reopen. MX does not need to totally stop and restart in the way the legacy software did.

Re: *** Data input appears to have stopped

Posted: Fri 09 Apr 2021 2:08 pm
by mcrossley
Good idea, I have coded it so that if the Data Stopped condition is set, it will attempt to find the station again and reopen it.

I have also tweaked the start-up code, so if it does not find the station it will keep trying until it does, or you kill MX.

Re: *** Data input appears to have stopped

Posted: Wed 14 Apr 2021 1:01 pm
by GeorgeF
My weather station also seems to stop at irregular intervals but the restart which usually means a shutdown of MX reseating the USB cable. The restart is fine and MX reads the archive data but seems to get some really random data. I think my fix request here is to apply spike removal to archive data reads because I am sure I did not have Force 12 Hurricane winds for 5 minutes (17740 m/sec) last week unless I was on Jupiter and this morning I had a temperature for 58 deg C at 07:50 am in Northern Britain.

These spikes leave all my historical data in ruins as well as the all time records etc. and it is a long editing of data files to get right. Please can MX just suppress these spikes on archive data reads?

I am running the latest build 3.10.5 build 3122

Regards

GeorgeF

Re: *** Data input appears to have stopped

Posted: Wed 14 Apr 2021 1:48 pm
by mcrossley
Hi George, spike removal is already applied to historic data.

Plus there are some additional range checks which mean the wind speed is ignored altogether if it lies outside the range 0 - 60 m/s so how you are getting those values is odd.

Can you post a message on the regular forum along with your log files and I'll take a look.

Re: *** Data input appears to have stopped

Posted: Sun 25 Apr 2021 10:14 pm
by euphbass
We've been having a similar issues on average once a week for the past month or so. There appears to be a loss of connection between the base unit and Cumulus MX (running on a very early Raspberry Pi). The error message in the logs is "Error reading data from station - it may need resetting". The only way to get it working again is to completely reset the base unit, then restart Cumulus, which is a faff. There is also, unfortunately, no way to know this has happened until you check one day and see that the last station update was ages ago! When it restarts it picks up the stored data from the base station to cover some of the gap, but a lot of the stored data comes out as nonsense, e.g. day sun patterns overnight, very large rainfall, temperature offset, that sort of thing. Causes havoc with the historical logs!

We're not sure if the problem is with the base station, the Raspberry Pi, or the software. We caught a failure in the log files this time - it happened at ~8am on The 22nd Apr. I've attached the log files. It all goes pear shaped from time stamp 2021-04-22 07:59:34.425. I've attached the log file in question. Any help greatly appreciated, even if it is just "get a better Raspberry Pi" or "your base station is broken"!

If it's a case of needing to update to the latest version of Cumulus MX (I'm running build 3117, installed via the quick and easy Raspberry Pi image), is there a new Raspberry Pi install image available, or do I need to go in and install it manually? Don't want to lose all the data so far, either. Thanks!

Re: *** Data input appears to have stopped

Posted: Sun 25 Apr 2021 11:11 pm
by PaulMy
If it's a case of needing to update to the latest version of Cumulus MX (I'm running build 3117, installed via the quick and easy Raspberry Pi image), is there a new Raspberry Pi install image available, or do I need to go in and install it manually? Don't want to lose all the data so far, either.
I don't use the Pi but there was some recent discussion on this here viewtopic.php?f=40&t=19275

Enjoy,
Paul

Re: *** Data input appears to have stopped

Posted: Mon 26 Apr 2021 11:27 am
by mcrossley
@euphbass What is your logging interval set to?

Re: *** Data input appears to have stopped

Posted: Tue 27 Apr 2021 8:03 pm
by euphbass
30 minutes logging interval.

Re: *** Data input appears to have stopped

Posted: Tue 27 Apr 2021 8:18 pm
by mcrossley
OK, that confirms what I was seeing from the console. That does mean that the "Synchronise reads" may be problematic, I'll need to give that a looking at - with long logging intervals it may actually be more likely to trigger lock-ups.

Re: *** Data input appears to have stopped

Posted: Tue 27 Apr 2021 11:09 pm
by Cortmalaw
These are the kind of glitches that a scheduled daily reboot of the Raspberry Pi (or any other computer) is fairly likely to avoid (if you are lucky) or fix (if you are half lucky).