Data values stuck

[malkie]

I started my Pi2 off on Sunday running version 3018. I then went away on a business trip early on Monday. I discovered later in the day that the Pi had stopped updating the values from my Vue console at 00:20 on Monday morning. It was still refreshing the web pages, but the data was all the same.

I got home tonight and found everything was still running, the Vue Console was showing live data, but the MX console (local) was also showing the same stuck values as were present on the web pages.

The MXdiag files shows one message of interest:

2015-02-09 00:20:00.874 Writing log entry for 09/02/2015 00:20:00
2015-02-09 00:20:00.876 Written log entry for 09/02/2015 00:20:00
2015-02-09 00:20:00.882 Writing today.ini, LastUpdateTime = 09/02/2015 00:20:00 raindaystart = 0.21 rain counter = 0.21
2015-02-09 00:22:20.294 Done uploading standard files, uploading graph data files
2015-02-09 00:23:06.023 Error uploading raindatad3.json to raindatad3.json : Timed out trying to read data from the socket stream!

I stopped CumulusMX and rebooted the Pi2 and the values were now refreshed but it did not update the values when it was stuck, as it thought it had valid data I guess.

So I deleted the 'stuck' values from Feb15log.txt, then changed the timestamp in today.ini to the time of the last good reading, and when I restarted CumulusMX again the data was retrieved from the USB logger, (thanks to belfryboy!). I then found two entries for the 9th and 10th in dayfile.txt, but that was easy to resolve. Everything else seems to have updated itself OK.

Steve, any idea what "Timed out trying to read data from the socket stream" is referring to?

[steve]

I think it just means that the ftp connection timed out - no response from the server. That shouldn't have stopped it reading the data from the Vue, though, as that's done in a separate thread.

[malkie]

Thanks Steve, all working fine so far, fingers crossed.

[malkie]

This happened to my Pi2 again on Saturday morning. The data from the Console was not updated after 7:07 in the morning. The MXdiag files has this train of events:


2015-03-14 07:00:42.990 Reading reception stats
2015-03-14 07:02:21.069 Reading reception stats
2015-03-14 07:03:58.949 Reading reception stats
2015-03-14 07:05:36.989 Reading reception stats
2015-03-14 07:07:15.000 Reading reception stats
2015-03-14 07:07:57.882 System.IO.IOException: Input/output error
at System.IO.Ports.SerialPortStream.ThrowIOException () [0x00000] in <filename unknown>:0
at System.IO.Ports.SerialPortStream.get_BytesToRead () [0x00000] in <filename unknown>:0
at System.IO.Ports.SerialPort.get_BytesToRead () [0x00000] in <filename unknown>:0
at (wrapper remoting-invoke-with-check) System.IO.Ports.SerialPort:get_BytesToRead ()
at . (Int32 ) [0x00000] in <filename unknown>:0
2015-03-14 07:07:57.882 Reading reception stats
System.IO.IOException: Input/output error
at System.IO.Ports.SerialPortStream.ThrowIOException () [0x00000] in <filename unknown>:0
at System.IO.Ports.SerialPortStream.DiscardInBuffer () [0x00000] in <filename unknown>:0
at System.IO.Ports.SerialPort.DiscardInBuffer () [0x00000] in <filename unknown>:0
at (wrapper remoting-invoke-with-check) System.IO.Ports.SerialPort:DiscardInBuffer ()
at . (System.IO.Ports.SerialPort , System.String ) [0x00000] in <filename unknown>:0
at . () [0x00000] in <filename unknown>:0
at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0
2015-03-14 07:10:00.668 Writing log entry for 14/03/2015 07:10:00
2015-03-14 07:10:00.670 Written log entry for 14/03/2015 07:10:00


I have set 'Logging=1' in the Cumulus.ini file to see if it records any more helpful information should it happen again. As before a simple restart of the Pi fixed it, no power off/on or reconnection of the USB cable needed.

I have had a lot of internet connection issues over the last week or so and MX has coped admirably, the above errors would seem to be related to 'serial' communication rather than a Network issue.

[steve]

Yes, you have some kind of communication problem to/from the console - it's impossible to say exactly where, a hardware or driver issue probably. Cumulus should really trap the error and report it in a more friendly way, but other than that, there's nothing it can do about it.

[malkie]

Amazingly it happened again while I was at work today.

I have attached the zipped diags file.

[steve]

Yes, looks like the same problem.

[malkie]

Ok time for an update on this issue.

I was originally using my Vantage Vue with a Bellfry-boy clone logger and it was that set-up that suffered the flat line data problem.

Since buying the Vantage Vue I succeeded to buy a genuine Davis USB logger along with an IP logger on eBay. I could not make the IP logger work with Cumulus MX, but since swapping to the genuine Davis USB logger the data has been flowing OK.

Then the Pi decided to stop uploading every day at 06:20.

My Pi (a Pi2) was running headless and when it stopped uploading I could not ssh to it nor could I see the MX Console Web pages. I could ping it though, which is odd. It has a fixed IP so no way that was another device with the same IP. A power cycle was the only option. This went on for some days then I found the time to reconnect a screen and keyboard.

When it stopped uploading I found it had crashed and there was a 'core dump' displayed on the screen.

I realised the logs were stopping at the same time of day, in fact 06:25, and nothing to do with Cumulus MX.

To cut a very long story short, at 06:25 the Pi runs all the tasks in the /etc/cron.daily folder. I edited cron tab to change the task run time and the crash time changed.

I started trying to eliminate the tasks in this folder, then decided to run apt-get update followed by apt-get upgrade. This updated lots of packages since I got my Pi2 soon after they came out and there have been two updates to the OS since then.

This seems to have fixed it. One of the cron.daily tasks is to update the 'man' database to ensure all the help files are in order. Of course running the Update scripts has rebuilt the 'man' database in quite a big way, so that may be the fix.

Fingers are crossed, it has run uninterrupted for two days now!

[mcrossley]

If you haven't done it already, it may also be worth checking for any firmware updates using rpi-update.

[malkie]

Two weeks on and time for another update.

After the last updates my Pi ran faultlessly for 2 weeks, and during that time I had horrendous ADSL connection issues, with the service coming and going almost at will, but the Pi2 and CumulusMX coped, it just worked, well done Steve!

Regarding the firmware updates posted above by Mark, there is conflicting information about this. This page at the Raspberry Foundation: https://www.raspberrypi.org/documentati ... pdating.md
implies that the the firmware updates come with the package updates, but I ran rpi-update and it did update the 'firmware' even after having run "api-get update" just before.

For those who have not looked into it, the 'firmware' on a Pi is actually stored on the boot partition of the SD (or microSD) card, so to my mind isn't really firmware as it is not stored in the hardware. OK, rant over(!) if the 'firmware' is updated the Pi has to be rebooted to load the new code.

I have only just done the reboot, as a nice man from Virgin Media came in on Friday and installed their 50Meg 'basic' Broadband for me. 40-50 Mbits down and 3-5 Mbits up is a bit of an improvement over the 5/0.8 that I had with ADSL when it was working well. Since my Pi is running on a fixed IP address, and the Virgin modem would not let me use the subnet I had been using, I had to change the IP of the Pi and my NAS boxes etc. Why is life never simple? Anyway it is behaving nicely, all the new updates have loaded and the web uploads are almost instant. Nearly time to switch off the Windows box running Cumulus 1!

[BCJKiwi]

So does this confirm that you have CumulusMX running on a Davis IP logger?

[malkie]

So does this confirm that you have CumulusMX running on a Davis IP logger?

No, unfortunately not. I'm using a USB Davis logger. When I tried the IP Logger MX refused to see any data. I could connect to the logger via a browser on my desktop PC, but MX did not connect to it.

I will try again with the IP logger, and connect to it via weather link just to make sure it is working.

[steve]

I put some improvements for IP loggers in 3024, see other threads. I think I may have some more in the pipeline in 3025. Adjusting the DavisIPResponseTime setting may also help.