Data input stops after IOException

[ninendal]

Hi,

Cumulus MX stops data input after an IOException;

2016-02-02 18:41:13.795 !!! loop data not received
2016-02-02 18:41:14.813 System.IO.IOException: Write failure ---> System.Net.Sockets.SocketException: Connection reset by peer
at System.Net.Sockets.Socket.Send (System.Byte[] buffer, Int32 offset, Int32 size, SocketFlags flags) <0x73874e00 + 0x000c4> in <filename unknown>:0
at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) <0x73874a88 + 0x000b7> in <filename unknown>:0
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) <0x73874a88 + 0x00128> in <filename unknown>:0
at . (System.Net.Sockets.TcpClient , System.String ) <0x7375ee20 + 0x0009b> in <filename unknown>:0
at . () <0x7375e6f0 + 0x0020f> in <filename unknown>:0
at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) <0x74c712bc + 0x000c7> in <filename unknown>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x74c6f934 + 0x001af> in <filename unknown>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, Boolean preserveSyncCtx) <0x74c6f900 + 0x0002b> in <filename unknown>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) <0x74c6f84c + 0x0005b> in <filename unknown>:0
at System.Threading.ThreadHelper.ThreadStart () <0x74c7148c + 0x00037> in <filename unknown>:0

2016-02-02 18:42:00.913 *** Data input appears to have stopped

Cumulus keeps running and sending data but without no new data.

Something to fix in the future or is there something I can do?

[steve]

The remote end has disconnected the TCP connection. See this very recent thread - https://cumulus.hosiene.co.uk/viewtopic.php?f=27&t=14884

How is your PC connected to your weather station?

[ninendal]

The remote end has disconnected the TCP connection. See this very recent thread - https://cumulus.hosiene.co.uk/viewtopic.php?f=27&t=14884

How is your PC connected to your weather station?

I'm using raspberry PI 2 model B and is connected with my Vantage 2 Pro + with a TCP/IP connection.

[steve]

I guess you've just demonstrated that the remote end can disconnect even when it's straight to an IP logger (i.e. no Meteohub or anything else handling the connection). I've been using MX with an IP logger for over a year now, and it's never happened to me. Oh well...

[ninendal]

I guess you've just demonstrated that the remote end can disconnect even when it's straight to an IP logger (i.e. no Meteohub or anything else handling the connection). I've been using MX with an IP logger for over a year now, and it's never happened to me. Oh well...

I keep an eye on it, in case it was my own doing Thanks for your reply.

[Rayj44]

Hi Steve,

Similar thing here this afternoon. Not seen it before. Using RPi headless, tcp connection to VP2 logger.

Ray

2016-02-04 15:00:00.315 Hour changed:15
2016-02-04 15:00:00.316 Calculating sunrise and sunset times
2016-02-04 15:00:00.322 Sunrise: 07:52:41
2016-02-04 15:00:00.323 Sunset : 16:56:41
2016-02-04 15:00:00.324 Tomorrow sunrise: 07:50:54
2016-02-04 15:00:00.324 Tomorrow sunset : 16:58:43
2016-02-04 15:00:00.337 Writing log entry for 04/02/2016 15:00:00
2016-02-04 15:00:00.338 Written log entry for 04/02/2016 15:00:00
2016-02-04 15:00:00.346 Writing today.ini, LastUpdateTime = 04/02/2016 15:00:00 raindaystart = 121.2 rain counter = 128.4
2016-02-04 15:00:01.482 WeatherCloud Response: OK: 200
2016-02-04 15:01:19.673 !!! loop data not received
2016-02-04 15:02:00.321 *** Data input appears to have stopped
2016-02-04 15:03:00.323 *** Data input appears to have stopped
2016-02-04 15:04:00.326 *** Data input appears to have stopped
2016-02-04 15:05:00.328 *** Data input appears to have stopped
2016-02-04 15:05:00.333 Writing log entry for 04/02/2016 15:05:00
2016-02-04 15:05:00.334 Written log entry for 04/02/2016 15:05:00
2016-02-04 15:05:00.341 Writing today.ini, LastUpdateTime = 04/02/2016 15:05:00 raindaystart = 121.2 rain counter = 128.4
2016-02-04 15:06:00.331 *** Data input appears to have stopped
2016-02-04 15:07:00.333 *** Data input appears to have stopped
2016-02-04 15:08:00.336 *** Data input appears to have stopped
2016-02-04 15:09:00.338 *** Data input appears to have stopped
2016-02-04 15:10:00.341 *** Data input appears to have stopped
2016-02-04 15:10:00.345 Writing log entry for 04/02/2016 15:10:00
2016-02-04 15:10:00.347 Written log entry for 04/02/2016 15:10:00
2016-02-04 15:10:00.355 Writing today.ini, LastUpdateTime = 04/02/2016 15:10:00 raindaystart = 121.2 rain counter = 128.4
2016-02-04 15:10:02.594 WeatherCloud Response: OK: 200
2016-02-04 15:11:00.343 *** Data input appears to have stopped
2016-02-04 15:12:00.345 *** Data input appears to have stopped
2016-02-04 15:13:00.348 *** Data input appears to have stopped
2016-02-04 15:14:00.350 *** Data input appears to have stopped
2016-02-04 15:15:00.353 *** Data input appears to have stopped
2016-02-04 15:15:00.357 Writing log entry for 04/02/2016 15:15:00
2016-02-04 15:15:00.360 Written log entry for 04/02/2016 15:15:00
2016-02-04 15:15:00.366 Writing today.ini, LastUpdateTime = 04/02/2016 15:15:00 raindaystart = 121.2 rain counter = 128.4
2016-02-04 15:16:00.356 *** Data input appears to have stopped
2016-02-04 15:16:48.577 System.IO.IOException: Write failure ---> System.Net.Sockets.SocketException: No route to host
at System.Net.Sockets.Socket.Send (System.Byte[] buf, Int32 offset, Int32 size, SocketFlags flags) [0x00000] in <filename unknown>:0
at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0
at . (System.Net.Sockets.TcpClient , System.String ) [0x00000] in <filename unknown>:0
at . () [0x00000] in <filename unknown>:0
at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0

[steve]

That one seems to be a routing problem, I don't think there's anything I can do about that - disconnecting/reconnecting the TCP/IP connection probably won't help.

[Rayj44]

That one seems to be a routing problem, I don't think there's anything I can do about that - disconnecting/reconnecting the TCP/IP connection probably won't help.

Looking back, there have been occasional unexplained disconnects and reconnects before. I suspect also with Cumulus 1. I decided to upgrade the wL IP firmware from 1.09 to 1.11 just in case that was the cause. The upgrade repeatedly failed. I now have a totally bricked logger. Seems like my logger has been going into failure mode for a while.

Ray

[UncleBuck]

At 08:55 on 11th Feb (local time) this started happening:

Code: Select all

2016-02-11 08:55:00.591 Writing today.ini, LastUpdateTime = 2/11/2016 8:55:00 AM raindaystart = 65 rain counter = 65
2016-02-11 08:55:08.118 !!! loop data not received
2016-02-11 08:55:08.118 Reading reception stats
2016-02-11 08:55:18.184 !!! No ack received in response to LPS 2 1
2016-02-11 08:55:23.936 !!! No ack received in response to LOOP 50
2016-02-11 08:55:29.684 !!! No ack received in response to LPS 2 1
2016-02-11 08:55:35.439 !!! No ack received in response to LOOP 50
2016-02-11 08:55:41.188 !!! No ack received in response to LPS 2 1
2016-02-11 08:55:46.931 !!! No ack received in response to LOOP 50
2016-02-11 08:55:52.693 !!! No ack received in response to LPS 2 1
2016-02-11 08:55:58.413 !!! No ack received in response to LOOP 50
2016-02-11 08:56:00.860 *** Data input appears to have stopped
2016-02-11 08:56:00.861 Writing log entry for 2/11/2016 8:56:00 AM
2016-02-11 08:56:00.861 Written log entry for 2/11/2016 8:56:00 AM
2016-02-11 08:56:00.862 Writing today.ini, LastUpdateTime = 2/11/2016 8:56:00 AM raindaystart = 65 rain counter = 65
2016-02-11 08:56:04.168 !!! No ack received in response to LPS 2 1
2016-02-11 08:56:09.937 !!! No ack received in response to LOOP 50
2016-02-11 08:56:15.666 !!! No ack received in response to LPS 2 1
2016-02-11 08:56:21.409 !!! No ack received in response to LOOP 50
2016-02-11 08:56:27.132 !!! No ack received in response to LPS 2 1
2016-02-11 08:56:32.874 !!! No ack received in response to LOOP 50
2016-02-11 08:56:38.628 !!! No ack received in response to LPS 2 1
2016-02-11 08:56:44.385 !!! No ack received in response to LOOP 50
2016-02-11 08:56:50.151 !!! No ack received in response to LPS 2 1
2016-02-11 08:56:55.901 !!! No ack received in response to LOOP 50
2016-02-11 08:57:00.045 *** Data input appears to have stopped

This caused the realtime file (and all other data) to repeat the same readings over and over again so (for example) the temperature reading stayed at 26.6 until I just picked up there was a problem. From 08:55 onwards that same group of log entries is repeated over and over.

Is this related to the same issue?

[steve]

Apparently not, as there was no exception caused by the connection being disconnected. It appears that the logger just stopped responding. Without data logging turned on it's hard to say more. How is the machine running MX connected to the station (what type of logger and how is that connected)?

[UncleBuck]

The set up is a weather link 6520 usb connected to a Mac mini running El Capitan server. If it happens again then I will turn on the data logging. Restarting CMX seems to have resolved the issue for now but unfortunately I have lost all the data from the time it flatlined.

[steve]

Restarting CMX seems to have resolved the issue for now but unfortunately I have lost all the data from the time it flatlined.

You can 'rewind' to the start of the day (or the last time you started MX before this happened - use the later of the two options) by copying the files from an appropriate Cumulus backup folder into the data folder (with Cumulus stopped), and then starting Cumulus. It will then fill the gap with logger data.

[UncleBuck]

Unfortunately I left it to late to do that and the data in the logger has rolled over.

At least I know that I can do that if it happens again.