SFTP does not recover

[HansR]

There was a major internet outage this morning that lasted only five minutes but CMX (SFTP, b 3132) didn't recover from it (it always did before). I solved it by a restart after 1.5 hrs (when I discovered it had not recovered)
The relevant part of the log until my restart of CMX is attached. Must say that strangely the webcam did not recover by itself either.
The disruption was from 7h23 - 7h28 resulting in (part of log):

Code: Select all

2021-06-22 07:27:25.995 Realtime[130]: Warning, a previous cycle is still trying to connect to FTP server, skip count = 11
2021-06-22 07:27:25.995 Realtime[130]: Realtime has been in progress for more than 5 minutes, attempting to reconnect.
2021-06-22 07:27:40.152 SendLoopCommand: Starting - LOOP 20
2021-06-22 07:27:51.016 Realtime[130]: Error reconnecting ftp server - Could not resolve host 'ftp.meteo-wagenborgen.nl'
2021-06-22 07:27:51.016 SFTP[130]: Attempting realtime SFTP connect to host ftp.meteo-wagenborgen.nl on port 22
2021-06-22 07:27:55.993 Realtime[131]: Warning, a previous cycle is still trying to connect to FTP server, skip count = 12
2021-06-22 07:27:55.993 Realtime[131]: Realtime has been in progress for more than 5 minutes, attempting to reconnect.
2021-06-22 07:28:00.158 DoLogFile: Writing log entry for 22-6-2021 07:28:00
2021-06-22 07:28:00.159 DoLogFile: Written log entry for 22-6-2021 07:28:00
2021-06-22 07:28:00.164 Writing today.ini, LastUpdateTime = 22-6-2021 07:28:00 raindaystart = 438.8 rain counter = 438.8
2021-06-22 07:28:01.044 SFTP[130]: Error connecting sftp - Could not resolve host 'ftp.meteo-wagenborgen.nl'
2021-06-22 07:28:06.005 Realtime[131]: Error reconnecting ftp server - Could not resolve host 'ftp.meteo-wagenborgen.nl'
2021-06-22 07:28:06.005 SFTP[131]: Attempting realtime SFTP connect to host ftp.meteo-wagenborgen.nl on port 22
2021-06-22 07:28:16.035 SFTP[131]: Error connecting sftp - Could not resolve host 'ftp.meteo-wagenborgen.nl'
2021-06-22 07:28:18.152 SendLoopCommand: Starting - LOOP 20
2021-06-22 07:28:25.998 Realtime[132]: Warning, a previous cycle is still trying to connect to FTP server, skip count = 13
2021-06-22 07:28:25.998 Realtime[132]: Realtime has been in progress for more than 5 minutes, attempting to reconnect.
2021-06-22 07:28:36.005 Realtime[132]: Error reconnecting ftp server - Could not resolve host 'ftp.meteo-wagenborgen.nl'
2021-06-22 07:28:36.006 SFTP[132]: Attempting realtime SFTP connect to host ftp.meteo-wagenborgen.nl on port 22
2021-06-22 07:28:46.031 SFTP[132]: Error connecting sftp - Could not resolve host 'ftp.meteo-wagenborgen.nl'
2021-06-22 07:28:56.001 Realtime[133]: Warning, a previous cycle is still trying to connect to FTP server, skip count = 14
2021-06-22 07:28:56.001 Realtime[133]: Realtime has been in progress for more than 5 minutes, attempting to reconnect.
2021-06-22 07:28:56.151 SendLoopCommand: Starting - LOOP 20

[philpugh]

It's almost like the original FTP request doesn't time-out - i.e. even when the network comes back it doesn't resolve the address as it is still trying the original request?

[mcrossley]

It looks like the DNS look-up was failing. It could be that the host was caching a null record from when the internet was down, or more likely .Net (Mono) was, and CMX was keeping that cache fresh by repeatedly requesting the lookup.

There does not seem to be a cross-platform method of flushing the .Net System.Net DNS cache that I can readily find?

[HansR]

Mmm... quick searching... not sure but...

Does this help:

1. https://stackoverflow.com/questions/727 ... -dns-cache
   https://stackoverflow.com/questions/210 ... sharp-mono (using ServicePointManager.DnsRefreshTimeout?)

(just back, cleaning the backlog, my mind is not set to code yet)

[mcrossley]

I'll add it in to set that cache time to zero on a host not found error, then reset to default on a successful connection, can't see it doing any harm.

[HansR]

I'll add it in to set that cache time to zero on a host not found error, then reset to default on a successful connection, can't see it doing any harm.

Yes, I thought the same.