Page 1 of 2
v. 3.6.2 crashing
Posted: Wed 20 May 2020 2:58 am
by Matt.j5b
I upgrade from 3.6.0 to v3.6.2 and now Cumulus MX crashes after around an hour. I have run Cumulus MX with -debug enabled and the diags file doesn't provide information at the end of the file as it to what has happened. It seems to silently crash. I didn't have this issue with 3.6.0 or previous versions.
I have attached the MX diags. Only the last two attempts has the debugging enabled.
MXdiags.zip
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 3:44 am
by Matt.j5b
I run MX as a crontask, but if I run it directly in the command line I get the exception as below:
Code: Select all
Current culture: English (United Kingdom)
Cumulus MX v.3.6.2 build 3078
Today.ini = data/today.ini
Last update=2020-05-20T12:35:00
Connected to station
Cumulus running at: http://*:8998/
(Replace * with any IP address on this machine, or localhost)
Open the admin interface by entering this URL in a browser.
20/05/2020 12:43:26
Type Ctrl-C to terminate
Downloading Archive Data
- processed 100%
Downloading Archive Data
- No historic data available
Normal running
System.ObjectDisposedException: Safe handle has been closed
at System.Runtime.InteropServices.SafeHandle.DangerousAddRef (System.Boolean& success) [0x0001c] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading.NativeEventCalls.SetEvent (Microsoft.Win32.SafeHandles.SafeWaitHandle handle) [0x00002] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading.EventWaitHandle.Set () [0x00000] in <61d5ec988156494684532710ebf1643d>:0
at (wrapper remoting-invoke-with-check) System.Threading.EventWaitHandle.Set()
at System.IOAsyncResult.set_IsCompleted (System.Boolean value) [0x0001e] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.SocketAsyncResult.Complete () [0x00031] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.SocketAsyncResult.Complete (System.Int32 total) [0x00007] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.Socket+<>c.<.cctor>b__313_8 (System.IOAsyncResult ares) [0x00070] in <99508e34019346e68bffacfcecbd997e>:0
at System.IOSelectorJob.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00000] in <99508e34019346e68bffacfcecbd997e>:0
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <61d5ec988156494684532710ebf1643d>:0
**** An error has occurred - please zip up the MXdiags folder and post it in the forum ****
Press Enter to terminate
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 8:30 am
by mcrossley
It does not appear to be crashing in the MX code. The only significant change from 3.6.0 to 3.6.2 was the update of the FluentFTP.dll file. That definitely got copied across did it?
I see from the logs it can run for up to an hour and half before it crashes, which again is odd.
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 9:01 am
by Matt.j5b
Thanks Mark.
I copied across the dll file just in case I missed that file, though I though I did. I will see if that solves it. MX have really been reliable with upgrades between versions up until now.
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 10:18 am
by Matt.j5b
I afraid that after copying the dll file I get this exception, crashing after 67 minutes:
Code: Select all
Current culture: English (United Kingdom)
Cumulus MX v.3.6.2 build 3078
Today.ini = data/today.ini
Last update=2020-05-20T19:15:00
Connected to station
Cumulus running at: http://*:8998/
(Replace * with any IP address on this machine, or localhost)
Open the admin interface by entering this URL in a browser.
20/05/2020 19:17:30
Type Ctrl-C to terminate
Downloading Archive Data
- No historic data available
Downloading Archive Data
- No historic data available
Normal running
System.ObjectDisposedException: Safe handle has been closed
at System.Runtime.InteropServices.SafeHandle.DangerousAddRef (System.Booleansuccess) [0x0001c] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading.NativeEventCalls.SetEvent (Microsoft.Win32.SafeHandles.SafeWaitHandle handle) [0x00002] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading.EventWaitHandle.Set () [0x00000] in <61d5ec988156494684532710ebf1643d>:0
at (wrapper remoting-invoke-with-check) System.Threading.EventWaitHandle.Set()
at System.IOAsyncResult.set_IsCompleted (System.Boolean value) [0x0001e] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.SocketAsyncResult.Complete () [0x00031] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.SocketAsyncResult.Complete (System.Int32 total) [0x00007] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.Socket+<>c.<.cctor>b__313_8 (System.IOAsyncResult ares) [0x00070] in <99508e34019346e68bffacfcecbd997e>:0
at System.IOSelectorJob.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00000] in <99508e34019346e68bffacfcecbd997e>:0
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <61d5ec988156494684532710ebf1643d>:0
**** An error has occurred - please zip up the MXdiags folder and post it in the forum ****
Press Enter to terminate
System.ObjectDisposedException: Safe handle has been closed
at System.Runtime.InteropServices.SafeHandle.DangerousAddRef (System.Boolean& success) [0x0001c] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading.NativeEventCalls.SetEvent (Microsoft.Win32.SafeHandles.SafeWaitHandle handle) [0x00002] in <61d5ec9881564946845327 10ebf1643d>:0
at System.Threading.EventWaitHandle.Set () [0x00000] in <61d5ec988156494684532710ebf1643d>:0
at (wrapper remoting-invoke-with-check) System.Threading.EventWaitHandle.Set()
at System.IOAsyncResult.set_IsCompleted (System.Boolean value) [0x0001e] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.SocketAsyncResult.Complete () [0x00031] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.SocketAsyncResult.Complete (System.Int32 total) [0x00007] in <99508e34019346e68bffacfcecbd997e>:0
at System.Net.Sockets.Socket+<>c.<.cctor>b__313_8 (System.IOAsyncResult ares) [0x00070] in <99508e34019346e68bffacfcecbd997e>:0
at System.IOSelectorJob.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00000] in <99508e34019346e68bffacfcecbd997e>:0
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <61d5ec988156494684532710ebf1643d>:0
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <61d5ec988156494684532710ebf1643d>:0
**** An error has occurred - please zip up the MXdiags folder and post it in the forum ****
Press Enter to terminate
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 11:07 am
by mcrossley
If you revert back to 3.6.0 does the crash still occur?
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 12:13 pm
by G6GUH
Had the same problem on two raspberry pi’s on different usb loggers, worked for about an hour or so then stopped. Reverted to previous version and all is ok.
Pete
www.cardingmillweather.co.uk
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 12:54 pm
by mcrossley
Are you both using FTP? If so which flavour (FTP, FTPS, SFTP)?
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 1:00 pm
by Matt.j5b
3.6.0 haven't crashed after running for 105 minutes and I never had this issue with 3.6.0 before I upgraded.
I am using plain FTP.
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 1:18 pm
by mcrossley
Thanks, looks like an issue with that FTP update, just letting my test env run with FTP going for an hour+ and see if I can reproduce it...
My 3.6.2 live install doesn't use FTP and its been running for 30 hours now.
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 1:29 pm
by mcrossley
Actually, if one of you is willing to test it, the FluentFTP.dll file from version 3.6.0 (version 32.3.1) *should* be compatible with CumulusMX 3.6.2
If that combination works then we know it is definitely the DLL update causing the problem.
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 1:41 pm
by Matt.j5b
Sure I now have that combination running at the moment. FTP is working correctly using the older FluentFTP.dll file.
Most of the crashes happened after around an hour, though sometimes it crashed after 10 or so minutes.
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 2:13 pm
by mcrossley
My test env has been running - using FTPS, and 1 minute uploads - for 75 minutes now, how I love it when you cannot reproduce issues
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 2:18 pm
by Matt.j5b
Not good, as that is pretty well the longest I could get it to run without crashing. Using the older dll with 3.6.2 after 40 minutes it is not crashing yet.
Re: v. 3.6.2 crashing
Posted: Wed 20 May 2020 2:26 pm
by mcrossley
What FTP upload interval are you using? Maybe 1 minute is not letting something get garbage collected.