Cumulus stalls at start-up

[SUBYDAZZ]

Cumulus has started hanging on startup for me too recently, I intially htough it was just my ancient, slow PC I was running it on but have since installed on a new system and the problem remains. In the Cumulus window it seems to be working fine until the point just after it says:

"Start reading history data...
dd/mm/yyyy xx:xx:xx AM/PM
Type Ctrl-C to terminate"

It then just appears to hang indefinitely, the only thing I can do to kick it into gear is to attempt to shut it down by pressing Ctrl+C, but then it seems to wake up and you get the:

"Connecting real time FTP
Normal Running"

I have set Cumulus to auto-start when my PC loses power or restarts, but it seems I cannot rely on it to do this any more as I have to interact with it to start properly. Any ideas? Not sure what logs to turn on and upload.

[flort]

Go into Program Settings and turn on "Data logging" under the Logging Options. Restart Cumulus and leave it sit at the start-up for a few minutes, then do your CTRL-C and post the MXDiags files here (upload both the ServiceConsoleLog.txt and the latest log file). Hopefully someone will be able to see what's happening.

[SUBYDAZZ]

Of course, now it's made a liar out of me, I let it go for 10 minutes or so and eventually it started by itself without CTRL+C-ing.

Here the recent MXDIAGS folder contents.

I will see if I can replicate the problem I was having and then re-upload. Please let me know if you find anything in there.

[SUBYDAZZ]

How long should it typically take to read the archive date and history data from the station on startup? Does it depend on PC resources much or more on frequency of logging? I rarely have it disconnected for very long but have noticed it can take a looooong time in the past to download a couple days worth.

[PaulMy]

Hi,
There is an error in your Feb log file that you should fix.

Code: Select all

2022-03-03 12:12:33.918 WMR200 station found
2022-03-03 12:12:34.127 Stream opened
2022-03-03 12:12:34.145 Loading last N hour data from data logs: 3/03/2022 12:09:00 PM
2022-03-03 12:12:34.150 LoadRecent: Attempting to load 7 days of entries to recent data list
2022-03-03 12:12:37.130 Error parsing log file record: Input string was not in a correct format.
2022-03-03 12:12:37.130 Log record:                                                                                                                                 27/02/22,22:54,20.3,97,19.8,8,9,154,0.0,0.0,1018.0,2882.6,25.9,46,9,20.3,20.3,0.0,0,0.00,0.00,22.4,0,0.0,157,0.0,0.0,21.7,27.5
2022-03-03 12:12:37.130 LoadRecent: Error at line 26044 of data\Feb22log.txt : Input string was not in a correct format.
2022-03-03 12:12:37.130 Please edit the file to correct the error

How long should it typically take to read the archive date and history data from the station on startup? Does it depend on PC resources much or more on frequency of logging? I rarely have it disconnected for very long but have noticed it can take a looooong time in the past to download a couple days worth.

Do you have the station logger interval and CumulusMX set at the same interval? I am not familiar with the WMR200 or its logging, but it is recommended that these are set to the same interval to avoid Cumulus having to download the full logger at startup.

Enjoy,
Paul

[flort]

Hi Mark,

I've still had no joy with Build 3170. The system is still hanging at the ping response. In fact, it is worse as before I could just restart the service and Cumulus would kick into action but now it won't even do that. I have to manually edit Cumulus.ini to remove the ping address to get it going.

I have attached two MXDiag files with data logging turned on. The first one shows it successfully got a ping reply but then it errored and that's where Cumulus stalled -

2022-03-10 10:01:19.268 Waiting for PING reply from 185.27.134.3
2022-03-10 10:01:19.718 Ping reply: Success
2022-03-10 10:01:21.860 Ping Error: The PING failed to return after the timeout, cancelling it...

The second one only got as far as waiting for the ping and then stalled.

Regards,
Trevor

[HansR]

I have similar problems but now I can't start CMX at all i.e. it not only does not start at a reboot but when I give the start manually from the commandline CMX does not start either:

Code: Select all

sudo systemctl start cumulusmx

does not work.

The logfiles are attached: first series is the reboot, second series is the manual restart.
These are my settings:

Code: Select all

[Program]
EnableAccessibility=0
StartupPingHost=meteo-wagenborgen.nl
StartupPingEscapeTime=3
StartupDelaySecs=0
StartupDelayMaxUptime=300

With the second attempt I noticed the following lines in htop (not changing):

Schermafbeelding 2022-03-10 091715.jpg

I am pretty sure the ping site is up and after 30 minutes these lines are still there. Nothing happens.

[HansR]

If I start CMX in the command window (not as a service) I get the next lines in the log:

Code: Select all

2022-03-10 09:40:29.110  ========================== Cumulus MX starting ==========================
2022-03-10 09:40:29.116 Command line: /home/CumulusMX/CumulusMX.exe 
2022-03-10 09:40:29.116 Cumulus MX v.3.15.1 build 3170
2022-03-10 09:40:29.126 Platform: Unix
2022-03-10 09:40:29.127 OS version: Unix 5.10.103.7
2022-03-10 09:40:29.127 Current culture: Dutch (Netherlands) [nl-NL]
2022-03-10 09:40:29.128 Mono version: 6.12.0.122 (tarball Mon Feb 22 17:40:01 UTC 2021)
2022-03-10 09:40:29.129 Checking required folders
2022-03-10 09:40:29.255 Reading Cumulus.ini file
2022-03-10 09:40:29.280 AvgSpdMins=10 AvgSpdTime=00:10:00
2022-03-10 09:40:29.282 Cumulus start date: dinsdag 11 juni 2019
2022-03-10 09:40:29.352 Reading Cumulus.ini file completed
2022-03-10 09:40:29.353 Directory separator=[/] Decimal separator=[,] List separator=[;]
2022-03-10 09:40:29.353 Date separator=[-] Time separator=[:]
2022-03-10 09:40:29.353 Standard time zone name:   CET
2022-03-10 09:40:29.353 Daylight saving time name: CEST
2022-03-10 09:40:29.357 Daylight saving time? False
2022-03-10 09:40:29.357 10-3-2022 09:40:29
2022-03-10 09:40:29.357 Stop second instance: No other running instances of Cumulus found
2022-03-10 09:40:29.357 No start-up delay - disabled
2022-03-10 09:40:29.358 Waiting for PING reply from meteo-wagenborgen.nl
2022-03-10 09:40:29.567 Ping reply: Success
2022-03-10 09:40:31.986 Ping Error: The PING failed to return after the timeout, cancelling it...
!!! Unhandled Exception !!!
System.NullReferenceException: Object reference not set to an instance of an object.
  at CumulusMX.Cumulus..ctor (System.Int32 HTTPport, System.Boolean DebugEnabled, System.String startParms) [0x01eb7] in <c346ebaa802445a8b02adfbfafb50be2>:0 
  at CumulusMX.Program.RunAsAConsole (System.Int32 port, System.Boolean debug) [0x00048] in <c346ebaa802445a8b02adfbfafb50be2>:0 
  at CumulusMX.Program.Main (System.String[] args) [0x0056b] in <c346ebaa802445a8b02adfbfafb50be2>:0 

[HansR]

If I start CMX in the command window (not as a service) I get the next lines in the log:

Code: Select all

2022-03-10 09:40:29.110  ========================== Cumulus MX starting ==========================
2022-03-10 09:40:29.116 Command line: /home/CumulusMX/CumulusMX.exe 
2022-03-10 09:40:29.116 Cumulus MX v.3.15.1 build 3170
2022-03-10 09:40:29.126 Platform: Unix
2022-03-10 09:40:29.127 OS version: Unix 5.10.103.7
2022-03-10 09:40:29.127 Current culture: Dutch (Netherlands) [nl-NL]
2022-03-10 09:40:29.128 Mono version: 6.12.0.122 (tarball Mon Feb 22 17:40:01 UTC 2021)
2022-03-10 09:40:29.129 Checking required folders
2022-03-10 09:40:29.255 Reading Cumulus.ini file
2022-03-10 09:40:29.280 AvgSpdMins=10 AvgSpdTime=00:10:00
2022-03-10 09:40:29.282 Cumulus start date: dinsdag 11 juni 2019
2022-03-10 09:40:29.352 Reading Cumulus.ini file completed
2022-03-10 09:40:29.353 Directory separator=[/] Decimal separator=[,] List separator=[;]
2022-03-10 09:40:29.353 Date separator=[-] Time separator=[:]
2022-03-10 09:40:29.353 Standard time zone name:   CET
2022-03-10 09:40:29.353 Daylight saving time name: CEST
2022-03-10 09:40:29.357 Daylight saving time? False
2022-03-10 09:40:29.357 10-3-2022 09:40:29
2022-03-10 09:40:29.357 Stop second instance: No other running instances of Cumulus found
2022-03-10 09:40:29.357 No start-up delay - disabled
2022-03-10 09:40:29.358 Waiting for PING reply from meteo-wagenborgen.nl
2022-03-10 09:40:29.567 Ping reply: Success
2022-03-10 09:40:31.986 Ping Error: The PING failed to return after the timeout, cancelling it...
!!! Unhandled Exception !!!
System.NullReferenceException: Object reference not set to an instance of an object.
  at CumulusMX.Cumulus..ctor (System.Int32 HTTPport, System.Boolean DebugEnabled, System.String startParms) [0x01eb7] in <c346ebaa802445a8b02adfbfafb50be2>:0 
  at CumulusMX.Program.RunAsAConsole (System.Int32 port, System.Boolean debug) [0x00048] in <c346ebaa802445a8b02adfbfafb50be2>:0 
  at CumulusMX.Program.Main (System.String[] args) [0x0056b] in <c346ebaa802445a8b02adfbfafb50be2>:0 

[EDIT:] Rollback to 3169 and everything is OK (same settings).

[mcrossley]

Hi Hans, thanks for the report, I'm struggling to see what object is null at moment, and I cannot reproduce the problem that the ping does not return. For me it always either: works, times out, or returns an error like DNS host not found.

[flort]

Hi Hans,

The only way I could get CMX to start again was to edit Cumulus.ini through the command terminal and remove the StartupPingHost entry. I tried several ways to edit Cumulus.ini but each method told me the file was in use or permission denied when I tried to save it. I ended up using Nano as superuser and it let me save the changes. After that Cumulus started okay.

Regards,
Trevor

[HansR]

Hi Hans, thanks for the report, I'm struggling to see what object is null at moment, and I cannot reproduce the problem that the ping does not return. For me it always either: works, times out, or returns an error like DNS host not found.

OK. Let me know if I need to experiment something. I haven't got a clue either atm.
And NOTE: before update and restart I updated the system/Linux.
And NOTE: system is Bullseye

The only way I could get CMX to start again was to edit Cumulus.ini through the command terminal and remove the StartupPingHost entry. I tried several ways to edit Cumulus.ini but each method told me the file was in use or permission denied when I tried to save it. I ended up using Nano as superuser and it let me save the changes. After that Cumulus started okay.

Yes, that will work no doubt but I prefer to leave it as is and experiment when necessary.
Reverting to 3169 is simpler

[mcrossley]

Test build sent to you Hans.

[HansR]

Test build sent to you Hans.

Tried and works fine:

Code: Select all

2022-03-10 15:15:04.290 No start-up delay - disabled
2022-03-10 15:15:04.292 Waiting for PING reply from meteo-wagenborgen.nl
2022-03-10 15:15:04.637 Ping reply: Success
2022-03-10 15:15:04.990 Received PING response from meteo-wagenborgen.nl, continuing...
2022-03-10 15:15:05.001 Data path = data/

[AndyKF650]

Hi there

I just saw this thread and would let you know that I loaded build 3170 first thing this morning with no issues at all.

I am running Davis VP2 plus through meteo pi to a RPi 4. I went through my usual upgrade process of stopping CMX and copy entire folder to a spare drive as a backup, download new version, extract and install on Pi, copy following files from backup ie data, reports, utils, web, web files, CMX .ini and CUtils.ini and paste onto the new CMX version. All checked out CMX admin OK and CUtils website OK.

The system is working just fine as expected.