Cumulus stalls at start-up

[mcrossley]

A fix for this issue is now released - it helps when you don't get your logic comparisons reversed!

[AndyKF650]

New upgrade installed no problem all functioning as expected

[HansR]

A fix for this issue is now released - it helps when you don't get your logic comparisons reversed!

Flawless fix
Not the first, not the last. Sometimes we need to prove we're human.

[sfws]

Flawless fix :D

Wrong.

A fix for this issue is now released - it helps when you don't get your logic comparisons reversed! :bash:

I'm no expert on C#, but my brain can cope with logic, so I understand the do (sleep) while (condition) concept, the first rule of programming is to have a guaranteed loop exit (in your case by using a time-out) and build 3171 has wrong logic for that.

Builds 3107 to 3169 logic: Continue loop if time now less than what was calculated as time to time-out;
Build 3171 logic: Continue loop forever if current time exceeds what was calculated as time to time-out.

Luckily the actual ping reply condition has been corrected in b.3171; to continue loop while ping reply is null (meaning code can exit from loop once ping reply successfully received).

In builds 3107 to 3169, the loop included a 50 millisecond sleep time and that loop was done if there was a successful ping reply AND the current time was less than time calculated for end of ping check time, so 3169 worked for HansR because it exited loop when time was exceeded. The logic for the ping reply condition was wrong (should be do loop if no ping reply), but the "AND" and less than of second condition was okay.

Back on 10 February in your GitHub commit, you left that reply condition wrong and instead changed the AND into a OR. That stayed in loop for either successful ping reply or time less for end of check. The first condition remained wrong, the rest was correct.

In March you increased the sleep time to 500 ms and reverted the OR back to AND ready for public release 3170.
This failed for anybody requesting a ping, and successfully getting a successful ping reply, within the timeout period, as HansR found.

For 3171 you have finally got the first condition right (continue loop if no ping response). However, you have reversed the linking logic (AND gone to OR), and reversed the condition that follows (so continues loop if current time is after the ping time-out). This logic is wrong for handling time-out. It works for HansR because he gets the ping reply before the time-out is reached.

[mcrossley]

That is what happens when you squeeze a fix into the working day! I'll fix the fix tomorrow.

[mcrossley]

OK, I have fixed this, but the fix will wait until the next routine release as the code error will only affect a the edge case where the ping hangs.

[flort]

Hi Mark,

I have upgraded to Build 3173 but, unfortunately, it has not resolved the issue. If I restart Cumulus with an active internet connection then the ping works flawlessly but if network is not available at the time then it hangs and doesn't go any further no matter how long I leave it.

These are my Program Start-up Settings:

Start-up.jpg

I have an 8 minute ping escape time because it takes around 4 minutes for my router to come online. I left it for 10 minutes before manually restarting Cumulus. The MXDiag file is attached. This was with debug turned on so it appears that if the first ping is unsuccessful then the program hangs.

Regards,
Trevor

[mcrossley]

Hi Trevor,

It appears to be hanging in the Mono stack and not returning control to Cumulus

Have you updated your service file to include the new line...

Code: Select all

Wants=network-online.target

You will need to refresh the daemon afterwards with...
> sudo systemctl daemon-reload

Also you could add a start-up delay to allow the network more time to come up.

[flort]

Hi Mark,

You missed putting in what the new line was for the service file but this is my complete file contents -

Code: Select all

[Unit]
Description=CumulusMX service
Documentation=https://cumuluswiki.org/a/Main_Page
Requires= time-sync.target local-fs.target
After=network-online.target
Wants=network-online.target

[Service]
User=root
Group=root
ExecStart=/usr/bin/mono-service -d:/home/pi/CumulusMX CumulusMX.exe -service -lang en-AU
Type=forking
ExecStopPost=/bin/rm /tmp/CumulusMX.exe.lock

[Install]
WantedBy=multi-user.target

I have been using a start-up delay as a workaround but it would be good to get it working as it's intended to.

Regards,
Trevor

[mcrossley]

That file looks OK.

Unfortunately the problem seems to be in Mono, when I make the ping request, control never comes back to MX - even though it is an asynchronous request.

The only step left to me is to try running the ping code in a completely separate thread and see/hope if the main thread remains responsive. Would you be willing to run a test version of MX to try this?

[flort]

Hi Mark,

Happy to run a test version for you.

Regards,
Trevor

[f4phlyer]

I had this issue and posted a new thread before I found this. I was in the process of bringing my new Compute Module 4 on the CM4 I/O board on line. I wanted to use the new Raspian Bullseye 64 bit O/S.

As I researched the issue I discovered that only the 32 bit version of Mono is in the Raspian package operating in “degraded” mode.

There used to be a thread about. “Mono” but I couldn’t find it so FWIW: I tried a couple of things and found that it was possible to use the “Debian” 64 bit Buster install of Mono from the “Mono project” site https://www.mono-project.com/download/ ... lin-ubuntu.

Alas, this setup does not remedy the CumulusMX stall problem but I’ve not encountered any other problems with this setup on either the CM4 setup or a similarly set up rPi-4 b.

[flort]

Mark has created a fix to the stalling issue and I have successfully tested it. This will be included in the next release.

Essentially, it will continue to ping the Host until it is successful or the ping escape time is reached.

Regards,
Trevor