Page 1 of 1
Problems with booting CumulusMX as a service
Posted: Sun 20 Mar 2022 9:02 pm
by f4phlyer
I originally posted this on the "running CumulusMX as a service but didn't receive any responses. Still dealing with the issue and just wondering if anyone has any ideas.
While rebuilding my rPi compute module 4 with Bullseye 64 bit I've encountered a problem on "reboot". I've encountered a problem that I can't find a solution to. I have CumulusMX running on a rPi 4 with Bullseye 64 bit set up exactly the same but runs without issue.
After reboot I check "sudo systemctl status cumulusmx" and MX starts up but at the end of the report:
● cumulusmx.service - CumulusMX service
Loaded: loaded (/etc/systemd/system/cumulusmx.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-03-10 17:28:30 CST; 42s ago
Docs:
https://cumuluswiki.org/a/Main_Page
Process: 381 ExecStart=/usr/bin/mono-service -d:/home/pi/CumulusMX CumulusMX.exe -service (code=exited, status=0/SUCCESS)
Main PID: 382 (mono)
Tasks: 11 (limit: 8987)
CPU: 5.568s
CGroup: /system.slice/cumulusmx.service
└─382 /usr/bin/mono /usr/lib/mono/4.5/mono-service.exe -d:/home/pi/CumulusMX CumulusMX.exe -service
Mar 10 17:28:29 rPi-computemod-4 systemd[1]: Starting CumulusMX service...
Mar 10 17:28:30 rPi-computemod-4 systemd[1]: Started CumulusMX service.
Mar 10 17:28:36 rPi-computemod-4 mono[382]:
CumulusMX.exe: Object reference not set to an instance of an object
When I ""sudo systemctl reboot cumulusmx" everything is fine.
Re: Problems with booting CumulusMX as a service
Posted: Sun 20 Mar 2022 11:13 pm
by jlmr731
You have it running there
● cumulusmx.service - CumulusMX service
Loaded: loaded (/etc/systemd/system/cumulusmx.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-03-10 17:28:30 CST; 42s ago
that last error you see is an error from mono, so is it recording data? so, to see if it is, open up the current mxdiag file and see if it updates. And if there are any errors within the file, post it here and Mark or others can help out
Re: Problems with booting CumulusMX as a service
Posted: Mon 21 Mar 2022 5:14 pm
by f4phlyer
My bad, I should have provided that, MX stops. attached diags:
20220321-093456.zip
Re: Problems with booting CumulusMX as a service
Posted: Mon 21 Mar 2022 11:27 pm
by f4phlyer
jlmr731 wrote: ↑Sun 20 Mar 2022 11:13 pm
You have it running there
● cumulusmx.service - CumulusMX service
Loaded: loaded (/etc/systemd/system/cumulusmx.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-03-10 17:28:30 CST; 42s ago
that last error you see is an error from mono, so is it recording data? so, to see if it is, open up the current mxdiag file and see if it updates. And if there are any errors within the file, post it here and Mark or others can help out
jlmr, good news, saw another post that Mark handled about
Cumulus stalls at startup, on version 3171. The discussion was similar to my issue in that it seemed to hang at the start with mono, Mark, Hans et. all added start up delay with ping. I added both and it cured my issue.
Thanks for the assistance.
Steve
Re: Problems with booting CumulusMX as a service
Posted: Tue 22 Mar 2022 1:36 am
by jlmr731
Good to hear startup delay works. Did see in your diag that the network is not fully up when MX starts you can see when it first checks for update github not found.
Re: Problems with booting CumulusMX as a service
Posted: Tue 22 Mar 2022 8:18 pm
by f4phlyer
No, I've not been very lucky looking at the logs, there's usually so much there and I don't have an IT background looking at logs, although I'm getting better at it. The cause and effect relationships don't pop out.
Again, thanks for the help.
Steve
Re: Problems with booting CumulusMX as a service
Posted: Wed 23 Mar 2022 7:23 pm
by BigOkie
f4phlyer wrote: ↑Sun 20 Mar 2022 9:02 pm
I originally posted this on the "running CumulusMX as a service but didn't receive any responses. Still dealing with the issue and just wondering if anyone has any ideas.
While rebuilding my rPi compute module 4 with Bullseye 64 bit I've encountered a problem on "reboot". I've encountered a problem that I can't find a solution to. I have CumulusMX running on a rPi 4 with Bullseye 64 bit set up exactly the same but runs without issue.
After reboot I check "sudo systemctl status cumulusmx" and MX starts up but at the end of the report:
● cumulusmx.service - CumulusMX service
Loaded: loaded (/etc/systemd/system/cumulusmx.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2022-03-10 17:28:30 CST; 42s ago
Docs:
https://cumuluswiki.org/a/Main_Page
Process: 381 ExecStart=/usr/bin/mono-service -d:/home/pi/CumulusMX CumulusMX.exe -service (code=exited, status=0/SUCCESS)
Main PID: 382 (mono)
Tasks: 11 (limit: 8987)
CPU: 5.568s
CGroup: /system.slice/cumulusmx.service
└─382 /usr/bin/mono /usr/lib/mono/4.5/mono-service.exe -d:/home/pi/CumulusMX CumulusMX.exe -service
Mar 10 17:28:29 rPi-computemod-4 systemd[1]: Starting CumulusMX service...
Mar 10 17:28:30 rPi-computemod-4 systemd[1]: Started CumulusMX service.
Mar 10 17:28:36 rPi-computemod-4 mono[382]:
CumulusMX.exe: Object reference not set to an instance of an object
When I ""sudo systemctl reboot cumulusmx" everything is fine.
You say 'When I ""sudo systemctl reboot cumulusmx" everything is fine'. Do you still have the Object reference error in the status call after you reboot it?
A lot of times when I see these one off errors it's typically a corrupted monthly log file. Not saying that's it, but in my experience (I've been running CMX now since 2016) that's almost always it.
In a lot of cases, the MXdiag log file will have entries like the following if that's the case.
Code: Select all
2021-12-15 12:05:59.728 Error parsing log file record: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
2021-12-15 12:05:59.728 LoadRecent: Error at line 4178 of data/Dec21log.txt : Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
Re: Problems with booting CumulusMX as a service
Posted: Thu 31 Mar 2022 2:23 am
by f4phlyer
You say 'When I ""sudo systemctl reboot cumulusmx" everything is fine'. Do you still have the Object reference error in the status call after you reboot it?
A lot of times when I see these one off errors it's typically a corrupted monthly log file. Not saying that's it, but in my experience (I've been running CMX now since 2016) that's almost always it.
In a lot of cases, the MXdiag log file will have entries like the following if that's the case.
Thanks for the comment, armed with your’s and ‘jamir 731’s comments. I’ll do a little more digging to see to what extant the problem is/was. The issue arose as I was rebuilding a rPi compute module on a rPi CM4 i/O board with 64 bit Raspian. Jeff Geerling has suggested that 32 bit Debian will be loosing support in the new 64 bit world so I thought I’d get start the transition.
Re: Problems with booting CumulusMX as a service
Posted: Fri 01 Apr 2022 2:12 am
by f4phlyer
I originally thought I had a problem with mono, it seems that 'mono' for Raspian is only 32 bit. I was transitioning to Raspian 64bit so was thinking that it was a mono problem.
More or less wrapping this up, looking a little deeper there may be something else but what stopped the startup of MX was it stopped when the ping request for the web site didn't respond prior to the minimum timeout. So setting the ping wait time to 10 solved it.