External Programs & Linux

[mcrossley]

I have a bash script that CumulusMX calls at the daily roll over to do various uploads and housekeeping. It can take some time to run, but seems to run OK normally.

Today I restarted my Pi after it hung yesterday (looks like a loose power connection), I noticed that the external program was called during the catchup daily roll over, but it seemed to stop the catchup process. The catchup proceeded to 00:20 and then silently CMX just stopped doing anything. It terminated with a Ctrl-C OK and on restart did the catchup from 00:20. ***

It seems CMX may run the external programs synchronously, so I wondered if I supply a parameter of "&" (my parameter field is currently blank) then the script may run in background and let Cumulus proceed?

Anyhow I've put that parameter in - I'll see if the script still runs OK tonight.

***It seems CMX builds a list of catch-up URLs for WU and WOW during the log processing and then plays them back at the end of the catch up. As my catchup ran in two segments only data from the second run was uploaded to WOW and WU. One to watch for in the future.

[steve]

It creates a separate process to run the external command, and doesn't wait for the command to terminate. What does the diags log look like immediately after the "Executing daily program" message? Perhaps there was some kind of system error which silently killed the logger download thread - similar to the problem Adrian reported recently? Anything in the system logs at that time?

[steve]

It's interesting that Adrian is also executing a command at rollover...

[mcrossley]

Steve

I can't see anything of interest in the diags file (attached).

All the system logs (syslog, kern) show that the system was rebooted 21 May @ 19:17. Which is weird as the diagnostic log shows MX stop logging at 19:16:57, then responding to my Ctrl-C at 19:22:57 and I don't recall the SSH session aborting! Maybe I am misinterpreting this?

Code: Select all

May 21 19:15:08 pi-live kernel: [ 2250.921965] usb 1-1.3.1.3: new full-speed USB device number 8 using dwc_otg
May 21 19:15:08 pi-live kernel: [ 2251.024650] usb 1-1.3.1.3: New USB device found, idVendor=067b, idProduct=2303
May 21 19:15:08 pi-live kernel: [ 2251.024692] usb 1-1.3.1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
May 21 19:15:08 pi-live kernel: [ 2251.024713] usb 1-1.3.1.3: Product: USB-Serial Controller
May 21 19:15:08 pi-live kernel: [ 2251.024731] usb 1-1.3.1.3: Manufacturer: Prolific Technology Inc.
May 21 19:15:08 pi-live kernel: [ 2251.029820] pl2303 1-1.3.1.3:1.0: pl2303 converter detected
May 21 19:15:08 pi-live kernel: [ 2251.036742] usb 1-1.3.1.3: pl2303 converter now attached to ttyUSB0
May 21 19:17:01 pi-live /USR/SBIN/CRON[3019]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
May 21 19:17:26 pi-live kernel: imklog 5.8.11, log source = /proc/kmsg started.
May 21 19:17:26 pi-live rsyslogd: [origin software="rsyslogd" swVersion="5.8.11" x-pid="2130" x-info="http://www.rsyslog.com"] start
May 21 19:17:26 pi-live kernel: [    0.000000] Booting Linux on physical CPU 0x0
May 21 19:17:26 pi-live kernel: [    0.000000] Initializing cgroup subsys cpu
May 21 19:17:26 pi-live kernel: [    0.000000] Initializing cgroup subsys cpuacct
May 21 19:17:26 pi-live kernel: [    0.000000] Linux version 3.18.13+ (dc4@dc4-XPS13-9333) (gcc version 4.8.3 20140303 (prerelease) (crosstool-NG linaro-1.13.1+bzr2650 - Linaro GCC 2014.03) ) #784 PREEMPT Sat May 9 15:49:42 BST 2015
May 21 19:17:26 pi-live kernel: [    0.000000] CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d
May 21 19:17:26 pi-live kernel: [    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
May 21 19:17:26 pi-live kernel: [    0.000000] Machine model: Raspberry Pi Model B
May 21 19:17:26 pi-live kernel: [    0.000000] cma: Reserved 8 MiB at 0x1b800000
May 21 19:17:26 pi-live kernel: [    0.000000] Memory policy: Data cache writeback

[steve]

Yes, it looks like the external command may be a red herring, it carried on for a short time after starting the process, but then just stopped. Perhaps it had problems reading the data from the station. The system log is odd - perhaps the messages are just delayed, and they are from booting the system a few minutes earlier? Do you have Cumulus set to start automatically - perhaps the problem is something to do with the system still starting up if so?

[mcrossley]

Cumulus wasn't set to autostart. I do have a script set to run on reboot to launch it but the actual command is commented out. I launched CMX from an SSH session.

I can't figure out the syslog either - though the Pi is complicatedhampered by not having a realtime clock, so time stamps could be unreliable until it it contacts an NTP server.