NOAA reports not correct

[HansR]

I thought I had send this but can't find the message back.
The NOAA reports apparently have the wrong average windspeed (I noticed when dashes started to be displayed).
Not really sure when that started, it seems to be wrong in every report but recently I did a recalculation for all reports, so difficult to say which version it went wrong.
Below are the reports as done by CMX (for august to date) and the coloured one are the calculated dayfile data by CumulusUtils. But when other tools are installed the same issue should be observed.

Aantekening 2020-08-15 104445.png

Aantekening 2020-08-15 105710.png

[HansR]

I noticed it has been reported in another thread as well.
I suggest you combine both. I don't think MXdiags is needed if you think otherwise, let me know.

[mcrossley]

Cross-posted...

Hans, could you switch on debug logging, regenerate a report, then post the logfile please - needless to say, I am not seeing the same problem, but another user reported exactly the same issue.

I'm doubting the log will show anything much as there is precious little logging in those routines.

[HansR]

There is something wrong: see image.
I attached a partial MXdiags log: from update of inifile to update inifile I activated debug logging and tried to recreate the august report.

If there is more you require, let me know.

Aantekening 2020-08-17 184654.png

[mcrossley]

Have you updated to v3.8.1? That is one of the fixes.

[mcrossley]

Also I am thinking the problem is probably in your monthly log file since it only shows on the days since you updated to v3.8.0

Could you post your August log file please.

[HansR]

The August 2020 file is attached.

[mcrossley]

There is a corrupt line in the log file at line 16677

[HansR]

Damn, I should have found that myself. Sorry for that.

So the question is where did that come from?

The time was around the update to 3.8.0 and most likely has to do with the regular stop for update and the restart as a service.

I will try some restarts/reboots again.

I assume that if I give a reboot (also from crontab sometimes), the service automatically gets a true stop command by the system. I did not check that actually but maybe a pause after the command might be required to give CMX time for a graceful exit? So far, between that moment and today, I did do some additional stops/starts through the commandline. That did not cause any problems.

BUT: I repaired the problem and I still can't read the report, still get the error so I can't regenerate and I can't load the year-reports either. I checked all datafiles (using "cumulusutils checkall") and found no further errors.

[mcrossley]

Damn, I should have found that myself. Sorry for that.

No problem, I didn't see it for a while.

So the question is where did that come from?

The time was around the update to 3.8.0 and most likely has to do with the regular stop for update and the restart as a service.

It does look like that - I still haven't got to the bottom of why this happens

I assume that if I give a reboot (also from crontab sometimes), the service automatically gets a true stop command by the system. I did not check that actually but maybe a pause after the command might be required to give CMX time for a graceful exit? So far, between that moment and today, I did do some additional stops/starts through the commandline. That did not cause any problems.

A term signal should cause a graceful shutdown - there should be evidence of that at the end of the log file.

BUT: I repaired the problem and I still can't read the report, still get the error so I can't regenerate and I can't load the year-reports either. I checked all datafiles (using "cumulusutils checkall") and found no further errors.

Have you updated to v3.8.1? Its a known fault in v3.8.0 when running as a service.

[HansR]

Here are two MXdiags: one shutdown with a command and the last with a reboot. Both have the same exception but seem to shutdown OK. I don't see any anomalies in the datafiles (dayfile and monthly) anymore.

I'll do the update later today and report back.

[mcrossley]

That exception should be fixed in 3.8.1 as well

[HansR]

OK, updated to 3.8.1 and I have the following observations:

1) The NOAA is fixed. However, the FTP now apparently does not send the NOAA report(s) Must that be done manually?
2) The exception at the end of the log indeed has disappeared. It all looks now more neat
2) I found the following things I never had before. Not a big issue, I'll remove the zero line in the logfile and just forget about those 4 minutes. Just for info. Logfiles attached.

1. First start of 3.8.1 it missed 4 minutes during the catching up. (see image 130212 and logfile of 125555)
   Aantekening 2020-08-18 130212.png
2. So, I tried that a second time (stop and start) and got the following (see image 132034 and logfiles 130758 (for the shutdown) and 131851 (for the startup).
   This resulted in the first catchup minute (13h10) with mostly zero's (temperature etc...)
   Aantekening 2020-08-18 132034.png

[mcrossley]

1) The NOAA is fixed. However, the FTP now apparently does not send the NOAA report(s) Must that be done manually?

Regen just recreates the report files. No FTP.

2) The exception at the end of the log indeed has disappeared. It all looks now more neat

Good

1. First start of 3.8.1 it missed 4 minutes during the catching up. (see image 130212 and logfile of 125555)

CMX set the console time at the start (12:56), it looks like it may have been a few minutes slow, because at 12:59 the logger only contains records for...
...
12:51
12:52
12:58 (last entry)

What I will do is add a log entry of the console time before the clock set (it already logs the console time afterwards) so we can see if it is adjusted significantly.

[*]So, I tried that a second time (stop and start) and got the following (see image 132034 and logfiles 130758 (for the shutdown) and 131851 (for the startup).
This resulted in the first catchup minute (13h10) with mostly zero's (temperature etc...)

Aantekening 2020-08-18 132034.png[/list]

No idea what happened there, data logging was not on, so the dump of the archive record that it used to create that record is not in the log file.