Monthly Reports are no longer updating

[gard72]

The last time Monthly Reports updated was 20-January. I have checked Dayfile and have data for 21 and 22, however it refuses to add onto the monthly report. When I attempt to re-generate report, errors occur. This installation has worked flawlessly until 20 January. Error log reports the following--each day at midnight and whenever I attempt a 're-generate':

2024-01-23 17:07:13 - Error at line 2952, column 1, value '12:00' of data/Jan24log.txt : System.FormatException: Input string was not in a correct format.
at System.Number.ThrowOverflowOrFormatException (System.Boolean overflow, System.String overflowResourceKey) [0x0001a] in <d636f104d58046fd9b195699bcb1a744>:0
at System.Number.ParseInt32 (System.ReadOnlySpan`1[T] value, System.Globalization.NumberStyles styles, System.Globalization.NumberFormatInfo info) [0x00016] in <d636f104d58046fd9b195699bcb1a744>:0
at System.Int32.Parse (System.String s, System.IFormatProvider provider) [0x00017] in <d636f104d58046fd9b195699bcb1a744>:0
at System.Convert.ToInt32 (System.String value) [0x0000b] in <d636f104d58046fd9b195699bcb1a744>:0
at CumulusMX.Utils.ddmmyyhhmmStrToDate (System.String d, System.String t) [0x00045] in <2254c0dcfbb14b4e97c3995e92c4164b>:0
at CumulusMX.NOAA.CreateMonthlyReport (System.DateTime thedate) [0x0088f] in <2254c0dcfbb14b4e97c3995e92c4164b>:0

[freddie]

Have you looked at line 2952 in your Jan24log.txt file to see if there is something wrong that you can see? That is what the error message is indicating.

[KenKenyon]

I've been struggling with NOAA monthly reports since I started on the raspberry pi journey. All was well for a day and then problems.
I've checked the MXDiags file and have pretty much the exact same problem as gard72. I've looked at line 1152 and can't see anything wrong (not that I know what I'm looking for - although I have checked puncutation against previous line and it's the same.
2024-01-26 00:08:45.814 Creating NOAA monthly report
2024-01-26 00:08:46.814 Error at line 1152, column 1, value '15:06' of data/Jan24log.txt : System.FormatException: Input string was not in a correct format.
at System.Number.ThrowOverflowOrFormatException (System.Boolean overflow, System.String overflowResourceKey) [0x0001a] in <de882a77e7c14f8ba5d298093dde82b2>:0
at System.Number.ParseInt32 (System.ReadOnlySpan`1[T] value, System.Globalization.NumberStyles styles, System.Globalization.NumberFormatInfo info) [0x00016] in <de882a77e7c14f8ba5d298093dde82b2>:0
at System.Int32.Parse (System.String s, System.IFormatProvider provider) [0x00017] in <de882a77e7c14f8ba5d298093dde82b2>:0
at System.Convert.ToInt32 (System.String value) [0x0000b] in <de882a77e7c14f8ba5d298093dde82b2>:0
at CumulusMX.Utils.ddmmyyhhmmStrToDate (System.String d, System.String t) [0x00046] in <b0ae9ad91692488da7ace3260fd3c598>:0
at CumulusMX.NOAA.CreateMonthlyReport (System.DateTime thedate) [0x00884] in <b0ae9ad91692488da7ace3260fd3c598>:0
2024-01-26 00:08:46.815 Please edit the file to correct the error
Here is line 1152
12/01/24,15:06,3.9,82,1.1,1,4,154,0.0,0.0,1034.9,0.0,19.9,56,1,3.9,3.9,0.0,18,0.00,0.00,1.7,27,0.0,219,0.0,0.0,3.9,3.9

[SamiS]

Could you post the problematic Jan24log.txt so we can take a look? One thing to check is that does the previous line have some error or hidden non-printable characters.

[KenKenyon]

Thanks for your offer to help SamiS. Here's the file.
Ken

[SamiS]

One thing that catches my eye is that on line 1151 you have data from 12/01/24,15:02, and the next line 1152 contains data from 12/01/24,15:06 so there is data missing. Why this results an error on NOAA Monthly report generation, I don't have a clue.

If you want to test my theory, you could make copies of the previous line and edit the clock time to fill the three minute gap and see if that fixes the issue on the report generation.

This is probably a bug that needs to be looked at by @mcrossley.

PS. Note for everyone, if you send zipped files, please use the very old and extremely compatible .zip format, not 7z or rar or anything else exotic that cannot be opened with mobile devices, tablets etc. Thanks.

Sami

[KenKenyon]

Thanks for that Sami. I will do what you suggest and report back with the result tomorrow.
Ken

[KenKenyon]

Hello Sami

Nine minutes past midnight and I've just checked my NOAA monthly report. At first I was disappointed that it was unchanged. I then clicked regenerate and as if by magic all the missing days appeared. Thank you so much.
Ken

[SamiS]

Excellent. Keep an eye on it for a while to see if the automatic generation resumes/continues now. Let's hope that Mark can find and fix the reason for this occurring.

@gard72 (who started this thread) could you also check if there is a gap on your data between the lines 2951-2952 or send the Jan24log.txt?

[mcrossley]

One thing that catches my eye is that on line 1151 you have data from 12/01/24,15:02, and the next line 1152 contains data from 12/01/24,15:06 so there is data missing. Why this results an error on NOAA Monthly report generation, I don't have a clue.

If you want to test my theory, you could make copies of the previous line and edit the clock time to fill the three minute gap and see if that fixes the issue on the report generation.

This is probably a bug that needs to be looked at by @mcrossley.

MX shouldn't care about the odd missing bit data - after all when it catches up, the archive logging interval is often different from the live logging interval.

I cannot see anything wrong at line 1152, but further down the file there is some corruption...
11617:

Code: Select all

[null][null].0,195,0.0,0.0,2.3,4.0

11623:

Code: Select all

20/01/24,08:10,0.0,1017.1,0.0,18.2,52,2,4.3,4.3,0.0,35,0.00,0.00,1.8,79,0.0,216,0.0,0.0,4.3,4.3
20/01/24,10:00,4.3,78,0.8,2,5,223,0.0,0.0,1017.0,0.0,18.3,52,2,4.3,4.3,0.0,35,0.00,0.00,1.8,80,0.0,193,0.0,0.0,4.3,4.3

13964:

Code: Select all

[null][null]7,20,247,0.0,0.0,1003.3,0.0,19.8,56,11,7.2,9.0,0.0,50,0.00,0.00,5.1,112,2.3,268,0.0,0.0,7.2,9.0

[SamiS]

MX shouldn't care about the odd missing bit data - after all when it catches up, the archive logging interval is often different from the live logging interval.

I cannot see anything wrong at line 1152, but further down the file there is some corruption...

Could there be some strange correlation regarding missing data and corrupted lines and the error triggering? After all, adding those missing lines resulted in successful report generation… It never even crossed my mind that the fault could be later in the data. But the catch-up and historical data resolution aspect was just the reason why I said I don’t have a clue why missing data would cause this.

[mcrossley]

It could be the line count is wrong. I'll check it, the routine will read the whole file, so it should still be erroring on those corrupt lines - I forget now, but iirc there is only one value in the report that needs the monthly log scan, most of it comes from the day file.