philpugh wrote: ↑Sat 13 Aug 2022 3:40 pm
I would be worth looking at your processing for the calculation as it was only a missing record for 1/1/2018 in the dayfile.txt that was the issue. I can't explain why it was missing but the Jan18log.txt was complete and so is my Mariadb for that day. Why it would add the previous year to the total must have something to do with it being the first day of the year?
[EDIT: 14/8] corrected some formulation.
OK, this took me some time because I started looking in the code determining the records (which I had never looked at again after creation). That's not where it is. When reading the dayfile I create a list of a structure of values with an entry for every day but while doing so I add some extra fields in the structure for
WetPeriod, DryPeriod, MonthlyRain and YearToDateRain for which the value is determined in some additional processing.
For
YearToDateRain that means CUtils simply adds the previous value in the list to the current value and stores it in the current value.
You guessed it: the previous value in your case that was the previous year total and because it missed the first of the month and the first of the year it did not reset the month total nor the years total. Then it simply starts adding the daily values for that month until it encounters a first day of the next month.
Now the question is do I regard this as a bug or as a data failure?
I see it as data failure: garbage in is garbage out; dayfile records need to be contiguous and if they are not, bad luck. It will correct after that month or after that year (in case the 1 jan is missing).
Any missing record will have similar processing but on the first day of the year unfortunately the resulting error is quite large because the previous value is the years total. Thanks for noticing
Any error of this kind will correct automatically, the user has the responsibility for the data.