Cumulus Support

As I understand it the daily rain resets at midnight however we have only 1 minute logging so the last entry for the end of day is 23:59 but that is 60 seconds before midnight and if it is raining still there could be several updates to rain prior to 23:59:59, but it resets at 00:00:00 so what happens to those updates? They should not be recorded in the 00:00 record but as part of the 23:59 record. Where is this rain recorded?

Stuart

If you measure for a minute and then write it out, the written value is always for the past minute.
So why would you worry for that amount which is written at 00:00? As long as it is done consistently there should be no problem. All time boundaries wrt the weather are kind of artificial aren't they? And the amount of minutes written is always 60 because you start at 0. I do not see the issue. If you attribute the amount of midnight to the previous minute you would assign a 2 minute value to 23:59. That would start me questioning.

Yes in a way you are correct so in that case there is an argument for a current day 00:00 record and a new day 00:00 record with the new day record having always 0 rain.

In the case of an archive catch up where there are only 5 minute records we have 23:55 and 00:00. Now we are downloading from Ecowitt and in the case of a midnight reset the archive 00:00 value will be zero and in those 5 minutes we could have a significant amount of rain - where does it go?

Stuart

broadstairs wrote: ↑Sat 10 Sep 2022 8:46 am Yes in a way you are correct so in that case there is an argument for a current day 00:00 record and a new day 00:00 record with the new day record having always 0 rain.

But that would still mean you count the last minute double and you would have 61 minutes in the hour or a double registration for rain in that last minute. That would give it is bias over all other registered minutes.

broadstairs wrote: ↑Sat 10 Sep 2022 8:46 am In the case of an archive catch up where there are only 5 minute records we have 23:55 and 00:00. Now we are downloading from Ecowitt and in the case of a midnight reset the archive 00:00 value will be zero and in those 5 minutes we could have a significant amount of rain - where does it go?

Well, as Mark has often said, the rain is quite complex in CMX so I am not going to tell you how everything works simply because I am not sure or don't know.

I think you are right that a drop fallen just before midnight will be registered at 00:00 which is the formally the first timestamp of the new day. However, if that is done consistently I don't see a problem. I would have a problem with attributing the value of those few drops to the previous sampling interval giving it a systematic statistical bias. That would raise the number of intervals for a day by one (interval being 1, 5, 10, 15 or 30 minutes I guess).

You make life very confusing

Having a consistent registration - of all weather parameters - is much more importent than to have it registered just before or after a time limit which is artificial by definition and therefore kind of irrelevant: I want to know how much rain fell if it fell 1 minute before or 1 minute after midnight would that change the state of the world we're in (it reminds me kind of Shakespeare with his 'what's in a name').

I think the danger here is that we could lose the accuracy of rain recording. In many ways we should not have a time stamp of 00:00 as it is not recognised as an official time in the 24 hors clock, the last should be 23:59 and the first after midnight is 00:01 and the 00:00 reset potentially deletes any rain falling in the last minute, it would in my view make more sense if we recorded everything in the last minute up to midnight in the 23:59 record and then any rain falling in the first minute after midnight in the 00:01 record.

Yes I know it is complicated but it is also important for data to be as accurate as possible.

Stuart

broadstairs wrote: ↑Sat 10 Sep 2022 10:28 am I think the danger here is that we could lose the accuracy of rain recording.

Accuracy of rain recording has nothing to do with attributing the amount measured to a time class.

broadstairs wrote: ↑Sat 10 Sep 2022 10:28 am In many ways we should not have a time stamp of 00:00 as it is not recognised as an official time in the 24 hors clock, the last should be 23:59 and the first after midnight is 00:01 and the 00:00 reset potentially deletes any rain falling in the last minute, it would in my view make more sense if we recorded everything in the last minute up to midnight in the 23:59 record and then any rain falling in the first minute after midnight in the 00:01 record.

Midnight (start of day) timestamp is by definition 00h00.

broadstairs wrote: ↑Sat 10 Sep 2022 10:28 amYes I know it is complicated but it is also important for data to be as accurate as possible.

Again: accuracy (of measurement) has nothing to do with attributing an amount to a period class (because that is actually what we are doing when registering weather parameters).

I will step aside now from this discussion to let others shed their light on it

[Edit:]I changed some details to make the text more clear (I hope)

Early in the C1 releases, the 00:00 (or 09:00, but I'll ignore that for simplicity) entry did contain the rainfall value up to midnight the previous days. That changed to resetting to zero.
I do not think there is a any good answer to this, recording the previous days total in the next days 00:00 entry makes things simpler in in some ways, but more complex in others.

To answer your question "what happens to the rainfall in that period up to midnight?". The answer is - it is recorded in the dayfile. The dayfile contains to total at the time of the rollover.

The CreateMissing utility when creating a dayfile record uses either the 00:00 value of the next day (if it is a very old record with the daily total), or the last logged value before midnight, plus the difference in the rainfall counter from that record to the 00:00 record.

Mark thanks for the clarification. I think the archive being loaded from Ecowitt who only record at 5 minute intervals potentially means we 'could' lose rain data as the gateway resets at 00:00.

Stuart

No, because CMX uses the yearly rainfall value, so it only has the potential to lose 5 minutes of rainfall per year when the yearly total resets at 00:00 on the 1st Jan