Page 1 of 1

DayFile - Annual Evapotranspiration bug

Posted: Fri 31 Jan 2014 3:29 pm
by TgT
Wiki says:
23 (X): Total evapotranspiration (Only valid for Davis stations, shows zero otherwise)

but on a new years day (1.1. aka 1st Jan) i get huge negative value, possible previous year sum?

yes, i just sum whole year ET and it matches a negative number on new years day.

Re: DayFile - Annual Evapotranspiration bug

Posted: Fri 31 Jan 2014 3:38 pm
by steve
The ET value provided to Cumulus by the console and/or DLL is buggy and unreliable. It often shows a negative value.

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 02 Jan 2018 5:08 pm
by Mapantz
I know this is a problem on the Davis side of things, but this hit me hard again.
I had a complete log full of this:

Code: Select all

2018-01-01 00:49:58.293 *** ET Reset
2018-01-01 00:50:00.298 *** ET Reset
But it didn't reset until midnight on the 1st of January, which meant that I had a whole day's worth of data logged, with a value that looked like this
-732.66 the value slowly decreased in size throughout yesterday, and then also logged it as -731.69 in yesterday dayfile.

In the end, I had to manually edit Jan18.txt, Dayfile.txt and all of the rogue values in sql - took me ages! :(

Does anyone have some kind of workaround for this? I know it's only once a year, but it happened the year before as well. My console date and time match my computer perfectly.

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 02 Jan 2018 5:58 pm
by mcrossley
I didn't have a great problem. It detected the negative values just after midnight, but they stopped after 40 seconds, and at 00:00:45 I got...

Code: Select all

2018-01-01 00:00:45.816 StartofdayET set to 0
And that was it.

I have never looked at the year rollover before - interesting to see the rain counter reset to zero - obvious that it is a year total but I hadn't thought about it before.

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 02 Jan 2018 6:15 pm
by Mapantz
mcrossley wrote:I didn't have a great problem. It detected the negative values just after midnight, but they stopped after 40 seconds, and at 00:00:45 I got...

Code: Select all

2018-01-01 00:00:45.816 StartofdayET set to 0
And that was it.

I have never looked at the year rollover before - interesting to see the rain counter reset to zero - obvious that it is a year total but I hadn't thought about it before.
I never got the 0, it just started as this:

Code: Select all

2018-01-01 00:00:01.070 StartofdayET set to 732.663
and then had these for an hour:

Code: Select all

2018-01-01 00:00:07.411 *** ET Reset
2018-01-01 01:00:00.279 *** ET Reset
It wasn't until Jan 2nd that it corrected itself:

Code: Select all

2018-01-02 00:00:01.477 StartofdayET set to 0.8636
I hadn't realised until then, so that's when I had to go through everything to remove the rogue values.

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 02 Jan 2018 6:21 pm
by mcrossley
Not a great deal of help, but you could add a pre-insert check on the SQL tables for negative values and set them to zero.

Re: DayFile - Annual Evapotranspiration bug

Posted: Thu 04 Jan 2018 1:03 pm
by BeaumarisWX
Hi Mapantz,
You are not on your own there, dito here.
To the point that after many years i have decided to ditch my "Monthly" table altogether.
Unlike many I pushed mine to the limit purposely at a one (1) minute Log time to see how far it would go.
I started recording in 2010 at (5) min intervals then over time changed to (1) that coupled with numerous Software and Hardware changes over the years resulted in my culling of all data pre 2013 last year in order to omit erroneous data from the display on my site.
However as of earlier today, my host server based MySql Monthly Table finally locked up for one reason or another, so I decided that was it.
I killed it and amended all scripts to not use it.
Pity on one front I suppose, but a major breath take on the other not having to be concerned about it any longer and too boot my envisaged thought that it may eventually fail due to size (whether it be in 1/5/10 whatever min intervals was like death, it is inevitable).
Regards,
Tony

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 12:16 am
by Mapantz
.. and again..

I dunno why mine never resets?

And of course, my diag file is filling up fast again.

I stopped CMX I quickly edited it out of today.ini and jan19.txt before it got out of hand.

Everything reset on the console though.

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 12:48 am
by mcrossley
Weird, mine reset after retrying for 34 seconds this year.

What console firmware version are you running. Mine is v3.12

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 1:19 am
by Mapantz
I'm on 3.15 mark.

I should've worded it better. It doesn't reset it to zero. It went to -801.8

2019-01-01 00:00:01.910 StartofdayET set to 801.7764

Then I had the log filling with

2019-01-01 00:00:08.931 *** ET Reset

Until I stopped CMX and changed it in the today.ini

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 8:56 am
by jdc
FWIW my MXdiags has 'ET Reset' up until 01:00 when 'hour changed' seems to clear it.

Code: Select all

2019-01-01 00:59:53.095 *** ET Reset
2019-01-01 00:59:55.220 *** ET Reset
2019-01-01 00:59:57.080 *** ET Reset
2019-01-01 01:00:00.345 Hour changed:1
2019-01-01 01:00:00.345 Calculating sunrise and sunset times
2019-01-01 01:00:00.345 Sunrise: 08:37:47
2019-01-01 01:00:00.345 Sunset : 15:45:39

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 10:34 am
by mcrossley
I'll take a look at how the code is handling this and see if there is anything I can do. The problem will be reproducing it, I don't really want to mess up my live data by resetting the clock for an extended period to build up some et for a year rollover.

Some logs with debug and data logging on over the rollover period may be a help, but again too late for this year ☹️

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 1:46 pm
by mcrossley
Tina, could you send me your diag file for the full rollover period please. Plus the sections of data logs file covering the same period.

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 4:03 pm
by mcrossley
OK, after much stepping through the calculations in various scenarios in Excel! I can see why you are having the issue, and why it stops at the first daily rollover. I'll put a fix in the next release.

The *think* reason some people see the issue and others don't is down to a timing thing. I saw the "*** ET Rest" messages between midnight and rollover processing being done. So if there are loop packets coming in over the 00:00 period and MX sees the ET reset before the rollover processing and the counters get reset correctly. If it sees ET reset after the rollover processing the counters are not reset correctly until the next rollover.
I'd need to see some diag files to confirm this though.

I'll add the change and amend it slightly so it detects not just a reset to zero, but a reset to any value lower than current - just in case that can occur as well.

Re: DayFile - Annual Evapotranspiration bug

Posted: Tue 01 Jan 2019 9:20 pm
by Mapantz
Apologies Mark. I missed it.

Interesting. I figured it might have been a bug with the 3.15 firmware due to the lack of reports.