[IMPLMENTED] Heat sum & growing season

[HansR]

@beteljuice: actually aren't all charts timeseries?
Only this one is per year.

[HansR]

And OK, sorry, my site had a typo/coding error so did not work properly
Now it does

[beteljuice]

... the difference between the integral average and the estimator can be quite large (up to two degrees) so if you can avoid it don't use it.

I looked at several definitions and they all use (Max + Min) / 2
Remember this is a 'reference' and it would have been created long before mean samples were easy to do. If you change the basis of the formula you can't compare with look-up tables or other(s) data.

[HansR]

@beteljuice:

I looked at several definitions and they all use (Max + Min) / 2

That is because the UK did not take the leap into the future and remains with ancient estimators
Believe me, if you automate things, discrete integration is right at your fingertips.

I don't know where you are looking ('several definitions' might have reference?) but the average CMX stores in dayfile is as far as I can read from the code the sum of all observations divided by the number of observations. That is a far better estimator than (max+min)/2. And that is what the graph 'clash of averages is about.

Useless to discuss that again.

Remember this is a 'reference' and it would have been created long before mean samples were easy to do.

It is also a Wiki, so maybe it should be noted that it is ancient.
We don't calculate pi by approximation of the circle by a polygon.

[beteljuice]

Hans .. stop prevaricating

I gave you the wiki reference. I also consulted several agriculture sites (which further broke down the GDD required for each phase of any plants growth).

The point is not that we can do averages a different way, but that all reference data uses a particular way.

[HansR]

Hans .. stop prevaricating

Had to look that one up. I disagree there as well, that is not what I am doing.

And on topic: why talk about estimators of the past if even CMX itself calculates the day average in a more modern way, integral if you wish? I really don't understand why we should discuss (min+max)/2 to be taken as reference. It is an estimator and as such it is out of date.

[beteljuice]

Had to look that one up.

Good

I disagree there as well, that is not what I am doing.

Quite correct, it was misapplied
... but that means you accepted the definition in order to draw a conclusion

... so why do you want to re-define GDD in such a way as all reference tables would have to be re-written ?

[sfws]

As I'm sure Freddie will confirm ...

I looked at several definitions and they all use (Max + Min) / 2

That remains the way the WMO defines average temperature, and therefore the way that all international comparisons are made.

It makes sense as, not all weather is recorded by automatic sampling, some is still recorded by observers at fixed times.

The WMO permits individual nations to define average temperature (for use within nation) by integration of temperatures, but I doubt if any plants can do mathematical integration!

[HansR]

... but that means you accepted the definition in order to draw a conclusion

... so why do you want to re-define GDD in such a way as all reference tables would have to be re-written ?

Sorry I really do not understand this line of argumentation. The definition of the GDD is given as the integral of the temperature deviation of the daily average Td relative to some temperature Tbase. Then they come up with

A simpler, approximately equivalent formulation

This apparently leads you to interpret this as a definition rather than an estimator of the integral. And that is wrong.

I simply take another estimator of Td namely the the sampling method used by CumulusMX: take a temperature sample every minute, sum these and divide it by the number of observations. That leads to a more accurate estimator of the average temperature and thus of the GDD.

Why should I be worried about tables they made a hundred years ago? The GDD is not hard science, it is

[...]a heuristic tool in phenology. GDD are a measure of heat accumulation used by horticulturists, gardeners, and farmers to predict plant and animal development rates such as the date that a flower will bloom, an insect will emerge from dormancy, or a crop will reach maturity.

A modern farmer works with computers and takes calculations produced buy their own weather stations.

Again: why stay with old tables? You don't use logarithmic tables or slide-rule anymore do you? You just take your calculator.

@sfws: I think I made my point above. And beyond that, are we (or is CumulusMX) a slave to the WMO, then you start better rewriting because it simply does not function with that equation as an estimator of average temperature so stop talking about that.

and

but I doubt if any plants can do mathematical integration!

1. You don't have to shout
2. They can't add and divide either

If we start discussing on this level we better stop.

[mcrossley]

... but I doubt if any plants can do mathematical integration!

Hmm, I would argue that is exactly what they do!

They grow anytime the conditions are right, not at a mean rate when (max-min)/2 over 24 hours meets their conditions.

I'm going to have re-read this thread and see where it leaves us...

[HansR]

Just checking: I received a mail about a reaction of @sfws which I read about 2 hours ago while drinking coffee and now that I am ready to answer it has disappeared. It is possible the message is withdrawn - which is OK btw - but a notice about the withdrawal would have been polite.

[HansR]

And apart from that: please read some related comment with my chart 'Clash of Averages'.

[sfws]

Hans, I re-read my earlier post written just before I went to bed, and decided it was confusing. It takes me time to decide on a structure and best wording, hence you had time for a couple of posts. For your information, the UK Meteorological service uses both definitions of average temperature internally, but it follows WMO guidance for information shred globally. So your Wiki page you reference has got it wrong.

I'm going to have re-read this thread and see where it leaves us...

I did read the whole thread before making my previous post.

I want to find out the length of the growing season and the amount of heat. That is, all days when the average daily temperature exceeds 5 degrees Celsius. Less than 5 degrees Celsius is not included.

Set the Cooling threshold to 5C, then Cooling Degree Days becomes your Growing Degree Days.

At the start of this thread, you interpreted that Cooling degree days (temperature and time combined) was the answer.

Effective heat sum is a figure describing the heat accumulated during the growing season.
The effective heat sum is obtained by summing the part that exceeds the average temperature of all days + 5 ° C during the summer. - Degree days
Different plants require different amounts of effective heat to grow.
If the effective heat sum is more than 600 degrees (or degree days, ° Cvrk), trees will grow.

For MX (while left running) calculation of degree days is based on tracking, during each day, the duration that the temperature is above a threshold. Yet the above quote is asking to track the number of days the average temperature is above the threshold. This is the reason why your first answer might not work.

I know Cumulus 1, when asked to regenerate a NOAA report, did indeed use the WMO daily average "(Max + Min) / 2" as a basis for regenerating the cold degree days it reported. Does MX do that?


However, the originator of this thread also wanted "heat accumulation"...

The site has charts of heat accumulation and heat sum for each year. I calculate them with an excel program from the average daily temperature. I compare years with each other like this. Heat sum graphics link https://jussilanet.com/tehotemp.htm and heat buildup graphics link https://jussilanet.com/sumheat.htm. Is it possible in the future to get cmx to run these graphics?

Heat accumulation graphics etc. the file is made by adding together the average temperatures of the days
....
If cmx ever computes these then you can probably create a graphic from a json file.

From those posts by the originator, it seems that average temperature is indeed the only criteria. As the graphics plotted this on a yearly basis, the thread moved on to arguing how to define average temperature. Mark, you will remember an early modification you made to the NOAA reports was to replace their use of the WMO average (on (max-min)/2 basis) with the "integrated average" Cumulus calculates.


Next, I will provide some scientific evidence to support my previous post that made you respond.

Hmm, I would argue that is exactly what they do!

They grow anytime the conditions are right, not at a mean rate when (max-min)/2 over 24 hours meets their conditions.

Some plants respond to temperature, some to light conditions, some to moisture levels, but the information I have seen, always quotes this in respect to day by day, not minute by minute (or second by second).

There is some evidence (as quote below) that temperature changes can be detected fairly quickly, and that will affect individual cells.
Ruelland E, Zachowski A. How plants sense temperature. Environ Exp Bot. 2010;69(3):225–232.

changes in the ambient temperature can be quickly detected by cell organelles, triggering specific pathways of biochemical and molecular responses in each of these cell compartments and making up an integrated cell response to temperature changes.

This proves a plant responds to temperature fluctuations. So you are partly right, growth can vary depending on current temperature. But a plant does not decide to pop out of the ground, or produce a bud based on a spot temperature, these responses take time to action. Heat may help crops to ripen, but I have seen no evidence that minute to minute temperature variation is a factor.

You could say a plant detects changes in the climate, rather than the instantaneous weather. For that assessment, my argument stands, a plant does not have the computational ability to calculate a cumulation total of how the temperature is varying , but a plant can sense daily extremes. I never claimed all plants calculate the WMO mean, but daily high temperatures and low temperatures will both play some part. The WMO mean is a way that human beings can represent these two extremes.

]I looked at several definitions and they all use (Max + Min) / 2
Remember this is a 'reference' and it would have been created long before mean samples were easy to do. If you change the basis of the formula you can't compare with look-up tables or other(s) data.

My previous reply, was to support this WMO definition of average temperature. I argued, this topic is about growing season, and that is defined in terms of days when plants can grow, and days when they cannot grow. Like beteljuice, I see that formula used in the scientific studies of how plants grow.

Here is how one paper describes all this

The growth and development of plants depend, in part, on the temperature. Shifts in the timing of seasons as a direct result of increases in global average temperatures show that growth and development of many plants during spring and summer occur earlier in the year than in the past. To better understand how changes in temperature influence plants, scientists use a well-known tool referred to as Growing Degree Days (GDD) as a way to track the stages of growth and development.
...
GDD are calculated by finding the average of the maximum (Tmax) and minimum (Tmin) daily temperatures and subtracting this value from a base temperature (Tbase) of interest
...
GDD are cumulative–they add up as the year progresses–since they are calculated daily.

This quote surely confirms my assertion that the growing season in question is based on a response to daily extremes. There is evidence of a response to extremes in this final quote.

It has been known since 1939 (Laude et al) that plants' response to heat stress fluctuates between day and night - if you apply heat stress to a plant during the middle of the day, it is much more likely to survive than if you applied the same heat stress at night.

One factor to remember is that on a commercial basis, there will be lots of plants growing together (a woodland or crop field generally has say thousands of plants), these provide shelter, and make most (not those on outside edge) of the plants will be less susceptible to minute by minute variations in air temperature (as the plants see a more constant temperature). This is one reason why WHO mean, not integral of individual temperatures is a better measure for the whole field.

[HansR]

I will not reply into detail on the reaction of sfws and I have no intention to follow up on this discussion the way it goes.
But I will give a short response.

1. The GGD and Temperature Sum have nothing to do with the physiology of the plant. They are heuristics used by farmers, foresters and in general agriculturals. The heuristics have never been intended to provide scientific explanation for plant growth nor have they demanded the plant to do what the heuristic says. Bringing plant physiology to an heuristic for the timing of agricultural actions is not correct to say it polite. Of course there is a background and some reasoning but there is no cause and effect. So leave it out. The discussion is about the estimator of the daily average temperature.
2. The heuristic of Temperature Sum is SUM(Tav) for all Tav > 0 from January 1 to december 31. Similar for GGD with a substraction of a constant base temperature
   Having said that it is easy to see the average temperature is what its all about. Nothing more. With respect to the average temperature it is clear that you have to make a distinction between a True average and an estimated average. Every estimator will take discrete samples and divides by the number of samples. The true average is the integral (over the day) of the continuous temperature. The more samples we have, the more accurate we estimate the True value. That is the basis of numerical integration which is basically what is being done in CMX for the determination of Tav

Discussing the truthfulness of an estimator of an heuristic is nonsense.
One is just more accurate than the other. The more accurate the estimator is, the better.
(Max+Min)/2 as an estimator may be used in the NOAA reports, I don't know, but it is not the value in the Dayfile. It is just an archaic estimator for Tav.
(and see my link above for additional remark to the difference between the averages)

[mcrossley]

OK, my view...

The argument between integrated and max+min / 2 methods for calculating the GDD is a bit moot. I calculated using both methods going back over all my data (5 minute intervals).

Most years the difference using a 5C base temp was around 10 GDD (either way) out of a total of ~ 1500 - 2000. Around 0.5%, so not really worth arguing about when most of the GDD "ranges" I have seen for growth events are at least an order of magnitude or two greater.

I do note that the integral method does seem count days earlier in the season than the max/min method - maybe using the integral method would allow them to refine the ranges? Though I suspect it is much more complex than that, and sunshine hours and soil moisture also need to be factored in to create a "growth index". Anyone up for that?!

I intend to implement the max+min / 2 method Variant A , there a few reasons for this...

1. Using the integral method is computationally expensive to perform retrospectively and you lose the minute by minute data, which means that the GDD really has to be recorded daily into the dayfile. This means extending the MySQL schema, and the ExportToMySQL and CreateMissing utilities. This is not something I wish to do.
2. The max+min / 2 method is simple enough to to implement on the fly, so no extension to the dayfile required
3. Calculating GDD on the fly means the base temperature can easily be amended for different crops

I intend to implement the ability to show two different GDD series which can have different base temperatures. The data selection gets a bit messy with lots of years of data, but I see this is a tool for farmers and horticulturists not a casual click and scan visitor. Maybe someone can come up with good way of handling lots of data series in HighCharts.