Correcting Max/Min readings

[sanramonrover]

With the spikes I am getting, there are a lot of erroneous temp readings. I have tried removing or correct them by various means, but they don't seem to always disappear.

This is the [alltimelog] for this morning:

2011-02-22 04:49 12.200 "All time low temperature" 2011-02-22 04:44 12.300
2011-02-22 04:49 12.200 "All time low wind chill" 2011-02-22 04:44 12.300
2011-02-22 05:09 12.000 "All time low temperature" 2011-02-22 04:49 12.200
2011-02-22 05:09 12.000 "All time low wind chill" 2011-02-22 04:49 12.200
2011-02-22 06:39 12.800 "All time low temperature" 2011-02-22 05:09 12.000
2011-02-22 06:39 12.800 "All time low wind chill" 2011-02-22 05:09 12.000

The help page says the order of entries is:
date time value "item name" old date old time old value

This doesn't appear right to me, as sometimes the old <low temp> or <low windchill> value (last entry) is higher than the current entry value (3rd from right). Examples, lines 1-4. Am I reading something wrong?

Yes, I have read FAQ, Help and forums.

[steve]

This doesn't appear right to me, as sometimes the old <low temp> or <low windchill> value (last entry) is higher than the current entry value (3rd from right). Examples, lines 1-4. Am I reading something wrong?

Those are fine - a new low temp has to mean that that the old record is higher than the new record. But it's the others that I don't understand:

2011-02-22 06:39 12.800 "All time low temperature" 2011-02-22 05:09 12.000
2011-02-22 06:39 12.800 "All time low wind chill" 2011-02-22 05:09 12.000

That first one says that at 06:39 you had a new low temp of 12.8, beating the existing low temp of 12.0. That doesn't make sense. What version of Cumulus are you using?

[sanramonrover]

I'm using 1.9.0, build 958. Yes, Steve, I meant lines 5 & 6 appeared abnormal.

[steve]

I don't understand how the code could have been working at 05:09, but then wrong at 06:39. I've looked at the code and I can't see how it could possibly do what it apparently did at 06:39. Had you done anything between those two times?

[sanramonrover]

I apologize for the confusion. Perhaps it was when I tried to eliminate the spikes from the feb11log manually.

[steve]

I apologize for the confusion. Perhaps it was when I tried to eliminate the spikes from the feb11log manually.

Well... maybe, maybe not. The all-time records are held separately, and I still can't see how the code could decide that there was a new all-time low when the current temperature (12.8) was apparently higher than the old record (12.0). Particularly when it was working OK a few hours earlier.

I've looked through all my alltimelog.txt files, and I can't find any instances like the ones you've shown here.

[ColinJS]

Hi,

I have recently found similarly wild errors in the alltime files. Yesterday morning, at 8.45 (just before rollover), the rainfall rate was claimed to be around five *metres* an hour. I thought a problem with the raingauge, such as is mentioned in several posts. But the log showed (correctly) zero rainfall at the time in question. Then I noticed that earlier in the night (02:40) the min temperature had been changed to -3.9, when it should have been +9.9

I guess I can correct these by editing the alltime*.* files. The other files seem to be correct. I'll keep a copy of them in case.

Any ideas as to what is wrong? My receiver is a way from the sender, but it seems to receive data correctly and the logged data is correct. So can't be that? Or am I misinterpretting this?

2011-02-24 02:40 -3.900 "All time low temperature" 2011-02-18 23:55 1.900
24/02/11,02:40,9.9,88,8.0,3.5,8.3,294,0.0,9.9,1002.9,156.3,20.3,36,2.2,9.4,9.9,0.0,0

Thanks,

Colin

[steve]

Any ideas as to what is wrong? My receiver is a way from the sender, but it seems to receive data correctly and the logged data is correct. So can't be that? Or am I misinterpretting this?

Any erroneous readings from the station which aren't filtered out by the various methods that Cumulus uses may affect the all-time (etc) records, but not necessarily appear in the logs as these are snapshots of the readings at the time of logging.

[ColinJS]

Thanks for the prompt response!

So I'll just fix the alltime files and see if it happens again. It has been running for a couple of weeks without obvious gliches. Maybe I'll write a scipt to compare the log with the alltime to check for such problems.

Thanks,

Colin

[steve]

You could attempt to filter them out, e.g. using the EWTempdiff setting in cumulus.ini.

I've managed to determine that you're using 1.9.0; the 1.9.1 beta has more settings like the above for other parameters, and they are now accessible without editing cumulus.ini.

[ColinJS]

Thanks for the suggestion, I'll try it later.

And sorry for not telling you my software version! I should know better (work for a software company). I entered the website in my details, but guess I should put it in the sig.

Thanks for the help. Have a good day!

Colin

[fractonimbus]

Any erroneous readings from the station which aren't filtered out by the various methods that Cumulus uses may affect the all-time (etc) records

Steve, out of curiosity, what sort of filtering goes in to the traces, records etc? I notice that the FO WH1091 trace is much noisier for the period since Cumulus was last booted, but when shut down and rebooted later, the sampling for that same period is evidently averaged over a longer period. I suspect a short "realtime" Hamming filter might help avoid some of the noise and maybe any rogue measurements still left after outliers are removed.

DN

[steve]

Steve, out of curiosity, what sort of filtering goes in to the traces, records etc?

Just a simple discarding of obviously off-the-scale values, plus the optional 'spike removal' facility.

I notice that the FO WH1091 trace is much noisier for the period since Cumulus was last booted, but when shut down and rebooted later, the sampling for that same period is evidently averaged over a longer period.

That's because while running live, Cumulus plots at 1-minute intervals. When it starts up, it pre-fills the graphs with data from the log files, which is at whatever interval you've set the logging intervals in Cumulus and/or your station.

[fractonimbus]

Thanks, Steve.

DN

[ColinJS]

Hi,

I installed 1.9.1 yesterday. Overnight got loads of lost sensor messages. And a ludicrous hourly rain rate of over thirty metres. Most of the other rain values were fine. The tip count went up by thousands and in the next reading reverted to the previous value - i.e. went backwards.

I 'fixed' it by hacking the various log files.

I couldn't see anything in the ini that would deal with this. Am I missing something? Is rejection of off-the-scale and/or spikes values something I have to turn on?

Thanks,

Colin