Silly rainfall spikes - filtering settings wrong?

[rwg]

First off, thanks for creating and sharing this software - it's really appreciated

I'm using an Oregon Scientific WMR180 station and it usually works well, aside from an occasional rainfall rate reading of a whopping 3810mm/hr!

Now, I thought that I'd set things up to filter these out - turned off the 'use data logger' check box and the rain rate limit is set to 900mm/hr, but it doesn't seem to help. I think that the problem is happening when the bucket tips twice in quick succession, possibly twice between two successive readings coming back from the station - I certainly see a two steps at the same time rise in the rainfall graph at the point where the spike occurs.

It's perhaps worth knowing that I've put a bigger collecting funnel on the rain gauge, so I have set the rainfall multiplication factor to 0.333 to compensate.

The most recent spike occurred at 17:21 on the 15th. Here is the data from the log file for that period :

15/05/2012 17:10:01.058 : Writing today.ini, LastUpdateTime = 15/05/2012 17:10:01 raindaystart = 66.5479965209961 rain counter = 69.5960006713867
15/05/2012 17:13:00.815 : *** Data input appears to have stopped
15/05/2012 17:15:00.577 : Writing today.ini, LastUpdateTime = 15/05/2012 17:15:00 raindaystart = 66.5479965209961 rain counter = 69.5960006713867
15/05/2012 17:20:00.083 : Writing today.ini, LastUpdateTime = 15/05/2012 17:20:00 raindaystart = 66.5479965209961 rain counter = 69.5960006713867
15/05/2012 17:25:00.598 : Writing today.ini, LastUpdateTime = 15/05/2012 17:25:00 raindaystart = 66.5479965209961 rain counter = 71.8819961547852
15/05/2012 17:30:00.259 : Writing today.ini, LastUpdateTime = 15/05/2012 17:30:00 raindaystart = 66.5479965209961 rain counter = 76.1999969482422
15/05/2012 17:35:00.636 : Writing today.ini, LastUpdateTime = 15/05/2012 17:35:00 raindaystart = 66.5479965209961 rain counter = 77.2159957885742
15/05/2012 17:40:00.328 : Writing today.ini, LastUpdateTime = 15/05/2012 17:40:00 raindaystart = 66.5479965209961 rain counter = 78.2319946289063
15/05/2012 17:45:00.845 : Writing today.ini, LastUpdateTime = 15/05/2012 17:45:00 raindaystart = 66.5479965209961 rain counter = 78.2319946289063

Any advice on how to filter these spikes out more successfully would be welcome!

cheers,

Robin

[steve]

turned off the 'use data logger' check box and the rain rate limit is set to 900mm/hr

Neither of those settings has any effect with your station. Turning off 'use data logger' has no effect because your station doesn't have a logger anyway, and the rain rate spike removal setting only applies to those stations which don't supply a rain rate so Cumulus has to calculate one, and which have shown a tendency to give erroneous rain data figures, namely the Fine Offset and La Crosse stations. Oregon Scientific stations do not normally invent large amounts of rain.

Any advice on how to filter these spikes out more successfully would be welcome!

I think you need to investigate the cause of the spikes - could the wind or something else be causing the buckets to tip? Or perhaps your modification means that the bucket does tip twice in quick succession in heavy rain, hence the high figure, which could well be accurate - but over a very short period. It's hard to say without knowing how the station determines its rain rate figure.

[steve]

I've just tried my WMR200 rain gauge (it's on my desk, so it's easy to experiment with) making two tips in short succession, and I got a rain rate of 11444 mm/hr. So it looks like the station calculates rain rate based on the time between two tips. So, in heavy rain, and particularly with a larger funnel, it will generate high rain rate figures.

[rwg]

Ok, thanks, getting a clearer picture now

I see what you are getting at that with the larger collecting funnel the station may be seeing two tips at what it sees as being the same time, and it then effectively does a division of rainfall change (1mm) by time taken (0s) and ends up dividing by zero. It may well be that my readings of 3810 were actually something like 65535 in some units before being converted to mm/hr and scaled by a third.

I guess I have a couple of further questions...

1. Is there any way I can make cumulus write a log of all the readings it gets from the station? Then I could go through and look at the raw data at the time the next spike happens and see if I can see anything interesting that I might learn something from as to what is going on.

2. Is there a setting to force cumulus to calculate the rain rate for my station like it does for some of the others from the rain amount differences?

thanks again!

Robin

[rwg]

I've just tried my WMR200 rain gauge (it's on my desk, so it's easy to experiment with) making two tips in short succession, and I got a rain rate of 11444 mm/hr. So it looks like the station calculates rain rate based on the time between two tips. So, in heavy rain, and particularly with a larger funnel, it will generate high rain rate figures.

11444 multiplied by 0.333 is 3810, so that is definitely the culprit!

However, it's also definitely silly... I spent 7 years working in the Meteorology department at the University of Reading and remember people there getting excited about rain rates in the 100's of mm/hr. Goodness knows what the 11444 mm/hr would be like - 3mm/s! So I guess I'd still like to find a way to filter these values out if at all possible.

cheers,

Robin

[steve]

1. Is there any way I can make cumulus write a log of all the readings it gets from the station? Then I could go through and look at the raw data at the time the next spike happens and see if I can see anything interesting that I might learn something from as to what is going on.

If you turn on the debug log, the raw packets are logged to the debug.log file. You can then use Mark's spreadsheet to decode the data: https://cumulus.hosiene.co.uk/viewtopic.php?f=16&t=5912

2. Is there a setting to force cumulus to calculate the rain rate for my station like it does for some of the others from the rain amount differences?

No, I'm afraid not. It uses a very simple calculation based on the amount of rain in the last 5 minutes, and generally if a station calculates its own rate it does a better job than Cumulus. It wouldn't be hard to add as an option, so I could consider adding it at some point.

[steve]

I've just noticed that when the rain rate in the data packets is 11444 (which is B000 in the hex, in hundredths of an in/hr) that the WMR200 console displays 'HHH', so perhaps this is actually an overflow value. If that's the case, I can check for it and ignore it. It would be useful to see a packet from your station while the high value is displayed to see if the same value is used.

[rwg]

Ok, I'll try to get the debug output for this over the weekend.

cheers,

Robin

[steve]

I found this thread which confirms that the WMR100 (etc) does use the same value (hex B000): https://cumulus.hosiene.co.uk/viewtopic.php?f=20&t=5355

All we have to do now is agree on what Cumulus should do; the options are to ignore it (and hence the rain rate figure at the last 'good' value) or substitute a lower figure instead.

[rwg]

I can think of at least 3 possibilities...

* calculate a rain rate from (difference in rain amount) / (time elapse since last reading)
* Use the maximum genuine rate the station can report (if we knew or could work out what that is)
* As you suggest use the previous time interval's reading.

I think that the first option is unbiased whereas the last two would always tend to give values lower than the real rain rate (whatever that may be).

cheers,

Robin

[steve]

Use the maximum genuine rate the station can report (if we knew or could work out what that is)

I saw one spec which said it's 9999 mm/hr.

[rwg]

Hmmm... from my experience that number seems somewhat optimistic!

To exceed 9999mm/hr you'd need to be doing tips at a rate of nearly 3 per second. Given the design of the bucket I think that getting to that sort of rate is unlikely, even if you run a hosepipe into the collecting funnel. I just don't think it's going to move backward and forward fast enough!

It might be possible to determine the real maximum rate experimentally - basically sit there tipping the bucket once a second, once every 2 seconds, etc and seeing what the rate comes out at. I can't actually say I fancy doing the experiment though!

Calculation is the only way to go

cheers,

[steve]

Calculation is the only way to go

Except that it isn't one of the two options I'm offering

I've found the spec for the WMR100 which gives the max rain rate as a more reasonable 999 mm/hr.

[mcrossley]

I've found the spec for the WMR100 which gives the max rain rate as a more reasonable 999 mm/hr.

which is a tip every 1.1 seconds isn't it?

[steve]

I've found the spec for the WMR100 which gives the max rain rate as a more reasonable 999 mm/hr.

which is a tip every 1.1 seconds isn't it?

3.66 seconds, I think. 3600/(999/1.016)