Cumulus Support

{Hope that a cross-post from wxforum.net is acceptable.)

For anyone who may be interested: Stephen Burt (who some may recognise as the author of the excellent and recently-published 'Weather Observer's Handbook') has completed a 15-month long evaluation of the Davis Vantage Vue against other professional-quality instrumentation. In other words, this is absolutely not a quick review by eg a gadget magazine, but a rigorous, long-term, scientific evaluation.

Stephen gave a presentation to the Royal Meteorological Society last weekend (hello, if anyone here attended and stopped by our stand) summarising the results and you can view the slides from that presentation in PDF format at:

http://measuringtheweather.com/wp-conte ... n-Burt.pdf

or, more succinctly, http://goo.gl/Gb2WzI

I understand that there will be a more detailed write-up at a later date.

The bottom line is that - unsurprisingly - the Vue is not perfect, for example under especially demanding conditions such as clear, calm midsummer days (and to a lesser extent on clear calm radiation nights) there is a noticeable temperature error relative to a large traditional Stevenson screen. But overall the Vue puts in a creditable performance for such a (relatively) inexpensive station.

(For the record, we supplied the Vue to Stephen for this evaluation, but other than in this connection, the review was totally objective and independent.)

prodata wrote:{Hope that a cross-post from wxforum.net is acceptable.)

For anyone who may be interested: Stephen Burt (who some may recognise as the author of the excellent and recently-published 'Weather Observer's Handbook') has completed a 15-month long evaluation of the Davis Vantage Vue against other professional-quality instrumentation. In other words, this is absolutely not a quick review by eg a gadget magazine, but a rigorous, long-term, scientific evaluation.

Stephen gave a presentation to the Royal Meteorological Society last weekend (hello, if anyone here attended and stopped by our stand) summarising the results and you can view the slides from that presentation in PDF format at:

http://measuringtheweather.com/wp-conte ... n-Burt.pdf

or, more succinctly, http://goo.gl/Gb2WzI

I understand that there will be a more detailed write-up at a later date.

The bottom line is that - unsurprisingly - the Vue is not perfect, for example under especially demanding conditions such as clear, calm midsummer days (and to a lesser extent on clear calm radiation nights) there is a noticeable temperature error relative to a large traditional Stevenson screen. But overall the Vue puts in a creditable performance for such a (relatively) inexpensive station.

(For the record, we supplied the Vue to Stephen for this evaluation, but other than in this connection, the review was totally objective and independent.)

It is an interesting review, but to be honest I think that the temperature readings from the Davis Vantage Vue are very accurate. Though, I do agree that the temperature readings from the Davis Vantage Vue on calm, clear and frosty nights (at 5 meters) are a little higher than (at 1 meter). But, this doesn't concern me as it is very rare that there are frosty nights, here.

It is an interesting review, but to be honest I think that the temperature readings from the Davis Vantage Vue are very accurate.

Well, yes, but with respect you're just hoping and guessing that your temperature data are fully accurate (unless you also have some accurate reference data measured at exactly the same site). Stephen's evaluation was a proper side-by-side scientific analysis and that's going to trump guesses any day.

The main caveat with Stephen's analysis is that it's just a sample of one Vue unit. Nothing to suggest that it's not representative of the the Vue more generally, but data from other independent analyses is also needed to get a complete picture. (I am aware that other similar studies are under way in the UK, so there may be more data before too long.)

Interesting - especially the odd comparisons against the VP2 (for me anyway). Has Stephen done a similar exercise for the VP2?

Assuming the Vue uses the same TH sensor as the VP2, it has claimed accuracies of:
Temperature worst +/-1% between 0-40°C
Humidity +/-3% between 20-20% RH, Stephen has measured +6% at mid range.

The Davis radiation shield will affect the temperature readings, but should not have much influence on the humidity?

The Thermal images of the stations confirm my thoughts - why does the VP2 have a black rain bucket?

The barometric pressure reading is pretty consistent, but the Davis also applies a 'fudge factor' to the MSL pressure depending on temperature and humidity. It would be interesting to see the raw station pressures compared.

prodata wrote:Well, yes, but with respect you're just hoping and guessing that your temperature data are fully accurate (unless you also have some accurate reference data measured at exactly the same site). Stephen's evaluation was a proper side-by-side scientific analysis and that's going to trump guesses any day.

The main caveat with Stephen's analysis is that it's just a sample of one Vue unit. Nothing to suggest that it's not representative of the the Vue more generally, but data from other independent analyses is also needed to get a complete picture. (I am aware that other similar studies are under way in the UK, so there may be more data before too long.)

Yeah, well I know that the Davis Vantage Vue is accurate due to measuring the temperature with a thermometer in the shade at the same site and it showed the same reading.

Yeah, Stephen needs to use more than 1 Davis Vantage Vue to record a fair test.

@Mark: Answers/comments as follows:

Yes Stephen has published a similar VP2 analysis, there's a PDF copy downloadable from our website at http://goo.gl/NVfpLN (there'll almost certainly also be a link from Stephen's own measuringtheweather website).

Yes the Vue has the same T/H sensor as the VP2.

Defining AWS accuracy specifications is not easy. (Not to say for a second that all manufacturers shouldn't do their best, but there are real problems.) To start with, what are usually quoted are the intrinsic sensor specs, which are typically not single values but vary according to the magnitude of the value being measured (best temperature accuracy, for example, is only valid over a relatively narrow temperature range, rainfall accuracy will depend on rainfall rate, and so on). And some of the parameters depend on others, so eg RH accuracy data is typically only quoted at one temperature (eg 25C) with no easy clue as to its temperature dependence. And it's also likely the case that any error in eg temperature measurement will then have a knock-on effect on the temperature correction of RH values and hence lead to additional errors.

In practice, these intrinsic errors are then overlaid with other potential errors from the AWS design, eg the shield performance and then finally a further overlay of additional errors arising from the mounting - eg sensor height - and limitations of the observing site (relative to a fully standardised exposure). This latter category is probably reasonably obvious but is also typically the largest source of error for amateur sites.

So for eg T/H specs, I'm pretty sure that Davis will be quoting the Sensirion data for the sensor itself - I don't believe that Davis select or individually calibrate the Sensirion sensors BICBW. But again there must be a production distribution in the sensor errors - some will be more accurate than others, but the Sensirion data doesn't provide any real idea of what the shape of that distribution might be. Errors should, strictly speaking, be statistically based, not quoted as single figures.

It's then difficult to know how to pull these different sources and expressions of error together. On a calm sunny day, there will be a significant screen error with a passive shield, but worst case days may be only a handful per year (in the UK at least). So if you quote the worst case errors then you're suggesting that the AWS will perform poorly, when in fact it will be substantially more accurate on 98% of days (and probably for >99% of readings all-told - given that worst case conditions will apply for only a few hours on the days when they do apply).

Finally, and getting back to the RH point, any such trial is a comparative trial - you're relying on the reference readings you have. And it's far from inconceivable that Stephen's RH sensor might have been under-reading by possibly up to 1-2% in the midrange - RH is not easy to measure accurately even with very expensive sensors.

The usual answer to the VP2 black bucket is that it needs extra UV protection in the plastic to stop it becoming too brittle over time. But there is an air gap between TBR baseplate and shield, plus quite an effective heatshield at the top of the radiation shield, so this isn't normally considered to be a source of any significant error, even though superficially you might think otherwise.

On the pressure readings: AIUI the 'fudge factor' is only applied to correct measured pressure value to sea level values. Stephen's site isn't too far from sea-level and so any such corrections will have been minimal to negligible. So to all intents and purposes you are seeing the 'raw' station pressures. (In any installation, you can always leave a Vue or VP2 station set to virtually zero altitude so that it reports local atmospheric pressure and then do any correction to sea level in software using whichever correction algorithm you prefer. The only drawback to this approach is that you do need to take the console down to sea level occasionally so that you can check its calibration.)

Apologies for the long post!

William Grimsley wrote:

prodata wrote:Well, yes, but with respect you're just hoping and guessing that your temperature data are fully accurate (unless you also have some accurate reference data measured at exactly the same site). Stephen's evaluation was a proper side-by-side scientific analysis and that's going to trump guesses any day.

The main caveat with Stephen's analysis is that it's just a sample of one Vue unit. Nothing to suggest that it's not representative of the the Vue more generally, but data from other independent analyses is also needed to get a complete picture. (I am aware that other similar studies are under way in the UK, so there may be more data before too long.)

Yeah, well I know that the Davis Vantage Vue is accurate due to measuring the temperature with a thermometer in the shade at the same site and it showed the same reading.

Yeah, Stephen needs to use more than 1 Davis Vantage Vue to record a fair test.

yes, but do you know if the control thermometer you used was accurate?

BigOkie wrote:yes, but do you know if the control thermometer you used was accurate?

That's certainly an important point. To get an optimum comparison you'd need a BS (British Standard) marked thermometer accurate to 0.2-0.3C and exposed in a recognised-pattern standard Stevenson screen in an open area close to the Vue. Also, on calm days, the air above a sunny garden area can be significantly different in temperature to that in a shaded area, especially if there's been any extra overnight cooling in the latter area from nearby brick/stone walls, paving slabs etc.

FWIW I would agree with regard to temperature. I have my VP2 (unaspirated) adjacent to a Stevenson screen (over mown grass) containing Instromet sensors and Met Office mercury thermometers. I find the VP2 does read slightly high during intense solar radiation and lower wind speeds, but do not notice much variation on low night-time temps.
I have adjusted the VP2 rain gauge and this conforms very closely to my funnel type 'manual' gauge.
The Instromet and VP2 anemometers are about 50 metres apart (but both at 10m) so there is a slight variation but max. gusts are similar.

BigOkie wrote:yes, but do you know if the control thermometer you used was accurate?

What?

prodata wrote:

BigOkie wrote:yes, but do you know if the control thermometer you used was accurate?

That's certainly an important point. To get an optimum comparison you'd need a BS (British Standard) marked thermometer accurate to 0.2-0.3C and exposed in a recognised-pattern standard Stevenson screen in an open area close to the Vue. Also, on calm days, the air above a sunny garden area can be significantly different in temperature to that in a shaded area, especially if there's been any extra overnight cooling in the latter area from nearby brick/stone walls, paving slabs etc.

Yeah, I only used a thermometer not a stevenson screen but comparing with other weather stations nearby, the Davis Vantage Vue is very accurate.

William Grimsley wrote:

prodata wrote:

BigOkie wrote:yes, but do you know if the control thermometer you used was accurate?

That's certainly an important point. To get an optimum comparison you'd need a BS (British Standard) marked thermometer accurate to 0.2-0.3C and exposed in a recognised-pattern standard Stevenson screen in an open area close to the Vue. Also, on calm days, the air above a sunny garden area can be significantly different in temperature to that in a shaded area, especially if there's been any extra overnight cooling in the latter area from nearby brick/stone walls, paving slabs etc.

Yeah, I only used a thermometer not a stevenson screen but comparing with other weather stations nearby, the Davis Vantage Vue is very accurate.

So if you compare inaccurate thermometers to OTHER inaccurate thermometers, how does that make them accurate?

BigOkie wrote:So if you compare inaccurate thermometers to OTHER inaccurate thermometers, how does that make them accurate?

When did I say they were inaccurate?

William Grimsley wrote:

BigOkie wrote:So if you compare inaccurate thermometers to OTHER inaccurate thermometers, how does that make them accurate?

When did I say they were inaccurate?

How do you know that they are?

William Grimsley wrote:When did I say they were inaccurate?

Surely you have to start with the assumption that all thermometers are inaccurate unless either they are of a type that is manufactured to specific published accuracy specifications (for example a British Standard or NIST specification or other credible specifications) or, alternatively, that you have carefully calibrated the thermometer in question against a known accurate standard.