Wunderground Upload Wind Specs?

[NorthNJwx]

I've been extremely impressed with Cumulus since I started running it several days ago, and I'm thinking of possibly using it to upload to Wunderground (for normal, non-rapidfire updates). Before I do so, though, I was wondering about the specifications of the wind data that Cumulus sends to WU. Here are my questions:

1. Is the wind direction uploaded an average direction? If so, over how long a period of time before the upload is this average calculated?

2. Is the wind speed uploaded an average speed? If so, over how long a period before the upload is this average calculated?

3. For the wind gust, how long a period before the upload is the peak gust sampled from? (as in...peak from the last 10 min, or from the last 5 min, etc)

4. Is it possible for Cumulus to/does Cumulus already upload the average speed/direction and peak gust over the upload interval period? So, for example, can/does Cumulus upload a 5-min average direction and speed and peak gust over the past 5 minutes if the upload interval is set to 5 min?

[steve]

1. Is the wind direction uploaded an average direction? If so, over how long a period of time before the upload is this average calculated?

Yes. 10 minutes.

2. Is the wind speed uploaded an average speed? If so, over how long a period before the upload is this average calculated?

Yes. 10 minutes.

3. For the wind gust, how long a period before the upload is the peak gust sampled from? (as in...peak from the last 10 min, or from the last 5 min, etc)

10 minutes.

4. Is it possible for Cumulus to/does Cumulus already upload the average speed/direction and peak gust over the upload interval period? So, for example, can/does Cumulus upload a 5-min average direction and speed and peak gust over the past 5 minutes if the upload interval is set to 5 min?

It could, but it would require to me to write extra code; I'm not sure how much. Anything less than the normal 10 minutes would probably be easier to do than anything more than 10 minutes.

[NorthNJwx]

Thanks a lot for the quick reply. Since average wind speed/direction and peak gust data is important to me, it's great to hear that Cumulus does indeed use these parameters for the WU upload.

[NorthNJwx]

While using Cumulus to send to WU, I noticed it was "missing" some of the 10-min peak gusts, as in the actual highest gust from the 10-min period wasn't always sent to WU, but a lesser number was sent instead (I know this from comparing the Cumulus 10-min data to my WeatherLink 10-min data from the same time periods) . The actual peak gust from the 10-min period was sent in most, but not all of the updates. For what it's worth, I've never noticed the Cumulus software missing data packets from the VP2 on the display screen. Not sure if this is something on my end/on the WU end.

[steve]

I think I may have an explanation. Cumulus works out the peak gust in the last ten minutes by saving the most recent readings and checking the ones which are from the last 10 minutes to find the highest. It saves the 200 most recent readings. The reading interval for the VP2 used to be just over 3 seconds, but I managed to improve it slightly so the average is now just under 3 seconds. So 200 readings isn't quite enough to hold 10 minutes worth of data.

I'm trying to work out what the effect of this is, and I think it would only mean peak gusts are 'missed' if your Wunderground update interval is more than the 200 readings worth, so somewhere around 9 to 10 minutes or more. And I think it means the peak would have to occur shortly after the Wunderground update. What interval are you using? Could this explain what you're seeing?

The value it sends to Wunderground should be the same as appears in the 'Gust' box on the Cumulus main display, if that helps you work out what's going on.

[NorthNJwx]

I think I may have an explanation. Cumulus works out the peak gust in the last ten minutes by saving the most recent readings and checking the ones which are from the last 10 minutes to find the highest. It saves the 200 most recent readings. The reading interval for the VP2 used to be just over 3 seconds, but I managed to improve it slightly so the average is now just under 3 seconds. So 200 readings isn't quite enough to hold 10 minutes worth of data.

I'm trying to work out what the effect of this is, and I think it would only mean peak gusts are 'missed' if your Wunderground update interval is more than the 200 readings worth, so somewhere around 9 to 10 minutes or more. And I think it means the peak would have to occur shortly after the Wunderground update. What interval are you using? Could this explain what you're seeing?

The value it sends to Wunderground should be the same as appears in the 'Gust' box on the Cumulus main display, if that helps you work out what's going on.

Davis WeatherLink actually records the number of data packets it receives in its archive, which I have set for 10 minutes. My console reception is usually 100%, which typically translates to about 218 data packets in the 10 minute period - this number varies for different people based on the station ID they use for the ISS/Anemometer transmitter kit/etc., but it is usually around 218 readings every 10 minutes for me. So, based on that, 91.7% of the readings (200 out of 218) are sampled by Cumulus when I use it to upload to WU, meaning that approximately the same percentage of WU updates would include the peak gust.

While studying the Cumulus display closely yesterday, though (not while uploading to WU...just studying the display), the "gust" box did not seem to ever keep a gust for 10 minutes - for example, if a gust to 10 mph occurred at 17:40, with lesser readings in the following minutes, the 10 mph gust did not stay in the box until 17:50 - in fact, it seemed to be 5 minutes or less that a high gust would remain in the "gust" box after the gust was recorded (only talking about times in which there were no higher gusts for at least 10 minutes here). Cumulus easily handled every single data packet and change in wind speed/direction that occurred, but the gusts didn't stay in the "gust" box for 10 minutes, or even 5 minutes, for that matter. I wonder if I there could be some setting or option that I checked by accident within the Cumulus software that could be causing this?

[steve]

I can't think of any settings that would effect it. It's possible that there's a bug in the code that handles all this. I don't have time at the moment to sit and watch mine, I'm packing for a business trip. I'll have a look at it as soon as I come back (next weekend).

[steve]

One thing does occur to me - are you using VirtualVP? I have no idea how quickly the data comes back from that. Basically, Cumulus sits in a loop reading the data then sleeping for 100ms. Reading directly from the hardware, this results in an interval of just under 3 seconds. VirtualVP probably returns its cached data much more quickly than that, resulting in many more readings. If you are using VirtualVP, could you watch the update times in the bottom right hand corner and see if they change more often than every two or three seconds?

[NorthNJwx]

One thing does occur to me - are you using VirtualVP? I have no idea how quickly the data comes back from that. Basically, Cumulus sits in a loop reading the data then sleeping for 100ms. Reading directly from the hardware, this results in an interval of just under 3 seconds. VirtualVP probably returns its cached data much more quickly than that, resulting in many more readings. If you are using VirtualVP, could you watch the update times in the bottom right hand corner and see if they change more often than every two or three seconds?

I am indeed using VirtualVP, and the updates are occurring every 1-2 seconds, as opposed to the 2-3 like you said. Interesting.

I'm guessing this explains why the gusts go away so quickly? For example, I just had a gust to 19 mph at 13:18 (not sure how many seconds after 13:18 though), but that gust was gone/replaced at 13:22 and roughly 50 seconds with an 18 mph gust that had occurred at 13:19. So, the gusts are lasting about 4 minutes before they are being replaced.

This also explains something I saw with another station's WU data that uses Cumulus. It had a 5-min update interval, but there were no duplicate gusts at all in the data (as in: a 20 mph gust followed by another 20 mph gust in the next update, since a 10-min gust interval while using a 5-min update interval would overlap for 5 min for gusts and cause roughly half of the uploads to have the same peak gust as the previous upload). Also, through my own tests last night alongside WeatherLink again, the average wind speeds Cumulus uploaded were almost always the same as the 5-min average winds I had WeatherLink generate at the same time interval. Since 200 readings while using VirtualVP seems to be akin to about 4 minutes' time, this makes sense.

EDIT: I just did a per-minute update count to see how quickly I get to 200 updates. In three separate 1-min time periods, I counted 43, 44, and 43 updates to Cumulus, respectively. So, 200 updates are achieved in approximately 4 minutes and 39 seconds while using VirtualVP. That, in turn, would make the average wind an average over 4 minutes and 39 seconds, and the peak gust is the highest gust from the last 4:39.

[steve]

That probably explains it. 200 reads is going to be much less than 10 minutes with VirtualVP.

The sleep interval is configurable via cumulus.ini - edit the file, and in the [Station] section, add a line:

VP2SleepInterval=2000

The value is in milliseconds, 2 seconds is probably a good place to start, and then you can experiment. I've created a new build of Cumulus which increases the number of stored readings to 300:

http://www.nybbles.co.uk/downloads/beta ... sSetup.exe

300 readings will cover 10 minutes at the 2 second interval, so even if VirtualVP returns the data very quickly, it should all be OK. And with a 2 second sleep, you shouldn't miss any data. You may even be able to reduce it slightly, depending on how quickly VirtualVP returns the data.

[NorthNJwx]

That probably explains it. 200 reads is going to be much less than 10 minutes with VirtualVP.

The sleep interval is configurable via cumulus.ini - edit the file, and in the [Station] section, add a line:

VP2SleepInterval=2000

The value is in milliseconds, 2 seconds is probably a good place to start, and then you can experiment. I've created a new build of Cumulus which increases the number of stored readings to 300:

http://www.nybbles.co.uk/downloads/beta ... sSetup.exe

300 readings will cover 10 minutes at the 2 second interval, so even if VirtualVP returns the data very quickly, it should all be OK. And with a 2 second sleep, you shouldn't miss any data. You may even be able to reduce it slightly, depending on how quickly VirtualVP returns the data.

Interestingly enough, after adding that line to the cumulus.ini file, the updates are now coming in every 3 seconds consistently - not every 2 seconds. Updates occasionally come in at 2 or 4 seconds, but almost always at 3 seconds. It appears to me on the display that is causing the occasional missed data packet to Cumulus, since the VP2 sometimes updates at intervals under 3 seconds. I'll try toying with the VP2SleepInterval a bit. I upgraded to the new version as well.

EDIT: Toyed around with the VP2SleepInterval. Results:

@ VP2SleepInterval = 2000: Updates approximately every 3 seconds; 20-21 per minute. Misses a data packet here and there.
@ VP2SleepInterval = 1000: Updates approximately every 2 seconds; 32 per minute. No missed data packets; close to the level that would best produce a 10-min average and gust sampling period.
@ VP2SleepInterval = 500: Updates every 1-2 seconds; 43 per minute. Same as not editing the VP2SleepInterval at all; no missed data packets.

Further tests revealed that VP2SleepInterval = 1100 seems to work the best. It updates every two seconds with few exceptions. It misses zero or close to zero data packets, with my observations showing 29, 30, and 30 observations in three respective minutes. 30 times 10 = 300, so this seems to be the optimal rate.