Page 1 of 1

WU bulk update of Cumulus data

Posted: Mon 08 Dec 2014 3:37 pm
by KarlS
I have been running Cumulus since October 2010, but never bothered to upload the data to Wunderground. A few days ago I finally registered my weather station with WU and now Cumulus updates my account every 10 minutes. Now: wouldn't it be nice to be able to upload all my historical data to WU?!

I searched the WU site but found no answer to my question. On the WU Wiki I found the PWS upload protocol and wrote a small script to read the Dec14log.txt file and generate the appropriate HTTP requests. I successfully uploaded the missing first 4 days of December; however it took about 25 minutes to process the about 600 log records. I don’t know if this is due to my rather slow internet connection, but uploading the remaining 220,000 log records this way seems ridiculous.

Has anybody had this problem before and found a 'doable' solution?

Re: WU bulk update of Cumulus data

Posted: Mon 08 Dec 2014 3:49 pm
by steve
I don't think WU has a 'bulk upload' facility, so I think that however you do it, the limiting factor will be the speed of response of the WU server to each update. I typically find an update takes around 5 seconds, and you're getting twice that speed.

I wonder how the WU server would respond if you sent multiple updates in parallel?

Re: WU bulk update of Cumulus data

Posted: Mon 08 Dec 2014 4:12 pm
by mcrossley
There is no bulk upload that I am aware of, I have a spreadsheet that performs the uploads from the data files, it varies but a couple of updates a second is probably the best I have got out of it.