A couple of broad SS questions:

[prodata]

I can't quickly find this explained anywhere: realtime.txt is obviously the Cumulus realtime file in a very specific CSV format.

But do I gather that customclientraw.txt is actually a JSON file and while eg targeted at WD users, this is effectively a generic format so that any other software that creates an equivalent file of name:value pairs will work equally well as a source of data for the SS gauges, provided of course that the standard variable names are used and the correct JSON syntax is observed?

And presumably (??) it doesn't matter if some name:value pairs are missing (if they're not being used in that particular gauge presentation) or if the order of the name:value pairs varies, provided again that correct syntax is observed? And if I'm guessing correctly then perhaps some sets of name:value pairs could be uploaded in one file (eg ccr1.txt) and others in another (ccr2.txt) subject of course to some suitable amendment of the JS that reads the files? (Maybe they don't even need to have different filenames - if the same filename was used then some variables would be updated by one upload and others by a subsequent upload?) Or am I misunderstanding how JSON works?

So the reason for this last question is wondering in general about how best to update web page presentations for different types of weather parameters. Most parameters only change measurably on a timescale of eg 1 minute, eg temperature, humidity, pressure or like solar/UV and rainfall are limited - eg in the context of a Davis station - by sensor updates to the same 1 min timescale. (It takes pretty heavy rain of >12mm/hr to score >1 0.2mm tip/minute.)

But the one parameter that does update much more frequently is wind speed/direction for which a 2.5 second granularity (Davis again) would be needed to show all the available data. But it really seems like overkill to upload the whole of realtime.txt or equivalent literally every 2.5 secs. Hence the thought about the possibility of maybe uploading only wind data every 2.5 secs and then everything else eg once per minute or even less often.

An alternative approach that might be possible with the JSON type file would be to assemble a set of eg 12 or 24 wind pairs into a JSON array and upload this as part of the whole file every 30 or 60 seconds. OK this would require some further adjustment of the SS code, but perhaps not too much, and exact synchronicity with the actual wind measurements would be lost - that wouldn't worry me too much ( after all, there's always some latency in the overall process and I'm not sure it really matters anyway), but it might concern others more.

[mcrossley]

John

The end result of all the different data files is a JSON blob of data that is essentially the same across all applications. So your can take the customclientraw.txt or the realtimegauges.txt the content should be the same. If you can generate that JSON by any means, the gauges will work.

The order of data in the JSON data is irrelevant - apart from the "ver" value, I place that at the end as a rudimentary check that a complete data set is present.

"it doesn't matter if some name:value pairs are missing"
- well only if they relate to data that you aren't going to display because you have removed the related gauge.

"And if I'm guessing correctly then perhaps some sets of name:value pairs could be uploaded in one file (eg ccr1.txt) and others in another (ccr2.txt) subject of course to some suitable amendment of the JS that reads the files?"
- No reason why not, but as the downloads happen asynchronously, the code would have to check/wait for both downloads to complete before proceeding.

I'm guessing that the thrust of this is to limit the upload bandwidth to the web server? There are approaches you can take to limit that if you accept some processing on the web server. For instance, I upload three data files from Cumulus; a realtime data file, a 'rest of the data' file, and all the 'monthly/annual records' file. The latter two updated every 5 minutes, the realtime every 5 seconds.

For example, if you request the 'realtime.txt' file from my web server, it is actually generated on the fly by PHP using data from the realtime data file, and the 'rest of the data' files. So I avoid having to upload two 'realtime files'.

Likewise, I do not upload a realtimegauges.txt in realtime, the gauge JSON is again created 'on the fly' by PHP from the three other data files I upload.

Sending batches of wind data in a singe update, and then 'trickling them out' to the gauges between updates is what I am doing with that Instromet display. It is not without its problems getting the timing somewhere near.

A different approach altogether is outlined in my thoughts here where you don not need to do the upload data at all if nobody is using it. You stream the data directly from Cumulus to the web server as and when it changes. You could also do the same onward to the client, then your wind data in your web browser could be updated almost as fast as it is read from the station.

I might come back and proof read this again a bit later!