Davis forecast text strings

[BCJKiwi]

UPDATED May 20 2013
As part of the vueconsole coding the forecast icon needed to match the Davis forecast strings.

The Saratoga scripts are used and so decided to use the langtransstr() function and added some extra chars to the front of the forecast string so I could use the translation to match the icon name to the forecast string.
This works fine but immediately I found strings that were not in my list of some 100+ lines.
It also looks like many of the short strings like 'Windy' or 'Mostly cloudy' are unlikely to appear on their own but generally on the end of something else.
So I came up with a little bit of php code to generate a list of all the strings as they turn up. These get added to the end of language-en.txt. This will over time generate a list of all the actual strings that a console generates.

Obviously it is going to take forever for my system to come up with all the strings so I wondered if those interested might add this bit of code to their main script. I have placed it in the ajax-dashboard.php just before the WU forecast section but it could be placed anywhere that is frequently run/refreshed (would suggest only once on your site).
Then say once per month the results could be posted and combined into a single file.

EDIT - Modified snippet in attached file - Found the file did not handle multi language setups well!
Attached snippet will write forecasts to the current language file instead of always to the 'en' file.

It will generate a line like this at the end of language-xx.txt;
langlookup|X_Increasing clouds and cooler. Precipitation possible and windy within 6 hours|new|
There will not be any repeats in each language file.
Of course you can use this to fine tune the forecast translation list in all your language translation files as required.

Thanks

[steve]

Isn't the list of all possible combinations already known, as it maps to the forecast number in the LOOP data? As posted previously to the forum, for example here: https://cumulus.hosiene.co.uk/viewtopic.p ... &start=705

[BCJKiwi]

Well no it's not.

In previous discussions there was criticism that the list of 100 or so that I had come up with was too long.
However as indicated in the previous post, immediately I started running the vueconsoleCU script, forecast strings showed up that were not in the list.

What is known and documented so far are all the sub phrases but only some of the combined strings.
It is the full set of combined strings that is required to make the standard Saratoga translation system work properly. The use by the vueconsoleCU script is just an adjunct to the primary need for the full set of strings for translations.

[steve]

Sorry, I thought that the list I referred to was the full list. As I said, it maps on the the 'forecast rule number' in the LOOP data, so it's what any software would use if it wanted to convert the rule number into text. Hence, as far as I'm aware it's the mapping that the Davis DLL used by Cumulus uses (with the small proviso that it appears to modify some of the wind directions depending on location) and hence it's the list of possible forecasts supplied by Cumulus - which is what I thought you wanted.

This has all been discussed before many times, and I'm getting tired of repeating myself. So, apologies for misunderstanding and interfering, I'll go back to minding my own business

[BCJKiwi]

I have re-read the previous posts you linked to and in which I played a part.
It appears to confirm my current understanding as it indicates that the list is incomplete.
It was from that thread and the combined input of those contributing, that the list I have been using of around 100 forecasts was derived. As I operate in English it is not obvious when a string is not translated.
In the last two days the code snippet posted has already added 5 more forecasts to the list and the weather has not changed all that much. The most recent addition is;
Mostly cloudy and cooler. Precipitation possible within 12 hours, possibly heavy at times. Windy.
which is an example of how these multi sentence forecasts are built up adding to the list.
This is the core issue - while there is a long list of 'sentences' many are very short and as far as I can tell will never actually appear in a forecast string as delivered to a website. e.g. I have not seen and can't conceive of an entire forecast existing simply of "Windy".

It just seems the simplest way to get the full list of actual forecasts is to accumulate them as delivered by Davis stations in the field.
However for this to be successful it would need to be a collaborative effort across all seasons and climates.

[mcrossley]

But as we do know all the component strings, can't you translate each of these individually where you find them within the composite forecast string?

[steve]

I have not seen and can't conceive of an entire forecast existing simply of "Windy".

Because that's not on the list.

It just seems the simplest way to get the full list of actual forecasts is to accumulate them as delivered by Davis stations in the field.
However for this to be successful it would need to be a collaborative effort across all seasons and climates.

The station doesn't supply the forecast string, the software does, as I keep saying. The result you will get will depend on the software you use to generate the forecast. But the assumption is that all software has used the same well-known list of just under 200 strings. I don't know who first came up with list and what software they were using; Weatherlink, I imagine.

[BCJKiwi]

Well now you have me really confused.
Windy appears in many of the lists. Those lists purport to come from Davis and as you have stated "Cumulus doesn't produce the text for the forecasts, it reads them from the Davis Vantagepro.dll."
I have equated the "station" with "Vantagepro.dll" as in what Cumulus delivers is derived from Davis equipment/code.

It's very late now in New Zealand and I will need to review and look again at all this with a clear head in the morning.

Thanks for your patience

[steve]

Windy appears in many of the lists.

Windy (on its own) does not appear in the list I referred you to, which is the list of complete forecast strings generated by any software which follows that list (obviously). I'm not aware of an official version of that list, my understanding is that someone worked it out a long time ago, presumably by force-feeding Weatherlink the 190-odd valid codes and seeing what it spat out for each one.

Those lists purport to come from Davis and as you have stated "Cumulus doesn't produce the text for the forecasts, it reads them from the Davis Vantagepro.dll."
I have equated the "station" with "Vantagepro.dll" as in what Cumulus delivers is derived from Davis equipment/code.

I'm equating the DLL with software as it's not the console, it's what does the mapping of forecast rule number to a forecast string on behalf of Cumulus.

The important thing to remember is that the Davis communication protocol only allows for a forecast number (0 to 200) to be retrieved from the console, it does not allow for any forecast text to be retrieved. The software, whether that's an executable or a DLL being used by an executable, has to map that number onto a forecast string.

[beteljuice]

.... coding the forecast icon needed to match the Davis forecast strings.

So why not search for 'keys' in the string and prioritize them.

eg. cloud* subsets [null], [mostly / increasing], [clearing], [partly]

precip* subsets [null], [heavy]

Of course you would also have to 'say' rain is more import than cloud etc.

Think like the various METAR decoders that give condition icons

[BCJKiwi]

Windy (on its own) does not appear in the list I referred you to, which is the list of complete forecast strings generated by any software which follows that list (obviously). I'm not aware of an official version of that list, my understanding is that someone worked it out a long time ago, presumably by force-feeding Weatherlink the 190-odd valid codes and seeing what it spat out for each one.

There are published lists of forecast sentences pulled from the dll which show this. However (as per your further advice below) these likely don't appear in a forecast on their own as I had surmised. Mark Crossley also published a list of the individual strings which contain sentences that do not appear anywhere in the '200'. Then I have another list supposedly from dll v2.4.2 which is different again.

The important thing to remember is that the Davis communication protocol only allows for a forecast number (0 to 200) to be retrieved from the console, it does not allow for any forecast text to be retrieved. The software, whether that's an executable or a DLL being used by an executable, has to map that number onto a forecast string.

This is new and useful information to me. I guess I missed wherever it may have been stated previously.

The list of 198 (0 thru 196 plus 200) strings that match the numbers contain a lot of duplicates. If the duplicates are removed the list contains 81 items. I presume the duplicates are there to facilitate the building of the full strings by the dll from the number keys?

If these various lists are combined we get a long list which definitely DOES NOT contain forecasts which are appearing on my system, nor do they include any of the typos that have been found over time.

So I can only conclude that while the principles and theory are all fine, the reality is unfortunately different.
Also I can see no other simple way of capturing all the actual strings that turn up than that I have already proposed.

[BCJKiwi]

.... coding the forecast icon needed to match the Davis forecast strings.

So why not search for 'keys' in the string and prioritize them.
eg. cloud* subsets [null], [mostly / increasing], [clearing], [partly]
precip* subsets [null], [heavy]
Of course you would also have to 'say' rain is more import than cloud etc.
Think like the various METAR decoders that give condition icons

Well I guess you could do something like that but it seems an unnecessary complication. One would have to effectively reproduce what algorithms the Davis console firmware is using to come up with the forecast icons from all the component parts of the forecast.
Using langtransstr() only requires applying the relevant icon name to the forecast string.

This thread was not aimed at the icon issue but at getting an actual list of all the forecast strings that do turn up and thus avoiding the recurring discussions about how it all works as we have just been through again in this thread.

[steve]

Also I can see no other simple way of capturing all the actual strings that turn up than that I have already proposed.

Just bear in mind that the list may vary depending on the software used. If you want a list specifically for Cumulus, you have to make sure that Cumulus (or some software using the Davis DLL) is used to populate the list. There's also the fact that the wind directions appear to come out differently depending on location.

[BCJKiwi]

Agreed Steve.
I think that does support the concept of collecting real data from working systems.
Then a full list will become available that is no longer dependent on systems or geography and is useful to all - the only requirement being the use of the Saratoga langtrans() functionality.

Anyhow, I have no idea if anyone is the least bit interested in assisting in this process as since I posted the code in line instead of as an attachment I can't tell if anyone has downloaded it let alone implemented it.

[steve]

What would be useful would be a bit of code to simulate a station (at least partially) and get it to return all possible rule numbers in the LOOP packet, and then see what the DLL maps them onto. Not trivial, though. I may have a go if I get very bored.