Cumulus behaviour when sensor contact lost

[BCJKiwi]

Looking for a way to indicate on a website (preferably via some change in realtime.txt) when contact is lost between the console and the sensors.
In this case for a Davis Vue.

As there is no "lost sensor contact alarm" for Davis and none of the Davis specific tags help, I was wondering what exactly happens to the data in realtime.txt when sensor contact is lost.
Does realtime stop being sent?
Does realtime continue being sent with the same data - if so does the date / time remain static or does it change?
Does realtime continue to be sent but with zeroes (or --- ?) in the fields.
Knowing what happens to realtime.txt should enable a way to determine if sensor contact has been lost.
(I did try masking the antenna with various metal tubes but was unable to stop reception only lower the quality by around 15%!)

Thanks

[steve]

With Davis stations, there is no overall 'contact lost' indication, each individual item returns a 'no reading' value. I'm not actually sure how the Davis DLL handles this.

Cumulus just keeps re-using the last valid data that it read, and carries on as normal. When I wrote Cumulus - for myself - I decided that any loss of communication would be very short term, as it would either be a temporary dropout, or it would be something I could attend to myself quickly. So that solution was acceptable to me. With thousands of people now using Cumulus, in many different situations, it is arguable that the 'solution' requires a re-think

[BCJKiwi]

Thanks for that Steve.

If Cumulus just keeps using the same data then I can use that by checking for changes.
I had thought that might be the case so already have a routine than sums the first 8 numeric data fields in realtime.txt (the fast changing ones i.e. not items like highs and lows etc.) and stores it. It then compares it on the next update. The assumption is that if nothing has changed, it is not updating. The application is to change the blinking data received symbol that I now have on the Davis 'Live' Console but of course could be used anywhere.

Presumably the realtime.txt time (second field) still updates? If it became static, it would be the simplest thing to check.

[steve]

Presumably the realtime.txt time (second field) still updates?

Yes, everything just carries on as normal. Even if the outdoor sensors lost contact, the indoor items would still be updating.

[mcrossley]

An added complication for Davis and OS stations is that can be more than one remote transmitter. I have my wind speed and direction on one transmitter and T/H, rain, solar on another. I did wonder at one time if the remote battery status could be useful, but Davis uses 'OK' for transmitters that are really OK, and those that are missing, argh! Who's bright idea was that. The packet received/missed stats (availble through Cumulus anyway) appear to be only for the main ISS transmitter so they are out too.

[steve]

I did wonder at one time if the remote battery status could be useful, but Davis uses 'OK' for transmitters that are really OK, and those that are missing, argh! Who's bright idea was that.

There is actually a function in the DLL which returns a list of which sensors are present, so Cumulus could parse the status string and adjust it. Assuming the function works. It's a shame the DLL doesn't do it anyway.

The packet received/missed stats (availble through Cumulus anyway) appear to be only for the main ISS transmitter so they are out too.

Yes, it appears to be only for the ISS, but does include the wireless anemometer if one is in use. Well, actually, it says: "Fills in the receptionStats parameter with the current ISS or Wireless Anemometer reception data. " so I'm not quite sure exactly what that means.

[BCJKiwi]

The remote battery status is not a good indicator anyway as it frequently misreports the status of the battery, and, the transmitter will work long after the battery is below the trigger voltage for the battery indicator.

Have a routine which happily sums the fast changing data with parseFloat and is currently returning a number around 1150.2 BUT will have to remove the inside data items!
The issue I have right now is that this is in the javascript module and I don't yet have it storing the previous value as each pass starts fresh and so the last total is lost - any suggestions?

[mcrossley]

I did wonder at one time if the remote battery status could be useful, but Davis uses 'OK' for transmitters that are really OK, and those that are missing, argh! Who's bright idea was that.

There is actually a function in the DLL which returns a list of which sensors are present, so Cumulus could parse the status string and adjust it. Assuming the function works. It's a shame the DLL doesn't do it anyway.

Maybe, but do we know what the behaviour is for a sensor that is 'registered' but has now gone away?

The packet received/missed stats (availble through Cumulus anyway) appear to be only for the main ISS transmitter so they are out too.

Yes, it appears to be only for the ISS, but does include the wireless anemometer if one is in use. Well, actually, it says: "Fills in the receptionStats parameter with the current ISS or Wireless Anemometer reception data. " so I'm not quite sure exactly what that means.

I checked when I was setting up my wireless anemometer for reception levels, the station shows separate 'screens' for the two transmitters, the stats Cumulus was showing were for the ISS only - which I moved to channel 2, and put the new wind transmitter on channel 1.

[steve]

Maybe, but do we know what the behaviour is for a sensor that is 'registered' but has now gone away?

No, but I would guess that it would still appear, having been configured.

the stats Cumulus was showing were for the ISS only - which I moved to channel 2, and put the new wind transmitter on channel 1.

Well, that does agree with the documentation (ISS OR anemometer). The serial protocol spec says the same thing.

[steve]

The serial spec also has this to say:

"The value in the “Dash Value” column is what you will see if that field is not updated at all during the archive interval. A dash value can appear for several reasons, and different weather variables are treated differently. For example, if you see 32767 for Outside Temperature that could be because of a communication problem, the sensor was unplugged, or the sensor has failed. Note, a dashed value is not always the sign of a problem. For example, the rainfall reading could be 0 if no rain fell in that interval. To determine if a problem exists, often times you will need to look at more than one weather variable."

Which is what Cumulus would have to do to reliably report communication problems.

[BCJKiwi]

To determine if a problem exists, often times you will need to look at more than one weather variable

Which is effectively what combining (by adding together) a number of the variables coming from the ISS to see if there is no change would do.
Isn't it reasonable to surmise that if there is no change in a 'basket' of ISS variables, that there is no longer a flow of information?

As indicated, I have this total working but since it is in a javascript module I don't seem to be able to use it as I don't see a way to pass it back to a variable so it can be compared to a new value later.

The only alternative I see right now would be to send a tag file of just those variables to php to generate the 'sensor lost contact' variable for use by the website.

[mcrossley]

The remote battery status is not a good indicator anyway as it frequently misreports the status of the battery, and, the transmitter will work long after the battery is below the trigger voltage for the battery indicator.

I had hoped that it would be a four or even five state:

five state...
OK - OK
Low - Battery warning
Failed - Battery bad, but running on Solar/Super Cap.
NoSig - Lost contact
n/a - channel not in use


four state...
OK - OK
Low - Battery warning
Failed - Battery bad, but running on Solar/Super Cap.
n/a - Lost contact or channel not in use

At least that is the way I would have implemented it.

[steve]

Isn't it reasonable to surmise that if there is no change in a 'basket' of ISS variables, that there is no longer a flow of information?

Yes, with the information that Cumulus currently supplies, that's probably the best way to do it.

If Cumulus did it, it would be able to do it more reliably, as it would actually see the 'dash values' rather than just having to rely on data values not changing, as you are having to do. Assuming the DLL does pass the 'dash value' information on in some form.

[BCJKiwi]

Ok, I now have "rainevent" and "sensorlost" routines running in the Cumulus PC, and sending the data to the (development) website. I'm also logging (at the website) the date, time, the var and in the case of the sensorlost, the sum of the tags, and in the case of the rainevent, the rain and rainevent values.
The rainevent has been through a few actual rain events but has also been tested with data of actual rain that has (and is still) being logged for some time as it happens.

The sensorlost routine is summing these tags:- <#apptemp> <#avgtemp> <#heatindex> <#humidex> <#hum> <#dew> <#wchill> <#wlatest> <#bearing> at each tag processing interval as they seem to be the only ones directly related to ISS data and that change frequently.
The routine replaces a --- with 0, replaces comma decimal with point decimal, adds them all together and compares the result with the last result then sets the sensor contact lost var to 0 or 1.

So far it has been running for over 24hrs and the closest data (around 3am) with no wind, was a 0.1 change for each of three cycles. So no matches so far.

I'm wondering further about the --- option. If sensor contact is lost and the ISS data all changes to ---, which of these data points would actually be passed on by Cumulus as ---? If more than a few, then a simpler test would be just to check to see if all of those are ---.

Steve, perhaps this was what you were getting at in your last post?
Once I'm sure these routines are doing what they should I could pass them along to you so you could see if it is feasible to include what they do into Cumulus rather than the 'lash-up" that I currently have.
This involves a ...T.txt file processed to fill the tags and save the result as a program. A batch file runs the program producing an output file as a php snippet with the var as a php var and FTP's the file to the website, the website 'includes' this snippet which makes the variable available to the website.

This same process is carried out for both routines. It would obviously be much tidier if it were part of Cumulus but these routines will do the job as for as long as required (provided they pass the testing!).
As they both work on tags, they will of course work for any station supported by Cumulus.

Any further thoughts/suggestions?

[steve]

As I said in my previous post, if I did this in Cumulus, I wouldn't do it this way, as Cumulus has access to the 'dash values' which let it know that contact really has been lost. I'm assuming when I say that, that the DLL does pass on the 'dash values' in some form, rather than doing what Cumulus does and re-send the last good values. I would think that assumption has a good chance of being correct.

In the 'new' version of Cumulus, if/when that ever finally appears, it has direct access to the LOOP data, so definitely does have access to the 'dash values'.