Detecting when data isn't received from sensor

[ScottM]

I have problems with my hardware, and I want to be able to detect this from handwritten software. Here's the deal:

I have an Ambient Weather (Fine Offset) sensor and receiver. The hardware apparently has issues, because even over short range, the display/receiver occasionally loses the sensor signal, and doesn't get it back until I power cycle it. I'd like to know when this is happening. The receiver knows, because it puts dashes on the display to indicate No Data. I'm guessing Cumulus knows as well. But what weather underground is reporting back, is periods of time with data that doesn't change (except for pressure, because that's measured by the receiver, not the sensor array, so receive problems don't affect it.)

Right now, I can detect the problem in a sloppy way, by checking what weather underground reports, and noticing if the temperature and wind data have not changed at all for, say, half an hour. But that's a poor technique - that could happen naturally, leading to false alarms.

Is there a better way? Ideally I'd like to be able to open a socket connection to cumulus, and be able to ask it if it's gotten fresh data in the last n seconds (and if so, what it was). Since I don't think that's available, I'm looking for alternatives. (Parsing the files cumulus writes is a technique I'd rather avoid - I could end up reading them just as it's writing them, leading to bad data.)

Suggestions welcome (other than sending Ambient the weather station back, which will happen if they can't clear up the problem with piecemeal replacements.)

[steve]

You could perhaps use the web tag <#SensorContactLost>.

[ScottM]

Ok. It's a little bit of a bulldozer solution, using a web server when I want one byte of info, but it's there and I'll take it.

One issue I've run into: The directory I specify to put the processed file into, isn't set until AFTER the file is moved.

Logging in to calliope
Uploading D:\WeatherForms\w.htmltmp
Remote file: w.html
Changing directory to htdocs/weather <----- too late!
Logging out

So while I can see the file got properly processed and FTP'd up just fine, it didn't land in the directory I hoped for. It's at the top level of my apache install, where I definitely do not want it. What did I miss?

[steve]

It uploads 'extra' files without changing directory, so you have to supply a full path. But you don't need to use a web server at all; use a 'local' file name and don't tick the ftp box.

[ScottM]

Thanks. I have a working solution now.

I still wish Cumulus offered a TCP-based port, to query things from. The code that wants to know about the sensor state is on a different machine, so for now I'm stuck with flinging files using FTP once a minute to ship the data. This means neither machine ever gets to spin down the disk... not ideal.

But for now I'm good. Thanks.

[ScottM]

Ok, I *almost* have a solution.

Recently I discovered Cumulus reporting that "the unit didn't appear to be sending any data". The unit itself was displaying current data, so I have no idea if the problem was the unit or Cumulus, but power cycling everything cleared it up.

My difficulty now is that there's no webtag I can use to detect this state. <#SensorContactLost> doesn't cover this case. I dont' want <#errorlight> because that seems to contain errors that arent' sensor related. I think I need a <#SensorDataMissing>, which is a superset of lost contact and anything else that means I'm not getting good data.

I can do my own analysis on the data to figure this out - if absolutely nothing in the data changes for, say, 4 hours, I can declare an error and take action. But since Cumulus appears to be doing something like this already, I'd like to be able to ask it.

[steve]

'Data stopped' indicates that the station is not talking to the PC. There's an option to make Cumulus restart when this occurs; it's unlikely to fix the problem, it usually requires a reset of the station, but it means that Cumulus doesn't repeatedly write the same data to the logs.

When this does occur, the <#LastError> tag will contain the message "Data input appears to have stopped - check your station and connections", so you could check for that. I'll add <#DataStopped> to 1.9.3.

But none of this is a solution; these are just examples of the many attempts in Cumulus to work around the various problems that some people have with Fine Offset stations, and there's a limit to what I can do in software when the hardware is broken.

[ScottM]

But none of this is a solution; these are just examples of the many attempts in Cumulus to work around the various problems that some people have with Fine Offset stations, and there's a limit to what I can do in software when the hardware is broken.

I know. Believe me, I know. I also write software for a living.

Ambient is shipping me a replacement for the receiver/display, and I'm hoping all of this will become moot. In the meantime, it's nice that Cumulus has ways to reveal this stuff.

Something to consider adding - an option to reboot the whole system, on selected errors. In my case that would help, because the receiver is powered by USB, and shutting down and rebooting would effectively power cycle the receiver. (Or, more generally, the option to spawn a process on selected errors, and then I could write a separate shutdown app.)

[mcrossley]

A reboot doesn't normally remove power from the USB ports does it - only a shutdown.

[41south]

Scott,

Take a look at this thread https://cumulus.hosiene.co.uk/viewtopic.php?f=13&t=7150

I've so far had great success using the ferritte beads on the usb lead. I'm well past the point where I normally lose contact with the base station unit. Still with fingers crossed of course

Cheers
Colin

[Ned]

Even a shutdown may not remove power from USB, something I noticed on my new PC, which I power off every night. Then only by physically unplugging/plugging the USB cable do I hear the console relay operate.

[mcrossley]

True, my laptop is the same - if there is something plugged into the port it maintains power to that port, handy when I am charging my 'phone