VP2 EnviroMonitor data stream fail

[salvatore.pappalardo]

Dear members of the community,
I am setting up five different VP2 with different CMX running.
I am struggling with one of them, which is a VP2 which is streaming data via EnviroMonitor module. I kept the same general settings (Davis WeatherLink Cloud (WLL/WLC), changed the ID station properly but nothing changed.
On the dashboards some daily data are recorded (min/max) but not real-time data.
Is there anybody that have some suggestion?
In case they might be useful, I attach some screenshots.
Thanks in advance.
SP

[mcrossley]

The EnviroMonitor will probably use JSON structures that MX is not currently decoding. The current version doesn't even decode all VP2 options. From what I remember from last time I looked at the sensor catalog the EM has many many different possible configurations.

I may be able to to do something if you switch on debug and data logging, then post the latest MXdiags log after stopping Cumulus for a while, waiting so it has some catch-up to do, then restarting. That will give me a log file with both historic and current data from wl.com to look at.

[salvatore.pappalardo]

Hi Mark,
thank you so much for your response and sorry for the little delay.
I am not sure I have the required informatics knowledge to do what you are asking for, but I do really want to test and implement Cumulus for our urban weather network and I'll do my best to try to fix it. This is what I done, by following your instructions:

0) I am running latest Cumulus MX release 3.28.3 (build 3280) on Windows 10
1) I enabled on "Program Settings">"Logging Options" (Figure 1)
2) I stopped cumulus by terminal (ctrl-c)
3) I re-launched by terminal Cumulus
4) I looked at the directory MXdiags and took the last two .txt files (with cumulus running and with cumulus after restart)
5) Don't know if they can be useful, I also share the console logs.

I attach screenshots here of what I have done, hoping it is possible to make EM data stream.
Thanks in advance.
/SP

[mcrossley]

Thanks for that.

So...

Adding a decode of the ISS historic data is easy, the current code will work, I just need to add a check for an additional sensor type.

Adding a decode of the ISS current data is OK except for rainfall. MX uses the annual rainfall counter to do its calculations. The EnviroMonitor ISS current data only contains a value for today's rain. This is a problem; it would require additional code to work around this limitation.

The data EnviroMonitor gateway itself (historic and current) is completely different to the VP2/WLL data. It would be quite a big job to add this.

The barometer data is the same as VP2 and should already be decoded.

The EnviroMonitor also supports a whole fleet of different sensor types than the VP2/WLL, many that Cumulus will probably never support.


TLDR: Supporting the EnviroMonitor is a significant piece of work, and I'm not sure how many users for it will be out there. If I do add basic ISS support, it will not be a priority at the moment.

[salvatore.pappalardo]

Dear Mark,
thank you for your reply and for your time. As far as I understand decondig ISS historic and current data for all parameter, excluding rainfall, is not just a "dinner party".
I understand the EnviroMonitor gateway is something not completelyt suitable for open-source technology nor open-data, nor open tech-communities, and I completely agree. However, VP2 and Davis weather stations - if properly installed - are at present one of the best option for acquiring accurate weather data.
Unfortunately, EnviroMonitor gateway was adopted in our weather network for all locations without internet connection (5), mostly agricultural areas in the peri-urban ring.
I think that actually many users has opted for VP2+EnviroMonitor gateway as this is a good - but close-mind - solution for having data streaming from remote areas without internet.

I kindly ask you if there is anything we can do to make pressure on Davis for making possible to use EnviroMonitor by Cumulus.
Is there any decoding options or change in protocols that might help you?
Thanks for your time.
SP

[prodata]

I kindly ask you if there is anything we can do to make pressure on Davis for making possible to use EnviroMonitor by Cumulus.
Is there any decoding options or change in protocols that might help you?

I'm afraid that the answer here is likely to be a firm no. The way that EM data is structured in the v2 API is deeply embedded now in the way that weatherlink.com operates. It has to be different, in part because of the huge number of extra sensors that the EM system supports. And the existing EM data structures in the API are quite widely used by others already pulling data from EM stations into their own software and so these users will very much want to retain the existing data structures.

[mcrossley]

Hi John, I still think it will be a no, but it would only require the addition of a yearly rainfall figure, rather than changing anything existing. The EM only supplies a rainfall today figure. But the answer from Davis is always "V2 API only supplies what the station sends, not any additional computed data like the V1 API". I'm guessing the EM does not provide an annual rainfall figure directly?

[prodata]

TBH I don't know exactly what the EM gateway sends - AFAIAA it's not documented anywhere that I have access to. But given the sheer number of different sensors that can attach to an EM system, I think that the only practical way for the EM nodes and gateway to operate is to perform only limited processing of the sensor data before the gateway uploads the data - it really is a somewhat different concept to a console.

That said, a rain gauge attached to an ISS is effectively a special case on EM stations because the ISS is connected directly to the EM gateway rather than to an EM node but even so I can well imagine that the same strategy still applies. What Davis are saying (about mostly only making available what the station uploads) does seem to be very much an article of faith with them and something that they are unlikely to deviate from.

There are quite a number of rainfall-related fields in the EM Current Conditions data structure (Type 6) so the gateway is obviously generating all of those, but yes the monthly and annual totals are missing if that's what you need for CMX (though I see that they are there for the 6313 console). But I guess you are more interested in the archive records? And I agree that the Archive Type 7 record is minimal for rainfall. For our Prodata Reports software for EM stations we poll the archive records at hourly intervals and then maintain full (ie including a running daily total) rainfall records for customer stations in our own PR database on our central server (some managed DO droplets).