Cumulus Support

Hi everyone, I recently bought a PM 2.5 & PM 10 sensor, which detects the molecular particulate present in the environment, and therefore possible pollution.
The sensor sends to the network purpleair a whole series of data, to populate the databases of their site www.purpleair.com, however the sensor makes available a json file with all the updated data every few seconds. Here is the example:

DateTime "2019/06/04T20:32:47z"
Geo "PurpleAir-aaa3"
Mem 2012
memfrag 16
memfb 17016
memcs 944
Id 681
lat 45.405125
lon 9.035582
Adc 0.01
loggingrate 15
place "outside"
version "4.02"
uptime 361250
rssi -79
period 120
httpsuccess 18158
httpsends 18162
hardwareversion "2.0"
hardwarediscovered "2.0+BME280+PMSX003-B+PMSX003-A"
current_temp_f 80
current_humidity 40
current_dewpoint_f 53
pressure 998.28
pm2.5_aqi_color_b "rgb(199,249,0)"
pm2.5_aqi_b 46
pm1_0_atm_b 8.37
p_0_3_um_b 1615.23
pm2_5_atm_b 10.99
p_0_5_um_b 470.63
pm10_0_atm_b 11.35
p_1_0_um_b 45.07
pm1_0_cf_1_b 8.37
p_2_5_um_b 3.66
pm2_5_cf_1_b 10.99
p_5_0_um_b 0.48
pm10_0_cf_1_b 11.35
p_10_0_um_b 0
pm2.5_aqi_color "rgb(174,246,0)"
pm2.5_aqi 44
pm1_0_atm 9.25
p_0_3_um 1813.28
pm2_5_atm 10.66
p_0_5_um 484.81
pm10_0_atm 11.35
p_1_0_um 43.53
pm1_0_cf_1 9.25
p_2_5_um 1.68
pm2_5_cf_1 10.66
p_5_0_um 0.65
pm10_0_cf_1 11.35
p_10_0_um 0.4
pa_latency 288
response 201
response_date 15596803
latency 1070
key1_response 200
key1_response_date 15596803
key1_count 12804
ts_latency 1023
key2_response 200
key2_response_date 155968032
key2_count 12791
ts_s_latency 1026
key1_response_b 200
key1_response_date_b 1559680
key1_count_b 12800
ts_latency_b 985
key2_response_b 200
key2_response_date_b 15596803
key2_count_b 12787
ts_s_latency_b 1256
wlstate "Connected"
status_0 2
status_1 2
status_2 2
status_3 2
status_4 2
status_5 2
status_6 2
status_7 0
status_8 2
status_9 2
ssid "saturno"


Why not think about implementing Cumulus even with this data? maybe it is possible to make an apparition with the purpleair network, would it be a possible thing?

Info: http://www.fracassi.net/meteopassione/s ... m25-10.htm
My data: http://andrea.altervista.org/pm10/index.php

tnx! andrea

Nice idea, but I'm afraid it will be way down the to-do list unless someone implements it themselves and contributes it back.

Yes I just installed the same and was wondering how difficult it may be to use the “put” command on this sensor for Cumulus as well. I don't write code, per se, so I’m unable to do it but it doesn't seem altogether difficult.

The temperature data would be unreliable as sensors in the sun read too high, but the particulate data is certified accurate. Seemingly, it’d mean sampling and capturing the data stream to incorporate a part of it to be added to Cumulus updates, as example. Cumulus already has provision for other external sensors.

Anyway, nice thought.

Hi everyone, I have only launched the idea ...
but unfortunately I am not able to help you.
I can tell you that even with the help of a friend who gave me some input I realized this:

http://andrea.altervista.org/pm10/index.php
http://www.fracassi.net/public/banner_pm.jpg
http://www.fracassi.net/aria_gaggiano.php
http://www.fracassi.net/public/webcam.jpg

I just needed some basic PHP information I didn't have, and some examples.

Hello! and thanks.

If it is only PHP-scripting, maybe you could share your implementation of these air-quality sensors?
Code sharing?

USEPA share their findings of the PA against there own certified sensors

https://cfpub.epa.gov/si/si_public_reco ... yId=348488

Wonder if the AirLink will also be tested.

I know the PA maps have a drop-down to select the corrected view from this new formula

Because I am currently working on a Davis AirLink emulator, diving deep into this type of sensors, I would like to make two remarks. The first here, the second in my next post.

1) The basic question in this thread is:

andrea_iw2ntf wrote: ↑Tue 04 Jun 2019 8:38 pm Why not think about implementing Cumulus even with this data? maybe it is possible to make an apparition with the purpleair network, would it be a possible thing?

Given the exercise I am working on, I can say that it might be possible to extend what I call 'hitchhiking the AirLink' to any device measuring the basic PM1 PM2.5 and PM10 together with the temperature, humidity and time. It would just require the user to create an input (could be a plain datafile a kind of realtime.txt or it could be a plain text file containing a json. Tbd).

The emulator would require a definition of the input per device (I don't assume the PA is the only one around) and convert that to the minute values for the AirLink and required averages including the nowcast value. If the user did define a fake AirLink in Cumulus, it would then poll the emulator for values. So getting the values into CMX is possible, given the current situation. As user you could then view your data and graphs (coming up If I am well informed) in CMX.

Getting the data to PurpleAir through CMX would be a different story. And apart from that I don't see why you would not make use of the standard PurpleAir facility to move your data to their system and display it on the map.

Anyway, if there is enough interest, I could create a - lets call it a PMSensor2CMX-utility. The code for calculating the averages and the nowcast is open and can be found on github (see link to my post above). Still under development but reaching maturity as it now runs continuously and has already configurable sensor capability.
On the userside, for any device, there is always the requirement to define the device specific interface and communicate to adapt the software.

2) My second remark is to the article @ConligWX refers to.

ConligWX wrote: ↑Fri 16 Oct 2020 7:29 pm USEPA share their findings of the PA against there own certified sensors

https://cfpub.epa.gov/si/si_public_reco ... yId=348488

Wonder if the AirLink will also be tested.

I know the PA maps have a drop-down to select the corrected view from this new formula

Thanks for the link, I read it with rising astonishment.
I have the following comments.

1. There is no indication to what they see as TRUE values. Their reference is unknown.
2. They focus on the nowcast value. But the nowcast is a calculated value of the measurements made. It is like measuring temperature, calculating the dew-point (had this discussion before) and judging the algorithm by its deviation to some other temperature value series they don't disclose.
3. testing a device like this should be on the level of the sensors used (in case of the PurpleAir that is the Plantower PMS1003 and PMS5003) and that has been done already like here and here.
4. Then, knowing how a device functions, one should verify the - well described - algorithm for the nowcast coming out of the PA.
5. They come up with a correction formula

   Code: Select all

   PM2.5 =0.541*PA_cf1( avgAB)-0.0618*RH +0.00534*T +3.634

   to correct the hourly average for T and RH. Without analysing this equation: any device (also the Plantower devices) have characteristics and correction in their specification for T and RH. I don't see any relation to that specification. OR do they not trust that specification?
6. Any device has deviations, standard deviation, errors etc... this is not the way to handle and judge that. In my opinion it is bad science and should not be used.

So in summary they focus on a correction of the nowcast output of the PurpleAir without any reference to the underlying devices or characteristics.
Do you have a link to the underlying research?

In my opinion, from what I have read so far, the American science on these particle sensors and AQI is about dampening the values and extremes. I do understand the politics behind it but it should be realised what is happening here. Making complex algorithms and subsequent algorithms to correct the result of the first algorithm is politics, not science. Especially if the AQI published is only for the 24h average or the nowcast.

As @beteljuice stated:

beteljuice wrote: ↑Wed 14 Oct 2020 1:56 pm I have mentioned elsewhere that if any level starts rising rapidly and you live next to a chemical factory you may want to go visit Aunty Mary before you see the 24hr value

If we all are going to install particle sensors, that is a great development. But be aware of data manipulations and sensor quality. I think that is more important than accepting any 'correction' that comes from the US. And this includes the use of averages. If my neighbour starts his wood stove at 20h00 and delivers 2 hrs of smoke I have to close my windows. This dampens heavily if you only look at (24 hr) averages. Or even week averages as the PurpleAir has. Yes sustained levels are interesting sometimes but intermittent peaks are as well.

Oh, and I can't find the drop down in the map which uses this correction.

NOTE: There is disclaimer. How should that be interpreted.

Disclaimer
The views expressed in this poster are those of the author(s) and do not necessarily represent the views or policies of the U S Environmental Protection Agency Any mention of trade names, products, or services does not imply an endorsement by the U S Government or the U S Environmental Protection Agency The EPA does not endorse any commercial products, services, or enterprises

It's always good to test devices, but this does not give me any clues.

Edit: Interesting comparison article. It's all about statistics and reference material.

I guess the AirLink will be put under a microscope at some point soon.

I guess they will do that. Hope a bit more info will be published about the reference and method.

NOTE: I said above the specification of the correction for T and RH is in the specification of the sensor. For my PMS1003 it is only for T and it is not in a formula so indeed some research needs to be done there. And apparently they did, though not published.

Hold on guys ...

You must remember that these 'home user' sensors are pretty much toys (Including the C02 and mixed gas sensors)

They are interesting, they give an appreciation of what's going on - but that's it.
As for the AQ calculations ....

@beteljuice:
I deleted my previous answer as i think i misread your remark.
I now understand you regard the AirLink and The PurpleAir as home toys. Yes?

Ok. But what is the reference then?
And what with the AQ calcs?

And yes, agreed for the gas sensors. That is more difficult.

HansR wrote: ...
I now understand you regard the AirLink and The PurpleAir as home toys. Yes?

Correct !

Ok. But what is the reference then?
And what with the AQ calcs?

I have no answers ...

... and as you will have discovered we are only discussing particle size here, and in order to produce an AQI someone has decided what 'normal' pollution is composed of, regardless of location

... also as you will have found from investigating VOC / gas. Every 'component' has it's own AQ classification from tolerable to dire.
These 'home' sensors are rather like the microwave, wonderful idea, sits in the corner, good if you're in a hurry - results, mixed

Good. I think we understand each other.
And to end it:

beteljuice wrote: ↑Sat 17 Oct 2020 9:29 pm These 'home' sensors are rather like the microwave, wonderful idea, sits in the corner, good if you're in a hurry - results, mixed

I guess it depends on the cook. Evidently they do say something...

I don't know if there is a Dutch equivalent ....

Here in the UK, frozen curry = Chicken 'Ding'