Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Hitchhiking with the Airlink interface of CMX

Discussion of the Cumulusutils tool and website generator.

Moderator: HansR

User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Hitchhiking with the Airlink interface of CMX

Post by HansR »

Yes, the Wiki is not very clear on this one. I will change it according to what I explain below.

CumulusMX knows the following country settings:

Code: Select all

	US, GB,  CA, EUAQI, EUCAQI, AU, NL, BE
I left the type indicator away except for the EU so USA-EPA becomes simply US.

CumulusUtils knows the following countries:

Code: Select all

	US, GB, CA, EU, AU, NL, BE
These match one to one which I think is easy to understand except for the EU. Both EU setting from CumulusMX are projected on EU in CumulusUtils.
These country systems are explained in this post here.

As you can see in the forum post, the values for the two EU AQI standards do not differ that much so I merged them into EU-CAQI.
The choice of these AQI calculations leads in CUtils to the reference lines. CUtils does chart the concentrations only, not the AQI.

The Canadian system is the only one which uses a formula and not a table. That is why it only accepts 3 hr averages.

To program the exact AQI representation at certain concentrations for specific particle size is according to me an incorrect presentation of the danger levels involved (apart from the complexity in doing so). Danger levels involved must be uniform and the same everywhere in the world, not dependent on politics of a country (compare e.g. the danger levels of the US to those of the EU). So CumulusUtils deviates here from CMX and It may in future even remove the AQI per country completely and replace that with the WHO reference which recently has become available. (see my link above)

As said, I will add this to the Wiki.

I assume you wish to use the GB settings, meaning that both in CMX and in CUtils you need to set GB (I see you did that already :) )

To make one step forward: I notice that the chart is not fluent which means it does not get the data fed regularly. Feeding the data in an orderly manner, you need to use the UserAskedData command on the interval frequency. So assuming you already run CumulusUtils on that frequency for e.g. SystemInfo, I would advise you to add UserAskedData to the commandline.

[EDIT] The wiki has been modified, you may wish to read/check.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

Hans,
I already have the following in my crontab :

Code: Select all

1 0 * * * cd /home/pi/CumulusMX; utils/bin/cumulusutils.exe thrifty website ; cd /home/pi/scripts
9-59/15 * * * * cd /home/pi/CumulusMX; utils/bin/cumulusutils.exe sysinfo UserAskedData
Isn't that suffient...?

For the country issue I have set

Code: Select all

[AirLink]
CountrySelected=1
in cumulusutils.ini to match

Code: Select all

AQIformula=1
in cumulusmx.ini which seemed to get rid of the error message. Using text always gave an error message.
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Hitchhiking with the Airlink interface of CMX

Post by HansR »

watsonm wrote: Tue 09 Aug 2022 2:57 pm Hans,
I already have the following in my crontab :

Code: Select all

1 0 * * * cd /home/pi/CumulusMX; utils/bin/cumulusutils.exe thrifty website ; cd /home/pi/scripts
9-59/15 * * * * cd /home/pi/CumulusMX; utils/bin/cumulusutils.exe sysinfo UserAskedData
Isn't that suffient...?
Yes, that is sufficient although I fail to understand why the datapoints are 30 minutes apart if you run it on a 15 min cycle.

watsonm wrote: Tue 09 Aug 2022 2:57 pm For the country issue I have set

Code: Select all

[AirLink]
CountrySelected=1
in cumulusutils.ini to match

Code: Select all

AQIformula=1
in cumulusmx.ini which seemed to get rid of the error message. Using text always gave an error message.
The CMX setting AQIformula is set in the interface and you will find that back in the inifile as a number. So if you see 1, apparently you chose the US.
The CUtils setting in the AirLink section for CountrySelected must be a two character string representing the country. If, during the conversion to compare with the CMX setting, it fails somewhere it defaults to EU.

Both issues can be see in the logfile.
Can you send me both: one for the full run when creating the AirLink module (so with the command AirLink) and the one with the UserAskedData command. Set logging on and on level 'info'.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

If I select USA then it sets AQIformula is set to 0
If I select UK then it sets AQIformula is set to 1


Attached log is for a run where I selected UK and it set AQIformula is set to 1
and I set
[AirLink]
CountrySelected=UK
in cumulusutils.ini
You do not have the required permissions to view the files attached to this post.
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Hitchhiking with the Airlink interface of CMX

Post by HansR »

Mike,

OK, my bad. index array starts at 0 so US indeed is 0 and UK indeed is 1.

Second issue: CUtils uses GB for UK. Highly confusing I agree. The log indeed shows this.
I suggest you use GB in CUtils configuration for now.

I will change the code GB to UK in CUtils such that it is in line with CMX and change the Wiki accordingly.
A new release will be done soon (with some other minor things).

(How did this get unnoticed?)
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

Hans,
GB being used and working, thanks :geek:

edit: Also added code to calculate Nowcast.
You do not have the required permissions to view the files attached to this post.
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

UK now working with latest build. :clap:
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Hitchhiking with the Airlink interface of CMX

Post by HansR »

Good :D
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

Hans,

As usual I am probably looking in the fiction section of Google rather than than non-fiction section :roll: but I can't seem to locate anything about the "1 hour filling grade" as listed on the extra ->Airlink page . What is the filling grade?

(28 degrees sat by the PC!!)
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Hitchhiking with the Airlink interface of CMX

Post by HansR »

Mike,

(Answering this by 29.5 degrees and increasing :| )

The Davis spec gives the answer:
After power loss or in rare circumstances after a soft restart, the data used to compute PM averages may be lost and require 24 hours to fully refill. The pct_pm_data_xxx fields can be used to estimate the time interval actually covered by each average.
Sorry if filling grade is not a real English term - nobody made a remark - but it stands for the percentage the buffer for the calculation of the averages has been filled. So the 1 hr average requires 60 minute values, the 3 hr average requires 180 minute values etc....

For the now-cast the wiki says :
Because the most recent hours of data are weighted so heavily in the NowCast when PM levels are changing, EPA does not report the NowCast when data is missing for c1 or c2.
so we only start calculating when two hours exist in the buffer for the 12 hr estimate.

The percentage is calculated and stored in the datastructure which is sent by the webserver:

Code: Select all

now => 1 sample (not an average)
1 hr => 60 samples (100%)
3 hr => 180 samples (100%)
24 hr => 1440 samples (100%)
now-cast => 720 samples (100%), only displayed when 120 samples available; 
Note that for the now-cast, CUtils does not obey the 120 samples rule and leaves it to the user to use the value or to discard it on the basis of the percentage filling.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

Hans,

Thanks. Will update that parameter soon!

I save the PM data in files so usually a restart has completely full buffers.
WIll flush one as a test!
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Hitchhiking with the Airlink interface of CMX

Post by HansR »

watsonm wrote: Sat 13 Aug 2022 11:18 am I save the PM data in files so usually a restart has completely full buffers.
No you haven't if you're out more than a minute. The empty spaces do not count, they have no value.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

I understand, will work on fixing that.

(save time stamp of most recent entry and calculate time difference and pop oldest elements appropriately)

Edit: maybe push -1 entries instead. Thinking cap on.
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
watsonm
Posts: 208
Joined: Sun 03 Jan 2016 3:39 pm
Weather Station: N96GY -but like Triggers Broom!!
Operating System: Raspberry Pi4 Vers:11 Bullseye
Location: Poundbury, Dorset

Re: Hitchhiking with the Airlink interface of CMX

Post by watsonm »

Hans,

Is cm utils pulling the filling grade from websitedata.json and if so what parameters is it using. I have added

Code: Select all

"pct_pm_data_last_1_hour":"73.3",
"pct_pm_data_last_3_hour":"53.3",
"pct_pm_data_last_24_hour":"89.375",
and it is not getting through.
Regards Mike
Image
Raspberry Pi 4 Bullseye
Directly connected : BMP388 pressure sensor. DHT22 For internal Humidity/temp Sensor. PMS5003 Particle Sensor
Wireless connected: 3 (WS2083)Temp TXr's, WH57 Lightning Sensor, FO Solar sensor
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Hitchhiking with the Airlink interface of CMX

Post by HansR »

Cutils does not use websitedata.json.

CMX reads the data from the json structure you send by the webserver and stores it in the database.
CUtils reads the database.
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
Post Reply