AirQuality charts

[AndyKF650]

Hi Hans

I have taken the plunge and gone to an EcoWitt gateway with an external temperature sensor and a CO2 AirQuality sensor.

After some work I have got the sensors talking to CUtils and recording data in realtime which now shows on my website.

I have managed to get the temperature sensor to graph its results but try as I might I cannot get the AirQuality data to graph. I worked through the wiki CDL pages and that suggested that a plot of AirQuality1 and CO2 should work but not for me.

I have attached the CUtils.ini, Charts.def and a compileonly log file for information.

Please can you point me in the correct direction.

[HansR]

Hi Andy,

Good plunge

As far as I can see, you did enable the CO2 sensor correctly: I see the values but not the chart.
You may disable sensor 2,3,4,5 and the AirQuality1 sensor. Apparently you don't have those.

I'll come back on the CO2 chart.

[HansR]

OK, reading the cumulusutils.ini I have the following remarks:

1. The ExtraSensor parameters must be as below:

   Code: Select all

   [ExtraSensors]
   ExtraSensors=true
   ParticipatesSensorCommunity=true
   ExternalExtraSensors=
   ExtraTemp=1
   ExtraHum=1
   ExtraDP=1
   SoilTemp=
   SoilMoisture=
   AirQuality=
   UserTemp=
   LeafTemp=
   LeafWetness=
   CO2=1
   LightningSensor=false
   UserModificationExtraSensorCharts=false
   CleanupExtraSensorslog=false
   

   Note the sensor numbers are the numbers as in CMX, only for the CO2 it has a number where CMX has none (since CMX only supports one).
2. The parameter UserModificationExtraSensorCharts needs to be false as long as you are setting up the sensors and verifying they are correctly working. When done you can set it to true and combine e.g. the three Extra sensors nr 1 plot variable into one chart (or change the line colours as blue is not really nice against your background ).
   
   The effect of UserModificationExtraSensorCharts is that the ExtraSensor charts will not be overwritten when running CUtils and as a result you can modify the charts to your liking. When adding sensors having this parameter to true, you will have to add the charts yourself using the EXTRA Plot Variable definitions from CDL.
3. While setting up in your website configuration it is easiest to use only the commandline argument 'ExtraSensors' which makes experimentation a lot faster.

Let me know how you get on.

[AndyKF650]

Hi Hans

Thanks for the pointers, I changed the suggested lines and ran the extrasensors cmd line and suddenly the mist cleared and now I have some interesting charts for th e new tech

As you say I will probably need to sort out colours and fine tune narratives but at least I can see something rather than nothing!!!!

[HansR]

Good

[AndyKF650]

Hi Hans

I think I have gone as far as possible with the tweaking within the confines of the standard model.

It looks good to me but how about making the table in alternating blue and white as per my user reports. Is this possible or just a step too far?


I found out about the UserModificationExtraSensorCharts True/False when I ran the website cmd without changing it to True

Now I just need to keep the sensors working for some time to get meaningful longer term data.

[HansR]

Andy,

I think I have gone as far as possible with the tweaking within the confines of the standard model.

It looks good to me but how about making the table in alternating blue and white as per my user reports. Is this possible or just a step too far?

The colour it takes is the ColorMenuBackground (default Lightgrey) alternated with the default background.
You could parametrize the ColorMenuBackground to another colour (blue) or a new parameter could be introduced which defaults to the menu colour. I am not sure about this. It additionally complicates things. No? Point is that User Defined reports are exactly that: User Defined. I can't presume colours there.

I found out about the UserModificationExtraSensorCharts True/False when I ran the website cmd without changing it to True

I could write a thousand wiki lines but nothing beats true hands on experience

Now I just need to keep the sensors working for some time to get meaningful longer term data.

The UserAskedData command takes the nr of hours for the RECENT CMX home charts. Only the RECENT CMX charts have a limit of 7 days with CUtils having no limit implemented (don't abuse this, there is a performance penalty if you do and I assume if you go beyond some limit javascript will start to complain)

[AndyKF650]

Hi Hans

Just a thought on your comment re long term extra sensor data, would it be possible to set up a ExtraSensor.txt file rather like the dayfile.txt which can be exported to MySQL automatically and then data can be called for much longer term than the current RECENT qualifier.

This maybe a step too far at the moment but looking at your comment about the latest CUtils upgrade it seems to me that you are probably using SQLlite already. Maybe this is an extra table that is created in CMX alongside the existing monthly, dayfile, and realtime tables.

[HansR]

Hi Andy,

Just a thought on your comment re long term extra sensor data, would it be possible to set up a ExtraSensor.txt file rather like the dayfile.txt which can be exported to MySQL automatically and then data can be called for much longer term than the current RECENT qualifier.

Always in for ideas on the possibilities of CUtils, but this one won't make it. Simply because Cutils itself is about data presentation and not data manipulation (some equations in the chartscompiler aside). Storing e.g. Extra Temperatures in a dayfile would require defining a new data store, averaging the numbers at some freuqnecy (probably what is being used for the dayfile) but all that simply is not the task for CUtils. Data aquisition and storage - in my view - is the task for CMX.

BUT: there is the CustomLogs feature of CMX where you can store any webtag/tekst/separator and manipulate that any way you want. Store it in a spreadsheet or in a MySQL database if you wish. And guess what: CUtils will (don't know when though) be able to display and chart the values in any CustomLog. At least that's the idea. Not much has been written yet on the CustomLogs but the way I look at it, it is the most powerfull feature of CMX. It even could make redundant its own standard logfiles simply by defining those as CustomLogs. Maybe that's a step too far but it could.

So in short: CUtils won't do what you ask, but you could take a look at the custom logs and see what can be found there (and wait for CUtils to handle those). CUtils will never create new datastores and/or data aggregates.

This maybe a step too far at the moment but looking at your comment about the latest CUtils upgrade it seems to me that you are probably using SQLlite already. Maybe this is an extra table that is created in CMX alongside the existing monthly, dayfile, and realtime tables.

1. No, CUtils (v6.x.y that is) is not using SQLite atm, it just reads the logfiles delivered by CMX as they are. SQLite is still future. The only SQLite databases in use are cumulusmx.db and diary.db.
2. And if and when that migration will be made is up to Mark. Some steps have been made but the project is not at speed. You have to ask Mark for his views on this.
3. Also the tables used is up to CMX. Modifications no doubt will be announced and discussed beforehand (I hope) but the initiative there is definitely not at CUtils. CUtils follows CMX in data storage, structure and technique. That has been the starting point and most likely won't be changed.

Hope this answers your questions. Not what you wanted but hey, that's life

[AndyKF650]

Hi Hans

Thanks for your very full reply to my seemingly innocuous question. Yes I understand that CUtils is all about data presentation on a website and it does that well. As for CMX, that does data store and manipulation well and maybe an extra sensors file in the style of the dayfile will be a step for the future.

I will take a look at the CustomLogs feature of CMX and let you know how I get on.

[AndyKF650]

Hi Hans

Is it possible to get the AirLink AirQuality table to work using data from my EcoWitt CO2 sensor. It produces PM 2.5 and PM10 and CO2 data but try as I might I have been unable to get any of the AirLink.txt files to appear.

I expect that I have missed a step but I have run the commands AirLink and useraskeddata which the wiki says are needed.

any thoughts would be appreciated

[HansR]

Hi Andy,

Nice idea again but sorry, no that is not possible because the AirLink is seen as a separate device and uses another table and a totally different setup (the AirLink configuration can have two AirLink devices in CMX). It also has standard more running averages (1, 3 and 24 hrs and the so called now-cast). And the AirLink does not have a CO2 sensor.

The only way possible would be to write some piece of software to duplicate the Ecowitt AirQuality sensor data into the table of the Airlink and as such mimic an AirlInk device. But then you would need to tell CMX it has an AirLink and NOT read the device while the outside world thinks it is a real AirLink (or if CUtils would take that task, it would have to duplicate and modify the data first before doing its normal job).

Hope this explains.

[AndyKF650]

Thanks Hans. So it seems that I have nearly reached the end of this development.

[HansR]

So it seems that I have nearly reached the end of this development.

I guess so yes.

But about the display: the CO2 looks good but I would not have expected the particulate matter to be so high (45 or even ebove 10) on Jersey with its direct openness to the ocean and fresh air. The WHO level is 5. Or is it the woodstoves? Does Jersey have a lot of woodstoves on (imported) wood or does it come from France (wind SW, this is where the windbarbs come in handy)?

[AndyKF650]

Hi Hans

Yes I had noticed the spikes and were when the wind is from the east ie France /Europe. Whilst Jersey is an island Normandy is only 20km away and all the PM produced in Europe comes our way. The dust haze in the atmosphere is noticeable when we look across the sea.

When the breeze is from the west the next land mass to us is the USA but across the Atlantic ocean so the PM levels tends to reduce and the air is so much clearer. It will be interesting to see just how a Sahara dust plume affects the PM count on the detector.