Page 1 of 1

Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 7:30 am
by Dador
The CustomLog module has problems drawing charts in version 7.x.x. Previously, when I turned CMX off and on after some time, the additional sensors returned the value "0" for that period. I removed them manually from the Extralog2024xx.txt file. Currently I am not doing this because there are empty fields and the values ​​do not show zero. However, this makes the graphs blurry.

For example, I need to remove such a line so that the graph starts drawing.

Code: Select all

08/12/24,16:10,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2.2,2.2,2.8,3.9,17,20,31,27,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
08/12/24,16:15,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2.2,2.2,2.8,3.9,17,20,31,27,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
I don't know whether the number of empty fields or the lack of specific data means that I have to remove them in order to create a chart.

Re: Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 7:35 am
by HansR
@Dador:
  1. Could you please stop using this thread and start a new thread for a new issue? Thank you.
  2. It is not clear to me whether you are talking about ExtraSensors or about Custom Logs
  3. Could you provide a link to a chart displaying such behaviour?

Re: Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 8:09 am
by Dador
Let me just finish thinking about CustomLog.

Of course, this is about CustomLog and not ExtraLog, sorry for the confusion :oops: .

I have this data in the customlog202412.txt file

Code: Select all

09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,0.0,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.0,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.2,21,60,1021.0,1021.0,0.0,2.0
09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,-2.2,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.2,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.3,21,60,1021.0,1019.6,990.4,2.0
In cumulusutils.log I get this feedback

Code: Select all

08:52:47.656 Information CustomLogs ReadRecentCustomLog: data/customlog-202412.txt - Start: 05-12-2024 08:50:00 ; End: 12-12-2024 08:50:00 ;
2024-12-12 08:52:47.796 Warning CustomLogs ReadRecentCustomLog for customlog: Field is not a value -, continuing
Does this affect the fact that the charts are not drawn?

Edit:
Ok, I see this has an impact. I removed all lines from the file where there were ,-,-, instead of numbers. The charts are back.

Re: Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 8:21 am
by HansR
Dador wrote: Thu 12 Dec 2024 8:09 am Let me just finish thinking about CustomLog.

Of course, this is about CustomLog and not ExtraLog, sorry for the confusion :oops: .
OK.
Dador wrote: Thu 12 Dec 2024 8:09 am I have this data in the customlog202412.txt file

Code: Select all

09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,0.0,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.0,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.2,21,60,1021.0,1021.0,0.0,2.0
09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,-2.2,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.2,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.3,21,60,1021.0,1019.6,990.4,2.0
In cumulusutils.log I get this feedback

Code: Select all

08:52:47.656 Information CustomLogs ReadRecentCustomLog: data/customlog-202412.txt - Start: 05-12-2024 08:50:00 ; End: 12-12-2024 08:50:00 ;
2024-12-12 08:52:47.796 Warning CustomLogs ReadRecentCustomLog for customlog: Field is not a value -, continuing
The dash is a consequence of the null values which are introduced in CMX 4.3

Dador wrote: Thu 12 Dec 2024 8:09 am Does this affect the fact that the charts are not drawn?
Yes.

The problem has been signalled here and a solution is being worked on.
You'll find out if a satisfactory solution in CMX will be found. If my solution would be implemented, you would need to use the proposed parameter in your CustomLog definition to prevent the dashes. However, Mark is currently thinking on how to deal with the question where webtags are being used.

It is not a CUtils issue.
If you would like to circumvent the problem I would advise to temporarily modify the CutomLogs definition to remove the null value producing webtags (disable that specific customlog).

Hope this clarifies.

Re: Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 10:06 am
by HansR
Note that this thread has been separated from its origin and has been given a new title.

Re: Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 3:55 pm
by mcrossley
The next build implements an optional "nv=xxx" for web tags

Re: Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 4:06 pm
by HansR
Great :clap:

Re: Null values in webtags and CustomLogs

Posted: Thu 12 Dec 2024 6:46 pm
by HansR
Dador wrote: Thu 12 Dec 2024 8:09 am Let me just finish thinking about CustomLog.

Of course, this is about CustomLog and not ExtraLog, sorry for the confusion :oops: .

I have this data in the customlog202412.txt file

Code: Select all

09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,0.0,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.0,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.2,21,60,1021.0,1021.0,0.0,2.0
09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,-2.2,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.2,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.3,21,60,1021.0,1019.6,990.4,2.0
In cumulusutils.log I get this feedback

Code: Select all

08:52:47.656 Information CustomLogs ReadRecentCustomLog: data/customlog-202412.txt - Start: 05-12-2024 08:50:00 ; End: 12-12-2024 08:50:00 ;
2024-12-12 08:52:47.796 Warning CustomLogs ReadRecentCustomLog for customlog: Field is not a value -, continuing
Does this affect the fact that the charts are not drawn?

Edit:
Ok, I see this has an impact. I removed all lines from the file where there were ,-,-, instead of numbers. The charts are back.
@Dador:
OK, Mark created a solution in the coming release which I assume will arrive shortly. With that you need to make some choices:
  1. Apparently you log a webtag which is not connected and creates null values. You should decide what you want with that webtag and maybe delete it from the custom logs. If you do the problem is solved (no more dashes).
  2. If you decide to keep it in the customlog apparenty it creates a dash where it expects a number as output - note that all allowed webtags produce numbers to be able to chart the values. This means that if CMX outputs a '-', the actual value CMX has for this webtag is a null value. You need to modify the definition of the webtag in the customlog with the nv=xxx (additional) modifier. You will need to check the ChangeLog of CMX on how this precisely will work.
I will be available Tuesday for support and to modify the wiki page (both CUtils and for the webtag modifiers) on this.