Null values in webtags and CustomLogs

[Dador]

The CustomLog module has problems drawing charts in version 7.x.x. Previously, when I turned CMX off and on after some time, the additional sensors returned the value "0" for that period. I removed them manually from the Extralog2024xx.txt file. Currently I am not doing this because there are empty fields and the values ​​do not show zero. However, this makes the graphs blurry.

For example, I need to remove such a line so that the graph starts drawing.

Code: Select all

08/12/24,16:10,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2.2,2.2,2.8,3.9,17,20,31,27,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
08/12/24,16:15,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2.2,2.2,2.8,3.9,17,20,31,27,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

I don't know whether the number of empty fields or the lack of specific data means that I have to remove them in order to create a chart.

[HansR]

@Dador:

1. Could you please stop using this thread and start a new thread for a new issue? Thank you.
2. It is not clear to me whether you are talking about ExtraSensors or about Custom Logs
3. Could you provide a link to a chart displaying such behaviour?

[Dador]

Let me just finish thinking about CustomLog.

Of course, this is about CustomLog and not ExtraLog, sorry for the confusion .

I have this data in the customlog202412.txt file

Code: Select all

09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,0.0,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.0,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.2,21,60,1021.0,1021.0,0.0,2.0
09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,-2.2,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.2,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.3,21,60,1021.0,1019.6,990.4,2.0

In cumulusutils.log I get this feedback

Code: Select all

08:52:47.656 Information CustomLogs ReadRecentCustomLog: data/customlog-202412.txt - Start: 05-12-2024 08:50:00 ; End: 12-12-2024 08:50:00 ;
2024-12-12 08:52:47.796 Warning CustomLogs ReadRecentCustomLog for customlog: Field is not a value -, continuing

Does this affect the fact that the charts are not drawn?

Edit:
Ok, I see this has an impact. I removed all lines from the file where there were ,-,-, instead of numbers. The charts are back.

[HansR]

Let me just finish thinking about CustomLog.

Of course, this is about CustomLog and not ExtraLog, sorry for the confusion .

OK.

I have this data in the customlog202412.txt file

Code: Select all

09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,0.0,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.0,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.2,21,60,1021.0,1021.0,0.0,2.0
09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,-2.2,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.2,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.3,21,60,1021.0,1019.6,990.4,2.0

In cumulusutils.log I get this feedback

Code: Select all

08:52:47.656 Information CustomLogs ReadRecentCustomLog: data/customlog-202412.txt - Start: 05-12-2024 08:50:00 ; End: 12-12-2024 08:50:00 ;
2024-12-12 08:52:47.796 Warning CustomLogs ReadRecentCustomLog for customlog: Field is not a value -, continuing

The dash is a consequence of the null values which are introduced in CMX 4.3

Does this affect the fact that the charts are not drawn?

Yes.

The problem has been signalled here and a solution is being worked on.
You'll find out if a satisfactory solution in CMX will be found. If my solution would be implemented, you would need to use the proposed parameter in your CustomLog definition to prevent the dashes. However, Mark is currently thinking on how to deal with the question where webtags are being used.

It is not a CUtils issue.
If you would like to circumvent the problem I would advise to temporarily modify the CutomLogs definition to remove the null value producing webtags (disable that specific customlog).

Hope this clarifies.

[HansR]

Note that this thread has been separated from its origin and has been given a new title.

[mcrossley]

The next build implements an optional "nv=xxx" for web tags

[HansR]

Great

[HansR]

Let me just finish thinking about CustomLog.

Of course, this is about CustomLog and not ExtraLog, sorry for the confusion .

I have this data in the customlog202412.txt file

Code: Select all

09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,0.0,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.0,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.2,21,60,1021.0,1021.0,0.0,2.0
09/12/24,21:05,2.2,88,0.4,0.4,-1.0,2.2,2.2,-1.9,-1.0,2.2,-1.0,-2.2,-,-,-,-,-,-,-,2.8,2.8,3.3,3.9,0,222,0.2,0.0,0.0,0.2,6.0,806.6,0.0,27.0,0,836.80,1948.5,-,11.3,21,60,1021.0,1019.6,990.4,2.0

In cumulusutils.log I get this feedback

Code: Select all

08:52:47.656 Information CustomLogs ReadRecentCustomLog: data/customlog-202412.txt - Start: 05-12-2024 08:50:00 ; End: 12-12-2024 08:50:00 ;
2024-12-12 08:52:47.796 Warning CustomLogs ReadRecentCustomLog for customlog: Field is not a value -, continuing

Does this affect the fact that the charts are not drawn?

Edit:
Ok, I see this has an impact. I removed all lines from the file where there were ,-,-, instead of numbers. The charts are back.

@Dador:
OK, Mark created a solution in the coming release which I assume will arrive shortly. With that you need to make some choices:

1. Apparently you log a webtag which is not connected and creates null values. You should decide what you want with that webtag and maybe delete it from the custom logs. If you do the problem is solved (no more dashes).
2. If you decide to keep it in the customlog apparenty it creates a dash where it expects a number as output - note that all allowed webtags produce numbers to be able to chart the values. This means that if CMX outputs a '-', the actual value CMX has for this webtag is a null value. You need to modify the definition of the webtag in the customlog with the nv=xxx (additional) modifier. You will need to check the ChangeLog of CMX on how this precisely will work.

I will be available Tuesday for support and to modify the wiki page (both CUtils and for the webtag modifiers) on this.