Page 3 of 4

Re: CustomLogs beta

Posted: Thu 12 Oct 2023 1:25 pm
by HansR
AndyKF650 wrote: Thu 12 Oct 2023 12:06 pm One point that is odd, when you start on the customtemplog page the chart does not automatically populate, you need to go to the daily chart and then back to the customtemplog page to get the full detail.
That is because your Daily logs apparently have no data yet and therefore the javascript fails at some point (in referencing a zero length array).
Don't change anything further and tomorrow morning there will be data in the Daily log and the charts will load normally.

Eventually Recent errors will all phase out over time.
AndyKF650 wrote: Thu 12 Oct 2023 12:06 pm This is no getting there, I will load some more custom logs in CMX and see how they work.
OK. Remember that after every change/addition you will have to remove your json's to have the system as the PHP upload can't handle the changes atm. But OK once configured things won't change very fast I assume.

Re: CustomLogs beta

Posted: Sat 14 Oct 2023 8:10 pm
by HansR
CMX v3.27.0 - b3254 - Beta 4 has just been released, that should fix the CO2 sensor issue.
The CMX solution for the JSON does not fix it for CUils CustomLogs, I'll look into that so it will take more time.
Also Daily logs without data need to syntactically be correct, will fix that too.

When the above is done, it should be ready for beta 4

Re: CustomLogs beta

Posted: Sun 15 Oct 2023 6:29 pm
by HansR
I just released v 6.21.0 beta 4 (which must run with CMX CMX v3.27.0 - b3254 currently under beta too, if you need that but can't find it just wait until it is released or ask me and I will forward it to you. Note it still is a beta so be careful and it is compiled under debug. When not certain, just wait!)

1) This release fixed the CO2 webtags which now have underscores and require both the new CMX and the new CUtils version
2) If you use PHP-upload: the issues with the JSON should be fixed for the CustomLogs. Any time you run CustomLogs, the corresponding JSONs will be uploaded full. After that all uploads are incremental. Btw... it may also be tested for FTP as that is more or less kind of hidden these days.


Note, if this is OK - remarks still welcome - it will be a release. However a subsequent version is already on the drawing board which will contain additional technical improvements and the addition of webtag metadata. The metadata will add the Unit, Chart Axis and possibly a description for a tooltip. Other functionalities can be added so you may suggest.

Re: CustomLogs beta

Posted: Sun 15 Oct 2023 7:56 pm
by AndyKF650
Hi Hans

I loaded this Beta having already had the CMX latest Beta loaded. The customlogs charts now start cleanly with the customtemplog not needing any of the previous fiddling :clap:

However the CO2 customlog does not want to start. I have cleared the RecentData.json from the internet host and then rerun customlogs and useraskeddata but it still is blank as you will see.

I have attached the logfile relating to customlog run for information.

Re: CustomLogs beta

Posted: Sun 15 Oct 2023 8:38 pm
by HansR
Hi Andy,

Good, its running :D
You can find the issues in the logfile yourself: search for warning and error. Beside that you can set the warning level to warning in the logfile meaning that only some 'where are you now' tracking and all errors and warnings will be shown. And don't make too many changes at the same time. If you want to know the system works take it one step at a time.

Also please take some time between commands (at least one minute) you fire the commands so fast that it is two commands in one logfile. That is confusing and difficult for storage. If commands are done in the same minute they end up in the same logfile.

But having said that, the logs shows:

Code: Select all

2023-10-15 20:19:35.942 Information InetPhp: extrasensorsdata.json: Response code = 500: InternalServerError
2023-10-15 20:19:35.943 Information InetPhp: extrasensorsdata.json: Response text follows:
No valid JSON data in the received data
2023-10-15 20:19:35.943 Information PHP UploadFile: Failed

Code: Select all

2023-10-15 20:19:11.910 Information CustomLogs ReadRecentCustomLog: CustomLogCO2
2023-10-15 20:19:11.910 Information CustomLogs ReadRecentCustomLog: data/CustomLogCO2-202310.txt - Start: 07/10/2023 20:15:00 ; End: 15/10/2023 20:15:00 ;
2023-10-15 20:19:11.911 Warning CustomLogs : There are more/less webtags than values in the log on line 0: 15/10/23,09:32,378
2023-10-15 20:19:11.911 Warning CustomLogs : The chart may not be what you want, please correct the content of the datafile. Continuing...
and

Code: Select all

2023-10-15 20:19:11.964 Information Constructor CustomLogs: handling Custom log CustomLogCO2 - Webtag <#CO2_24hr>
2023-10-15 20:19:11.964 Warning Constructor CustomLogs: Not a valid Webtag CO2_24hr used in Custom log CustomLogCO2
2023-10-15 20:19:11.964 Information Constructor CustomLogs: handling Custom log CustomLogCO2 - Webtag <#CO2-pm2p5>
2023-10-15 20:19:11.964 Warning Constructor CustomLogs: Not a valid Webtag CO2-pm2p5 used in Custom log CustomLogCO2
The first part tells me that you have an error in the ExtraSensors JSON file. So please remove it from the server. There is no further info in the log so if this error returns we will have to investigate.

The second part says there are more/less values in the log than there are in the definition of the log. This is probably caused because of the third error: you made the following typos in the definition of the log:

1) <#CO2_24hr> Must be <#CO2_24h> (spot the difference)

and

2) <#CO2-pm2p5> Must be <#CO2_pm2p5> (again spot the difference)

Remove the Customlogs JSON data files and the ExtraSensors data JSON and run again after correcting the errors. Be ware it takes some time to get data in the actual data logs. No data no charts. I would say do the corrective actions, run CUtils and take a night sleep to analyse tomorrow. That gives at least one dataline in the daily logs.

Let me know how you get on.

Re: CustomLogs beta

Posted: Sun 15 Oct 2023 9:00 pm
by AndyKF650
Hi Hans

Thanks for your support for this. As ever you were correct and my spelling needed to be sorted out. :bash:

This of course was masked by the old named webtags which still gives the same result. A quick correction and run customlogs and useraskeddata and all is well :D

As for the multiple log file I think that a manual command coincided with an automatic one giving a rather complicated result.

You are correct, a nights sleep will aid my tired brain. :)

Re: CustomLogs beta

Posted: Mon 16 Oct 2023 6:20 am
by AndyKF650
Hi Hans

All working just fine this morning, the Daily chart looks a bit odd with only one data point set so now I need to leave it alone and watch long term trends emerge.

Re: CustomLogs beta

Posted: Mon 16 Oct 2023 7:32 am
by HansR
AndyKF650 wrote: Mon 16 Oct 2023 6:20 am All working just fine this morning, the Daily chart looks a bit odd with only one data point set ...
Good, thnx!
AndyKF650 wrote: Mon 16 Oct 2023 6:20 am ... so now I need to leave it alone and watch long term trends emerge
Exactly.

NOTE: Once you are satisfied with the charts (menu and titles) and configuration you can think of the Texts/Translation for the charts. After all your experimentation I think it is best to remove all CustomLogs related texts and restart that. A lot of zombie texts may be present atm. But OK, that is just how you wish to handle that. There is no parameter/Text management from with CUtils itself.

As soon as you change a Log Title your menu and chart wil change and as such your texts will change. Modifying the Logs rapidly may create a labyrinth of texts. The only option then is delete and redo it (this is also true for other modules but to a much lesser extend).

Re: CustomLogs beta

Posted: Mon 16 Oct 2023 7:39 am
by HansR
Note to all: this version (6.21.0 - CustomLogs) will be released as soon as CMX v3.27.0 is released.

Re: CustomLogs beta

Posted: Tue 17 Oct 2023 7:27 am
by AndyKF650
Hi Hans

Just to say that all is working just fine at the moment and the daily log looks fine.

One comment, at the midnight roll over I get a full series of copy_CustomLog.... data files for each of the CustomLogs. Is this normal or is it a bug in the roll over procedure?

Re: CustomLogs beta

Posted: Tue 17 Oct 2023 7:44 am
by HansR
Andy,
AndyKF650 wrote: Tue 17 Oct 2023 7:27 am Just to say that all is working just fine at the moment and the daily log looks fine.
Great, thnx.
AndyKF650 wrote: Tue 17 Oct 2023 7:27 am One comment, at the midnight roll over I get a full series of copy_CustomLog.... data files for each of the CustomLogs. Is this normal or is it a bug in the roll over procedure?
That should only be for the DAILY Custom Logs and will be fixed in the final release.

But NOTE: all experimental CustomLogs - those you abandon after some time - need to be cleaned up by yourself.
There is no method for doing that automatically.

Re: CustomLogs beta

Posted: Tue 17 Oct 2023 8:15 am
by AndyKF650
Thanks for the info.

Re: CustomLogs beta

Posted: Tue 17 Oct 2023 3:18 pm
by AndyKF650
Hi Hans

Now that we have got the customlogs working properly, for me they have more functionality than the extralogs charts.

As a wish rather than a request is it possible to stop and remove the extralog charts from the website. I have tried but keep on getting a residual presence which I have not found a way to remove. Any thoughts or is this something I will have to live with?

Re: CustomLogs beta

Posted: Tue 17 Oct 2023 3:39 pm
by HansR
AndyKF650 wrote: Tue 17 Oct 2023 3:18 pm Now that we have got the customlogs working properly, for me they have more functionality than the extralogs charts.
Haha... yes that is a possible use but take care. The realtime table on the left may be overcharged a bit. I would have to redesign that and that will not be short term.
AndyKF650 wrote: Tue 17 Oct 2023 3:18 pm As a wish rather than a request is it possible to stop and remove the extralog charts from the website. I have tried but keep on getting a residual presence which I have not found a way to remove. Any thoughts or is this something I will have to live with?
I think that setting the parameter

Code: Select all

ExtraSensors=false
should do the trick. If it does not then please get back with specifying what you call residual presence!

Beside that you MUST remove the Extra section from the CutilsCharts.def file.

If you wish to cleanup your utils directory, you will have to remove yourself the Extra log, Extra realtime file and the compiled Extra charts and from the server the Extra JSON. I think the removal of those files is not required but it is always good to do some cleanuo.

If any issues, please make a new thread.

Re: CustomLogs beta

Posted: Tue 17 Oct 2023 3:52 pm
by AndyKF650
Hi Hans

As ever the perfect solution :D

I think when I tried this earlier I had not set the ExtraSensors flag to false and hence I could see some residual ExtraSensors detail on my site which has now completely gone.

Thanks