Cumulus Support

Posted: **Fri 12 Jan 2024 3:55 pm**

I'm getting error - "Error Parsing User Charts Definitions : No Chart at position '561'" when the compiler hits this chart definition...

CHART Temperature TITLE Temperature and Derivatives (Splines)
; PLOT InsideTemp AS Spline COLOUR #DDDF00 ZINDEX 100
PLOT Temperature AS Spline COLOUR #058DC7 ZINDEX 90
PLOT FeelsLike AS Spline COLOUR #ED561B ZINDDEX 80
PLOT DewPoint AS Spline COLOUR #50B432 ZINDEX 70
PLOT HeatIndex AS Spline COLOUR CornflowerBlue ZINDEX 60
PLOT ApparentTemp AS Spline COLOUR Orange ZINDEX 50
PLOT WindChill AS Spline COLOUR Blue ZINDEX 40
PLOT Humidex AS Spline COLOUR Brown ZINDEX 30
ENDCHART

There's no reference to 561 anywhere in the .def file so I don't know what it references.

Posted: **Fri 12 Jan 2024 4:40 pm**

OK, don't know where the position '561' comes from (most likely this is part of a larger CutilsCharts.def), but when I try this fragment alone, I get a similar message:

2024-01-12 17:26:53.532 Error Parsing User Charts Definitions : Error at EndChart of Chart 'Temperature'
2024-01-12 17:26:53.534 Error Errors in Charts definition. See logfile, please correct and run again.

IN the next version I will extend that message to:

2024-01-12 17:26:53.532 Error Parsing User Charts Definitions : Error at EndChart of Chart 'Temperature'
2024-01-12 17:26:53.532 Error Parsing User Charts Definitions : After position 'ZINDDEX'
2024-01-12 17:26:53.534 Error Errors in Charts definition. See logfile, please correct and run again.

To find the message you are getting, could you post the actual file you are parsing (change .def extension to .txt)?
I will look at that and improve that message as well (assuming I get a similar message).

But the actual error causing this all is the keyword ZINDDEX (must have single D, spelling is a bitch in computer languages

)

NOTE: The '561' you are getting must be the keyword position (so the 561-first keyword) in your file.

Posted: **Fri 12 Jan 2024 8:36 pm**

561'st keyword...sounds like that could be it, although finding that position would be fun

The change you're suggesting sounds good.

I've attached the file I'm trying to use. It's largely the same as one that Nossie uploaded but I'm trying to learn by changing bits of the charts to see what the effect is. Unfortunately I don't recall what I changed when this broke because I didn't notice that it had broken at first

Posted: **Fri 12 Jan 2024 9:21 pm**

packman2008 wrote: ↑Fri 12 Jan 2024 8:36 pm Unfortunately I don't recall what I changed when this broke because I didn't notice that it had broken at first

That's where it starts: memory and precision

I'll see what the file does, I I get anything I'll get back to you.

Posted: **Fri 12 Jan 2024 9:46 pm**

OK first run under the debugger one advise: take it one chart at a time with such issues. This is too complex.

The fragment you gave me is also different from what is in the large file. So, on step at a time...

Posted: **Fri 12 Jan 2024 10:45 pm**

It is large but I assumed what had been uploaded was working so I thought that was a safe starting point.

I tried to be methodical and started looking at each chart and found most worked as intended. I changed small things like titles, removed inside temperature, removed a BOM chart that's not applicable in the UK, etc. I sometimes made multiple changes per test because the compile, upload and review cycle is quite slow (I have 45 years of data loaded in CMX).

I then found some charts that didn't work, e.g. just blank plots, so I set about commenting parts of those charts out to see if I could find a subset of the chart that worked so that I could more closely examine the commented out parts to find out why they didn't work for me.

The snippet is slightly different to the uploaded file because I tried a few more things to see if I could fix the problem between reporting the problem and you requesting the full file. As each iteration gave the same error seemingly in the same chart I assumed I was still chasing the same issue.

Posted: **Sat 13 Jan 2024 4:02 am**

packman2008 wrote: ↑Fri 12 Jan 2024 10:45 pm I tried to be methodical and started looking at each chart and found most worked as intended. I changed small things like titles, removed inside temperature, removed a BOM chart that's not applicable in the UK, etc. I sometimes made multiple changes per test because the compile, upload and review cycle is quite slow (I have 45 years of data loaded in CMX).

You should use the command CompileOnly i.s.o. Website.
CompileOnly is very fast as it only compiles, makes a big difference

packman2008 wrote: ↑Fri 12 Jan 2024 10:45 pm I then found some charts that didn't work, e.g. just blank plots, so I set about commenting parts of those charts out to see if I could find a subset of the chart that worked so that I could more closely examine the commented out parts to find out why they didn't work for me.

Not working charts may be related to the version of HighCharts (see this post and my advise)

packman2008 wrote: ↑Fri 12 Jan 2024 10:45 pm The snippet is slightly different to the uploaded file because I tried a few more things to see if I could fix the problem between reporting the problem and you requesting the full file. As each iteration gave the same error seemingly in the same chart I assumed I was still chasing the same issue.

OK.... we'll get there. I'll look at it again.

Posted: **Sat 13 Jan 2024 4:46 am**

And AHA... found the actual error: your comment does not start with a semicolon on the first position in the line:

" ; Not working 11/01/24 - CJM
"

Code: Select all

; Not working 11/01/24 - CJM
;Test 1 - comment out InsideTemp 
CHART Temperature TITLE Temperature and Derivatives (Splines)
;	PLOT InsideTemp AS Spline COLOUR #DDDF00 ZINDEX 100
	PLOT Temperature AS Spline COLOUR #058DC7 ZINDEX 90
	PLOT FeelsLike AS Spline COLOUR #ED561B ZINDEX 80
	PLOT DewPoint AS Spline COLOUR #50B432 ZINDEX 70
	PLOT HeatIndex AS Spline COLOUR CornflowerBlue ZINDEX 60
	PLOT ApparentTemp AS Spline COLOUR Orange ZINDEX 50
	PLOT WindChill AS Spline COLOUR Blue ZINDEX 40
	PLOT Humidex AS Spline COLOUR Brown ZINDEX 30
ENDCHART

 ; Not working 11/01/24 - CJM
;Test 1 - comment out InsideTemp
CHART TempTodayYesterday TITLE Temperature and Derivatives (Lines)
;	PLOT InsideTemp AS line COLOUR #DDDF00 ZINDEX 100

But I do agree that the errormessages can be better. I changed this one to:

2024-01-13 05:19:21.840 Information Parsing User Charts Definitions : Chart Temperature'
2024-01-13 05:19:21.844 Error Parsing User Charts Definitions : No Chart at position 'Not'
2024-01-13 05:19:21.844 Error Errors in Charts definition. See logfile, please correct and run again.

I will work through the messages.

Find attached v 6.23.3 beta 1

If you need improved errorhandling (messages), let me know and I'll change the needed message and give you an intermediate release.

Posted: **Sat 13 Jan 2024 9:11 am**

Thanks Hans!

I'd missed that despite looking at it many times. I'm usually a bit OCD about putting things at line starts and using correct tabbing/spacing too!

I'll fix this and install the new version later today.

One other related thing, the error says 'See logfile' but the only place I'm seeing these errors is in the log files in utils/utilslogs. Is there another file I'm not seeing that has more detail?

Posted: **Sat 13 Jan 2024 9:13 am**

Have got the beta files running on mine, Thanks Hans, out of the box appeared to work A OK for compile only
Just tested then for my full kit using

START /D C:\CumulusMX C:\CumulusMX\utils\bin\cumulusutils.exe pwsFWI Top10 Graphs Yadr Records NOAA DayRecords UserAskedData Thrifty

I did note that the thrifty appeared to be stopping some of the pages from being updated. BUT that was for a week or so like that, this was something else then, not related to testing here. Doing a full without thrifty is not a big deal since only have a few years of data
(note to self, read the docco's and one learns, I need to do this regularly

)
Ran this and everything updates as needed (I run charts as a separate batch file with the CompileOnly for that one.)

START /D C:\CumulusMX C:\CumulusMX\utils\bin\cumulusutils.exe pwsFWI Top10 Graphs Yadr Records NOAA DayRecords UserAskedData

For the file that was uploaded "CutilsCharts.txt" changed its name to be CutilsCharts.def and used it for testing
Didn't succeed in processing it out of the box.
Logs gave this error (using the Beta cumulusutils.exe and cumulusutils.exe.config from Hans)
Excerpt of logs file C:\CumulusMX\utils\utilslog
2024-01-13 16:53:30.539 Information Parsing User Charts Definitions : Chart DailySolar'
2024-01-13 16:53:30.539 Information Parsing User Charts Definitions : Chart Temperature'
2024-01-13 16:53:30.540 Error Parsing User Charts Definitions : No Chart at position 'Not'
2024-01-13 16:53:30.540 DoAirLink / AirQualitySensor = False / False
2024-01-13 16:53:30.541 Information CustomLogs constructor: start

Looked for "Not" in the def file
Found this at line 142-144
Note the leading space in front of the comment " ;"

Code: Select all

 ; Not working 11/01/24 - CJM
;Test 1 - comment out InsideTemp
CHART TempTodayYesterday TITLE Temperature and Derivatives (Lines)

Removed the leading space and the file then processed fine for me, with no other changes

It is now this for those 3 lines

Code: Select all

; Not working 11/01/24 - CJM
;Test 1 - comment out InsideTemp
CHART TempTodayYesterday TITLE Temperature and Derivatives (Lines)

Enjoy

Posted: **Sat 13 Jan 2024 9:40 am**

@Chris:

packman2008 wrote: ↑Sat 13 Jan 2024 9:11 am One other related thing, the error says 'See logfile' but the only place I'm seeing these errors is in the log files in utils/utilslogs. Is there another file I'm not seeing that has more detail?

Nope, that's the only file. Each run creates a logfile and those files are kept for two days.
There is not more detail required and the compiler stops when an error occurs (no stop at warnings).

There is no error recovery so that it continues at a next recognised piece of CDL - that is something real compilers do have but it requires a lot of work so I did not do that in first instance. I may work on that later but for now it is not even on the ToDo list.

@Barry: Looks good.

nossis wrote: ↑Sat 13 Jan 2024 9:13 am START /D C:\CumulusMX C:\CumulusMX\utils\bin\cumulusutils.exe pwsFWI Top10 Graphs Yadr Records NOAA DayRecords UserAskedData

One advise: all commands but UserAskedData only require a daily run so I would take UserAskedData out of that commandline and create an entry of its own to run every 10 minutes or so.

If you use Thrifty on that first line (so for the modules) it would economise your bandwidth and CPU simply because e.g. it will process Records only partly if no records are detected and therefore it won't upload the file. For YADR the years before now won't change anymore so it will only generate this year etc... So you can use thrifty when running once daily. Don't use thrifty on UserAskedData.

On a PC Thrifty is less relevant than on a slower performing Raspberry Pi.

When updating a full release ALWAYS run Website once without thrifty so the system files (libraries) are updated automatically. Even if you do not use the website. If you do not, you will have to upload the libraries manually to the lib directory below your webroot. As you use the cumulusutils.js library, you sometimes will have to run Website because that is the only way to regenerate that library.

Posted: **Sat 13 Jan 2024 10:50 am**

I've just installed the beta and started looking at the code again. The error is less cryptic now. Would it be possible to output the line containing the error as well as the keyword that the error occurred at? If the DEF file had numerous instances of 'Not' in it the error would be difficult to trace.

One thing I note is that the log file contains the error, e.g.

2024-01-13 10:30:44.240 Error Parsing User Charts Definitions : No Chart at position 'Not'
2024-01-13 10:30:44.240 Error Errors in Charts definition. See logfile, please correct and run again.
2024-01-13 10:30:44.240 Error No new cumuluscharts.txt is generated and the old one remains in place!

...but neither the output from 'website'...

2024-01-13 10:30:44.208 DefineUsercharts: Parsing User charts definitions - start
2024-01-13 10:30:47.868 MapsOn: Start
2024-01-13 10:30:47.973 DateIsToday: thisSpan: 07:41:42.9733080 | thisSpan.TotalDays = 0.320636265138889
2024-01-13 10:30:47.973 DateIsToday returning True

...or 'compileonly'...

2024-01-13 10:35:51.138 DefineUsercharts: Parsing User charts definitions - start
2024-01-13 10:35:51.579 All done, Entering the finally section...; Closing down.

...gives and indication that there's an error. Is there something else I need to do to get errors to appear in the console output when I'm compiling? I suspect this is why I missed the change that broke things because I was looking for an error but there didn't appear to be anything wrong in the compiler output.

Posted: **Sat 13 Jan 2024 11:18 am**

Thanks Hans and all noted, made changes.
Yep makes sense in a Pi. And will make more sense with more data over time for me.
Cool thanks again, time to stop hijacking this post by me

Posted: **Sat 13 Jan 2024 11:24 am**

packman2008 wrote: ↑Sat 13 Jan 2024 10:50 am I've just installed the beta and started looking at the code again. The error is less cryptic now. Would it be possible to output the line containing the error as well as the keyword that the error occurred at? If the DEF file had numerous instances of 'Not' in it the error would be difficult to trace.

Ouputting the line can't be done because the whole file is broken up in keyword parts for parsing. The concept line does not exist with whitespace as separator, you could write everything separated by spaces (except titles of course) on a newline and it would work as well.

So the method is to develop chart by chart, check the logfile for errors and iterate. But also the screen output stops at error so you can see what chart was the last correct one. That gives you a fair enough indication of where the error is.

packman2008 wrote: ↑Sat 13 Jan 2024 10:50 am One thing I note is that I the log file contains the error, e.g.
[...]
...but neither the output from 'website'...
[...]
...or 'compileonly'...
[...]

...gives and indication that there's an error. Is there something else I need to do to get errors to appear in the console output when I'm compiling? I suspect this is why I missed the change that broke things because I was looking for an error but there didn's appear to be anything wrong in the compiler output.

OK, the next beta (attached) will output the error occurrence on screen:

2024-01-13 12:19:08.594 Errors in Charts definition. See logfile, please correct and run again.

The word Errors will be in red on the RPi, that is satisfying

But you will have to look in the logfile to get the actual info. That is how Cutils was designed: the screen is only a confirmation for the user that the program is working and where it more or less is. The logfile is intended for detailed info.

Note: you can also switch the logfile off, the screen output off and set the loglevel (Error, Warning, Info and Verbose) with the parameters:

Code: Select all

LoggingOn=true
NormalMessageToConsole=true
TraceInfoLevel=info

Verbose you should only use when I ask you to.

Posted: **Sat 13 Jan 2024 11:26 am**

nossis wrote: ↑Sat 13 Jan 2024 11:18 am Cool thanks again, time to stop hijacking this post by me

Don't worry, you both are in a similar phase and the beta's with improved compiler messages will be published here.

Cumulus Support

ChartCompiler - No Chart at position '561'

ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'

Re: ChartCompiler - No Chart at position '561'