Highcharts realtime.log size limitation

[kingqueen]

This is just in case it's off use to anybody,

I increased the number of measurements saved in realtime.log for the 96 hour Tnet graph. At 1 per minute, that made the log file very very long.

Turns out highcharts doesn't cope once the file gets above a certain size, it shows a graph with axes but no data.

Cutting the file size made the graphs come back. Changing to 5 minute interval in the log should prevent it getting that big again

That took some troubleshooting so I thought I'd post here in case it's of use to anybody else!

Cheers

[kingqueen]

speaking of which, random question: does anybody know - or know how I can find out - how long realtime.log can be before Cumulus falls over?

I suppose the alternative is to have two realtime.log files - hmmm yes that would do it wouldn't it - one for the jpgraphs and one shorter one for the highcharts - with a different cron job and filename for each. I shall implement that.

thanks!

[steve]

Cumulus doesn't know anything about the realtime.log file, so it's not going to affect it no matter how big it gets.

[kingqueen]

Yes apologies, I had a brain fart, I should have said, how big the realtime.log file gets before highcharts falls over.

[gemini06720]

I increased the number of measurements saved in realtime.log for the 96 hour Tnet graph. At 1 per minute, that made the log file very very long.

Turns out highcharts doesn't cope once the file gets above a certain size, it shows a graph with axes but no data.

One could always create and run a PHP script once a day (a CRON job on an Apache web server) to truncate (reduce) the file to a size that is acceptable by the Highcharts scripts...

[water01]

Turns out highcharts doesn't cope once the file gets above a certain size, it shows a graph with axes but no data.

As far as can find out there is no limit on the number of plot points (i.e. there is an example of a Highchart/Histock chart plotting 1.7 million points on their API area). What I think is the problem is the performance which they admit when plotting a lot of points like that takes a considerable time. I also think Mark found this when he tried to plot a lot of his weather data and he had to reduce the number of plot points to make it usuable. So I think that if you had waited the blank chart would have eventually filled out!! I personally use Mark's little DOS batch file to keep the realtime.log file to the level that is required for the Near Realtime plot, couldn't that be extended/modified to provide what you want?

By the way what is the Tnet graph?

[mcrossley]

Hi, David is correct, the Highcharts demo site shows a graph with 1.7 million data points. Highcharts itself makes an attempt to decimate points that will not be visible after the number reaches a configurable limit. The other limitation is the browser and host machine. Each data point on the chart is another DOM element on the page.

My PHP scripts that query a SQL database have a parameter that lets the script return every nth row. I use that on my 72 hour charts sourced from 1 minute data to thin the data down a bit - and reduce the returned data size.

[kingqueen]

So I think that if you had waited the blank chart would have eventually filled out!!

No; there's definitely a "cut off" point whereby it just stops working, I left the page open for some time (an hour?) It doesn't seem to be the number of points on one graph either; I had only set it to plot two days data, 2880 points per series, which it quite happily does if the log file is only 2880 lines long, but not if it's 5760 lines long even though it's only plotting the same number.

Thanks for the suggestions everybody but I had already sorted the problem by using a cron job to restrict the file to a sensible size, and having a separate file for the other graphs. I posted here just because it took some head scratching to work out what was going wrong, with no error messages. Thought it might help A N Other person to know that if your Highcharts weather graph just displays axes and no data, the size of the log file might be responsible.

By the way what is the Tnet graph?

"TNET Cumulus JpGraph". I haven't played with mine yet or put them in my website as I would like to tweak their appearance, but here are the sample graphs for my data.