Page 2 of 2
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 1:02 pm
by mcrossley
HansR wrote: ↑Tue 21 Jul 2020 12:10 pm
But unfortunately my FF says:
SyntaxError: JSON.parse: unexpected character at line 1 column 9 of the JSON data and I have to look at the RAW data.
And tbh, it makes me wonder why the editor takes so much time to read the data of the month files?
That's because the URL is missing a parameter, add "&draw=1" and it should be OK.
Atlas wrote: ↑Tue 21 Jul 2020 12:26 pm
Well..... MX on Windows 10 (latest updates), MSI windbox dc111 (5 years old) and logfile 1 minute interval MX running through VVP....VVP hanged and need to restart MX after It used a lot of CPU during minutes.
Both testing of you are in Raspberrys...some test on Windows ?
Just to check if It is a thing related to my system.
Windows 10 - 10,000 requested for 06-2020 = 0.13 seconds
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 1:21 pm
by HansR
mcrossley wrote: ↑Tue 21 Jul 2020 1:02 pm
That's because the URL is missing a parameter, add "&mark=1" and it should be OK.
Should be, but it isn't unfortunately.
Code: Select all
http://192.168.178.6:8998/api/data/logfile?month=07-2019&start=0&length=50000&mark=1
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 2:11 pm
by Atlas
Impossible for me....
1,000 requests to 06-2020 log ----- 195 kB ------- 19.4 seconds (Chrome developer tools), Cumulus MX CPU usage about 45% while processing
06-2020 log with 1 minute interval recording has 5,628 kB and 43,027 lines
Maybe the log size would be the problem....
P.D. mcrossley your 06-2020 logfile with 5 minutes interval must have around 8,640 lines (12 * 24 * 30)... and you have requested 10,000 lines... no problem about it ??
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 3:06 pm
by freddie
I see similar poor performance - /api/data/logfile?month=07-2020&start=0&length=20000&mark=1 returns all rows (my archive interval is 1 minute) but it took 20.34 seconds. If I request any more rows I run into a timeout (about 25 seconds): in Developer tools (Chrome) the status is given as net::ERR_CONNECTION_RESET.
TBF I am temporarily running Cumulus on a Windows 10 laptop - so it probably isn't optimised for "serving". I'll run the same test when my server is back on its feet.
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 4:23 pm
by mcrossley
HansR wrote: ↑Tue 21 Jul 2020 1:21 pm
mcrossley wrote: ↑Tue 21 Jul 2020 1:02 pm
That's because the URL is missing a parameter, add "&mark=1" and it should be OK.
Should be, but it isn't unfortunately.
Code: Select all
http://192.168.178.6:8998/api/data/logfile?month=07-2019&start=0&length=50000&mark=1
Cranial flatulence strikes again, it should be "&draw=1"
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 4:24 pm
by mcrossley
freddie wrote: ↑Tue 21 Jul 2020 3:06 pm
I see similar poor performance - /api/data/logfile?month=07-2020&start=0&length=20000&mark=1 returns all rows (my archive interval is 1 minute) but it took 20.34 seconds. If I request any more rows I run into a timeout (about 25 seconds): in Developer tools (Chrome) the status is given as net::ERR_CONNECTION_RESET.
TBF I am temporarily running Cumulus on a Windows 10 laptop - so it probably isn't optimised for "serving". I'll run the same test when my server is back on its feet.
Weird - how can it be soo different!?
Which version of MX?
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 4:33 pm
by freddie
mcrossley wrote: ↑Tue 21 Jul 2020 4:24 pm
freddie wrote: ↑Tue 21 Jul 2020 3:06 pm
I see similar poor performance - /api/data/logfile?month=07-2020&start=0&length=20000&mark=1 returns all rows (my archive interval is 1 minute) but it took 20.34 seconds. If I request any more rows I run into a timeout (about 25 seconds): in Developer tools (Chrome) the status is given as net::ERR_CONNECTION_RESET.
TBF I am temporarily running Cumulus on a Windows 10 laptop - so it probably isn't optimised for "serving". I'll run the same test when my server is back on its feet.
Weird - how can it be soo different!?
I don't know. I usually use Linux and its performance is a bit crisper on there.
mcrossley wrote: ↑Tue 21 Jul 2020 4:24 pmWhich version of MX?
3.6.12 build 3088. I had been using the latest beta but forgot that in my panic/haste to get it up-and-running again on Sunday
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 5:03 pm
by mcrossley
Atlas wrote: ↑Tue 21 Jul 2020 2:11 pm
P.D. mcrossley your 06-2020 logfile with 5 minutes interval must have around 8,640 lines (12 * 24 * 30)... and you have requested 10,000 lines... no problem about it ??
No problem asking for more lines than exist - it will just return all the lines it can.
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 5:44 pm
by HansR
mcrossley wrote: ↑Tue 21 Jul 2020 4:23 pm
Cranial flatulence strikes again, it should be "&draw=1"
ROF
Working though! Cold start - so no cache - less than 3 seconds.
Re: Graphs datepicker
Posted: Tue 21 Jul 2020 6:34 pm
by mcrossley
OK, now I have restarted MX I am getting response times that are varying from 0.4 seconds to 7+ seconds (Windows 10, i5, local machine).
For all of the them the response time (to read the data and start sending it) is < 0.1 seconds, then the transmission time to the browser that is varying from 0.4 up to a couple of seconds.
But then the rendering time for the browser to interpret the JSON and display the data can take up to another 7 seconds, I think it is this rendering that is also slowing the data download as...
Running the query in Postman rather than a browser and the data is sent for the first query in <600ms then once the log file has been cached it is consistently being downloaded in 70-80ms.
Re: Graphs datepicker
Posted: Wed 22 Jul 2020 8:24 am
by Atlas
Well.... my previous tests have been with build 3086 but after upgrading to 3088 there have been a lot of improvements in performance because MX has almost instanly served the data.... Chrome has been a little bit lazy loading the whole json.... I've requested 50,000 lines and Chrome has served 28,000 in 1.4 minutes !!
So I think it is not a matter of PC specs but the 3088 version improvement... can you confirm that mcrossley ??
Anyway I am going to do some tests requiring the json data for the graph... asking for the whole json and selecting only the day I want.
Re: Graphs datepicker
Posted: Wed 22 Jul 2020 9:04 am
by mcrossley
3088 did have performance enhancements around constructing the JSON strings.
I've just made some additional performance tweaks to the API code for the next release.
Re: Graphs datepicker
Posted: Wed 22 Jul 2020 11:05 am
by Atlas
if it's not too much to ask for… could some of these tweaks include to ask for a specific day of the log ? the way
http://127.0.0.1:8998/api/data/logfile? ... 020&day=23
for example ??
If not thank you anyway
Re: Graphs datepicker
Posted: Sat 01 Aug 2020 1:51 pm
by galfert
I second this request instead of having to construct URL manually.