Graphs datepicker

[Atlas]

Hi

I have not found nothing related to a graph datepicker in the forum. Could It be add to the graph page ? Sometimes is more "visual" to see graphs than raw data logfile.

It would be great if possible.

Greetings

[HansR]

What exactly do you mean by a datepicker for graphs?
What graphs are you referring to?

The standard Cumulus graphs have a slider below the graph.
If you want similar graphs for the whole timespan of your dataset with date selection, that might be problematic if it were only for the amount of data involved.

[Atlas]

A datepicker or calendar in graphs HTML page of MX web interface.

I suppose you mean that all data has to be loaded and then selected, I mean to be loaded and selected just the picked day. I think it would be easy by loading the choosen day from the logfile.

[sfws]

MX's standard functionality will upload to database tables on the web server, and getting data from them is just as easy for past dates as for recent dates.
Thus one can code web pages to select using a menu of navigation-type buttons, your date-picker, (or any other inputs in a HTML form) what data they read from such tables.

I am unsure whether you want to pick the date, and have just 24 hours displayed?

If you are seeking something equivalent to what is in current trends page, but for past periods, then search Historic Charts in this forum. See the author's implementation at https://weather.wilmslowastro.com/graphs_historic.php to confirm whether this will give you something close to what you want, it is without your date-picker.

I think it would be easy by loading the chosen day from the logfile

The standard web pages are not able to send a request back to Cumulus to ask for specific data to be uploaded, selections in MX decide what is uploaded when. There is some control over the number of days/hours included in the JSON files used for the MX trends web page, but not the ability for a web page to tell MX to change the json into a past day that you want.

You can upload log files, and therefore have them on your web server where your date picker can choose what to read from the log file.
If you are prepared to search, there are various oldish third party produced web pages that can read log files, probably including one I wrote for plotting from log file (in a topic about extending the Cumulus 1 Trends web page) and another I wrote where you read one day off the daily summary log (it is in a topic about displaying numbers for a year ago).

For all log file readers, search forum (possibly starting from Wiki documentation) for what was on offer before MX was made available as a beta, because I suspect all of them were written when only Cumulus 1 was available, and therefore comparatively few people updated database tables.

[Atlas]

Thanks for the sources.

As I said my intention is to load a logfile's day in a chart with the datepicker.

I'll see and try if I can implement that

[Atlas]

Well I'm searching a way to get logfiles from the data folder but being in a local system folder and MX interface in a webserver (EmbedIO) Cross Origin Policy doesn't allow to get those logfiles.

Can be mapped data folder in some way to be accessed within MX webserver ??

Thanks

[sfws]

Cross Origin Policy doesn't allow to get those logfiles

You can upload log files

[Atlas]

I mean to be mapped to be accessed from a file (charts.html for example) in MX interface.

I suppose it would need to write MX executable code.

[HansR]

@Atlas: not sure how difficult you want to make it yourself, but I decided to circumvent that type of problems by running my dataselection/graphgenerator and more on the CMX machine itself, using direct access to the data (on a copied version of course). From there the results are send to the webserver over ftp et voila. Result like in the signature link. Not what you want (you want realtime feedback to dataselection) but maybe you find some ideas.

There is a tendency to bring raw data from the CMX machine to the client and start working the data there.
That itself is a technique much used in the Cumulus-world and you may find many procedures doing all kinds of things, creating all kinds of output.
So, shop around, somebody might have gotten close to what you want.

Btw: what is MX executable code?

[Atlas]

This is not for me (well I'd like to ) but for a guy in the meteoclimatic forum that wanted a way to plot past days in the MX's graph HTML page

The workarounds are not easy for people that don't have ideas about local webservers or don't have a website in a hosting , so I've tried to see if it was posible to add that feature in MX.

Maybe one day it could possibly be done.

Btw: what is MX executable code?

I mean the CumulusMX.exe code.


By the way… great work with cumulusutils and special thanks to Javier (aka kocher) that belongs to the meteoclimatic staff (like me) for his translations and great work supporting it.

Thanks

[sfws]

A datepicker or calendar in graphs HTML page of MX web interface.

Sorry, your original post confused me talking about web interface. I assumed that meant what was uploaded by MX from the web folder to a web server and you were talking about the web page with graphs (we call that trends.htm). My first reply was trying to help you with that, where the data is supplied by json files that are uploaded from that web folder. Most people would customise these main web pages to achieve your request, but I accept what you mean by

The workarounds are not easy for people that don't have ideas about local web-servers or don't have a web site in a hosting

=======================================================================================================================

Please be aware that Cumulus 1 had ability to report statistics for any period (1 day to all time) see https://cumuluswiki.org/a/Cumulus_Screenshots#View_data and would have answered the original query (see end of this post) easily, but this is one feature that has not been implemented in MX.

If you were to ask for this View a Period Cumulus 1 feature to be added to MX, then there is a better chance it could be added than your graph idea.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------

I mean to be mapped to be accessed from a file (charts.html for example) in MX interface.

I suppose it would need to write MX executable code.

Now I understand you are talking about interface/charts.html, my answer is totally different. That page is in the admin interface. That gets its json files via a Application Programming Interface. Look at https://github.com/cumulusmx/CumulusMX/ ... sMX/Api.cs from line 319 onwards where it generates what is sent to
interface/charts.html.
That is the bit of C# complied into CumulusMX.exe that would need to alter so it could talk to your date picker.

to see if it was possible to add that feature in MX

The admin interface is basically for changing settings and seeing a quick summary of latest outputs, so it is not intended to be customised for particular requirements such as what you suggest. Theoretically it is possible, whether the developer of MX will add such a feature is not for me to answer, although I doubt it.

The interface/datalogs.html has a JavaScript file that includes a paginator function for choosing 10 lines, then sends a request to MX for 10 lines from the standard log file,and then the api sends back those 10 lines. I did play around with that part of the admin interface so I understand it. It shows the admin interface can ask for particular part of the log file, so that is why I say what you want is theoretically possible.

You don't indicate your own skills, so I can't say whether you could find a way, given my now relevant pointers, to make this work for yourself and your friend.
=======================================================================================================================================
By the way, I applied Google translate to the forum post you mention, so I understand the context. The various daily summary third part routines, see https://cumuluswiki.org/a/Daily_Summary would (as well as the Cumulus 1 feature I mentioned earlier) actually answer the original question in figures (not a graph) as they report maximum wind gusts on a daily basis, but those are designed to run on a web server and are not part of MX.

Estación: Paterna - Casas Verdes - ESPVA4600000046980C / Bejis - ESPVA1200000012430A

Formas ágiles de localizar datos de viento en CumulusMX
« en: 17 de Julio del 2020, 23:28:07 pm »
Buenas noches,

Planteo la siguiente situación por si se puede realizar la consulta en los datos de CumulusMX de forma mas rápida. He recibido un mail de una persona a través del formulario de la web de Meteoclimatic de mi estación de Paterna-Casas Verdes solicitándome los datos de rachas máximas de viento de la noche del 12 al 13 de julio. Para revisar dichos datos solo se me ha ocurrido dentro de la aplicación de CumulusMX ir al menú de data log y revisar los datos de lo que había ocurrido esa noche. El problema es que para llegar hasta los datos del día 12-13 me ha tocado ir pasando todas las páginas hasta ese día, lo que me ha llevado un buen rato, ya que se graban los datos cada 5 minutos. Me preguntaba si existe alguna forma mas ágil de consultar dichos datos o si se puede pedir algún gráfico seleccionando un rango de fechas concreto. Los gráficos de velocidad del viento que se muestran son los de las últimas 24h.

Gracias de antemano
---------------------------------------------------------------
Ver Perfil www.meteoarchena.es
Estación: Archena - ESMUR3000000030600B

Re:Formas ágiles de localizar datos de viento en CumulusMX
« Respuesta #1 en: 18 de Julio del 2020, 09:16:44 am »
Hola Lraso

Sí, la verdad es que a MX le falta un calendario o "datepicker" en los gráficos para elegir el gráfico del día que quiera consultarse mejor que estar navegando por los logs.

En la interfaz web del programa es engorroso ver esos logs por el tema de ir en tablas de pocos registros. Para consultas así yo cerraría el programa y vería el logfile correspondiente de la carpeta data con un editor de texto (también se puede abrir con un navegador).

De todas formas he abierto un hilo sobre lo del datepicker en las sugerencias del foro de MX.

A ver cómo se desarrolla la cosa, supongo que mucha gente estaría interesada en ello.

Saludos
En línea

Station: Paterna - Casas Verdes - ESPVA4600000046980C / Bejis - ESPVA1200000012430A

Agile ways to locate wind data in CumulusMX
«On: July 17, 2020, 23:28:07 pm»
Good night,

I raise the following situation in case the query in the CumulusMX data can be done more quickly. I have received an email from a person through the Meteoclimatic website form from my Paterna-Casas Verdes station requesting the data on maximum wind gusts for the night of July 12 to 13. To review such data, it has only occurred to me within the CumulusMX application to go to the data log menu and review the data of what had happened that night. The problem is that to get to the data for the 12-13 day I had to go through all the pages until that day, which has taken me a long time, since the data is recorded every 5 minutes. I was wondering if there is a more agile way to consult these data or if you can request a graph by selecting a specific date range. The wind speed graphs shown are those of the last 24h.

Thanks in advance
---------------------------------------------

Re: Agile ways to locate wind data in CumulusMX
«Answer # 1 on: July 18, 2020, 09:16:44 am»
Hi Lraso

Yes, the truth is that MX lacks a calendar or "datepicker" in the charts to choose the chart of the day that it wants to consult better than to be browsing the logs.

In the web interface of the program it is cumbersome to see these logs due to the fact of going in tables with few records. For queries like this I would close the program and see the corresponding logfile of the data folder with a text editor (it can also be opened with a browser).

Anyway I have opened a thread about the datepicker in the MX forum suggestions.

To see how the thing develops, I suppose that many people would be interested in it.

Regards

[Atlas]

The interface/datalogs.html has a JavaScript file that includes a paginator function for choosing 10 lines, then sends a request to MX for 10 lines from the standard log file,and then the api sends back those 10 lines. I did play around with that part of the admin interface so I understand it. It shows the admin interface can ask for particular part of the log file, so that is why I say what you want is theoretically possible.

You hit the nail here.

This is what I'm looking for but after some testing it's not possible to ask for a single day unless there's an option about it in the URL query string. There are just start and length options in the query but you can pick a day because it's all sorted by indexes and not by dates.

I can load, for example:

http://192.168.1.101:8998/api/data/logf ... length=100

without problems but if you ask more length MX has to deal with too much search and takes a long time processing with high CPU usage.

Indeed the API part is involved about asking the data and now I realize why 10 entries are only shown in the logs.

It would be interesting if there's an option to load the logfile without that long processing to json conversion but I don't think so

And sorry for my English and MX scheme if I made a little mess about it.

And yes... you did a good search and translation/understanding about meteoclimatic original thread.

[mcrossley]

I can load, for example:

http://192.168.1.101:8998/api/data/logf ... length=100

without problems but if you ask more length MX has to deal with too much search and takes a long time processing with high CPU usage.

I just tried it asking for 10,000 rows for 06-2020 and it returned 8640 rows (the whole month at 5 minute logging intervals) in 0.45 seconds from my Raspberry Pi - not that slow?

[HansR]

I can load, for example:

http://192.168.1.101:8998/api/data/logf ... length=100

without problems but if you ask more length MX has to deal with too much search and takes a long time processing with high CPU usage.

I just tried it asking for 10,000 rows for 06-2020 and it returned 8640 rows (the whole month at 5 minute logging intervals) in 0.45 seconds from my Raspberry Pi - not that slow?

Indeed, the same here, flash and there it is.
But unfortunately my FF says: SyntaxError: JSON.parse: unexpected character at line 1 column 9 of the JSON data and I have to look at the RAW data.

And tbh, it makes me wonder why the editor takes so much time to read the data of the month files?

[Atlas]

Well..... MX on Windows 10 (latest updates), MSI windbox dc111 (5 years old) and logfile 1 minute interval MX running through VVP....VVP hanged and need to restart MX after It used a lot of CPU during minutes.

Both testing of you are in Raspberrys...some test on Windows ?

Just to check if It is a thing related to my system.