Welcome to the Cumulus Support forum.

Latest Cumulus MX release 3.9.1 (build 3096) - 31 October 2020 (please see announcement regarding releases since 3.5.0)
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of the NOAA report and Snow Index drop-down menus to 2030)

Use this Wiki link to Download the Software (Cumulus MX / Cumulus 1 and other related items).

Highcharts Graphs

Discussion and support for 3rd-party (non-Sandaysoft) tools for Cumulus
User avatar
kingqueen
Posts: 60
Joined: Tue 26 Feb 2013 6:52 pm
Weather Station: Maplin N96GY / Fine Offset 1081
Operating System: Windows 7 HP 64 bit
Location: Wetherby, UK
Contact:

Re: Highcharts Graphs

Post by kingqueen »

My web hosts support cron jobs, which can be created using their control panel. I've therefore set that up to create realtime.log, adding realtime.txt every time and trimming it.

Here's my current setup:

Image

Here's the command for the default 120 readings / 2 hours:

cat /home/wetherby/public_html/realtime.log /home/wetherby/public_html/realtime.txt > /home/wetherby/public_html/realtime.tmp && tail -q -n 120 /home/wetherby/public_html/realtime.tmp > /home/wetherby/public_html/realtime.log && rm /home/wetherby/public_html/realtime.tmp

Obviously anybody else will have to adjust the absolute paths to their hosting. No doubt also using clever piping it would be possible to do the above in one command, without a temporary file, and do without the absolute paths. It's quick and dirty - but it works, and it means realtimelog doesn't have to be FTPed up every minute, with the resultant bandwidth considerations.
Image

User avatar
mcrossley
Posts: 7150
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Highcharts Graphs

Post by mcrossley »

Jachym, in your case it would make sense to ignore all the real time stuff. All you need to do is copy the Cumulus day file up to your web server once a week after it is updated. Then run the importcumulusfile.PHP script to copy the new day file data into the SQL database table. The import script is in the Wiki.

Sorry, but this is brief post as I am on a tablet at the moment and typing is slow.

jachym
Posts: 53
Joined: Mon 07 Jan 2013 3:47 pm
Weather Station: WH 1080
Operating System: Win XP
Location: Czech republic

Re: Highcharts Graphs

Post by jachym »

HI,

thx a lot for reply. Well, the thing is, I am an absolute beginner in this perspective. As I mentioned, so far I used Excel and I only have experience with HTML and javascript - so I for example know how to implement HighCharts on a webpage etc. But I have never worked with MySQL or PHP so I dont really know anything about it apart from roughly what it does.
My question was primarily what software or technology you use for your webpages and from your answer I understand it that you use PHP and MySQL. So my question then is, where would you recommend for me to begin... I mean I guess I have to learn how all this works... any tips for a webpage where to learn it? And the stuff you mentioned about already existing scripts etc... I will probably use those but currently I have no idea how it all works, how to implement it.... I moreless wanted to know whether the stuff I want can be done using MySQL and PHP (so that I dont actually spend time learing it only to realize this is not what I wanted, but it seems like this is actually what I need).

I would therefore like to ask you a few very basic general questions:
1. where would you recommend I begin - given I only now javascript, html and I have a personal webpage where I can run MySQL and PHP
2. if I set up a database in mysql, how do I then work with the data... I mean can I for example do my own calculations? How for example do I extract specific data from the DB (lets say comparison of a particular month over various years - just like you have on your webpage)... what language is used to do this? Javascript?

Sorry to bother you with this, just that I am really an absolute beginner and I know it probably will take some time to learn it, but one has to start one day :)

jachym
Posts: 53
Joined: Mon 07 Jan 2013 3:47 pm
Weather Station: WH 1080
Operating System: Win XP
Location: Czech republic

Re: Highcharts Graphs

Post by jachym »

In other words,

I basically understand the concept of MySQL and I see how I would import my updated dayfile into SQL format. That would be the first step. And then I now how to do the last step - how to display the results using HTML and HighCharts. But what I still dont know how to do is how to work with the data in mySQL and how to extract the particular data for the various graphs I then use. I know how to insert the data and how to display them - but not how to work with them...

User avatar
mcrossley
Posts: 7150
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Highcharts Graphs

Post by mcrossley »

To perform the data extracts I call a PHP script on the server using some JavaScript on the web page. The PHP script contains the code to connect othe MySQL data base, perform a SQL query (you need to learn a bit of basic SQL too!) and return the data to the JavaScript formatted as JSON data.

The PHP is fairly straight forward (if you know JavaScript you will have most of the basics for understanding the commonly used PHP syntax).

For instance here is my php for returning the total rainfall each day for a period passed as a parameter...
http://weather.wilmslowastro.com/highch ... p?view=sce

jachym
Posts: 53
Joined: Mon 07 Jan 2013 3:47 pm
Weather Station: WH 1080
Operating System: Win XP
Location: Czech republic

Re: Highcharts Graphs

Post by jachym »

I see... I quickly looked at the file you posted, of course I dont know how exactly it works, but Im beginning to get the point - this is the actual thing that gets the data based on specific criteria. So the major thing is I need to learn SQL to handle the data right?... Thx a lot. How long approximately did it take you to the the data handling necessary to generate all the various graphs and tables you currently have on your web?

User avatar
kingqueen
Posts: 60
Joined: Tue 26 Feb 2013 6:52 pm
Weather Station: Maplin N96GY / Fine Offset 1081
Operating System: Windows 7 HP 64 bit
Location: Wetherby, UK
Contact:

Re: Highcharts Graphs

Post by kingqueen »

I am really enjoying the HighCharts, thank you for that.

There's something odd with the Gusts measurement. Comparing the highcharts version against the Cumulus generated version:

Image

They're over different timescales obviously (my realtime.log not having fully populated yet) but it's not just that. In the Cumulus graph, the gust data is much more erratic than the averaged data, understandably. It goes below and above the average. Yet in the highchart graph, they're always well above, and are static.

I wonder why? And is it something silly I've done?

Thank you
Image

User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Highcharts Graphs

Post by steve »

It looks like you have 'plot latest gust' selected in Cumulus, so it will plot the last gust reading from the station, rather than the peak from the last 10 minutes, which is presumably what the highchart graph is plotting.
Steve

User avatar
mcrossley
Posts: 7150
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Highcharts Graphs

Post by mcrossley »

Also the wind speed differences are probably Highcharts is using the 10 min average, and the Cumulus graph the 'latest'?

User avatar
kingqueen
Posts: 60
Joined: Tue 26 Feb 2013 6:52 pm
Weather Station: Maplin N96GY / Fine Offset 1081
Operating System: Windows 7 HP 64 bit
Location: Wetherby, UK
Contact:

Re: Highcharts Graphs

Post by kingqueen »

Thank you both, that's great. You are quite right, It was as I was using wgust instead of wlatest, so the maximum gust over the last 10 minutes instead of the latest wind reading. That and some tinkering with zindex has produced something much more akin.

Thanks
Image

User avatar
jdc
Posts: 142
Joined: Tue 19 Jun 2012 8:51 pm
Weather Station: Davis VP2 : Instromet
Operating System: Win 10 : Win XP
Location: Berwick-upon-Tweed, England.
Contact:

Re: Highcharts Graphs

Post by jdc »

Thanks to Mark for bringing Highcharts to our attention - and his guidance in this thread.
As someone with average php skills and negligible JS knowledge, I managed to produce graphs from my MySql database with very little difficulty. Formatting the JSON statement was, I found, the trickiest and getting JS 'punctuation' correct.
I can see that the JS learning curve is going to have to continue!

User avatar
actioman
Posts: 118
Joined: Sat 20 Mar 2010 1:01 am
Weather Station: Davis Vantage Pro2 Plus
Operating System: Windows XP SP3
Location: Elvas, Portugal
Contact:

Re: Highcharts Graphs

Post by actioman »

mcrossley wrote:Today's tinkering... My initial stab at putting some Highcharts graphs together. I have just plonked them in my existing page whilst I figure out a better date range selection mechanism.

These are grabing the data via ajax and PHP generated JSON data from a MySQL database on my creaky old laptop out in the shed. So performance may not be up to much.

I also have a problem with the xAxis date stamps being ahead of the data by one day on some graphs - I have yet to figure this out!

After the date range selection mechanism, a decent 'theme' for the graphs is in order.

Edit: I know there is already a 'package' out there that pulls the data from WU, but I do not wish to use WU as a data source, and where is the fun if it is all already figured out for you!
Hello Mark.

With this Highcharts Graphs script that you created it is possible to use data from a mySQL database?

With the "ImportCumulusFile.php" file I created two databases, one of dayfile and one of the monthly logs ( a huge one with my data since April 2010 until now)

I've been trying to use these databases in WUgraphs, but does not recognize the format of the date table and so far I could not speak to the author of the script. I discovered then that here was another method of charting created by you and really liked what I saw:
Highcharts graphs

Tahnks in advance and let's see if I can apply your excellent graphics in my meteoelvas website.
Kind Regards, Manuel.

Image

User avatar
mcrossley
Posts: 7150
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Highcharts Graphs

Post by mcrossley »

Manuel, sure it is possible to MySQL. My "Historic Graphs" are all driven from MySQL using a table built from the Cumulus Dayfile (using ImportCumulusFile.php). Now you are going to ask 'how do I do that?' :lol:

User avatar
gwheelo
Posts: 453
Joined: Wed 11 Jun 2008 7:36 pm
Weather Station: WMR-88
Operating System: Windows 8.1
Location: L'Estartit, Spain
Contact:

Re: Highcharts Graphs

Post by gwheelo »

'how do I do that?'
I hear echos :lol: :lol:

gwheelo
Image

User avatar
mcrossley
Posts: 7150
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Highcharts Graphs

Post by mcrossley »

Arghhh! I'm working on it George - still fiddling now and again, but time...

Post Reply