Highcharts Graphs

[kingqueen]

My web hosts support cron jobs, which can be created using their control panel. I've therefore set that up to create realtime.log, adding realtime.txt every time and trimming it.

Here's my current setup:



Here's the command for the default 120 readings / 2 hours:

cat /home/wetherby/public_html/realtime.log /home/wetherby/public_html/realtime.txt > /home/wetherby/public_html/realtime.tmp && tail -q -n 120 /home/wetherby/public_html/realtime.tmp > /home/wetherby/public_html/realtime.log && rm /home/wetherby/public_html/realtime.tmp

Obviously anybody else will have to adjust the absolute paths to their hosting. No doubt also using clever piping it would be possible to do the above in one command, without a temporary file, and do without the absolute paths. It's quick and dirty - but it works, and it means realtimelog doesn't have to be FTPed up every minute, with the resultant bandwidth considerations.

[mcrossley]

Jachym, in your case it would make sense to ignore all the real time stuff. All you need to do is copy the Cumulus day file up to your web server once a week after it is updated. Then run the importcumulusfile.PHP script to copy the new day file data into the SQL database table. The import script is in the Wiki.

Sorry, but this is brief post as I am on a tablet at the moment and typing is slow.

[jachym]

HI,

thx a lot for reply. Well, the thing is, I am an absolute beginner in this perspective. As I mentioned, so far I used Excel and I only have experience with HTML and javascript - so I for example know how to implement HighCharts on a webpage etc. But I have never worked with MySQL or PHP so I dont really know anything about it apart from roughly what it does.
My question was primarily what software or technology you use for your webpages and from your answer I understand it that you use PHP and MySQL. So my question then is, where would you recommend for me to begin... I mean I guess I have to learn how all this works... any tips for a webpage where to learn it? And the stuff you mentioned about already existing scripts etc... I will probably use those but currently I have no idea how it all works, how to implement it.... I moreless wanted to know whether the stuff I want can be done using MySQL and PHP (so that I dont actually spend time learing it only to realize this is not what I wanted, but it seems like this is actually what I need).

I would therefore like to ask you a few very basic general questions:
1. where would you recommend I begin - given I only now javascript, html and I have a personal webpage where I can run MySQL and PHP
2. if I set up a database in mysql, how do I then work with the data... I mean can I for example do my own calculations? How for example do I extract specific data from the DB (lets say comparison of a particular month over various years - just like you have on your webpage)... what language is used to do this? Javascript?

Sorry to bother you with this, just that I am really an absolute beginner and I know it probably will take some time to learn it, but one has to start one day

[jachym]

In other words,

I basically understand the concept of MySQL and I see how I would import my updated dayfile into SQL format. That would be the first step. And then I now how to do the last step - how to display the results using HTML and HighCharts. But what I still dont know how to do is how to work with the data in mySQL and how to extract the particular data for the various graphs I then use. I know how to insert the data and how to display them - but not how to work with them...

[mcrossley]

To perform the data extracts I call a PHP script on the server using some JavaScript on the web page. The PHP script contains the code to connect othe MySQL data base, perform a SQL query (you need to learn a bit of basic SQL too!) and return the data to the JavaScript formatted as JSON data.

The PHP is fairly straight forward (if you know JavaScript you will have most of the basics for understanding the commonly used PHP syntax).

For instance here is my php for returning the total rainfall each day for a period passed as a parameter...
http://weather.wilmslowastro.com/highch ... p?view=sce

[jachym]

I see... I quickly looked at the file you posted, of course I dont know how exactly it works, but Im beginning to get the point - this is the actual thing that gets the data based on specific criteria. So the major thing is I need to learn SQL to handle the data right?... Thx a lot. How long approximately did it take you to the the data handling necessary to generate all the various graphs and tables you currently have on your web?

[kingqueen]

I am really enjoying the HighCharts, thank you for that.

There's something odd with the Gusts measurement. Comparing the highcharts version against the Cumulus generated version:



They're over different timescales obviously (my realtime.log not having fully populated yet) but it's not just that. In the Cumulus graph, the gust data is much more erratic than the averaged data, understandably. It goes below and above the average. Yet in the highchart graph, they're always well above, and are static.

I wonder why? And is it something silly I've done?

Thank you

[steve]

It looks like you have 'plot latest gust' selected in Cumulus, so it will plot the last gust reading from the station, rather than the peak from the last 10 minutes, which is presumably what the highchart graph is plotting.

[mcrossley]

Also the wind speed differences are probably Highcharts is using the 10 min average, and the Cumulus graph the 'latest'?

[kingqueen]

Thank you both, that's great. You are quite right, It was as I was using wgust instead of wlatest, so the maximum gust over the last 10 minutes instead of the latest wind reading. That and some tinkering with zindex has produced something much more akin.

Thanks

[jdc]

Thanks to Mark for bringing Highcharts to our attention - and his guidance in this thread.
As someone with average php skills and negligible JS knowledge, I managed to produce graphs from my MySql database with very little difficulty. Formatting the JSON statement was, I found, the trickiest and getting JS 'punctuation' correct.
I can see that the JS learning curve is going to have to continue!

[actioman]

Today's tinkering... My initial stab at putting some Highcharts graphs together. I have just plonked them in my existing page whilst I figure out a better date range selection mechanism.

These are grabing the data via ajax and PHP generated JSON data from a MySQL database on my creaky old laptop out in the shed. So performance may not be up to much.

I also have a problem with the xAxis date stamps being ahead of the data by one day on some graphs - I have yet to figure this out!

After the date range selection mechanism, a decent 'theme' for the graphs is in order.

Edit: I know there is already a 'package' out there that pulls the data from WU, but I do not wish to use WU as a data source, and where is the fun if it is all already figured out for you!

Hello Mark.

With this Highcharts Graphs script that you created it is possible to use data from a mySQL database?

With the "ImportCumulusFile.php" file I created two databases, one of dayfile and one of the monthly logs ( a huge one with my data since April 2010 until now)

I've been trying to use these databases in WUgraphs, but does not recognize the format of the date table and so far I could not speak to the author of the script. I discovered then that here was another method of charting created by you and really liked what I saw:
Highcharts graphs

Tahnks in advance and let's see if I can apply your excellent graphics in my meteoelvas website.

[mcrossley]

Manuel, sure it is possible to MySQL. My "Historic Graphs" are all driven from MySQL using a table built from the Cumulus Dayfile (using ImportCumulusFile.php). Now you are going to ask 'how do I do that?'

[gwheelo]

'how do I do that?'

I hear echos

gwheelo

[mcrossley]

Arghhh! I'm working on it George - still fiddling now and again, but time...