In an ideal world how should a Cumulus web site work?

[mcrossley]

Title says it all really. Bear with me as I don't really know where this is going...

Background
I have switched all my pages to be PHP processed now.

I have the dayfile.txt being put into MySQL daily just after it is updated at midnight.

The realtime.txt is being archived into a 'rolling' MySQL table at a one minute sampling period.

The monthly log files are dumped into a MySQL table daily - but currently I do do not make any use of this data.

Random thoughts
Minimising the data transfer from Cumulus to the web site is probably a good thing.

Cumulus has it's 'own' PHP variables file, and the popular Saratoga templates has another.

The one problem with both these solutions is that they use a single file, so a comprise on the update time has to be accepted. Currently they 'normally' update at the Cumulus web update interval (5-20 minutes for most people), but they contain some data that only changes daily, some that changes over a 5-10 minute time scale, and some that changes in real-time.

That says to me that really there should be three PHP variable files, one for each type of data, updating at appropriate intervals. Any PHP script then pulls in which files it needs to complete the task in hand. There may or may not be some overlap in the variables in the real-time and current files, depending on how efficient it is to include two files over duplicating the data (harking back to my DB days long ago I am averse to duplication unless there is a obvious performance benefit).

Then there is the question of the format of the data files. Structure as PHP code, or bare 'csv' text, or use something like JSON or XML (no!). The advantage of PHP code is that it is efficient for PHP pages, but using something like JSON allows the same files to easily be used for client side browser code - I like that efficient, no separate files required for each new client side do-dah that comes along.

Or, are the data files just used to populate MySQL tables, and PHP and client requests pull the data from there?

I'm sure there are no 'right' answers but I'd be interested in informed views.

Asides
I was going to standardise on the Saratoga PHP data file format, but I don't use the templates and I then thought I need the 'full' files plus a real-time sub-set of the variables in another file. So if I was breaking away from the 'standard' why not create three JSON format data files?

The next version of the SS gauges will probably have the option to move away from the realtimegaugesT.txt file, I am looking to implement an optional PHP script to generate the required JSON data on the fly for those that already have the data sat in PHP variables anyway! - Like me

Whacky thought
Another issue is that all the 'weather programs' use different names for the same variables, what is really required is a 'standard' JSON data format and naming convention that all the developers started using. Imagine that, templated solutions you could use across platforms with minimal change! Pie in the sky.

[steve]

I was seriously considering using JSON in the version of Cumulus that I'm working on. I'd be happy for you to specify the format/naming, in fact it would make things much easier for me if you did. And as the new version is written in a language/environment which is reasonably compatible with Delphi, it wouldn't be too hard to backport any new code that I write to the current version of Cumulus.

[laulau]

The next version of the SS gauges will probably have the option to move away from the realtimegaugesT.txt file, I am looking to implement an optional PHP script to generate the required JSON data on the fly for those that already have the data sat in PHP variables anyway! - Like me

Hi Mark,
Do you plan to share the php script that outputs JSON data from a cumuluswetag.php file in a near future ?
Thanks

[mcrossley]

Sometime!

The version I am using is not a straightforward generate JSON from PHP tags, it also implements a HTTP long polling technique. I intend to make this available in the next release - as an option for those who have fully PHP enabled websites.

If you already have all the relevant tags available, then a simple PHP equivalent of the text file would be something like this (untested cut'n'paste)...

Code: Select all

<?php
//----------------------------------------------------------------------
// Generate JSON data for the Weather SteelSeries Gauges
// Author: Mark Crossley
//
// Ver 0.1 - 05/01/14 - Initial release, 0.X = a work in progress, breakages and changes on the fly likely! 
//----------------------------------------------------------------------
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program.  If not, see <http://www.gnu.org/licenses/>.
//----------------------------------------------------------------------

$RealtimeFilename = $_SERVER['DOCUMENT_ROOT'].'/cumuluswebtagsRealtime.php';  // realtime PHP variable file
$CurrentFilename = $_SERVER['DOCUMENT_ROOT'].'/cumuluswebtags.php';  // current PHP variable file

// set up the response headers
header('Cache-Control: private');
header('Cache-Control: no-cache, must-revalidate');
header('Content-type: text/json');

$response = decodeCumulus($CurrentFilename, $RealtimeFilename);

function decodeCumulus($currFile, $RealFile) {
	global $response;
	error_reporting(E_ALL);

	try {
		// incorporate all the Cumulus variables
		include($currFile);
		include($RealFile);

		return array(
			'date' => $timehhmmss,
			'timeUTC' => $timeUTC_ss,
			'temp' => $temp,
			'tempTL' => $tempTL,
			'tempTH' => $tempTH,
			'intemp' => $intemp,
			'dew' => $dew,
			'dewpointTL' => $dewpointTL,
			'dewpointTH' => $dewpointTH,
			'apptemp' => $apptemp,
			'apptempTL' => $apptempTL,
			'apptempTH' => $apptempTH,
			'wchill' => $wchill,
			'wchillTL' => $wchillTL,
			'heatindex' => $heatindex,
			'heatindexTH' => $heatindexTH,
			'humidex' => $humidex,
			'wlatest' => $wlatest,
			'wspeed' => $wspeed,
			'wgust' => $wgust,
			'wgustTM' => $wgustTM,
			'bearing' => $bearing,
			'avgbearing' => $avgbearing,
			'press' => $press,
			'pressTL' => $pressTL,
			'pressTH' => $pressTH,
			'pressL' => $pressL,
			'pressH' => $pressH,
			'rfall' => $rfall,
			'rrate' => $rrate,
			'rrateTM' => $rrateTM,
			'hum' => $hum,
			'humTL' => $humTL,
			'humTH' => $humTH,
			'inhum' => $inhum,
			'SensorContactLost' => $SensorContactLost,
			'forecast' => $forecastenc,
			'tempunit' => $tempunitnodeg,
			'windunit' => $windunit,
			'pressunit' => $pressunit,
			'rainunit' => $rainunit,
			'temptrend' => $temptrend,
			'TtempTL' => $TtempTL,
			'TtempTH' => $TtempTH,
			'TdewpointTL' => $TdewpointTL,
			'TdewpointTH' => $TdewpointTH,
			'TapptempTL' => $TapptempTL,
			'TapptempTH' => $TapptempTH,
			'TwchillTL' => $TwchillTL,
			'TheatindexTH' => $TheatindexTH,
			'TrrateTM' => $TrrateTM,
			'ThourlyrainTH' => $ThourlyrainTH,
			'LastRainTipISO' => $LastRainTipISO,
			'hourlyrainTH' => $hourlyrainTH,
			'ThumTL' => $ThumTL,
			'ThumTH' => $ThumTH,
			'TpressTL' => $TpressTL,
			'TpressTH' => $TpressTH,
			'presstrendval' => $presstrendval,
			'Tbeaufort' => $Tbeaufort,
			'TwgustTM' => $TwgustTM,
			'windTM' => $windTM,
			'bearingTM' => $bearingTM,
			'BearingRangeFrom10' => $BearingRangeFrom10,
			'BearingRangeTo10' => $BearingRangeTo10,
			'UV' => $UV,
			'UVTH' => $UVTH,
			'SolarRad' => $SolarRad,
			'SolarTM' => $solarTH,
			'CurrentSolarMax' => $CurrentSolarMax,
			'domwinddir' => $domwinddir,
			'WindRoseData' => array_map('intval', explode(',', $WindRoseData)),
			'windrun' => $windrun,
			'cloudbasevalue' => $cloudbasevalue,
			'cloudbaseunit' => $cloudbaseunit,
			'version' => $version,
			'build' => $build,
			'ver' => '12'
			);
	} catch (Exception $e) {
		return $e->getMessage() . ' line:' . $e->getLine();
	}
}

// JSON encode the response
echo json_encode($response);
// all done!
?>

[laulau]

Thanks for that.
A copy/paste will shorten what I started this morning.
edit:

Code: Select all

<?php
require_once("cumuluswebtags.php");
$return[temp24h] = $temp24h;
$return[lastrain] = $lastrain;
$return[presstrend] = $presstrend;
$return[temptrendtext] = $temptrendtext;
echo json_encode($return);
?>

I can imagine that getIndexPageData.php script which updates your main webpage is similar

[n9mfk]

Can this be used in the current version of ss gauges?
that would cumuluswebtagsRealtime.php' look like?

I like to lean new ways to do things
I tried the above code it gives you

Code: Select all

{"date":"11:50:01","timeUTC":null,"temp":"27.7","tempTL":"22.2","tempTH":"28.8","intemp":"72.4","dew":"16.0","dewpointTL":"13.6","dewpointTH":"20.4","apptemp":"17.3","apptempTL":"11.4","apptempTH":"18.5","wchill":"18.9","wchillTL":"11.8","heatindex":"27.7","heatindexTH":"28.8","humidex":"27.7","wlatest":"8.0","wspeed":"9.0","wgust":"17.0","wgustTM":"21.0","bearing":"22","avgbearing":"5","press":"30.596","pressTL":"30.308","pressTH":"30.627","pressL":"29.010","pressH":"30.872","rfall":"0.00","rrate":"0.00","rrateTM":"0.00","hum":"61","humTL":"56","humTH":"82","inhum":"42","SensorContactLost":"0","forecast":" Mostly clear and cooler. ","tempunit":"F","windunit":"mph","pressunit":"in","rainunit":"in","temptrend":"+1.5","TtempTL":"07:50","TtempTH":"11:37","TdewpointTL":"08:32","TdewpointTH":"00:00","TapptempTL":"08:29","TapptempTH":"11:39","TwchillTL":"11.8","TheatindexTH":"11:37","TrrateTM":"00:00","ThourlyrainTH":"00:00","LastRainTipISO":"2014-11-30 06:42","hourlyrainTH":"0.00","ThumTL":"10:44","ThumTH":"01:29","TpressTL":"00:00","TpressTH":"11:04","presstrendval":"0.000","Tbeaufort":"F3","TwgustTM":"10:34","windTM":"11.3","bearingTM":"344","BearingRangeFrom10":"330","BearingRangeTo10":"090","UV":"0.0","UVTH":"0.0","SolarRad":"0","SolarTM":"0","CurrentSolarMax":"424","domwinddir":"N","WindRoseData":[12118,7128,1645,130,23,0,0,0,0,0,0,0,0,0,96,6048],"windrun":"94.0","cloudbasevalue":"2659","cloudbaseunit":"ft","version":"1.9.4","build":"1099","ver":"12"}

Lets say you wet to print on a php file the temp 27.7 how do you do it ?
thanks beau

[water01]

That says to me that really there should be three PHP variable files, one for each type of data, updating at appropriate intervals. Any PHP script then pulls in which files it needs to complete the task in hand. There may or may not be some overlap in the variables in the real-time and current files, depending on how efficient it is to include two files over duplicating the data (harking back to my DB days long ago I am averse to duplication unless there is a obvious performance benefit).

I came to that conclusion, but the problem with doing this in the current version of Cumulus is you can only have one interval time for upload. I don't know if Steve intends to change this but it would be nice to have a interval time per file for each upload, and then you could achieve this and many more options. Thinking this further it might be useful to have a realtime interval per file as well, although at the moment I cannot think why you would need it, but I am sure someone will come up with a reason.

[duke]

but the problem with doing this in the current version of Cumulus is you can only have one interval time for upload.

That's not really a problem, there are several software's out there that will reliably upload a file (after you have processed it) at any time interval you desire.

For my money and as already mentioned in a thread here this week. You cannot do better (IMO) than Syncovery.

[n9mfk]

I have been think about this
let say your doing something where you went realtime data and a five minute ftp
for the ftp I use cumuluswebtags.php' for realtime you use cumuluswebtagsRealtime.php from interfering
as both files have some of the same tags and both files are in the script
thoughts?
Beau

[duke]

You could rename the duplicated tags if you wish.

However, if you call the cumuluswebtagsRealtime.php tag file after cumuluswebtags.php those duplicate and more recent tags would simply over ride the previously loaded tags.

Code: Select all

require_once ( 'cumuluswebtags.php ' );  // loads all initial tags
require_once ( 'cumuluswebtagsRealtime.php' );  // loads any more frequent tags

[mcrossley]

That is what I do, include both, but the realtime after the main file so it overwrites the values.

For the longer term variables (monthly/yearly records) I have those in a separate file again that gets updated/uploaded at the archive interval, but I only include that file in the pages that require them.

[n9mfk]

How can I get deg symbol to show in above code


im thinking

Code: Select all

\xc2\xb0

[mcrossley]

*If* you are using UTF8 everywhere, *and* one of the latest versions of PHP, *and* your web server is configured to send JSON as UTF8, you should be able to insert the character directly, or use one of the escape functions for JSON (forget what they are called now), or play safe and escape it directly at the start

[n9mfk]

Hi Mark
I do not understand what you mean by play safe and escape it directly at the start
is this what you mean

Code: Select all

<?php
error_reporting(E_ALL);
ini_set("display_errors", 1);

// set up the response headers
header('Cache-Control: private');
header('Cache-Control: no-cache, must-revalidate');
header('Content-type: text/json; charset= UTF-8');


$response = "\xc2\xb0";

echo json_encode($response,JSON_UNESCAPED_UNICODE);

?>

http://n9mfk.info/newidea/mydata/test.php

im mot sure that correct how would you do it ?