PHP code for using historic data

[Nerja]

Has anyone got any PHP code for making use of the historic data that cumulus saves out? I want to be able to pick out high/low temperatures etc for particular dates in the past and am not sure how to do this.

Thanks for any suggestions.

[tobyspond]

Hi,

If you do not have access to MySQL, there is a script available that parses the dayfile: http://cumulus.tnetweather.com/projects/phpdayfile, which produces monthly tables: http://cumulus.tnetweather.com/trends/monthly.

If you have access to MySQL, then import the dayfile and/or monthly log files into a database and query the database for the information you want to show. This is the method I use for my data tables and graphs.

Kerry

[PaulMy]

Hi Kerry,
I have added a MySQL database on my webhost but have no idea yet how to use, add data, etc. I have Cumulus Toolbox uploading my dayfile.txt and have downloaded Kevin's script and uploaded to http://www.komokaweather.com/weather/dayfile.php so a start Now what?

I copied Kevin's css code and saved/named it dayfile.css but zilch understanding of this.

Any quick pointers/lesson?

Paul

[water01]

That depends what you want to do with it Paul. The .css file is just a Style file which defines the way in which your HTML or PHP code displays items in your web page i.e. weatherstyle.css defines the basic style of the Cumulus webpages.

If you look at your MySQL database using phpMyAdmin (should be available in your Hosting service CPanel or equivalent) you will see the you have a table called DayData (or whatever name you called it), each row of which is a day's data from Cumulus as per the dayfile.txt file.

You now need to write some code that extracts the data as you wish from that database/table in the format that you wish (totals, fields etc.) and displays it in a web page which uses the .css file as it's style. Because this is a MySQL database this code will have to be written in a computer language that contains the syntax to extract data from a MySQL database, usually PHP but there are others available. That for instance is how Mark's Near Realtime Data display works which by passing data to the jscript graphing tool HighChart from PHP queries on the database.

Hope that helps.

[Nerja]

Hi,

If you do not have access to MySQL, there is a script available that parses the dayfile: http://cumulus.tnetweather.com/projects/phpdayfile, which produces monthly tables: http://cumulus.tnetweather.com/trends/monthly.

If you have access to MySQL, then import the dayfile and/or monthly log files into a database and query the database for the information you want to show. This is the method I use for my data tables and graphs.

Kerry

Hi Kerry

Thanks for that. Do you have any code you could share to go the MySQL route?

[tobyspond]

Paul - I don't use/ haven't used that script, so I'm not sure I can be of much help. It appears as though you are missing the css file. I think you need to create links for each month as Kevin has done.

There is another set of scripts here: http://weather.wildwoodnaturist.com/downloadsCVW.php that you may be interested in. Again, I have not used those scripts.

Nerja - I'll try to put an example together today to get you started. One caveat: I am self taught and not a programmer.

edited: I have attached to examples of MySQL and PHP code. Both are bare bones i.e. no css applied, but should give you an idea of how it works. You can see the tables produced by tempexample.php here tobyspondweather.com/temp.php. However, the tempexample.php does not have any formatting - that's up to you. The annualexample demonstrates two different ways of querying the database.

Kerry

[PaulMy]

Thanks Kerry, I have the Wildwood NOAA scripts running http://www.komokaweather.com/wxnoaarecords.php and they work great, after a few initial issues that Murry helped to overcome. I have been interested in MySQL for a while but no knowledge where to start. I'll see if I can get Kevin's script properly/completely installed to use my dayfile.txt first and then on to MySQL.

If you look at your MySQL database using phpMyAdmin (should be available in your Hosting service CPanel or equivalent) you will see the you have a table called DayData (or whatever name you called it), each row of which is a day's data from Cumulus as per the dayfile.txt file.

You now need to write some code that extracts the data as you wish from that database/table in the format that you wish (totals, fields etc.) and displays it in a web page which uses the .css file as it's style. Because this is a MySQL database this code will have to be written in a computer language that contains the syntax to extract data from a MySQL database, usually PHP but there are others available.

Thanks for the information. I had created the MySQL database in my GoDaddy account but don't think any data is entered as I haven't got far on how to. I will get into this some more but writing in a computer language will be quite a challenge for me

Thanks for your help and suggestions,
Paul

[tobyspond]

Paul,

I use cumulus toolbox and the importcumulusfile : http://wiki.sandaysoft.com/a/ImportCumulusFile to upload data into mysql. I upload both the dayfile and monthly log file just shortly after midnight.

Give it a try.

Kerry

[water01]

I had created the MySQL database in my GoDaddy account but don't think any data is entered as I haven't got far on how to

To do this install ImportCumulusFile.php on your server using these instructions http://wiki.sandaysoft.com/a/ImportCumulusFile. Make sure you amend the five things mentioned (servername, username, password, database and security key) in the php file before you upload it. You must know these if you have created a MySQL database on your host server.

Once you have done this you then upload your dayfile.txt to the same position as you loaded the php file and run it using the command line in the URL area of your browser e.g.

Code: Select all

http://www.myserver.com/ImportCumulusFile.php?type=dayfile&key=letmein&table=Dayfile&file=./data/Jan10log.txt 

It should tell you it has created the table successfully and you then have a MySQL database to work with. Ideally this import should be done on a daily basis after the Cumulus rollover time e.g Midnight or 9am.

I use Cumulus Toolbox to automate that as it contains functions to upload via FTP and execute Remote Commands which is what you did when you ran the abouve command line.

[PaulMy]

Thanks Kerry and David, that has been very helpful in urging me to delve into this unknown

Paul

[Nerja]

Nerja - I'll try to put an example together today to get you started. One caveat: I am self taught and not a programmer.

edited: I have attached to examples of MySQL and PHP code. Both are bare bones i.e. no css applied, but should give you an idea of how it works. You can see the tables produced by tempexample.php here tobyspondweather.com/temp.php. However, the tempexample.php does not have any formatting - that's up to you. The annualexample demonstrates two different ways of querying the database.

Kerry

Thanks Kerry - the code is exactly what I need to get started. I wouldn't have a clue otherwise, so I will play around with this.

[Andre]

I startup this script for my page but, and it's working Translate it to Dutch.

But now i will place the footer in the script but it's gray out and the page is now longer loaded on the site when i use the footer.

Any idea what is wrong ?

Code: Select all

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
	<meta http-equiv="content-type" content="text/html; charset=utf-8" />
	<title>Historie per dag</title>
	<meta name="keywords" content="" />
	<meta name="description" content="" />
	<link href="styles.css" rel="stylesheet" type="text/css" />
	<link href="dayfile.css" rel="stylesheet" type="text/css" />
	<link rel="stylesheet" href="nivo-slider.css" type="text/css" media="screen" />
  <script type="text/javascript" src="lib/dropmenu/dropdowntabs.js"></script>
</head>
<body>
<div id="bg_clouds">
	<div id="main">
		<!-- header and main menu begins -->
    <?php
      require_once('header_menu.php');
    ?>
    <script type="text/javascript">
    // Select menu current item highlight
    //SYNTAX: tabdropdown.init("menu_id", [item menu, start from 0])
    tabdropdown.init("mainmenu", 0)
    </script>
		<div style="height:13px"></div>
		<!-- header ends -->
		<div class="header_top"></div>
		<div class="header">
			<div id="wrapper">
				<div id="slider-wrapper">        
					<div id="slider" class="nivoSlider">
						<img src="style/header1.jpg" alt="" />
						<img src="style/header2.jpg" alt=""/>
						<img src="style/header3.jpg" alt="" />
					</div>
				</div>
			</div>
			<script type="text/javascript" src="lib/jquery-1.4.3.min.js"></script>
			<script type="text/javascript" src="lib/jquery.nivo.slider.pack.js"></script>
			<script type="text/javascript">
				$(window).load(function() {
				$('#slider').nivoSlider();
				});
			</script>
		</div>
		<div class="header_bot"></div>
		<div style="height:15px"></div>
		
<?php
############################################################################
# A Project of TNET Services, Inc.
############################################################################
#                     E X P E R I M E N T A L   C O D E
############################################################################
#   Project:    Cumulus Project 
#   Package:    dayfile.php
#   Module:     dayfile.php
#   Version:    0.12 - 2010-02-12
#   Purpose:    Reads a copy of the dayfile.txt
#               And displays it
#   Authors:    Kevin W. Reed <programs@tnet.com>
#               TNET Services, Inc.
#   Copyright:  (c) 1992-2010 Copyright TNET Services, Inc.
#
#   License:    
#
#   This program is free software; you can redistribute it and/or
#   modify it under the terms of the GNU General Public License
#   as published by the Free Software Foundation; either version 2
#   of the License, or (at your option) any later version.
#   
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#   
#   You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#   Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, 
#   USA.
#
############################################################################
# Location of the dayfile.txt
# 
# This needs to point to where your dayfile.txt is relative to where this
# script is running.
#
# Normally... This will work because the daytime.txt and this script are
# in the same directory (folder)
$SITE['dayfile']    = './dayfile.txt';
#
# Need to know where the realtime.txt file is too.  We look there to get
# the Unit types of the values for temp, wind, press, and rain...
#
# Normal would be something like
$SITE['realtime']   = './realtime.txt';
#
############################################################################
## OVERRIDE VALUES... REMOVE IF YOU ARE USING THIS SCRIPT
$SITE['realtime']   = './realtime.txt';
$SITE['dayfile']   = './dayfile.txt';
############################################################################

############################################################################
# Really shouldn't need to change anything below here unless you want to
# change the language...
############################################################################

// Fields of realtime.txt file - DON'T CHANGE!! Even for language
$SITE['rfields'] = array(
    "date","time","temp","hum","dew","wspeed","wgust","bearing","rrate",
    "rfall","press","wdir","beaufort","windunit","tempunit","pressunit","rainunit",
    "windrun","presstrend","rmonth","ryear","rfallY","intemp","inhum","wchill",
    "temptrendval","tempTH","TtempTH","tempTL","TtempTL",
    "windTM","TwindTM","wgustTM","TwgustTM",
    "pressTH","TpressTH","pressTL","TpressTL",
    "version","build","rmaxgust","heatindex","humidex","UV","ET","SolarRad",
    "avgbearing","rlhour", "forecast", "daylight", "finestat");
    
// Values in the dayfile.txt - DON'T CHANGE!! Even for language
$SITE['dfields']  = array (
    'date',
    'gusthi','gusthidir','gusthitm',
    'tempmin','tempmintm',
    'tempmax','tempmaxtm',
    'pressmin','pressmintm',
    'pressmax','pressmaxtm',
    'rainratemax','rainratemaxtm', 'raintot',
    'tempavg',
    'winrun', 'winavghi','winavghitm'
    );    

// Values for correlating Windrun units based on windunits

$windrununits = array (
    'm/s' => 'Meters/Sec',
    'mph' => 'Miles',
    'km/h' => 'Kilometers',
    'kts' => 'Nautical Miles' );
    
// Check for Options

check_sourceview();

// Option to override year?

if ( isset($_GET['y'] )) {
    $year = intval($_GET['y']);
} else {
    $year = date('Y');
}

// Option to override month?

if (isset($_GET['m'] )) {
    $month = intval($_GET['m']);
} else {
    $month = date('m');
}

// Get Units Info from the realtime.txt file

// Read data into array
$DATA = get_raw($SITE['realtime'],' ');
$SITE['tempunit'] = ret_rval('tempunit');
$SITE['windunit'] = ret_rval('windunit');
$SITE['pressunit'] = ret_rval('pressunit');
$SITE['rainunit'] = ret_rval('rainunit');
$SITE['windrununit']    = $windrununits[ret_rval('windunit')];

// READ the dayfile into the $raw array
$raw = file($SITE['dayfile']);

// Display the data

echo '<h1>Maandoverzicht</h1>';

display_month($year,$month) ; 

echo 'De gegevens worden iedere nacht om 00:12 bijgewerkt';

// Function to parse the month needed.
// Allows multiple display_month calls above.

function display_month($year,$month) {
    global $raw, $SITE;
    
    $months = array ("Januari","Februari","Maart","April","Mei","Juni","Juli","Augustus",
    "September","Oktober","November","December");

    echo '<h2>' . $months[$month -1 ] . ' ' . $year . '</h2><br/>';

    echo '<table id="daytable">
    <th>Dag</th>
    <th>Hoogste Temp</th><th>Laagste Temp</th><th>Gemiddelde Temp</th>
    <th>Luchtdruk Hoog<br/>(' . $SITE['pressunit'] . ')</th><th>Luchtdruk Laag<br/>(' . 
        $SITE['pressunit'] . ')</th>
    <th>Hoogtse windvlaag<br/>(' . $SITE['windunit'] . ')</th><th>Wind richting</th><th>Wind snelheid<br/>(' . 
        $SITE['windrununit'] . ')</th>
    <th>Totale neerslag<br/>(' . $SITE['rainunit'] . ')</th><th>Max Regeninstensiteit<br/>' . 
        $SITE['rainunit'] . '/hr</th></tr>';
    
    // Set found counter to 0
    $found = 0;
    
    // Read the array now looking for data in the data set we want
    
    foreach($raw as $key) {
        
        if (strpos($key, ";") !==FALSE ){
            $values = preg_split('/;/',$key);
        } else {
            $values = preg_split('/,/',$key);
        }

    
        // Check to see if this is in the date data set
        if ($year == "20" . substr($values[ret_val('date')],6,2) 
            && $month == substr($values[ret_val('date')],3,2) ) {
            
            // Date
            echo '<tr><td class="day">' . substr($values[ret_val('date')],0,2) . '</td>';
            
            // Temps
            echo '<td>' . $values[ret_val('tempmax')] . '&deg;' . $SITE['tempunit'] . '</td><td>' . 
                $values[ret_val('tempmin')] . 
                '&deg;' . $SITE['tempunit'] . '</td><td>' . $values[ret_val('tempavg')] . '&deg;' . 
                $SITE['tempunit'] . '</td>';
            
            // Pressure
            echo '<td>' . $values[ret_val('pressmax')] . '</td><td>' .
                $values[ret_val('pressmin')] . '</td>';
                    
            // Wind
            echo '<td>' . $values[ret_val('gusthi')] . '</td><td>' . 
                windDir($values[ret_val('gusthidir')]) . 
                '</td><td>' . $values[ret_val('winrun')] . '</td>';
        
            // Rain
            echo '<td>' . $values[ret_val('raintot')] . '</td><td>' . 
                $values[ret_val('rainratemax')] . '</td>';
            echo '</tr>' . $LF;
            
            $found++;
        }
    }
    
    echo '</table>';
    
    // If we found no records, output an error:
    
    if (!$found) {
        echo "<p><strong>Sorry, er zijn geen gegevens gevonden voor " . 
            $months[$month -1 ] . ' ' . $year . "</strong></p>";
    }

}

// Return the array position of the variable

function ret_val($lookup) {
    global $SITE;
    
    $rtn = array_search  ( $lookup  , $SITE['dfields'] );
    
    if ($rtn !== FALSE) {
        return( $rtn );
    } else {
        return("-");
    }
}

// Return the array position of the variable

function ret_rval($lookup) {
    global $SITE, $DATA;
    
    $rtn = array_search  ( $lookup  , $SITE['rfields'] );
    
    if ($rtn !== FALSE) {
        return( $DATA[$rtn] );
    } else {
        return("-");
    }
}

// Standard Source view option check

function check_sourceview () {
    global $SITE;
    
    if ( isset($_GET['view']) && $_GET['view'] == 'sce' ) {
        $filenameReal = __FILE__;
        $download_size = filesize($filenameReal);
        header('Pragma: public');
        header('Cache-Control: private');
        header('Cache-Control: no-cache, must-revalidate');
        header("Content-type: text/plain");
        header("Accept-Ranges: bytes");
        header("Content-Length: $download_size");
        header('Connection: close');
        readfile($filenameReal);
        exit;
    }
}

// Calculate Word WinDir from Bearing

function windDir ($winddir) {
    if (!isset($winddir)) {
        return "---";
    }
    $windlabel = array ("N","NNO", "NO", "ONO", "O", "OZO", "ZO", "ZZO", "Z",
       "ZZW","ZW", "WZW", "W", "WNW", "NW", "NNW");
    $dir = $windlabel[ fmod((($winddir + 11) / 22.5),16) ];
    return "$dir";
}

// Function to read the data file with the
// Delimiter provided

function get_raw( $rawfile , $del ) {
    $rawdata = array();
    $fd = fopen( $rawfile, "r" );
    if ( $fd ) {
        $rawinfo = '';
        while (! feof ( $fd ) ) {
            $rawinfo .= fread( $fd, 8192 );
        }
        fclose($fd);
        $rawdata = explode ( $del, $rawinfo );
    } else {
        $rawdata[0]= -9999;
    }
    return $rawdata;
	}
</div>
		<div class="cont_bot"></div>
    <!-- content ends --> 

		<!-- footer begins -->
		<div id="footer">
			<a href="http://validator.w3.org/check/referer" title="This page validates as XHTML 1.0 Transitional"><abbr title="eXtensible HyperText Markup Language">XHTML</abbr></a> | <a href="http://jigsaw.w3.org/css-validator/check/referer" title="This page validates as CSS"><abbr title="Cascading Style Sheets">CSS</abbr></a> | Weather Blues Template by <a href="http://weatherbyyou.com">Weather by You!</a>
			<p style="width:100%; font-family:helvetica; font-size:10px; color:#fff">Copyright  2011. <!-- Do not remove -->Designed by <a href="http://www.metamorphozis.com/free_templates/free_templates.php" title="Free Web Templates">Free Web Templates</a>, coded by <a href="http://www.myfreecsstemplates.com/" title="Free CSS Templates">Free CSS Templates</a><!-- end --></p>
		</div>
		<!-- footer ends -->
  </div>
</div>
</body>
</html>

[beteljuice]

At this point php is 'crashing' into html:

Code: Select all

   } else {
        $rawdata[0]= -9999;
    }
    return $rawdata;
   }
</div>
      <div class="cont_bot"></div>
    <!-- content ends --> 

Try

Code: Select all

   } else {
        $rawdata[0]= -9999;
    }
    return $rawdata;
   }
?>

</div>
      <div class="cont_bot"></div>
    <!-- content ends --> 

[Andre]

Thanks it's fixed now, it's only a little outlined.

[PaulMy]

Paul - I don't use/ haven't used that script, so I'm not sure I can be of much help. It appears as though you are missing the css file. I think you need to create links for each month as Kevin has done.

There is another set of scripts here: http://weather.wildwoodnaturist.com/downloadsCVW.php that you may be interested in. Again, I have not used those scripts.

Nerja - I'll try to put an example together today to get you started. One caveat: I am self taught and not a programmer.

edited: I have attached to examples of MySQL and PHP code. Both are bare bones i.e. no css applied, but should give you an idea of how it works. You can see the tables produced by tempexample.php here tobyspondweather.com/temp.php. However, the tempexample.php does not have any formatting - that's up to you. The annualexample demonstrates two different ways of querying the database.

Kerry

Well I made great stride, for me anyway, and successfully uploaded the dayfile.txt into MySQL. Have done some db queries to help understand and so far so good. I should be able to schedule the daily update through Toolbox. Now how do I get data from that database to display on my website! Looking at Kerry's annualexample.zip and tempexample.zip it looks like I need a dbconfig.php to include, which I presume has the same or similar info as in the settings in the ImportCumulusFile.php to allow it access to my MySQL database. Would you be able to share a sample for my trials?

Thanks for your help,
Paul