Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

New Script that displays a selected daily statistic for every day

Other discussion about creating web sites for Cumulus that doesn't have a specific subforum

Moderator: daj

sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

New Script that displays a selected daily statistic for every day

Post by sfws »

This topic has been started to give the script in the topic https://cumulus.hosiene.co.uk/viewtopic.php?f=14&t=16880 a better title for future people to find it on forum.


I have now created a newer zip than the one in the other thread.
daily(pick_statistic).zip
This attachment now obsolete, but retained to show how many times it was downloaded and in case anyone finds it useful.

+++++++++++++++
EDIT - Due to some confusion - see my 16 Apr post - I have made it clearer how to change between my schema and Mark's schema in the two critical places - where you display the columns available and where you determine the column name of an associated time-stamp
NEWER ZIP
Updated_following_comments.zip
This 18 Apr zip is the one to use now, it also reinstates the use of 'selected' so it is clear which column was selected in the form and restores the caption that gives the full English description of the selected column(s). See the postings bearing dates of 18 April 2018 or earlier.
+++++++++++++++
[/color]
This time round, I have included my own schema in the script, I have also worked out and typed in the schema that Mark uses giving it a different name.
I have indicated where you should make changes to my script to use his schema (generally just remove the '#' that signifies a PHP comment, but sometimes delete part of my script).
My schema has been designed to make selection of weather statistic required for retrievals easy, the order of columns means that related columns are grouped (e.g. all wind or all rain), and the naming standard I use makes it easy to write into the retrieval script the inclusion of the time-stamp as well as the value (that is complex for Mark's schema). Also I define useful arrays and functions in common scripts that I include in specific ones, so converting my script to something to post here was actually rather a lot of work. Please reply here, if you find this a useful script, to reward me for the time that I have invested in making it suitable to share.

For those who don't look at the original thread, PaulMy asked for a query to show highest daily temperature for every day:
PaulMy wrote:I have been trying to create a new script that would have a table of each day's temp for the 10 years of my data -
Year as column header (11 columns from 2008 to 2018)
Each day's date in the left column (365 lines or 366 in leap year) possibly beak it down in monthly blocks
Each day's high temp filled in the table
The query I produced in response is the main query in my attached script, it adds the time-stamp:

Code: Select all

SELECT DATE_FORMAT(LogDate, '%e %b'),MaxTemp, TMaxTemp, DATE_FORMAT(LogDate, '%m%d') AS d1, YEAR(logdate) AS y1 FROM daytable WHERE MONTH(LogDate) = 1 ORDER BY d1 ASC, y1 DESC  
I decided to make my query available to others, by wrapping a web page round it.

I repeat the following from earlier topic:
The main feature of the script is the query to retrieve all daily values (that of course takes slightly longer the more days you have in your database table) ... to output that in a simple table with days as rows and years as columns showing a single criterion in the table cells (and my script adds time-stamps where available as my schema makes that easy).

I am using this web page on my own server as a replacement for all the functionality in the data summary design originated by David Jamieson and developed further by others. For those who come to this thread expecting more information about what can be done with daily summary statistics, you might find the Wiki article http://wiki.sandaysoft.com/a/Daily_Summary is a useful pointer to some of the many scripts available. Reading the other topics linked from the Wiki might even provide useful background to help you understand and appreciate what I offer above better.
You do not have the required permissions to view the files attached to this post.
Last edited by sfws on Wed 18 Apr 2018 10:56 pm, edited 4 times in total.
Mapantz
Posts: 1775
Joined: Sat 17 Dec 2011 11:55 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 11 x64
Location: Dorset - UK
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by Mapantz »

Hi sfws!

Interesting script, so I thought i'd try it out. I got it working quite easily

I presume this layout is correct?

https://www.warehamwx.co.uk/cumulus/daily_pick.php

A couple of errors i'm currently getting:

Code: Select all

PHP Notice:  Undefined offset: 2 in /home/warehamw/public_html/cumulus/daily_pick.php on line 206
Line 206: echo "<option title='". $dayTableColumnsAndCaptions[$i][2] . '; ' . $i . '=' . $dayTableColumnsAndCaptions[$i][0]

Code: Select all

PHP Notice:  A non well formed numeric value encountered in /home/warehamw/public_html/cumulus/daily_pick.php on line 364
Line 364: $tot[$rowArray['y1']] += $rowArray[$columnNameValue];

If I have some time later in the week, i'll have a go at tarting it up with some CSS.

I like it! Thank you for posting it, greatly appreciated.
Image
User avatar
PaulMy
Posts: 3776
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by PaulMy »

I got it working quite easily
Well done, it only took me several days :oops: and lots of help. But finally got it this evening http://www.komokaweather.com/weather/da ... test1f.php
Now to remove the data that is not in my Dayfile table, and improve the presentation...
Thanks sfws, you've been patient.

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www.komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX www.komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX www. komokaweather.com/cumulusmx4/index.htm

Image
sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

Re: New Script that displays a selected daily statistic for every day

Post by sfws »

Thank you to both of you for trying my script and giving feedback.

The schema I use for my database has worked well for me for many years, and made life simple for me. I guess David A Jamieson when he selected column names in creating ImportCumulusFile (the earliest I saw was version 1.3 when it had been revised by Mark Crossley) did not anticipate the sort of queries that might be used in the future. David, on his original website, simply had a historic graph using his database, his other scripts read the Cumulus log file directly. Mark Crossley's top ten script was something new to use the daily table, but it did not output any time-stamps and so did not consider the column names for those. My own version of Mark's top ten, did add time-stamps to the output and that was just one of several scripts by me that benefited from my choice of column names. Mark has changed David's schema in terms of the number of decimal places in some columns, but never (as far as I know) changed the original column names. Steve Loft when he added SQL generation to MX made provision for people to use their own schema, but used the Mark schema as his default. That is a long paragraph to set the context, but what it boils down to is that scripts that simply work well for me on my set-up (including handling time-stamps), need a lot of difficult tweaking to work with the set ups that others use.

PaulMy, you are using a database table that is set up using Mark's schema (ImportCumulusFile), but the HTML <option> tags in the HTML form are being populated with column names from my (SPAWS) schema. So it works if you happen to select a name in my schema that also appears in Mark's schema, but gives an error like
Unknown column 'TLastRainTip' in 'field list' query failure
if you pick a column in my schema that is not in Mark's schema. Well done for getting there otherwise, but you need to change the code around line 250 to read

Code: Select all

for($i = 0; $i < count($dayfileColumnsAndCaptions); $i++)
{						
		if(substr($dayfileColumnsAndCaptions[$i][1], 0, 10) != 'Time-stamp')
		{
			echo "<option title='". $i . '=' . $dayfileColumnsAndCaptions[$i][0]
				. "' value='" . $dayfileColumnsAndCaptions[$i][0] . "'>" . $dayfileColumnsAndCaptions[$i][1] . '</option>';
		}									
}
PaulMy wrote:and improve the presentation...
Have fun! I have changed the looks on my own website on average probably about twice a year since I started, it is part of how I have learnt CSS. The pictures I posted in the other topic show the current look although the background picture is old it does give a good impression of how my latest arrangement of sensors looks.
======================
Mapantz wrote:PHP Notice: Undefined offset: 2 in /home/warehamw/public_html/cumulus/daily_pick.php on line 206

Line 206: echo "<option title='". $dayTableColumnsAndCaptions[$i][2]
Mapantz, you are using Cumulux MX, that generates a database table using by default what I call Mark's schema. You are correctly using the array that I generated for that schema "$dayfileColumnsAndCaptions" in the HTML form, but it does not include a "$dayTableColumnsAndCaptions[$i][2]" and that is still mentioned in your implementation, hence your error. The 'better' solution I give below will remove this error.
Mapantz wrote:Line 364: $tot[$rowArray['y1']] += $rowArray[$columnNameValue];
Actually, Mapantz, I realised after my post (but before your post) that this error could occur if, when selecting the column to display, you select a time-stamp. The above PHP instruction cannot add for example "06:20" and "09:01"!
There are two ways round this, either amend that line into

Code: Select all

if(is_numeric($rowArray[$columnNameValue])) $tot[$rowArray['y1']] += $rowArray[$columnNameValue];
or
and
this is a much better solution amend the instructions around line 240

Code: Select all

		<li>Select either a time-stamp, or an observation without an associated time-stamp, to display corresponding column</li>
								<li>Select any field containing an observation that is associated with a time-stamp, to display both that column and the associated time-stamp</li>
							</ol>
						</aside>
						<label for="field">Select <em>One</em> Daily Summary Column:</label>
						<div class="enlarged"> <!-- the quoted class name should be associated with a larger font size -->
							<select name="field" size="15">
								<?php
									for($i = 0; $i < count($dayTableColumnsAndCaptions); $i++)
									{						
											// If you are using Mark's schema, use "$dayfileColumnsAndCaptions" version here, and delete other version:
										#	echo "<option title='". $i . '=' . $dayfileColumnsAndCaptions[$i][0]
										#	. "' value='" . $dayfileColumnsAndCaptions[$i][0] . "'>" . $dayfileColumnsAndCaptions[$i][1] . '</option>';
											
											// The following "$dayTableColumnsAndCaptions" version is for SPAWS schema, if using it delete above version: 
											echo "<option title='". $dayTableColumnsAndCaptions[$i][2] . '; ' . $i . '=' . $dayTableColumnsAndCaptions[$i][0]
											. "' value='" . $dayTableColumnsAndCaptions[$i][0] . "'>" . $dayTableColumnsAndCaptions[$i][1] . '</option>';										
									}
								?>
							</select>

to

Code: Select all

								<li>Select an observation without an associated time-stamp, to display corresponding column</li>
								<li>Select any field containing an observation that is associated with a time-stamp, to display both that column and the associated time-stamp</li>
							</ol>
						</aside>
						<label for="field">Select <em>One</em> Daily Summary Column:</label>
						<div class="enlarged"> <!-- the quoted class name should be associated with a larger font size -->
							<select name="field" size="15">
								<?php
									for($i = 0; $i < count($dayfileColumnsAndCaptions); $i++)
									{						
										if(substr($dayfileColumnsAndCaptions[$i][1], 0, 10) != 'Time-stamp')
										{										
											echo "<option title='". $i . '=' . $dayfileColumnsAndCaptions[$i][0]
											. "' value='" . $dayfileColumnsAndCaptions[$i][0] . "'>" . $dayfileColumnsAndCaptions[$i][1] . '</option>';											
										}											
									}
								?>
							</select>
In case it is not clear, the changes are to remove mention of selecting a time-stamp, to change which array is counted in the for loop, and to place an 'if' instruction round the echo within the 'for' loop to exclude any '<option>' line that displays a caption starting with 'Time-stamp'. Apologies, that I did not make it possible for you to get this right with the code in my zip.
Mapantz wrote: I got it working quite easily

I presume this layout is correct?
Glad you got it working easily. Yes, that is the layout I saw when I was testing the script I was going to put in this topic.
Mapantz wrote:If I have some time later in the week, I'll have a go at tarting it up with some CSS.
The previous "MySQL code help" topic initiated by Paul includes some screen shots of how my own CSS makes it look with the HTML <summary> tags formatted as buttons (I don't actually like those button colours, but they were selected for a different script) and various colours for backgrounds and borders of different parts of the page. My website can only be seen on my two PCs, but I am always saying when I have time I will work out a better look. But I like to be outdoors, and I tend to use my computer either when I can't sleep or when I am tired and I retreat back indoors, not the best times to create masterpieces of design!

Given that I have needed to give some replacement code above, I will try to find time to update my zip making it a little clearer in that particular area, and if I remember I will include some CSS as an example starting point to help you (and anyone else) develop CSS to tart it up!
User avatar
PaulMy
Posts: 3776
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by PaulMy »

I have made changes and should now be working correctly http://www.komokaweather.com/weather/dailydata-pick.php

Again thank you sfws for your patience and assistance.

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www.komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX www.komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX www. komokaweather.com/cumulusmx4/index.htm

Image
User avatar
BeaumarisWX
Posts: 357
Joined: Mon 09 Apr 2012 2:38 pm
Weather Station: Davis VP2 Plus - 24hr FARS
Operating System: Windows 10 Pro Hades Canyon
Location: Beaumaris, Tasmania, AU
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by BeaumarisWX »

Hi sfws,

Very Interesting script, so I thought i'd try it out also. : https://southfranklinweather.com/weathe ... istics.php

As I use a narrow template I added overflow scroll, so when the years pass by I can scroll right. Still need to work out how to fix the Date Column when scrolling.

Added other bits of css styling for headers/date/hover over etc.

And also because I was always wanting to see what criterion had been chosen, I added the $columnNameValue to the Months for now,until (if possible) I can work out how to display the Chosen Caption Name instead.

Still have a bit to tidy up, but brilliant script, cheers.

Kind regards,
Tony Beaumaris, Tasmania (AUS)

CMX Mobile : https://beaumaris-weather.com/BWX/
CMX Default: https://beaumaris-weather.com/cumulusmx_default/
Colour Dashboard : https://beaumaris-weather.com/dashborad_color.php
Click below for Saratoga Template :
Image
sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

Re: New Script that displays a selected daily statistic for every day

Post by sfws »

HRVistaWeather wrote:Very Interesting script
Thanks Tony for that comment and for picking up a bug. If I remember this evening, I will update the script yet again, but it is a lovely warm sunny day here today, and I am intending to be mostly outdoors for daylight hours
HRVistaWeather wrote:because I was always wanting to see what criterion had been chosen
Good idea, I did not do that as on my implementation I see the selected criterion still showing in the form because it shows as "selected", it seems in simplifying the script for sharing I lost the "selected" option feature and as your page shows, the list reverts to the start. I think I will slightly revise the script so the table caption is replaced by a heading always visible.

Well done for highlighting my caption bug, I see the caption does not show when the table is opened, because I somehow left out the caption setting line in the script I shared. In my own script I have the caption set in the foreach loop working through column names:

Code: Select all

	foreach($dayTableColumnsAndCaptions as $name)
	{
		if($columnNameValue == $name[0]) $caption = $name[1];
		if('T' . $columnNameValue == $name[0])
		{
In your case, the foreach array name is $dayfileColumnsAndCaptions.

A couple of tiny typing errors of your own....On your time-out message https://southfranklinweather.com/40timeout.shtml it says
Sorry the page you where viewing timed Out.
The 'where' should read 'were' to make sense! And the standard bit on some of your pages
A few differnt displays
should be spelt 'different'!
User avatar
BeaumarisWX
Posts: 357
Joined: Mon 09 Apr 2012 2:38 pm
Weather Station: Davis VP2 Plus - 24hr FARS
Operating System: Windows 10 Pro Hades Canyon
Location: Beaumaris, Tasmania, AU
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by BeaumarisWX »

Hi sfws,
Thanks for the spelling pick up, definitely not my standard. :oops: Worst part is, they been there since last year. A quick fnr has fixed them now.

That would be great if you can fix the "selected" option feature, will look forward to that.

I did try but failed to fix.

Kind regards,
Tony Beaumaris, Tasmania (AUS)

CMX Mobile : https://beaumaris-weather.com/BWX/
CMX Default: https://beaumaris-weather.com/cumulusmx_default/
Colour Dashboard : https://beaumaris-weather.com/dashborad_color.php
Click below for Saratoga Template :
Image
Mapantz
Posts: 1775
Joined: Sat 17 Dec 2011 11:55 am
Weather Station: Davis Vantage Pro2
Operating System: Windows 11 x64
Location: Dorset - UK
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by Mapantz »

Hi there!

Thanks sfws. No more errors, all working good. :)

I have dabbled a bit with the CSS, a bit more to do when I get some more time (maybe this evening)

https://www.warehamwx.co.uk/cumulus/daily_pick.php

I don't have years of data unfortunately, but I do like it!

I was also going to suggest about the 'selected' option, so that's good news.

Thank you for your hard work.
Last edited by Mapantz on Fri 20 Apr 2018 11:48 pm, edited 1 time in total.
Image
sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

Re: New Script that displays a selected daily statistic for every day

Post by sfws »

It was only fitted in just before midnight, but I did manage to update my script yesterday and added it to earliest post. Alterative schema sections clearer, Caption now converted to heading and a value set into it, 'selected' now present in form select. It does not quite work out of the box, because I choose to show coding for both schemas, but I believe I have made it much easier for you to see which snippets of code to uncomment.

I don't display the measurement units anywhere because you may use different units to me, but I will leave indiviual users to work out the best way to add those, the script already turned out a lot more complicated that I originally imagined and planned because of adding the database existance validation!

I have generated a sample style sheet, it is based on the one I used to produce the screen shot in the earlier thread, that is in the zip, and is intended to be just a starting point for whatever styling suits you. As posted there is one table for each month of the year, HTML tags let those be optionally displayed, and I have included monthly summary statistics (total, highest, and lowest in month). All of those features can be changed fairly easily with minimum understanding of my coding.

I believe that is all requests satisfied, and hope I have not introduced any more bugs by burning midnight oil! One possible reason for bugs is that
I have not updated my schema within the script, but I have made a couple of minor changes on my own site, so there are actually substantial differences between my own fully tested script and the one I post here. I apologise if, in generating the script to post here, there are any transcription errors that stop it from working out of the box after you have removed the comments round the snippets that are schema dependent.

Enjoy. I got a book called 'SQL in easy steps' as a present from my neice last Christmas, so my venture into writing SQL is my new voyage of exploring my ten year dataset!
User avatar
kocher
Posts: 233
Joined: Sat 19 Apr 2014 7:57 pm
Weather Station: Davis Vantage Pro2+
Operating System: Windows 10
Location: San Sebastian - Spain
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by kocher »

Greetings to all

This script seems very useful and quick to see the data of our table 'dayfile' :clap:

http://kocher.es/daily-pick.php

Many thanks sfws
Image
User avatar
PaulMy
Posts: 3776
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by PaulMy »

Well done too, kocher. It looks nice!

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www.komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX www.komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX www. komokaweather.com/cumulusmx4/index.htm

Image
User avatar
kocher
Posts: 233
Joined: Sat 19 Apr 2014 7:57 pm
Weather Station: Davis Vantage Pro2+
Operating System: Windows 10
Location: San Sebastian - Spain
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by kocher »

Greetings Paul

Here I attach a compressed file with the edited files of my script.

To work, you must adapt it to your connection data, name of the table in your database ..etc

Download link:

http://kocher.es/cajon/dailypick.rar

Saludos desde San sebastian
Image
sfws
Posts: 1183
Joined: Fri 27 Jul 2012 11:29 am
Weather Station: Chas O, Maplin N96FY, N25FR
Operating System: rPi 3B+ with Buster (full)

Re: New Script that displays a selected daily statistic for every day

Post by sfws »

In https://cumulus.hosiene.co.uk/viewtopic.p ... 584#p90584 (January 2014) I said that I had abandoned my daily summary script using JavaScript (I see from looking now the final version of that managed 25 downloads) that was my project back in 2012 in favour of one using PHP, but did not attach my new script as it was not relevant to that topic. Given that this topic has collected some favourable comments from users of my recent "pick" variant, I have decided to share my PHP version of the daily summary now.
daily(select_2_criterions_and_12_months).zip
If you do make use of either my daily pick or my daily summary, I would appreciate a link back to this topic as a credit.

Please note, I have deleted out references to my CSS files and my navigation menu from this source, although you could use the CSS I provided with my new pick script. Also the script in this zip uses my schema, so if you use the David Jamieson/Mark Crossley/Cumulus MX schema you will need to make all required changes, as I have neither time nor inclination to do that for you. The changes you need are along the lines of those I describe in my daily_pick.php script whose download is in my first post (April 2018) of this newer topic, it has 26 downloads so far counting both first version and latest version.

The difference between this daily summary script and my daily pick script is that my daily summary shows a single 12 month period in a calendar like style, while my daily pick shows all years in a simple table, but you can show/hide month by month. The daily summary script I release here allows you to show one or two criterion (using the terminology I invented for daily summary in my November 2012 post), but does not have the restriction to those selected to be on buttons by David Jamieson, as almost any two will be permitted by the limited validation I built in, but I also offer range and mean from those two (I didn't include validation to ensure a sensible pair are picked). The daily summary version I now share has no JavaScript, so there is no row/column highlighting (although I have not removed all the structure id that was previously used by that highlighting) and all the selections are done using a standard HTML form (you will see the intentional similarities in the form in my new pick script!). The statistical summary presented is slightly different to the final Javascript version in 2014, you can look up the original topic for why the published version of that evolved slightly differently to my own version. I added extra validations for database table existence in my new pick script in the updated version as described earlier in this topic, I have not added that to this older development where you get only the most basic validation.

As I said back in 2014, I still use JavaScript for another variant where there is still a calendar style look, but you choose a particular month and the output table has columns for all available years. It is that variant that is more directly updated by my new daily pick script.
You do not have the required permissions to view the files attached to this post.
User avatar
rogerthn
Posts: 489
Joined: Thu 11 Apr 2013 6:31 pm
Weather Station: Ecowitt GW1000/GW1003
Operating System: Raspberry Pi OS bullseye aarch64
Location: Trollhättan Sweden
Contact:

Re: New Script that displays a selected daily statistic for every day

Post by rogerthn »

Image
Post Reply