Page 1 of 3
Add rain totals for month to dayfile or monthly data file
Posted: Sun 01 Sep 2024 4:18 pm
by broadstairs
There is no rain total recorded in the month data file neither is it in the day file. Would it be possible to the add the monthly rainfall total to date to the month data file or the dayfile?
Stuart
Re: Add rain totals for month to dayfile or monthly data file
Posted: Mon 02 Sep 2024 4:15 pm
by Weerhaas
It is in the realtime.txt.
Re: Add rain totals for month to dayfile or monthly data file
Posted: Mon 02 Sep 2024 4:26 pm
by broadstairs
Yes there is also a web tag for current month but what I want to get is the month total at the end of the month which means waiting until after rollover at midnight so you don't miss any last second tips and by that time it is a new month. I think adding something to either month data file or dayfile would be a valuable thing to have and being able to look back at all previous months easily is also a bonus. In addition this would mean calculating monthly averages is made easier.
Stuart
Re: Add rain totals for month to dayfile or monthly data file
Posted: Mon 09 Sep 2024 6:41 pm
by mcrossley
We (Steve before me) have tried to avoid adding data that is fairly easily computed/derived. The day file particularly gives a total for each day so you can total up whatever period you like.
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 10:19 am
by broadstairs
mcrossley wrote: ↑Mon 09 Sep 2024 6:41 pm
We (Steve before me) have tried to avoid adding data that is fairly easily computed/derived. The day file particularly gives a total for each day so you can total up whatever period you like.
Yes in many ways I understand but in the case of the monthly rain total you keep track of it all month anyway so there is no need to calculate anything extra in CMX. In this case all that is needed is to add a rain-total-month-to-date field to either the dayfile or the monthly data file then the month total is the last line of the month data file of the last day of month line in dayfille.
Stuart
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 12:02 pm
by rogerthn
My 2 cents having data in MySQL
Code: Select all
SELECT substr(LogDate,1,7) AS Month, count(*) AS Days, sum(TotRainFall) As MonthRain FROM Dayfile group by substr(LogDate,1,7);
Screenshot 2024-09-10 140120.png
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 12:44 pm
by broadstairs
That assumes you use mysql but I don't and don't see the need especially just for the sake of having these values which in my view could easily be saved by CMX as they are already created.
Stuart
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 1:39 pm
by PaulMy
The NOAA reports have this.
Enjoy,
Paul
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 1:49 pm
by broadstairs
PaulMy wrote: ↑Tue 10 Sep 2024 1:39 pm
The NOAA reports have this.
Enjoy,
Paul
Yes I know but try processing a NOAA report with a program, it's much more complex than reading a data file with specific columns.
Sorry but I just don't see the issue with saving a value which is already calculated in a data file which can very easily be processed programmatically and is already used in reports and web tags. To me it's a no brainer. If I was asking for data to be collected in order to create the value then I could understand it but all I'm suggesting is that the value be saved in a data file, which in my view should have been in the original design spec anyway.
Stuart
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 2:58 pm
by RayProudfoot
@broadstairs, I'm curious. You say you want to process this info using a program. For what purpose? The monthly rain totals are stored in the NOAA reports. Or if you want to extract them then the process @rogerthn described is about as simple as it gets.
I use Microsoft Access to extract info from my dayfile.txt in Cumulus1.
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 4:10 pm
by broadstairs
I believe in automation so I need a process where I can easily extract the data I need. So I can currently get 99.99% of the data I need for my website from the day file or the month data file. The one thing missing is for the rainfall total for the month which frankly I expected to find on one or other of those files and it is in neither which I did not expect, honestly I think it is wrong for it not to be at least in the monthly data file probably as a total-rainfall-to-date value which again is readily available all through the month.
Now yes I could process either file and create the data but this is for my website and I do not want to use the band width to upload all that data just to extract that value every time it is needed to display, yes I could do it locally but why when CMX has done the work right up to month end and then not written to the month data file.
The monthly rain total is needed to be able to calculate the average rainfall for that month which in my view is a very important statistic when looking back over time to see how rainfall varies and compare with the current total for the month.
Stuart
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 4:14 pm
by sutne
Have you checked the different DayFileReaders (YADR)?
They do this, either directly from the Dayfile or from MySql.
Ex:
IMG_0332.png
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 4:18 pm
by broadstairs
sutne wrote: ↑Tue 10 Sep 2024 4:14 pm
Have you checked the different DayFileReaders (YADR)?
They do this, either directly from the Dayfile or from MySql.
No
I simply do not understand why this value is not already available in the fairly comprehensive data files built by CMX. The data exists anyway why not write it to the month data file? This is such a basic simple request why do I have to run all sorts of other software to get the data?
Stuart
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 4:35 pm
by RayProudfoot
broadstairs wrote: ↑Tue 10 Sep 2024 4:18 pm
I simply do not understand why this value is not already available in the fairly comprehensive data files built by CMX. The data exists anyway why not write it to the month data file? This is such a basic simple request why do I have to run all sorts of other software to get the data?
Stuart
I've been running Cumulus for 15 years and never thought the omission of monthly rainfall in any of the data files was a problem. As sutne says try Beteljuice's Day File Reader which will give you all the stats you need on averages for temp, rainfall and wind you could possibly need.
Mine is shown here.
https://www.cheadlehulmeweather.co.uk/b ... ummary.php
Re: Add rain totals for month to dayfile or monthly data file
Posted: Tue 10 Sep 2024 4:50 pm
by PaulMy
There are many ways to skin a cat.
I've often wondered if the Monthly and Dayfile logs could have an additional line at the bottom where a Start and End date can be entered (similar to the Data Log Viewer) and then have the sum for that Start to End dates below the various columns - i.e. Rain, Wind run, etc. This doesn't add any additional fields to dayfile so no new CreateMissing needed. But likely this wouldn't create a new source of data as Stuart is asking.
Like others have mentioned, the beteljuice Annual Data script is my most used go-to website - my own and others!
p.s. Mark, and if I haven't already thanked you for adding that Start Date and End Date to the Data Log Viewer,

it is one of my most used CMX Dashboard functions.
Enjoy,
Paul