Page 2 of 2
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 12:03 am
by water01
His date notation is correct for the USA.
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 12:10 am
by Cambium
Gyvate wrote: ↑Sun 12 Dec 2021 10:00 pm
what does the "old" file look like ?
is the date-time appropriate ?
shouldn't it be
Code: Select all
01/12/21,17:09,7.0,96,6.4,4.6,13.7,82,0.0,8.4,998.0,1057.3,21.5,49,6.5,7.0,7.0,0.0,0,0.16,27.67,5.3,0,0.0,85,0.0,8.4,7.0,7.0
=>
1. date notation should be 01/12/21 and not 12/01/21
2. time notation should be 17:00 and not 05:00 PM (if it has to be PM, then also AM for the times between midnight and noon)
3. not existing values should be marked "0.0" (I have to correct my earlier statement here after looking into my log files in detail)
Doesn't your normal ("old") Dec21log.txt record from CMX via http interface look like the example from mine ?
You have to use the same notation as CMX usually has - the console export may have a different format and needs possibly to be converted beforehand
(your notation might or might not be different from mine in the above example)
Oh snap.. you're right! The date is wrong format. Attached what the working log looks like. Good eye on the 17:00 vs 5pm and the "0.0". I will try those tomorrow.
Thanks for you help!!
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 12:37 am
by Cambium
I couldnt resist to try now.
So I got the CumulusMX Data Logs, Monthly Logs to show up after those changes but the NOAA Month report is not loading. Hmmm
What file/log does that read from? Seems like its not reading when I paste the new data in the Dec21 log so I assume its that file only? Just trying to figure out if its another error in the log Im not seeing or I need to change something else in another file along with it
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 10:03 am
by Gyvate
It is possible that the NOAA reports are build from cumulusmx.db - in fact I don't know.
But I think there is a way to fill gaps in the cumulusmx.db from the log files.
I've seen even a log file where at startup CMX would do that itself - but not sure if it's an automatism.
Might be worth going through the wiki and see if there's something/somewhere mentioned.
You can also use sqlitebrowser (there are versions for WIndows and Linux) to work on the cumulusmx.db.
sqlitebrowser needs to be run with administrative rights/root permission #, sudo under Linux.
There you can manually work on the db or execute SQL commands (e.g. reading your logs and inserting missing/updating existing records).
Of course you need to familiar with mySQL commands/language.
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 12:32 pm
by Cambium
Gyvate wrote: ↑Mon 13 Dec 2021 10:03 am
It is possible that the NOAA reports are build from cumulusmx.db - in fact I don't know.
But I think there is a way to fill gaps in the cumulusmx.db from the log files.
I've seen even a log file where at startup CMX would do that itself - but not sure if it's an automatism.
Might be worth going through the wiki and see if there's something/somewhere mentioned.
You can also use sqlitebrowser (there are versions for WIndows and Linux) to work on the cumulusmx.db.
sqlitebrowser needs to be run with administrative rights/root permission #, sudo under Linux.
There you can manually work on the db or execute SQL commands (e.g. reading your logs and inserting missing/updating existing records).
Of course you need to familiar with mySQL commands/language.
No idea what you just said. lol. I'll try to search for NOAA data stuff in the forum or wiki. Thanks
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 3:02 pm
by PaulMy
I think the NOAA Reports are generated from dayfile.txt
Enjoy,
Paul
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 3:15 pm
by Gyvate
PaulMy wrote: ↑Mon 13 Dec 2021 3:02 pm
I think the NOAA Reports are generated from dayfile.txt
Enjoy,
Paul
If so, which would make sense as the lowest resolution of a monthly NOAA is the day, you will have to make sure that you update your dayfile.txt correspondingly.
There you have the high/low values (where this makes sense) and max values (e.g. solar radiation, uv, gust ...) or summary values (rain) for each day.
So if some are missing after your import or are not complete/updated, you will have to do so based on your newly imported data.
By the way, the cumulusmx.db only holds minute data from the past 7 days for better depiction of graphs for short period reporting.
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 6:18 pm
by sfws
Cambium wrote: ↑Mon 13 Dec 2021 12:32 pm
I'll try to search for NOAA data stuff in the forum or wiki. Thanks
Paul is right, for MX
dayfile.txt is used.
Please see
here in Wiki for precisely where Mark's monthly/annual NOAA report gets its data.
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 8:47 pm
by Cambium
sfws wrote: ↑Mon 13 Dec 2021 6:18 pm
Cambium wrote: ↑Mon 13 Dec 2021 12:32 pm
I'll try to search for NOAA data stuff in the forum or wiki. Thanks
Paul is right, for MX
dayfile.txt is used.
Please see
here in Wiki for precisely where Mark's monthly/annual NOAA report gets its data.
Nope.. For some reason why I edit the Dec21 log thats when it disappears. The day file is there and NOAA report still not reading. I know its probably something simple I cant find. I'll check the Wiki link but its something with the Dec21 log after I edit it
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 9:10 pm
by Gyvate
you have created a second thread for the same (?) issue as it appears.
Looks like your time format in the Dec21log.txt file is not correct (at least not for the manually added records) - the time has always to have 4 places HH:MM - 00:01 and not 0:01 etc. - and 24h notation. When the log file is corrupted, CMX goes" on strike" - so first clear your log file issue and then look into the NOAA.
Re: NOAA Reports - Day/Month Log File
Posted: Mon 13 Dec 2021 9:50 pm
by mcrossley
Cambium wrote: ↑Mon 13 Dec 2021 12:37 am
What file/log does that read from?
Primarily the dayfile.txt.
But the monthly report also reads that months log file to determine the average wind speed for each day.
What does your MXdiags log file say when you generate the reports? It should log the error, the file it was reading and the line number.
Re: NOAA Reports - Day/Month Log File
Posted: Tue 14 Dec 2021 10:14 am
by Gyvate
unfortunately the issue seems to be handled in two threads - also
viewtopic.php?f=40&t=19964
there are also the error messages from MXdiags
latest assessment was that the time format seems to need correction for the manually imported records into the Dec21log.txt file.