Page 1 of 1

missing data ?!

Posted: Fri 18 Aug 2023 8:06 pm
by GaxUser
I'm not sure how to interpret what I'm seeing. Any advise would be appreciated.

the last entry in dayfile.txt is 07/08/23
Aug23log.txt has data from 01/08/23,00:00 to 08/08/23,20:07
The temperature historic chart
temperature.png
What can I do to correct this?

Re: missing data ?!

Posted: Fri 18 Aug 2023 9:30 pm
by PaulMy
Hi,
If your data logger has the missing data then you may be able to get that data in CumulusMX by backdating. What station and datalogger? Do you have the CumulusMX automatic \backup\daily from August 7

Enjoy,
Paul

Re: missing data ?!

Posted: Fri 18 Aug 2023 11:51 pm
by GaxUser
Hi thanks.
I do have the backup/daily files up to August 8. Its a Davis 2. I'm curious though, where does the charts get the its data? Why is it missing data when the data exist in the dayfile and showing data when there is no data? Alos the station and Cumulus have been running where is the data going?

Re: missing data ?!

Posted: Sat 19 Aug 2023 2:02 am
by PaulMy
Hi,
the last entry in dayfile.txt is 07/08/23
Aug23log.txt has data from 01/08/23,00:00 to 08/08/23,20:07
I do have the backup/daily files up to August 8. Its a Davis 2.
That would indicate it stopped during 08/08 so the last good backup would be end of day 07/08.
Depending on your data logger interval, after stopping CMX and copying-in all the backup files to your CumulusMX\data, a CMX restart would get the historical data to catch up from 07/08 to current.
I'm curious though, where does the charts get the its data?
I am not sure but presume from the monthly log files. If there is no logged data, then can't display charts...
Why is it missing data when the data exist in the dayfile and showing data when there is no data?
The dayfile.txt contains the various data for the whole day when it is processed at the end of day. If you have some days missing in dayfile.txt but you do have the data logged in the monthly log files then the CreateMissing utility can populate dayfile.txt for the missing days. If there is no data for a day in the monthly log files then CMX cannot add that day to dayfile.txt. If there is data for a day in dayfile.txt but no data for that day in the monthly log file then I presume it has been removed in error.
Alos the station and Cumulus have been running where is the data going?
If your station console is receiving live data from the ISS, and you have a data logger, the ongoing data is saved at the logger's interval settings, and if CumulusMX is also running at the same time then CMX also receives that live data and logs that in its monthly log files, as well as in the various today, yesterday, monthly .ini files. If CMX is stopped for a while, a CMX restart will get that data logger's historical data and add that to the monthly log and .ini files, and to dayfile at the end of any day while CMX was stopped.

Enjoy,
Paul

Re: missing data ?!

Posted: Sat 19 Aug 2023 8:00 pm
by GaxUser
I'm afraid I wasn't clear. If you look at my first post I point out that the data seemed to stop being logged on the 8th. However, if you look at the graph there are many days going back though June where the graph flat lines (With the exception of two days in July) and then starts recording on the 8th. The dayfiles and monthly logs HAVE data up to the 8th. Through out this whole period the weather station was working properly (I check the web page daily).

My questions would be
1) how come the charts don't show data when I have data (e.g most of July)
2) how come the charts show data when I don't have data (e.g. after the 8th)
3) how come data is no longer getting logged but cumulus recieves the data from the station (e.g no data in the dayfile and monthly log file)
4) how can I fix it

Re: missing data ?!

Posted: Sat 19 Aug 2023 8:29 pm
by PaulMy
Your CUtils Yadr also is missing Aug 8 to 16th and the same for NOAA.
Your CMX template website Reports August is missing Aug 1 - 6, ok for 7 - 13, and then missing to date. What is in your MXdiags file around end-of-day for NOA A reports processing? Also what is being logged in your MXdiags files for the days the data is missing, as well as in utils\utilslog after its daily run?
Could be errors in your dayfile.txt and/or log files, have you checked that thoroughly? You could zip and attach to your dayfile.txt and Aug23log.txt so others could have a look.

Enjoy,
Paul

Re: missing data ?!

Posted: Sat 19 Aug 2023 8:55 pm
by GaxUser
I'm more confused then ever. I shut everything down and restarted and ran all of the CMX modules I usually run. Now everything looks "better" but I'm still missing data for the period of the 8th to the 17th. But at least everything is consistent (e.g. the charts, NOAA, etc. show data for the up to the 8th and then pick up again on the 17th).

Re: missing data ?!

Posted: Sun 20 Aug 2023 6:18 am
by AndyKF650
Hi there

I just looked at your CUtils site and in particular the YADR data and your data set looks good, so you have the data for the "missing" period.

My suggestion would be to recreate the NOAA report for August in the CMX admin site and that should sort you out.

Re: missing data ?!

Posted: Wed 30 Aug 2023 11:23 am
by HansR
When I look at your site it looks as if all issues have been solved? Am I correct?
In general this was a CMX issue so I would have moved it to the CMX thread had I been around, leave it as is now ;)

Re: missing data ?!

Posted: Wed 30 Aug 2023 4:39 pm
by GaxUser
Thanks, I was able to recover the lost data from backups. All is good now