Page 3 of 3
Re: Ecowitt Api Historic Data
Posted: Wed 16 Mar 2022 7:46 pm
by Box
mcrossley wrote: ↑Wed 16 Mar 2022 5:26 pm
You would need to go back to a backup from before "the gap". The logs you posted were from after.
The screen shot showed going back to 22 Feb which was before, but there were no logs from that run.
ok
Strange, I thought I had done that
Maybe I misread the folder
I will try again
Re: Ecowitt Api Historic Data
Posted: Wed 16 Mar 2022 8:10 pm
by Phil23
mcrossley wrote: ↑Wed 16 Mar 2022 2:48 pm
You have the same error as @box was getting "API.GetHistoricData: Exception:"
I have a new build to fix this.
So I assume I can just keep letting things run for the time being & then when the next build is relaeased,
revert to my 9:05am 15/03 data & then restart & let it all process.
Will make sure I have the most relevant level of debugging on when I get to that point.
Re: Ecowitt Api Historic Data
Posted: Wed 16 Mar 2022 10:53 pm
by Box
Box wrote: ↑Wed 16 Mar 2022 7:46 pm
mcrossley wrote: ↑Wed 16 Mar 2022 5:26 pm
You would need to go back to a backup from before "the gap". The logs you posted were from after.
The screen shot showed going back to 22 Feb which was before, but there were no logs from that run.
ok
Strange, I thought I had done that
Maybe I misread the folder
I will try again
Success!!!
I used a backup from before 'the gap' AND deleted my Mar22log.txt file (backed up first of course

), then ran CMX 3172 and the whole mar22log.txt file was reconstructed with 'the gap' filled in
Many thanks for your help
Re: Ecowitt Api Historic Data
Posted: Thu 17 Mar 2022 1:41 am
by Phil23
mcrossley wrote: ↑Wed 16 Mar 2022 5:26 pm
You would need to go back to a backup from before "the gap".
I "Thought" I tried my midnight one before the 9:00 last update issue, but maybe the Linux archiver didn't overwrite as expected.
Box wrote: ↑Wed 16 Mar 2022 10:53 pm
Success!!!
I used a backup from before 'the gap' AND deleted my Mar22log.txt file (backed up first of course

), then ran CMX 3172 and the whole mar22log.txt file was reconstructed with 'the gap' filled in
I may not have that option now due to my backup rotation.
Only have the March 15th 9:05am backup preserved.
Would be no data prior to the above date if I deleted Mar2log.txt
Presently when I restore that I end up with a gap from 9:00am 15th up to the time I start MX after the restore.
Phil.
Re: Ecowitt Api Historic Data
Posted: Thu 17 Mar 2022 8:24 am
by Box
Phil23 wrote: ↑Thu 17 Mar 2022 1:41 am
mcrossley wrote: ↑Wed 16 Mar 2022 5:26 pm
You would need to go back to a backup from before "the gap".
I "Thought" I tried my midnight one before the 9:00 last update issue, but maybe the Linux archiver didn't overwrite as expected.
Box wrote: ↑Wed 16 Mar 2022 10:53 pm
Success!!!
I used a backup from before 'the gap' AND deleted my Mar22log.txt file (backed up first of course

), then ran CMX 3172 and the whole mar22log.txt file was reconstructed with 'the gap' filled in
I may not have that option now due to my backup rotation.
Only have the March 15th 9:05am backup preserved.
Would be no data prior to the above date if I deleted Mar2log.txt
Presently when I restore that I end up with a gap from 9:00am 15th up to the time I start MX after the restore.
Phil.
If you backed up previous installs before upgrades you should have some older backups squirrelled away if you dig
The one I used was from three version ago
I never delete anything, just bung it in an Old Cumulus folder
Check your recycle bin?
Good luck!
Re: Ecowitt Api Historic Data
Posted: Thu 17 Mar 2022 2:58 pm
by PaulMy
The issue with going back too far for historical backup data is that it may miss some of the extremes that are logged with live data. Logger data is recorded at what it is at the logging interval time. While running, Cumulus saves highs and lows from ongoing live data from the station. But at start up Cumulus gets the data provided from the logger, which may have missed some highs and lows between logging times. So in restoring from backup I try to find the latest prior to a data loss/corruption issue - check dayfile and month log files, and also keep in mind how far back the Davis logger has data. This past logged data may not be an issue if Ecowitt data access is unlimited.
Enjoy,
Paul
Re: Ecowitt Api Historic Data
Posted: Thu 17 Mar 2022 5:38 pm
by Box
PaulMy wrote: ↑Thu 17 Mar 2022 2:58 pm
The issue with going back too far for historical backup data is that it may miss some of the extremes that are logged with live data. Logger data is recorded at what it is at the logging interval time. While running, Cumulus saves highs and lows from ongoing live data from the station. But at start up Cumulus gets the data provided from the logger, which may have missed some highs and lows between logging times. So in restoring from backup I try to find the latest prior to a data loss/corruption issue - check dayfile and month log files, and also keep in mind how far back the Davis logger has data. This past logged data may not be an issue if Ecowitt data access is unlimited.
Enjoy,
Paul
Good point
My backup was about 3 weeks old. I guess it might be a good idea to stop/start CMX regularly, say every few days, to generate more backups?
Re: Ecowitt Api Historic Data
Posted: Thu 17 Mar 2022 6:39 pm
by freddie
Box wrote: ↑Thu 17 Mar 2022 5:38 pm
PaulMy wrote: ↑Thu 17 Mar 2022 2:58 pm
The issue with going back too far for historical backup data is that it may miss some of the extremes that are logged with live data. Logger data is recorded at what it is at the logging interval time. While running, Cumulus saves highs and lows from ongoing live data from the station. But at start up Cumulus gets the data provided from the logger, which may have missed some highs and lows between logging times. So in restoring from backup I try to find the latest prior to a data loss/corruption issue - check dayfile and month log files, and also keep in mind how far back the Davis logger has data. This past logged data may not be an issue if Ecowitt data access is unlimited.
Enjoy,
Paul
Good point
My backup was about 3 weeks old. I guess it might be a good idea to stop/start CMX regularly, say every few days, to generate more backups?
Backups are performed daily as routine. Do you need more frequent than that?
Re: Ecowitt Api Historic Data
Posted: Thu 17 Mar 2022 9:36 pm
by Box
freddie wrote: ↑Thu 17 Mar 2022 6:39 pm
Box wrote: ↑Thu 17 Mar 2022 5:38 pm
PaulMy wrote: ↑Thu 17 Mar 2022 2:58 pm
The issue with going back too far for historical backup data is that it may miss some of the extremes that are logged with live data. Logger data is recorded at what it is at the logging interval time. While running, Cumulus saves highs and lows from ongoing live data from the station. But at start up Cumulus gets the data provided from the logger, which may have missed some highs and lows between logging times. So in restoring from backup I try to find the latest prior to a data loss/corruption issue - check dayfile and month log files, and also keep in mind how far back the Davis logger has data. This past logged data may not be an issue if Ecowitt data access is unlimited.
Enjoy,
Paul
Good point
My backup was about 3 weeks old. I guess it might be a good idea to stop/start CMX regularly, say every few days, to generate more backups?
Backups are performed daily as routine. Do you need more frequent than that?
Not sure how the backup system works, the named folders are not 24 hours apart but 24 hours is more than enough
Re: Ecowitt Api Historic Data
Posted: Thu 17 Mar 2022 10:01 pm
by Box
Box wrote: ↑Thu 17 Mar 2022 9:36 pm
freddie wrote: ↑Thu 17 Mar 2022 6:39 pm
Box wrote: ↑Thu 17 Mar 2022 5:38 pm
Good point
My backup was about 3 weeks old. I guess it might be a good idea to stop/start CMX regularly, say every few days, to generate more backups?
Backups are performed daily as routine. Do you need more frequent than that?
Not sure how the backup system works, the named folders are not 24 hours apart but 24 hours is more than enough
Since found the daily backup folders

Re: Ecowitt Api Historic Data
Posted: Sun 20 Mar 2022 5:41 pm
by mcrossley
The patched release 3.15.3 is now available.
Re: Ecowitt Api Historic Data
Posted: Sun 20 Mar 2022 9:14 pm
by Box
mcrossley wrote: ↑Sun 20 Mar 2022 5:41 pm
The patched release 3.15.3 is now available.
Many thanks for your help