Welcome to the Cumulus Support forum.
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Nearly 3 weeks of data lost in log file
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Nearly 3 weeks of data lost in log file
Steve,
I was going through some of the select-a-graphs for July and noticed a straight line joing two lots of data. On closer inspection of my log file for July the 5-minute entries end at 21:20 on 7 July and only start again at 23:36 on 26 July. It would appear I've lost nearly 3 weeks of data.
Can you shed any light on how this could have happened and secondly, is there any way of retrieving it? I am also running WeatherLink and that appears unaffected.
I notice when I restart Cumulus the downloading data box doesn't show any activity and disappears after a few seconds. I'm guessing that's linked to the TimeStamp date/time in today.ini. If I set that back to 7 July is there any chance of recovering the lost data?
I was going through some of the select-a-graphs for July and noticed a straight line joing two lots of data. On closer inspection of my log file for July the 5-minute entries end at 21:20 on 7 July and only start again at 23:36 on 26 July. It would appear I've lost nearly 3 weeks of data.
Can you shed any light on how this could have happened and secondly, is there any way of retrieving it? I am also running WeatherLink and that appears unaffected.
I notice when I restart Cumulus the downloading data box doesn't show any activity and disappears after a few seconds. I'm guessing that's linked to the TimeStamp date/time in today.ini. If I set that back to 7 July is there any chance of recovering the lost data?
- steve
- Cumulus Author
- Posts: 26702
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Nearly 3 weeks of data lost in log file
It's hard to see how Cumulus could have been running all that time without logging anything. Presumably the data was there at some point, you would have noticed if it wasn't? What does you dayfile.txt look like? A corresponding gap?
Could you have done anything that might have caused Cumulus to log to a different location? You could try searching your computer for jul09log.txt.
I doubt that you'll be able to get anything back from the logger by resetting the date; at 5 minute intervals that's 9 days worth of data. Do you normally run Cumulus most of time, or stop it overnight? What have you got in the backup folder?
Could you have done anything that might have caused Cumulus to log to a different location? You could try searching your computer for jul09log.txt.
I doubt that you'll be able to get anything back from the logger by resetting the date; at 5 minute intervals that's 9 days worth of data. Do you normally run Cumulus most of time, or stop it overnight? What have you got in the backup folder?
Steve
- steve
- Cumulus Author
- Posts: 26702
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Nearly 3 weeks of data lost in log file
Interestingly, those dates are both a day after a new beta was released. Could you have installed the one from 6th July (831) to a different location?
Steve
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Re: Nearly 3 weeks of data lost in log file
It's baffling. Cumulus was stopped on the evening of the 7th (probably to upgrade to a new version) and again on 26th July probably for the same reason. It's run on a netbook 24/7 solely for wx use. The dayfile.txt has an entry for all the missing dates which is even odder. During this period Cumulus looked perfectly okay and even recorded a new record but crucially the 15 minute saves weren't happening.steve wrote:It's hard to see how Cumulus could have been running all that time without logging anything. Presumably the data was there at some point, you would have noticed if it wasn't? What does you dayfile.txt look like? A corresponding gap?
I've searched the entire netbook drive and the only jul09log.txt files were those in the backup and data folders.steve wrote:Could you have done anything that might have caused Cumulus to log to a different location? You could try searching your computer for jul09log.txt.
Pity I've lost those records. I don't suppose there's any way of extracting them from WeatherLink is there? Assuming you use that program. Cumulus runs non-stop including overnight. I only stop it when I update the exe. It seems as though after installing the new version it never started logging until the next one overwrote it. I don't keep old versions so it wouldn't be possible to say what version was in use on what date.steve wrote:I doubt that you'll be able to get anything back from the logger by resetting the date; at 5 minute intervals that's 9 days worth of data. Do you normally run Cumulus most of time, or stop it overnight? What have you got in the backup folder?
In the backup folder I have several folders and two correspond to the two dates involved. Unfortunately the log for the second date is missing all records from the two dates mentioned.
Oh well, put it down to experience. Worse things happen at sea.
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Re: Nearly 3 weeks of data lost in log file
Good suggestion but the setup always suggests C:\Program Files\Cumulus and I just go with that. I suppose if anyone else reports a similar problem then it might be linked to that beta but otherwise it has to be down to something at my end but I really wouldn't have a clue what caused it.steve wrote:Interestingly, those dates are both a day after a new beta was released. Could you have installed the one from 6th July (831) to a different location?
C'est la vie!
- steve
- Cumulus Author
- Posts: 26702
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Nearly 3 weeks of data lost in log file
[quote="RayProudfoot"Cumulus was stopped on the evening of the 7th (probably to upgrade to a new version) and again on 26th July probably for the same reason. It's run on a netbook 24/7 solely for wx use. The dayfile.txt has an entry for all the missing dates which is even odder. [/quote]
If Cumulus was running uninterrupted between the times where the data is missing, and given that the dayfile is OK, then I have to admit that the most likely explanation is that it simply failed to write to the log file during that run.
Could you zip up the contents of your Diags folder and upload them, please?
If Cumulus was running uninterrupted between the times where the data is missing, and given that the dayfile is OK, then I have to admit that the most likely explanation is that it simply failed to write to the log file during that run.
Could you zip up the contents of your Diags folder and upload them, please?
It's certainly possible, and probably not too difficult. I think Weatherlink has an option to export a set of data between two dates as text? Could you do one for me for the missing period, and upload it, please?Pity I've lost those records. I don't suppose there's any way of extracting them from WeatherLink is there?
Steve
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Re: Nearly 3 weeks of data lost in log file
Thanks for your quick response on this Steve. I've zipped up the entire Diags folder and it's attached.steve wrote:If Cumulus was running uninterrupted between the times where the data is missing, and given that the dayfile is OK, then I have to admit that the most likely explanation is that it simply failed to write to the log file during that run.RayProudfoot wrote:Cumulus was stopped on the evening of the 7th (probably to upgrade to a new version) and again on 26th July probably for the same reason. It's run on a netbook 24/7 solely for wx use. The dayfile.txt has an entry for all the missing dates which is even odder.
Could you zip up the contents of your Diags folder and upload them, please?
I'll start that now and upload shortly - many thanks!steve wrote:It's certainly possible, and probably not too difficult. I think Weatherlink has an option to export a set of data between two dates as text? Could you do one for me for the missing period, and upload it, please?Pity I've lost those records. I don't suppose there's any way of extracting them from WeatherLink is there?
You do not have the required permissions to view the files attached to this post.
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Re: Nearly 3 weeks of data lost in log file
Hi again Steve,
Here's the data from WeatherLink for the affected period - 7-26 July. I exported the whole of July and removed the data Cumulus already has.
These records are every 15 mins compared to 5-min with Cumulus. That hardly matters if I can fill in that big gap. Thanks once again for brilliant support.
PS. Also attaching dayfile.txt for info.
Here's the data from WeatherLink for the affected period - 7-26 July. I exported the whole of July and removed the data Cumulus already has.
These records are every 15 mins compared to 5-min with Cumulus. That hardly matters if I can fill in that big gap. Thanks once again for brilliant support.
PS. Also attaching dayfile.txt for info.
You do not have the required permissions to view the files attached to this post.
Last edited by RayProudfoot on Tue 04 Aug 2009 6:38 pm, edited 1 time in total.
- steve
- Cumulus Author
- Posts: 26702
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Nearly 3 weeks of data lost in log file
Yes, sadly the logs are full of:
2009-07-17 22:40:00 : 17/07/2009 22:40:00 Error writing to data file: C:\Program Files\Cumulus\data\Jul09log.txt. I/O error 32
2009-07-17 22:40:00 : 17/07/2009 22:40:00 Error writing to data file: C:\Program Files\Cumulus\data\Jul09log.txt. I/O error 103
Error 32 is a file sharing violation - i.e. the file was in use by another process. Error 103 is just a knock-on from that (failed to close the file because it hadn't managed to open it).
From the logs, it looks like there was some sort of problem closing Cumulus down on the evening of the 6th, so I suspect the other process accessing the file was Cumulus itself - the error may have resulted in two copies of Cumulus running at the same time. There's an option in the station settings to warn when this happens - assuming you haven't set this already, it might be worth doing it.
Perhaps Cumulus ought to shout out if it fails to write to the log file; but occasional transient errors may occur, so I wonder if an error pop-up might be over the top?
2009-07-17 22:40:00 : 17/07/2009 22:40:00 Error writing to data file: C:\Program Files\Cumulus\data\Jul09log.txt. I/O error 32
2009-07-17 22:40:00 : 17/07/2009 22:40:00 Error writing to data file: C:\Program Files\Cumulus\data\Jul09log.txt. I/O error 103
Error 32 is a file sharing violation - i.e. the file was in use by another process. Error 103 is just a knock-on from that (failed to close the file because it hadn't managed to open it).
From the logs, it looks like there was some sort of problem closing Cumulus down on the evening of the 6th, so I suspect the other process accessing the file was Cumulus itself - the error may have resulted in two copies of Cumulus running at the same time. There's an option in the station settings to warn when this happens - assuming you haven't set this already, it might be worth doing it.
Perhaps Cumulus ought to shout out if it fails to write to the log file; but occasional transient errors may occur, so I wonder if an error pop-up might be over the top?
Steve
- steve
- Cumulus Author
- Posts: 26702
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Nearly 3 weeks of data lost in log file
Could I have your July log as well, please?
The conversion won't be perfect, due the differences in the things that WL and Cumulus logs, but most things will be correct and the rest not far off.
The conversion won't be perfect, due the differences in the things that WL and Cumulus logs, but most things will be correct and the rest not far off.
Steve
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Re: Nearly 3 weeks of data lost in log file
Would a warning message interfere with Cumulus outputting data to the log? If so then that might be best avoided. How about an option to force an immediate log write when Cumulus first starts? If that works then it's probably okay but in a situation like mine it could well fail and at least the user would be immediately alerted to the problem. We definitely need a warning for failure to write to the log. It's just how we go about it.steve wrote:Yes, sadly the logs are full of:
2009-07-17 22:40:00 : 17/07/2009 22:40:00 Error writing to data file: C:\Program Files\Cumulus\data\Jul09log.txt. I/O error 32
2009-07-17 22:40:00 : 17/07/2009 22:40:00 Error writing to data file: C:\Program Files\Cumulus\data\Jul09log.txt. I/O error 103
Error 32 is a file sharing violation - i.e. the file was in use by another process. Error 103 is just a knock-on from that (failed to close the file because it hadn't managed to open it).
From the logs, it looks like there was some sort of problem closing Cumulus down on the evening of the 6th, so I suspect the other process accessing the file was Cumulus itself - the error may have resulted in two copies of Cumulus running at the same time. There's an option in the station settings to warn when this happens - assuming you haven't set this already, it might be worth doing it.[/quote
I didn't have that option set but it is now. It seems I have just been unlucky with the previous instance not terminating. So should that happen again your warning flag should at least alert me to a potential problem.
steve wrote:Perhaps Cumulus ought to shout out if it fails to write to the log file; but occasional transient errors may occur, so I wonder if an error pop-up might be over the top?
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Re: Nearly 3 weeks of data lost in log file
July log attached. Beggars can't be choosers! I'm just grateful you can give me most of what's I've lost back.steve wrote:Could I have your July log as well, please?
The conversion won't be perfect, due the differences in the things that WL and Cumulus logs, but most things will be correct and the rest not far off.
You do not have the required permissions to view the files attached to this post.
- steve
- Cumulus Author
- Posts: 26702
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Nearly 3 weeks of data lost in log file
I suppose it could pop up an error message if the first write fails - which will be straight away if there is data downloaded from the logger, or within the log interval if not. The message wouldn't stop anything else working. It's a bit messy to force a write if one isn't needed.
Steve
-
- Posts: 3372
- Joined: Wed 06 May 2009 6:29 pm
- Weather Station: Davis VP2 with Daytime FARS
- Operating System: Windows XP SP3
- Location: Cheadle Hulme, Cheshire, England
- Contact:
Re: Nearly 3 weeks of data lost in log file
Am I the first person to report this problem? If so then it's a pretty rare occurence so I don't think you need to do too much. I'll certainly monitor the software after installing a new version so a message box seems fine.steve wrote:I suppose it could pop up an error message if the first write fails - which will be straight away if there is data downloaded from the logger, or within the log interval if not. The message wouldn't stop anything else working. It's a bit messy to force a write if one isn't needed.
- steve
- Cumulus Author
- Posts: 26702
- Joined: Mon 02 Jun 2008 6:49 pm
- Weather Station: None
- Operating System: None
- Location: Vienne, France
- Contact:
Re: Nearly 3 weeks of data lost in log file
I'm not aware of anyone else having the problem; occasionally data has gone 'missing', but I don't think it has ever been down to a problem like this in Cumulus.
Steve