Dayfile.txt create missing

[PaulMy]

While starting to use the data from my MySQL I have just noticed that there is still a lot of missing data after using the Edit Dayfile.txt Create Missing facility. I understand from the Help

The 'Create Missing' facility attempts to create missing entries based on the data it can find in the monthly log files. It's not perfect, and of course it relies on the data in the monthly log files being OK. It creates a file in the Cumulus folder called dayfileeditlog.txt which contains the entries it created - it creates this file even if you click 'cancel', so you can see what it would have added if you had clicked OK. It makes no attempt to create values for 'highest hourly rainfall' but does the best it can with everything else.

From the dates in my dayfile.txt it looks like after I started with Cumulus in October 2008 when there were 17 data fields, since then I have updated Cumulus on or about Sep 17/10 when 2 new fields were added; Mar 27/11 when 20 were added; Aug 24/11 when 3 were added; and October 21/12 when 4 were added.

The monthly log files all seem to be ok.

I have used the Dayfile.Edit facility a few times in the past and all "missing" fields have been added from Nov 11/08 and before up to when I first ran Cumulus; about 1/3rd from Aug 20/10 to Nov 11/08; and then afterwards none of the new fields seem to have been created. This http://www.komokaweather.com/mysql/monthly-data.php highlights some of the missing data. I have noted from the dayfile.txt that when the missing data has been added it is all of the new data fields so that a particular day either has all of the data fields used currently or only the data fields that Cumulus version provided when that day's data was saved. Also, the missing data in that MySQL page all seem to be calculated rather than sensor data.

I have tried the Dayfile Edit again today and no missing data was created, and the dayfileeditlog.txt has 0 kb (I may have deleted this file in the past, just can't recall).

As the Help says "but does the best it can with everything else" so not expecting any miracles but just checking in case someone has any other idea.

Paul

Edit: Just to mention that all the missing data is marked as NULL in my MySQL database.

[steve]

Just to clarify - the 'create missing' facility creates missing entries from log file data, it doesn't add new fields to existing entries. It sounds to me that you're expecting it to do the latter? Its purpose is to make sure that you have an entry in dayfile.txt for all of the days that you have data in your monthly log files, it's not a way of adding the new fields that have been added to Cumulus over time.

[sfws]

it's not a way of adding the new fields that have been added to Cumulus over time.

Also see http://wiki.sandaysoft.com/a/Monthly_lo ... ular_dates

[PaulMy]

Thanks Steve for the explanation. Yes my mistake, I was under the incorrect impression that it would create prior data records as it appeared it had in the earlier period from Oct 23/08 to Nov 11/08, but then I can't recall what occurred or what I might have done at the time. Also, I should have looked at the Wiki as pointed out by sfws and will look at trying the merging example.

Paul

[steve]

I think you must have had some entries missing, for whatever reason, so it was able to create those entries (and hence put in all of the 'new' fields as well).