Missing UV grapths Historic charts

[The PIT]

If you got to https://www.sheffieldweather.net/weathe ... .htm#solar and then look at 2012 you'll see there's no UV data.
I've checked in a random day file and UV values are in there.
The issue is on local cumulusmx interface and the uploaded historic charts.
So where is the issue?

[HansR]

Did you check the Data Series Visibility (Station Settings=>Solar=>Graphs=>)?

[Mapantz]

A very quick scan shows you do have problems in your dayfile.txt

Here is a random one I found:

Code: Select all

30/12/12,18.0,220,12:37,9.3,02:49,17.2,13:10,1020.46,04:29,1022.05,13:15,0.0,00:00,0.0,11.5,69.0,15.5,12:45,78,12:55,84,02:39,0.91,1.6,17.2,13:10,14.6,12:35,6.3,03:51,0.0,00:00,7.5,03:51,13.3,13:10,6.7,02:39,208,3.8,0.0,719,12:45,5.5,12:55
30/12/12,60.0,215,18:45,3.1,02:19,10.1,23:45,992.86,00:00,1000.44,15:59,4.4,19:51,2.2,5.9,439.1,32.3,18:55,75,11:58,89,00:00,1.22,4.3,10.1,23:45,3.6,23:59,-5.2,05:56,1.4,20:05,-2.5,05:56,7.3,22:45,0.6,08:47,229,12.4,0.0,,,,

In fact, your dayfile is full or errors..

Code: Select all

31/03/09,16.0,202,02:05,6.6,01:05,14.6,12:30,1027.94,01:45,1029.83,21:15,0.0,00:05,0.0,10.3,79.4,10.0,01:10,61,12:30,87,01:05,1.37,0.0,13.9,12:25,12.7,12:25,2.2,01:10,0.0,00:00,3.4,01:10,7.9,12:25,4.4,00:05,0,8.0,0.0,865,12:00,4,12:20
01/02/09,32.0,68,13:15,-2.3,18:35,1.4,13:40,1017.81,23:59,1023.06,08:45,0.0,00:05,0.0,-0.2,233.1,17.0,13:15,64,14:50,89,21:10,0.81,0.0,1.2,13:40,-3.1,11:05,-9.5,18:00,0.0,00:00,-8.6,18:00,-1.7,12:55,-4.9,19:05,45,18.5,0.0,301,10:25,1,10:30

It goes from February 09, to March 09, then a repeat of February 09 again, and March 09 after that.

[HansR]

aha...

[The PIT]

Now that's going to be interesting on how to correct.

[The PIT]

Well I suppose I could remove the entries from the dayfile.txt and then run create missing which should do the trick. I'll need to wait for the fixed versions I guess.

[sfws]

I could remove the entries from the dayfile.txt and then run create missing

I am not sure which entries you plan to remove. I also doubt if there is any reason for you to wait until "Create Missing version 1.1.0" is released, as I am not convinced it will help you.

I have not studied your dayfile.txt and your other log files, but from what has been said it appears you may have some duplicate dates, and some dates out of sequence. You need to correct those line ordering problems first, before you start removing anything.

I would have thought your first step would be to try to work out what corrupted your dayfile.txt? The same date being stored twice, usually happens when you stop and restart MX just before, or just after, when it is trying to do an end of day (if your standard interval was every 10 minutes, you should not start/stop within plus or minus 11 minutes of rollover time. Getting dates out of sequence most often happens when you use a spreadsheet or other external mechanism to edit the file, or you move the file from one place to another, or you swap between 2 Cumulus installations. It might be that the lines out of sequence are two sets of exact duplicates, so easy to resolve.

You should get the lines rearranged so they are in date order in a copy of the file, the reasons you discover for causing corruption might identify which line is correct one for a particular date, or may indicate you need to merge two lines with same date. Alternatively, have you got an old backup that is correct (not got lines out of order)? You could build a new file starting from such a backup, that would reduce the number of lines in the file to sort out!

You must understand that the "Create Missing" utility does not replace wrong data, nor does it resequence lines. All it can do is to look for missing fields in a dayfile.txt line, it is designed for seeing if fewer fields exist in some lines, and adding the extra fields to those lines, by an attempt to approximately recalculate them. It is not designed for (e.g.) recalculating maximum UV index and the time of that. You also need to read here to understand why you can never get back the accuracy of original entries!

[The PIT]

MX hasn't been stopped or started before the roll over time. Neither has a spreadsheet been used. The some data isn't a duplication either it's totally different data from what I've seen so cannot simply remove duplications. I suspect a crash probably caused the issue.

Looking at the cumulus file the duplicates are in there so have been there for a while.

If I removed the data from the files but leave the dates it should recreate the data.

[The PIT]

Well I discovered why the UV values are missing they're not in the dayfile.txt. I've removed the duplicates. I guess I've got some editing to do at some time. The trouble starts 2011
Looking at the monthly files the data is in those which is good.