Data not showinjg for last week

[avvy65]

I left CumulusMX v4 running while I was away for the last week, and now I see that the last day of data in the dayfile is 29th May 2024.

I checked the dayfile in CumulusMV v3 and the data is all upto date, but when I run CumulusMX v3 and click on the /Data logs/ dayfile, there is a gap between 27th May and 7th June.

How can I get the information / data to show up in the dayfile so I can see the data.

Thanks

[PaulMy]

Hi,

I left CumulusMX v4 running while I was away for the last week, and now I see that the last day of data in the dayfile is 29th May 2024.
I checked the dayfile in CumulusMV v3 and the data is all upto date, but when I run CumulusMX v3 and click on the /Data logs/ dayfile, there is a gap between 27th May and 7th June.

So a similar issue in both V3 and V4?

If the data is ok and up to date in the monthly log files but not in the dayfile then need to look at the MXDiags log file with the time of the daily rollover and see what it shows during that time.
Fortunately the CreateMissing utility will update the dayfiles if you have the up to date monthly log files.

Enjoy,
Paul

[avvy65]

Thanks but I ran CreateMissing in both versions, and the data is still missing from the day file in CumulusMX v3 and v4

In MXDiags for v4 there are a number of CreateMissing txt files but none for the actual missing dates, and yet it is all there in the dayfile.txt file. Also all the data is there in both the monthly log files of May and june
I have never come across this situation in the 6 years i have been using CumuluMX. There must be some way of getting the data to show up in CumulusMX day file

Any other ideas.

Thanks

[PaulMy]

Hi,

Thanks but I ran CreateMissing in both versions, and the data is still missing from the day file in CumulusMX v3 and v4

In MXDiags for v4 there are a number of CreateMissing txt files but none for the actual missing dates, and yet it is all there in the dayfile.txt file.

Are you saying the data is all there in the actual dayfile.txt but not viewable from the local Dashboard > Data Logs > Dayfile?
If so, is there any message in local Dashboard > Utils > Latest errors?

Is the data in the monthly log files current and in the correct format for each version - i.e. correct date format and separator, and correctly named? Remember v3 log files use different names compared to v4 log files.

Try a new CreateMissing and restart CMX, and then zip up the CreateMissing and latest log file after the restart from MXdiags, and attach to a reply post. Since it happens on both v3 and v4 try v3 first and enable Data and Debug Logging in Program settings > Logging Options before shutting down and doing the CreateMissing and restart.

Enjoy,
Paul

[avvy65]

Yes the data is in the dayfile.txt file, and not viewable from the dashboard.
As far as I can see the data is correctly formatted.
I ran CreateMissing again and restarted CMX, but the data is still missing from the dashboard.
Please note that the above is for CMX v3

I also did the same with CMX v4, and the data is now showing in the dashboard
In Utils /Latest errors, it says that Console Clock: Console is 175 seconds adrift but automatic setting is disabled - you should set the clock manually.
Where do I set it to automatic.

I have uploaded the file requested, but these are for v3.

[PaulMy]

Hi,

In Utils /Latest errors, it says that Console Clock: Console is 175 seconds adrift but automatic setting is disabled - you should set the clock manually.
Where do I set it to automatic.

Yes I noticed that in the MXdiags file. Not likely the main issue but in Station settings > Davis VP/VP2/Vue > Davis Advanced Options > Clock

I have uploaded the file requested, but these are for v3.

But in the attached this shows MX v.4.0.1 build 4023

Code: Select all

2024-06-08 20:00:28.455  ========================== Cumulus MX starting ==========================
2024-06-08 20:00:28.456 Command line: /home/rob/CumulusMX/CumulusMX.dll 
2024-06-08 20:00:28.456 Cumulus MX v.4.0.1 build 4023
2024-06-08 20:00:28.460 Platform       : Linux
2024-06-08 20:00:28.460 OS Description : Linux Mint 21.3
2024-06-08 20:00:28.460 Current culture: English (United Kingdom) [en-GB]
2024-06-08 20:00:28.460 Running as a 64 bit process
2024-06-08 20:00:28.461 Running under userid: root

Possibly wrong file attached?

I am now a bit confused on what in which version is working correctly and what is not... I suspect some issue in dayfile.txt and possibly in monthly log file. If you'd like us to look at them zip them and attach, clearly marking the file and what CMX version.

Enjoy,
Paul

[avvy65]

Oops, sorry about that, yes wrong files, doh!
This should be the correct ones now

[PaulMy]

Hi,
That attachment appears to be a combined file of several CreateMissing and normal CMX startup and running log files.
There are numerous errors in data and the CreateMissing identifies several dates with no monthly data and log file errors.

The CMX startup and running logs show an issue with second instance and duplicate entry in dayfile

Code: Select all

2024-05-26 11:19:02.048 Creating lock file /tmp/cumulusmx-8998.lock
2024-05-26 11:19:02.049 Stop second instance: File Error! - System.UnauthorizedAccessException: Access to the path "/tmp/cumulusmx-8998.lock" is denied.
  at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in <d636f104d58046fd9b195699bcb1a744>:0 
  at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in <d636f104d58046fd9b195699bcb1a744>:0 
  at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
  at CumulusMX.Cumulus.CheckForSingleInstance (System.Boolean Windows) [0x00064] in <0832ac3bdc4144b28316939f2eeaa0bb>:0 
2024-05-26 11:19:02.050 Stop second instance: File HResult - -2147024891
2024-05-26 11:19:02.050 Stop second instance: File HResult - 5
2024-05-26 11:19:02.050 Stop second instance: 'Stop second instance' is disabled - continuing this instance of Cumulus

Errors in dayfile stopped the NOAA reports to be created

Code: Select all

2024-05-26 11:19:02.530 LoadDayFile: Attempting to load the day file
2024-05-26 11:19:02.627 ERROR: Duplicate entry in dayfile for 27/10/2020
2024-05-26 11:19:02.628 ERROR: Duplicate entry in dayfile for 28/10/2020
2024-05-26 11:19:02.763 LoadDayFile: Found 2 duplicate entries, please correct your dayfile and try again

If you used that CMX v3 \data for Migrating to v4 all those errors will carry forward.
If you want to continue with both CMX v3 and v4 then you should fix up the v3 log files and dayfile first so that there is no issues shown in the v3 MXdiags log file before migrating.

Enjoy,
Paul

[avvy65]

Thanks.
Ideally, I want to use just CMX4.

So to be clear, do I have to fix the errors in the dayfile, and then migrate the data again. If that is so, then surely more data will be missing as I have not used CMX v3 for over a week now
Could you tell me how I can fix the errors, apart from the duplicate entries. Also what files are the error in, as I have looked at the CMX v3 dayfile but can't see what you are showing me.

One other thing I've just come across is that the data in in CMX v4 day file is not showing anything for yesterday, probably because yesterday is not in the dayfile!!! So I ran CreateMissing again to get the missing date to show

Thanks

[PaulMy]

Hi,
Yes, the dayfile errors, i.e. duplicate or out of sequence, shown in the Utils > Latest errors and in MXdiags log file need to be fixed. If you have missing days in your monthly log files there is nothing you can do unless you have data somewhere that can be added. The dayfile and any reports depending on it will just be missing those days. Rerunning CreateMissing will not likely do anything more unless you have added missing monthly log file(s).

If you only want to continue with CMX v4, and the migrated data seems ok in the new data and date format (other than what is also identified in the v3 MXdiags logs) I'd proceed with v4 and fix the dayfile. I think rerunning CreateMissing is mute unless you have new or updated monthly log files.

Enjoy,
Paul

[avvy65]

Yes, I am only going to run or use CMX v4. I kept CMX v3 because when I tried to update an earlier v4 version, it returned loads of errors when I tried to run it, as I upgraded that as I usually upgraded v3. I'm not sure if that was the correct way or not.
I'm not aware of any errors in the v4 dayfile as when I ran CreatMissing earlier, it finished without error.

The last monthly log file is 202406log.txt, and the last month ini file ismonth202405.ini.
I'll check the day file in the console tomorrow to see if todays data is there.

Thanks

[avvy65]

It's happened again. yesterdays data is not showing in the console day file of CMX v4.
How do I get it to update automatically, or is it a case of running CreatemMssing or reatrtingCMX.

thanks

[freddie]

Could you zip up your logfiles from your MXdiags folder and attach to a reply to this topic, please? We're only guessing if we can't see the logfiles. It does sound as though you may still have errors in your dayfile.

[avvy65]

Apologies, I didn't think to upload my MXdiags files before.

As requested now

[freddie]

From the log:

Code: Select all

2024-06-09 00:00:00.033 Error writing to dayfile.txt: Access to the path '/home/rob/CumulusMX/data/dayfile.txt' is denied.
2024-06-10 00:00:00.247 Error writing to dayfile.txt: Access to the path '/home/rob/CumulusMX/data/dayfile.txt' is denied.

Looks like permissions are incorrect on your dayfile.txt. Specifically, the "rob" user doesn't have appropriate access.