Welcome to the Cumulus Support forum.
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4019) - 03 April 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024
Cumulus MX V4 beta test release 4.0.0 (build 4019) - 03 April 2024
Legacy Cumulus 1 release 1.9.4 (build 1099) - 28 November 2014
(a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)
Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki
Using a backup creates double entries in dayfile
Moderator: mcrossley
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Using a backup creates double entries in dayfile
I had to use a backup this morning and catch up the data since then.
So the files in backup/20201014090048/* were copied to the data directory, I restarted CMX and it was catching up OK.
However it resulted in two entries of the 14th of October in dayfile.txt which is not correct.
I enclose the standard logfile (no debug or data was switched on) if needed I can send you the dayfile of the 14th 9h00 (so up to the 13th) and the resulting dayfile with the double entries of the 14th.
So the files in backup/20201014090048/* were copied to the data directory, I restarted CMX and it was catching up OK.
However it resulted in two entries of the 14th of October in dayfile.txt which is not correct.
I enclose the standard logfile (no debug or data was switched on) if needed I can send you the dayfile of the 14th 9h00 (so up to the 13th) and the resulting dayfile with the double entries of the 14th.
You do not have the required permissions to view the files attached to this post.
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
- mcrossley
- Posts: 12763
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: Using a backup creates double entries in dayfile
Can you zip up the backup and send it to me please?
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: Using a backup creates double entries in dayfile
Backup copy attached
You do not have the required permissions to view the files attached to this post.
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
- mcrossley
- Posts: 12763
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: Using a backup creates double entries in dayfile
Looks like a race condition on the AirLink and the main station.
Needs fixing!
Needs fixing!
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: Using a backup creates double entries in dayfile
OK.
Strange though, because I have a different but seemingly unrelated problem in dayfile when somebody tries to generate pwsFWI of CumulusUtils and then misses a day if it is generated before 9h00. It is OK when generated after 9h00. That person stops CMX before 00h00 and restarts around 8h00 - 8h30. As if a rollover is doubles/missed. I don't understand it yet but we are investigating.
I was investigating that issue, had a problem, used the backup etc... There seems to be something strange around 9h00.
May not have anything to do with this issue. Just saying, you never know if it rings a bell.
Strange though, because I have a different but seemingly unrelated problem in dayfile when somebody tries to generate pwsFWI of CumulusUtils and then misses a day if it is generated before 9h00. It is OK when generated after 9h00. That person stops CMX before 00h00 and restarts around 8h00 - 8h30. As if a rollover is doubles/missed. I don't understand it yet but we are investigating.
I was investigating that issue, had a problem, used the backup etc... There seems to be something strange around 9h00.
May not have anything to do with this issue. Just saying, you never know if it rings a bell.
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
- mcrossley
- Posts: 12763
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: Using a backup creates double entries in dayfile
I don't think it is related.
Your condition occurred because your "AirLink" is a standalone device. My testing had been using AirLink with a WLL main station, then the AirLink is not started until after the WLL completes it initialisation. Your AirLink was starting during the VP2 initialisation and messing up the date/time of the last data.
The AL logged data as Now, which was the day after where the VP2 history catch-up was up to, so forced a day rollover, which the history catch-up then repeated when it processed through midnight. So the second entry in your dayfile for the 14th should be correct one.
Your condition occurred because your "AirLink" is a standalone device. My testing had been using AirLink with a WLL main station, then the AirLink is not started until after the WLL completes it initialisation. Your AirLink was starting during the VP2 initialisation and messing up the date/time of the last data.
The AL logged data as Now, which was the day after where the VP2 history catch-up was up to, so forced a day rollover, which the history catch-up then repeated when it processed through midnight. So the second entry in your dayfile for the 14th should be correct one.
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: Using a backup creates double entries in dayfile
Aha... that's an interesting accidental test situation
But I don't understand the relation between the Airlink and dayfile. My shortcoming I guess
But I don't understand the relation between the Airlink and dayfile. My shortcoming I guess
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: Using a backup creates double entries in dayfile
Any idea when a new release comes up with this problem solved?
(if I want to investigate the other issue I would like to see this one fixed )
(if I want to investigate the other issue I would like to see this one fixed )
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
- mcrossley
- Posts: 12763
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: Using a backup creates double entries in dayfile
Soon I hope, I'm testing memory leak fixes, and that takes time as I need to let MX run for a day or so each time.
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: Using a backup creates double entries in dayfile
Ok.
Didn't know C# makes memory leaks possible unless runtime system bugs... Garbage collection not working?
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
-
- Posts: 2475
- Joined: Wed 08 Jun 2011 11:19 am
- Weather Station: Davis Vantage Pro 2 + Ecowitt
- Operating System: GNU/Linux Ubuntu 22.04 LXC
- Location: Alcaston, Shropshire, UK
- Contact:
Re: Using a backup creates double entries in dayfile
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: Using a backup creates double entries in dayfile
Yes, but how come?
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
-
- Posts: 2475
- Joined: Wed 08 Jun 2011 11:19 am
- Weather Station: Davis Vantage Pro 2 + Ecowitt
- Operating System: GNU/Linux Ubuntu 22.04 LXC
- Location: Alcaston, Shropshire, UK
- Contact:
Re: Using a backup creates double entries in dayfile
Because you need to release file handles, large collections, etc. Perhaps that is happening? I don't know, I haven't looked at the code.
- mcrossley
- Posts: 12763
- Joined: Thu 07 Jan 2010 9:44 pm
- Weather Station: Davis VP2/WLL
- Operating System: Bullseye Lite rPi
- Location: Wilmslow, Cheshire, UK
- Contact:
Re: Using a backup creates double entries in dayfile
I *think* it was the JSON parser causing it, but it only seems to occur under Mono, so the root issue is probably in there. That also makes diagnosis much harder for me as my dev environment is under Windows.
I've switched to a new parser package and so far it looks better....
I've switched to a new parser package and so far it looks better....
- HansR
- Posts: 5964
- Joined: Sat 20 Oct 2012 6:53 am
- Weather Station: GW1100 (WS80/WH40)
- Operating System: Raspberry OS/Bookworm
- Location: Wagenborgen (NL)
- Contact:
Re: Using a backup creates double entries in dayfile
Aha, interesting.. would the| json lib have been written in some other language?
That would mean a switch of the json dll i assume... Means i have to switch as well (trying to match the cmx environment to prevent distribution of other files).
That would mean a switch of the json dll i assume... Means i have to switch as well (trying to match the cmx environment to prevent distribution of other files).
Hans
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3