Welcome to the Cumulus Support forum.

Latest Cumulus MX release 3.9.1 (build 3096) - 31 October 2020 (please see announcement regarding releases since 3.5.0)
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of the NOAA report and Snow Index drop-down menus to 2030)

Use this Wiki link to Download the Software (Cumulus MX / Cumulus 1 and other related items).

Using a backup creates double entries in dayfile

From build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Using a backup creates double entries in dayfile

Post by HansR »

I had to use a backup this morning and catch up the data since then.
So the files in backup/20201014090048/* were copied to the data directory, I restarted CMX and it was catching up OK.
However it resulted in two entries of the 14th of October in dayfile.txt which is not correct.

I enclose the standard logfile (no debug or data was switched on) if needed I can send you the dayfile of the 14th 9h00 (so up to the 13th) and the resulting dayfile with the double entries of the 14th.
You do not have the required permissions to view the files attached to this post.
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

User avatar
mcrossley
Posts: 7024
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Using a backup creates double entries in dayfile

Post by mcrossley »

Can you zip up the backup and send it to me please?

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Using a backup creates double entries in dayfile

Post by HansR »

Backup copy attached
You do not have the required permissions to view the files attached to this post.
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

User avatar
mcrossley
Posts: 7024
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Using a backup creates double entries in dayfile

Post by mcrossley »

Looks like a race condition on the AirLink and the main station. :bash:

Needs fixing!

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Using a backup creates double entries in dayfile

Post by HansR »

OK.

Strange though, because I have a different but seemingly unrelated problem in dayfile when somebody tries to generate pwsFWI of CumulusUtils and then misses a day if it is generated before 9h00. It is OK when generated after 9h00. That person stops CMX before 00h00 and restarts around 8h00 - 8h30. As if a rollover is doubles/missed. I don't understand it yet but we are investigating.

I was investigating that issue, had a problem, used the backup etc... There seems to be something strange around 9h00.

May not have anything to do with this issue. Just saying, you never know if it rings a bell.
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

User avatar
mcrossley
Posts: 7024
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Using a backup creates double entries in dayfile

Post by mcrossley »

I don't think it is related.

Your condition occurred because your "AirLink" is a standalone device. My testing had been using AirLink with a WLL main station, then the AirLink is not started until after the WLL completes it initialisation. Your AirLink was starting during the VP2 initialisation and messing up the date/time of the last data.

The AL logged data as Now, which was the day after where the VP2 history catch-up was up to, so forced a day rollover, which the history catch-up then repeated when it processed through midnight. So the second entry in your dayfile for the 14th should be correct one.

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Using a backup creates double entries in dayfile

Post by HansR »

Aha... that's an interesting accidental test situation :D :clap:
But I don't understand the relation between the Airlink and dayfile. My shortcoming I guess ;)
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Using a backup creates double entries in dayfile

Post by HansR »

Any idea when a new release comes up with this problem solved?
(if I want to investigate the other issue I would like to see this one fixed ;) )
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

User avatar
mcrossley
Posts: 7024
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Using a backup creates double entries in dayfile

Post by mcrossley »

Soon I hope, I'm testing memory leak fixes, and that takes time as I need to let MX run for a day or so each time.

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Using a backup creates double entries in dayfile

Post by HansR »

mcrossley wrote:
Thu 22 Oct 2020 4:53 pm
Soon I hope, I'm testing memory leak fixes, and that takes time as I need to let MX run for a day or so each time.
Ok.
Didn't know C# makes memory leaks possible unless runtime system bugs... Garbage collection not working?
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

freddie
Posts: 904
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux Ubuntu 20.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Using a backup creates double entries in dayfile

Post by freddie »

HansR wrote:
Thu 22 Oct 2020 6:15 pm
mcrossley wrote:
Thu 22 Oct 2020 4:53 pm
Soon I hope, I'm testing memory leak fixes, and that takes time as I need to let MX run for a day or so each time.
Ok.
Didn't know C# makes memory leaks possible unless runtime system bugs... Garbage collection not working?
Usually failure to release resources.
Freddie
Image

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Using a backup creates double entries in dayfile

Post by HansR »

Yes, but how come?
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

freddie
Posts: 904
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux Ubuntu 20.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Using a backup creates double entries in dayfile

Post by freddie »

HansR wrote:
Thu 22 Oct 2020 7:02 pm
Yes, but how come?
Because you need to release file handles, large collections, etc. Perhaps that is happening? I don't know, I haven't looked at the code.
Freddie
Image

User avatar
mcrossley
Posts: 7024
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Using a backup creates double entries in dayfile

Post by mcrossley »

I *think* it was the JSON parser causing it, but it only seems to occur under Mono, so the root issue is probably in there. That also makes diagnosis much harder for me as my dev environment is under Windows.

I've switched to a new parser package and so far it looks better....

User avatar
HansR
Posts: 1439
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Using a backup creates double entries in dayfile

Post by HansR »

Aha, interesting.. would the| json lib have been written in some other language?
That would mean a switch of the json dll i assume... Means i have to switch as well (trying to match the cmx environment to prevent distribution of other files).
https://meteo-wagenborgen.nl / Cumulus 3.8.4 (build: 3094)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.51-v7+ armv7l
Mono JIT compiler version 6.12.0.90 (tarball Fri Sep 4 14:21:54 UTC 2020)

Post Reply