Page 1 of 2

Using a backup creates double entries in dayfile

Posted: Thu 15 Oct 2020 10:23 am
by HansR
I had to use a backup this morning and catch up the data since then.
So the files in backup/20201014090048/* were copied to the data directory, I restarted CMX and it was catching up OK.
However it resulted in two entries of the 14th of October in dayfile.txt which is not correct.

I enclose the standard logfile (no debug or data was switched on) if needed I can send you the dayfile of the 14th 9h00 (so up to the 13th) and the resulting dayfile with the double entries of the 14th.

Re: Using a backup creates double entries in dayfile

Posted: Thu 15 Oct 2020 11:23 am
by mcrossley
Can you zip up the backup and send it to me please?

Re: Using a backup creates double entries in dayfile

Posted: Thu 15 Oct 2020 11:27 am
by HansR
Backup copy attached

Re: Using a backup creates double entries in dayfile

Posted: Thu 15 Oct 2020 12:12 pm
by mcrossley
Looks like a race condition on the AirLink and the main station. :bash:

Needs fixing!

Re: Using a backup creates double entries in dayfile

Posted: Thu 15 Oct 2020 1:32 pm
by HansR
OK.

Strange though, because I have a different but seemingly unrelated problem in dayfile when somebody tries to generate pwsFWI of CumulusUtils and then misses a day if it is generated before 9h00. It is OK when generated after 9h00. That person stops CMX before 00h00 and restarts around 8h00 - 8h30. As if a rollover is doubles/missed. I don't understand it yet but we are investigating.

I was investigating that issue, had a problem, used the backup etc... There seems to be something strange around 9h00.

May not have anything to do with this issue. Just saying, you never know if it rings a bell.

Re: Using a backup creates double entries in dayfile

Posted: Thu 15 Oct 2020 1:49 pm
by mcrossley
I don't think it is related.

Your condition occurred because your "AirLink" is a standalone device. My testing had been using AirLink with a WLL main station, then the AirLink is not started until after the WLL completes it initialisation. Your AirLink was starting during the VP2 initialisation and messing up the date/time of the last data.

The AL logged data as Now, which was the day after where the VP2 history catch-up was up to, so forced a day rollover, which the history catch-up then repeated when it processed through midnight. So the second entry in your dayfile for the 14th should be correct one.

Re: Using a backup creates double entries in dayfile

Posted: Thu 15 Oct 2020 3:24 pm
by HansR
Aha... that's an interesting accidental test situation :D :clap:
But I don't understand the relation between the Airlink and dayfile. My shortcoming I guess ;)

Re: Using a backup creates double entries in dayfile

Posted: Thu 22 Oct 2020 4:44 pm
by HansR
Any idea when a new release comes up with this problem solved?
(if I want to investigate the other issue I would like to see this one fixed ;) )

Re: Using a backup creates double entries in dayfile

Posted: Thu 22 Oct 2020 4:53 pm
by mcrossley
Soon I hope, I'm testing memory leak fixes, and that takes time as I need to let MX run for a day or so each time.

Re: Using a backup creates double entries in dayfile

Posted: Thu 22 Oct 2020 6:15 pm
by HansR
mcrossley wrote: Thu 22 Oct 2020 4:53 pm Soon I hope, I'm testing memory leak fixes, and that takes time as I need to let MX run for a day or so each time.
Ok.
Didn't know C# makes memory leaks possible unless runtime system bugs... Garbage collection not working?

Re: Using a backup creates double entries in dayfile

Posted: Thu 22 Oct 2020 6:25 pm
by freddie
HansR wrote: Thu 22 Oct 2020 6:15 pm
mcrossley wrote: Thu 22 Oct 2020 4:53 pm Soon I hope, I'm testing memory leak fixes, and that takes time as I need to let MX run for a day or so each time.
Ok.
Didn't know C# makes memory leaks possible unless runtime system bugs... Garbage collection not working?
Usually failure to release resources.

Re: Using a backup creates double entries in dayfile

Posted: Thu 22 Oct 2020 7:02 pm
by HansR
Yes, but how come?

Re: Using a backup creates double entries in dayfile

Posted: Thu 22 Oct 2020 7:23 pm
by freddie
HansR wrote: Thu 22 Oct 2020 7:02 pm Yes, but how come?
Because you need to release file handles, large collections, etc. Perhaps that is happening? I don't know, I haven't looked at the code.

Re: Using a backup creates double entries in dayfile

Posted: Thu 22 Oct 2020 7:45 pm
by mcrossley
I *think* it was the JSON parser causing it, but it only seems to occur under Mono, so the root issue is probably in there. That also makes diagnosis much harder for me as my dev environment is under Windows.

I've switched to a new parser package and so far it looks better....

Re: Using a backup creates double entries in dayfile

Posted: Fri 23 Oct 2020 4:08 am
by HansR
Aha, interesting.. would the| json lib have been written in some other language?
That would mean a switch of the json dll i assume... Means i have to switch as well (trying to match the cmx environment to prevent distribution of other files).