Page 1 of 1

Missing data

Posted: Wed 27 Oct 2021 1:03 pm
by Benji60
Hello, I've looked through the wiki and I saw that it was possible to insert missing data, but I didn't understand the exact procedure in my case.
To make it simple, following a power cut, my Ecowitt GW1000 changed IP address yesterday evening (it won't happen again, I set my box to fix the address).
So there is a lot of data missing. I just thought I could reinsert my missing data thanks to my console which saves the data on a csv file.
In attachment I put my file.
How can I transform this file into data that can be inserted into the Oct21log.txt file?
Thank you.

Translated with www.DeepL.com/Translator (free version)

Re: Missing data

Posted: Tue 02 Nov 2021 10:13 pm
by Benji60
Hello,
Is this not possible?
I lost again 8 hours of data due to a disconnected cable while I have an sql database (export in csv) where I have all these data.

Re: Missing data

Posted: Wed 03 Nov 2021 7:22 am
by LindaFNM
Hi,

You’ll need to convert the October text file to a .csv one, then insert the missing lines of data from your backup. Save this file back to a .txt file and copy back into CumulusMX. You can run the Insert Missing utility to update the day file.

Alternatively, I assume you could convert the whole database data from October to a .txt file and use this to replace your October data.

If you need more detail let me know. Before making adjustments (my CumulusMX is running on a Pi) I take a copy and do all of the manipulation on my PC so I don’t make any mistakes on the live running copy.

Re: Missing data

Posted: Wed 03 Nov 2021 7:30 am
by LindaFNM
Sorry, I’ve just realised the backup is from an Ecowitt console. This is going to give you some problems as the data doesn’t exist in the correct format. It depends on how ‘fussy’ you want to be. I’ll need to look at my notes etc again, I get confused with this as I’m (stupidly!) trying to consolidate data from WeatherLink, PC weather, Weather Display and CumulusMX!! I’ve had to build a database as much of the data is held in different ways. You’ll have to convert the October .txt to .csv and insert as best as you can the matching data from the Ecowitt file.

Re: Missing data

Posted: Wed 03 Nov 2021 9:34 am
by LindaFNM
Oh dear, I should have waited till I was properly awake this morning before doing a comprehensive reply!

Another, much simpler option may be available. Are you up loading to Ecowitt.net? If so, you can get the daily max/min figures from there and then amend your day that’s incorrect directly in the CumulusMX dayfile. Unless you had a particularly active period of weather when your data was down this should take care of the things that are important like daily rain, gust temperature etc. In trying to sort out 11 years of mixed mess, I’m learning that some of the data that is provided isn’t necessarily worth the huge amount of effort to recreate it!! Hope you manage to solve the problem.

Re: Missing data

Posted: Fri 05 Nov 2021 8:07 am
by Benji60
Thank you very much for the answer!
Indeed with the csv file of the console I can try to modify it.
I already modified the daily max/min, you are right!
It's just that it's quite tedious to replace the missing data but it's not serious, even if it makes "holes" in the graphs