Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Lost all my data

Topics about the Beta trials up to Build 3043, the last build by Cumulus's founder Steve Loft. It was by this time way out of Beta but Steve wanted to keep it that way until he made a decision on his and Cumulus's future.

Moderator: mcrossley

Locked
tumutbound
Posts: 154
Joined: Thu 11 Mar 2010 11:03 am
Weather Station: WH1081
Operating System: Linux, Raspberry Pi (Wheezy)
Location: Port Huon, Tasmania , Australia
Contact:

Lost all my data

Post by tumutbound »

:groan:
In between updating to the latest CumulusMX, evicting spiders from the rain gauge and having a corrupt SD card on my RPi, I've managed to lose the data directory and all the readings I've collected in the last month. Not happy!

Fortunately, the backup directory under CumulusMXDist was recoverable and I still have the data from the pre beta Cumulus on another computer so I may have only lost a days data. Time to go and read the wiki to see how to recover. I'd rather be in the shed making sawdust. :cry:

There are weather stations a couple of kilometres north and south of me and both are on Wunderland so can probably fudge most of todays figures from there.
My weather
CumulusMX (3036)
Raspberry Pi: Wheezy
FineOffset WH1081
WoodburyMan
Posts: 166
Joined: Sun 04 Sep 2011 2:02 am
Weather Station: Ecowitt GW1000
Operating System: Windows 10 20H2
Location: Woodbury, Connecticut, U.S.A.
Contact:

Re: Lost all my data

Post by WoodburyMan »

SD Card corruption on RaspPi is a major issue. If you're using the SD Card, get a Class10 SDHC, and get it from a well known good manufacturer. Apart from that it might be a good idea to get a USB SSD or HDD and power it via a powered USB hub and run Cumulus off that. Another option I've been toying with is running it off a NAS of some sort. You can force it to mount a filesystem using CIFS or NFS if you have a NAS that supports it. However, if it came to that point I would most likely offload it from the RaspPi to a small Intel NUC system I have set up as a Media computer and DVR thats also very low power. CPU is only 15watt TDP and it draws very low power on idle, and has a Samsung 840 Evo mSata SSD.

Likewise, I currently have a image file of my current Pi setup made ready to reflash to a new card. I also have the Pi's home directory mounted on a PC via SMB file share which I back up every day just in case as well.

Apparently the major problems with microSD / SD Cards on Pi's is unexpected shutdowns. The latest firmware versions apparently handle it a little better, and the latest Pi revisions as well such as the B+. I suggest updating to make sure you have the latest. http://www.raspberrypi.org/documentatio ... pdating.md
Woodbury, CT Weather Ecowitt GW1100, Solar, Wind, Rain, Temp, Soil, Lightning
Southington, CT Weather Ecowitt GW2000 Wittboy Solar, Wind, Rain, Temp, Soil, Lightning, CO2/PM2.5/PM10

Running CumuluxMX 3.24.1 b3234 both sites on Windows 11 22H2
tumutbound
Posts: 154
Joined: Thu 11 Mar 2010 11:03 am
Weather Station: WH1081
Operating System: Linux, Raspberry Pi (Wheezy)
Location: Port Huon, Tasmania , Australia
Contact:

Re: Lost all my data

Post by tumutbound »

My webcamera is RPi based and it runs off a USB hard disk. No problems to date.
My Cumulus RPi has a USB flash drive where all data is stored, it too will get a hard disk so the only SD card usage will be during a boot or rpi-update.
One thing I should do to reduce I/O to the SD card is to have all my logging done over the network. I already do this on the camera.

They are a couple of jobs to consider when I've recovered my data and rebuilt all the records etc.
Are there any tools to assist with Cumulus data recovery?
My weather
CumulusMX (3036)
Raspberry Pi: Wheezy
FineOffset WH1081
tumutbound
Posts: 154
Joined: Thu 11 Mar 2010 11:03 am
Weather Station: WH1081
Operating System: Linux, Raspberry Pi (Wheezy)
Location: Port Huon, Tasmania , Australia
Contact:

Re: Lost all my data

Post by tumutbound »

Am I correct in assuming that Cumulus will recreate the record files i.e. alltime.ini, month.ini, monthlyalltime.ini year.ini, the first time it is run or at end of day processing?
Other than the edit menus in Cumulus 1, are there any available tools to help rebuild/recreate missing files or is it just a hard slog with an editor?

I have now put a proper backup scheme in place to save the files to another server - better late than never. :oops:
My weather
CumulusMX (3036)
Raspberry Pi: Wheezy
FineOffset WH1081
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Lost all my data

Post by steve »

The editors in Cumulus 1 can recreate all of the other files from the monthly log files, starting with dayfile.txt if necessary. It is a semi-automatic process, but you have to press the buttons yourself - sorry about that. There is currently no equivalent in Cumulus MX. It's still very much a beta, you should expect to lose data, even it wasn't the fault of the software in this case. Taking a daily backup is a good idea if you are concerned about data loss (as you are right to be).

I'm not aware of anyone having written any tools to aid with data recovery beyond the large amount of work that I have done in Cumulus, but I may be doing someone an injustice.
Steve
User avatar
mcrossley
Posts: 12689
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Lost all my data

Post by mcrossley »

steve wrote:I'm not aware of anyone having written any tools to aid with data recovery beyond the large amount of work that I have done in Cumulus, but I may be doing someone an injustice.
Cumulus 1 does such a good job I don't think it has been necessary.
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Lost all my data

Post by steve »

I think so too, but you can't please everybody...
Steve
Locked