Lost all my data

[tumutbound]

In between updating to the latest CumulusMX, evicting spiders from the rain gauge and having a corrupt SD card on my RPi, I've managed to lose the data directory and all the readings I've collected in the last month. Not happy!

Fortunately, the backup directory under CumulusMXDist was recoverable and I still have the data from the pre beta Cumulus on another computer so I may have only lost a days data. Time to go and read the wiki to see how to recover. I'd rather be in the shed making sawdust.

There are weather stations a couple of kilometres north and south of me and both are on Wunderland so can probably fudge most of todays figures from there.

[WoodburyMan]

SD Card corruption on RaspPi is a major issue. If you're using the SD Card, get a Class10 SDHC, and get it from a well known good manufacturer. Apart from that it might be a good idea to get a USB SSD or HDD and power it via a powered USB hub and run Cumulus off that. Another option I've been toying with is running it off a NAS of some sort. You can force it to mount a filesystem using CIFS or NFS if you have a NAS that supports it. However, if it came to that point I would most likely offload it from the RaspPi to a small Intel NUC system I have set up as a Media computer and DVR thats also very low power. CPU is only 15watt TDP and it draws very low power on idle, and has a Samsung 840 Evo mSata SSD.

Likewise, I currently have a image file of my current Pi setup made ready to reflash to a new card. I also have the Pi's home directory mounted on a PC via SMB file share which I back up every day just in case as well.

Apparently the major problems with microSD / SD Cards on Pi's is unexpected shutdowns. The latest firmware versions apparently handle it a little better, and the latest Pi revisions as well such as the B+. I suggest updating to make sure you have the latest. http://www.raspberrypi.org/documentatio ... pdating.md

[tumutbound]

My webcamera is RPi based and it runs off a USB hard disk. No problems to date.
My Cumulus RPi has a USB flash drive where all data is stored, it too will get a hard disk so the only SD card usage will be during a boot or rpi-update.
One thing I should do to reduce I/O to the SD card is to have all my logging done over the network. I already do this on the camera.

They are a couple of jobs to consider when I've recovered my data and rebuilt all the records etc.
Are there any tools to assist with Cumulus data recovery?

[tumutbound]

Am I correct in assuming that Cumulus will recreate the record files i.e. alltime.ini, month.ini, monthlyalltime.ini year.ini, the first time it is run or at end of day processing?
Other than the edit menus in Cumulus 1, are there any available tools to help rebuild/recreate missing files or is it just a hard slog with an editor?

I have now put a proper backup scheme in place to save the files to another server - better late than never.

[steve]

The editors in Cumulus 1 can recreate all of the other files from the monthly log files, starting with dayfile.txt if necessary. It is a semi-automatic process, but you have to press the buttons yourself - sorry about that. There is currently no equivalent in Cumulus MX. It's still very much a beta, you should expect to lose data, even it wasn't the fault of the software in this case. Taking a daily backup is a good idea if you are concerned about data loss (as you are right to be).

I'm not aware of anyone having written any tools to aid with data recovery beyond the large amount of work that I have done in Cumulus, but I may be doing someone an injustice.

[mcrossley]

I'm not aware of anyone having written any tools to aid with data recovery beyond the large amount of work that I have done in Cumulus, but I may be doing someone an injustice.

Cumulus 1 does such a good job I don't think it has been necessary.

[steve]

I think so too, but you can't please everybody...