Welcome to the Cumulus Support forum.

Latest Cumulus MX release 3.9.6 (build 3101) - 20 January 2021 (please see announcement regarding releases since 3.5.0)
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of the NOAA report and Snow Index drop-down menus to 2030)

Use this Wiki link to Download the Software (Cumulus MX / Cumulus 1 and other related items).

Backing up and restoring CMX on the RPi

Talk about anything that doesn't fit elsewhere - PLEASE don't put Cumulus queries in here!
Post Reply
User avatar
HansR
Posts: 1681
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Backing up and restoring CMX on the RPi

Post by HansR »

(No doubt this is somewhat superfluous, but you never know)
I had a SD-card crash for the first time on my RPi recently so I had to recover. I draw some experience from this exercise which I would like to share.

For a situation like this I had been making backups: once a week a total system backup and a daily backup of the CumulusMX directory tree. The backup was saved to my NAS and I deleted the CMX backups older than a week and system backups older than a month. So I thought to restore the most recent system backup and then overwrite CMX with the most recent CMX backup. And - don't be surprised - the system backup failed to restore. Why? I don't know. A test restore some time ago worked.

So, what to do? I decided to install a complete new Raspbian with the new Raspberry Pi Imager.
Piece of cake. Create the SSH file on /boot (to have the headless boot) and create the (wpa_supplicant.conf for the WiFi). Personally I also make an entry for a fixed IP address as well. Then I ran my UpdateSystem script (I have some scripts which I also backup separately) to be up to date with everything and install:
  1. Mono complete (required)
  2. lshw (required for CumulusUtils)
  3. zip (required for backing up)
  4. reinstate the crontab things you want to do (including the backup)
Reboot.
Then I unzip my Cumulus Backup on the PC and copy it to the RPi, copy the systemd script to the systemd directory (adjust directory).
Done. Within 30 minutes I had manually restored my RPi system. I am done with the full system backup which takes almost an hour and which does not restore.
It is a very simple attitude towards crashes without complex backup systems: just reinstall the system

Conclusion: just backing up the CMX directory is enough.
The advantage is that once in a while you need to restore your system and it keeps your knowledge up to date reviewing everything :D

So here is the backup script I use which appeared very useful. If you don't have a NAS, adjust to wherever you want to backup.
It is ran through crontab, output in a logfile.

Code: Select all

#!/bin/bash -v
#
#######################################################################################################################
#
# Create and restore a backup of a Raspberry running raspbian
# Need to run as root
#

#Usage: pause "Press [return] to continue---"
function pause(){
   read -p "$*"
}

# Goto the /home directory and stop CumulusMX
cd /home
systemctl stop cumulusmx

DATE=$(date +%Y%m%d-%H%M%S)
HOSTNAME=$(hostname)
NL=$'\n'
CURRENT_DIR=$(pwd)
FILENAME="$DATE"'-CumulusMX.zip'
BCKPDIR="/mnt"

date
echo "Creating CumulusMX backup in "$FILENAME

if ( /sbin/showmount -e 192.*.*.* )
then
  echo "The NAS is online!"

    if ( mount -t nfs -o defaults -v 192.*.*.*:/volume1/Backups/Meteo $BCKPDIR  )
    then
      echo "Mount succesful"

      echo "Starting the zip/backup operation. Timing:"
      time zip -r -q $BCKPDIR/"$FILENAME" CumulusMX

      echo "Starting the find/delete operation. Timing:"
      time find $BCKPDIR/ -type f -name '*CumulusMX.zip' -mtime +7 -exec rm '{}' \;

      echo "Starting the unmounting operation. Timing:"
      time umount -v $BCKPDIR
      echo "Backup CumulusMX succesfully completed"
    fi
else
  echo "The NAS is not online, cumubackup failed, script halted!"
fi

systemctl start cumulusmx

echo "Backup CumulusMX is finished"
date
And OK,why not, also the UpdateSystem script (with commented out lines to use at your convenience)

Code: Select all

#!/bin/bash
#
#######################################################################################################################
#
# Update the raspbian system and installed apps
# Need to run as root
# 
# nov 2019: Upgrade for RAspbian buster
#

apt update
apt full-upgrade
apt autoremove
apt clean

#apt-get purge libmono* cli-common mono-runtime

#apt install apt-transport-https dirmngr gnupg ca-certificates
#apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
#echo "deb https://download.mono-project.com/repo/debian stable-raspbianbuster main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
#apt update
#apt-get update && sudo apt-get upgrade 
#apt-get install mono-complete
#apt autoremove
https://meteo-wagenborgen.nl / Cumulus 3.9.4 (build: 3099)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.79-v7+ armv7l
Mono JIT compiler version 5.18.0.240 (Debian 5.18.0.240+dfsg-3 Sat Apr 20 05:16:08 UTC 2019)

freddie
Posts: 1055
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux Ubuntu 20.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Backing up and restoring CMX on the RPi

Post by freddie »

I agree that start from scratch is the best way. I run Cumulus in a container that is easily replaceable. My approach for critical Cumulus files is to just copy files from /data /web and Cumulus.ini to a remote location, and just copy them into place when the container is recreated. I use rsync called from a simple cron script such that only changed files actually get copied. I copy the /data directory once per hour, and the /web and Cumulus.ini at less frequent intervals.
Freddie
Image

User avatar
mcrossley
Posts: 7580
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Buster Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Backing up and restoring CMX on the RPi

Post by mcrossley »

If you use the pre-built Cumulus MX image file, then all you would need to do (apart from the WiFi configuration if you use that) is restore the /data, /reports, /web folders, Cumulus.ini and off you go again. ;)

User avatar
HansR
Posts: 1681
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 10 (Buster)
Location: Wagenborgen (NL)
Contact:

Re: Backing up and restoring CMX on the RPi

Post by HansR »

@freddie: yes. Once per hour seems a very high frequency if there is buffering in the station itself.
@mcrossley:yes

For both: I seem to be adding other files and dirs as well (obviously for cutils but also other stuff). Others will have other (web)mods as well. To be sure you don't forget small things: just backup and restore the whole cmx directory.
https://meteo-wagenborgen.nl / Cumulus 3.9.4 (build: 3099)
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian Buster / 5.4.79-v7+ armv7l
Mono JIT compiler version 5.18.0.240 (Debian 5.18.0.240+dfsg-3 Sat Apr 20 05:16:08 UTC 2019)

Matt.j5b
Posts: 495
Joined: Mon 28 Nov 2011 2:13 am
Weather Station: Davis VP2 with DFARS
Operating System: RPi Raspbian (Buster)
Location: Ferny Grove, Brisbane, Australia
Contact:

Re: Backing up and restoring CMX on the RPi

Post by Matt.j5b »

I have a script in cron that runs a backup each day and creates a zipped file of the Cumulus MX directly. I do similar with other backups as well, including the databases as I also use the Pi as a local web server to test out changes to the online website.

What I found helpful if/when reinstalling is creating notes that describe the configurations I have made, installing packages and various processes/ scripts that I use the Pi for. This is much easier than trying to figuring out the changes I have made over time.
Regards, Matt of Brisbane, Australia
Ferny Grove Weather
Image

Post Reply