Welcome to the Cumulus Support forum.

Latest Cumulus MX release v3.0.2 build 3052 - 13 September 2019.
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014

See the Wiki for downloading either version.

Catching up does not work

Discussion of version 3 of Cumulus, which runs on Windows, Linux, and OS X. All Cumulus MX queries in here, please.

Moderator: mcrossley

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Catching up does not work

Post by HansR » Tue 06 Aug 2019 8:10 pm

Hi,

I run my station for two months now without a problem (system description see signature). But today there was something wrong with my wireless. This was while making a backup so Cumulus was not running at that time and did not start until I had the opportunity to repair and restart. Cumulus had not been running for appr. 10 hrs. Normally, when using cumulus1, I was used to cumulus catching up when starting. However this did not happen with cumulus MX.

Anybody an explanation why that did not happen (losing 10 hrs of data)?
Did I not configure something? What is the point here?
Even if the period had been too long, I assume Cumulus has to catch up at least with the data still available. But nothing.

Best
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

freddie
Posts: 494
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux (Ubuntu 18.04)
Location: Dorrington, Shropshire
Contact:

Re: Catching up does not work

Post by freddie » Tue 06 Aug 2019 9:10 pm

What is your update interval?
Freddie
Image

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR » Sat 10 Aug 2019 12:32 pm

Thanks for the reaction.

Actually, I do not have the original Davis software (weatherlink 5.8 in my case) installed anymore, so I do not really know. It fetches every few (5?) seconds from the station and cumulus uses a log interval of 1 minute. The Davis probably is in its default update interval.

But anyway, normally it catches up, this time it did not. So what can be the cause for this?

Best,
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

User avatar
PaulMy
Posts: 1956
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis Vantage Pro 2+ w/Envoy USB
Operating System: Windows-8
Location: Komoka, ON Canada
Contact:

Re: Catching up does not work

Post by PaulMy » Sat 10 Aug 2019 1:36 pm

Was there anything helpful in the MX startup DOS screen or in the MXdiags?
Do you upload to WU, etc. and are those set to Catch Up, and if so did that happen?
One reason that it may not have been able to download the data would be if you have the Davis logger interval and the Cumulus logging interval set to different values.
A search in the FAQ will explain how you can "rewind" from backup.

Enjoy,
Paul

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR » Sat 10 Aug 2019 2:34 pm

Mmm... I have my Raspberry attached through SSH, so no DOS screen. Rewinding from backup is a bit late now (it's 4 days later) but I would like to know what happened in case it occurs again. I do upload to WU etc... but they did not catch up either. The part of the MXdiags when starting Cumulus after the problem, before it starts its normal work, is shown below. The backup exists, made at startup time, but the files in it stop at 10:00 hrs when I shutdown cumulus.

Maybe the failing backup process (which uses the dd-comand) did something wrong?
Anyway, if you can make something of this MXdiags that would be nice. Otherwise I will consider it simply bad luck :(

Code: Select all

2019-08-06 21:49:07.934  ========================== Cumulus MX starting ==========================
2019-08-06 21:49:07.943 Command line: /home/CumulusMX/CumulusMX.exe -port 8998
2019-08-06 21:49:07.943 Cumulus MX v.3.0.0 build 3050
2019-08-06 21:49:07.946 Platform: Unix
2019-08-06 21:49:07.947 OS version: Unix 4.19.57.7
2019-08-06 21:49:07.948 Mono version: 4.6.2 (Debian 4.6.2.7+dfsg-1)
2019-08-06 21:49:07.948 Current culture: Dutch (Netherlands)
2019-08-06 21:49:07.948 Directory separator=[/] Decimal separator=[,] List separator=[;]
2019-08-06 21:49:07.949 Date separator=[-] Time separator=[:]
2019-08-06 21:49:07.949 Standard time zone name:   CET
2019-08-06 21:49:07.949 Daylight saving time name: CEST
2019-08-06 21:49:07.949 Daylight saving time? True
2019-08-06 21:49:07.949 6-8-2019 21:49:07
2019-08-06 21:49:07.950 Data path = data/
2019-08-06 21:49:08.075 Creating backup folder backup/20190806214908/
2019-08-06 21:49:08.170 Created backup folder backup/20190806214908/
2019-08-06 21:49:08.208 Reading Cumulus.ini file
2019-08-06 21:49:08.227 ASM=10 AST=00:10:00
2019-08-06 21:49:08.228 Cumulus start date: dinsdag 11 juni 2019
2019-08-06 21:49:08.240 Debug logging is disabled
2019-08-06 21:49:08.240 Data logging is disabled
2019-08-06 21:49:08.240 Logging interval = 1
2019-08-06 21:49:08.240 NoSensorCheck = 0
2019-08-06 21:49:08.532 Calculating sunrise and sunset times
2019-08-06 21:49:08.544 Sunrise: 05:56:36
2019-08-06 21:49:08.544 Sunset : 21:19:00
2019-08-06 21:49:08.545 Tomorrow sunrise: 05:58:15
2019-08-06 21:49:08.545 Tomorrow sunset : 21:17:04
2019-08-06 21:49:08.568 Station type: Davis Vantage Pro2
2019-08-06 21:49:08.569 WindUnit=km/h RainUnit=mm TempUnit=°C PressureUnit=hPa
2019-08-06 21:49:08.569 YTDRain=284,000 Year=2019
2019-08-06 21:49:08.569 RainDayThreshold=-1,000
2019-08-06 21:49:08.569 Offsets and Multipliers:
2019-08-06 21:49:08.569 PO=2,000 TO=1,000 HO=0 WDO=0 ITO=0,000 UVO=0,000
2019-08-06 21:49:08.570 WSM=1,000 WGM=1,000 TM=1,000 TM2=0,000 HM=1,000 HM2=0,000 RM=1,000 UVM=1,000
2019-08-06 21:49:08.570 Spike removal:
2019-08-06 21:49:08.570 TD=999,000 GD=999,000 WD=999,000 HD=999,000 PD=999,000
2019-08-06 21:49:08.570 MR=999,000 MH=999,000
2019-08-06 21:49:08.570 Cumulus Starting
2019-08-06 21:49:08.572 Opening station
2019-08-06 21:49:08.610 Last update time from today.ini: 6-8-2019 10:00:00
2019-08-06 21:49:08.610 Read today file: Date = 6-8-2019, LastUpdateTime = 6-8-2019 10:00:00, Month = 8
2019-08-06 21:49:08.612 Sensor clock  00:00:00
2019-08-06 21:49:08.612 Station clock 00:00:00
2019-08-06 21:49:08.613 ReadTodayfile: set notraininit false
2019-08-06 21:49:08.624 /home/CumulusMX/data/alltime.ini
2019-08-06 21:49:08.626 Alltime.ini file read
2019-08-06 21:49:08.662 MonthlyAlltime.ini file read
2019-08-06 21:49:08.668 Month.ini file read
2019-08-06 21:49:08.675 Year.ini file read
2019-08-06 21:49:08.678 Finding raintoday from logfile data/aug19log.txt
2019-08-06 21:49:08.679 Expecting listsep=; decimal=,
2019-08-06 21:49:09.811 Midnight rain found in the following entry:
2019-08-06 21:49:09.811 06-08-19;00:00;19,5;80;16,0;0,4;8,0;131;0,0;0,0;1009,59;194,6;24,5;58;0,0;19,5;19,5;0,0;0;0,10;203,96;21,4;0;0,0;131;0,0;0,0
2019-08-06 21:49:09.903 Setting raintoday from logfile = 0
2019-08-06 21:49:09.903 Calculating rain counter = 194,6
2019-08-06 21:49:09.903 Midnight rain found = 194,6
2019-08-06 21:49:09.906 Getting rain totals, rain season start = 1
2019-08-06 21:49:09.906 Today = 06/08/19
2019-08-06 21:49:09.925 Rainthismonth from dayfile.txt: 0,8
2019-08-06 21:49:09.925 Rainthisyear from dayfile.txt: 56,6
2019-08-06 21:49:09.925 Adding YTD rain: 284
2019-08-06 21:49:09.930 Station type = Davis
2019-08-06 21:49:09.930 LOOP2 disabled
2019-08-06 21:49:09.930 Serial device = /dev/ttyUSB0
2019-08-06 21:49:09.930 Serial speed = 19200
2019-08-06 21:49:09.942 Connected OK
2019-08-06 21:49:09.945 Reading firmware version
2019-08-06 21:49:11.759 FW version = 1.73
2019-08-06 21:49:11.760 Last update time = 6-8-2019 10:00:00
2019-08-06 21:49:11.766 Setting console time
2019-08-06 21:49:12.571 Console time set OK
2019-08-06 21:49:12.575 Reading console time
2019-08-06 21:49:14.177 Console clock: 6-8-2019 21:49:00
2019-08-06 21:49:14.181 Reading history data from log files
2019-08-06 21:49:14.182 Loading last N hour data from data logs: 6-8-2019 10:00:00
2019-08-06 21:49:15.613 Loaded 61 entries to last hour data list
2019-08-06 21:49:16.978 Loaded 181 entries to last 3 hour data list
2019-08-06 21:49:18.460 Loaded 2881 entries to graph data list
2019-08-06 21:49:28.988 Loaded 10080 entries to recent data list
2019-08-06 21:49:29.008 Loaded 56 entries to daily data list
2019-08-06 21:49:29.009 Reading archive data from logger
2019-08-06 21:49:29.046 Get Archive Data
2019-08-06 21:49:29.046 Rollover hour = 0
2019-08-06 21:49:29.047 Last Archive Date: 6-8-2019 10:00:00
2019-08-06 21:49:29.047 Date: 9990
2019-08-06 21:49:29.047 Time: 1000
2019-08-06 21:49:29.144 629 web tags initialised
2019-08-06 21:49:29.197 HTML root path = /home/CumulusMX/interface
2019-08-06 21:49:29.448 Sending DMPAFT
2019-08-06 21:49:29.648 Received response to DMPAFT, sending start date and time
2019-08-06 21:49:29.648 Sending: 06-27-E8-03-8D-B7
2019-08-06 21:49:29.849 No ACK in response to sending date and time
2019-08-06 21:49:29.849 Get Archive Data
2019-08-06 21:49:29.849 Rollover hour = 0
2019-08-06 21:49:29.849 Last Archive Date: 6-8-2019 10:00:00
2019-08-06 21:49:29.849 Date: 9990
2019-08-06 21:49:29.849 Time: 1000
2019-08-06 21:49:30.450 Sending DMPAFT
2019-08-06 21:49:30.651 Received response to DMPAFT, sending start date and time
2019-08-06 21:49:30.651 Sending: 06-27-E8-03-8D-B7
2019-08-06 21:49:30.852 No ACK in response to sending date and time
2019-08-06 21:49:30.858 Logger archive reading thread completed
2019-08-06 21:49:30.858 Updating highs and lows
2019-08-06 21:49:30.864 Start normal reading loop
2019-08-06 21:49:30.908 Start Timers
2019-08-06 21:49:30.908 Starting 1-minute timer
2019-08-06 21:49:30.923 Attempting realtime FTP connect
2019-08-06 21:49:31.363 Starting Realtime timer, interval = 30 seconds
2019-08-06 21:49:31.364 Normal running
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

freddie
Posts: 494
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux (Ubuntu 18.04)
Location: Dorrington, Shropshire
Contact:

Re: Catching up does not work

Post by freddie » Sat 10 Aug 2019 3:00 pm

HansR wrote:
Sat 10 Aug 2019 12:32 pm
normally it catches up, this time it did not. So what can be the cause for this?
I have the same problem, and I have a very short update interval as well. I think it's a problem where the response from the VP when you request a large number of records tends to time out. Mark Crossley, who is currently maintaining Cumulus, kindly included a longer timeout in an alpha version of the program and gave me the alpha version to try. It has solved the problem for me.
Freddie
Image

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR » Sat 10 Aug 2019 6:32 pm

OK, thnx. And which build is that, as it's alpha, has it been released yet?
(see my signature for the version I use)
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

freddie
Posts: 494
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux (Ubuntu 18.04)
Location: Dorrington, Shropshire
Contact:

Re: Catching up does not work

Post by freddie » Sat 10 Aug 2019 6:52 pm

It's 3051 and hasn't been released. I will PM Mark.
Freddie
Image

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR » Sun 11 Aug 2019 9:40 am

OK. Thnx, I'll hear what will happen.

Best,
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR » Tue 13 Aug 2019 10:33 am

One additional remark.

I think you are right about the cause of the problem. Today I backed-up my Raspberry which took a bit more than 2 hrs. That did not appear to be a problem when catching up so it must be the data size which let it break down.

But I am not sure changing (enlarging) the timeout is the good solution since the timeout is there for another problem (no connection). I would suggest to start reading smaller blocks of data (e.g. per hr) instead of trying to read the whole missed period in one go.

Maybe Mark can shed a light on this?

Best,
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

freddie
Posts: 494
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux (Ubuntu 18.04)
Location: Dorrington, Shropshire
Contact:

Re: Catching up does not work

Post by freddie » Tue 13 Aug 2019 12:11 pm

HansR wrote:
Tue 13 Aug 2019 10:33 am
But I am not sure changing (enlarging) the timeout is the good solution since the timeout is there for another problem (no connection). I would suggest to start reading smaller blocks of data (e.g. per hr) instead of trying to read the whole missed period in one go.
The timeout only refers to the interval between Cumulus requesting the data and the VP acknowledging that request - not the interval required to download the data. We are talking milliseconds here, so not noticeable. Maybe the VP does something behind the scenes (counting the records?) before responding to the request. I don't know. What I do know, though, is that the increase in timeout has resulted in the acknowledgement of the request not timing out when there are a large number of records to return, so looks to be a good change to me.
Freddie
Image

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR » Wed 14 Aug 2019 1:11 pm

Mmm... OK. We'll see.
Wait for acknowledgement by Mark then.

OK, 6 hrs later I am reviewing the problem again and I have forgotten to say that the console firmware version: 1.73. That means it can not use Loop2.
Not sure this is relevant.

Best,
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

User avatar
mcrossley
Posts: 5677
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Catching up does not work

Post by mcrossley » Tue 20 Aug 2019 8:15 pm

Hans - PM awaiting you re the test code...

water01
Posts: 1551
Joined: Sat 13 Aug 2011 9:33 am
Weather Station: WH1081
Operating System: Windows 7 64bit HP SP1
Location: Somewhere in the USA
Contact:

Re: Catching up does not work

Post by water01 » Tue 20 Aug 2019 8:34 pm

Rewinding from backup is a bit late now (it's 4 days later)
Cumulus backups happen at midnight every night, so you should be able to go back 1 or 2 days, so I do not understand this comment unless you are talking about your backups and not the ones done by Cumulus. It is these backups that the Wiki refers to when using the rewind that Paul pointed you to.

User avatar
HansR
Posts: 265
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR » Wed 21 Aug 2019 5:09 pm

Yes, i am talking about my own external backup. Once a week i backup the complete machine to my NAS. That's where a problem occurred. So the data were not in the cumulus backup either. They were simply not read from the davis.

Best,
https://meteo-wagenborgen.nl / Cumulus MX 3.0.2 b3052
Davis Vantage Pro 2+ / Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)

Post Reply