Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4017) - 17 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Catching up does not work

From build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Catching up does not work

Post by HansR »

Hi,

I run my station for two months now without a problem (system description see signature). But today there was something wrong with my wireless. This was while making a backup so Cumulus was not running at that time and did not start until I had the opportunity to repair and restart. Cumulus had not been running for appr. 10 hrs. Normally, when using cumulus1, I was used to cumulus catching up when starting. However this did not happen with cumulus MX.

Anybody an explanation why that did not happen (losing 10 hrs of data)?
Did I not configure something? What is the point here?
Even if the period had been too long, I assume Cumulus has to catch up at least with the data still available. But nothing.

Best
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
freddie
Posts: 2433
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Catching up does not work

Post by freddie »

What is your update interval?
Freddie
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR »

Thanks for the reaction.

Actually, I do not have the original Davis software (weatherlink 5.8 in my case) installed anymore, so I do not really know. It fetches every few (5?) seconds from the station and cumulus uses a log interval of 1 minute. The Davis probably is in its default update interval.

But anyway, normally it catches up, this time it did not. So what can be the cause for this?

Best,
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
User avatar
PaulMy
Posts: 3775
Joined: Sun 28 Sep 2008 11:54 pm
Weather Station: Davis VP2 Plus 24-Hour FARS
Operating System: Windows8 and Windows10
Location: Komoka, ON Canada
Contact:

Re: Catching up does not work

Post by PaulMy »

Was there anything helpful in the MX startup DOS screen or in the MXdiags?
Do you upload to WU, etc. and are those set to Catch Up, and if so did that happen?
One reason that it may not have been able to download the data would be if you have the Davis logger interval and the Cumulus logging interval set to different values.
A search in the FAQ will explain how you can "rewind" from backup.

Enjoy,
Paul
Davis Vantage Pro2+
C1 www.komokaweather.com/komokaweather-ca
MX www.komokaweather.com/cumulusmx/index.htm /index.html /index.php
MX www.komokaweather.com/cumulusmxwll/index.htm /index.html /index.php
MX www. komokaweather.com/cumulusmx4/index.htm

Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR »

Mmm... I have my Raspberry attached through SSH, so no DOS screen. Rewinding from backup is a bit late now (it's 4 days later) but I would like to know what happened in case it occurs again. I do upload to WU etc... but they did not catch up either. The part of the MXdiags when starting Cumulus after the problem, before it starts its normal work, is shown below. The backup exists, made at startup time, but the files in it stop at 10:00 hrs when I shutdown cumulus.

Maybe the failing backup process (which uses the dd-comand) did something wrong?
Anyway, if you can make something of this MXdiags that would be nice. Otherwise I will consider it simply bad luck :(

Code: Select all

2019-08-06 21:49:07.934  ========================== Cumulus MX starting ==========================
2019-08-06 21:49:07.943 Command line: /home/CumulusMX/CumulusMX.exe -port 8998
2019-08-06 21:49:07.943 Cumulus MX v.3.0.0 build 3050
2019-08-06 21:49:07.946 Platform: Unix
2019-08-06 21:49:07.947 OS version: Unix 4.19.57.7
2019-08-06 21:49:07.948 Mono version: 4.6.2 (Debian 4.6.2.7+dfsg-1)
2019-08-06 21:49:07.948 Current culture: Dutch (Netherlands)
2019-08-06 21:49:07.948 Directory separator=[/] Decimal separator=[,] List separator=[;]
2019-08-06 21:49:07.949 Date separator=[-] Time separator=[:]
2019-08-06 21:49:07.949 Standard time zone name:   CET
2019-08-06 21:49:07.949 Daylight saving time name: CEST
2019-08-06 21:49:07.949 Daylight saving time? True
2019-08-06 21:49:07.949 6-8-2019 21:49:07
2019-08-06 21:49:07.950 Data path = data/
2019-08-06 21:49:08.075 Creating backup folder backup/20190806214908/
2019-08-06 21:49:08.170 Created backup folder backup/20190806214908/
2019-08-06 21:49:08.208 Reading Cumulus.ini file
2019-08-06 21:49:08.227 ASM=10 AST=00:10:00
2019-08-06 21:49:08.228 Cumulus start date: dinsdag 11 juni 2019
2019-08-06 21:49:08.240 Debug logging is disabled
2019-08-06 21:49:08.240 Data logging is disabled
2019-08-06 21:49:08.240 Logging interval = 1
2019-08-06 21:49:08.240 NoSensorCheck = 0
2019-08-06 21:49:08.532 Calculating sunrise and sunset times
2019-08-06 21:49:08.544 Sunrise: 05:56:36
2019-08-06 21:49:08.544 Sunset : 21:19:00
2019-08-06 21:49:08.545 Tomorrow sunrise: 05:58:15
2019-08-06 21:49:08.545 Tomorrow sunset : 21:17:04
2019-08-06 21:49:08.568 Station type: Davis Vantage Pro2
2019-08-06 21:49:08.569 WindUnit=km/h RainUnit=mm TempUnit=°C PressureUnit=hPa
2019-08-06 21:49:08.569 YTDRain=284,000 Year=2019
2019-08-06 21:49:08.569 RainDayThreshold=-1,000
2019-08-06 21:49:08.569 Offsets and Multipliers:
2019-08-06 21:49:08.569 PO=2,000 TO=1,000 HO=0 WDO=0 ITO=0,000 UVO=0,000
2019-08-06 21:49:08.570 WSM=1,000 WGM=1,000 TM=1,000 TM2=0,000 HM=1,000 HM2=0,000 RM=1,000 UVM=1,000
2019-08-06 21:49:08.570 Spike removal:
2019-08-06 21:49:08.570 TD=999,000 GD=999,000 WD=999,000 HD=999,000 PD=999,000
2019-08-06 21:49:08.570 MR=999,000 MH=999,000
2019-08-06 21:49:08.570 Cumulus Starting
2019-08-06 21:49:08.572 Opening station
2019-08-06 21:49:08.610 Last update time from today.ini: 6-8-2019 10:00:00
2019-08-06 21:49:08.610 Read today file: Date = 6-8-2019, LastUpdateTime = 6-8-2019 10:00:00, Month = 8
2019-08-06 21:49:08.612 Sensor clock  00:00:00
2019-08-06 21:49:08.612 Station clock 00:00:00
2019-08-06 21:49:08.613 ReadTodayfile: set notraininit false
2019-08-06 21:49:08.624 /home/CumulusMX/data/alltime.ini
2019-08-06 21:49:08.626 Alltime.ini file read
2019-08-06 21:49:08.662 MonthlyAlltime.ini file read
2019-08-06 21:49:08.668 Month.ini file read
2019-08-06 21:49:08.675 Year.ini file read
2019-08-06 21:49:08.678 Finding raintoday from logfile data/aug19log.txt
2019-08-06 21:49:08.679 Expecting listsep=; decimal=,
2019-08-06 21:49:09.811 Midnight rain found in the following entry:
2019-08-06 21:49:09.811 06-08-19;00:00;19,5;80;16,0;0,4;8,0;131;0,0;0,0;1009,59;194,6;24,5;58;0,0;19,5;19,5;0,0;0;0,10;203,96;21,4;0;0,0;131;0,0;0,0
2019-08-06 21:49:09.903 Setting raintoday from logfile = 0
2019-08-06 21:49:09.903 Calculating rain counter = 194,6
2019-08-06 21:49:09.903 Midnight rain found = 194,6
2019-08-06 21:49:09.906 Getting rain totals, rain season start = 1
2019-08-06 21:49:09.906 Today = 06/08/19
2019-08-06 21:49:09.925 Rainthismonth from dayfile.txt: 0,8
2019-08-06 21:49:09.925 Rainthisyear from dayfile.txt: 56,6
2019-08-06 21:49:09.925 Adding YTD rain: 284
2019-08-06 21:49:09.930 Station type = Davis
2019-08-06 21:49:09.930 LOOP2 disabled
2019-08-06 21:49:09.930 Serial device = /dev/ttyUSB0
2019-08-06 21:49:09.930 Serial speed = 19200
2019-08-06 21:49:09.942 Connected OK
2019-08-06 21:49:09.945 Reading firmware version
2019-08-06 21:49:11.759 FW version = 1.73
2019-08-06 21:49:11.760 Last update time = 6-8-2019 10:00:00
2019-08-06 21:49:11.766 Setting console time
2019-08-06 21:49:12.571 Console time set OK
2019-08-06 21:49:12.575 Reading console time
2019-08-06 21:49:14.177 Console clock: 6-8-2019 21:49:00
2019-08-06 21:49:14.181 Reading history data from log files
2019-08-06 21:49:14.182 Loading last N hour data from data logs: 6-8-2019 10:00:00
2019-08-06 21:49:15.613 Loaded 61 entries to last hour data list
2019-08-06 21:49:16.978 Loaded 181 entries to last 3 hour data list
2019-08-06 21:49:18.460 Loaded 2881 entries to graph data list
2019-08-06 21:49:28.988 Loaded 10080 entries to recent data list
2019-08-06 21:49:29.008 Loaded 56 entries to daily data list
2019-08-06 21:49:29.009 Reading archive data from logger
2019-08-06 21:49:29.046 Get Archive Data
2019-08-06 21:49:29.046 Rollover hour = 0
2019-08-06 21:49:29.047 Last Archive Date: 6-8-2019 10:00:00
2019-08-06 21:49:29.047 Date: 9990
2019-08-06 21:49:29.047 Time: 1000
2019-08-06 21:49:29.144 629 web tags initialised
2019-08-06 21:49:29.197 HTML root path = /home/CumulusMX/interface
2019-08-06 21:49:29.448 Sending DMPAFT
2019-08-06 21:49:29.648 Received response to DMPAFT, sending start date and time
2019-08-06 21:49:29.648 Sending: 06-27-E8-03-8D-B7
2019-08-06 21:49:29.849 No ACK in response to sending date and time
2019-08-06 21:49:29.849 Get Archive Data
2019-08-06 21:49:29.849 Rollover hour = 0
2019-08-06 21:49:29.849 Last Archive Date: 6-8-2019 10:00:00
2019-08-06 21:49:29.849 Date: 9990
2019-08-06 21:49:29.849 Time: 1000
2019-08-06 21:49:30.450 Sending DMPAFT
2019-08-06 21:49:30.651 Received response to DMPAFT, sending start date and time
2019-08-06 21:49:30.651 Sending: 06-27-E8-03-8D-B7
2019-08-06 21:49:30.852 No ACK in response to sending date and time
2019-08-06 21:49:30.858 Logger archive reading thread completed
2019-08-06 21:49:30.858 Updating highs and lows
2019-08-06 21:49:30.864 Start normal reading loop
2019-08-06 21:49:30.908 Start Timers
2019-08-06 21:49:30.908 Starting 1-minute timer
2019-08-06 21:49:30.923 Attempting realtime FTP connect
2019-08-06 21:49:31.363 Starting Realtime timer, interval = 30 seconds
2019-08-06 21:49:31.364 Normal running
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
freddie
Posts: 2433
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Catching up does not work

Post by freddie »

HansR wrote: Sat 10 Aug 2019 12:32 pmnormally it catches up, this time it did not. So what can be the cause for this?
I have the same problem, and I have a very short update interval as well. I think it's a problem where the response from the VP when you request a large number of records tends to time out. Mark Crossley, who is currently maintaining Cumulus, kindly included a longer timeout in an alpha version of the program and gave me the alpha version to try. It has solved the problem for me.
Freddie
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR »

OK, thnx. And which build is that, as it's alpha, has it been released yet?
(see my signature for the version I use)
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
freddie
Posts: 2433
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Catching up does not work

Post by freddie »

It's 3051 and hasn't been released. I will PM Mark.
Freddie
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR »

OK. Thnx, I'll hear what will happen.

Best,
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR »

One additional remark.

I think you are right about the cause of the problem. Today I backed-up my Raspberry which took a bit more than 2 hrs. That did not appear to be a problem when catching up so it must be the data size which let it break down.

But I am not sure changing (enlarging) the timeout is the good solution since the timeout is there for another problem (no connection). I would suggest to start reading smaller blocks of data (e.g. per hr) instead of trying to read the whole missed period in one go.

Maybe Mark can shed a light on this?

Best,
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
freddie
Posts: 2433
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Catching up does not work

Post by freddie »

HansR wrote: Tue 13 Aug 2019 10:33 amBut I am not sure changing (enlarging) the timeout is the good solution since the timeout is there for another problem (no connection). I would suggest to start reading smaller blocks of data (e.g. per hr) instead of trying to read the whole missed period in one go.
The timeout only refers to the interval between Cumulus requesting the data and the VP acknowledging that request - not the interval required to download the data. We are talking milliseconds here, so not noticeable. Maybe the VP does something behind the scenes (counting the records?) before responding to the request. I don't know. What I do know, though, is that the increase in timeout has resulted in the acknowledgement of the request not timing out when there are a large number of records to return, so looks to be a good change to me.
Freddie
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR »

Mmm... OK. We'll see.
Wait for acknowledgement by Mark then.

OK, 6 hrs later I am reviewing the problem again and I have forgotten to say that the console firmware version: 1.73. That means it can not use Loop2.
Not sure this is relevant.

Best,
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
User avatar
mcrossley
Posts: 12686
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Catching up does not work

Post by mcrossley »

Hans - PM awaiting you re the test code...
water01
Posts: 3215
Joined: Sat 13 Aug 2011 9:33 am
Weather Station: Ecowitt HP2551
Operating System: Windows 10 64bit
Location: Burnham-on-Sea
Contact:

Re: Catching up does not work

Post by water01 »

Rewinding from backup is a bit late now (it's 4 days later)
Cumulus backups happen at midnight every night, so you should be able to go back 1 or 2 days, so I do not understand this comment unless you are talking about your backups and not the ones done by Cumulus. It is these backups that the Wiki refers to when using the rewind that Paul pointed you to.
David
Image
User avatar
HansR
Posts: 5870
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: GW1100 (WS80/WH40)
Operating System: Raspberry OS/Bullseye
Location: Wagenborgen (NL)
Contact:

Re: Catching up does not work

Post by HansR »

Yes, i am talking about my own external backup. Once a week i backup the complete machine to my NAS. That's where a problem occurred. So the data were not in the cumulus backup either. They were simply not read from the davis.

Best,
Hans

https://meteo-wagenborgen.nl
CMX build 4017+ ● RPi 3B+ ● Raspbian Linux 6.1.21-v7+ armv7l ● dotnet 8.0.3
Post Reply