Welcome to the Cumulus Support forum.

Latest Cumulus MX release v3.0.0 build 3050 - 1 May 2019.
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014

See the Wiki for downloading either version.

Possible memory leak

Discussion of version 3 of Cumulus, which runs on Windows, Linux, and OS X. All Cumulus MX queries in here, please.

Moderator: mcrossley

philpugh
Posts: 132
Joined: Tue 24 May 2011 8:34 am
Weather Station: DAVIS VantageVue
Operating System: Debian Stretch Lite
Location: Antrobus, Cheshire
Contact:

Re: Possible memory leak

Post by philpugh » Sun 28 Apr 2019 10:40 am

Seem to have had problem loading the ftplog file - now reduced to <5MB !
You do not have the required permissions to view the files attached to this post.
Phil Pugh
Davis Vantage Vue Raspberry Pi Zero W Stretch-Lite
Using TCP/IP connection to Meteobridge
http://goosegate.plus.com/

freddie
Posts: 430
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2
Operating System: GNU/Linux (Ubuntu 18.04)
Location: Dorrington, Shropshire
Contact:

Re: Possible memory leak

Post by freddie » Sun 28 Apr 2019 12:25 pm

You appear to have intermittent network connectivity. This could be due to a problem with your network interface that is present on the machine hosting Cumulus MX - or it could equally be a problem on your LAN. I don't know anything about your LAN topology, so I can't comment any further.

I base this assessment on the error messages logged in your MXDiags file, and the variable logging of FTP sessions in your FTP log. In particular, messages such as:

Code: Select all

2019-04-27 13:37:28.011 TCP Logger Connect attempt 1
2019-04-27 13:37:29.242 TCP Logger reconnected
2019-04-27 13:37:29.247 Flushing input stream
2019-04-27 13:37:29.476 Connected OK
2019-04-27 13:37:29.488 Reading firmware version
2019-04-27 13:37:33.074 WakeVP Error: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respondafter a period of time, or established connection failed because connected host has failed to respond.
Looks as if you are making TCP connections to your weather station - and they are not always succeeding, or fail at some point during data transfer.

Code: Select all

2019-04-27 13:44:24.506 Test of realtime FTP connection failed: Timed out trying to read data from the socket stream!
2019-04-27 13:44:24.507 Realtime ftp not connected - reconnecting
Realtime connections had succeeded before this point, and succeeded after this point too.

Code: Select all

2019-04-27 13:44:38.419 Warning, previous realtime ftp still in progress, skipping this period.
Suggests slow network connectivity.

Code: Select all

2019-04-27 13:46:19.939 ACK received
2019-04-27 13:46:19.950 invalid LOOP packet
2019-04-27 13:46:22.952 Discarding bytes from pipeline: 3
Some sort of interruption to the packets received from your weather station?

Code: Select all

2019-04-27 13:51:03.342 Error uploading realtime.txt to /var/www/realtime/realtime.txt : Timed out trying to read data from the socket stream!
Not receiving response code from the remote server.

Code: Select all

2019-04-27 18:44:52.703 Error encountered during Realtime MySQL operation.
2019-04-27 18:44:52.704 Lost connection to MySQL server during query
Network problem during data transfer.

Code: Select all

2019-04-27 18:44:54.877 Error encountered during Realtime delete MySQL operation.
2019-04-27 18:44:54.877 Can't connect to MySQL server on '192.168.1.95' (10061):
No route to host
Network down at this point.

Code: Select all

2019-04-27 18:45:04.141 Error connecting ftp - No route to host
Network down at this point.

Code: Select all

2019-04-27 18:45:21.195 Windy update: Error: NameResolutionFailure
Network down at this point.

So it appears you have an intermittent problem - and a network-resource-heavy program like Cumulus MX will suffer.

Are there any wireless network connections on your LAN?
Are all your switches and routers performing as expected?
Is your Internet connection reliable? Does it have sufficient bandwidth?
Have you changed any of the network interface drivers on your Cumulus MX host?

As you can probably gather, diagnosing network issues is tricky. But you now have a few points to check.
Freddie
Image

philpugh
Posts: 132
Joined: Tue 24 May 2011 8:34 am
Weather Station: DAVIS VantageVue
Operating System: Debian Stretch Lite
Location: Antrobus, Cheshire
Contact:

Re: Possible memory leak

Post by philpugh » Sun 28 Apr 2019 1:00 pm

My LAN has both wired (most systems) and wireless (mostly portable devices) through two access points.

Originally CumulusMX was being run on my RPi 3B WWW server. Then I started to get problems with CMX 'hogging the system' - CPU loads in excess of 60% average! I first noticed this with V3.3043 of CMX I believe.

Having run a meteobridge with my Weatherflow system very successfully I thought I would move to using one of those for the Vantage Vue. The connection to CMX is intended to be for a short time while I re-engineer the web interface to use the meteobridge derived database. I have a number of other feeds into the database - solar power and electric usage....

Since removing CMX from the server the average server CPU load is closer to the 'usual' 2% that I had for many months before.

The LAN has two 8-port switches with the meteobridge and server on the same switch - the Pi Zero W is obviously connected via wifi.
I am just configuring a spare Pi3B to replace the ZeroW and will try that connected into the same switch. I suspect the ZeroW wifi isn't up to the job!

Will keep people informed.
Phil Pugh
Davis Vantage Vue Raspberry Pi Zero W Stretch-Lite
Using TCP/IP connection to Meteobridge
http://goosegate.plus.com/

philpugh
Posts: 132
Joined: Tue 24 May 2011 8:34 am
Weather Station: DAVIS VantageVue
Operating System: Debian Stretch Lite
Location: Antrobus, Cheshire
Contact:

Re: Possible memory leak

Post by philpugh » Wed 08 May 2019 4:15 pm

Just to close this down....

I have been running .3049 on the RPi 3B+ using TCP connection to Meteobridge ( version 3.9 bld 2057 - running on TL-MR3020V3) since 28/4/19 without any further issues.

It would appear that a Pi Zero W just doesn't have the power to cope with the demands of running this way !
Phil Pugh
Davis Vantage Vue Raspberry Pi Zero W Stretch-Lite
Using TCP/IP connection to Meteobridge
http://goosegate.plus.com/

User avatar
mcrossley
Posts: 5518
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Possible memory leak

Post by mcrossley » Wed 08 May 2019 5:25 pm

Thanks for letting me know Phil.
I use a Pi 3, that is overkill, but what the heck, it runs two web sites, Cumulus and my database on a small SSD with ease.

Post Reply