Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Possible memory leak

From build 3044 the development baton passed to Mark Crossley. Mark has been responsible for all the Builds since. He has made the code available on GitHub. It is Mark's hope that others will join in this development, but at the very least he welcomes your ideas for future developments (see Cumulus MX Development suggestions).

Moderator: mcrossley

User avatar
philpugh
Posts: 381
Joined: Tue 24 May 2011 8:34 am
Weather Station: See Signature
Operating System: Debian 11 (RPi4)
Location: Antrobus, Cheshire, UK
Contact:

Re: Possible memory leak

Post by philpugh »

Seem to have had problem loading the ftplog file - now reduced to <5MB !
You do not have the required permissions to view the files attached to this post.
Phil Pugh
GW1000 + WH65/WH57/WH31; HP2551 + WS68/WH40A; GW2001 WittBoy
cumulusutils generating website
http://goosegate.uk/
freddie
Posts: 2434
Joined: Wed 08 Jun 2011 11:19 am
Weather Station: Davis Vantage Pro 2 + Ecowitt
Operating System: GNU/Linux Ubuntu 22.04 LXC
Location: Alcaston, Shropshire, UK
Contact:

Re: Possible memory leak

Post by freddie »

You appear to have intermittent network connectivity. This could be due to a problem with your network interface that is present on the machine hosting Cumulus MX - or it could equally be a problem on your LAN. I don't know anything about your LAN topology, so I can't comment any further.

I base this assessment on the error messages logged in your MXDiags file, and the variable logging of FTP sessions in your FTP log. In particular, messages such as:

Code: Select all

2019-04-27 13:37:28.011 TCP Logger Connect attempt 1
2019-04-27 13:37:29.242 TCP Logger reconnected
2019-04-27 13:37:29.247 Flushing input stream
2019-04-27 13:37:29.476 Connected OK
2019-04-27 13:37:29.488 Reading firmware version
2019-04-27 13:37:33.074 WakeVP Error: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respondafter a period of time, or established connection failed because connected host has failed to respond.
Looks as if you are making TCP connections to your weather station - and they are not always succeeding, or fail at some point during data transfer.

Code: Select all

2019-04-27 13:44:24.506 Test of realtime FTP connection failed: Timed out trying to read data from the socket stream!
2019-04-27 13:44:24.507 Realtime ftp not connected - reconnecting
Realtime connections had succeeded before this point, and succeeded after this point too.

Code: Select all

2019-04-27 13:44:38.419 Warning, previous realtime ftp still in progress, skipping this period.
Suggests slow network connectivity.

Code: Select all

2019-04-27 13:46:19.939 ACK received
2019-04-27 13:46:19.950 invalid LOOP packet
2019-04-27 13:46:22.952 Discarding bytes from pipeline: 3
Some sort of interruption to the packets received from your weather station?

Code: Select all

2019-04-27 13:51:03.342 Error uploading realtime.txt to /var/www/realtime/realtime.txt : Timed out trying to read data from the socket stream!
Not receiving response code from the remote server.

Code: Select all

2019-04-27 18:44:52.703 Error encountered during Realtime MySQL operation.
2019-04-27 18:44:52.704 Lost connection to MySQL server during query
Network problem during data transfer.

Code: Select all

2019-04-27 18:44:54.877 Error encountered during Realtime delete MySQL operation.
2019-04-27 18:44:54.877 Can't connect to MySQL server on '192.168.1.95' (10061):
No route to host
Network down at this point.

Code: Select all

2019-04-27 18:45:04.141 Error connecting ftp - No route to host
Network down at this point.

Code: Select all

2019-04-27 18:45:21.195 Windy update: Error: NameResolutionFailure
Network down at this point.

So it appears you have an intermittent problem - and a network-resource-heavy program like Cumulus MX will suffer.

Are there any wireless network connections on your LAN?
Are all your switches and routers performing as expected?
Is your Internet connection reliable? Does it have sufficient bandwidth?
Have you changed any of the network interface drivers on your Cumulus MX host?

As you can probably gather, diagnosing network issues is tricky. But you now have a few points to check.
Freddie
Image
User avatar
philpugh
Posts: 381
Joined: Tue 24 May 2011 8:34 am
Weather Station: See Signature
Operating System: Debian 11 (RPi4)
Location: Antrobus, Cheshire, UK
Contact:

Re: Possible memory leak

Post by philpugh »

My LAN has both wired (most systems) and wireless (mostly portable devices) through two access points.

Originally CumulusMX was being run on my RPi 3B WWW server. Then I started to get problems with CMX 'hogging the system' - CPU loads in excess of 60% average! I first noticed this with V3.3043 of CMX I believe.

Having run a meteobridge with my Weatherflow system very successfully I thought I would move to using one of those for the Vantage Vue. The connection to CMX is intended to be for a short time while I re-engineer the web interface to use the meteobridge derived database. I have a number of other feeds into the database - solar power and electric usage....

Since removing CMX from the server the average server CPU load is closer to the 'usual' 2% that I had for many months before.

The LAN has two 8-port switches with the meteobridge and server on the same switch - the Pi Zero W is obviously connected via wifi.
I am just configuring a spare Pi3B to replace the ZeroW and will try that connected into the same switch. I suspect the ZeroW wifi isn't up to the job!

Will keep people informed.
Phil Pugh
GW1000 + WH65/WH57/WH31; HP2551 + WS68/WH40A; GW2001 WittBoy
cumulusutils generating website
http://goosegate.uk/
User avatar
philpugh
Posts: 381
Joined: Tue 24 May 2011 8:34 am
Weather Station: See Signature
Operating System: Debian 11 (RPi4)
Location: Antrobus, Cheshire, UK
Contact:

Re: Possible memory leak

Post by philpugh »

Just to close this down....

I have been running .3049 on the RPi 3B+ using TCP connection to Meteobridge ( version 3.9 bld 2057 - running on TL-MR3020V3) since 28/4/19 without any further issues.

It would appear that a Pi Zero W just doesn't have the power to cope with the demands of running this way !
Phil Pugh
GW1000 + WH65/WH57/WH31; HP2551 + WS68/WH40A; GW2001 WittBoy
cumulusutils generating website
http://goosegate.uk/
User avatar
mcrossley
Posts: 12695
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Possible memory leak

Post by mcrossley »

Thanks for letting me know Phil.
I use a Pi 3, that is overkill, but what the heck, it runs two web sites, Cumulus and my database on a small SSD with ease.
Post Reply