CUtils v 6.17.0 beta

[HansR]

In my haste I installed an old v6.7.10 yesterday, and sorry for the confusion that caused.

I tried v6.17.0 today in both a CMX release v3.23.1-b3221 and that worked ok and updated to the /cumulusmx website.
I did not try it on the other b3221 and will update that later /cumulusmxwll.

OK. Take your time

You asked if v3.24.0 tester with PHP could try v6.17.0 beta so I did that this morning as well. This CMX test install also had CUtils installed quite a while ago which I had not updated since perhaps summer of 2022. I copied in all the v6.17.0 unzipped files and did a website run. Seemed ok in the \utils folder but nothing uploaded to website. There are some errors in the utilslog file and that is attached. I haven't taken the time yet to double check the CUtils install, just wanted to correct the wrong version of yesterday.

Well, according to the log your still run CumulusUtils version: 6.7.10 so we'll leave that until the double checks.

I have done a time compare of the cumulusutils.exe website runs:
CMX v3.23.1-b3221 CUtils v6.17.0 started at 10:37:02 and finished 10:39:35 as recorded in the log file so about 2.5 minutes (my guess of 3 to 4 minutes yesterday was off a bit but then time seems to take forever when watching the PC time, and that is about the same time that I have observed in previous website runs after installing a new version) website is /cumulusmx/index.html
CMX v3.23.1-b3221 CUtils v6.16.0 started at 10:02:02 and finished 10:04:20 so about the same 2 plus minutes to run website /cumulusmxwll/index.html
CMX v3.24.0-b3228 CUtils v6.17.0 started at 10:44:02 and finished 10:44:24 so at about 22 seconds a lot quicker than with v3.21.1 but then this did not have any actual uploads.

Some more work to do on my part to get CUtils installed properly on the test install.

That timing issue is still something I do not really comprehend: afaiu you run all those instances on the same Windows machine under the same OS. So the time differences can not be in the order of 20 secs vs 2+ mins (I assume they have the same datasets?) but lets agree we take this as the final check when the installs and runs are OK and I can analyse the correct logs.

As I said above: take your time, speed is OK but quality is better

[HansR]

@Nossie: The important thing for me is now that an install does run correct in the situation you are used to (in your case local webserver with copy if I remember well). Any change you make with the coming CMX (i.e. the PHP upload) we'll take later. One step at a time

[HansR]

@Paul: having said all that, I suddenly realised that if you did not use the most recent version of CUtils, you will definitely have problems for the site because the libraries (notably gauges) will have changed. So you will need a full install which I add here in the first post. Wrt the executables there is no difference with the previous zip but this one is a normal release and contains the up to date libraries you need to upgrade correctly.

[PaulMy]

Hi Hans,
I am in the process of further testing.
My updating error was that I copied all the update files to the usual \utils folder and that placed these files in \cutils instead of \utils\bin
v6.17.0 beta 1 runs in CMX b3221 (there are many errors but these are data issues and have been there for a while that I hadn't noticed)
v6.17.0 beta 1 run starts in CMX with updates.php but continues without finishing. I've Ctrl C to stop it.

Code: Select all

2023-02-24 13:32:07.367 Testing PHP upload compression on http://komokaweather.com/cumulusmxwlltest/upload.php
2023-02-24 13:32:28.417 Error PhpInit: Error - An error occurred while sending the request.
2023-02-24 13:32:28.417 Error UploadFile: Nothing uploaded because of connection error.
2023-02-24 13:32:28.422 Testing PHP upload compression on http://komokaweather.com/cumulusmxwlltest/upload.php
2023-02-24 13:32:49.452 Error PhpInit: Error - An error occurred while sending the request.
2023-02-24 13:32:49.452 Error UploadFile: Nothing uploaded because of connection error.
2023-02-24 13:32:49.452 Error CheckPackageAndCopy: Upload of utils/index.html to index.html failed.

I've tried with several different path in cumulusutils.ini which initially had \public_htm\cumulusmxwlltest\ but haven't found anything that works yet.

Code: Select all

[FTP site]
FtpLog=Off
DoUploadFTP=true
UploadDir=cumulusmxwlltest

Enjoy,
Paul

[Nossie]

@hans

I'm aware that this will have no visible impact on me - but anyway beta 3 installed and everything seems ok

thanks.

[HansR]

Hi Paul,

OK... I'll start with the first log.

It says the following:

Code: Select all

2023-02-24 13:37:29.708 CumulusUtils version: 6.17.0 beta 1
2023-02-24 13:37:29.722 InetSupport: Constructor start => no complaints so everything seems OK
2023-02-24 13:37:29.991 CumulusMX Version: 3.23.1 build 3221 => b3221 so no PHP upload
 [...] => lots of data errors which are there because of old data (2008) => ignore
2023-02-24 13:37:49.593 Information Uploading = cumuluscharts.txt
2023-02-24 13:37:49.600 Testing PHP upload compression on http://komokaweather.com/cumulusmxwlltest/upload.php
2023-02-24 13:38:10.700 Error PhpInit: Error - An error occurred while sending the request.

This is basically impossible because CMX 3.23.1 b3221 cannot have protocol nr 3 in its inifile it should upload over some FTP.
I do not understand this.
I did check the URL for upload.php and that is correct but you cannot use this with b3221

Could I ask you to install beta 3 (from the complete install I just published)?
And could you confirm your CMX version and protocol?

------

The second log says:

Code: Select all

2023-02-24 13:20:17.162 CumulusUtils version: 6.17.0 beta 1
2023-02-24 13:20:17.174 InetSupport: Constructor start
2023-02-24 13:20:18.399 Information InetSupport: FTP Setup (After connect):
2023-02-24 13:20:18.399 InetSupport: Plain Old FTP activated.
2023-02-24 13:20:18.399 Information InetSupport: FTP Server: PureFTPd on Unix  => OK so now we're on FTP
2023-02-24 13:20:18.670 CumulusMX Version: 3.23.1 build 3221 => OK still 3221 but now with FTP it should be OK
[...]
And the uploads go OK but I took the path out of the log so I have no idea where they go - that is something I need to correct in the log

It seems to me that you try to switch between PHP and FTP uploads repetedly within this one install. That is awkward because the upload paths are very different (FTP path versus relative to the PHP procedure URL) and you probably will make errors.

The following steps need to be taken:

1. Test the PHP and the FTP protocol on two different CMX instances with corresponding CUtils installs, don't start switching within one instance. That is very error prone.
2. For the PHP test you just need the CMX 3.24.0 beta (b3228) and you need to be able to set the upload protocol to PHP. If you can't don't set things manually in the ini file (or switch ini files)
3. Use the lates beta published here in first post.

Let's see where this goes.

[PaulMy]

Hi Hans,
My first test with beta 3 is with b3221 using WiFiLogger, and which is FTP 0 - Just plain FTP which on my PC uses port :8998 and uploads to /cumulusmx
The website run started at 2023-02-24 16:12:59 and finished 16:15:22 and ends with ready...
The CUtils log is file 2302241612cumulusutils.log

Code: Select all

2023-02-24 16:13:01.203 SystemStatus : starting
2023-02-24 16:13:01.213 GetUrlData Start: URL - http://localhost:8998/api/tags/process.json?version&build&ProgramUpTime&NewBuildAvailable&NewBuildNumber&CpuCount&CPUTemp 
2023-02-24 16:13:01.227 Information GetUrlData Calling GetAsync
2023-02-24 16:13:01.362 CumulusMX Version: 3.23.1 build 3221

This works and is updating

The second test is with b3228 using WLL, and which is PHP upload which on the same PC uses port :8993 and uploads to /cumulusmxwlltest
The website run started at 23-02-24 16:21:36 and never ends so Ctrl C at about 16:29:15
The CUtils log is file 2302241621cumulusutils.log

Code: Select all

2023-02-24 16:21:36.791 SystemStatus : starting
2023-02-24 16:21:36.798 GetUrlData Start: URL - http://localhost:8999/api/tags/process.json?version&build&ProgramUpTime&NewBuildAvailable&NewBuildNumber&CpuCount&CPUTemp 
2023-02-24 16:21:36.798 Information GetUrlData Calling GetAsync
2023-02-24 16:21:36.922 CumulusMX Version: 3.23.1 build 3221
2023-02-24 16:21:36.923 Information  SystemStatus: Found device 11 Davis WLL

I wonder if I am going a bit haywire, but you will see the log says port :8999 instead of :8993 and version 3.21.1 build 3221 but that is not correct.
Attached is a screen capture of the run which is in C:\KomokaWeather\CumulusMXwlltest

I do run more than one CMX on the same PC. In addition to those 2, there is another b3221 using WLL and port :8999 (the port shown in the Utils log instead of :8993. This still has CUtils v6.16.0.0 and runs fine at every midnight rollover /cumulusmxwll. And also another test v.xxxx using WLL and port :8994 and I don't have CUtils on this as I believe not yet supported.

Enjoy,
Paul

[HansR]

Hi Paul,

Thanks.
I took a quick look before the end of day here, you info makes some things clear.

I think the version issue with CMX may be a problem with CUtils which asks the version to

Code: Select all

http://localhost:{CMXport}

The CMXport can be set in cumulusutils.ini, section SysInfo (default is 8998) so you may have to change that for a specific CUtils. That might solve the version issue.

So the FTP runs fine, let's leave that now and turn our attention to the PHP upload which apparently does not work. I assume it will not work either if the version reads correct (which I assume it will after you set the port right).

The important thing here is to check if the URL of the upload procedure is correct (can you post a printscreen of the Web/Upload screen where you set the protocol and the URL? Can you verify the key you have configured in CMX is the same as what you entered in the upload.php file?

Thanks, we will continue tomorrow.

[HansR]

@all: It is advised to switch off the windbarbs for the airlink (WantToSeeWind=false) as that will give problems.

[PaulMy]

Hi Hans,
I had forgotten about that port setting and it was still :8999 so changed it in cumulusutils.ini and a new website run now shows that correct. However still the PHP error and upload failure.

Unfortunately there are some other issues as just realized that the usual CMX PHP uploads stopped earlier today after working well for days.

Code: Select all

2023-02-24 20:12:39.948 PHP[6]: Error - An error occurred while sending the request.
2023-02-24 20:12:39.948 PHP[6]: Base exception - A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 23.229.161.0:80
2023-02-24 20:12:39.948 PHP[6]: Error - An error occurred while sending the request.
2023-02-24 20:12:39.948 PHP[6]: Base exception - A connection attempt failed because the connected party did not properly respond after a period of time, or established connection 

I have verified the 'secret' but still the same.

Also likely about the same time all my www komokaweather.com sites give me a

Code: Select all

Hmmm… can't reach this page
www.komokaweather.com took too long to respond

I've had this happen a few times in the past months so another support call to my ISP. I presume all my pages are visible to you.
In the meantime most testing is on hold.

Trying to enjoy,
Paul

[HansR]

Hi Paul,

Indeed I can reach all your sites. It would not surprise me if your provider detects the new PHP upload access and defines that as an issue so yes, a call would be appropriate, interesting what comes out of this.

I have currently two sites on my domain running on the upload for days and no problems so far.

I wait the results.

You know, this is very succesful beta testing isn't it: finding problems before release and finding causes ...

[HansR]

Beta 4 is available.

This solves the issue of the AirLink JSON getting too large (they now can be truly incremental).
This was an issue for all CUtils AirLink users, not only concerning the CMX beta users.

[AndyKF650]

Hi Hans

From my limited view as a current CMX build 3221 user the latest Cutils 6.17.0 beta 4 has all the usual functionality.

[HansR]

Beta 5 is available.

Further improvements to the Airlink Windbarbs.
If you are not a user of Airlink Windbarbs you may ignore this beta.

[PaulMy]

Hi Hans and thanks,

Indeed I can reach all your sites. It would not surprise me if your provider detects the new PHP upload access and defines that as an issue so yes, a call would be appropriate, interesting what comes out of this.

I've had long calls with both my webhost and ISP. I think all is good with webhost and v 6.17.0 runs fine on CMX b3221. My current issue with b3228 v6.17.0 I think is that CMX can communicate out to my webhost but not back in, which is causing upload.php to fail and giving the error

Code: Select all

2023-02-25 11:50:13.565 PHP[249]: Error - An error occurred while sending the request.
2023-02-25 11:50:13.565 PHP[249]: Base exception - A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 23.229.161.0:80
2023-02-25 11:50:13.567 Realtime[249]: End cycle

I presume this is related to my issue with ISP where FTP uploads work and website is updated and the world can see the webpage but I can't see anything at komokaweather.com from my home network. I can see it using my cell phone and turning WiFi off. This is the 4th time this has happened over the last few months and I had raised a ticket with ISP and within a few days the issue would go away but no information why it happened and how it started to work again. I have a new active ticket with ISP.

Enjoy,
Paul