Cumulus Support

Support forum for Cumulus weather station software
https://cumulus.hosiene.co.uk/

CMX crash and realtime retain failure

https://cumulus.hosiene.co.uk/viewtopic.php?t=19246

Page 3 of 4

Re: CMX crash and realtime retain failure

Posted: Thu 08 Apr 2021 5:26 pm
by HansR
No. Intermittently it does work (ftps) most of the time it doesn't. Plain ftp works. Sftp says open ssh not permitted. I put it aside for the moment. There are more interesting things in the world :(

Re: CMX crash and realtime retain failure

Posted: Thu 08 Apr 2021 9:23 pm
by HansR
And before I forget: of course WinSCP works flawlessly on all FTP techniques possible.

Re: CMX crash and realtime retain failure

Posted: Thu 08 Apr 2021 10:06 pm
by ConligWX
HansR wrote: ↑Thu 08 Apr 2021 5:26 pm There are more interesting things in the world :(

Indeed. I want to know which covid vaccines will turn us all into mutants :lol:

Re: CMX crash and realtime retain failure

Posted: Fri 09 Apr 2021 2:22 pm
by mcrossley
HansR wrote: ↑Thu 08 Apr 2021 5:26 pm No. Intermittently it does work (ftps) most of the time it doesn't. Plain ftp works. Sftp says open ssh not permitted.
The later versions of the SSH component do support many more key file types, but then we hit some other bugs, and there is no support for running under Mono. :(

Re: CMX crash and realtime retain failure

Posted: Fri 09 Apr 2021 2:50 pm
by HansR
Ok, so what is the instruction for keygeneration fot the current version used?
I can't find any instruction in the wiki/forum. (Keywords not discriminating enough)

Re: CMX crash and realtime retain failure

Posted: Fri 09 Apr 2021 3:37 pm
by mcrossley
I cannot find the steps I did, but I'm pretty sure I just used OpenSSL on my NAS box to generate an RSA 2048 bit key pair. Extract and copy the public key into the server authorized_keys file, then added the local path to the private key into CMX.

Re: CMX crash and realtime retain failure

Posted: Fri 09 Apr 2021 5:58 pm
by HansR
I did similar using ssh-keygen with default. Stored the private key directly in the CMX dir and the public in the server part. So far so good. I can access with MobaXterm or other client using a keypair (I use another, similar created keypair) . While running CMX it does not accept openssh. Could be the server part.

To finish off my contribution in this field, attached is the log for my SSH run (private key local in CMX dir, public key on the server according to their specs).

I'll ask the hoster on monday if they intend it only to be for terminals ... :?

Re: CMX crash and realtime retain failure

Posted: Fri 09 Apr 2021 7:35 pm
by mcrossley
I think that error is saying it does not like the the OpenSSH format private key, it needs to be converted to RSA format...

I found this command (untested)...

Code: Select all

ssh-keygen -p -N "" -m pem -f /path/to/key
That sets a blank password - "" - and converts the file "in place" so will overwrite your existing file. Make a copy first!

Re: CMX crash and realtime retain failure

Posted: Fri 09 Apr 2021 9:05 pm
by HansR
Yes, that helps. I get connected now. Thnx.
But it's not all right. There are apparently still some paths to set which seems strange to me because it looks correct for realtime.txt. After the first error it disposes the SFTP object and it's back to square one. Tomorrow the rest.
    Schermafbeelding 2021-04-09 225436.jpg

    Re: CMX crash and realtime retain failure

    Posted: Sat 10 Apr 2021 7:47 am
    by HansR
    I looked into the failing SFTP uploads and I think it may have to do with ownership of the whole directory structure of my website.

    The owner is not me (my userid or whatever) but the owner is ftp and therefore(?) the SFTP process cannot access the files and issues the error message. I am not sure, but I have no other explanation. I can't use SFTP from WinSCP either because I cannot give a key, only un/pw.

    I will have to contact the provider.

    Re: CMX crash and realtime retain failure

    Posted: Sat 10 Apr 2021 5:55 pm
    by mcrossley
    HansR wrote: ↑Sat 10 Apr 2021 7:47 am I can't use SFTP from WinSCP either because I cannot give a key, only un/pw.
    ?? You set the key file under the connection Advanced Site Settings | SSH | Authentication. It also has tools for generating a key pair (it uses PuTTY KeyGen) and to upload your public key to the server.

    Re: CMX crash and realtime retain failure

    Posted: Sun 11 Apr 2021 9:54 am
    by HansR
    OK. I did expect the key entry when modifying the protocol to SSH on the same level as un/pw. Not so, so thanks.
    In the process I also found that the path required two more levels to the root of the server machine. Those had not been visible before without WinSCP on SSH.
    So, everything is working fine now apart from the FTPS protocol.
    I also have to renew the key every year (provider demand), so I prefer FTPS assuming it's working: much easier.

    Re: CMX crash and realtime retain failure

    Posted: Sun 11 Apr 2021 11:03 am
    by HansR
    OK, now that I have SFTP working I went back to FTPS and looked at that one.

    I noticed that when setting the protocol for my hoster I have to use port 21 for FTPS (yes I know, it is not common, but still it is true). When I set 21 and save, I go elsewhere to set the path in the extra web files and then looking at the log tail it does connect but it fails again for the datafiles because it tries to do the upload through port 990 (the default. Looking at the settings then indeed the default 990 has been reset iso the 21. Setting it again to 21 it works.

    Apart from that, it still does not upload, looking further... I'll maybe get back on this.

    There is another issue that's open: the MySQL realtime (see first logfile I posted here). As I don't do anything with the SQL itself it seems to be an error in the realtime retainment statement. Is that correct? (I switched the realtime SQL off to avoid disturbance).

    Re: CMX crash and realtime retain failure

    Posted: Sun 11 Apr 2021 2:43 pm
    by mcrossley
    Hi Hans, I'd need to see a log with debug switched on to look at the real time SQL retain issue. Unfortunately the current code does not dump the whole SQL statement on error.

    Re: CMX crash and realtime retain failure

    Posted: Sun 11 Apr 2021 2:53 pm
    by HansR
    This is the fragment between the setting of the debug and the end of cycle.
    If you need more let me know.

    Code: Select all

    2021-04-11 16:50:33.245 Updating program settings
2021-04-11 16:50:33.308 Writing Cumulus.ini file
2021-04-11 16:50:33.339 Completed writing Cumulus.ini file
2021-04-11 16:50:34.095 LOOP: Data - 8: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-09-0A-4E-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-DA-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-6C-FB
2021-04-11 16:50:34.095 LOOP: 8 - Data packet is good
2021-04-11 16:50:36.093 LOOP: Data - 9: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0B-0A-4E-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-DA-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-A1-20
2021-04-11 16:50:36.094 LOOP: 9 - Data packet is good
2021-04-11 16:50:38.099 LOOP: Data - 10: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0B-0A-51-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-DA-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-B5-D8
2021-04-11 16:50:38.100 LOOP: 10 - Data packet is good
2021-04-11 16:50:40.077 LOOP: Data - 11: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0B-0A-51-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-DA-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-B5-D8
2021-04-11 16:50:40.077 LOOP: 11 - Data packet is good
2021-04-11 16:50:42.095 LOOP: Data - 12: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0B-0A-51-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-DA-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-B5-D8
2021-04-11 16:50:42.095 LOOP: 12 - Data packet is good
2021-04-11 16:50:44.092 LOOP: Data - 13: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0B-0A-52-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-DA-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-01-D7
2021-04-11 16:50:44.093 LOOP: 13 - Data packet is good
2021-04-11 16:50:46.100 LOOP: Data - 14: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0A-0A-53-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-D5-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-D3-C3
2021-04-11 16:50:46.100 LOOP: 14 - Data packet is good
2021-04-11 16:50:48.350 LOOP: Data - 15: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0A-0A-52-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-D5-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-4F-D9
2021-04-11 16:50:48.351 LOOP: 15 - Data packet is good
2021-04-11 16:50:50.096 LOOP: Data - 16: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0A-0A-52-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-D5-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-4F-D9
2021-04-11 16:50:50.096 LOOP: 16 - Data packet is good
2021-04-11 16:50:52.104 LOOP: Data - 17: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-0A-0A-48-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-D5-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-97-11
2021-04-11 16:50:52.105 LOOP: 17 - Data packet is good
2021-04-11 16:50:52.635 GetAlCurrent: Outdoor - Waiting for lock
2021-04-11 16:50:52.636 GetAlCurrent: Outdoor - Has the lock
2021-04-11 16:50:52.636 GetAlCurrent: Outdoor - Sending GET current conditions request 1 to AL: http://192.168.178.5/v1/current_conditions ...
2021-04-11 16:50:52.705 GetAlCurrent: Response - {
  "data":{
    "did":"000000000000",
    "name":"FakeAirLink",
    "ts":1618152652,
    "conditions":[ {
        "lsid":000000,
        "data_structure_type":6,
        "temp":61.0,
        "hum":56.1,
        "dew_point":-1,
        "wet_bulb":-1,
        "heat_index":-1,
        "pm_1_last":-1,
        "pm_2p5_last":-1,
        "pm_10_last":-1,
        "pm_1":0.00,
        "pm_2p5":0.00,
        "pm_2p5_last_1_hour":0.00,
        "pm_2p5_last_3_hours":0.01,
        "pm_2p5_last_24_hours":1.32,
        "pm_2p5_nowcast":0.01,
        "pm_10":0.50,
        "pm_10_last_1_hour":0.16,
        "pm_10_last_3_hours":0.20,
        "pm_10_last_24_hours":2.01,
        "pm_10_nowcast":0.2,
        "last_report_time":1618152652,
        "pct_pm_data_last_1_hour":100,
        "pct_pm_data_last_3_hours":100,
        "pct_pm_data_last_24_hours":100,
        "pct_pm_data_nowcast":100
      } ]
    },
    "error":null
  }

2021-04-11 16:50:52.706 DecodeAlCurrent: Outdoor - Found AirLink data
2021-04-11 16:50:52.706 DecodeAlCurrent: Outdoor - Using temp/hum data
2021-04-11 16:50:52.706 GetAlCurrent: Outdoor - Releasing lock
2021-04-11 16:50:53.756 Realtime[196]: Start cycle
2021-04-11 16:50:53.758 Realtime[196]: Creating realtime.txt
2021-04-11 16:50:53.762 Realtime[196]: Running SQL command: INSERT IGNORE INTO Realtime (LogDateTime,temp,hum,dew,wspeed,wlatest,bearing,rrate,rfall,press,currentwdir,beaufortnumber,windunit,tempunitnodeg,pressunit,rainunit,windrun,presstrendval,rmonth,ryear,rfallY,intemp,inhum,wchill,temptrend,tempTH,TtempTH,tempTL,TtempTL,windTM,TwindTM,wgustTM,TwgustTM,pressTH,TpressTH,pressTL,TpressTL,version,build,wgust,heatindex,humidex,UV,ET,SolarRad,avgbearing,rhour,forecastnumber,isdaylight,SensorContactLost,wdir,cloudbasevalue,cloudbaseunit,apptemp,SunshineHours,CurrentSolarMax,IsSunny,FeelsLike) Values('21-04-11 16:50:53',5.4,82,2.6,18.3,20.1,328,0.0,3.4,1014.7,'NNW','3','km/h','C','hPa','mm',394.1,'+0.8',24.6,202.8,6.2,20.1,49,1.8,+0.1,7.0,'16:15',2.9,'01:35',34.9,'12:43',46.3,'02:47',1014.7,'16:50',1007.0,'05:14','3.10.5','3122',36.2,5.4,5.4,0.5,0.97,213,325,0.0,1,'1','0','NW',1156,'ft',0.3,0.5,540,'0',1.8)
2021-04-11 16:50:53.794 Realtime[196]: 1 rows were added.
2021-04-11 16:50:53.795 Realtime[196]: Running SQL command: DELETE IGNORE FROM Realtime WHERE LogDateTime < DATE_SUB('2021-04-11 16:50:53', INTERVAL retainVal=2 retainUnit=day)
2021-04-11 16:50:53.819 Realtime[196]: Error encountered during Realtime delete MySQL operation.
2021-04-11 16:50:53.820 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'retainUnit=day)' at line 1
2021-04-11 16:50:53.820 Realtime[196]: Processing realtime file - realtimegauges.txt
2021-04-11 16:50:53.829 Realtime[196]: Processing extra file[1] - utils/airlinkrealtime.txt
2021-04-11 16:50:54.103 LOOP: Data - 18: 4C-4F-4F-3C-00-DD-06-D0-74-AA-02-31-AA-01-09-0A-4A-01-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-FF-52-FF-FF-FF-FF-FF-FF-FF-00-00-05-D5-00-37-00-95-44-11-00-7B-00-24-04-24-00-3C-00-54-01-FF-FF-FF-FF-FF-FF-FF-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-2A-03-06-4B-82-02-EA-07-0A-0D-9C-22
2021-04-11 16:50:54.103 LOOP: 18 - Data packet is good
2021-04-11 16:50:54.366 Realtime[196]: Uploading - realtime.txt
2021-04-11 16:50:54.550 Realtime[196]: Uploading - realtimegauges.txt
2021-04-11 16:50:54.725 Realtime[196]: Uploading extra web file[1] utils/airlinkrealtime.txttmp to /public_html/airlinkrealtime.txt
2021-04-11 16:50:54.910 Realtime[196]: End cycle
    All times are UTC
    Page 3 of 4

    Powered by phpBB® Forum Software © phpBB Limited