Welcome to the Cumulus Support forum.

Latest Cumulus MX release v3.0.2 build 3052 - 13 September 2019.
Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014

See the Wiki for downloading either version.

MeteoAdvisory

Discussion of Ken True's web site templates

Moderator: saratogaWX

User avatar
ConligWX
Posts: 750
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis VPro2 Plus +DFARS
Operating System: MeteoBridge Nano SD
Location: Bangor, NI
Contact:

MeteoAdvisory

Post by ConligWX » Sun 22 Jul 2018 1:33 pm

Anyone in the EU using MeteoAdvisory advisors from Ken?


When calling https://www.conligwx.org/wxadvisory.php I get a blank page as the cached file keeps showing 0 bytes and no data is downloaded. I've not changed anything in the configuration but it's seems now broken until I call the following:

https://www.conligwx.org/check-fetch-times.php

--checking METEOalarm warning URL --
Using Settings.php $SITE['EUwarningURL'] entry for test.
URL: https://www.meteoalarm.eu/en_UK/0/0/UK0 ... eland.html
Cache: ./cache/meteoalarm-en.txt
curl fetching 'https://www.meteoalarm.eu/en_UK/0/0/UK0 ... eland.html'
HTTP stats: RC=200 dest=81.31.132.86 port=443 (from sce=172.x.x.x)
Times: dns=0.004 conn=0.006 pxfer=1.201 get=0.107 total=1.308 secs
RC=200 OK, bytes=11248
Data age=0 sec 'Sun, 22 Jul 2018 13:30:08 GMT'
Wrote 11248 bytes to ./cache/meteoalarm-en.txt successfully.
Test was SUCCESSFUL.
--end METEOalarm warning URL check --
Only thing I have changed recently is PHP upgraded to 7.2.8.

Apache is at 2.4.33

ALSO Data age=0 sec 'Sun, 22 Jul 2018 13:30:08 GMT' is 1 hour behind real time, yet webserver time is correct.
Regards Simon

https://www.conligwx.org - @conligwx
Davis Vantage Pro2 Plus + DFARS - Meteobrige Nano SD + Saratoga Templates

User avatar
saratogaWX
Posts: 987
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: MeteoAdvisory

Post by saratogaWX » Sun 22 Jul 2018 8:23 pm

D'Oh.. I forgot to update get-meteoalarm-warning-inc.php to use cURL/https .. the old version was HTTP only.

Please try https://saratoga-weather.org/wxtemplate ... p?sce=view and let me know if that fixes the issue. If it does, I'll release for distribution.

User avatar
ConligWX
Posts: 750
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis VPro2 Plus +DFARS
Operating System: MeteoBridge Nano SD
Location: Bangor, NI
Contact:

Re: MeteoAdvisory

Post by ConligWX » Sun 22 Jul 2018 9:03 pm

saratogaWX wrote:D'Oh.. I forgot to update get-meteoalarm-warning-inc.php to use cURL/https .. the old version was HTTP only.

Please try https://saratoga-weather.org/wxtemplate ... p?sce=view and let me know if that fixes the issue. If it does, I'll release for distribution.
seems to have worked when I polled the full link. my php maybe caching the old one still so will give it a few hours of testing just to see if thats now working.

Thanks for the heads up! :clap:
Regards Simon

https://www.conligwx.org - @conligwx
Davis Vantage Pro2 Plus + DFARS - Meteobrige Nano SD + Saratoga Templates

User avatar
ConligWX
Posts: 750
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis VPro2 Plus +DFARS
Operating System: MeteoBridge Nano SD
Location: Bangor, NI
Contact:

Re: MeteoAdvisory

Post by ConligWX » Sun 22 Jul 2018 10:26 pm

seems to be stable now Ken. I have been refreshing quite often and data has populated the cache file each time whereas before it was always 0 bytes.
Regards Simon

https://www.conligwx.org - @conligwx
Davis Vantage Pro2 Plus + DFARS - Meteobrige Nano SD + Saratoga Templates

User avatar
saratogaWX
Posts: 987
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: MeteoAdvisory

Post by saratogaWX » Sun 22 Jul 2018 10:32 pm

Thanks, Simon! I've released the update via the standalone script page and the template update tool.

User avatar
HansR
Posts: 153
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: MeteoAdvisory

Post by HansR » Sun 23 Jun 2019 9:06 am

Hi,

For one reason or another I can't get the advisory script to work either. I did download the most recent versions (Version 2.10 - 22-Jul-2018 of the 'get-meteoalarm-warning-inc'-script). The debug output is below. After making changes I emtpy my FF-cache and try again, so far unsuccessful and now out of ideas.

Code: Select all

<div id="main-copy">
  
  <!-- lang=nl (ne_NL) used -->
<!-- get-meteoalarm-warning-inc.php V2.10 - 22-Jul-2018 -->
<!-- Warning data courtesy of and Copyright &copy; EUMETNET-METEOalarm (https://www.meteoalarm.eu/).Used with permission.
Time delays between this website and the www.meteoalarm.eu website are possible, for the most up to date information about alert levels as published by the participating National Meteorological Services please use meteoalarm.eu. -->
<!-- curl fetching 'https://www.meteoalarm.eu/ne_NL/0/0/NL-Nederland.html' -->
<!-- curl Error: Operation timed out after 6000 milliseconds with 0 out of 0 bytes received -->
<!-- HTTP stats:  RC=0 dest=81.31.132.86 port=443 (from sce=46.235.40.106)
      Times: dns=1.511 conn=4.551 pxfer=0.000 get=6.000 total=6.000 secs -->
<!-- headers returned:

 -->
<!-- Meteoalarm data load from from URL https://www.meteoalarm.eu/ne_NL/0/0/NL-Nederland.html -->
<!-- data not available cachename: ./cache/meteoalarm-nl.txt 
  -->
  
</div><!-- end main-copy -->
My PHP settings have:

Code: Select all

allow_url_fopen=on
date.timezone = Europe/Amsterdam
display_errors = Off
Sorry, can you see where I am missing the point? Something seems to be similar tot ConligWX's problem which is why I replied to this thread. Maybe I am wrong and it's a simple setting somewhere. Or would you prefer an new thread.

NOTE: 'allow_url_fopen=off' does not make a difference.
NOTE2: When I have 'display_errors = On', I can not display the graph because there is an error output in CU-defs.php line 81 and 82. Would you like me to open a specific thread for this? The error reads as "Notice: A non well formed numeric value encountered in /var/www/clients/client13104/web75415/web/CU-defs.php on line 81
"
Hans Rottier

Davis Vantage Pro 2+
Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)
Cumulus MX 3.0.2 b3052
https://meteo-wagenborgen.nl

User avatar
saratogaWX
Posts: 987
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: MeteoAdvisory

Post by saratogaWX » Tue 25 Jun 2019 2:13 am

The error message above

curl Error: Operation timed out after 6000 milliseconds with 0 out of 0 bytes received -->

really says it all. Your webserver was unable to contact meteoalarm.eu servers after 6 seconds so the curl fetch operation terminated.

Timeout errors like this can be due to:
1) a routing issue between your webserver and meteoalarm.eu
2) a firewall block by meteoalarm.eu on the ip address of your webserver
3) a firewall block by your hoster for outbound https traffic to meteoalarm.eu

The only way to resolve this is to contact your webhoster and ask them to diagnose the connectivity problem.
Tell them that your webserver at IP 46.235.40.106 is not able to connect for https (port 443) to meteoalarm.eu (81.31.132.86)

User avatar
HansR
Posts: 153
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: MeteoAdvisory

Post by HansR » Tue 25 Jun 2019 2:38 pm

Hi, I asked the provider and he has tested it with a small file containing the following:

Code: Select all

<?php

echo shell_exec("curl https://meteoalarm.eu");

?>
You can execute it from my server https://www.meteo-wagenborgen.nl/test_webreus.php

So I went back to the original files to make sure I made no error. Now on my server the filesdates are 5 jan 2011 (wxadvisory) and 22 july 2018 (get-meteoalarm-warning-inc). Straight from your site onto my server. Alas, it does not work. So, could it be something else?

Also your 'check-fetch-times.php' does not see any problems (although btw the UV-test fails on cURL). Continued from here I notice that the cachefile is created and contains the data wanted. So running wxadvisory should give the data from the cache. It does not, so apparently something is wrong with the subsequent handling of the resulting html?

I still don't see what is happening here. Sorry.
Hans Rottier

Davis Vantage Pro 2+
Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)
Cumulus MX 3.0.2 b3052
https://meteo-wagenborgen.nl

User avatar
saratogaWX
Posts: 987
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: MeteoAdvisory

Post by saratogaWX » Tue 25 Jun 2019 6:22 pm

I see that check-fetch-times.php does indicate a connection issue with temis.nl for the UV forecast, but strangely enough, does fetch the meteoalarm.eu page (and cache it). Their test script just shows that the curl command in shell is functioning correctly (and returning output). In PHP the curl functions are provided by a library function which may be different than the curl command in shell, so that is not really diagnostic.

Using check-fetch-times.php?show=info reveals that the curl associated with PHP is
cURL version: 7.29.0
cURL SSL version: NSS/3.36
cURL libz version: 1.2.7
SSL is available
LIBZ is available
cURL protocols supported: dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, ldaps, pop3, pop3s, rtsp, scp, sftp, smtp, smtps, telnet, tftp
On my host running PHP Version: 7.3.6 it shows
cURL version: 7.29.0
cURL SSL version: NSS/3.36
cURL libz version: 1.2.7
SSL is available
LIBZ is available
cURL protocols supported: dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, ldaps, pop3, pop3s, rtsp, scp, sftp, smtp, smtps, telnet, tftp
which is the same as your PHP Version: 7.3.4

Looking at check-fetch-times.php on your site shows two failures:

curl fetching 'http://www.temis.nl/uvradiation/nrt/uvi ... =53.258901'
curl Error: Operation timed out after 4001 milliseconds with 0 out of -1 bytes received
HTTP stats: RC=0 dest=145.23.254.72 port=80 (from sce=46.235.40.106)
Times: dns=0.005 conn=1.011 pxfer=1.011 get=2.991 total=4.002 secs
and

curl fetching 'https://www.meteoalarm.eu/ne_NL/0/0/NL-Nederland.html'
curl Error: Connection timed out after 4001 milliseconds
HTTP stats: RC=0 dest=81.31.132.86 port=443 (from sce=)
Times: dns=0.061 conn=0.000 pxfer=0.000 get=4.001 total=4.001 secs

both with the time-out issue. The meteoalarm.eu fetch has worked before (I saw it), but seems intermittent. That leads me to suspect it is a routing issue between your webserver's IP and both temis.nl and meteoalarm.eu -- the network folks at your hoster are the likely ones to be able to fix that.

User avatar
HansR
Posts: 153
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: MeteoAdvisory

Post by HansR » Tue 25 Jun 2019 8:55 pm

I partly agree: you did not address the issue that the info in the cache (with check-fetch-times) is not handled correc!tly when used within the expiry time after the run of check-fetch-times. I will inform the provider of your remarks concerning the routing. I must say that I have been with this provider for 12 years now with multiple websites and without any problem. We will see, i would be surprised. I'll get back to you witlh his remarks.
Hans Rottier

Davis Vantage Pro 2+
Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)
Cumulus MX 3.0.2 b3052
https://meteo-wagenborgen.nl

User avatar
beteljuice
Posts: 2714
Joined: Tue 09 Dec 2008 1:37 pm
Weather Station: None !
Operating System: ex XP SP3 - now W10 laptop grr
Location: Dudley, West Midlands, UK

Re: MeteoAdvisory

Post by beteljuice » Tue 25 Jun 2019 10:08 pm

Trying the eu pages in my browser, the 'fetch' times were just over 4s ...

So, increase the curl time-out.
Image
......................Imagine, what you will KNOW tomorrow !

User avatar
saratogaWX
Posts: 987
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: MeteoAdvisory

Post by saratogaWX » Wed 26 Jun 2019 12:02 am

That's a good idea to try. In get-meteoalarm-warning-inc.php and get-UV-forecast-inc.php change

Code: Select all

$numberOfSeconds=6;
to

Code: Select all

$numberOfSeconds=9;
and in check-fetch-times.php change

Code: Select all

$numberOfSeconds=4;
to

Code: Select all

$numberOfSeconds=9;
and see if that eliminates the issue.

Be aware that longer timeouts may add to your page-load time for pages that use either/both the get-* scripts. The index.php uses both, so could see an additional 18 seconds if both sites are unavailable and time-out.

User avatar
saratogaWX
Posts: 987
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: MeteoAdvisory

Post by saratogaWX » Wed 26 Jun 2019 1:09 am

Just for reference, I ran the check-fetch-times.php on my default World site on my 1and1 server in the USA, and it showed:

curl fetching 'http://www.temis.nl/uvradiation/nrt/uvi ... 7.27153397'
HTTP stats: RC=200 dest=145.23.254.72 port=80 (from sce=74.208.252.86)
Times: dns=0.253 conn=0.371 pxfer=0.371 get=1.002 total=1.373 secs

and

curl fetching 'http://www.meteoalarm.eu/index3.php?are ... lang=en_UK'
HTTP stats: RC=200 dest=81.31.132.86 port=80 (from sce=74.208.252.86)
Times: dns=0.511 conn=0.652 pxfer=0.652 get=0.173 total=0.825 secs

for temis.nl and meteoalarm.eu -- those are what I'd expect from a well-connected webserver. Your timeouts at 4 seconds are anomalous IMHO and shows an underlying connectivity issue with your webserver and hoster's networking infrastructure.

User avatar
HansR
Posts: 153
Joined: Sat 20 Oct 2012 6:53 am
Weather Station: Davis Vantage Pro 2+
Operating System: Raspbian GNU/Linux 9 (stretch)
Location: Wagenborgen (NL)
Contact:

Re: MeteoAdvisory

Post by HansR » Wed 26 Jun 2019 4:47 am

OK. I did set the timeout to 30 seconds (why try 9 first,if it works timeout is not important). You can try for yourself.

Results: UV-fetch does not work, MeteoAlarm-fetch does work.
Let's be clear: the UV forecast sometimes does work, I have seen it myself work intermittently.

Still, when the meteoalarm-fetch works (as it did this morning) I have no display of the warning. Which leads me to my yesterday's remark
you did not address the issue that the info in the cache (with check-fetch-times) is not handled correctly when used within the expiry time after the run of check-fetch-times
So that remains an issue. I will forward your remarks on the timeouts to the provider. So far, a timeout of 30 seconds does not solve the problem, even if data are fetched. It seems we have two problems:
  1. Timeout
  2. Handling of the result (if any)
-----

EDIT 26 june / 11h32 UTC: My provider gets a response after 63 seconds and advised to set the timeout at 300. I told him that could not be the solution. The problem is that I can not reach temis.nl either from my browser so that has nothing to do with my provider. It has to be somewhere/something else.

EDIT 26 june / 11h46 UTC: I contacted KNMI, the Dutch Meteorological Society, for an explanation on the responsetime of temis.nl . They control the site, I assume the response may be related to the distribution system of the site. Or whatever ... I'll let you know their response.
Hans Rottier

Davis Vantage Pro 2+
Raspberry Pi 3 Model B Plus Rev 1.3
Raspbian GNU/Linux 9 (stretch) / Linux 4.19.58-v7+
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1)
Cumulus MX 3.0.2 b3052
https://meteo-wagenborgen.nl

User avatar
saratogaWX
Posts: 987
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: MeteoAdvisory

Post by saratogaWX » Wed 26 Jun 2019 3:11 pm

Regarding
you did not address the issue that the info in the cache (with check-fetch-times) is not handled correctly when used within the expiry time after the run of check-fetch-times
The check-fetch-times.php script does not pay attention to the update time of the cache file -- it always does a reload from the URL in order to test it.
The get-meteoalarm-warning-inc.php script does pay attention to the cache file last-modified time and updates only if it is older than the

Code: Select all

$EUArefetchSeconds = 300;  // refresh every 5 minutes


Would you mind elaborating on what you mean by 'is not handled correctly'?

KNMI/temis.nl has been known to block IP addresses/ranges when they feel abused. It's possible that your webserver may be on their naughty list and so is dropping connection requests, or.. it still could be a routing issue webserver->temis.nl

I agree with you that a recommendation of a 300 second timeout by your hoster is not a useful suggestion and wildly impractical.

Post Reply