Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

get-USNO-sunmoon.php Slow Page Load

Discussion of Ken True's web site templates

Moderator: saratogaWX

WoodburyMan
Posts: 166
Joined: Sun 04 Sep 2011 2:02 am
Weather Station: Ecowitt GW1000
Operating System: Windows 10 20H2
Location: Woodbury, Connecticut, U.S.A.
Contact:

get-USNO-sunmoon.php Slow Page Load

Post by WoodburyMan »

Greetings!

I am having a issue with the template on my page WoodburyWeather.com with slow page loads. It only happens when cached data is being refreshed, when it has been a while since the last page view. These slow page loads will result in a blank page for anywhere from 30-120 seconds while PHP is processing in the background. I figured out that it only happened when certain cached data was being refreshed.

I'm unsure of when this started. It's been happening for at least a week almost now perhaps a month.

I narrowed it down to a issue in wxastronomy.php, more specifically get-USNO-sunmoon.php, and very specifically I think it's having issues connecting to

Code: Select all

http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset
which in my case is

Code: Select all

http://api.usno.navy.mil/rstt/oneday?date=03/21/2016&coords=41.553333,-73.224167&tz=-4
when it goes to pull for data, or something else in the function where it pulls data from the url is timing out.

On the wxastronomy.php page, I get the following above the Sun and Moon.

Code: Select all

Warning: Invalid argument supplied for foreach() in /home/woodburyman/woodburyweather.com/get-USNO-sunmoon.php on line 359

Warning: Invalid argument supplied for foreach() in /home/woodburyman/woodburyweather.com/get-USNO-sunmoon.php on line 368
To trigger the slow loads, I can clear the cache folder, refresh the page. I sat and watched the page load while refreshing the directory listing on the cache folder. After anywhere from 30 to almost 120 seconds, the page finally loads, and at the same time a 0byte USNO-moondata.txt file is created in the cache folder, then the scripts all work fine until it tries to refresh the data again. If there is a 0 byte file, those PHP errors will appear.

Once in a while, if I clear the cache, I will get a healthy 900byte+ byte USNO-moondata.txt in the cache folder. The PHP errors will disappear. Even on these load times, it can take 30-60 seconds of waiting. I believe their site or something is having issues, and is causing a time out.

Just wondering if anyone else is having these issues, or if there's any further testing I can do to narrow it down?
Woodbury, CT Weather Ecowitt GW1100, Solar, Wind, Rain, Temp, Soil, Lightning
Southington, CT Weather Ecowitt GW2000 Wittboy Solar, Wind, Rain, Temp, Soil, Lightning, CO2/PM2.5/PM10

Running CumuluxMX 3.24.1 b3234 both sites on Windows 11 22H2
User avatar
saratogaWX
Posts: 1170
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by saratogaWX »

If you do a view-source on wxastronomy.php?debug=y&force=1 you'll see
<!-- get-USNO-sunmoon.php - Version 2.04 - 11-Nov-2015 -->
<!-- refetch seconds=1 -->
<!-- loading ./cache/USNO-moondata.txt from http://api.usno.navy.mil/rstt/oneday?da ... 4167&tz=-4 -->
<!-- GET /rstt/oneday?date=03/21/2016&coords=41.553333,-73.224167&tz=-4 HTTP/1.1
Host: api.usno.navy.mil Port: 80 IP=api.usno.navy.mil-->
<!-- Network error: php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution (0) -->
<!-- HTTP stats: dns=28.025 conn=56.049 put=n/a get( blocks)=n/a close=n/a total=84.074 secs -->
<!-- fetch function elapsed= 84 secs. -->
which indicates an issue with your hoster's DNS resolver -- it should only take < 1 second to look up the IP address,
and it failed on the first try. Then the connect failed because it tried to use the failing DNS lookup again, and likely multiple times.

Call the tech support for your hoster and let them know their caching DNS is choking trying to resolve api.usno.navy.mil -- it should return
Query for api.usno.navy.mil type=255 class=1
api.usno.navy.mil A (Address) 199.211.133.93
in a fraction of a second, and not 28+ seconds!

Here's what a normal one looks like
<!-- get-USNO-sunmoon.php - Version 2.04 - 11-Nov-2015 -->
<!-- local date changed v.s. cache date .. reloading from USNO -->
<!-- refetch seconds=1 -->
<!-- loading ./cache/USNO-moondata.txt from http://api.usno.navy.mil/rstt/oneday?da ... 4323&tz=-7 -->
<!-- GET /rstt/oneday?date=03/21/2016&coords=37.27153397,-122.02274323&tz=-7 HTTP/1.1
Host: api.usno.navy.mil Port: 80 IP=199.211.133.93-->
<!-- Request:
GET /rstt/oneday?date=03/21/2016&coords=37.27153397,-122.02274323&tz=-7 HTTP/1.1
Host: api.usno.navy.mil
Connection: Close
User-agent: Mozilla/5.0 (get-USNO-sunmoon.php; saratoga-weather.org)
Accept: text/plain,text/html
Accept-Encoding: gzip;q=0,compress;q=0


-->
<!-- HTTP stats: dns=0.098 conn=0.065 put=0.000 get(42 blocks)=0.186 close=0.000 total=0.350 secs -->
<!-- fetch function elapsed= 0 secs. -->
WoodburyMan
Posts: 166
Joined: Sun 04 Sep 2011 2:02 am
Weather Station: Ecowitt GW1000
Operating System: Windows 10 20H2
Location: Woodbury, Connecticut, U.S.A.
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by WoodburyMan »

Oh wow. 84 seconds for a DNS lookup! Thank you! I'll contact them to find out what's going on.
I get the same result via SSH using my hosting providers DNS servers. Using 8.8.8.8 I get a instant responce.
Woodbury, CT Weather Ecowitt GW1100, Solar, Wind, Rain, Temp, Soil, Lightning
Southington, CT Weather Ecowitt GW2000 Wittboy Solar, Wind, Rain, Temp, Soil, Lightning, CO2/PM2.5/PM10

Running CumuluxMX 3.24.1 b3234 both sites on Windows 11 22H2
WoodburyMan
Posts: 166
Joined: Sun 04 Sep 2011 2:02 am
Weather Station: Ecowitt GW1000
Operating System: Windows 10 20H2
Location: Woodbury, Connecticut, U.S.A.
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by WoodburyMan »

My hosting provider has been very slow to respond. They've confirmed they have an issue with the DNS server my server uses, but have yet to do anything.

As a work around for now, I've edited line 122 of get-USNO-sunmoon.php from

Code: Select all

$USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
to

Code: Select all

// $USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
$USNOUrl = "http://199.211.133.93/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
Thankfully the server it pulls from is a dedicated server and not a shared host with many sites sharing the same IP so I am able to substitute the IP for now. I will have to update it if it changes...
Woodbury, CT Weather Ecowitt GW1100, Solar, Wind, Rain, Temp, Soil, Lightning
Southington, CT Weather Ecowitt GW2000 Wittboy Solar, Wind, Rain, Temp, Soil, Lightning, CO2/PM2.5/PM10

Running CumuluxMX 3.24.1 b3234 both sites on Windows 11 22H2
User avatar
rogerthn
Posts: 489
Joined: Thu 11 Apr 2013 6:31 pm
Weather Station: Ecowitt GW1000/GW1003
Operating System: Raspberry Pi OS bullseye aarch64
Location: Trollhättan Sweden
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by rogerthn »

Possible workaround while waiting for your hosting provider to behave ;-)

Replace or add nameserver line below in /etc/resolv.conf

Code: Select all

nameserver 8.8.8.8
This is working OK for me until I reboot when nameserver is replaced by "dhcp"
To disable the dhcp update of /etc/resolv.conf I'm currently testing with "write protected" /etc/resolv.conf as below

Code: Select all

sudo chattr +i /etc/resolv.conf
To remove write protection do

Code: Select all

sudo chattr -i /etc/resolv.conf
Image
WoodburyMan
Posts: 166
Joined: Sun 04 Sep 2011 2:02 am
Weather Station: Ecowitt GW1000
Operating System: Windows 10 20H2
Location: Woodbury, Connecticut, U.S.A.
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by WoodburyMan »

rogerthn wrote:Possible workaround while waiting for your hosting provider to behave ;-)

Replace or add nameserver line below in /etc/resolv.conf

Code: Select all

nameserver 8.8.8.8
This is working OK for me until I reboot when nameserver is replaced by "dhcp"
To disable the dhcp update of /etc/resolv.conf I'm currently testing with "write protected" /etc/resolv.conf as below

Code: Select all

sudo chattr +i /etc/resolv.conf
To remove write protection do

Code: Select all

sudo chattr -i /etc/resolv.conf
Shared hosting :(. Anything outside my profile is not accessible. I have to modify php settings in a special php folder in my profile the provider provides. Dont have enough traffic to warrant a VPS at this point.
Woodbury, CT Weather Ecowitt GW1100, Solar, Wind, Rain, Temp, Soil, Lightning
Southington, CT Weather Ecowitt GW2000 Wittboy Solar, Wind, Rain, Temp, Soil, Lightning, CO2/PM2.5/PM10

Running CumuluxMX 3.24.1 b3234 both sites on Windows 11 22H2
User avatar
rogerthn
Posts: 489
Joined: Thu 11 Apr 2013 6:31 pm
Weather Station: Ecowitt GW1000/GW1003
Operating System: Raspberry Pi OS bullseye aarch64
Location: Trollhättan Sweden
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by rogerthn »

WoodburyMan wrote:Shared hosting :(. Anything outside my profile is not accessible. I have to modify php settings in a special php folder in my profile the provider provides. Dont have enough traffic to warrant a VPS at this point.
Whoops, I did see "Operating System: RaspberryPi Model B 3.0" in your profile and the sudo chattr +i /etc/resolv.conf seems to have fixed my issue :bash:
I do not know how to use https://pear.php.net/manual/en/package. ... .query.php but maybe you can?
Image
Simnm
Posts: 4
Joined: Tue 15 Nov 2016 11:11 pm
Weather Station: Vantage Pro2
Operating System: Windows 10

Re: get-USNO-sunmoon.php Slow Page Load

Post by Simnm »

WoodburyMan wrote:My hosting provider has been very slow to respond. They've confirmed they have an issue with the DNS server my server uses, but have yet to do anything.

As a work around for now, I've edited line 122 of get-USNO-sunmoon.php from

Code: Select all

$USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
to

Code: Select all

// $USNOUrl = "http://api.usno.navy.mil/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
$USNOUrl = "http://199.211.133.93/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
Thankfully the server it pulls from is a dedicated server and not a shared host with many sites sharing the same IP so I am able to substitute the IP for now. I will have to update it if it changes...
Thank For the solution. I had the same problem. I need to keep the same ip address?

My host provided this answer to the problem :

the
issue is tied to api.usno.navy.mil itself and their servers rate-limiting
our own queries to their DNS servers, I'm afraid. As this issue is due
to rate-limiting from an outside source which we have no direct control
over, unfortunately, we are not able to make any changes to their own
rate-limiting settings from our side of things directly, and I apologize
for the trouble with this.

...........................................................................
User avatar
ConligWX
Posts: 1570
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis vPro2+ w/DFARS + AirLink
Operating System: Ubuntu 22.04 LTS
Location: Bangor, NI
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by ConligWX »

I seem to be getting the same issue from my site today.
Warning: Invalid argument supplied for foreach() in /share/CACHEDEV1_DATA/Web/weather/get-USNO-sunmoon.php on line 359

Warning: Invalid argument supplied for foreach() in /share/CACHEDEV1_DATA/Web/weather/get-USNO-sunmoon.php on line 368
but http://api.usno.navy.mil seems to be down...

anyone else having issues?
Regards Simon

https://www.conligwx.org - @conligwx
Davis Vantage Pro2 Plus with Daytime FARS • WeatherLink Live • Davis AirLink • PurpleAir •

Image
User avatar
saratogaWX
Posts: 1170
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by saratogaWX »

Right now, it's not a hoster issue.. it's that usno.navy.mil and api.usno.navy.mil DNS zones fail to resolve (even using OpenDNS, Google DNS, etc).

I've sent a note to hostmaster at navy.mil asking for their help to get the usno.navy.mil Zone working again.

Meanwhile.. you can rename get-USNO-sunmoon.php to get-USNO-sunmoon-temp.php to take it out of the page-load process. Your sunrise/sunset/moonrise/moonset times will be missing and the moonphase only approximated (based on the built-in functions in common.php), but your page will load ok.
User avatar
ConligWX
Posts: 1570
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis vPro2+ w/DFARS + AirLink
Operating System: Ubuntu 22.04 LTS
Location: Bangor, NI
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by ConligWX »

Thank for the update. Tried using IP address but that failed too. looks like they have took it down or moved hosts/addresses then
Last edited by ConligWX on Wed 23 Nov 2016 9:26 pm, edited 1 time in total.
Regards Simon

https://www.conligwx.org - @conligwx
Davis Vantage Pro2 Plus with Daytime FARS • WeatherLink Live • Davis AirLink • PurpleAir •

Image
User avatar
ConligWX
Posts: 1570
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis vPro2+ w/DFARS + AirLink
Operating System: Ubuntu 22.04 LTS
Location: Bangor, NI
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by ConligWX »

but the direct path here:

http://199.211.133.93/rstt/oneday?date= ... 5.672&tz=0

WORKS!
{
"error":false,
"apiversion":"1.3.0",
"year":2016,
"month":11,
"day":23,
"dayofweek":"Wednesday",
"datechanged":false,
"lon":-5.671917,
"lat":54.634060,
"tz":0,

"sundata":[
{"phen":"BC", "time":"07:27"},
{"phen":"R", "time":"08:09"},
{"phen":"U", "time":"12:09"},
{"phen":"S", "time":"16:09"},
{"phen":"EC", "time":"16:51"}],

"moondata":[
{"phen":"R", "time":"01:15"},
{"phen":"U", "time":"07:52"},
{"phen":"S", "time":"14:17"}],

"closestphase":{"phase":"Last Quarter","date":"November 21, 2016","time":"08:33"},
"fracillum":"29%",
"curphase":"Waning Crescent"
}
I've edited the url to use the ip address for now.
Regards Simon

https://www.conligwx.org - @conligwx
Davis Vantage Pro2 Plus with Daytime FARS • WeatherLink Live • Davis AirLink • PurpleAir •

Image
Supercell
Posts: 16
Joined: Sat 13 Oct 2012 9:37 pm
Weather Station: Vue + B-L Sunrecorder
Operating System: Win7
Location: Dublin, Ireland

Re: get-USNO-sunmoon.php Slow Page Load

Post by Supercell »

Thanks Simon, used the IP on mine too and all working fine again.
Image
User avatar
saratogaWX
Posts: 1170
Joined: Wed 06 May 2009 5:02 am
Weather Station: Davis Vantage Pro Plus
Operating System: Windows 10 Professional
Location: Saratoga, CA, USA
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by saratogaWX »

Yes, thanks..

To bypass the problem, just put

Code: Select all

$USNOUrl = "http://199.211.133.93/rstt/oneday?date=$lclToday&coords=$myLat,$myLong&tz=$myTZOffset";
after the line that has $USNOUrl =... so the IP address will be used instead. There still seems to be a problem with the entire usno.navy.mil DNS zone. I did send a note about it to hostmaster at navy.mil
User avatar
ConligWX
Posts: 1570
Joined: Mon 19 May 2014 10:45 pm
Weather Station: Davis vPro2+ w/DFARS + AirLink
Operating System: Ubuntu 22.04 LTS
Location: Bangor, NI
Contact:

Re: get-USNO-sunmoon.php Slow Page Load

Post by ConligWX »

Ken it might be because the US navy got hacked!

http://www.bbc.co.uk/news/technology-38090234
Regards Simon

https://www.conligwx.org - @conligwx
Davis Vantage Pro2 Plus with Daytime FARS • WeatherLink Live • Davis AirLink • PurpleAir •

Image
Post Reply