Error Processing "Extra Web Files" - CMX 3.24.2

astroed · Post by **astroed** » Fri 17 Mar 2023 3:31 pm

After updating to CMX 3.24.0 build 3231, I notice my web pages were occasionally failing. The home page was blank with the error shown below displayed.

System.ArgumentOutOfRangeException: startIndex cannot be larger than length of string. Parameter name: startIndex at System.String.Substring(Int32 startIndex, Int32 length) at CumulusMX.TokenParser.Parse3() at CumulusMX.TokenParser.ToString()

This problem continued even after upgrading to CMX 3.24.1 build 3234 and CMX 3.24.2 build 3235. Yesterday I finally located the cause, the Cumulus Template File was not processing correctly. I have this file configured in "Extra Web Files" to process and upload via FTP every 5 minutes. When a failure occurs, I found the uploaded CUtags file contained that very same error and nothing else. All the tags and their values were missing. This processing failure seems to be a random event. Most of the time the CUtags file was processed and uploaded correctly. Usually, the next update after a failure the uploaded CUtags file does process correctly, and the web site starts to work again. There were no errors associated with these events in the MXdiags files. Last night I went back to CMX v.3.23.1 build 3221. No processing failures were observed after running that version all night.

Any help in resolving this issue would be appreciated!
Ed
https://weather.ejmj.net/

mcrossley · Post by **mcrossley** » Fri 17 Mar 2023 3:43 pm

Do you have just one Extra web files entry processing that template file?

astroed · Post by **astroed** » Fri 17 Mar 2023 3:48 pm

Yes, just that one file.

astroed · Post by **astroed** » Fri 17 Mar 2023 4:39 pm

Mark,

I may have misunderstood your question. I am processing two files every 5 minutes. The one that's giving me trouble is for the main web pages and a second file is for an iphone page. Both files do have some of the same tags. Sorry about the confusion.

Ed

mcrossley · Post by **mcrossley** » Fri 17 Mar 2023 5:01 pm

I did mean two entries processing the same template file. I wondered if they were trying to read the template file at the same time.

What may help is a log file with debug logging enabled, you could switch back once you have captured one error. I know there is no error in the log file, but if you also log the time of the failure I can look and see what was happening.

Meantime I have added some code to capture and log this error.

astroed · Post by **astroed** » Mon 20 Mar 2023 12:37 am

I had several failures today and finally caught one with a known time stamp. The attached MXdiag file had an event at the 18:40:00 updates. I had both debug logging and FTP logging enabled. The CUtags file appeared to upload normally but as before, the file uploaded was not processed correctly (see the attached CUtags.txt file). I will start using the previous version of CMX pending a fix to this issue. I will be happy to run any additional tests you might need, just let me know.
Thanks for working on this!

Ed

mcrossley · Post by **mcrossley** » Mon 20 Mar 2023 3:52 pm

OK thanks, I'm not 100% sure why it is happening, but I have changed the token parser code to try and prevent it, and at least detect the issue if it does happen.

Found it! An error in the real-time upload of realtimegauges.txt meant it could corrupt the interval upload processed file content.

astroed · Post by **astroed** » Mon 20 Mar 2023 6:13 pm

That's good news! I've been having another issue that I planned to report separately, but it may be related to the issue you found uploading the realtimegauges.txt. This problem concerns the new php upload feature. The only file that uploads using this method is the realtime.txt file. All other files generate errors beginning with the realtimegauges.txt file. Here is a typical block of errors found in the MXdiag.

Code: Select all

2023-03-16 13:13:49.708 PHP[78]: realtimegauges.txt: Response code = 406: NotAcceptable
2023-03-16 13:13:49.708 PHP[78]: realtimegauges.txt: Response text follows:
<head><title>Not Acceptable!</title></head><body><h1>Not Acceptable!</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>

The block repeats until other files need uploading which generate similar errors.

Code: Select all

2023-03-16 13:15:04.518 PHP[Int]: Error - Error while copying content to a stream.
2023-03-16 13:15:04.518 PHP[Int]: Base exception - Unable to read data from the transport connection: The connection was closed.
2023-03-16 13:15:04.612 PHP[Int]: Uploading daily graph data file: alldailytempdata.json
2023-03-16 13:15:04.752 PHP[Int]: winddata.json: Response code = 406: NotAcceptable
2023-03-16 13:15:04.752 PHP[Int]: winddata.json: Response text follows:
<head><title>Not Acceptable!</title></head><body><h1>Not Acceptable!</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>
2023-03-16 13:15:04.752 PHP[Int]: Error - Error while copying content to a stream.
2023-03-16 13:15:04.752 PHP[Int]: Base exception - Unable to read data from the transport connection: The connection was closed.
2023-03-16 13:15:05.049 PHP[Int]: dailyrain.json: Response code = 406: NotAcceptable
2023-03-16 13:15:05.049 PHP[Int]: dailyrain.json: Response text follows:
<head><title>Not Acceptable!</title></head><body><h1>Not Acceptable!</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>
2023-03-16 13:15:05.049 PHP[Int]: Error - Error while copying content to a stream.
2023-03-16 13:15:05.049 PHP[Int]: Base exception - Unable to read data from the transport connection: The connection was closed.
2023-03-16 13:15:05.362 PHP[Int]: wdirdata.json: Response code = 406: NotAcceptable
2023-03-16 13:15:05.362 PHP[Int]: wdirdata.json: Response text follows:
<head><title>Not Acceptable!</title></head><body><h1>Not Acceptable!</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>
2023-03-16 13:15:05.455 PHP[Int]: Error - An error occurred while sending the request.
2023-03-16 13:15:05.455 PHP[Int]: Base exception - The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.
2023-03-16 13:15:05.471 PHP[Int]: Uploading daily graph data file: alldailypressdata.json
2023-03-16 13:15:05.690 PHP[Int]: Uploading daily graph data file: alldailywinddata.json
2023-03-16 13:15:05.862 PHP[Int]: raindata.json: Response code = 406: NotAcceptable
2023-03-16 13:15:05.862 PHP[Int]: sunhours.json: Response code = 406: NotAcceptable
2023-03-16 13:15:05.862 PHP[Int]: raindata.json: Response text follows:
<head><title>Not Acceptable!</title></head><body><h1>Not Acceptable!</h1><p>An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.</p></body></html>

In case these two issues are related I thought it was a good idea to let you know now.

mcrossley · Post by **mcrossley** » Mon 20 Mar 2023 9:03 pm

Your error looks like that reported by someone else - viewtopic.php?p=170978#p170978 - where the server security is blocking the uploads.

Cumulus Support

Error Processing "Extra Web Files" - CMX 3.24.2

Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2

Re: Error Processing "Extra Web Files" - CMX 3.24.2