Cumulus Support

Looking at the settings for Extra Web Files there is an option for end of day but what time does that happen? I have a cron job which runs currently just after midnight to extract data from the day file (I need data for the day just finished!) so ideally I like to use this setting but if it happens at or just before midnight it wont work. Currently I allow it to upload all day but that is a waste of bandwidth as it does not change during the day.

Stuart

If you use End of Day it does upload it once when all the other End of Day files get uploaded and the daily backup is produced. I use it for my extratags.php and my dayfile.txt and they were uploaded .03 and .04 seconds after midnight. If you start the cron job 10 seconds after midnight you should get it every time.

water01 wrote: ↑Thu 01 Aug 2024 5:28 pm If you use End of Day it does upload it once when all the other End of Day files get uploaded and the daily backup is produced. I use it for my extratags.php and my dayfile.txt and they were uploaded .03 and .04 seconds after midnight. If you start the cron job 10 seconds after midnight you should get it every time.

Sorry that does not compute. Ten seconds sounds too late.

Stuart

OK do 5 seconds, it was only a suggestion.

water01 wrote: ↑Thu 01 Aug 2024 5:56 pm OK do 5 seconds, it was only a suggestion.

But can I be sure the dayfile.txt will have the previous day in it?

Stuart

Well mine did 3 seconds after midnight, and I am using SFTP which is usually slower than the other FTP methods. I use it for the dayfile reader.

Extra files with EOD ticked are transferred at the next upload interval after rollover.

mcrossley wrote: ↑Thu 01 Aug 2024 7:50 pm Extra files with EOD ticked are transferred at the next upload interval after rollover.

Thanks Mark I'll see what I can do to catch that.

Stuart

Mark I just tried this and got an error as it tried to copy the file which obviously fails due to the destination path. For the file I had set up I only ticked EOD and nothing else. Can you confirm whether I need to tick upload as well? Also does this mean that it will also upload at every interval as well, if so that was not what I hoped. I only want it uploaded once at EOD.

Stuart

Without upload ticked the file will be copied to the destination - so the destination must be a valid file path.

With upload ticked the file will be transferred using your normal upload method (FTP, PHP etc) the destination is the path on the upload server.

With EOD ticked the copy/transfer will only occur once a day, during the first interval after rollover along with the "standard" daily files.

Thanks Mark I was just about to post this

Looks like I've answered my own question, yes I need upload and with EOD it only uploads at EOD. So that's fine exactly what I need.

but got sidetracked and you then beat me to it!

Stuart

Mark I don't think the EOD option will work for me, or perhaps anyone, who needs to get data from files after rollover to then upload that extracted data, in my case I am extracting from the dayfile. My objective of using the EOD was to reduce upload bandwidth by only uploading the file once as it only gets updated once per 24 hours. My normal upload interval is set to 1 minute so I assumed that the first upload would be at 00:01.

I looked at the log file and could see the dayfile was updated in the first second after midnight but the first upload happened at fractionally over 1 second past midnight. I just don't see how this can be used for uploading data which has to be created following rollover unless the file is created at rollover.

Stuart

Have you checked the uploaded file as mine uploaded 2 seconds after midnight definitely contains the line of data for the previous day, so I see no reason why yours shouldn't.

water01 wrote: ↑Tue 06 Aug 2024 8:13 am Have you checked the uploaded file as mine uploaded 2 seconds after midnight definitely contains the line of data for the previous day, so I see no reason why yours shouldn't.

OK but is your file created at rollover by CMX or do you create it?

Stuart

It is created by CMX. Again have you checked the uploaded file, I have and it is correct?