Extra web files end of day timing

[broadstairs]

It is created by CMX. Again have you checked the uploaded file, I have and it is correct?

Yes that I'd expect.

However I have to create my file after the dayfile has been updated by CMX at rollover and so I cannot start extracting data until after rollover which means that the first upload has already happened before I can create the file containing the extracted data! The rollover and first upload happens within less than 2 seconds after midnight as I discovered by checking the log. I have to run my extract as a cron job so that is only on the minute so if I run it at 1 minute past midnight I am already about 58 seconds too late!

Stuart

[freddie]

If you set your cron job to run at midnight and have a sleep command as the first statement, you can delay execution by x seconds. If that is what you are trying to do - it isn't entirely clear from your description. Like this:

Code: Select all

!#/bin/bash
sleep(5)
/your/script.sh

[broadstairs]

If you set your cron job to run at midnight and have a sleep command as the first statement, you can delay execution by x seconds. If that is what you are trying to do - it isn't entirely clear from your description. Like this:

Code: Select all

!#/bin/bash
sleep(5)
/your/script.sh

Yes I know I could do that but the rollover and first upload both happen within 1.2 seconds after midnight!

Stuart

[freddie]

I've just re-read the thread and I don't think you can use EOD upload. You're running a script that requires the EOD content to be present in the dayfile, and the result of that script is a file that you would like transferred in the EOD upload.

That's like a self-induced race condition. Sometimes it may work but I wouldn't want to rely on it.

[broadstairs]

I've just re-read the thread and I don't think you can use EOD upload. You're running a script that requires the EOD content to be present in the dayfile, and the result of that script is a file that you would like transferred in the EOD upload.

That's like a self-induced race condition. Sometimes it may work but I wouldn't want to rely on it.

That's exactly what I believe so for now I've modified my script to do the create and upload, but I do need to find a Linux commandline FTP client which will support more than plain old FTP.

Stuart

[rogerthn]

... but I do need to find a Linux commandline FTP client which will support more than plain old FTP.

Stuart

I would try lftp.

[freddie]

sftp is in most distros. Or there's always scp. Or even rsync which will just transfer the delta if you're interested in saving bandwidth.