ImportCumulusFile.php - version 2.0

[mcrossley]

That will do it, or
retainVal=7
retianUnit=day

[thekow]

What is the best practice to import the realtime data. At the moment i am storing 3 hours of data in a txtfile. Then processing it every 3 hours via the import php. Does this script overwrite identical entries or ignore them? Trying to figure out if i am going to run into issues when i eventually start running the recent graphs.

[mcrossley]

The realtime table has a primary key on the date time so it will not allow duplicate rows.

I am running the ImportCumlusFile.PHP script from the command line on my web server once a minute to populate the real time table.

[laulau]

Hi,
I'm trying this script to generate a realtime database.
I set up a cron job to populate the database (every 5 min).
I started this 3 hours ago and the database is populated but the old data are not deleted!
my command line :

Code: Select all

php /PathToWeb/test/ImportCumulusFile.php type=realtime key=**** table=Realtime file=/PathToWeb/realtime.txt retainVal=1 retainUnit=hour

The database should contain only 1 hour of data but for the moment all the data are kept in.
I tryed the command manualy :

• 24/05/13 - 17:29:49
  Importing to table: Realtime ...
  Processing realtimefile: ../realtime.txt
  Inserting data... done.
  Deleting old realtime records...done.
  24/05/13 - 17:29:51

But the old realtime records are still here.

What is going wrong ?
Thanks

[laulau]

Hi again,
After a longer observation, it seem's that my realtime database retains about 3 hours data (one hour was expected with my parameters).
there are 36 lines in the database.
The script run every 5 min, with retainVal=1 retainUnit=hour i should only have 12 lines in the DB.

I don't understand where it hangs!

[laulau]

It's a time zone issue i think.
I have put 2:00 instead of 0:00 in this part of the code :

Code: Select all

if (!mysql_query('SET time_zone="+[b]2[/b]:00"')) {
        die('Error: TZ Statement');
    }

And now it works i have only 1 hour of data instead 3 hours.

What would happened at the next hour change (winter time UTC+1 <> summer time UTC+2) ?
Is there a way to make this change in the script automaticaly?

[mcrossley]

I need to look at the tz again, I thought that forcing the time zone to UTC would avoid these sorts of problems - obviously not.