Cumulus Support

Hi everybody!
Have one question......
I made a two tables on my database (dayfile and monthlylog file). By Cumulus toolbox I upload dayfile on my web server and with remote commands I place all data on dayfile table and that is all ok.
Now, I saw on my table a number of rows, one row for each day. What's gonna happend for three years. Is that mean that my table gonna have 1095 rows and 3650 rows for ten years?
.....regarding a monthly log table. On which way monthly log file upload on my web server? Every month a new file? On which way monthly's log change month on the last day of every month? Will I have a 3 monthly log files on my web server for three months?

Thanks a lot............

Yes the day file table will grow larger by one row a day, that shouldn't be an issue in your lifetime

Again you will have one monthly file per month, I add the new data from the current monthly file every night after I do the day file import. That way there isn't to much data to import in one hit, and all the previous days data is available. Depending on what you want to use the log data for you could import it more or less frequently.

Yeah that's my lifetime issue and hope it will take a long time
But regarding to my "long time-life time" issue I got a very "long" monthly data table. I am talking about my march14 monthly log file. It is so huge that I receive error message from my web server…. As I have my "data log interval" on my Cumulus Station settings set on 1min, I am talking about my 4 or 5 MB data by month. So it is a 43 200 rows on my data table on my web space……more than 500 000 rows by year.

To be honest , I am a bit of statistic freak I wish that one day, when I grow up have very similar website as yours. Actually I am already working on that

What I do not understand the "mechanism" which allows to (e.g. on your website) have so much data. I'm talking about the dayfiles for more than four years? It is 1460 rows in one table?

I'm totally new to all this and I read almost the entire forum and wiki pages that have helped me very much . However, very often It seems to me that these materials are written for those who already know everything about a subject. As a beginner, usually I have a hundreds of question ….
And of course, members of the forum have helped me very much and I am very grateful to all of you.
As such, frequently I have to make a lot of effort that finally found out that some issue is a trivial thing. On the other hand it is also the best way to learn something. Someone already wrote about this on the forum.
Anyway, for this moment I'm interested for the
1. "mechanism" that allows storage large (few years e. g. ) on my web server and as I wrote in a previous post,
2. On which way "automatically" enter data in a table for next month or how to change
remote command line from the Tool box when month is over . Seems to me illogical that I have to do manually at the end of each month?
3. When you say that you "add the new data from the current monthly file every night after I do the day file import" how you did that. In monthly log file I have 1440 rows per day.....

Many thanks on your help

I use a 5 minute logging interval so only a fifth of your data!

1. So the day file table is tiny in database terms, it is only thousands of rows. Take my 'top 10s' page, that queries the full day file table 42 times (actually more iirc because of he way some queries are constructed) and with 4 years worth of data it can generally do that in under 0.1 seconds on a 'home' computer with a slow disk.

2. I no longer use Toolbox, I use a script called by Cumulus at the end of day roll-over. That copies any 'updated' log files to the web server, and then works out the current log filename from the date and imports it.

3. Do you have a problem importing that many rows?

I am not actually using the log file data at present, and I will probably look at a way of rolling it up to say hourly data in the future (Cumulus already provides the daily and monthly roll ups).

mcrossley,

the good news is that my tables is not limited by the row number, just by the web space....word "just" sounds so "ominous" in this case

When you say that you copy your updated log file, is that mean that you on every end of day roll-over copy entire e. g. monthly file or just data of day just roll over which be added to existing monthly file data on your web server

So I have no problem about row importing just was an error when I try to change the number of rows which be displayed on my monitor from my data table

I still curious......do I need to change command line in toolbox on end of every month or that process will be executed automatically
What kind of data I must to upload if I want to display data as you do on your web pages....dayfile, monthly, annual, records?.....is that all?

Thaaaaaaaaanks

....btw I am really envy you on your web page . Great work, just as Cumulus (refers to Steve)......

mm23 wrote:When you say that you copy your updated log file, is that mean that you on every end of day roll-over copy entire e. g. monthly file or just data of day just roll over which be added to existing monthly file data on your web server

The whole file.

mm23 wrote:I still curious......do I need to change command line in toolbox on end of every month or that process will be executed automatically

You have to change it, or not use toolbox to do the insert.

mm23 wrote:What kind of data I must to upload if I want to display data as you do on your web pages....dayfile, monthly, annual, records?.....is that all?

Most data is just from the dayfile or Cumulus tags, the realtime charts obviously use a realtime data table.

Thanks very much Mark on your answers. In this moment I try to menage my NOAA report. Your suggestions is welcome and surely I will try to use It in a days upcoming.....