dayfile.txt

[muppet77]

i am thinking of moving from WD>Cumulus

can someone please check these 2 ideas?

(1)
i have some key weather variables recorded in Excel that were generated by WD (max temp, time, min temp, time etc)
If I can mock up a dayfile.txt (with some values as blanks as i don't have them) can i then seamlessly drop this into the correct folder and moive forward with Cu adding to it?

so for example if i didn't have the maximum humidity that day i would just leave a blank between the commas
,,

However I note that in http://www.sanday.org.uk/weather/dayfile.txt the number of columns increases with the version changes and updates. This makes me think that it's not easy.

(2)
second q is a little more of a gamble and i fear the answer.
can the monthly, yearly and all time records and extremes be generated automatically in CU using this mocked up dayfile.txt??

fingers crossed.

thanks

[steve]

If I can mock up a dayfile.txt (with some values as blanks as i don't have them) can i then seamlessly drop this into the correct folder and moive forward with Cu adding to it?

so for example if i didn't have the maximum humidity that day i would just leave a blank between the commas
,,

I think you will need to put valid figures in rather than nulls.

second q is a little more of a gamble and i fear the answer.
can the monthly, yearly and all time records and extremes be generated automatically in CU using this mocked up dayfile.txt??

Yes, the editors in Cumulus can generate highs and lows from dayfile.txt and/or the monthly log files.

[muppet77]

That's great Steve, so you think a zero value rather than a null.

Cheekily i edited my post to include

However I note that in http://www.sanday.org.uk/weather/dayfile.txt the number of columns increases with the version changes and updates. This makes me think that it's not easy.

How do the column headings cope with knowing which is which if the number of columns is different?

[steve]

How do the column headings cope with knowing which is which if the number of columns is different?

Sorry, I don't understand. The columns are always in the same position. New columns added over time always go at the end. The code copes by checking where the last valid item is.

[muppet77]

But if I copied the data into excel a day with 20 columns back in 2010 wouldn't match up with a day from 2013 that has 43 columns.

[steve]

Sorry, I still can't understand what point you're trying to make. If you do anything in Cumulus which requires it to read dayfile.txt, it reads as many columns in each entry as exist in that entry. If the entry is from an older version of Cumulus, then it can't use the data that isn't there, and it doesn't try.

[muppet77]

Taking the eg at http://www.sanday.org.uk/weather/dayfile.txt

If I export this to excel with the 40 odd column headings in the wiki file, the 40 odd headings don't match up with the early data, which has less columns.

So i would need to put a ,, or a zero to make sure that the column headings match up if I create my own ( as my version will have some variables missing).

Does that make sense?

But you think that the records and extremes can be created just from the dayfile that I build?

[steve]

If I export this to excel with the 40 odd column headings in the wiki file, the 40 odd headings don't match up with the early data, which has less columns.

The columns which are there match up with the headings. The columns which aren't there obviously don't match up with the headings, because... they aren't there.

So i would need to put a ,, or a zero to make sure that the column headings match up if I create my own ( as my version will have some variables missing).

You can supply as many columns as you want, as long as you supply as a minimum those columns which existed in the early versions of Cumulus, about 15 or so. For the columns after the last figure that you actually supply, you can either leave them out or just put commas.

But you think that the records and extremes can be created just from the dayfile that I build?

Yes.

[muppet77]

Ah. My misunderstanding Steve. I forgot that the new column headers would be added to the right rather than randomly in the middle. Silly I know.

So if I create a dayfile then I drop it in the data folder, ans link up CU to my logger it should auto create everything I need from the past?

If so, wow. I will get to work on formatting my data.

[mcrossley]

I 'think' someone on the forum offers a data conversion 'service' to convert the WD log files directly into the Cumulus format files. They do this offer by pm, if you wait a little while, or rename the thread to something more obvious it may attract their attention.

[steve]

Ah. My misunderstanding Steve. I forgot that the new column headers would be added to the right rather than randomly in the middle. Silly I know.

Particularly as I said this in my second reply

So if I create a dayfile then I drop it in the data folder, ans link up CU to my logger it should auto create everything I need from the past?

It depends on what you mean by "auto" and "everything". It will create the highs and lows, but you will need to use the editors in Cumulus to make it do it - I assume you've looked at these and the associated help entries? You would still be missing the monthly log files, so wouldn't be able to use anything that relied on that data. But as Mark points out, it would be much easier if you could convert your data into monthly log files, and then Cumulus could create the dayfile.txt for you.

[muppet77]

thanks steve - i admit that i missed that! apologies.

the reson that i wanted to go straight in with making my own dayfile is that my WD log files are not always 100% and accurate whereas i know that my day by day stats that i have compiled in Excel are. the times for each variable are held in one row.

the data in each daily row that i have is
Date
av T
max T
Time
min T
Time
av wind
av dir
av bear
hi wind
av baro
rain
av dew pt
av RH
av cloud base m

i was planning to use this to build a dayfile and see how much i get back from the Cumulus software?
from what you say i will need more.

i suppose the links at the bottom of the Cumulus webpage can just be linked backed to my historical data that is hosted somewhere?

[steve]

the reson that i wanted to go straight in with making my own dayfile is that my WD log files are not always 100% and accurate whereas i know that my day by day stats that i have compiled in Excel are. the times for each variable are held in one row.

Yes, getting Cumulus to build dayfile.txt does rely on the actual highs and lows being present in the log files to get accurate results.

i was planning to use this to build a dayfile and see how much i get back from the Cumulus software?
from what you say i will need more.

You won't get very good results with what you've listed, I'm not sure there's any point in creating a dayfile.txt with just those, you may as well edit the highs and lows in Cumulus manually. Most of the ones you have don't have places in dayfile.txt anyway.

i suppose the links at the bottom of the Cumulus webpage can just be linked backed to my historical data that is hosted somewhere?

Yes, you can change the links as you wish, or modify the pages significantly, or provide your own pages from scratch.

[muppet77]

Thank you so much with your help Steve. I really appreciate it.
I want to get a plan before I dive into the water.

After guidance I think that I may just move forwards and not worry about converting everything. I am guessing that I can manually enter most of the monthly and yearly and all time records which then get automatically changed by the software if they are broken? Is that true?

[steve]

I am guessing that I can manually enter most of the monthly and yearly and all time records which then get automatically changed by the software if they are broken? Is that true?

Yes, that's right.