Page 1 of 1

Viewing reports, ini's &text files

Posted: Tue 09 Apr 2013 12:00 pm
by Touchtone
What is the best program to use to view information from Cumulus & to be able to print it. I would really like to be able to create a condensed report covering all the days highs & lows & also to be able to do the same for each month.
I have just started using Cumulus with my Davis Vantage Pro2 Plus weather station. I really like the console & various details that are available from the drop down menus. If I copy some of the information to the clipboard & I then find the clipboard only has a series of number & not the information that was on the console menu.

Re: Viewing reports, ini's &text files

Posted: Tue 09 Apr 2013 12:33 pm
by steve
You can view most if not all of the information from within Cumulus. The dayfile.txt editor will show you the daily highs and lows, and there are various monthly/yearly etc summary screens. The 'this period' display can be set to a single day to get a basic summary for that day. The monthly/yearly summary screens should give a reasonably well formatted result when the 'copy to clipboard' button is used - they seem to for me at least. The 'copy to clipboard' menu option on the main screen also gives me a reasonably well formatted result. Perhaps it depends on what you use to view the result; I use notepad++.

If you could give an example of where 'copy to clipboard' isn't producing a reasonable result, I could investigate.

Admittedly, the displays available are rather basic; for other analysis etc, the data files are in CSV format so can be imported into a spreadsheet such as Excel. And some people have produced some quite nice web-based data summary/comparison displays.

Re: Viewing reports, ini's &text files

Posted: Tue 09 Apr 2013 8:22 pm
by Touchtone
Thanks Steve. I will look for Notepad ++ on the web. I only have Notepad that is supplied with Windows 7 Home Premium. I don't have Excel but I do have Open Office which has a spreadsheet almost the same as Excel.
When I copy the information to Notepad & then open Notepad I get a row of numbers & no text information that could be re-formatted. In other cases there are rows of information down the left hand edge of the page & although it is detailed with numbers & text in each row, but while some of it can be read there is a lot that doesn't make sense. There are no columns at all.
Best regards.
Touchtone.

Re: Viewing reports, ini's &text files

Posted: Tue 09 Apr 2013 8:36 pm
by steve
I haven't used any strange formatting, just spaces, so I don't understand why it's not working for you. Here's the output of 'this month' pasted into notepad:
2013-04-09 21_33_34-Untitled - Notepad.png

Re: Viewing reports, ini's &text files

Posted: Tue 09 Apr 2013 10:41 pm
by Touchtone
Hi Steve. I have now discovered that the "copy to clipboard" is in rich text format & I have to check on the desktop for new text document, by right clicking mouse to find "new text document. 2 versions are there But only the RTF one seems to work. Below are the results. Thanks for your help.

Current data for Bligh's Weather Station
Wednesday, 10 April 2013 8:29:19 AM

Temperature (°C):
Current 19.8
Trend (per hour) +0.4
Average today 18.8
Wind chill 19.8
Heat Index 19.8
Dew Point 16.7
Rel Humidity 82%

Wind (km/h):
Current Gust 14.5 ESE
Average Speed 4.8 SSE

Rainfall (mm):
Current rate 0.0
Last hour 0.0
Total today 0.0
Total yesterday 1.4
Total this month 1.4
Total this year 1072.2

Pressure (hPa):
Current 1018.56
Trend (per hour) +0.24

Re: Viewing reports, ini's &text files

Posted: Wed 10 Apr 2013 6:50 am
by steve
I think it's something to do with what you're doing with the clipboard copy; Cumulus isn't using RTF. This is what I do: Having clicked 'copy to clipboard', start notepad (or whatever) so a new blank document is displayed, or open an existing one that you want the information to appear in. Then either do CTRL+V or select 'Paste' from the Edit menu.

Re: Viewing reports, ini's &text files

Posted: Wed 10 Apr 2013 9:27 pm
by sfws
steve wrote:I think it's something to do with what you're doing with the clipboard copy; Cumulus isn't using RTF. This is what I do: Having clicked 'copy to clipboard', start notepad (or whatever) so a new blank document is displayed, or open an existing one that you want the information to appear in. Then either do CTRL+V or select 'Paste' from the Edit menu.
Touchtone wrote:I do have Open Office which has a spreadsheet almost the same as Excel.
The 'Copy to Clipboard' can be easily seen (just as Steve Loft describes) if you use the "writer" component of Apache Open Office (or Libre Office) where Steve says 'notepad (or whatever)' - all you need to do is click 'Paste' icon or select from menu as he says. Once in Writer, you can 'Convert text to table' using 3 (you might need to experiment on this number) spaces as column splitter. You may need to manually edit table to get it perfect. Once arranged as a table you can easily paste into 'Calc' if you want a spreadsheet layout.

I took output of a Cumulus view 'This Period' using Copy to clipboard that I pasted into Libre Office Writer and turned it into table layout on a HTML web page to display statistics for my imported observations prior to my first installation of Cumulus having populated some monthly logs for the earlier period. But I did need to learn some HTML syntax first!
(Edits - correcting my grammar and spelling errors!)

Re: Viewing reports, ini's &text files

Posted: Wed 10 Apr 2013 10:50 pm
by Touchtone
Thanks for the replies & the help you have been. I will try all these options & hopefully get an easy solution working.

Re: Viewing reports, ini's &text files

Posted: Fri 12 Apr 2013 1:13 pm
by JimDantin
The format of the data that is 'copied to clipboard' is a bit difficult to work with if you are attempting to extract the 'data values' from the 'titles'. In the example shown, most of the data is aligned rather nicely, with the data values starting at the same character number in each line. In Excel (or other spreadsheets) you could parse the data at column 17 and isolate the titles from the data values.

However, when you get down to "Days with snow falling", you have a problem, since the title is more than 17 characters.

What to do?

It's Notepad++ to the rescue! (other advanced text editors can also be used, of course)

If you aren't a user of Notepad++, and you need complex text manipulations, you are missing out on a great tool. The magic here is using a "Regular expression" to help with the data manipulation. I'll show you how to replace any occurrence of two or more spaces with a tab. That will effectively turn the clipboard text into a series of data records with the data title, a tab, and then the data itself. Once you get to that point, you can do further cleanup or feed the data to some other program.

In Notepad++, let's set one option to help visualize the data. Select View...Show Symbol...Show All Characters. This will display tabs and end of line characters.

In Cumulus, select View and the data you are interested in, then "Copy to clipboard". Open Notepad++ and then paste. (use Edit...Paste, or Ctrl+V). You should see the data as shown above, with the addition of the special <CR><LF> characters.

Now open the Replace window (Search...Replace..., or Ctrl+H). We're going to search for all the instances of two or more spaces, and replace each instance with a single Tab character. You'll have to do some independent study on "Regular expressions" or "Regex" to learn what's possible here - trust me, it's a whole universe of possibilities, and Google is your friend!

In the Replace window, select "Regular expression" in the "Search Mode" section. Then enter the following in the "Find what:" box --
([\s]+)([\s]+)
In the "Replace with:" box, enter
\t

Now, click the "Replace All" button and you'll see the multiple spaces replaced with a yellow "-->" tab character.

I'll leave further data manipulations up to your imagination and needs. You might, perhaps, want to isolate the dates from the various min/max values. You can even create and save complex edit sequences as macros for convenient reuse. Have fun.

Re: Viewing reports, ini's &text files

Posted: Sat 13 Apr 2013 10:04 pm
by Touchtone
Thanks Jim very much for the information using Notepad++. I downloaded Notepad++ from the web & I followed your instructions & the vertical text became a horizontal line similar to a spreadsheet. Using formulas & macro's is certainly not a strong point with me. I have searched the web under "regex" as you suggested but I really don't know what I should be looking for.
What I would like to be able to do is convert vertical text files in to data similar to a spread sheet that has all the headings across the top & the numerical information underneath in columns.
Thanks again & best regards.
Bligh.

Re: Viewing reports, ini's &text files

Posted: Sun 14 Apr 2013 8:16 am
by steve
Have you looked at the 'NOAA-style' reports? Again, they are also just 'plain text', but offer an alternative way of viewing monthly/annual summaries, and they contain some information that Cumulus doesn't currently supply by any other route.

Re: Viewing reports, ini's &text files

Posted: Sun 14 Apr 2013 9:52 am
by Touchtone
Thanks Steve. Yes I have viewed the NOAA reports & they certainly do contain extra information which is very useful, I will keep working away at getting more proficient with Notepad++ & see how I go.

Re: Viewing reports, ini's &text files

Posted: Sun 14 Apr 2013 1:00 pm
by JimDantin
Touchtone -- can you give us a big-picture idea of what you are trying to do? There are probably a half-dozen ways of doing anything like this, and the key is understanding exactly what you need as a final solution. It sounds like you want a way to capture the summary info and save it for further analysis - if this is just an occasional need, then manual manipulations with Notepad++, Excel, or whatever, could be sufficient.

If, however, you're looking at an ongoing or frequent task, then we might want to figure out a more automated process. My suggestions with regex functions was only one way to accomplish a slightly complicated manual edit. The Notepad++ macros provide a way to save and reuse multi-step edit processes. To get additional sets of data without the text labels, you can use other regex functions in Notepad++ to delete everything before the first tab. Transposing the columns and rows is another task - an Excel command comes to mind as an easy solution (paste special...transpose).

So for my assumed task of "occasionally grab the summary data and add it as a row in an excel spreadsheet" you could do the following after creating your Excel data sheet and copying the data from Cumulus.

Open Notepad++ and then:

1. Paste the data you just copied.
2. Use a regex expression to find the first occurrence of more than one space on every row.
3. Delete everything at the start of the line (including the multiple spaces).
4. Delete any blank lines
5. Copy the remaining column of data

Then, in Excel, paste special, transpose -- and place the data on a new row under any existing data.

All of this could be handled by creating a macro in Notepad++ that did the steps 1-5.

FYI - this is fun stuff for me. I do database development and consulting as a profession. Data manipulation is a common task. I'm basically lazy, so I always look for ways to automate tedious tasks! :mrgreen:

Re: Viewing reports, ini's &text files

Posted: Sun 14 Apr 2013 1:33 pm
by JimDantin
FYI -- if you used my first example to replace multiple spaces with tabs, this will then delete everything at the start of the line including the first tab:

Find what: ^[^\t]+\t(.+)
Replace with: \1

You could probably do it all in one step, but I haven't figured out the proper regex expression yet. Please note that I am NOT a regex expert - I just use Google a lot until I find an appropriate example that I can modify. I've also learned just enough regex syntax to understand what I am looking at. Like I said -- I'm lazy!

Depending on if you are looking at monthly or yearly data in Cumulus, you could also split the various "day 03" into additional lines and delete the "day " label. You could also delete the title lines like "Temperature (°F):" so that you end up with a simple column of numbers that matches your pre-determined Excel columns.

Re: Viewing reports, ini's &text files

Posted: Sun 14 Apr 2013 9:07 pm
by Touchtone
Thanks Jim. You have been very helpful. I really have not decided what I really want to do with the Cumulus data as yet. I don't really want to be able to print out detailed daily records. So I am tossing up whether to have a hand written diary of the daily information, which I used to with my old Oregon Scientific WMR 928. The advantage with this system is that it can be contained in one book covering perhaps a year or more & it is very easy to find any particular day etc. & it is compact.
Cumulus doesn't show UV & Solar in the daily reports other than the dayfile.txt which is a very big file. So to get each day I can refer to this file & manually write them down or refer to the Davis console daily high & low details & get them from there.
Thanks again. I will keep you posted on what I eventually end up doing.
Best regards. Bligh.