Cumulus Support

There are a couple solutions to make a program that is designed to run at the console (on your screen) as a service.

For the non-geeks: Cumulus is a program that will run interactively, on your Windows box, if you logon and start it. A Windows Service can be configured to run whenever the PC is powered on, even when you don't log on to it. Programs that run as a service do not need any user interaction to do their work. The concept of a service is ideal for software that needs to constantly be active, for instance to monitor conditions and act upon them. a Cumulus service would monitor your weather station's input, and update a data repository based on that input.

A service will (if so configured) be started by the operating system as soon as the operating system starts, even when you don't log on to the computer. It will create its own user context (something like "log on itself"), either under a System context (which gives it unlimited access to the system), or under a specific user context.

To be able to do this, the program must be specifically written as a Windows Service. There are, however, a couple ways around this, such as srvAny (an MS utility), or ServiceEx (http://serviceex.com/). This would enable you to run Cumulus as a service.
Again: the main advantage of this would be that, if your computer restarts for whatever reason (power failure and subsequent recovery, system crash and reboot), Cumulus restarts too, without you having to log on and start it.

There is, however, a huge disadvantage to this: if you do this, you loose the Cumulus user interface. You cannot start Cumulus twice; it would create mayhem, because there would be two instances of Cumulus competing for the same weather station input. I have not tried this, but I can foresee chaos.

What I am tempted to do is:
- write a service that monitors the Cumulus data store, and stores all values in a SQL database (I could currently support MS SQL, Oracle, and MySql, but SqLite seems to be possible as well).
- write a front end that accesses the database to replicate the functionality of the Cumulus user interface.

Why do I want to do this?
1. well, first of all, because. Just because.
2. because it allows me to run Cumulus as a service, thus enabling continuous data collection, even when my machine restarts while I am on vacation. It would be rather inconvenient to have to return from, say, Loch Tay, to Cothen because my server has rebooted.
3. because I can (and secretly because I believe I am actually effin' good at this, but I cannot say things like these aloud)
and...

4. because, even now with 4 grandchildren, I am a kid at heart. I want to play, and to me, this constitutes playing. This is actually the real explanation of reasons 1 and 3.

If there is anyone who can give me a fifth reason, like "oh yes I absolutely need this feature but I *hate* programming so I am not gonna do it myself", please chime in. I might just gather enough motivation to actually *do* it rather than think about the possibilities.

Having said (and whispered) quite ambitious things, I have to add that I am not very well-versed at writing UI's. I can do the front end that extracts data from a data repository, but UI's ... let's just say I don't do UI's professionally. I am purely a back end guy. So if I have to come up with a user interface, it may well be crappy.

I'd use it Pete

Fifth reason?
So you can run the 'front end' console on a different PC from the one running Cumulus-as-a-service.
Now that would be really useful.

I suspect that people who liked Cumulus 2 would like this (apart from, perhaps, those who liked Cumulus 2 purely because of the 'bling') because this proposal offers a similar architecture.

Steve, you are *not* doing yourself propâh justice. Cumulus 2 would've been a lot better.

Compared to Cumulus 2, the solution I am thinking of, while offering similar functionality... is still a horrible kludge. Cumulus 2 would have been a *lot* more elegant from an architectural point of view.

BCJKiwi wrote:Fifth reason?
So you can run the 'front end' console on a different PC from the one running Cumulus-as-a-service.
Now that would be really useful.

That goes without saying. I'd be really (and I mean REALLY) stupid if this would not be possible.

Well,
So sorry for responding to your request!

It may be obvious to you and its great that it is (even though it was not mentioned anywhere in your treatise) but there are many examples of programs (many of which are commercial) where local only access is all that is provided despite using a database backend.

No prob. It wasn't mentioned anywhere in my rant because I simply hadn't considered the idea that it would NOT be possible.
The omission is entirely on my part - I admit that I am not always consciously aware that whatever is obvious to be is also obvious to others. Hence my reaction that it went without saying.

So thanks for pointing that out.

Time permitting (hospitalized daughter yesterday, expected visitors today), I'll draft up an object model and see if I can understand the why's of Steve's repository. I'm currently thinking some of the files are there for performance reasons (such as being able to get things like yesterday's extremes, this months extremes, or all time records, quickly rather than having to wade through the entire data set), and I may not have to bother with all the files.

cheers,
peter

I've started mapping out a data model. It will probably look like this: separate tables for CurrentConditions, ConditionsToday (daily summaries, which will include the running day), MonthlyConditions (which will include the running month), YearlyConditions (which -- you saw this coming -- will include the current year), and AllTimeRecords.

I'm currently working on the CurrentConditions model, which will look somewhat like Realtime.txt - and actually, that is where we will be getting our data from.

I have a few questions for the True Cumulus Experts here -- which, compared to me, probably includes 95% of this forum.

First question: in my Cumulus main window, there is a section 'Solar', with a field called 'Light', which presents the light value in lux. I cannae find thes! in the Realtime.txt file definition on the Wiki. Is that correct?
If it is, is this calculated?

Second question: I cannot find a value for Average Temperature (Avg Temp in the Outdoor section) in the wiki. Is that correct, or am I myopic?
If I am not proven myopic and this is indeed not there, is it calculated? If so, how? I seem to remember reading that this is calculated by taking the average of each 10 minute period and averaging that.... is that true?

So far for me being stupid (I'm sure this will happen more often in the weeks to come).

I also have a couple considerations that I'd like to get opinions on.

<humour>
If anything in the text below makes you think I'm a nerd, you're definitely right. If anything in the text below makes you think you're stupid, you are almost certainly wrong. This is nerd stuff. It needs to be dealt with. By nerds, so that you normal people can go on living your normal lives while we do nerdy things in secluded rooms, feeding on yesterday's pizza. Yes, we will slave for the benefit of your well-being.
</humour>

<nerd stuff>
Time considerations
I would like to populate each record with the date and time (duh). But because dates and times are screwed up because of our planet being a globe, I'd like to store the UTC DateTime as well as the Local DateTime.
I am thinking that the UTC DateTime could be useful for making global comparisons at a given point in time. However, for useful interpretation of data, the local time may be way more important.
I cannot at this point formulate a valid use case for storing both... but experience in developing across-the-globe solutions tells me that I will probably regret it at some stage if I don't store both timepoints.
Can anyone think of a valid usage scenario for storing UTC datetime to support my case... or of a reason to NOT do this?
</nerd stuff>

Unit considerations
I am strongly veering to storing all data in one unit system... being the unit system that most meteorological organisations default to (and a few meteorogical organiZations won't want to deal with, unfortunately). That would mean temperature data in degrees Celcius (degrees Centigrade if you live across the Atlantic), pressure data in hPa, rainfall data in mm's, wind speed data in m/sec.
That doesn't mean that you have to LOOK at the data in the unit system that I choose, or even that your realtime.txt has to present the data in these units... the realtime.txt file reports the units used by your Cumulus configuration, and I can calculate these to the unit system that I want to use.

The advantage of this would be that the data is portable between systems, and that the data repository (the database) is not broken when you decide to change any of the units later.

The disadvantage of this is that if you live in a country that fights the SI unit system with a vengeance (the USA comes to mind) the data in the database is not in a unit system that you are familiar with. Again, when the data is presented to you, we could calculate the units in the database to the units of your preference.

This is not really a question, actually. My gut feeling is that it would be a very good idea to stick to the internationally recognised system (or recognized system, depending on your location) would actually be a Good Idea. However, I'm still interested in differing opinions, and debate, on this.

That's it for today... my alarm clock goes off 7 hours from now.

cheers,
peter

You will find that Cumulus does no write everything 'current' to the realtime file. In order to capture all the data I think you will have to construct your own web tags file. Or use something like the PHP web tags file and extract what you need from that.

Re the dates;
Cumulus stores every date in the local PC Windows system date/time format (except LastRainTipISO).
So if you don't use the same date that Cumulus does you will have to convert everthing to UTC then back again.

As Mark has indicated, if you restrict yourself to realtime.txt then the system will be very limited. There are over 600 variables available via the webtags.
Before you get in too deep with your data formats it might be useful to look at what others have already done, e.g. Saratoga templates via the CUtgs.php/CU-defs.php which converts/massages the raw tags from Cumulus where required and uses others as-is.
There are a number of existing realtime and tags -> sql conversion routines already in existence.
DAJ, Mark (for steelseries gauges and highcharts) and a few others I will have missed!

I decided that storing the data in 'fixed' units and using UTC was the way to go when I started from scratch with Cumulus 2, so that's what Cumulus 2 does/did. It's easy enough to convert to/from UTC and local time; as you no doubt know, .NET has routines built in.

There's one issue (at least) with using UTC, and that's Daylight Savings Time. If the switch over times for a particular region have changed at any time, then it becomes difficult to do the conversion historically. You can get a library which contains all of the switch over times that have ever been (and I don't think it's huge), or maybe better would be a flag to say that DST is currently in use.

Regarding realtime.txt: I stopped adding things to that some time ago, as it was always intended to be a small file (it was initially just for feeding a couple of gauges), and adding lots of extra items to that penalises people (in terms of time and bandwidth) who only use a few items.

Thanks guys!

BCJ: conversion from UTC to local is indeed a trivial matter - 8 keystrokes in code (a bit more if you mistype .ToLocal )
Plus I store the local time as well, so no worries there. If anyone wants to extract the time without the help of DateTime conversion libraries, they would just query my DateTimeLocal field.

Steve: On top of flexibility (if someone would want to query the database without the help of a conversion library), it is your reservation about DST that leads me to want to store the local time as well. I know it's an extra field, but hey... 8 extra bytes is not going to bite me.

So... I guess webtags is the way to go? This morning, before I left for the office, I took a brief look at XML Tags. Together with the XmlSerializer class, that would be terribly convenient... as in, almost too easy!

Peter,
I thought you were intending to use the existing Cumulus data;

What I am tempted to do is:
- write a service that monitors the Cumulus data store, and stores all values in a SQL database (I could currently support MS SQL, Oracle, and MySql, but SqLite seems to be possible as well).
- write a front end that accesses the database to replicate the functionality of the Cumulus user interface.

If so you will have to determine and deal with each local time configuration in order to get it into UTC in the first place.
Presumably also equally trivial so no problem?