General questions about Cumulus MX software and DIY data

[newburyweather]

Hi,

I'm new to the site but not software/sensor interfaces etc..

A few (5-7)years ago I built a weather station based on a Modtronix board which had it's own web server. Because of family and other commitments I never really finished it to my satisfaction, although still does graphs reading etc.
http://www.newburyweather.org.uk:8888/ if you want to look. Note this is a 40MHz PIC18f running a TCP/IP stack and a web-server. It uses c to generate javascript to display data graphs.

So when the Raspberry pi came out I started a new one. This again was never finished because of other commitments (moving to a house that needed more work than envisaged). The major stumbling block was the performance of the Pi using mysql. It may have been the way the dbs were setup but after a year the table became very large and became excessively long so search and display data.

So I've bought a new 2+ Pi and am stating again.
So far I have written data collection daemons for onewire and a couple of serial devices which write to a SQLite3 database and I'm now looking at the database/website interface.

Previous to today I have looked at Cumulus and seen that it only ran on a PC and didn't want the carbon footprint of my PC being on all the time.

Today I see that Cumulus runs on linux which means that the carbon foot print can be low using a raspberrypi etc.

So intro over, the real question is can I use Cumulus software.
I am writing my own data collection software as a daemon and placing the data into a SQLite3 db. My intention was to use PHP to extract the data and display graphs using highcharts (which is how I did it with the raspberrypi previously) but the Cumulus software looks so much better and would save me a load of coding.

Currently I'm using onewire temp-sensors and some thermistors (+/-0.1degC), I have a AAG wind direction and anamometer and a couple of humidity sensors.

So the questions:
How can I integrate home built sensors and data tables into Cumulus software?
How does Cumulus collect data, maybe I can mod my software to simulate a currently used protocol?

Another thought is that if Steve of Cumulus fame had a standard i/f through serial or tcp the DIYers could use it.

I'm not after fully detailed answers but some indication of the possibilities.

Many thanks

Andy

[steve]

People who have done this have tended to do one of the following:

1. Generate a weatherlink.dat file - http://wiki.sandaysoft.com/a/EasyWeather_Format
2. Emulate a Davis weather station, as the protocol is made available by Davis on their web site (warning: there are errors in the protocol document)
3. Emulate a Fine Offset station, as there are basic protocol documents available, and the protocol is reasonably well understood in the community.

I think someone also emulated an Oregon Scientific weather station.

MX may have a 'universal' interface at some point in the future, but development is slow, so I don't know when this might be.

See the 'Weatherduino' as an example of system using the Davis protocol, and the 'Windblown' station using the Fine Offset protocol.

[newburyweather]

Many thanks for your reply Steve,

I will look into the possibilities you have suggested.
They show me that I can interface to Cumulus using my own data collection software. I now have to decide the most practical way?

I'll let you know how I progress.

Many thanks

Andy

[AllyCat]

Hi Andy,

If you may want to use "extra" sensors or perhaps even multiple instances of Cumulus (at the same time) then emulating (part of) the Davis serial protocol is probably the way to go.

But are you aware of the WeatherDuino project which already uses that approach?

Cheers, Alan.

[newburyweather]

Hi Alan,

But are you aware of the WeatherDuino project which already uses that approach?

I was not aware of many things I have found out in the last few hours. I need to read lots and see which way to go.

Simulating a protocol causes a couple of physical issues primarily in the length of USB cable between one data collection point and the Pi running Cumulus.
It may of course be possible to use loopback ( ie virtual port to virtual port but I'm not sure how this would work at the moment) at the Pi with Cumulus and collect the data from the remote Pi over wifi or CATV.

Now I have too many choices.

Much reading ahead.

Many thanks

Andy

[newburyweather]

Hi All,

[EDIT] I've stated a new thread because I have answered some of my own questions but some more have arisen

Hope you've all had a great Christmas. The geek in me has because I've successfully put some live data in to the EasyWeatherPlus.dat file and seen it displayed.

I have a few general and specific question which may be answered elsewhere so I apologise if they have, some I could probably answer myself after some trials;

What unit is the EWInterval in the Cumulus ini file?
Has anyone done a write up using on MX through apache2 so that I don't have to use port forwarding at the router. It's been a few years since I've set up a web server and I remember having problems in this area before.
I note that MYSQL and SQLite3 are mentioned in various files. How are these used? It doesn't seem to be a requirement for the installation.

Many thanks and happy Boxing Day.

Andy

[steve]

What unit is the EWInterval in the Cumulus ini file?

Seconds

Has anyone done a write up using on MX through apache2 so that I don't have to use port forwarding at the router

Not that I'm aware of. But I don't really understand the question. Are you talking about the simple web pages which MX generates for upload or copy to a web site, or the MX user interface? The former should work with any web server, and the latter can only use MX itself as the web server.
I note that MYSQL and SQLite3 are menti ... tallation.
MySQL updates are an option for those who want to use them. SQLite is currently used internally for an in-memory database, mainly for the 'recent data' web tags. I am currently looking at adding logging of data to an SQLite database, i.e. in parallel with the text-based log files.

[newburyweather]

But I don't really understand the question. Are you talking about the simple web pages which MX generates for upload or copy to a web site, or the MX user interface? The former should work with any web server, and the latter can only use MX itself as the web server.

After further reading I have found that my question was irrelevant because this has already been taken care of using ftp or file transfer to the web root pages.

I am currently looking at adding logging of data to an SQLite database, i.e. in parallel with the text-based log files.

This could be the start of a universal interface. If the user could write to the db files then Cumulus could read and display the data.

Many thanks

Andy