Welcome to the new home of the Cumulus Support forum.

Latest Cumulus release v1.9.4 (build 1099) - Nov 28 2014
Latest Cumulus MX release - v3.0.0 build 3044 7 December 2018. See the Wiki for download

Future data formats

Discussion and questions about Cumulus weather station software version 1. This section and its subforums are the main place to get help with Cumulus. Anything which is not specific to the type of weather station goes in here; for anything specific to a type of weather station, please use the appropriate subforum. Use the 'website development' section for any questions relating to creating or running a web site for Cumulus data. Discussion of the stations themselves in these sections is fine.

How should a future version of Cumulus store its data?

In a local SQL database
16
11%
In 'flat' text files like the current version
39
26%
In both simultaneously
93
63%
 
Total votes: 148

User avatar
steve
Cumulus Author
Posts: 26714
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Future data formats

Post by steve » Wed 12 Sep 2012 9:13 am

Purely hypothetical...
Steve

User avatar
mcrossley
Posts: 5237
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2
Operating System: Stretch Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: Future data formats

Post by mcrossley » Wed 12 Sep 2012 10:01 am

None of the above! Maybe :?

I am happy with flat files as you can import them into SQL fairly easily, but if you offer SQL as an option, then Cumulus should be able to use either a local OR remote database - maybe using a flat file as well, then it can buffer data if the remote db is unavailable ;)

User avatar
steve
Cumulus Author
Posts: 26714
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve » Wed 12 Sep 2012 10:24 am

I considered a 'none of the above' option, but didn't want it getting out of hand. I was thinking that if it did have local SQL support, then there might be an option to maintain a remote database as well. But if that remote database was MySQL, we get into expensive licensing issues, unless the client was open source.
Steve

riordankj
Posts: 4
Joined: Sat 21 Jul 2012 3:06 am
Weather Station: Vantage Vue
Operating System: Windows 7-64bit
Location: Rockford, Illinois USA

Re: Future data formats

Post by riordankj » Thu 13 Sep 2012 12:55 am

I was very impressed with the screen shots of the now defunct version 2. I could never get it to run on my system so I couldn't see it real time. I understand that used SQL.

Spider-Vice
Posts: 207
Joined: Sat 24 Sep 2011 2:46 pm
Weather Station: PCE FWS-20 (Fine Offset)
Operating System: Windows 10

Re: Future data formats

Post by Spider-Vice » Thu 13 Sep 2012 8:32 pm

I'm blind when it comes to databases and benefits so I think the plain data is fine, but well, if you could maintain the database remotely, you could always suffer a hardware failure and keep your data safe.

RayProudfoot
Posts: 2633
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: Future data formats

Post by RayProudfoot » Thu 13 Sep 2012 9:36 pm

I voted for both options as eventually I would like to store data in a database for the power it gives the user. But those who don't wish to go down that route should not need to change hence the option to retain the existing system.
Cheers,
Ray, Cheshire.

Image

cats_five
Posts: 55
Joined: Sat 26 Dec 2009 4:24 pm
Weather Station: WH1081PC
Operating System: Win 7 x64 Pro
Location: N 56° 17' 29", W 3° 13' 39"

Re: Future data formats

Post by cats_five » Sun 21 Oct 2012 9:00 am

What about SQLite?

http://www.sqlite.org/
Pegase 101a
Pentax K7
Weather Station: Not currently available
Photography: http://www.fifescapes.co.uk
Cats: http://shunracats.wordpress.com

User avatar
steve
Cumulus Author
Posts: 26714
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve » Sun 21 Oct 2012 9:30 am

SQLite is what I would use for the SQL database, yes, as I did in Cumulus 2.
Steve

cats_five
Posts: 55
Joined: Sat 26 Dec 2009 4:24 pm
Weather Station: WH1081PC
Operating System: Win 7 x64 Pro
Location: N 56° 17' 29", W 3° 13' 39"

Re: Future data formats

Post by cats_five » Sun 21 Oct 2012 11:00 am

I think ideally we would have a configuration choice of flat files or a database. Those of us that are sophisticated to want a database should also be able to extract the data from it as we require. However not sure if it would be easier from your PoV to do either flat files, or flat files plus a database so you only need one mechanism for generating the output from the saved data.
Pegase 101a
Pentax K7
Weather Station: Not currently available
Photography: http://www.fifescapes.co.uk
Cats: http://shunracats.wordpress.com

atb
Posts: 10
Joined: Sun 03 Jan 2010 1:58 pm
Weather Station: Fine Offset WH1080
Operating System: XP SP3 & Raspberry Pi
Location: Bredfield, Woodbridge, UK
Contact:

Re: Future data formats

Post by atb » Sun 21 Oct 2012 11:33 am

I can see the potential benefit in storing in SQL, however the simple text also provides useful access at times, even if via a spreadsheet.

My main concern in opting for both is whether that would place higher demand on the system, unless there could be an option to disable the SQL. My current setup relies on an old laptop running 24/7 as a dedicated "server", also servicing a USB Webcam (using Yawcam) that uploads via FTP every 10 minutes.

This old laptop usefully takes little power (15 watts on power meter, if I recall correctly), is 597MHZ Celeron with 512MB (max allowed!) running WinXPSP3 Pro (local maintenance via Remote Desktop). TaskManager indicates about 50 - 70% CPU load rising to 100% briefly when Cumulus or Webcam uploads (I try to set them interleaved!) and less than 300MB PF usage.
So, would a SQL or even both version require greater CPU or RAM?

User avatar
steve
Cumulus Author
Posts: 26714
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve » Sun 21 Oct 2012 12:27 pm

My current thinking is that a future version would continue to use the existing file format and may offer an option to also add data to a local SQL database.
Steve

OliII
Posts: 54
Joined: Mon 19 Mar 2012 11:47 am
Weather Station: WMR 200 by Oregon Scientific
Operating System: WinXP SP3
Location: Belgium

Re: Future data formats

Post by OliII » Sat 03 Nov 2012 9:37 pm

I voted for both options.
Image
http://www.7331.be/meteo website based on the phpBB3 forum and template. WMR200 + UVN 800 - Cumulus 1.9.4 1062 / Windows XP Service Pack 3 build 2600 FR

uncle_bob
Posts: 476
Joined: Wed 17 Aug 2011 2:58 pm
Weather Station: WeatherDuino Pro2
Operating System: 2008
Location: Canberra

Re: Future data formats

Post by uncle_bob » Sat 03 Nov 2012 10:49 pm

I'm on the fence :)
Whatever option that could be taken would need a simple way of editing bad data (spikes).
It would also be great if Cumulus was OS independent so it could run on different platforms.
Interested in building your own Weather Station? Maybe check out the WeatherDuino Pro Project Here
Conder, Canberra Weather
Image

User avatar
steve
Cumulus Author
Posts: 26714
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Future data formats

Post by steve » Sun 04 Nov 2012 11:27 am

uncle_bob wrote:It would also be great if Cumulus was OS independent so it could run on different platforms.
I'm working on it ;)
Steve

dale
Posts: 8
Joined: Tue 29 May 2012 9:15 pm
Weather Station: WMR968
Operating System: Windows 7
Location: Glen Arbor, MI

Re: Future data formats

Post by dale » Fri 16 Nov 2012 2:10 pm

I probably would not use SQL so voted for flat file, but if SQL was there I'd really like the flat file to remain and option.

Post Reply