Page 1 of 4

Future data formats

Posted: Wed 12 Sep 2012 9:13 am
by steve
Purely hypothetical...

Re: Future data formats

Posted: Wed 12 Sep 2012 10:01 am
by mcrossley
None of the above! Maybe :?

I am happy with flat files as you can import them into SQL fairly easily, but if you offer SQL as an option, then Cumulus should be able to use either a local OR remote database - maybe using a flat file as well, then it can buffer data if the remote db is unavailable ;)

Re: Future data formats

Posted: Wed 12 Sep 2012 10:24 am
by steve
I considered a 'none of the above' option, but didn't want it getting out of hand. I was thinking that if it did have local SQL support, then there might be an option to maintain a remote database as well. But if that remote database was MySQL, we get into expensive licensing issues, unless the client was open source.

Re: Future data formats

Posted: Thu 13 Sep 2012 12:55 am
by riordankj
I was very impressed with the screen shots of the now defunct version 2. I could never get it to run on my system so I couldn't see it real time. I understand that used SQL.

Re: Future data formats

Posted: Thu 13 Sep 2012 8:32 pm
by Spider-Vice
I'm blind when it comes to databases and benefits so I think the plain data is fine, but well, if you could maintain the database remotely, you could always suffer a hardware failure and keep your data safe.

Re: Future data formats

Posted: Thu 13 Sep 2012 9:36 pm
by RayProudfoot
I voted for both options as eventually I would like to store data in a database for the power it gives the user. But those who don't wish to go down that route should not need to change hence the option to retain the existing system.

Re: Future data formats

Posted: Sun 21 Oct 2012 9:00 am
by cats_five
What about SQLite?

http://www.sqlite.org/

Re: Future data formats

Posted: Sun 21 Oct 2012 9:30 am
by steve
SQLite is what I would use for the SQL database, yes, as I did in Cumulus 2.

Re: Future data formats

Posted: Sun 21 Oct 2012 11:00 am
by cats_five
I think ideally we would have a configuration choice of flat files or a database. Those of us that are sophisticated to want a database should also be able to extract the data from it as we require. However not sure if it would be easier from your PoV to do either flat files, or flat files plus a database so you only need one mechanism for generating the output from the saved data.

Re: Future data formats

Posted: Sun 21 Oct 2012 11:33 am
by atb
I can see the potential benefit in storing in SQL, however the simple text also provides useful access at times, even if via a spreadsheet.

My main concern in opting for both is whether that would place higher demand on the system, unless there could be an option to disable the SQL. My current setup relies on an old laptop running 24/7 as a dedicated "server", also servicing a USB Webcam (using Yawcam) that uploads via FTP every 10 minutes.

This old laptop usefully takes little power (15 watts on power meter, if I recall correctly), is 597MHZ Celeron with 512MB (max allowed!) running WinXPSP3 Pro (local maintenance via Remote Desktop). TaskManager indicates about 50 - 70% CPU load rising to 100% briefly when Cumulus or Webcam uploads (I try to set them interleaved!) and less than 300MB PF usage.
So, would a SQL or even both version require greater CPU or RAM?

Re: Future data formats

Posted: Sun 21 Oct 2012 12:27 pm
by steve
My current thinking is that a future version would continue to use the existing file format and may offer an option to also add data to a local SQL database.

Re: Future data formats

Posted: Sat 03 Nov 2012 9:37 pm
by OliII
I voted for both options.

Re: Future data formats

Posted: Sat 03 Nov 2012 10:49 pm
by uncle_bob
I'm on the fence :)
Whatever option that could be taken would need a simple way of editing bad data (spikes).
It would also be great if Cumulus was OS independent so it could run on different platforms.

Re: Future data formats

Posted: Sun 04 Nov 2012 11:27 am
by steve
uncle_bob wrote:It would also be great if Cumulus was OS independent so it could run on different platforms.
I'm working on it ;)

Re: Future data formats

Posted: Fri 16 Nov 2012 2:10 pm
by dale
I probably would not use SQL so voted for flat file, but if SQL was there I'd really like the flat file to remain and option.