Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

Cumulus - manually edit faulty data?

Discussion specific to Fine Offset and similar rebadged weather stations
IslandBoy77
Posts: 8
Joined: Wed 04 Apr 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Cumulus - manually edit faulty data?

Post by IslandBoy77 »

Hi
I've been using the Cumulus software for my WS1081 for most of this year. I've found that either due to a program glitch or something to do with my weather station, Cumulus will periodically record wildly inaccurate data. A few minutes ago Cumulus showed an alert and new record, claiming the temp was -10. It would be nice if it were that cold (as it would mean it was snowing, which it does only once every 20 or so years here), but since it's 16 degrees I'm sure that reading is wrong. I also note the the pressure reading last night showed a sudden 0 rating, then back to normal.

Anyway, it's not that these glitches happen often, I just want to know how to get into the data file to manually edit the values to something approximating normal.

Advice please?

Thanks
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Post by steve »

The data files are all in the Cumulus data folder, and the formats are described in the help and in the wiki. It's best to stop Cumulus before editing the files. There's a built-in editor for dayfile.txt.

You can use the calibration screen in Cumulus to set some 'spike removal' settings so that Cumulus can attempt to filter out any future strange readings from the station.
Steve
IslandBoy77
Posts: 8
Joined: Wed 04 Apr 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Re: Cumulus - manually edit faulty data?

Post by IslandBoy77 »

Thanks for your reply Steve. I opened the monthly file into Excel, but I couldn't see the values for either the wrong pressure or the wrong temperature reflected in the data - it looks like only the hi's & low's file picked it up. What's the easiest way to reset the hi's & lows to their previous values? Thanks.
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Post by steve »

Have you checked dayfile.txt? They are likely to be in there even if they aren't in the monthly log files. Once you've corrected dayfile.txt, you can use the built in record editor(s) to scan dayfile.txt and rebuild the records from that.
Steve
IslandBoy77
Posts: 8
Joined: Wed 04 Apr 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Re: Cumulus - manually edit faulty data?

Post by IslandBoy77 »

steve wrote:Have you checked dayfile.txt? They are likely to be in there even if they aren't in the monthly log files. Once you've corrected dayfile.txt, you can use the built in record editor(s) to scan dayfile.txt and rebuild the records from that.
Yes. I found the temperature discrepancy in the dayfile & made an arbitrary correction to approx what it should have been. The odd thing is that the Sep12log file, when opened in Excel (comma deliminated) doesn't show either the temperature aberration nor the 0 pressure reading late on the 14th. The dayfile doesn't show the pressure reading either, but using select a graph does. All very strange.
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Post by steve »

Select-A-Graph gets its data from the monthly log files.
Steve
rwilliam99
Posts: 42
Joined: Sun 01 Apr 2012 2:55 pm
Weather Station: WS-2080
Operating System: Windows 7
Location: Oregon City, OR

Re: Cumulus - manually edit faulty data?

Post by rwilliam99 »

I thought I would post my question here since it is related to editing faulty data. After my experiment with using EasyWeather instead of Cumulus to talk to the console, my rain totals are completely messed up. I've gone back through the data files and figured out the correct totals, however nothing I have tried has made it stick. I've edited all of the *.ini files where the monthly total was, and no matter what I do, as soon as Cumulus starts up, it changes it back to the value in error. I have not fixed the actual data files (the data was getting duplicated over and over in the EasyWeather.dat file it appears), but I did sort through it to figure out what the totals should be. So I have accurate totals for November and also accurate totals for December to date.

What do I need to do in order to fix this permanently? I hope I don't need to go back and fix the monthly .txt files. I'm hoping I can override the monthly and all time totals somewhere I'm not seeing it. I've tried editing the files manually, and also doing it through the Cumulus Toolbox editor and also using the editor within Cumulus (Edit/All time records, Monthly records, This month's records, This year's records). They all did the same thing. I made sure that Cumulus was not running when I edited them.

Any suggestions on how to fix these bad values for the high values and also fix the value for December?

Thanks

Rob
Matt.j5b
Posts: 512
Joined: Mon 28 Nov 2011 2:13 am
Weather Station: Davis VP2/ WLL with DFARS
Operating System: RPi Raspbian (Buster)
Location: Ferny Grove, Brisbane, Australia
Contact:

Re: Cumulus - manually edit faulty data?

Post by Matt.j5b »

To fix the monthly totals I think you need to edit the daily totals in the dayfile and then restart Cumulus. To edit the all time, this month, this year and monthly records either correct the log file for the affected month/s or the dayfile and use the editors in the edit menu to fetch the correct reading for the records or insert the correct figure manually into the editor. Editing the ini files for the records such as month.ini and year.ini I think is the same as using the editor. If this broke the all time records you can look at the alltimelog in the data folder to see what the previous record was.
Regards, Matt of Brisbane, Australia
Ferny Grove Weather
Image
rwilliam99
Posts: 42
Joined: Sun 01 Apr 2012 2:55 pm
Weather Station: WS-2080
Operating System: Windows 7
Location: Oregon City, OR

Re: Cumulus - manually edit faulty data?

Post by rwilliam99 »

I tried both methods (using the editor and also editing the ini files manually when Cumulus is not running)t, but when Cumulus restarts, it "fetches" the prior all time high value (which is incorrect). I can't find that value in any of the data files, which leads me to believe it is somehow reprocessing the data files, which have the invalid data in them.
Matt.j5b
Posts: 512
Joined: Mon 28 Nov 2011 2:13 am
Weather Station: Davis VP2/ WLL with DFARS
Operating System: RPi Raspbian (Buster)
Location: Ferny Grove, Brisbane, Australia
Contact:

Re: Cumulus - manually edit faulty data?

Post by Matt.j5b »

So the problem is with incorrect all time records. Is that right? Unless you are editing the wrong file (the all time ini file is "alltime.ini") or when you used the editor you did not click copy as well as OK. I have used the all time editor in Cumulus with the spikes in the readings lots of times and once I removed them from the log file and dayfile, I clicked "fetch" and clicked "copy" for the incorrect fields and pressed "OK" and it updated correctly. After you correct the all time records using the editor and press OK the alltime.ini file is updated and as I understand that file stores the figures for the all time records. Full details of the records editor is in the help file under "The displays" and then click on "The Records (Highs and Lows) Editors".
Regards, Matt of Brisbane, Australia
Ferny Grove Weather
Image
IslandBoy77
Posts: 8
Joined: Wed 04 Apr 2012 10:38 pm
Weather Station: WS1081
Operating System: Vista Business 32-Bit
Location: Napier, New Zealand

Re: Cumulus - manually edit faulty data?

Post by IslandBoy77 »

Hi Matt

You will probably end up doing what I'm thinking of doing: pull the battery from the transmitter. It appears that the culprit that keeps "resetting" any changes one makes to the various files is the rain gauge within the weather station transmitter itself. I've twice in the last 2 months had the sprinkler going on the front lawn and accidentally got water into the rain gauge. My plan is to make sure I have a backup of everything on 31/12/12, then pull the battery on the unit on 1/1/13, put the battery back in, and see what happens (after correcting my rain totals as per the notes I've made). I'm picking that all should be well, since one has to replace the batteries in the weather station transmitter itself from time to time anyway.
Matt.j5b
Posts: 512
Joined: Mon 28 Nov 2011 2:13 am
Weather Station: Davis VP2/ WLL with DFARS
Operating System: RPi Raspbian (Buster)
Location: Ferny Grove, Brisbane, Australia
Contact:

Re: Cumulus - manually edit faulty data?

Post by Matt.j5b »

My spikes I get generally occur in winter with the lower temperatures with the temperature and humidity readings. Last winter I most likely edited Cumulus of the spikes about 20 or 30 times and got about 40 to 50 spikes. Between July 27 and September 12 I got 33 spikes of which 17 needed editing as I run Cumulus off the logger when I started to keep a count of the spikes. I did try to reset the transmitter but nothing avoids the trouble, its the low temperatures that do it. I have not have much of false rain readings to correct as I tested the gauge before I used Cumulus and we get lots of rain except for recently.
Regards, Matt of Brisbane, Australia
Ferny Grove Weather
Image
User avatar
steve
Cumulus Author
Posts: 26702
Joined: Mon 02 Jun 2008 6:49 pm
Weather Station: None
Operating System: None
Location: Vienne, France
Contact:

Re: Cumulus - manually edit faulty data?

Post by steve »

rwilliam99 wrote:I'm hoping I can override the monthly and all time totals somewhere I'm not seeing it.
As Matt says, the monthly totals are calculated from the daily totals in dayfile.txt, so you need to correct those and then your monthly totals will be correct (after a restart). You can then use the high/low editor(s) in Cumulus to regenerate the monthly high.
Steve
rwilliam99
Posts: 42
Joined: Sun 01 Apr 2012 2:55 pm
Weather Station: WS-2080
Operating System: Windows 7
Location: Oregon City, OR

Re: Cumulus - manually edit faulty data?

Post by rwilliam99 »

That was the culprit. I did not edit the dayfile.txt. Once I did that and restarted, my totals are now accurate. I knew it had to be something simple....

THANK YOU!
Phil23
Posts: 884
Joined: Sat 16 Jul 2016 11:59 pm
Weather Station: Davis VP2+ & GW1000 (Standalone)
Operating System: Win10 Pro / rPi Buster
Location: Australia

Re: Cumulus - manually edit faulty data?

Post by Phil23 »

steve wrote:Have you checked dayfile.txt? They are likely to be in there even if they aren't in the monthly log files. Once you've corrected dayfile.txt, you can use the built in record editor(s) to scan dayfile.txt and rebuild the records from that.
Is it safe to do the above by transferring MX's data over to version 1 then back again?, or is that a big NO-NO?

Phil
:Now: :Today/Yesterday:

Image

Main Station Davis VP2+ Running Via Win10 Pro.
Secondary Stations, Ecowitt HP2551/GW1000 Via rPi 3 & 4 Running Buster GUI.
:Local Inverell Ecowitt Station: :Remote Ashford Ecowitt Station:
Post Reply