Page 1 of 1

Spikes in Data, data file corrected, but graph spike remains

Posted: Wed 31 Oct 2012 9:54 pm
by BigDave
I wonder if anyone can shed any light on this for me?

On he 27th, at around 18:00, I got a lot of spikes appear in various fields of data (windspeed gust of 275mph, outside temp of -27.5 etc). I have manually gone in to remove these erroneous figures from the Oct12 data file, and all of my large graphs have now correctly adjusted with the exception of the Wind Gust reading.

I wondered if the graphs were re-drawn at startup, so I restarted Cumulus a number of times (incidentally, I shut down cumulus before editing the file) but the large graph still shows this spike. The trouble is, as it's so much higher than all the other readings, the graph has re-calibrated it's Y axis in order to allow room for the spike to fit, meaning that all my real readings are little bumps at the bottom of the graph.

Can anyone offer any advice as to how I can clean up my graphs?

Also, any ideas why my F/O might have generated these nonsense figures? Radio noise, or iffy unit?

FYI they didn't all happen at exactly the same time, but in the course of 10 minutes or so. Some were a singe spurious entry, others including the wind gust reading was 10 or so lines of data. And my thumbnail graph for wind doesn't show this spike.

Any help gratefully received, if there is no solution, I'll just have to hang on until the 27th pops off my graphs!

Re: Spikes in Data, data file corrected, but graph spike rem

Posted: Wed 31 Oct 2012 10:36 pm
by steve
Yes, the graphs are redrawn at start up from the monthly log file, so your spurious gust figure must still be in there.

You can use the 'spike removal' settings on the calibration screen to try to avoid the odd readings that Fine Offset stations sometimes give. I have no idea what causes them, radio interference is one candidate.

Re: Spikes in Data, data file corrected, but graph spike rem

Posted: Thu 01 Nov 2012 1:01 pm
by BigDave
Thanks very much Steve.

I've looked through the logfiles, and even used the new (ish) facility in "Edit all time records" to scan the data files for this spurious high gust value (that's actually how I originally pinpointed the exact time of the incident in the first place), and I can't find anything there. I then went through the CSV file manually looking for anything I had missed and still no sign of it. I did find a matching high figure in the dayfile for that day and I have edited that also, but the spike remains!

Thanks for the information though. In two days the spike will be gone anyway, I just wanted to better understand how Cumulus worked, and your explanation has served that purpose very well thank you!

Cheers


Dave

Re: Spikes in Data, data file corrected, but graph spike rem

Posted: Thu 01 Nov 2012 1:28 pm
by steve
If you attach your October log, I'll have a look.

Re: Spikes in Data, data file corrected, but graph spike rem

Posted: Thu 01 Nov 2012 1:54 pm
by BigDave
Thanks so much Steve, most helpful as always!

File is attached.

Re: Spikes in Data, data file corrected, but graph spike rem

Posted: Thu 01 Nov 2012 2:20 pm
by steve
Three entries with 287.0 in:

27/10/12,18:42,6.3,81,3.3,3.1,15.6,314,0.0,0.0,1019.6,29.7,18.9,56,11.0,5.6,6.3,0,0,0.00,0.00,3.9,0,0.0,315,0.0
27/10/12,18:43,5.4,89,3.7,3.0,15.6,348,0.0,0.0,1019.4,29.7,18.9,56,287.0,4.6,5.4,0,0,0.00,0.00,3.1,0,0.0,90,0.0
27/10/12,18:44,5.4,89,3.7,2.7,15.6,80,0.0,0.0,1019.4,1258.5,18.9,56,287.0,5.4,5.4,0,0,0.00,0.00,3.2,0,0.0,90,0.0
27/10/12,18:45,5.4,89,3.7,2.5,15.6,86,0.0,0.0,1019.4,1258.5,18.9,56,287.0,5.4,5.4,0,0,0.00,0.00,3.2,0,0.0,90,0.0
27/10/12,18:46,6.3,81,3.3,2.5,15.6,86,0.0,0.0,1019.6,1258.5,18.9,56,9.1,6.3,6.3,0,0,0.00,0.00,4.1,0,0.0,315,0.0
27/10/12,18:47,6.3,81,3.3,2.4,15.6,86,0.0,0.0,1019.5,29.7,18.9,56,6.5,6.3,6.3,0,0,0.00,0.00,4.1,0,0.0,360,0.0

You could also change those 'total rainfall counter' values from 1258.5 to 29.7 while you're in there, but those aren't critical and not really used after 24 hours have passed.

Re: Spikes in Data, data file corrected, but graph spike rem

Posted: Thu 01 Nov 2012 4:01 pm
by BigDave
:oops: Sorry Steve! Schoolboy error on my part!

Thanks very much once more!