Cumulus Support

With my Fine Offset W-8681-Solar, using datalogger, somehow when external sensor readings fail humidity values are being written as 10%, even with a spike defined.

I am using latest 1.94 beta.

Cumulus Apr13log.txt:

25/04/13,23:55,18.6,47,7.1,5.0,8.6,1,0.0,0.0,1018.7,6.0,20.6,60,8.6,18.6,18.6,0,0,0.00,0.00,16.9,0,7.8,360,0.0,0.0
25/04/13,23:55,0.0,10,4569.0,0.0,0.0,0,0.0,0.0,1018.7,6.0,20.6,60,0.0,0.0,0.0,0,0,0.00,0.00,0.0,0,7.8,0,0.0,0.0
26/04/13,00:00,0.0,10,4569.0,0.0,0.0,0,0.0,0.0,1018.8,6.0,20.6,60,0.0,0.0,0.0,0,0,0.00,0.00,0.0,0,0.0,0,0.0,0.0
26/04/13,00:05,0.0,10,4569.0,0.0,0.0,0,0.0,0.0,1018.6,6.0,20.6,60,0.0,0.0,0.0,0,0,0.00,0.00,0.0,0,0.0,0,0.0,0.0
26/04/13,00:10,0.0,10,4569.0,0.0,0.0,0,0.0,0.0,1018.8,6.0,20.6,60,0.0,0.0,0.0,0,0,0.00,0.00,0.0,0,0.0,0,0.0,0.0
26/04/13,00:15,0.0,10,4569.0,0.0,0.0,0,0.0,0.0,1018.7,6.0,20.6,60,0.0,0.0,0.0,0,0,0.00,0.00,0.0,0,0.0,0,0.0,0.0
26/04/13,00:20,15.2,67,9.1,11.2,14.8,1,0.0,0.0,1018.7,6.0,20.6,60,14.8,14.7,15.2,0,0,0.00,0.00,12.8,0,0.0,360,0.0,0.0
26/04/13,00:25,14.9,69,9.3,15.8,19.4,26,0.0,0.0,1018.5,6.0,20.9,62,19.4,13.9,14.9,0,0,0.00,0.00,11.7,0,0.0,45,0.0,0.0

EasyWeatherPlus.dat:

8357, 2013-04-26 15:21:32, 2013-04-25 23:55:07, 5, 60, 20.6, ---, ---, ---, ---, 1010.3, 1018.7, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 6.3, 6.3, 352.2, 352.2, ---, ---, 0, 0, 0, 0, 0, 0, 1, 0, 005448, 2414, 05 3C CE 00 FF FF FF 77 27 FF FF FF 80 14 00 40 FF FF FF FF ,
8358, 2013-04-26 15:21:32, 2013-04-26 00:00:07, 5, 60, 20.6, ---, ---, ---, ---, 1010.3, 1018.7, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, ---, ---, 0, 0, 0, 0, 0, 0, 1, 0, 00545C, 2415, 05 3C CE 00 FF FF FF 77 27 FF FF FF 80 14 00 40 FF FF FF FF ,
8359, 2013-04-26 15:21:32, 2013-04-26 00:05:07, 5, 60, 20.6, ---, ---, ---, ---, 1010.4, 1018.8, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, ---, ---, 0, 0, 0, 0, 0, 0, 1, 0, 005470, 2416, 05 3C CE 00 FF FF FF 78 27 FF FF FF 80 14 00 40 FF FF FF FF ,
8360, 2013-04-26 15:21:32, 2013-04-26 00:10:07, 5, 60, 20.6, ---, ---, ---, ---, 1010.2, 1018.6, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, ---, ---, 0, 0, 0, 0, 0, 0, 1, 0, 005484, 2417, 05 3C CE 00 FF FF FF 76 27 FF FF FF 80 14 00 40 FF FF FF FF ,
8361, 2013-04-26 15:21:32, 2013-04-26 00:15:07, 5, 60, 20.6, ---, ---, ---, ---, 1010.4, 1018.8, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, ---, ---, 0, 0, 0, 0, 0, 0, 1, 0, 005498, 2418, 05 3C CE 00 FF FF FF 78 27 FF FF FF 80 14 00 40 FF FF FF FF ,
8362, 2013-04-26 15:21:32, 2013-04-26 00:20:07, 5, 60, 20.6, ---, ---, ---, ---, 1010.3, 1018.7, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, ---, ---, 0, 0, 0, 0, 0, 0, 1, 0, 0054AC, 2419, 05 3C CE 00 FF FF FF 77 27 FF FF FF 80 14 00 40 FF FF FF FF ,
8363, 2013-04-26 15:21:32, 2013-04-26 00:25:07, 5, 60, 20.6, 67, 15.2, 9.1, 12.6, 1010.3, 1018.7, 3.1, 2, 4.1, 3, 0, N, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, 0.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0054C0, 2420, 05 3C CE 00 43 98 00 77 27 1F 29 00 00 14 00 00 00 00 00 00 ,

It's because the same value (255) is sent when the humidity falls below 10%, so Cumulus converts this to 10%. When using live data the 'lost sensor contact' flag being set would prevent the 'set to 10%' code being reached. When using logger data, it doesn't look at the 'sensor contact lost' flag. Right now, I'm not sure why that's the case.

If I can't think of a good reason for it, I'll look at adding a check on the 'lost sensor contact' flag when processing logger data. The result of that would be that it would then log a value of zero, if it hadn't been able to get a valid reading from any previous logger entries.

Personally I'd rather have even data gaps and voids in graphs than this issue, which is very recurrent for anyone with a Fine Offset not turned on 24/24h :/

Unfortunately that's not possible without completely rewriting large parts of Cumulus from scratch. And while I have written a lot of code already specifically for the many problems which are unique to Fine Offset stations, I am not prepared to go that far.

steve wrote:And while I have written a lot of code already specifically for the many problems which are unique to Fine Offset stations

I understand and appreciate that. In fact, that is why I am using Cumulus. It does seem to be quite hard to keep any sort of accurate log using these things.

Still, another issue I found was an almost 20000 mm rain spike also during a sensor failure period. This seems to have been recognized as something to remove, is not accounted for in total rain for the day but does appear in the "Rainfall Today" graph.

One doubt: even if the sensor lost flag is not raised, shouldn't these spikes (humidity included) be discarded since they fall within the "Spike removal" parameters defined in "Calibration"?

26/04/13,01:36,13.9,10,4569.0,6.1,9.7,1,0.0,0.0,1018.5,6.0,20.8,61,7.2,13.9,13.9,0,0,0.00,0.00,12.7,0,0.0,360,0.0,0.0
26/04/13,01:41,13.9,10,4569.0,6.1,9.7,1,0.0,19583.0,1018.4,19589.0,20.9,61,7.2,13.9,13.9,0,0,0.00,0.00,12.7,0,0.0,360,0.0,19583.0
26/04/13,01:46,13.9,10,4569.0,6.1,9.7,1,0.0,0.0,1018.5,6.0,20.9,61,7.2,13.9,13.9,0,0,0.00,0.00,12.7,0,0.0,360,0.0,0.0

8377, 2013-04-26 15:21:32, 2013-04-26 01:35:07, 5, 61, 20.8, 76, 13.9, 9.8, 13.9, 1010.2, 1018.6, 1.7, 2, 2.0, 2, 0, N, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, 0.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0055D8, 2434, 05 3D D0 00 4C 8B 00 76 27 11 14 00 00 14 00 00 00 00 00 00 ,
8378, 2013-04-26 15:21:32, 2013-04-26 01:40:07, 5, 61, 20.8, ---, ---, ---, ---, 1010.1, 1018.5, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, 0.0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0055EC, 2435, 05 3D D0 00 FF FF FF 75 27 FF FF FF 80 14 00 40 00 00 00 00 ,
8379, 2013-04-26 15:21:32, 2013-04-26 01:45:07, 5, 61, 20.9, ---, ---, ---, ---, 1010.0, 1018.4, ---, ---, ---, ---, ---, ---, 20, 19666.8, 0.0, 0.0, 0.0, 6.3, 352.2, 352.2, ---, ---, 0, 0, 0, 0, 0, 0, 1, 0, 005600, 2436, 05 3D D1 00 FF FF FF 74 27 FF FF FF 80 14 00 40 FF FF FF FF ,

There are no spike removal settings for accumulated rain, because it's not possible - or at least extremely difficult - to provide them. Any faulty values that occur while the sensor contact lost flag is set (as with the case you've quoted) would get ignored with the change I am proposing to make.

steve wrote:Any faulty values that occur while the sensor contact lost flag is set (as with the case you've quoted) would get ignored with the change I am proposing to make.

Hello Steve,

Does this change correspond to this bit in the log of improvements made in 1.9.4 beta build 1071: "Ignore outdoor data if 'sensor contact lost' set in Fine Offset logger data"?

If so, I have to report that I am currently using build 1072 and the problem with humidity is still there. This might perhaps be related with the fact that not all sensors were lost (UV and Lux were still communicating)?

Best Regards

hvalentim wrote:Does this change correspond to this bit in the log of improvements made in 1.9.4 beta build 1071: "Ignore outdoor data if 'sensor contact lost' set in Fine Offset logger data"?

It does, yes.

If so, I have to report that I am currently using build 1072 and the problem with humidity is still there. This might perhaps be related with the fact that not all sensors were lost (UV and Lux were still communicating)?

Does the console not set the 'sensor contact lost' flag in that case? Can you show me the same output that you showed previously?

Sorry, ignore that request. It's a bug, I'm checking the wrong byte. Fix in next build.

Good to know you nailed the issue down. Will be looking forward for the next release.

Thanks.

hvalentim wrote:Good to know you nailed the issue down. Will be looking forward for the next release.

Already released when you posted

Already released when you posted

Yes. Only noticed it this morning

Still, I fear the problem is there with 1073 :/

This is the easyweatherplus.dat:

12254, 2013-05-11 09:01:49, 2013-05-11 06:41:48, 5, 59, 18.8, 74, 11.1, 6.7, 11.1, 1013.6, 1022.0, 0.3, 1, 0.7, 1, 2, NE, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, 1708.8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 00AB64, 507, 05 3B BC 00 4A 6F 00 98 27 03 07 00 02 00 00 00 C0 42 00 00 ,
12255, 2013-05-11 09:01:49, 2013-05-11 06:46:48, 5, 59, 18.9, ---, ---, ---, ---, 1013.7, 1022.1, ---, ---, ---, ---, ---, ---, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, 3116.8, 0, 0, 0, 0, 0, 0, 0, 1, 0, 00AB78, 508, 05 3B BD 00 FF FF FF 99 27 FF FF FF 82 00 00 40 C0 79 00 00 ,
12256, 2013-05-11 09:01:49, 2013-05-11 06:51:48, 5, 59, 18.9, ---, ---, ---, ---, 1013.7, 1022.1, ---, ---, ---, ---, ---, ---, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, 4292.8, 0, 0, 0, 0, 0, 0, 0, 1, 0, 00AB8C, 509, 05 3B BD 00 FF FF FF 99 27 FF FF FF 82 00 00 40 B0 A7 00 00 ,
12257, 2013-05-11 09:01:49, 2013-05-11 06:56:48, 5, 59, 18.9, ---, ---, ---, ---, 1013.7, 1022.1, ---, ---, ---, ---, ---, ---, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, 5684.8, 0, 0, 0, 0, 0, 0, 0, 1, 0, 00ABA0, 510, 05 3B BD 00 FF FF FF 99 27 FF FF FF 82 00 00 40 10 DE 00 00 ,
12258, 2013-05-11 09:01:49, 2013-05-11 07:01:48, 5, 59, 18.9, ---, ---, ---, ---, 1013.7, 1022.1, ---, ---, ---, ---, ---, ---, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, 5684.8, 0, 0, 0, 0, 0, 0, 0, 1, 0, 00ABB4, 511, 05 3B BD 00 FF FF FF 99 27 FF FF FF 82 00 00 40 10 DE 00 00 ,
12259, 2013-05-11 09:01:49, 2013-05-11 07:06:48, 5, 59, 18.9, ---, ---, ---, ---, 1013.6, 1022.0, ---, ---, ---, ---, ---, ---, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, 5684.8, 0, 0, 0, 0, 0, 0, 0, 1, 0, 00ABC8, 512, 05 3B BD 00 FF FF FF 98 27 FF FF FF 82 00 00 40 10 DE 00 00 ,
12260, 2013-05-11 09:01:49, 2013-05-11 07:11:48, 5, 58, 18.9, 71, 11.9, 6.8, 10.1, 1013.6, 1022.0, 2.4, 2, 4.1, 3, 0, N, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, ---, ---, 0, 0, 0, 0, 0, 0, 0, 0, 00ABDC, 513, 05 3A BD 00 47 77 00 98 27 18 29 00 00 00 00 00 FF FF FF FF ,
12261, 2013-05-11 09:01:49, 2013-05-11 07:16:48, 5, 58, 18.9, 71, 11.9, 6.8, 10.1, 1013.8, 1022.2, 2.4, 2, 4.1, 3, 0, N, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, ---, ---, 0, 0, 0, 0, 0, 0, 0, 0, 00ABF0, 514, 05 3A BD 00 47 77 00 9A 27 18 29 00 00 00 00 00 FF FF FF FF ,
12262, 2013-05-11 09:01:49, 2013-05-11 07:21:48, 5, 58, 18.9, 68, 12.3, 6.6, 8.9, 1013.9, 1022.3, 3.4, 3, 5.1, 3, 0, N, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 352.5, 14652.3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 00AC04, 515, 05 3A BD 00 44 7B 00 9B 27 22 33 00 00 00 00 00 5B 3C 02 01 ,

This is the may13log.txt:

11/05/13,06:40,11.1,74,6.6,1.1,3.6,46,0.0,0.0,1022.0,0.0,18.8,59,2.5,11.1,11.1,0,15,0.00,0.00,10.1,3,0.0,45,0.0,0.0
11/05/13,06:45,11.1,10,4569.0,1.1,3.6,46,0.0,0.0,1022.1,0.0,18.9,59,2.5,11.1,11.1,0,27,0.00,0.00,10.1,8,0.0,45,0.0,0.0
11/05/13,06:50,11.1,10,4569.0,1.1,3.6,46,0.0,0.0,1022.1,0.0,18.9,59,2.5,11.1,11.1,0,38,0.00,0.00,10.1,16,0.0,45,0.0,0.0
11/05/13,06:55,11.1,10,4569.0,1.1,3.6,46,0.0,0.0,1022.1,0.0,18.9,59,2.5,11.1,11.1,0,50,0.00,0.00,10.1,27,0.0,45,0.0,0.0
11/05/13,07:00,11.1,10,4569.0,1.1,3.6,46,0.0,0.0,1022.1,0.0,18.9,59,2.5,11.1,11.1,0,50,0.00,0.00,10.1,41,0.0,45,0.0,0.0
11/05/13,07:05,11.1,10,4569.0,1.1,3.6,46,0.0,0.0,1022.0,0.0,18.9,59,2.5,11.1,11.1,0,50,0.00,0.00,10.1,55,0.0,45,0.0,0.0
11/05/13,07:10,11.9,71,6.8,8.6,14.8,1,0.0,0.0,1022.0,0.0,18.9,58,14.8,11.1,11.9,0,50,0.00,0.00,9.5,71,0.0,360,0.0,0.0
11/05/13,07:15,11.9,71,6.8,8.6,14.8,1,0.0,0.0,1022.2,0.0,18.9,58,14.8,11.1,11.9,0,50,0.00,0.00,9.5,88,0.0,360,0.0,0.0
11/05/13,07:20,12.3,68,6.6,12.2,18.4,1,0.0,0.0,1022.3,0.0,18.9,58,18.4,11.1,12.3,0,130,0.00,0.00,9.1,106,0.1,360,0.0,0.0

I don't understand that, I'm definitely checking the correct byte now, the one with "40" in it that indicates sensor contact lost. I'll do some testing with my station, it doesn't have solar and most of it is not working, but it might be enough to find the problem. In the meanwhile, could you attach the diags file from the start up when that data was downloaded, please?

steve wrote:could you attach the diags file from the start up when that data was downloaded, please?

Here it is. Should you need further info please let me know.

I screwed up the packaging of the release, it didn't actually contain the fix. I've just uploaded build 1074 which hopefully does fix it.