Very Slow Data Processing

[logjam]

Hello

I've been using this software for weeks with no problems.
Today I've noticed that whenever Cumulus communicates with the console it takes about 16 seconds to successfully process each line of information and write the data in the monthly data file. While it does this Cumulus 'freezes' and I can't do anything else with the programme. The main concern is when I catch up the data after Cumulus has been off for a while. It takes 16 seconds to read 1 minute of data. When it finally finishes reading all the data, (which can take hours) it ignores everything since the catch up starts, and that data is not downloaded.

I'm using an FO N96GY with Cumulus 1.9.3 Build 1058
I have tried resetting the console, restarting the computer, rolling back the software etc, but with no change. It seems to have nothing to do with the software version. I'm not having any other problems with USB devices

[steve]

The actual download of the data is very quick, so it's not a USB issue. The actual processing of the data is also quick. It's the stuff it does after it's processed the data that is taking 20 seconds - writing the log file, and calculating the trends etc. And it's not just the logger data where this happens, it takes 20 seconds each time it writes an entry to the log files (and does the trends etc) for live data also.

There's not enough in the diags files to see what is taking so much time. Please turn on the debug log, leave it for 5 minutes, and then attach the debug.log file. That might give some more clues.

[logjam]

Many thanks Steve

Here is the debug log as requested.

[mcrossley]

Looks like creating the WXNow File takes 20 seconds for some reason?

[steve]

That's narrowed it down a little, but doesn't help much:

2628.009 : 20:19:00 Creating WXNow File
2647.915 : 20:19:20 Updating WOW

There are quite a number of actions between those two, that don't get logged - adding entries to the graphs, a few 'trend' calculations, writing the entry to the log file.

Could you temporarily change the logging interval in Cumulus to 5 minutes, and then run again for a few minutes with the debug log switched on, please? That would at least tell us whether it's writing the log file entry that's causing the delay. Beyond that I'll have to add more trace points to the debug log.

[logjam]

The debug log after changing the logging interval

[steve]

It's the writing of the log file (or today.ini, which it does at the same time).

1223.718 : 20:48:00 Creating WXNow File
1223.734 : 20:48:00 Updating WOW

But when it writes the log file:

1343.746 : 20:50:00 Creating WXNow File
1363.776 : Creating web graphs
1364.821 : 20:50:21 Updating WOW

I don't think the creation of the web graphs is relevant, as we saw it was taking 20 seconds previously without that.

I'm afraid I have no idea why writing the log files takes 20 seconds; perhaps you have some kind of corruption, or a bad sector on the hard drive? You should perhaps run some disk checks.

[logjam]

Many thanks. That gives me something to work on. Come to think about it I did have problems starting Windows yesterday evening. . . . . . . . .

[logjam]

That was the problem! I've now corrected the hard drive corruption and all is working as it was.

Many thanks for the help.