Welcome to the Cumulus Support forum.

Latest Cumulus MX V3 release 3.28.6 (build 3283) - 21 March 2024

Cumulus MX V4 beta test release 4.0.0 (build 4018) - 28 March 2024

Legacy Cumulus 1 release v1.9.4 (build 1099) - 28 November 2014 (a patch is available for 1.9.4 build 1099 that extends the date range of drop-down menus to 2030)

Download the Software (Cumulus MX / Cumulus 1 and other related items) from the Wiki

.NET error related to Toolbox

Discussion and support for 3rd-party (non-Sandaysoft) tools for Cumulus
Post Reply
RayProudfoot
Posts: 3372
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

.NET error related to Toolbox

Post by RayProudfoot »

For the last 2 days I've had this error message when I checked Cumulus in the morning.
NET Error.JPG
The only change I've made to the netbook running Cumulus is to change the backup (xcopy via a batch file) from sending data to an attached memory stick to a NAS box. CumulusToolboxHTTP.txt exists and is empty except for a single sentence... This is the HTTP list for Cumulus Toolbox. Please do NOT edit this file
---------------------------------------------------------------------------


Cumulus Toolbox still uploads my dayfile.txt to my website ready for IICCP to download it.

I'd appreciate any help on how to solve it.
You do not have the required permissions to view the files attached to this post.
Cheers,
Ray, Cheshire.

Image
RayProudfoot
Posts: 3372
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: .NET error related to Toolbox

Post by RayProudfoot »

I think I know what is causing the problem. The backup is called from a Scheduled Task that runs every 6 hours. It runs very close to the time Toolbox uploads dayfile.txt to my webserver. I've changed the schedule from every 6 hours to 12 avoiding the spell around midnight.

This time tomorrow I should know if that has fixed it. I think the backup must be taking longer to a NAS box than to a USB stick.
Cheers,
Ray, Cheshire.

Image
BCJKiwi
Posts: 1255
Joined: Mon 09 Jul 2012 8:40 pm
Weather Station: Davis VP2 Cabled
Operating System: Windows 10 Pro
Location: Auckland, New Zealand
Contact:

Re: .NET error related to Toolbox

Post by BCJKiwi »

Interesting you should suspect the NAS box.

I have tried two different units recently and find the second much better than the first in general use.
However both are absolute snails when it comes to backup - they are just so slow as to be impractical. However I can run the backup to a large USB drive in a fraction of the time it takes trying to backup to the NAS.

Its the old story, large files backup relatively quickly but the OS overhead dealing with large numbers of small files just bogs down the Linux OS all these NAS boxes seem to use.

My problem now is how to backup the data on the NAS!
RayProudfoot
Posts: 3372
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: .NET error related to Toolbox

Post by RayProudfoot »

BCJKiwi wrote:Interesting you should suspect the NAS box.

I have tried two different units recently and find the second much better than the first in general use.
However both are absolute snails when it comes to backup - they are just so slow as to be impractical. However I can run the backup to a large USB drive in a fraction of the time it takes trying to backup to the NAS.

Its the old story, large files backup relatively quickly but the OS overhead dealing with large numbers of small files just bogs down the Linux OS all these NAS boxes seem to use.
I think the problem is the old backup was to a USB stick which is going to be much quicker than going via the network irrespective of how fast it is. But the netbook only has 10/100 connectivity via a homeplug so is always going to be slower. I'd blame the slow connection more than the speed of the NAS box. The Synology DS212j got really good reviews in PC Pro (UK). On my other PCs with gigabit connectivity the data transfer is extremely good.
My problem now is how to backup the data on the NAS!
That was concerning me so I chose a box with RAID1 capability. Two 2Tb WD Red drives are fitted and remove any possibility of losing data short of a fire or burglary.
Cheers,
Ray, Cheshire.

Image
BCJKiwi
Posts: 1255
Joined: Mon 09 Jul 2012 8:40 pm
Weather Station: Davis VP2 Cabled
Operating System: Windows 10 Pro
Location: Auckland, New Zealand
Contact:

Re: .NET error related to Toolbox

Post by BCJKiwi »

My NAS also has raid5 but that does not cover the situation where there is data corruption or a file is inadvertently deleted. There is no 'recycle bin' on a NAS (that I'm aware of!).

I also considered the networking issues but found the backup speed was much the same on a 10mbit and a gigbit network when it came to the queues of small files.
The NAS boxes can handle normal file transfers and the like well. Win8 has a 'chart' display of file transfer speed when files are being copied and even a regular copy process to the NAS slows to a crawl with a bunch of small files - of which there are usually many when making backups or copying whole drive contents. This is visible with any similar copy process, however it seems most of these NAS boxes use relatively low power processors and the overhead of managing large numbers of small files becomes very noticeable. The second NAS I used has much more ram and a bigger processor than the first and normal activity is much better but once those file queues build up it is no better than the first.
RayProudfoot
Posts: 3372
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: .NET error related to Toolbox

Post by RayProudfoot »

BCJKiwi wrote:My NAS also has raid5 but that does not cover the situation where there is data corruption or a file is inadvertently deleted. There is no 'recycle bin' on a NAS (that I'm aware of!).
Chances of data corruption are very low. Yes, there isn't a recycle bin but I am only backing up my Cumulus data, not editing or deleting it. And it's a copy of what's on my netbook anyway.
I also considered the networking issues but found the backup speed was much the same on a 10mbit and a gigbit network when it came to the queues of small files.
Yes, lots of small files will take longer to copy but I'm not copying all files. Only those that have changed since the last xcopy a few hours earlier.
...however it seems most of these NAS boxes use relatively low power processors and the overhead of managing large numbers of small files becomes very noticeable. The second NAS I used has much more ram and a bigger processor than the first and normal activity is much better but once those file queues build up it is no better than the first.
There's definitely a trade-off between cost and performance. If time was money it would be important but it isn't so I'm not too bothered if it takes 2 minutes compared to 1 minute.
Cheers,
Ray, Cheshire.

Image
BCJKiwi
Posts: 1255
Joined: Mon 09 Jul 2012 8:40 pm
Weather Station: Davis VP2 Cabled
Operating System: Windows 10 Pro
Location: Auckland, New Zealand
Contact:

Re: .NET error related to Toolbox

Post by BCJKiwi »

I also copy the entire Cumulus folder to my NAS at 23:52 each night using a batch file to rename/retain the last 15 days' zip files and 7-zip to create a new single zip file (~23,500kb - original size > 64Mb) the whole process takes < 55 seconds.

As for the rest of my backups;
Case 1
I can make a complete backup of ~ 250Gb data on my 2008 R2 server to a local USB 3.0 Drive.
It takes twice as long as that to copy the same data to the NAS.
Same scenario and similar times for a PC.
If I attempt to Backup the same data to the NAS (using the NAS purpose made Backup software installed the PC) it takes over 9hrs!

Case 2
I need to back up the NAS as it has data on it that does not exist elsewhere (won't fit anywhere else!).
So I now need to try the NAS doing the backup to yet another locally connected (to the NAS) USB drive. All reports from other users indicate this will also be really slow - only time will tell (pun intended).
BCJKiwi
Posts: 1255
Joined: Mon 09 Jul 2012 8:40 pm
Weather Station: Davis VP2 Cabled
Operating System: Windows 10 Pro
Location: Auckland, New Zealand
Contact:

Re: .NET error related to Toolbox

Post by BCJKiwi »

Just tested a 'synchronise' of a 'data' folder on the NAS to a locally connected USB (2.0) drive. Initial copy of the folder containing 796 folders, 45370 files with a total size of 58.32Gb took 36 mins 25 secs at an average speed of 27.32Mb/sec.

Will have to get a decent sized USB drive to back up the lot!
RayProudfoot
Posts: 3372
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: .NET error related to Toolbox

Post by RayProudfoot »

BCJKiwi,

I'll respond to your post later today. Having cancelled the scheduled backup due to run just after midnight I didn't expect to see this error again. Sadly it appeared proving it's nothing to do with the backup.

I've rebooted the netbook and we'll see what happens tonight. But Toolbox did transfer the file so whatever is going wrong it's after the action. :?
Cheers,
Ray, Cheshire.

Image
RayProudfoot
Posts: 3372
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: .NET error related to Toolbox

Post by RayProudfoot »

BCJKiwi wrote:I also copy the entire Cumulus folder to my NAS at 23:52 each night using a batch file to rename/retain the last 15 days' zip files and 7-zip to create a new single zip file (~23,500kb - original size > 64Mb) the whole process takes < 55 seconds.
Why the complete backup every night when only a few files have changed?
As for the rest of my backups;
Case 1
I can make a complete backup of ~ 250Gb data on my 2008 R2 server to a local USB 3.0 Drive.
It takes twice as long as that to copy the same data to the NAS.
Same scenario and similar times for a PC.
If I attempt to Backup the same data to the NAS (using the NAS purpose made Backup software installed the PC) it takes over 9hrs!
The advantage of a NAS box over an external USB3 drive is not speed, but convenience. It can be accessed from every PC in the house instead of having to carry a drive around and plug and unplug it. Also, a USB drive will not give you the security of RAID1.
Case 2
I need to back up the NAS as it has data on it that does not exist elsewhere (won't fit anywhere else!).
So I now need to try the NAS doing the backup to yet another locally connected (to the NAS) USB drive. All reports from other users indicate this will also be really slow - only time will tell (pun intended).
Can you fit a second drive to it and implement RAID1? That would guarantee you wouldn't lose any important files. Where do all these backups end? Will the NAS backup needs backing up? It could go on forever! :shock:
Cheers,
Ray, Cheshire.

Image
BCJKiwi
Posts: 1255
Joined: Mon 09 Jul 2012 8:40 pm
Weather Station: Davis VP2 Cabled
Operating System: Windows 10 Pro
Location: Auckland, New Zealand
Contact:

Re: .NET error related to Toolbox

Post by BCJKiwi »

Why the complete backup every night when only a few files have changed?
Because I don't want to have to start over if the drive fails!
The advantage of a NAS box over an external USB3 drive is not speed, but convenience. It can be accessed from every PC in the house instead of having to carry a drive around and plug and unplug it. Also, a USB drive will not give you the security of RAID1.
Ray, you don't get it. I know what a NAS is and does.
RAID is NOT a BACKUP.
I do not accept that data/installs/whole systems will not fail at some point.
I maintain a strategy that allows rapid recovery of each system that is important to me (and most of them are). Therefore they all need to be backed up.

When I got the NAS I expected to be able to backup my computers to the NAS and would need also to backup the unique data on the NAS to somewhere else (excluding the computer backups of course).
I found this to be impractical as it is too slow.
So I back up my main two systems (main work PC and Server) to their own USB drives. Other lesser systems back up what is critical to the server or the NAS
And now I backup the NAS to its own USB drive.

So when the disaster hits I just need the 3 USB drives.
User avatar
mcrossley
Posts: 12695
Joined: Thu 07 Jan 2010 9:44 pm
Weather Station: Davis VP2/WLL
Operating System: Bullseye Lite rPi
Location: Wilmslow, Cheshire, UK
Contact:

Re: .NET error related to Toolbox

Post by mcrossley »

and for off-site backup?

I replicate key data to 'the cloud' just in case there is a local disaster (fire, flood, lightning, theft, airplane crash etc) that takes out all my local data. So irreplaceable stuff (documents, photo's etc) is replicated up to either my Google Drive, or Skydrive which are both free and between them provide me with 29GB of storage which is enough for my really important data. Both have simple little applets that replicate your data in semi-real time when it changes.

I also use these cloud services to replicate data between some PCs, so for instance my weather website from my home 'server' is replicated (via Google Drive) to my work laptop, then if required when I am away from home I can make a local change on my laptop, and within a few minutes the change is replicated to my web server at home.
RayProudfoot
Posts: 3372
Joined: Wed 06 May 2009 6:29 pm
Weather Station: Davis VP2 with Daytime FARS
Operating System: Windows XP SP3
Location: Cheadle Hulme, Cheshire, England
Contact:

Re: .NET error related to Toolbox

Post by RayProudfoot »

BCJKiwi wrote: Ray, you don't get it. I know what a NAS is and does.
RAID is NOT a BACKUP.
I do not accept that data/installs/whole systems will not fail at some point.
I maintain a strategy that allows rapid recovery of each system that is important to me (and most of them are). Therefore they all need to be backed up.
RAID1 as distinct from RAID0 is about as safe as you can get. If one drive fails you lose nothing. For two to simultaneously fail plus to lose the original data is so remote I haven't even considered it. I consider that to be as safe as I can get at home.
When I got the NAS I expected to be able to backup my computers to the NAS and would need also to backup the unique data on the NAS to somewhere else (excluding the computer backups of course).
I found this to be impractical as it is too slow.
So I back up my main two systems (main work PC and Server) to their own USB drives. Other lesser systems back up what is critical to the server or the NAS
And now I backup the NAS to its own USB drive.

So when the disaster hits I just need the 3 USB drives.
If that works for you then fine. I'm happy with the system I have. So we're both happy. :D

Going back to my original reason for this thread the reboot solved the error.
Cheers,
Ray, Cheshire.

Image
Post Reply