Incredibly slow WHS over gigabit?

K

KevinG

Limp Gawd
Joined
Jun 21, 2003
Messages
348
After reading a thread here, I decided to check out WHS.

For now, it's installed on a machine with a single 500 gig drive, and a gigabit nic.

At first, throughput was fine (40 MB/s) while putting stuff on the server. However, after a little while, I noticed that my network utilization dropped to 3%, and copies were obviously CRAWLING. At this point, there was only about 150 gigs of data on the disk.

I did a bunch of google searches, and turned up an issue where WHS can slow to a crawl when it decides to move stuff from the landing zone to another disk to make room...However, I've only got a single disk in this box, so I don't know what it might be doing...

I tried restarting the server, no help.
I tried copying data to another gigabit machine on the network from the same client...right up where it should be...FAST. So, it's only moving data to the WHS that is slow.

Any thoughts? I'm glad I'm only on a trial version, because right now it looks like it's a no-go.

Thanks.
-Kevin
 
This worked for me with my network slowness on Vista.

Disable IPv6 in your NIC's properties by unchecking the box next it the IPv6 entry.
 
IPv6 is not installed on WHS by default, just as it isn't installed on Server 2003 by default. If you meant the client, he already eliminated it by testing with another box.

How is speed taking data off of the server?

How is performance on the server only? For example, copy something from the system volume to a share on data while in with RDP.

Also, install HD tune. Check the health tab for the hard drive and run a scan on the disk.
 
I hate it when people judge WHS due to network problems. WHS is Windows 2003 Server with an application on top of it, it's not the operating system if you are getting low performance. I can saturate 4 gig nics with no problems on my WHS boxes.
 
Ok, then disable IPv6 on the clients then. It's a simple check box to try and doesn't even need a reboot. It was the root of my problem on my network at home.
 
MrMike said:
IPv6 is not installed on WHS by default, just as it isn't installed on Server 2003 by default. If you meant the client, he already eliminated it by testing with another box.

How is speed taking data off of the server?

How is performance on the server only? For example, copy something from the system volume to a share on data while in with RDP.

Also, install HD tune. Check the health tab for the hard drive and run a scan on the disk.
Click to expand...

Didn't have too much more time to test, but I did test pulling data from the server... It's no better.

I'll have to install teracopy on the server to get a good feeling for how fast it is locally. And I'll install HD tune at the same time.

All that being said...all other nodes on the network are screaming fast to each other, and this box (which was XP pro before this attempt at WHS) used to be just as fast as all of the others... That sorta points to WHS, in my mind, anyway.
 
Yes, PowerPack1 (and a whole slew of other stuff) came down as soon as I turned on automatic updates and told it to "update now".

Interestingly enough, this morning (9 hours later), transfer speeds are back to 20 MB/s (not what they were yesterday, but that could be the source drive, or something else).

So, WHS was definitely doing *something*...the question is how often is it going to do that? I have more data to move over tonight, so I'll see if this happens again.

-Kevin
 
Did you have file/folder duplication turned on for the folder you were copying items to? WHS will attempt to balance your storage and create those copies on-the-fly, so I wouldn't be surprised if you were experiencing slower performance immediately after copying files over.
 
KevinG said:
For now, it's installed on a machine with a single 500 gig drive, and a gigabit nic.
Click to expand...

-=Antimatter=- said:
Did you have file/folder duplication turned on for the folder you were copying items to? WHS will attempt to balance your storage and create those copies on-the-fly, so I wouldn't be surprised if you were experiencing slower performance immediately after copying files over.
Click to expand...

You can't have file duplication on with one hard drive.
 
Nope. All file duplication is off for now. I plan to turn it on when I add another hard drive...but first I have to come to grips with how often the performance is going to drop to "unacceptable".

When this first started I thought that it might be related to Windows Desktop Search...So I turned off the search service, and then turned off the indexing service...That was before I rebooted the box. It was the same after reboot, so it wasn't an indexing issue either.

I still fully expect this to happen again...but the question remains why, and what is it doing during that time? I'm going to put HDTune on the box, and see if the HD is slow during that time...but I wouldn't be surprised to find that it is if the OS is using it to do something else. We'll see...it's all speculation for now.
 
If you think WHS is using the drive, then you can also install the Disk Management add-on and enable Automatic Disk Monitoring.

It's not unusual to see 5KB/s to 500KB/s for seemingly no reason.
 
MrMike said:
It's not unusual to see 5KB/s to 500KB/s for seemingly no reason.
Click to expand...

Well, that's a little concerning... Are you suggesting that this happens on all Windows OS versions, or is this a WHS thing only? In your experience, how long does this last, and how often does it happen? I'll be sure to install this when I get some time with the server again. Thanks for the pointer.
 
KevinG said:
Well, that's a little concerning... Are you suggesting that this happens on all Windows OS version, or is this a WHS thing only? In your experience, how long does this last, and how often does it happen? I'll be sure to install this when I get some time with the server again. Thanks for the pointer.
Click to expand...

I'd be surprised if all modern operating systems didn't have 5 to 500KB/s of usage on the hard drive occasionally. It's next to nothing. I was just letting you know so you didn't think it was abnormal. Disk Management just had my OS drive hovering between 1KB/s and 20KB/s for a bit, then it went to 0, and then it went back up for a bit again.
 
MrMike said:
I'd be surprised if all modern operating systems didn't have 5 to 500KB/s of usage on the hard drive occasionally. It's next to nothing. I was just letting you know so you didn't think it was abnormal. Disk Management just had my OS drive hovering between 1KB/s and 20KB/s for a bit, then it went to 0, and then it went back up for a bit again.
Click to expand...

Oh, sorry. I misunderstood you. I thought you were saying that peek throughput dropped to 5 to 500KB/s....that would be bad. :)

Yes, I fully expect random sporadic small usage of the HD while doing nothing otherwise...

-Kevin
 
Something else to check is your network cabling. I had a "flakey" gigabit ethernet adapter in a new server build that didn't want to stay at 1000, kept going back to 100. It turned out to be a bad network cable from the switch to the server. Swapped that out and no problems since.

That's one thing I got from my CCNA classes early: if something doesn't work, always check layer 1 first....
 
Interesting to see this complaint appear. I tried and abandoned WHS quite a while ago, well prior to Power Pack 1, mainly due to this exact problem. File transfers would start fast (on GigE) and then become excruciatingly slow. I played around with some things at the time, but never found anything that fixed it, finally abandoning WHS and moving to a less friendly, but quick and simple W2K3 server setup. Same hardware working perfectly ever since. I was actually thinking to going through the trouble to put WHS back on the system this month since PP1 was out, but I will likely be ditching that idea...
 
d259 said:
Something else to check is your network cabling. I had a "flakey" gigabit ethernet adapter in a new server build that didn't want to stay at 1000, kept going back to 100. It turned out to be a bad network cable from the switch to the server. Swapped that out and no problems since.
Click to expand...

This is certainly possible, and I can take a look at it, but the evidence so far doesn't support that... Appreciate the tip.
 
I would check for packet drops.

What NIC (make/brand) do you have on WHS?
Same for the switch.
 
If 2K3 works....WHS should work just the same

I would check your drivers for your nics....WHS is not packaged with all the drivers and components that 2K3 is because its designed for consumer hardware.
 
Okay, so I installed HDTune, and Teracopy on the server.

HDTune looks perfectly normal. The disk is a WD5000AAKS with an average of 65.5 MB/sec, 13.4 ms access time, 123.6 MB/sec burst, with 2.5% cpu usage. Only interesting thing about HDTune is that the info tab says that the drive supports UDMA mode 6, but is currently using UDMA mode 5.

Anyway, nothing else really matters until I can recreate this slowness...for now, the server seems to be back to about 30 MB/sec during file copies.

The computer is one of those [H]ot deals Dell SC440 server machines. So the Nic is the onboard Broadcom NetXtreme Gigabit nic. I have to admit that I only used the Windows Server 2003 drivers that came with the computer...I should go see if there are updated drives on Broadcom's site. It is connected via 2 (gigabit) switches to my main computer in the office, but, like I mentioned before....it was always fast before the WHS installation, so I don't think this is a hardware issue.

I'll see if I can make this happen again and then get a sense of what's happening to disk performance on the server itself...

-Kevin

Edit: There were *way* newer Nic drivers. I've installed them.
 
Well, that didn't take long at all to reproduce! The transfers are back to 3.0 MB/sec.

Running HD tune while experiencing this shows that the harddrive speed is exactly the same as the copy speed...HD Tune thinks the hard drive is only capable of 3.0 MB/sec on average. Great...so what next?

CPU usage is effectively 0% while this is happening...

I installed the Disk Management add-in, and turned on auto-monitoring. It shows activity equal to the slow copy speed. If I pause the copy, activity drops to something very low (a few hunded KB/sec)...so it doesn't think anything ELSE is happening on the disk...

Interestingly enough...while doing this, speed JUMPED up to 35 MB/sec in front of my eyes...like it *finished* doing something and went back to full speed. *SIGH*
 
Even 35 mb / sec isn't full bore for a modern hard disk. It kind of sounds like you have run into a bug in the disk drive itself.
 
Run a full disk scan in HD tune overnight.

The next morning, report what it says under the health tab for reallocated sectors. When a hard drive reallocates sectors, performance and seek time for those sectors suffers considerably.
 
Better yet, back anything up (on that disk) that isn't alreqady, then do a full format (not a quick) on that drive. Reinstall and see what happens.
 
I've kicked off a full error scan of the drive. I'll report back what it says.

I'm not too concerned about only hitting 35 mb/sec. This is fairly close to the max I've ever seen on my home network...I don't know if it's the source drives, the destination drives, the network, or something else...but that's about as good as I've seen it (I think the max ever was 40 mb/sec). I'm willing to dig in on that and find out how to get it higher....but for now I have to deal with it dropping to 3 mb/sec on the WHS...and that won't cut it.

If HDTune doesn't turn anything up, should I really go ahead and format the disk and start over? Is that likely to help? It's not a big deal, really, since this is basically a fresh WHS install with a bunch of stuff moved to it from my primary machine...not hard to recreate even without a backup (all of the data still exists in its original location as well.)

Thanks for the help.
-Kevin
 
A full format looks for bad sectors that may have appeared since the original format. Bad sectors are marked in a bad sector table and from then on are never used for storage. If you have bad sectors, then the drive may be trying to write to those locations. It will automatically do N retries etc etc.

The full format (not quick format) finds and marks them and gets rid of this problem.
 
try to enable jumbo frames on all NICs... make sure switch supports it however
 
jcolby said:
The full format (not quick format) finds and marks them and gets rid of this problem.
Click to expand...

Understood...but is there a reason to do that if HDTune doesn't find any bad sectors? (It's currently about 1/2 way done, and it's all green "Ok") The Health tab shows Reallocated Sector Count at 200 Current, 200 Worst, 140 Threshold, 0 Data and Ok Status...Not sure if this tab updates in real-time while doing the error scan though, so I'll report back when it is finished.

mjz_5 said:
try to enable jumbo frames on all NICs... make sure switch supports it however
Click to expand...

Switch does support it, but I currently have it turned off. I was under the impression that this was more about cpu usage during transfers as opposed to throughput. Is it both? That's easy enough to turn on, so I should do it...

Thanks.
 
Okay, HD Tune finished. It didn't find a single problem.

For now, I'm running sysinternals process explorer on the WHS. I'm tracking cpu and disk I/O. I'll try to reproduce the problem and see where the performance is going...
 
KevinG said:
Understood...but is there a reason to do that if HDTune doesn't find any bad sectors?
Click to expand...

In order to REALLY find bad sectors on a hard disk, just as in REALLY finding bad memory, you need to write / read different patterns. I am guessing that HDTune is just doing a quick peek to see if it can read a sector. That does not truly address whether that sector can actually store all different patterns.

The true format is a utility specifically designed to find, and map as bad, sectors which have problems storing data.

Which do you trust to do the job?

I ALWAYS do a full format on every disk before putting it into service, on old disks before putting them back into service.

I cannot say that this is going to fix your problem, however it is a good practice, seeing that you are having problems with this disk.

As for all the network stuff... If you can see this behavior sitting directly on the computer containing the disk, without doing transfers from some network source onto this disk, then the odds of it being a network issue are vanishingly small. To test, DISCONNECT the network cable, then test the disk. If you can still see the issue, it is NOT the network.

From everything you have said, it sure doesn't sound like a network problem to me!

I understand that this is THE DISK in the system and that doing a full format will be a pain. You have run into a case where finding the issue is not going to be fun. I do think that just blaming WHS, while tantalizing, is a bit premature. Do you have another disk that you could image with your current disk, and then swap the disk entirely?

While it is not cheap, SpinRite is a good investment for finding and handling disk issues if that is what it turns out to be.
 
jcolby said:
I understand that this is THE DISK in the system and that doing a full format will be a pain. You have run into a case where finding the issue is not going to be fun. I do think that just blaming WHS, while tantalizing, is a bit premature. Do you have another disk that you could image with your current disk, and then swap the disk entirely?
Click to expand...

I don't think it's the network anymore...I'm convinced it is either something WHS is doing, or a fault with the disk.

I do have another 500 gig disk (that I had planned to put into the server after this was all figured out). I think I'm going to "format" this disk, and then use acronis to image to WHS boot disk. I'll restore that onto this other disk and see if I can recreate the problem.

-Kevin
 
I have seen some weird actions by WHS before with stupid slow transfers. It starts off slow then speeds up at the end. The only time I have seen it though is when I'm doing a share to share transfer on the same disk where the disk is VERY full. My guess is the drive is horribly fragmented (I do lots of file moves) and the file move is causing the head to be screaming all over the place.
 
KevinG said:
I don't think it's the network anymore...I'm convinced it is either something WHS is doing, or a fault with the disk.

I do have another 500 gig disk (that I had planned to put into the server after this was all figured out). I think I'm going to "format" this disk, and then use acronis to image to WHS boot disk. I'll restore that onto this other disk and see if I can recreate the problem.

-Kevin
Click to expand...

Thats what I would do

If that does not work I remember when I first installed WHS about a year ago I had a similar issue. During a drive transfer my throughput would go through the floor, and the I would lose the connection. What I figured out is that I did not install the network drivers for my nic during the install. I only say that because you mentioned you were using the ones that came with the install disc.
So if the image is bad do a full format and slipstream your drivers for the new install.
 
Ockie said:
I hate it when people judge WHS due to network problems. WHS is Windows 2003 Server with an application on top of it, it's not the operating system if you are getting low performance. I can saturate 4 gig nics with no problems on my WHS boxes.
Click to expand...


Ockie -

What nics are you using. I thought teaming was disabled in WHS?
 
Teaming is done by the hardware settings, it has nothing to do with WHS.

I have 4nics teamed on my WHS as well.
 
brycejones said:
Which NICS are you using? I've been wanting to do this to my WHS.
Click to expand...

I have two Onboard Broadcom Nics....the broadcom utility software allows me to team those and my other nics together. I have 2 Rosewill gbit nics I got for like $20 and the broadcom utility allows me to team them.
 
You must log in or register to reply here.
Back
Top