Tech-Hounds.com

Looking at Vista's I/O Performance

Nearly 10 months after its released, Windows Vista's adoption rate is still pretty low, particularly compared to Windows XP. Of course, XP has quite of a lead over Vista - it's been the default choice for many users for several years now. As for the reason, well, it's actually nothing we didn't expect from Microsoft and their products. Keeping in tradition to other previous versions of Windows, Vista needs quite a hefty system to be able to run smoothly. You'll definitely NEED at least 1 GB of RAM and a DirectX 9 graphics card to enjoy what Vista has to offer. At least - if you really want to get your work done in Vista smoothly, prepare to get at least 2 GB of RAM and a dual core processor - it's that power hungry. With that kind of a machine, Windows XP is blazing fast - no wonder users prefer to stick to Windows XP..

Even the promise and (unfulfilled) potential of DirectX 10 is not sufficient to sway the crowd. Ten months after the release of Vista and you can count the number of DirectX 10 titles on one hand - Company of Heroes, Lost Planet: Extreme Condition and Bioshock. Granted, Crysis is just a few weeks away and let's not forget the latest installment of Unreal Tournament. However, as many reviews and previews have shown, don't expect blazing fast performance from the available, first generation DirectX 10 hardware. Driver problems didn't made things any easier. Just look at the first six months of Vista release. GeForce 8800 graphics card owners are probably the hardest hit. There are also other problems as well - one is related to hard drive / storage and the other network performance.

Here;s an interesting tidbit from Adrian Kingsley-Hughes' blog, regarding the network performance problem.

I" have received a response to the Vista network performance issue from Microsoft.  Here are some points of interest:"

• We have been looking into this problem and are working on a doc that will go into the technical details of what we have found.
• Please note that some of what we are seeing is expected behavior, and some of it is not.  In certain circumstances Windows Vista will trade off network performance in order to improve multimedia playback. This is by design.
• The connection between media playback and networking is not immediately obvious.  But as you know, the drivers involved in both activities run at extremely high priority.  As a result, the network driver can cause media playback to degrade.  This shows up to the user as things like popping and crackling during audio playback.  Users generally hate this, hence the trade off.
• In most cases the user does not notice the impact of this as the decrease in network performance is slight.  Of course some users, especially ones on Gigabit based networks, are seeing a much greater decrease than is expected and that is clearly a problem that we need to address.
• Two other things to note. First, we have not seen any cases where a users internet performance would be degraded, in our tests this issue only shows up with local network operations.
• Second, this trade-off scheme only kicks in on the receive side.  Transmit is not affected.

So, users with standard 10/100 MBps Ethernet connections will likely not notice the issue (too much). Hopefully, a fix will be available soon - preferably before Vista SP 1 comes out.

Vista's I/O Problems

Microsoft posted several updates and fixes - one of which is related to I/) storage performance. This issue have been plaguing the fledgling operating system for quite a while now. Earliest reports of such problems were from users using Windows Vista RC1 (yes, it's that long). Microsoft finally released an update addressing this problem at September 11, 2007.

Windows Vista Performance and Reliability Update (938979)

"This update resolves issues that may affect some Windows Vista-based computers. These issues have been reported by customers who are using the Error Reporting service or Microsoft Customer Support Services."

• You experience a long delay when you try to exit the Photos screen saver.
• A memory leak occurs when you use the Windows Energy screen saver.
• If User Account Control is disabled on the computer, you cannot install a network printer successfully. This problem occurs if the network printer is hosted by a Windows XP-based or a Windows Server 2003-based computer.
• When you write data to an AVI file by using the AVIStreamWrite function, the file header of the AVI file is corrupted.
• When you copy or move a large file, the "estimated time remaining" takes a long time to be calculated and displayed.
• After you resume the computer from hibernation, it takes a long time to display the logon screen.
• When you synchronize an offline file to a server, the offline file is corrupted.
• If you edit an image file that uses the RAW image format, data loss occurs in the image file. This problem occurs if the RAW image is from any of the following digital SLR camera models:
• Canon EOS 1D
• Canon EOS 1DS
• For more information, click the following article number to view the article in the Microsoft Knowledge Base 932988 (http://support.microsoft.com/kb/932988/) Files from a Canon EOD 1D or 1DS camera may be corrupted after you use Windows Photo Gallery, Windows Live Photo Gallery, or Windows Explorer to edit the file metadata in Windows Vista or in Windows XP Service Pack 2

• After you resume the computer from hibernation, the computer loses its default gateway address
• Poor memory management performance occurs.

Though it is easy to blame Microsoft, they're not the only one to blame. Hardware manufacturers are also having teething problems with Vista drivers. Intel posted a workaround for users having problems with RAID 0 arrays in Vista. They did post this fix (and updated drivers) way earlier than Microsoft - May 25th, 2007 or just slightly over three months earlier.

Possible issues with Windows Vista* and Intel® RAID

The following issues may occur on Windows Vista* with a supported Intel® RAID Controller hub:

• Inability to install Windows Vista (i.e. system halts during installation)
• Performance degradation
• Timeout errors reported by iastor or iastor(v) in Microsoft Event Viewer*

Now, with all these fixes and updates have Vista I/O performance improved to where it should be? That's what we're going to find out in this article. But first, let's look at some other factors that makes measuring storage performance a little bit more 'trickier' than in Windows XP.

Vista's Background Tasks

As we mentioned earlier (and user's will no doubt notice), Vista uses - some say need - more resources than Windows XP. Though many users will likely point to the use of Aero, this graphics side of Vista actually runs about the same whether you use Aero Glass, Basic or the standard classic 'Windows' user interface. What about services and other background processes? This are the more likely culprits. Windows XP has about 43 services - not including third party services. In comparison, Windows Vista has about 120 services. That's about three times as much. Of course, not all of these services runs at the boot up, however the number of services (and process) running will likely be higher with Vista than it is with Windows XP.

In addition to services, Vista has more background tasks as well. For instance, Windows Defender. This program will monitor your system in real time for file and configuration changes to guard your Windows installation against spy ware and other malware. So, it acts pretty much like an antivirus software. In addition to Windows Defender, you can also find the usual suspects of background tasks - Internet Connection Firewall, Automatic Updates, System Restore - also available on Windows XP.

Ordinarily, you would want to turn off these 'features'. They tend to take up space on your hard drive (System Restore) and may read and write data to the hard drive in the background (Automatic Updates and System Restore). These background tasks may influence results in a storage performance, so as a general rule, you should turn them off before doing any benchmarks. 

I/O Prioritization in Vista

One improvement Microsoft likes to point out with Vista is I/O Prioritization. Just like application / process priority, I/O Prioritization allows the operating system to prioritize which I/O tasks should be completed first or take precedence over others. Consider this situation - you want to open a file on your hard drive, but at the same time you're listening to an MP3 and is in the process of copying a folder with large amount of files from one drive to another. I/O Prioritization will prioritize I/O so that you can quickly search for the file you want to open, but still allows the MP3 player to stream data and file copying to resume though at a slightly slower pace.

From Microsoft's I/O Prioritization in Windows Vista whitepaper:

"When a user interacts with an application that accesses a file while other threads in the system are also creating I/O requests, the storage system might choose to service the system threads so the system can maintain a higher overall throughput. When responsiveness is considered, the user’s I/O is given higher priority. This causes the application to be more responsive, even though overall I/O throughput might decrease. If the system thread’s I/O is serviced first at the expense of the application’s ability to make progress, the user perceives the system as slower, even though throughput is actually higher."

Now some might say, isn't' that what command queuing are for. Well, here's another interesting little tidbit, still from the same document.

"The NCQ feature of the Serial ATA (SATA) protocol is an example of a hardware queue that could be extended to incorporate priority in a meaningful way. However, rather than proposing SATA NCQ Priority processing as a solidly defined I/O hierarchy strategy, strong momentum exists to make it only a recommendation for action. This makes the device responsible for trying to deliver a better quality of service for high-priority I/O while it balances overall throughput of outstanding NCQ commands with the shortest possible completion times on targeted I/O but does not provide the device with any context other than a general recommendation in the form of a priority value. The current proposal is so general that actually no requirement exists for a drive to process high-priority I/O any differently than if the drive was not sensitive to the priority bit at all. Consequently, all NCQ drives can claim to support Priority NCQ without actually being required to deliver better quality of service for high-priority I/O."

Here is a reminder on just what is Native Command Queuing and Volume Write Back cache. Volume Write Back cache enables multiple I/O requests from the host (controller) to be grouped into fewer requests and written from the cache to the volumes at defined intervals. Native command queuing acts in a similar manner, but goes further by also reordering commands for better throughput. We'll see later on how I/O Prioritization affect benchmarks results. 

RAM: Cache not Resource

Now its time to turn to higher levels of I/O - memory. Windows Vista introduced a more advanced method of prefetching data used in Windows XP dubbed SuperFetch  Just what does it do? Much like Prefetch in Windows XP, it will speculatively load application data ahead of time (prefetching) so that application load faster. How is it different from Windows XP Prefetch. Well, in addition to being 'smarter' - remembering your PC usage habits, SuperFetch goes one step further in doing its job. Here's a snip from Ask the Performance Team's blog, explaining Vista's SuperFetch and ReadyBoost:

"Many people tend to think of RAM as some sort of resource, and when it starts getting used up, they believe that they have a problem.  In reality however, RAM is more like a cache.  If your system is only using a small percentage of your cache is a huge waste.  Imagine how your processor would work if its L2 cache never used more than 25% of its capacity.  L2 cache is a small amount of high-speed memory that allows for high-speed access to the system's most commonly accessed data. In previous versions of Windows, the Prefetcher wasn’t very aggressive in populating RAM. With Windows Vista however, SuperFetch tries its best to use as much RAM as it can, because if you have it you might as well make use of it. "

In other words, SuperFetch will use any available memory on your PC, filling it with data you might / will need. So, when you start an application, the data it needs is already in RAM, and Windows doesn't need to access data from the slower hard drive. But what if the application uses lots of memory? Or you start an application that SuperFetch isn't aware and haven't Prefetch the data for?

Again from the same page:

"...if you have multiple programs that are using lots of RAM, this could still cause system resource depletion, however you would be able to see that in the Memory Usage graph in Task Manager.  If you disable SuperFetch, then Windows Vista behaves more like Windows XP in terms of the Prefetcher."

 

Above are two shots showing RAM usage just after you enter the Windows desktop. RAM usage is pretty high at the beginning, but becomes lower after a short time has passed (about 40 seconds on our setup). The high RAM and page file usage is normal since Windows needs to start up processes and applications when you start the desktop. After those applications and processes have finished starting up, things settle down a bit. At this point, Vista actually behaves pretty much like XP. Why is this important to know, well - again you'll see the reason later on.

Performance

Since the focus of this article is software, more appropriately the difference between Vista and XP, we didn't think a low level performance test such as HDTach or IOMeter will do us any good. They focus on hardware as with other low level synthetic benchmarks. So, we will have to rely on our usual game level load test to see I/O performance differences between Vista and XP.

Our test setup
Intel Core 2 Duo E6300 socket LGA-775 (running at 7 x 400 MHz)
4 x 512 MB A-DATA Vitesta 5-5-5-18 PC6400 DDR2-SDRAM
Gigabyte Radeon X1950 Pro 256 MB graphics card
Gigabyte P965-DS3P Intel P965 motherboard
2 x Hitachi Deskstar 250 GB Serial ATA 16 MB buffer 7200 rpm hard drives
LiteOn 1673S DVD-RW
FSP Epsilon 800 watts ATX/BTX power supply

Windows XP Professional with Service Pack 2 installed
Windows Vista Home Premium 32 bit with all the latest updates.
ATI Catalyst 7.8 reference driver
Intel Chipset Software Installation Utility 8.3.0.1011
Realtek Semiconductor High Definition Audio System Software Ver:R1.72
DirectX 9.0c
all respected games used for benchmarks have been updated to their latest, final builds.

These results are load times in miliseconds, so lower is better.

The results:

These results are very interesting. With first time loads, there are very little variances between runs. So, at least this test seems valid enough for the moment. There's no doubt, that Vista is slower than XP - particularly with a single drive setup. The gap is smaller with RAID 0 arrays, but Vista is still slower than XP by about 1 second. Processor overhead is less likely a factor here - in the past we've seen results that show better performance with RAID 0 arrays compared to single drives - just like what we're seeing here from both operating systems. F.E.A.R first run level loads is a good benchmark sensitive to hard drive transfer rates, not processor overhead.

The consecutive load results are a little bit difficult to explain. Although overall we can see Vista is slower, there's too big of a variation between runs to be absolutely sure - they fall somewhere between 10 and 25 percent. That's just too high than just normal variations between runs.What's causing this behavior? Let's look at Task Manager shots to see what's happening.

First Run Load



Consecutive Load



Because we restarted the system after each run for first time loads, Windows starts 'fresh' every time. This means Windows has more 'free' RAM to work with, hence the lower variations between runs. Now, look at the Task Manager shot with consecutive loads. You'll see RAM usage is higher with consecutive loads. It also tends to grow with each successive loads. Notice how F.E.A.R needs to 'reload' data from the hard drive to RAM - the dips in the graph. These factors are likely the cause we're seeing such high variations between runs - lots of data transfers  between RAM and the hard drive. Could it be F.E.A.R is 'leaking' memory? We don't think so. We do not see such high variations with XP between runs, so the problem lies elsewhere. The only suspect left is the operating system and / or drivers, and of course, SuperFetch. 


Quake 4 behaves in a more predictable manner than F.E.A.R. With Quake 4, we're seeing very small variations between runs - the only exception seems to be first run loads with a RAID 0 array. Even then, the variations are generally smaller than what we saw earlier with F.E.A.R (around 10 percent). As expected, Vista is slower than XP  - about 5 seconds with a single drive. However, it's actually 1 second faster on average with a RAID 0 array (2 seconds faster or lower due to variations between runs). There's very little variations with consecutive loads, and here Vista is about 1 second slower than XP. That 1 second may come from variations between runs, but we don't think so - variations between runs are very low in this test. It doesn't really matter anyway, since you wouldn't notice that kind of a difference in real life.

Now, what can be the cause of slower performance in Vista? Look closely - if we compare those first run results, Vista storage performance with both single and RAID 0 arrays is similar to Windows XP with a RAID 0 array? Now, if transfer rates is the influencing factor here, the RAID 0 array should've been faster. Obviously, that is not the case. The only explanation is processor overhead or CPU utilization during storage operations - that certainly will explain the performance difference between a single drive setup and a RAID 0 array in Windows XP.

First Run Load



Consecutive Load



Ah, that is enlightening. Quake 4 RAM usage stays pretty much constant with consecutive loads. The amount of RAM usage with consecutive loads is not that much different from first run loads. So, it looks like there's still plenty of RAM for Vista, Quake 4 and SuperFetch. If there's not enough memory, it's very likely Windows will try to swap some of the data into its page file - a bad thing since we actually want all data in the much faster RAM. Notice that the amount of free RAM is higher than what we saw earlier with F.E.A.R. This is likely the ideal scenario for doing I/O test related to hard drive in Vista - free as much RAM as possible. However, first run level loads in Quake 4 is really a measure of processor overhead rather than transfer rates.

We do think the results are valid enough to explain why Vista is generally slower than XP when it comes to hard drive performance. These results seem to indicate processor loads to be the culprit. However, we need a second test to confirm this. Let's see the Serious Sam II results.


Like Quake 4, this benchmark is more sensitive to processor overhead than transfer rates. See how the RAID 0 array is slower than a single drive in XP? If we look at Vista, there seems to be no difference between a single drive and a RAID 0 array. That may seem to be good news, but it doesn't change the fact that Vista is slower than XP - at least with a single drive. This does confirm our conclusion concerning higher processor overhead in Vista..

First Run Load



Consecutive Load



Compared to the other games / benchmarks, Serious Sam II 'eats' very little RAM. Physical memory usage stays under 1 GB. RAM usage the whole time and SuperFetch is less likely a factor here. There's also very little variations between runs, though this game dynamically uses RAM like F.E.A.R, but in much smaller proportions. Level loads are shorter - generally about 5 to 7 seconds - too short a time and not enough data to push the storage subsystems. So, it looks like Serious Sam II level loads are more influenced by processor overhead than anything else.

There is no doubt about it - Vista is slower than XP when ti comes to I/O and storage performance. There's a lot of factor to be considered here, but from these benchmarks we seem to narrow down two factors - processor overhead (probably more due to drivers than anything else) and transfer rates. There is that memory issue, where Vista does not have enough memory so it had to swap data into the pagefile

Company of Heroes ISO Test

We need another tests - one that is not prone to high RAM usage. Generally, a game installation is basically a file copy operation involving reads and writes. There may be some decompressing or other processor intensive tasks, but usually not enough to burden the whole system. RAM usage for these tasks are usually very low. From experience, our Company of Heroes ISO tests generally responds very well to differences in transfer rates and processor overhead.

The test involved mounting an ISO image of Company of Heroes, running the installation and measuring the time it takes for the installation process to finish. For today's tests, the ISO image is copied onto the RAID 0 array and mounted. We're going to focus on RAID 0 array performance alone this time around, so no single drive results. Because of I/O Prioritization in Vista, we ran the test two times - once with the installation as the active task (in the foreground) and then on the background.

So, let's see how Vista fare in this test.

	XP (Volume Write Back Cache Enabled) RAID 0 array
	Average	1st	2nd	3rd
ISO on the same drive	148	149	149	147
	Vista (Volume Write Back Cache Enabled) RAID 0 array
	Average	1st	2nd	3rd
ISO on the same array (in the background)	361	448	413	433
ISO on the same array (in the foreground)	218	197	223	235

(results are in seconds - lower is better)

Ouch.

Even when we 'forced' the test to run in foreground (the active task), there's no denying Vista in its present form is way slower than Windows XP. The differences are staggering - between 50 to 88 seconds or 30 to 50 percent slower! We believe this difference is caused by more than just processor overhead - its simply is too high. The situation gets worse if you're running the installation process on the background - be prepared for a very long wait (about 7 minutes or so - almost twice as long).  Well, at least we know I/O Prioritization works.

Conclusion:

Originally, we set out to compare various RAID and AHCI SATA controllers and what kind of performance we can expect with Vista. But problems during testing lead us elsewhere - this article. These preliminary testing clearly shows that kind of comparison is just not possible right now - something is still definitely broken with Vista's I/O performance. Even with the performance and reliability updates installed, performance is way off from where it should be. This mean any benchmarks we do will not be able to show the hardware's optimal performance. Microsoft's recent performance and reliability patches may help some users, but it's still far from perfect.

Go to top