Tech-Hounds.com

The Quest for Lag Free Gaming - Part 2

In the first part of this article (which you can read here), we looked into what we can do to avoid or at least minimize memory related stuttering. That article shows that most stuttering issues are caused by not having enough memory. The solution was easy enough - supply enough memory for the games and applications you're running. That means real memory (RAM) and not just virtual memory. We want our games with frame rates as high as possible, so we don't have the luxury of waiting for hard disk accesses with virtual memory usage.

But how about storage related stuttering? At one time or another, even PCs with 2 or 4 GB of RAM must access the hard disk, either to load new levels or saving game saves. Having more RAM does make for faster loads, particularly in newer games, but even the fastest RAM won't do much good if you have a too slow hard disk. With game saves, that's even more important - we want the hard disk to write them as fast as possible. Smart developers 'coped' with this problem by forcing us to access a menu and / or pausing the game every time we want to save or load a save game. This way, we won't notice the stutter caused by slow hard disk access. However, bringing up a menu every time you want to save a game does 'interrupt' the gameplay experience. So, more and more games are using an auto save feature, where automated game saves occur at different points of the game. Quick saving and loading works pretty much the same way - you skip the menu (but not the hard disk access).

With action oriented games where you need high frame rates, saving and loading will undoubtedly cause stutter or at least a significant frame rate drop. So, what can we do about it? There are a few steps you can take (in addition to having more memory) - get a faster hard drive or use a RAID 0 (stripe) array. The fastest hard disk for desktop users right now is undoubtedly Western Digital's Raptors. These 10K rpm hard disks have the fastest access time, but they also came with quite a hefty price tag and smaller capacities than most 7200 rpm drives. On the other hand, most chipsets and motherboards now come with SATA controllers supporting RAID 0 and 1. Since we don't have any of WD's Raptors right now, we're going for the latter option - two small capacities 7200 rpm SATA hard disks in one RAID 0 array. We think this is the most inexpensive way to boost storage performance without losing much capacity. So let's see whether or not RAID 0 is the right choice for lag and stutter free gaming.

Testing Preparations

During testing for the first article, we saw which games are prone to stuttering issues and which games are not. We can actually separate them into two groups - those which ran just fine with 512 MB and those who needs up to 2 GB or RAM. Here is the table from the first part of this article.

	Windows XP	Battlefield 2	Brothers In Arms	Call of Duty	Dungeon Siege	Dungeon Siege 2
Memory available	382.55	39.47	181.18	12.14	287.68	124.56
Memory installed	512	512	512	512	512	512
Memory used	129.45	472.53	330.82	499.86	224.32	387.44
Page file used	172	1032	314	445	256	406
Page file normal	172	172	172	172	172	172
	0	860	142	273	84	234
Total memory used (estimated)	129.45	1332.53	472.82	772.86	308.32	621.44

	Windows XP	F1 Career Challenge	FEAR	Full Spectrum Warrior	Homeworld 2	Lock On
Memory available	382.55	168.9	178.96	365.41	169.41	142.15
Memory installed	512	512	512	512	512	512
Memory used	129.45	343.1	333.04	146.59	342.59	369.85
Page file used	172	355	556	320	373	406
Page file normal	172	172	172	172	172	172
	0	183	384	148	201	234
Total memory used	129.45	526.1	717.04	294.59	543.59	603.85

	Windows XP	Nascar 2003	Quake 4	Richard Burns Rally	Splinter Cell	Splinter Cell Chaos Theory
Memory available	382.55	249.26	13.68	288.52	299.66	78.21
Memory installed	512	512	512	512	512	512
Memory used	129.45	262.74	498.32	223.48	212.34	433.79
Page file used	172	260	656	322	242	415
Page file normal	172	172	172	172	172	172
	0	88	484	150	70	243
Total memory used (estimated)	129.45	350.74	982.32	373.48	282.34	676.79

	Windows XP	Serious Sam II	SW: KOTOR
Memory available	382.55	36.84	266.04
Memory installed	512	512	512
Memory used	129.45	475.16	245.96
Page file used	172	472	266
Page file normal	172	172	172
	0	300	94
Total memory used (estimated)	129.45	775.16	339.96

all numbers are in MBs

Of all these games, not all of them can be used in this article. We need not only games with various memory footprints, but also those with both an automated and quick save features. Out of these games, we choose Call of Duty, Dungeon Siege, F.E.A.R, Homeworld 2, Quake 4, Serious Sam II and SW: KOTOR. In particular interest are Dungeon Siege, F.E.A.R and Quake 4 - Dungeon Siege (and Dungeon Siege II) still exhibits some stuttering even with 2 GB of RAM - most likely due to level loads while F.E.A.R and Quake 4 have the largest memory footprint (sans Battlefield 2 that we can't use in this article). On a side note, you really have to hand it to Croteam developers - even with that large memory footprint, the game runs very well with no stuttering in a 512 MB system. So, we're going to separate them into two groups - Dungeon Siege, Homeworld 2 and SW: KOTOR are games that already runs well enough with 512 MB, while Call of Duty, F.E.A.R, Quake 4 and Serious Sam II are games that are theoretically needs more than 512 MB of RAM.

How to 'capture' stuttering

While it's easy enough to display stuttering in real life, showing stuttering in a written article is not as easy. It's very hard to do since most stuttering issues can be very unpredictable both in length and when it occurs - that means they're pretty unrepeatable. So, we ask ourselves, just how can we capture stuttering. Here's what we come up with.

Most random stuttering issues are caused by not having enough memory (barring driver and game bugs). So, we're going to test these games again with both 512 MB and 2 GB MB systems. Games with larger than 512 MB memory footprint should force Windows to swap some memory into the page file - and of course, hard disk accesses. To 'capture' stuttering, we'll focus on the first test run results - where stuttering will most likely occur. We'll compare them to average test results from several 'stutter-free' runs (from the last article) to see whether or not there's a difference. A difference in numbers means more likely the games has stuttering issues. Or more appropriately, we have captured the stuttering. Next, we'll look at the fps progress graph to see when and just how much frame rate drops are causing the lower than average test results. Since we want to see whether or not RAID 0 helps stuttering issues, we'll ran the test again with both systems, but with a RAID 0 array. Of course, we also watch the tests as they are run to see whether or not there's any stuttering. Needless to say, this method really only works well with games that have larger than 512 MB memory footprints.

Thankfully, capturing stuttering caused by level loading, automated saves and quick saving are easier - at least we know when they will occur. It's just a matter of measuring how long. To do that, we measure the time it takes for the game to complete saving files to the hard disk - the shorter the better. There are variances with each saves, so we repeat this process three times and compute an average. Level loads measurement are done pretty much the same way. We also did a quick gameplay session to see whether or not we notice any stuttering during these saves. We know it's far from perfect, but it will have to do for now.

Performance Issues with RAID 0

Using RAID arrays with RAID controllers are not without same caveats. The most relevant here are not just storage performance, but also overall system performance. RAID controllers tend to have higher overheads than ordinary SATA or IDE controllers. That means they take more processor time, so your game might run slower since it gets less processor time. We'll take a look at just how much performance difference are there between using ol' plain, ordinary SATA and SATA RAID 0 arrays.

Performance

Just as before, we have to use a system where performance differences can be easily shown. We reused our test setup from the last article, replacing the single hard drive with an RAID 0 array for RAID 0 testing. A reference clocked GeForce 7800GTX which is system limited with this setup, is again used, with a resolution of 1024 x 768, 32 bit - no AA, AF or any other settings are enabled. This also mean not enabling HDR on Splinter Cell: Chaos Theory and Serious Sam II, no Ultra Settings in Quake 4, or Soft Shadows in F.E.A.R. All other settings are maxed out - full detail.

We'd like to thank both Tagan and Kingston for supplying with the additional power supply and 1 GB memory modules for this article.

Our test setup
AMD Athlon 64 3500+ socket 939
2 x 256 MB Kingston KVR 3-3-3 PC3200 DDR-SDRAM
4 x 1024 MB Kingston KVR 3-3-3 PC3200 DDR-SDRAM
MSI K8N NForce 4 SLI motherboard
ASUS EN7800GTX TOP 256 MB GeForce 7800GTX graphics card
(running at reference clocks - 430/600 MHz)
2 x Maxtor DiamondMaxPlus9 80 GBs Serial ATA 8 MB buffer
ASUS E-616 DVD-ROM
450 watts ATX power supply
Tagan TG530-U15 530 watts ATX/BTX power supply

Windows XP Professional with Service Pack 2 installed
NVIDIA Forceware 81.98 reference driver
NVIDIA NForce 4 6.70 reference driver
Creative SoundBlaster Live! 24 bit 5.12.1.512 driver.
DirectX 9.0c

The results:

Looks like there's no point of using more than 512 MB and / or RAID 0 in Dungeon Siege. As a matter of fact, we're actually running the game slightly slower, most likely because of the RAID controller's overhead. It's not much and hardly noticeable in gameplay, but is is slower. So, what about the stuttering with level loads in this game? Well, the stutter is still noticeable, even with 2 GB RAM and RAID on your system.


Unfortunately, the FRAPS fps counter can't really capture the stutter because it was much too quick (under a second). We have to turn to the in game benchmark tool. According to the log, the minimum fps during the benchmark was 6 fps, most likely the frame rate when the stutter occur. So, it looks like having more memory or RAID 0 won't help at all with this particular stutter.


Homeworld 2 is another game that already runs pretty well with 512 MB RAM. Its not surprising to see there's practically no improvement at all from using RAID 0. Like Dungeon Siege, we're actually running slower with RAID 0, though it's nothing significant. We didn't notice any stutter in this game, with or without RAID 0. Let's move on.


We seem to have gain a very small increase in frame rates with RAID 0. Even with repeated runs the increase stays relatively the same. It's too small to matter though, only about 2 - 3 fps at most. During testing, we didn't see any noticeable stuttering. That's about it for games with small memory footprints. What about those with footprints larger than 512 MB? Let's find out.


Call of Duty already ran pretty well with a 512 MB system but it still can gains a little bit of performance with larger memory. RAID 0 is a different story. The RAID controller's overhead is again causing lower frame rates. One thing to note, while we didn't see any random stuttering, we did notice a little stutter when saving games on a 512 MB system. With 2 GB of RAM and RAID, the stutter is definitely gone. So, it looks like that there is something to be gain in games with RAID 0.


You can definitely see 512 MB isn't really enough for F.E.A.R with Texture and Video Resolution set to maximum. It's enough for the performance test, but stuttering was everywhere in actual gameplay. You really need 2 GB of RAM and a high end graphics card to play this game on the highest settings. It's interesting to note that RAID 0 does indeed help frame rates, somewhat - we're certainly seeing higher frame rates on the 2 GB systems. Let's look at the fps progress graph.


You can see the large drop in the beginning of the benchmark with the 512 MB system. Oddly enough, on various points of the benchmark, the 512 MB system with RAID is faster than the others. F.E.A.R definitely appreciates the use of RAID 0.


Here, we're seeing pretty much the same story with Quake 4, although by a smaller degree. Using RAID 0 may help, but it's still no substitute for real RAM. Look at the minimum frame rates - 20 to 60 fps is a big difference. Even with RAID 0, you can still notice stuttering during the first run. On the second run, with all of textures and objects loaded, there's no visible stuttering.


It's hard to believe these graphs come from the same system, the only difference being the amount of memory and hard disk configuration. The graph may look a little strange, but that's completely normal. Playing the game with 'com_fixedTic' set to '1' means the game will try to run as fas as possible, much like a timedemo. Look at how much frames we're losing - at one point we went from about 120 fps to around 50 fps.


There's little difference here, not surprising since Serious Sam II is very system limited with no AA, AF or HDR on the GeForce 7800GTX. We didn't gain much with 2 GB of RAM in this game, so it shouldn't be a surprise to see the same thing happen with RAID 0. Now, let's look at load times on these games.

	In msec				In percent
	512 MB	2 GB	512 MB RAID	2 GB RAID	512MB to 512 MB RAID	2 GB to 2 GB RAID	512 MB to 2 GB RAID	512 MB to 2 GB
Call of Duty	5061	4910.33	9208.33	6353	81.95%	29.38%	25.53%	-2.98%
FEAR	33988.33	3594	29677.33	8205.67	-12.68%	128.32%	-75.86%	-89.43%
Quake 4	256521	19782	75129.67	22877.33	-70.71%	15.65%	-91.08%	-92.29%
Serious Sam II	9960.33	8270.33	10131	7848	1.71%	-5.11%	-21.21%	-16.97%
Average					0.07%	42.06%	-40.65%	-50.41%

These are load times, so the smaller the number, the better. Negative percentage means gains here. RAID 0 does offer some improvement in this case. However, not all games will see it. Call of Duty actually loads slower, regardless of configuration. We can also see there's no improvement from combining 2 GB of RAM and RAID 0 (except for Serious Sam II). That gain alone can come just from having more RAM. How about saving games then?

	In msec				In percent
	512 MB	2 GB	512 MB RAID	2 GB RAID	512MB to 512 MB RAID	2 GB to 2 GB RAID	512 MB to 2 GB RAID	512 MB to 2 GB
FEAR (Quick)	1301	1131.33	646.33	789.33	-50.32%	-30.23%	-39.33%	-13.04%
FEAR (Checkpoint)	2973.5	2422.33	2497	2404.33	-16.02%	-0.74%	-19.14%	-18.54%
Quake 4 (Quick)	847	632.67	498	560.33	-41.20%	-11.43%	-33.84%	-25.30%
Quake 4 (Checkpoint)	463.33	528.33	189	164	-59.21%	-68.96%	-64.60%	14.03%
Serious Sam II (Quick)	599.67	982.33	1102.33	523.33	83.82%	-46.73%	-12.73%	63.81%
Homeworld 2 (Quick)	959.67	1034	791	964.33	-17.58%	-6.74%	0.49%	7.75%
Homeworld 2 (Checkpoint)	1515.67	1427.33	1475.33	1398