Tech-Hounds.com

Looking Past the Present - Part 2

Last month, we took a look at what the GeForce 7900 series has to offer in terms of longevity. While the focus was on the GeForce 7900GTX, when we compared it to the Radeon X1900 series we found the latter to be more efficient. This and the number of shader processing units on the Radeon X1900 made us conclude that the Radeon X1900 is more 'futureproof' than the GeForce 7900. Even in old games, where the GeForce 7900GTX is faster in frame rates, it progressively lost more frame rate than the Radeon X1900XTX with higher resolution and AA / AF.

In newer games, we found they're pretty much equal - the Radeon X1900XTX is much more efficient in some games, the GeForce 7900 in others. However, even we have to admit that the GeForce 7900GTX provides a higher frame rate than the Radeon X1900XTX in most of the games in our benchmark suite, at least in 1024 x 768 with AA and AF (F.E.A.R and Splinter Cell Chaos Theory). It also consistently offer slightly higher minimum frame rates, one thing gamers are very particular with. So, it looks like more testing is in order or at least more games. We need games that are more forward looking, utilizing a lot more shader processing power of these cards and less on texture processing. After much testing and retesting, we chose three other games that we think fit the bill.

Need for Speed: Most Wanted

The newest installment in the series continues the tradition started by Need for Speed: Underground - the liberal use of shaders. The game environment is similar to NFS:U 2, but with more destructible objects. ie buildings, construction site, fences etc. Most of the graphical settings in this game is not that different with its predecessors. During testing, we found we could actually push all but one setting to the max with newer cards, even with AA and AF up to 1600 x 1200.



As you can see, there is only one setting we kept to minimum - Car Reflection Update Rate. From what we've seen in the game, there's little difference between enabling and disabling this option. With this option set to minimum, we get much higher and sustainable frame rates throughout the entire game so we think this is an acceptable compromise since we don't really lose that much, visually. Since there's no saved replay, we resort to benchmarking a realtime cutscene. It's actually the first realtime cutscene in the game. There's both closeups and chase cam positions, not to mention several collisions, similar to what you find throughout gameplay.

Call of Duty 2

The long awaited patch for Call of Duty is finally here, bringing the award-winning game from developer Infinity Ward to version 1.2. This patch finally brings Punkbuster support to the game, in addition to fixes mostly related to dual core processors (patch 1.1 enables support for dual core processors). It's still somewhat hard to work with as a benchmark, but we think the hassle is worth the trouble. Graphically, the game is impressive and surprisingly, it's fast enough on a GeForce 4, though you need to turn down most graphical settings. For testing, we've decided to use a custom demo using the second level in the game - the 'Demolition' mission of the USSR campaign. This level features things you most likely see throughout the entire game: a mixture of outdoor and indoor levels, lots of AI and smokes / explosions.

Below you'll see some screenshots we've taken using basically three set of settings - the first is with the number of dynamic lights set to 'Normal' and soften smoke edges set to 'World Only'. The second screenshot is with 'High' and 'Everything' for both settings, respectively. For the third screenshot, in addition to the latter, we change the texture settings from 'High' to 'Extra'. You can definitely see that 'Extra' settings adds much more detail with higher resolution normal and specular maps.



Obviously, you want to play with 'Extra' if you want the highest image quality. The number of dynamic lights and soften smoke edges settings is more subjective, you can enable them if you want, but there is performance penalty for each and it's hardly worth it in our opinion. There's no noticeable, let alone significant differences in image quality like we're seeing with the texture settings.

Of course, when talking about image quality, we don't mean non AA AF screenshots. Below are some AA and AF screenshots to look at. The first is with texture settings set to 'High', number of dynamic lights set to 'Normal' and soften smoke edges set to 'World Only'. The second screenshot is with 'Extra' and number of dynamic lights set to 'High' and soften smoke edges set to 'Everything'.



Apart from the textures, there's not much difference between the two (discount the fog / smoke). With a 4 fps loss, we got much sharper textures mostly on the nearest MIP levels. Everything else is pretty much the same. However, if you're playing at that setting, memory use for both video and main RAM is much, much larger. That's why only 512 MB cards are recommended with 'Extra' settings. However, that's not the whole story. Higher resolution textures also eats more bandwidth, sometimes more than what's available on the fastest cards available today. After much testing, we decided against using 'Extra' texture settings, because the frame rate will drop too low (below 25 fps at 1024 x 768 with AA and AF) at some points. We're pretty sure these drops are not system limited, since they do vary with resolution changes and AA / AF. Since this is a fast paced FPS game, we want to maintain at least a 25 fps minimum. So, we're using a much 'conservative' setting to benchmark Cal of Duty, which you can see below.

The Elder Scrolls IV: Oblivion

By now, most of you will probably be sick to death hearing about this game. Well, bare with us since this is 'THE GAME' of 3D graphics review this month (and quite possibly this year). Oblivion is very demanding on your PC's processor, memory and graphics cards, all at the same time. With a variety of graphical settings, there's a plethora of tweaks you can use to bring your GeForce 7900 or Radeon X1900 cards to its knees. Graphically, it's the most impressive game out this year, especially with High Dynamic Range rendering enabled.

For those of you who are not interested in discussing the quirks of Oblivion, you can jump to the settings we've chosen by clicking this link.

Despite the graphical splendor in the game, Oblivion is not without some quirks. For one, it's also a very system limited game. Most of the time, you'll see a very large drop in frame rates when you're viewing a character (even yourself). If you played the game, you'll know what we're talking about. Below is one example of this. This is the Arch-Mage's room in the Arcana University , quite possibly one of the smallest levels in the game (after all it's just one room). In first person mode, you'll get more than 150 fps here, regardless of whatever direction you're looking. However, switch to third person mode, you'll see the frame rate drops to around 70 fps. Actually, that's a 100 fps drop! Granted, there's more objects in the second screenshot, but that drop is just too much to be graphics related.



How about lots of characters? Well, if you play the game and follow the story, you'll see at least one or two scenes with lots of characters. One of the heaviest scenes is quite possibly the second battle of Bruma, the one with the Great Gate to Oblivion. You can see a screenshot we took below of that scene. Despite a very bland landscape with very little objects and quite far characters, we're seeing less than 30 fps.



There are two things we can do to make Oblivion a graphics benchmark. For one, we can use a fast enough processor to keep up with both the AI and graphics cards demands. However, we don't think this is the answer. Even with a faster processor, some scenes will likely be always system limited, particularly if there are characters in view. So, we choose another alternative - we simply choose scenes with no characters at all. We know that this hardly captures the gameplay of Oblivion, so in this regard our benchmarks is not fully representative of gameplay. This is a compromise, but it is a necessary one, otherwise the results will be system limited and won't tell us anything, graphical wise.

With that out of the way, let's take a closer look at the game. There's a plethora of graphics options to use, so let's look at them one at a time. Below you'll see some screenshots. It's the first glimpse you get of the land of Tamriel, just right outside the Imperial Prison sewers - right in front of the door actually. The first screenshot is with maximum detail, with no AA and AF or HDR. The next shots are taken with each setting slider set to minimum, beginning with 'Tree' (2nd screenshot), 'Actor' (3rd), 'Item' (4th), 'Object' (5th), 'Grass' (6th) and 'View' distance (7th).





The first four screenshots are pretty similar, so in this scene we can actually 'get away' with 'Tree', 'Actor', and 'Item' set to minimum. However, when the 'Object' slider is set to minimum, we begin to see a major difference in detail. Most of the objects are not rendered at all! Setting 'Grass' to minimum isn't that much different, but the lowest quality, image wise is of obviously 'View' set to minimum. So, it looks like to get the 'full effect', we need to keep 'Item', 'Object' and 'View' sliders at their maximum values or find a fair compromise.

Now, let's look at another scene, this time with a view of the Imperial City from near Harm's Folly. The screenshots below are arranged the same way as before - full detail, minimum 'Tree', 'Actor', 'Item', 'Object', 'Grass' and 'View'.



We can see, that setting 'Grass' to minimum means not rendering it at all. So, from the looks of it, there's only two settings we can 'minimize' without too much affecting gameplay and image quality - 'Tree' and 'Actor'. We'll be looking at the 'Actor' setting a bit more later, for now let's examine just how much compromise we can make with 'Item' / 'Object' and 'Grass'.



We move the sliders about a quarter at a time, so the first screenshot is about 3/4, the second is 2/4, the third is 1/4 and the fourth is minimum (none). Even at 3/4 setting, you can see some objects disappearing, though not completely. At 2/4, you begin to see some of the rocks fully disappearing, and at 1/4, they all disappear. Gameplay wise, you might get away with 2/4, but then you'll notice one of the more annoying quirks of Oblivion and the GameBryo engine - objects popping out of thin air in front of you. So, for our benchmark settings, we decided to set 'Item' and 'Object' at their maximum values. While objects popping out can still happen, its not as often or as annoying.



Just like before, these screenshots are taken at about 3/4, 2/4, 1/4 and minimum (none) settings.While the screenshots don't show it, even at full settings, you can see the grass sprucing up on the far hill on the left when you walk or run forward. With each degraded setting, we see less and less grass - that means you'll notice the grass sprucing up more often. We recommend 2/4 for comfortable gameplay, but for our benchmark setting, we chose to set the 'Grass' slider at maximum.



The 'Actor' slider actually controls the distance of how far the actor must be for the game not to render them. For you snipers out there, see if you can hit a target at the maximum setting. No zooming now. Did you hit it? Cool!!! Seriously now, for the rest of us, we can get away with a bigger compromise. Like the previous shots, the screenshots are taken with the slider set to about 3/4, 2/4 and 1/4, in that order. Unlike objects and items, actors don't pop up, but fade. Since actors / characters is a system limiting factor, we choose a setting of about 1/4 for graphics benchmark purposes. This way we can limit the rendering actors, so the scenes stays pretty much graphics limited than system limited.



Textures sizes are also a big problem with Oblivion. In fact, one of the very first mods to appear was a texture and normal map LOD replacement. Even with the texture size to 'Large' you can see there's very little detail if the objects are faraway. To maintain the level of detail in outdoor scenes, there's no compromise. We lost so many detail even at 'Medium', and 'Small' looks very much like, well, crap. Needless to say, we chose 'Large' for benchmark purposes.



In indoor and city scenes, you can actually get away with 'Medium' most of the time. Look at the store sign, it's still pretty much readable at 'Medium'. If you flip at these images, you'll notice that most of the difference between them are on the ground textures. Looks like Bethesda is very conservative on texture usage, with some very progressive MIP levels.





And at last, the Bloom and HDR screenshots. We enable bloom for both default and AA / AF benchmarks. For gamers with SM 2.0 cards, this is the highest image quality they can have. HDR is only supported with SM 3.0 cards and it will not work with AA, even with ATI's Radeon X1K series cards. There is a way around this. You can force it from the driver panel once you've installed the 'Chuck' patch, available on ATI's website. Currently, It's the only way to get HDR with AA in Oblivion. Although it works and quite beautifully we might add, we still much prefer Bethesda releasing their patch to enable HDR with AA, so until then, we will not be providing benchmarks results of HDR with AA in Oblivion. For benchmarks purposes, we will test HDR with AF only.

There are a lot of other graphical settings in Oblivion, but to keep this article short, we won't discuss them all. Out of these other settings, we disable two for our benchmarks - 'Self Shadows' because,well, it doesn't work quite right and 'Shadows on grass' which is very heavy on NVIDIA cards and causes texture flashing on ATI cards. All other settings are set to their maximum values (Full, High or On).



To keep the results 'relevant' to gameplay, we choose three scenes with elements you'll find everywhere in the world of Oblivion. The first scene is in Imperial City Market District, walking around the level from the spawn point. With several AI in the scenes, we expect this scene to be very system limited. The second scene is the one we've shown with the grass, near Harm's Folly. With lots of grass, there's actually very few objects or trees in this scene. The third one is what we call the Tree test, with both generous amounts of trees and grass. This scene is in the Great Forest area, where we walked south from Fort Coldcorn, heading towards Glademist Cave. All benchmarks are done in first person mode, with the above settings.

Performance

AA and AF settings are applied in game, whenever possible. We had to apply AF from the driver panel for NFS:MW and Oblivion. In Call of Duty, we enable both AA and AF from the graphical menu, but we change the number of AF samples to 16 with the command 'r_anisotropy' set to 16. V sync was disabled both from inside the game and on the driver panel. Image quality settings was set to 'High Quality'. In addition to the Radeon X1900XTX and GeForce 7900GTX, we also tested a slightly higher clocked X1800XT from ASUS, the EAX1800XT TOP for comparison purposes.

We'd like to thank both Tagan and Kingston for supplying with the additional power supply and 1 GB memory modules for this article.

Our test setup
AMD Athlon 64 3500+ socket 939
2 x 1024 MB Kingston KVR 3-3-3 PC3200 DDR-SDRAM
MSI K8N NForce 4 SLI motherboard
GeForce 7900GTX DDR3 512 MB graphics card
Radeon X1900XTX DDR3 512 MB graphics card
ASUS EAX1800XT TOP DDR3 512 MB graphics card
(running at standard clocks - core 700 MHz / memory 800 MHz (1600 MHz effective))
Maxtor DiamondMaxPlus9 80 GB Serial ATA 8 MB buffer
ASUS E-616 DVD-ROM
Tagan TG530-U15 530 watts ATX/BTX power supply

Windows XP Professional with Service Pack 2 installed
ATI Catalyst 6.4 reference driver
NVIDIA Forceware 84.21 reference driver
NVIDIA NForce 4 6.70 reference driver
Creative SoundBlaster Live! 24 bit 5.12.1.512 driver.
DirectX 9.0c

Performance

The results:

The graphs are pretty self explanatory, but in case you can't see the text, they're arranged in groups of three colors: green for minimum fps, blue for average fps and red for maximum fps. The first group represent test results under default settings, and the second represent results with 4xAA and 16xAF enabled.The third group, if present, are HDR with AF results. These results are in frame rate per second.

NFS: Most Wanted

It does look like the Radeon X1900XTX is faster than the GeForce 7900GTX, but the differences can still be attributed to normal variations between runs, at least at 1024 x 768 and 1280 x 1024 (2 - 3 fps). It's only at 1600 x 1200 do we see a confirmation of this - while both cards' average fps is nearly identical, the minimum fps is higher on the Radeon X1900XTX, much like it is at 1024 x 768 and 1280 x 1024. If that's not enough proof for you, let's at how these three cards progress through the benchmark.

	GeForce 7900GTX		EAX 1800XT TOP		Radeon X1900XTX
	Default	4x AA 16x AF	Default	4x AA 16x AF	Default	4x AA 16x AF
<30 fps	2	4	3	10	0	3
30-45 fps	16	19	23	32	13	28
45-60 fps	18	18	17	4	24	13
60-90 fps	11	6	4	1	10	3
90-120 fps	0	0	0	0	0	0
>120 fps	0	0	0	0	0	0
Total	47	47	47	47	47	47

results are in seconds

In case you're wondering, the table above is actually a brokedown of the benchmark results, displaying the time spent throughout the replay organized into several different frame rates ranges. Because this is a replay and not a timedemo, the replay runtime is the same between all cards. With a timedemo, the total number of seconds will vary according to settings, ie a heavier setting will have a higher number of seconds because the benchmarks takes longer to finish.

If you look at the progress graph, there's at least two areas that's quite interesting. On the first drop, the Radeon X1900XTX offers a higher frame rate than the GeForce 7900GTX, but on the last drop, the two are virtually neck to neck with the Radeon X1900XTX just edging out the GeForce 7900GTX. Unfortunately, the graph can't show what we can see in the table - throughout the rest of the gameplay, the GeForce 7900GTX spent less time between 30 - 45 fps than the Radeon X1900XTX.

Now, this becomes much more subjective. Do you prefer a slightly higher fps but with some sharper drops of the GeForce 7900GTX or the slightly lower fps but more shallow drops of the Radeon X1900XTX? Personally, we like the more 'sustained' frame rate of the Radeon X1900XTX better, because we notice less of a lag / stutter when playing. Frame rate wise, we think these results is convincing enough to say that both cards can still handle this game quite well and are evenly matched.

NFS: Most Wanted

	7900GTX			X1800XT TOP			X1900XTX
	Default	AA AF	Performance Lost	Default	AA AF	Performance Lost	Default	AA AF	Performance Lost
1024
Min	30	31	-3.33%	29	33	-13.79%	33	33	0.00%
Avg	55.64	56.91	-2.29%	57.98	57.83	0.26%	60.49	58.22	3.75%
Max	79	79	0.00%	76	78	-2.63%	78	75	3.85%
Average			-1.87%			-5.39%			2.53%
1280
Min	30	28	6.67%	31	29	6.45%	30	32	-6.67%
Avg	54.81	52.13	4.88%	53.95	46.77	13.31%	58.14	53.27	8.39%
Max	73	73	0.00%	78	76	2.56%	74	77	-4.05%
Average			3.85%			7.44%			-0.78%
1600
Min	26	24	7.69%	22	20	9.09%	32	26	18.75%
Avg	50.45	45.33	10.14%	44	36.78	16.42%	51.5	43.89	14.79%
Max	73	77	-5.48%	72	69	4.17%	76	73	3.95%
Average			4.12%			9.89%			12.50%
Weighted Average			2.03%			3.98%			4.75%

From the frame rates, it's easy enough to conclude that both the GeForce 7900GTX and Radeon X1900XTX is fast enough for comfortable gameplay at least up to 1280 x 1024 in NFS: Most Wanted with virtually maximum detail. At higher resolutions, the Radeon X1900XTX slightly edges the GeForce 7900GTX with its higher minimum fps, but you probably will still notice the drop anyway.

Due to the variability of results (from normal variations between runs), it's hard to estimate just how much performance we lost with higher resolutions and AA / AF. The trend seems to show that the GeForce 7900GTX is more efficient here. However, if you look at the results from the EAX1800XT TOP, you'll notice it's also slightly more efficient than the standard Radeon X1900XTX, so maybe all the Radeon X1900XTX needs is a slightly higher clocked core and memory. Or at least more time for the driver developers to 'fine tune' the driver for the Radeon X1900XTX. This is most likely true here, particularly with a sharp drop in efficiency at 1600 x 1200. It's too large a drop , so it can't be just normal variations between runs.

With this in mind, we can't really say from this benchmark alone which one will fare better with future games. So, lets hold our conclusions and look at the next game in our benchmark suite, Call of Duty 2.

Call of Duty 2

Right of the bat we can all see that Call of Duty basically runs faster with the Radeon X1K series. Even the EAX1800XT TOP is outperforming the GeForce 7900GTX here. Of course, you may want to reconsider playing this game at such high resolutions, all three cards is definitely not offering high enough frame rate for gameplay. Remember, this is with texture settings set to 'High', not 'Extra'.

The highest resolution you should choose is probably just 1024 x 768, or maybe you could put up with 1280 x 1024 with the Radeon X1900XTX. At 1024 x 768, the Radeon X1900XTX offers a 3 minimum fps lead with only 2 fps drop when AA and AF is enabled. Meanwhile, the GeForce 7900GTX can barely reach the 30 minimum fps mark and loses nearly 8 fps (a 10 percent drop) in average fps. It would be interesting to see how this benchmark behave with SLI and Crossfire, but that's not what we're interested in right now (another article perhaps?). Let's see each card's progress throughout the benchmark

	GeForce 7900GTX		EAX 1800XT TOP		Radeon X1900XTX
	Default	4x AA 16x AF	Default	4x AA 16x AF	Default	4x AA 16x AF
<30 fps	0	1	0	0	0	0
30-45 fps	43	47	30	36	30	32
45-60 fps	37	70	45	71	47	48
60-90 fps	121	91	123	105	119	121
90-120 fps	27	31	32	26	32	28
>120 fps	17	5	15	7	17	16
Total	245	245	245	245	245	245

results are in seconds

Of course, what we're more interested in is the lowest frame rates, so focus on the last parts of the graph. It may look like its too close to call, but we can see that even the Radeon EAX1800XT TOP consistently offer higher frame rates here. It's not much (2-3 fps), but it's enough to matter. Now, if you look at the other parts of the benchmark, you'll also see that in most drops, the trend persists. The numbers we're seeing the table confirms this - the GeForce 7900GTX spent more than 40 seconds between 30 - 45 fps compared to 30 something seconds on both the Radeon X1900XTX and the EAX1800TX TOP. Hell, we could say that we're not really seeing that much frame rate difference between running with AA and AF and without on the Radeon X1900XTX.

Unlike NFS: Most Wanted, we're seeing a different picture here. The Radeon X1900XTX is not only offering more shallow drops, but also higher fps on average. Since we're also seeing the same thing even with the EAX1800XT TOP, its a safe bet that the Call of Duty 2 graphics engine is more suited to ATI's design than NVIDIA's here - most likely the game is much more shader limited than texture limited with our choice of settings. Now, let's look at the efficiency of each card.

Call of Duty 2

	7900GTX			X1800XT TOP			X1900XTX
	Default	AA AF	Performance Lost	Default	AA AF	Performance Lost	Default	AA AF	Performance Lost
1024
Min	29	29	0.00%	34	31	8.82%	33	33	0.00%
Avg	72.58	64.95	10.50%	72.97	67.72	7.20%	74.42	72.37	2.76%
Max	163	135	17.18%	158	151	4.43%	171	167	2.34%
Average			9.23%			6.82%			1.70%
1280
Min	27	21	22.22%	30	24	20.00%	32	26	18.75%
Avg	60.37	49.28	18.37%	58.57	51.56	11.96%	64.8	54.65	15.67%
Max	120	100	16.67%	113	104	7.96%	131	117	10.69%
Average			19.09%			13.31%			15.04%
1600
Min	21	17	19.05%	21	19	9.52%	25	19	24.00%
Avg	47.6	38	20.18%	45.96	39.04	15.05%	52.25	42.05	19.52%
Max	91	72	20.88%	89	79	11.24%	98	84	14.29%
Average			20.04%			11.94%			19.27%
Weighted Average			16.12%			10.69%			12.00%

Overall, the results are not unexpected, but they're certainly worth looking at more closely. We can see all cards dropping significantly when we raise the resolution from 1024 x 768 to 1280 x 1024. Surprisingly, its the Radeon X1900XTX that has the sharpest drop - from close to 2 % to 15 %.Moving to 1600 x 1200 adds a 4 % penalty, much bigger than 1 % on the GeForce 7900GTX. We believe this is bandwidth related, look at the EAX1800XT TOP results, it actually lost less than the standard Radeon X1900XTX at 1280 x 1024 and 1600 x 1200. This is also confirmed by the GeForce 7900GTX results - both the GeForce 7900GTX and EAX1800XT TOP has the same memory clock.

Obviously, ATI's decision to push shader processing on the Radeon X1900 series paid off. However, their decision to scale back the memory clock is costing them performance. At higher resolutions, particularly with AA and AF, shader processing becomes less of a bottleneck. Even the most efficient of designs will not be that useful if the bottleneck is caused by external factors - memory bandwidth. No doubt, a refresh of R580 will likely rectify this problem. For now, if you're adventurous enough, you could always overclock the card to have at least the same bandwidth as the GeForce 7900GTX.

The Elder Scrolls IV: Oblivion