Benchmarking on Trial - Part 2
Gamers usually want high enough frame rates for comfortable gameplay and that means less low frame rates and stutter (sharp drops in frame rates). Low frame rates can be caused by a variety of factors, including but not limited to graphics cards. Running benchmarks does provide an idea what kind of performance we can expect. After all, benchmarks was originally used to pinpoint system performance bottlenecks. However, benchmark results alone is not enough to make a judgment of performance.What do we actually want to achieve with a benchmark? Since we are talking about graphics cards, we're talking about graphics benchmarks. A graphics benchmark is a benchmark that focuses on graphics card's performance. So we want to measure the graphics card's performance. In this case, it would be best to remove any system limitations that may hinder the card from using all its potential. However, that's not entirely possible with game benchmarks. Even timedemo runs in game benchmarks are not immune to system limitations, but it is as close as we can get without resorting to a pure synthetic graphics benchmark. Gameplay testing sessions will more likely be heavily affected by system limitations than timedemo runs.
Experience from our last article shows us that test results from replays / timedemo runs can be significantly different at times. That's actually pretty natural because the game may be graphically intensive in some areas and more system intensive in others. The benchmark may also not be 'heavy' enough to represent the most graphically intensive situations and scenes you will encounter throughout the entire game. That's one thing we want to explore more deeply in this article.
There's also the issue of gameplay relevance. Results that show significant difference between two cards on less graphically situations is not going to accurately portray what these two cards will do in a much graphically intensive scenes. Results from a graphics benchmark that 'capture' parts of the game with the lowest frame rates (that's graphically related) is more informative in examining the differences (if any) between two cards.
The purpose? To find out what kind of performance difference we can expect between two graphics cards that have varying degrees of pixel processing power. We've narrowed down the game and the game levels we're going to use in the previous article. This time we're going to focus on finding situations and scenes where we most likely have lower / lowest frame rates in the game. However, the results must indicate that particular scene scale with more pixel processing power - a common focus point determining graphics cards performance.
For this test, we're still using the same test setup we used in the first part of the article. To vary the degree of pixel processing power, we disable one and two quads of the GeForce 7900GT we used. So, effectively we have a GeForce 7900GT with 4, 5 and 6 quads. The 4 and 5 quads setup can be considered a proxy for a graphics card that' has slightly lower pixel processing power - 33 and 16 percent slower, respectively.
F.E.A.R
First of, we have F.E.A.R. This game prove to be very scalable, graphically speaking. We ran test three times, computing an average of three runs. We also compute the standard deviation of those three runs - this is very likely the normal variations between runs. To get a better idea of how these variations may effect result, we compare the normal variations with the average as a percentage. Here are the results..FEAR Performance Test 1024
| 6 Quads | ||||||
| 1st | 2nd | 3rd | Average | Variations | Variations (Percent) | |
| Average fps | 104.82 | 105.75 | 104.51 | 105.02 | 0.48 | 0.46% |
| Minimum fps | 41 | 40 | 42 | 41 | 0.67 | 1.63% |
| 5 Quads | ||||||
| 1st | 2nd | 3rd | Average | Variations | Variations (Percent) | |
| Average fps | 101.33 | 99.95 | 101 | 100.76 | 0.54 | 0.54% |
| Minimum fps | 45 | 40 | 41 | 42 | 2 | 4.76% |
| 4 Quads | ||||||
| 1st | 2nd | 3rd | Average | Variations | Variations (Percent) | |
| Average fps | 92.6 | 94.24 | 93.47 | 93.44 | 0.56 | 0.60% |
| Minimum fps | 33 | 39 | 42 | 38 | 3.33 | 8.77% |
FEAR Performance Test 1600
| 6 Quads | ||||||
| 1st | 2nd | 3rd | Average | Variations | Variations (Percent) | |
| Average fps | 64.27 | 65.33 | 64.87 | 64.82 | 0.37 | 0.57% |
| Minimum fps | 39 | 40 | 41 | 40 | 0.67 | 1.67% |
| 5 Quads | ||||||
| 1st | 2nd | 3rd | Average | Variations | Variations (Percent) | |
| Average fps | 60 | 60.36 | 59.6 | 59.99 | 0.26 | 0.43% |
| Minimum fps | 36 | 36 | 35 | 35.67 | 0.44 | 1.25% |
| 4 Quads | ||||||
| 1st | 2nd | 3rd | Average | Variations | Variations (Percent) | |
| Average fps | 54.55 | 53.51 | 54.05 | 54.04 | 0.35 | 0.65% |
| Minimum fps | 32 | 31 | 31 | 31.33 | 0.44 | 1.42% |
all numbers are in frame rates (except for percentages)
F.E.A.R Performance Test is unique as benchmarks go. While it's still is basically a replay,. there's always some differences on each run because of the game's physics engine. Let's talk about variations between runs. At 1024 x 768, there's still some areas in the performance test where we are slightly more system bound than others. At 1600 x 1200, we're practically graphics bound all the time, that's why variations are smaller - particularly for minimum fps.
[Previous Page]
[Go to top]
[Next Page]