Problems
Of the three games we're going to test today (F.E.A.R, Quake 4 and Serious Sam II), we have pretty much relied solely on gameplay testing session for Quake 4. While this method of testing does have higher system overhead than a timedemo, the graphics rendering is more accurate. Quake 4's timedemo is 'broken', meaning some shadows are not displayed, some textures are missing and the wrong fonts are used on OSD displays like the ones you see on elevator / lift control panels and switches. Here are some examples:
left: a misplaced OSD texture
right: notice there's shadows on the gameplay screenshot and none on the timedemo screenshot
Lately we have found results we're getting in Quake 4 don't seem to really represent frame rates we've experienced throughout the entire game, but we did choose the Data Processing Terminal (the 2nd level with the tower guardian) level because it scales pretty well with changes in resolution and graphical features. The same can also be said for Serious Sam II. With powerful graphic cards such as the GeForce 7900 and Radeon X1900 series, it's obvious we have to enable AA and AF to make the Greendale demo scale with changes in resolution. We hope by examining which level in Serious Sam II scales better than the developer supplied Greendale demo, we can use that new level for additional benchmarks in our performance evaluation.
Another problem in doing gameplay testing sessions is a boon for gamers, but a nightmare for reviewers - checkpoint saves. All three games have checkpoint saves in most of the levels. While they are helpful in playing the game, they often cause a stutter / a sharp drop in frame rate because the game needs to write data to the hard drive. Because of this, we will only relying on average frame rate for most of this article. Hopefully, we can find levels that doesn't have any checkpoints, or if there's any, can be circumvented in some way.
The test setup and settings for the game are the same as our usual standard setup and settings, an Athlon 3500+ and a GeForce 7900GT running at 520 / 700 MHz (non reference core and memory clocks). Our focus is on graphical benchmarks, so we are trying to find levels that scale graphically. Here are the steps we've taken to narrow the choices down.
- We've decided to first test each level within each game under normal playing conditions at 1024 x 768, 1280 x 1024 (or 1280 x 960) and 1600 x 1200, without AA and AF. Differences in the results between those resolutions may indicate that a level is more graphically intensive than other levels. Of course, they may also be variations between runs.
- After selecting the top five levels that scales best with resolution, we will then focus on minimizing variations between runs.
- Afterwards, we compare results taken from the game's internal benchmarking tool and compare them to the results we got. Hopefully by this time we've narrowed the choice to a single level. Tthat would make testing so much easier and quicker for the final test.
- The final test will be to see how the levels we've chosen scale in resolution, with and without AA and AF compared to our usual benchmarks.
F.E.A.R
To get an idea of what frame rates we could expect from this game, we decided to play the game from start to finish three times, each with different resolutions. Please note that these are raw numbers and these numbers will likely be different with each run. We're not focusing on repeatability or reproducible results at this moment, instead trying to get a general idea which of F.E.A.R's levels are very responsive to resolution changes.The number of pixels at 1280 x 960 and 1600 x 1200 are roughly, 1.5 and 2.4 times the number of pixels at 1024 x 768. So, in a 100 percent graphically limited situations, we expect the performance hit to be around 30 and 50 percent, respectively. Of course, that's a raw estimate and we will never be fully 100 percent graphically limited in a game. Now, let's look at those numbers
| 1024 x 768 | 1280 x 960 | 1600 x 1200 | Performance Hit 1280 | Performance Hit 1600 | ||
| Point of origin | 124.27 | 106.23 | 86.55 | 14.51% | 30.35% | |
| First encounter | 120.01 | 100.3 | 71.65 | 16.42% | 40.30% | |
| Infiltration | 129.53 | 108.44 | 81.77 | 16.29% | 36.87% | |
| Heavy resistance | 116.22 | 94.96 | 71.96 | 18.30% | 38.09% | |
| Bad water | 129.24 | 101.69 | 78.05 | 21.32% | 39.61% | |
| Exeunt omnes | 117.71 | 102.06 | 78.84 | 13.29% | 33.02% | |
| LZ is hot | 119.57 | 101.69 | 75.06 | 14.95% | 37.22% | |
| Watchers | 121.74 | 100.11 | 73.4 | 17.77% | 39.71% | |
| Bishop | 116.2 | 97.52 | 70.98 | 16.07% | 38.91% | |
| Blindside | 129.08 | 105.44 | 81.61 | 18.31% | 36.78% | |
| Sayonara, sucker | 139.8 | 110.58 | 84.02 | 20.90% | 39.90% | |
| Unauthorized personnel | 120.16 | 100.25 | 73.31 | 16.56% | 38.99% | |
| Afterimage | 121.27 | 89.63 | 72.01 | 26.09% | 40.62% | |
| Alice Wade | 110.03 | 93.04 | 68.32 | 15.43% | 37.91% | |
| Flight | 114.61 | 92.93 | 73.38 | 18.92% | 35.97% | |
| Urban decay | 106.91 | 92.64 | 64.61 | 13.35% | 39.57% | |
| Point of entry | 97.09 | 82.65 | 63.48 | 14.87% | 34.62% | |
| Lapdog | 119.29 | 97.77 | 70.84 | 18.04% | 40.62% | |
| Bypass | 143.21 | 114.22 | 83.01 | 20.24% | 42.04% | |
| The Vault | 122.46 | 102.13 | 66.34 | 16.60% | 45.83% | |
| Ground Zero | 125.41 | 94.46 | 68.44 | 24.68% | 45.43% | |
| Aftermath | 106.19 | 87.18 | 64.89 | 17.90% | 38.89% | |
| Average (entire game) |
120.45 | 98.91 | 73.75 | 17.76% | 38.69% |
all numbers are in frame rates (except for percentages)
F.E.A.R reacts so nicely to resolution changes, which means you most likely will be seeing lots of graphically limited scenes throughout the entire game. Although we can prove this by benchmarking the entire game, that would take too much time and effort. To simplify things a bit, we took the five top levels with the most performance hit at 1600 x 1200 and 1280 x 960 and put them in the table below. Undoubtedly, you'll notice some other levels such as Ground Zero and The Vault have higher performance hits that the ones we picked, but we think these levels don't really represent what we expect gameplay to be throughout the entire game.
[Previous Page]
[Go to top]
[Next Page]