| 1024 x 768 | 1280 x 1024 | 1600 x 1200 | Performance Hit 1280 | Performance Hit 1600 | ||
| Perimeter Defense Station | 90.54 | 89.48 | 72.86 | 1.17% | 19.76% | |
| Hangar Perimeter | 89.5 | 86.01 | 72.4 | 3.89% | 19.88% | |
| Interior Hangar | 98.48 | 92.06 | 82.81 | 6.53% | 15.92% | |
| The Nexus | 82.13 | 79.84 | 69.34 | 2.79% | 16.03% | |
| MCC Landing Site | 76.83 | 76.59 | 65.27 | 0.31% | 15.05% | |
| Average | 87.5 | 84.8 | 72.53 | 2.94% | 17.33% |
all numbers are in frame rates (except for percentages)
Ah, that's more like it. Do note that frame rates in the table above of these levels are generally lower than what you see throughout the entire game, though it's still fairly close. The performance hit of the various levels at 1280 x 1024 range from nothing to slightly higher and at 1600 x 1200 they are pretty much similar. The Interior Hangar level stands out as the one with the largest hit at 1280 x 1024, but The Nexus's performance hit is pretty close to the average. The Nexus also has the second lowest frame rate of the five, the MCC Landing Site being the first. However, we still need to confirm these results with more testing - are they repeatable? Let's look at the next table.
These results are taken at 1024 x 768, without AA and AF.
| 1st | 2nd | 3rd | Average | Variations | Variations (%) | ||
| Perimeter Defense Station | 89.85 | 89.17 | 89.78 | 89.6 | 0.37 | 0.41% | |
| Hangar Perimeter | 85.79 | 84.48 | 83.2 | 84.49 | 1.3 | 1.54% | |
| Interior Hangar | 90.98 | 93.23 | 90.39 | 91.53 | 1.5 | 1.63% | |
| The Nexus | 79.41 | 75.29 | 77.39 | 77.36 | 2.06 | 2.66% | |
| MCC Landing Site | 79.11 | 72.29 | 70.92 | 74.11 | 4.39 | 5.92% |
all numbers are in frame rates (except for percentages)
Only the Perimeter Defense Station results are pretty close to our initial results. The other levels are roughly 2 to 6 frames lower. Remember, we wanted a worst case scenario that really pushes the graphics card. The good news is the results from these levels are repeatable - only the MCC Landing Site level have slightly larger variations between runs. The Nexus level remains the one with the second lowest average frame rate.
Alright, now we're going to see whether or not there are differences between frame rate in actual gameplay and a timedemo run. Since we're measuring frame rate in gameplay testing sessions, we are playing the game with all the bells and whistles on - AI, sound, physics etc. Differences with timedemo runs are expected, but it sure is good to see just a small difference between gameplay testing sessions and timedemo results. In the table below, you'll see the average of three runs compared to a timedemo run for each of the levels we've picked. Just to make sure FRAPS is working correctly, we've included results from both Quake 4 internal timedemo statistics and FRAPS.
These results are taken at 1024 x 768, without AA and AF.
| Gameplay | FRAPS benchmark | internal |
Difference | Difference (%) | ||
| Perimeter Defense Station | 89.6 | 104.31 | 104.2 | 14.66 | 16.36% | |
| Hangar Perimeter | 84.49 | 91.37 | 91.5 | 6.95 | 8.22% | |
| Interior Hangar | 91.53 | 88.36 | 88.45 | -3.13 | -3.42% | |
| The Nexus | 77.36 | 77.72 | 77.5 | 0.25 | 0.32% | |
| MCC Landing Site | 74.11 | 68.48 | 68 | -5.87 | -7.92% |
all numbers are in frame rates (except for percentages)
We'll that's - weird. We usually expect timedemo results to be higher than gameplay testing sessions results with the absence of sound (and some texture errors). But we sure didn't expect timedemo results to actually be lower. From the looks of it, the only level that stays fairly close is The Nexus The results from the other levels is too different (or too strange) from a timedemo run. It's probably safe to say that you shouldn't rely only on timedemo results as a measure of actual gameplay performance with Quake 4. Timedemo results can actually be very different to frame rates in actual gameplay.
After more detailed testing and some additional research, this weird behavior with Quake 4 timedemo is actually not weird at all. Timedemo don't run AI, sound, physics routines, they're straight graphical instructions for the graphics cards to render the game's graphics. That means it's very close to a pure graphics stress test, though processor, memory and bus speeds / bandwidth still have some influence on the results. By taking out the burden of AI, sound, and physics from the processor, the processor can focus most of it's processing power to supplying the graphics cards, which in turn tries to render the game as fast as it can - hence the word 'timedemo'. In our case, higher results from certain levels indicate that we are system limited in those levels. Vice versa, a lower results may indicate we are graphically limited (pay attention to the word 'may').
So, out of the five levels we've picked, the choice comes down to two - Interior Hangar and The Nexus. We've decided to choose the Nexus for gameplay testing sessions - the primary reason is the results indicate that this level have close to equal load on both the system and the graphics subsystem. There are other points in favor of The Nexus level - it lacks a checkpoint save which makes it ideal to get minimum frame rates thus no hard disk writes that will cause a stutter or a sharp drop in frame rates. After all, we believe a lower minimum fps is more of a concern than a lower average fps. Generally speaking, average fps from The Nexus are pretty close to the lowest frame rates we've seen throughout the entire game, making it quite a perfect fit for a worst case scenario.
[Previous Page]
[Go to top]
[Next Page]