Serious Sam II
This game is quite possibly the longest of the three - 41 levels in total. There are some pretty funny, albeit dry jokes in it like, "These rocks can shake, but I won't Quake." or even "I love collecting big guns. They're so Unreal." Hmm, what's a matter Croteam? No Half-Life jokes? The last level, Mental Institution, was quite probably the longest level we have ever played in an FPS. Even the guys at Croteam put an estimated playtime of 1 hour for this level. Wonder what they will call the sequel: Serious Sam II, well, 2? I know Serious Sam II: Double the Fun!!! Oh, well - here are the results of many long hours of playing Serious Sam II.| 1024 x 768 | 1280 x 960 | 1600 x 1200 | Performance Hit 1280 | Performance Hit 1600 | ||
| Jungle | 92.26 | 91.01 | 82.37 | 1.35% | 10.72% | |
| Riverdance | 89.28 | 84.3 | 82.59 | 5.58% | 7.49% | |
| M'Keke Village | 84.05 | 92.44 | 82.99 | -9.98% | 1.26% | |
| Road to Ursul | 70.58 | 75.21 | 67.91 | -6.56% | 3.78% | |
| Ursul Suburbs | 83.98 | 84.99 | 80.84 | -1.20% | 3.74% | |
| Kukulele Prison | 80.78 | 81.31 | 72.89 | -0.66% | 9.76% | |
| Ursul Gardens | 75.77 | 78.25 | 77.64 | -3.28% | -2.47% | |
| Kwongo | 104.6 | 89.58 | 81.84 | 14.36% | 21.76% | |
| Deadwood | 86.62 | 86.06 | 84.88 | 0.65% | 2.01% | |
| Forsaken Compound | 74.92 | 75.81 | 81.69 | -1.19% | -9.04% | |
| Branchester | 81.44 | 78.36 | 78.31 | 3.78% | 3.84% | |
| Woodstock | 95.39 | 96.54 | 95.27 | -1.21% | 0.12% | |
| Giant Junkyard | 56.41 | 60.46 | 62.45 | -7.18% | -10.69% | |
| Zum-Zum | 108.23 | 95.88 | 128.07 | 11.41% | -18.34% | |
| Hong-Pong | 83.67 | 79.72 | 68.5 | 4.72% | 18.13% | |
| Wheels of Fortune | 65.19 | 64.18 | 62.74 | 1.55% | 3.75% | |
| Chanolin Chambers | 105.79 | 108.7 | 97.54 | -2.76% | 7.79% | |
| Chanolin Temple | 88.7 | 83.77 | 74.2 | 5.55% | 16.34% | |
| Prince Chan | 99.5 | 90.52 | 91.94 | 9.02% | 7.60% | |
| Unreal Wasteland | 113.9 | 115.69 | 113.48 | -1.57% | 0.37% | |
| Doomed Canyon | 135.17 | 129.42 | 129.16 | 4.26% | 4.45% | |
| Boneyard | 132.35 | 123.6 | 118.57 | 6.61% | 10.41% | |
| Count Kleerofski | 159.96 | 98.21 | 80.21 | 38.60% | 49.86% | |
| Greendale | 81.5 | 81.87 | 83.09 | -0.46% | -1.95% | |
| Coast to Cost | 80.63 | 85 | 88.97 | -5.42% | -10.35% | |
| Kingsburg | 90.83 | 91.75 | 91.98 | -1.01% | -1.26% | |
| Royal Sewers | 141.35 | 140.95 | 138.95 | 0.28% | 1.70% | |
| Castle of Rock | 108.3 | 109.12 | 97.33 | -0.75% | 10.13% | |
| Floaterra | 86.17 | 95.8 | 92.63 | -11.18% | -7.50% | |
| Cecil the Dragon | 170.89 | 179.86 | 162.56 | -5.25% | 4.87% | |
| Area 5100 | 96.6 | 101.97 | 97.39 | -5.56% | -0.82% | |
| Command Center | 110.49 | 95.93 | 100.66 | 13.18% | 8.90% | |
| Freezepad | 104.03 | 106.38 | 9.54 | -2.26% | 90.83% | |
| Frostpost | 78.14 | 83.63 | 80.9 | -7.03% | -3.54% | |
| Hugo the Robot | 124.22 | 119.96 | 109.33 | 3.43% | 11.98% | |
| Siriuspolis Uptown | 74.86 | 83.21 | 60.67 | -11.15% | 18.95% | |
| Shield Generator | 59.74 | 60.39 | 59.52 | -1.08% | 0.38% | |
| Be Quick of Be Dead | 97.31 | 93.71 | 97.14 | 3.70% | 0.17% | |
| Welcome to the Jungle | 87.51 | 95.28 | 83.55 | -8.88% | 4.53% | |
| Jump in the Fire | 125.03 | 106.64 | 98.95 | 14.71% | 20.86% | |
| Siriuspolis Downtown | 78.64 | 83.57 | 78.87 | -6.27% | -0.29% | |
| Mental Institution | 103.81 | 96.63 | 85.52 | 6.91% | 17.61% | |
| Average (entire game) | 96.87 | 94.66 | 88.42 | 1.14% | 7.33% |
all numbers are in frame rates (except for percentages)
The entire game seems to be quite system limited. It would've been a much more different picture with a faster processor or a slower graphics card but we didn't have either of them in hand so these will have to do. Average performance hit for the entire game at 1600 x 1200 is just around 7 percent, that's just how system limited this game is in our test setup. Obviously, we have to pick the more graphically intensive levels, where the performance hit is substantially larger than that. We didn't pick the Boss levels (Kwongo, Count Kleerofski) since they're pretty small and not really your typical Serious Sam II level. You can see the top 5 levels below.
| 1024 x 768 | 1280 x 960 | 1600 x 1200 | Performance Hit 1024 to 1280 | Performance Hit 1024 to 1600 | ||
| Jump in the Fire | 125.03 | 106.64 | 98.95 | 14.71% | 20.86% | |
| Siriuspolis Uptown | 74.86 | 83.21 | 60.67 | -11.15% | 18.95% | |
| Hong-Pong | 83.67 | 79.72 | 68.5 | 4.72% | 18.13% | |
| Mental Institution | 103.81 | 96.63 | 85.52 | 6.91% | 17.61% | |
| Chanolin Temple | 88.7 | 83.77 | 74.2 | 5.55% | 16.34% | |
| Average |
95.21 | 89.99 | 77.57 | 4.15% | 18.38% |
all numbers are in frame rates (except for percentages)
Average frame rates does seem to differ from the entire game, ranging from 6 to 30 fps - a very big difference. The performance hit seems to indicate these levels are more graphically intensive than others, but would it be enough? Out of these levels, only Mental Institution have an average fps that's close to what we've seen throughout the entire game. Personally, we would rather not use this level - it's just too damn long. From a performance penalty perspective, the Hong-Pong level is fairly close to the average performance penalty, followed by Chanolin Temple. Now, let's see if these numbers are repeatable.
These results are taken at 1024 x 768, without AA and AF.
| 1st | 2nd | 3rd | Average | Variations | Variations (%) | ||
| Jump in the Fire | 90.75 | 92.95 | 91.86 | 91.86 | 1.1 | 1.20% | |
| Siriuspolis Uptown | 71.21 | 67.78 | 66.42 | 68.47 | 2.47 | 3.61% | |
| Hong-Pong | 73.6 | 74.47 | 73.04 | 73.7 | 0.72 | 0.98% | |
| Mental Institution | 100.68 | 99.47 | 100.43 | 100.2 | 0.64 | 0.64% | |
| Chanolin Temple | 87.88 | 85.47 | 82.79 | 85.38 | 2.55 | 2.98% |
all numbers are in frame rates (except for percentages)
We tried to play these levels in a similar manner each time and by doing so, we've managed to keep variations pretty low. However, we can also see actual frame rates are much lower than our initial run. Only Mental Institution's and Chanolin Temple's results are fairly close to our initial results. However, Chanolin Temple seems to have a slightly higher variation between runs than Hong-Pong and Mental Institution. For our sanity's sake, let's hope it would be either Chanolin Temple or Hong-Pong, preferable Hong-Pong since it's much shorter and reacts much like other levels with resolution changes.
OK. Time to see if we could rely on replay instead of playing the game manually by hand. Will frame rates from a replay be indicative of actual frame rates in gameplay? Serious Sam II don't have a timedemo, but it does have a replay that can be used as a benchmark tool with the proper console command. We record a replay for each level, then we played that replay, measuring performance both with Serious Sam II internal benchmarking tool and FRAPS. We did this just to make sure FRAPS is recording and reporting accurately. Here are the results for each level.
These results are taken at 1024 x 768, without AA and AF.
| Gameplay | FRAPS benchmark | internal | Difference | Difference (%) | ||
| Jump in the Fire | 91.86 | 86.84 | 87.8 | -4.53 | -4.94% | |
| Siriuspolis Uptown | 68.47 | 56.27 | 56.75 | -11.96 | -17.47% | |
| Hong-Pong | 73.7 | 68.58 | 69.55 | -4.64 | -6.29% | |
| Mental Institution | 100.2 | 90.36 | 91.1 | -9.47 | -9.45% | |
| Chanolin Temple | 85.38 | 72.54 | 73.45 | -12.38 | -14.50% |
all numbers are in frame rates (except for percentages)
Ouch. Another strange behavior, but thankfully more predictable this time. Looks like replay results are lower than gameplay, making them poor not representative of frame rates in actual gameplay. The difference between replays and gameplay testing sessions can even be significant, so it's definitely not just normal variations between runs. Remember, variations between runs are fairly small with these levels (0.64 - 3.61 percent). In all honesty, our option are now limited to two levels - Jump in the Fire and Hong-Pong - two levels that behave similarly in replays and gameplay testing sessions. Out of the two, we pick Jump In The Fire because it's shorter.
We've seen the results from three newer games in our benchmark suite and it's not pretty. Out of the three, only F.E.A.R performance test offer results that are close enough to frame rate in actual gameplay. Quake 4 timedemo is not only broken (misplaced textures, fonts), but also behave rather erratically with some levels. Serious Sam II replay incurs quite a significant overhead in some cases, offering results that are actually lower than frame rate in actual gameplay.
It's beginning to look like timedemo and replays can not really represent what kind of frame rate you can expect from a game, even with a developer supplied demo / replay. However, that doesn't they're completely useless. They can still provide us some idea of how a game will behave in graphically limited situations, just not exact level of frame rates in actual gameplay. For a more representative test results, actual gameplay testing sessions is a better tool. Now that we have our candidates for gameplay testing, let's see how do they stand up in a real performance test.
[Previous Page]
[Go to top]
[Next Page]