Gaming Benchmarks - Direct3D
There are an incredible number of tests we wanted to run in order to explore all the different aspects of gaming on Vista versus gaming on XP both in 32- and 64-bit modes. For this first look, however, we will be sticking with single card setups on Vista x86, Vista x64, and Windows XP x86. In the future, we plan on looking at multi-GPU performance as well as the performance of games on Windows XP x64. With Vista, Microsoft has improved the gaming on 64-bit landscape significantly, and we would like to see just how far we have come.
For now, we will be looking at a selection of cards from AMD and NVIDIA: five cards from the Radeon X1K series and five cards from the GeForce 7 series, as well as both current 8800 parts. These tests will look at GPU and CPU limited situations, and we will be comparing Vista x86 with Windows XP as well as Vista x64 to see where there are any differences. We will also look at resolution scaling on each of our three test operating systems, but only with two cards representing AMD and NVIDIA: the X1950 XTX and 8800 GTX.
In terms of APIs, one DirectX and one OpenGL game will be tested. Oblivion will represent the MS API, while Quake 4 will be showing off OpenGL performance on Vista. We have also taken a look at a native 64-bit game. Valve's Half-Life 2: Lost Coast will be run on all three operating systems in native mode. We would like to have compared HL2:LC running in WoW at 32-bit under x64, but we have not been able to figure out how to test it without running in 64-bit mode yet.
First up is Oblivion performance. DirectX performance should be as close as possible to Windows XP performance as this is Microsoft's baby. First, let's take a look at Vista x86 numbers divided by Windows XP scores for CPU limited and GPU limited cases. This will give us the speed up (numbers above one) or slow down (numbers less than one) as compared to Windows XP. Just remember that there is some normal fluctuation in performance on both sides, so we could see a wider margin of error here than in our standard comparisons.
For CPU bound tests, almost every card performs better under Windows Vista than under Windows XP; the lone exception is the X1900 XT 256MB. This indicates that Vista is better able to provide system resources to DirectX games, which is actually quite surprising considering the overhead that Vista adds to the system.
When we take a look at the GPU bound case, we see almost the opposite with only the 8800 GTS performing better than it does on Windows XP. This does line up with what we would expect. There is more involvement from the OS in the rendering pipeline and less direct access to the hardware by games. Vista is now able to manage graphics memory and graphics drivers must support sharing of hardware resources between multiple programs. Luckily, the differences aren't that huge. While the hardcore gamers won't be happy with any performance loss, Microsoft is betting that in the long term the advantages will outweigh the loss in framerate.
Now let's take a look at how Vista stacks up against XP in an x86 vs. x64 comparison
On the CPU limited results, most of the cards again run faster on Vista x64. This is frankly amazing, as even Microsoft expects performance to be slightly slower on Vista. In the GPU limited results, we see basically a random scattering of cards that are slightly slower or faster under Vista x64. Given that Oblivion is a 32-bit application running in WoW, we would be pretty happy with only small performance losses, and any performance improvements are unexpected but welcome. All in all, x64 performance looks good, and we haven't seen the types of compatibility and stability issues between the two that we did with XP x64.
Last up for Oblivion is resolution scaling.
Both the 8800 GTX and X1950 XTX scale very similarly across platforms. There does seem to be more difference in CPU limited settings, while performance seems to converge as resolution increases.
There are an incredible number of tests we wanted to run in order to explore all the different aspects of gaming on Vista versus gaming on XP both in 32- and 64-bit modes. For this first look, however, we will be sticking with single card setups on Vista x86, Vista x64, and Windows XP x86. In the future, we plan on looking at multi-GPU performance as well as the performance of games on Windows XP x64. With Vista, Microsoft has improved the gaming on 64-bit landscape significantly, and we would like to see just how far we have come.
For now, we will be looking at a selection of cards from AMD and NVIDIA: five cards from the Radeon X1K series and five cards from the GeForce 7 series, as well as both current 8800 parts. These tests will look at GPU and CPU limited situations, and we will be comparing Vista x86 with Windows XP as well as Vista x64 to see where there are any differences. We will also look at resolution scaling on each of our three test operating systems, but only with two cards representing AMD and NVIDIA: the X1950 XTX and 8800 GTX.
In terms of APIs, one DirectX and one OpenGL game will be tested. Oblivion will represent the MS API, while Quake 4 will be showing off OpenGL performance on Vista. We have also taken a look at a native 64-bit game. Valve's Half-Life 2: Lost Coast will be run on all three operating systems in native mode. We would like to have compared HL2:LC running in WoW at 32-bit under x64, but we have not been able to figure out how to test it without running in 64-bit mode yet.
First up is Oblivion performance. DirectX performance should be as close as possible to Windows XP performance as this is Microsoft's baby. First, let's take a look at Vista x86 numbers divided by Windows XP scores for CPU limited and GPU limited cases. This will give us the speed up (numbers above one) or slow down (numbers less than one) as compared to Windows XP. Just remember that there is some normal fluctuation in performance on both sides, so we could see a wider margin of error here than in our standard comparisons.
For CPU bound tests, almost every card performs better under Windows Vista than under Windows XP; the lone exception is the X1900 XT 256MB. This indicates that Vista is better able to provide system resources to DirectX games, which is actually quite surprising considering the overhead that Vista adds to the system.
When we take a look at the GPU bound case, we see almost the opposite with only the 8800 GTS performing better than it does on Windows XP. This does line up with what we would expect. There is more involvement from the OS in the rendering pipeline and less direct access to the hardware by games. Vista is now able to manage graphics memory and graphics drivers must support sharing of hardware resources between multiple programs. Luckily, the differences aren't that huge. While the hardcore gamers won't be happy with any performance loss, Microsoft is betting that in the long term the advantages will outweigh the loss in framerate.
Now let's take a look at how Vista stacks up against XP in an x86 vs. x64 comparison
On the CPU limited results, most of the cards again run faster on Vista x64. This is frankly amazing, as even Microsoft expects performance to be slightly slower on Vista. In the GPU limited results, we see basically a random scattering of cards that are slightly slower or faster under Vista x64. Given that Oblivion is a 32-bit application running in WoW, we would be pretty happy with only small performance losses, and any performance improvements are unexpected but welcome. All in all, x64 performance looks good, and we haven't seen the types of compatibility and stability issues between the two that we did with XP x64.
Last up for Oblivion is resolution scaling.
Both the 8800 GTX and X1950 XTX scale very similarly across platforms. There does seem to be more difference in CPU limited settings, while performance seems to converge as resolution increases.
105 Comments
View All Comments
thebrown13 - Thursday, February 1, 2007 - link
This is TOTALLY false. Microsoft allows content providers to do what they want with their content. Again, THE CONTENT PROVIDERS DECIDE WHAT THEIR CONTENT PLAYS ON. MICROSOFT DECIDES NOTHING.MAIA - Thursday, February 1, 2007 - link
The architecture is still x86 with 64-bit extensions. The correct naming should be 32-bit x86 and 64-bit x86. Although some vendors use the x64 "slang", this only serves to create more confusion, making people believe it's a different architecture. More, editors and reviewers should use the correct definitions, they also have an educating factor which have to live up by a good standart.Ryan Smith - Thursday, February 1, 2007 - link
It makes little sense on our part to use something different than Microsoft's own terminology here when talking about Vista. The x86-64/EMT64/AMD64 versions of Vista are all called Vista * x64, so we're going to be consistent on using Microsoft's naming to avoid possible confusion among Windows versions.Myrandex - Thursday, February 1, 2007 - link
Eh I don't know how x64 can get confusing, but x86_64 is pretty typical as well.smitty3268 - Thursday, February 1, 2007 - link
Actually, it is a new architecture that just happens to be backwards compatible with x86. That is why there are twice the number or registers available and not just the same amount doubled in size. You're right that x64 is not the correct name, (it's technically called x86-64) but what does it matter? Everyone knows what it means.Gunlance - Thursday, February 1, 2007 - link
I wish I could agree about the new installer. My experience with trying to boot windows vista so far is worse than when I learned how to put NetBSD on an Apple G3.In fact I am still trying to get into the Windows Vista setup as I type this comment. I have been up all night :( I simply just don't get it. Every OS I have ever put on my desktop at least boots, and the vista beta's installed fine. Ugh.
The article was great! Heh. It has made me a bit more frustrated though. Because here I am with vista but still only being able to read about.
erwos - Thursday, February 1, 2007 - link
Your OpenGL performance numbers are radically higher than what Tom's has. Could you comment on the drivers and installation that you used?Ryan Smith - Thursday, February 1, 2007 - link
Normally I tend to avoid commenting on anything involving competitors, but...Assuming I'm looking at the right article here, they were using the Catalyst 8.31.100.3.2.1 driver, which was released back in December. We were using the Catalyst 7.1 driver(version # 8.33.something). ATI did not include an OpenGL driver until 7.1(and just barely at that).
ktgktg - Thursday, February 1, 2007 - link
I'm surprised that the article didn't mention how much ReadyBoost could lower the boot time. A member of notebooreview forums claimed that http://forum.notebookreview.com/showthread.php?t=1...">he lowered the boot time on a laptop (slower HD) from 80 to 43 seconds (including POST) with a 2 GB SD card. Remember that http://en.wikipedia.org/wiki/Comparison_of_memory_...">SD cards are slow compared to USB and CF. Although they're all slow compared to RAM, they have the benefit of storing the data until next boot.SuperFetch seems to be just another reason for using standby mode.
yacoub - Thursday, February 1, 2007 - link
Any word on FireFox compatibility with Vista? That is, does it run just as flawlessly as it does under XP x86 and x64?Just wondering. The comment on the Conclusions page about IE 7+ being the new standard for Safari is what made me think to ask.