This is my first time posting any benchmarks but I felt the results would help others and shine some light on the current state of Vista Gaming - including the Vista SP1 vs XP SP3 performance claims being made around the web. I hope you all enjoy it
A Comparative Analysis of XP vs Vista for Gaming
1-12-08
Hypothesis
------------
I expect to see XP beating Vista performance-wise by about 15% in all benchmarks just by being more lean and having a 7 year game and driver devlopment advantage over Vista. However, I also believe that Vista will be more than playable, albeit not boasting the raw power XP will have. I also believe that the games that have Direct X 10 extras will look noticable bette than their dx9 counter-parts. Enough so to warrant using Vista from a purely gaming perspective since you will be getting better looking games at a small performance cost.
*Update* Forgot the computer specs these benchmarks were done on! Sorry!
System Configuration
Abit IP35 Pro
Intel Q6600 @ 3Ghz
4GB G.SKILL PC2-6400
XFX 8800GT Alpha Dog Edition (PVT88PYDD4)
Seagate Barracuda 7200.11 1TB
PC Power & Cooling Silencer 610
Benchmark Software List:
All run 3 times with Avg taken from the Results.
------------
-3Dmark06
-Cinebench Release 10
-Company of Heroes Patch v2.201
-Crysis Patch v1.1
-F.E.A.R. Patch v1.08
XP Pro SP2 32bit Setup
------------
-Only 3GB of the 4GB of RAM visible
-Installed all updates available via Windows Update website
-Installed Newest Available drivers as of 1-10-08 (Nvidia 169.21)
-Installed updated DirectX Components (http://www.microsoft.com/downloads/...38-DB71-4C1B-BC6A-9B6652CD92A3&displaylang=en)
-Installed Benchmarking Apps
Results
3Dmark06 13782 13708 13667 Avg. 13719
Cinebench Release 10
OpenGL 5571 5482 5550 Avg. 5534
Single CPU 3072 3066 3066 Avg. 3068
Multi CPU 10752 10585 10808 Avg. 10715
Company of Heroes Patch v2.201
Avg FPS 58.1 58.4 58.3 Avg. 58
Max FPS 61 61 61 Avg. 61
Min FPS 12 29.5 29.5 Avg. 23.6
Crysis Patch v1.1
GPU @9am Avg 55.095
CPU @9am Avg 54.91
Assault Harbour @5pm Avg 46.67
F.E.A.R. Patch v1.08
Min FPS 42 43 41 Avg. 42
Avg FPS 83 82 83 Avg. 82.6
Max FPS 201 203 204 Avg. 202.6
FPS % below 25 0 0 0
FPS % between 25 and 40 0 0 0
FPS % above 40 100 100 100
XP SP3 RC1 v.3244
------------
After running the above benchmarks I installed the SP3 RC and rebooted - then ran the benchmarks again to see if any difference was noticable.
Results
3Dmark06 13791 13708 13667 Avg. 13728
Cinebench Release 10
OpenGL 5840 5773 5810 Avg. 5807
Single CPU 3075 3082 3066 Avg. 3074.3
Multi CPU 10753 10794 10736 Avg. 10761
Company of Heroes Patch v2.201
Avg FPS 58.1 58.3 58.3 Avg. 58.23
Max FPS 61 61 61 Avg. 61
Min FPS 11 29.5 30 Avg. 23.5
Crysis Patch v1.1
GPU @9am 55.645
CPU @9am 55.015
Assault Harbour @5pm 46.86
F.E.A.R. Patch v1.08
Min FPS 42 41 43 Avg. 42
Avg FPS 83 82 82 Avg. 82.3
Max FPS 201 204 205 Avg. 203.3
FPS % below 25 0 0 0
FPS % between 25 and 40 0 0 0
FPS % above 40 100 100 100
Vista Enterprise 32bit Setup
------------
-Only 3GB of the 4GB of RAM visible
-Disabled UAC
-Installed all updates available via Windows Update website
-Installed Newest Available drivers as of 1-10-08 (Nvidia 169.25)
-Installed Hotfix 940105 which is not available via Windows Updates
http://support.microsoft.com/kb/940105
-Installed updated DirectX Components(http://www.microsoft.com/downloads/...38-DB71-4C1B-BC6A-9B6652CD92A3&displaylang=en)
-Installed Benchmarking Apps
-Company of Heroes and Crysis both offer DirectX 10 graphics options not available that were used in these tests.
Results
3Dmark06 12959 12973 12995 Avg. 12975.6
Cinebench Release 10
OpenGL 4131 4158 4110 Avg. 4133
Single CPU 3051 3052 3040 Avg. 3047.6
Multi CPU 10753 10794 10736 Avg. 10761
Company of Heroes Patch v2.201
Avg FPS 29.7 30 29.1 Avg. 29.6
Max FPS 59.9 59.9 61 Avg. 60.2
Min FPS 14.6 15.9 16.5 Avg. 15.6
Crysis Patch v1.1
GPU @9am Avg 24.85
CPU @9am Avg 25.73
Assault Harbour @5pm Avg 23.895
F.E.A.R. Patch v1.08
Min FPS 40 24 23 Avg. 29
Avg FPS 68 39 27 Avg. 44.6
Max FPS 96 64 30 Avg. 63.3
FPS % below 25 0 11 12 Avg. 7.6
FPS % between 25 and 40 2 52 88 Avg. 47.3
FPS % above 40 98 37 0 Avg. 45
Vista 32bit SP1 RC1
------------
After running the above benchmarks I installed the SP1 RC1 and rebooted - then ran the benchmarks again to see if any difference was noticable.
Results
3Dmark06 13160 13012 13029 Avg. 13067
Cinebench Release 10
OpenGL 4355 4408 4386 Avg. 4383
Single CPU 3061 3058 3040 Avg. 3053
Multi CPU 10611 10681 10675 Avg. 10655.6
Company of Heroes Patch v2.201
Avg FPS 30.2 30 30.1 Avg. 30.1
Max FPS 59.9 60 59.9 Avg. 59.9
Min FPS 15.9 16.9 17.1 Avg. 16.6
Crysis Patch v1.1
GPU @9am Avg 24.765
CPU @9am Avg 25.435
Assault Harbour @5pm Avg 23.715
F.E.A.R. Patch v1.08
Min FPS 41 41 40 Avg. 40.6
Avg FPS 80 70 62 Avg. 70.6
Max FPS 161 104 84 Avg. 116.3
FPS % below 25 0 0 0 Avg. 0
FPS % between 25 and 40 0 0 2 Avg. .6
FPS % above 40 100 100 98 Avg. 99.3
Crysis Test all in Vista 64-bit DX9 Vs DX10
For this test I decided to try running crysis in Vista with all options the same in both dx9 mode and dx10 mode. All the settings remained the same - i did not turn on the extra options Direct X 10 mode unlocks for crysis. All settings were set to High @ 1024 x 768. I ran the Assault Harbor timedemo made available in the Crysis Benchmark Tool for this test since it is the most intensive benchmark for crysis that I used.
Vista 64bit High Quality DX9 Avg FPS 40.82
Vista 64bit High Quality DX10Avg FPS 30.3
Wow! Big difference there. Lets compare this to XP SP2 & SP3 RC1 scores:
XP SP2 High Quality DX9 Avg FPS 46.67
XP SP3 High Quality DX9 Avg FPS 46.86
So - DX9 mode in Vista gets pretty darn close to XP.
I ran this same kind of test with Company of Heroes (I got lazy and didn't record the exact scores) but I found that running Company of Heroes with that same exact settings as I was running it in XP yielded exactly the same scores. It was only when I enabled the Direct X 10 mode only "extras" that the frame rate dropped substantially.
Conclusion
First off - SP3 for XP and SP1 for Vista show almost no effect positive or negative speed boosts. This contradicts that already skeptical claim posted by CNET that SP3 boosted XP performance 15%.
The one strange thing is that SP3 did seem to boost the OpenGL benchmark for Cinebench by a good 5%. Other than that - nothing amaizing. SP1 seemed to help F.E.A.R by increasing its Avg FPS a huge 37% versus Vista without SP1. Obviously one game cannot dictate the future but F.E.A.R. is the oldest game being bechmarked here so maybe we will see older games running smoother with SP1
XP vs Vista scores showed a large performance gap and XP is faster than Vista on every front. Althought, Company of Heroes and Crysishad higher graphics options enabled in Vista and before the Vista fanboys shout "Apples to Oranges" testing let me defend myself in saying that if you are mainly a gamer and you don't care anything about Vista's features over XP's features then Direct X 10 eye candy is the only reason to consider switching to Vista - that is why I enabled it. And the difference was staggering to say the least. And personally, in Company of Heroes I could barely tell the difference while playing the game.
Company of heroes XP vs Vista showed nearly a 50% difference in Average FPS.
Heyzus Christo!
Crysis wasn't much better. The GPU benchmark showed a 55% difference in performance, the CPU benchmark showed a 54% difference, Assault Harbour showed 50% difference. I got two words for those results, TEH SUCK! The game does look noticeable better running it dx10 vs dx9 but sacrificing that much performance is a very, very tough pill to swallow. Espicially when it brings you below that golden 30FPS mark (which every dx10 Crysis run ran under).
Cinebench differences were a mixed bag. OpenGL favored XP AP3 vs Vista SP1 by being 25% faster. However the CPU benchmarks were nearly identical - showing no real distingushible difference between the two.
F.E.A.R just confused the hell out of me in Vista. For one, thanks to the huge changes made in Vista with DirectSound mixed with Creative lazyness for creating drivers you cannot enable EAX sound support for FEAR (or any other pre-vista game) in Vista. This shouldn't however create as big a difference as we see in the benchmark scores. Vista's 3 benchmark runs were so dissimilar you would think that I was running each one on a different box. It was really bizarre to see the wide range in scores from the three seperate runs in Vista. XP on the other hand, smooth as butta, all three benchmark runs matching each other extremely closely. The difference between running FEAR in XP and Vista? An average framerate boost of 37% in XP.
What I find to be the most interesting results are the Crysis running in Vista 64bit (no benchmarks listed above for 64-bit because they were pretty darn close to the 32bit benchmarks - so much so it's hard for me to find reason why I should go with a 64-bit OS and give up peer guardian). I ran the most intensive Crysis benchmark (assault harbour) using the same settings I ran the game at under XP (High vs the Vista only Very High) and then forced the game to run in dx9 and then dx10 with those settings to see the difference. A whopping 24% increase between dx9 and dx10 while still under Vista. The only difference being the version of Direct X being used. Running in dx9 mode cut the performance difference between Vista and XPto 13% and pulls it up above the 30 FPS marker. XP was still faster but only by about 6 FPS.
I wondered about what this meant for awhile and came to the conclusion that the reason why games are slow in Vista is not Vistas fault entirely. It is directx 10s fault too.
So, from a purely gaming perspective.....XP wins hands down. The only game that looks marginally better when running in Vista is Crysis but is nearly but the performance hits hurt it alot. I for one would rather have smooth gameplay at the expensive of some relativly minor graphics enhancements. Sure you can run Crysis in dx9 mode in Vista but then why shell out the money for a new operatiing system?
But from a non-gamer perspective, I like Vista. I like its interface, it's super fast and accurate indexing searches, the usefull sidebar & gadgets, it's high res icons, and even some of the new programs like Windows Calendar and the Snipping Tool. And unlike others, I have had no problems running any of my programs with it. These are the things that drew me in to Vista. I don't play games 24/7 on my computer. I browse the web, watch movies, multi-task, and write poignent benchmark comparissons. But alas, I do play alot of games.
So where does this leave me?
Basically, the gamer in me is ready to go back to XP without reservation. The average-joe in me appeals to Vista for its looks and (in my apparently minority opinion) better interface. As of now I'm undecided to be honest. I am currently leaning towards running Vista and playing games in dx9 mode so that I can enjoy the best of both worlds and be ready for a day (if it ever comes) where game performance in Vista will match XP's. I think that if game developers embrace Vista like they did XP we will see that day. However, right now that trend seems not to be the case. The word on the street is that Vista sucks - and while XP is the leads the OS market in terms of number of users I think that developers are going to make sure their OS works better on XP first and worry about Vista second. At least, thats how i think it's been going this past year - things will probably change when XP is no longer available for purchase starting later this year.
If you liked this - Digg it!
A Comparative Analysis of XP vs Vista for Gaming
1-12-08
Hypothesis
------------
I expect to see XP beating Vista performance-wise by about 15% in all benchmarks just by being more lean and having a 7 year game and driver devlopment advantage over Vista. However, I also believe that Vista will be more than playable, albeit not boasting the raw power XP will have. I also believe that the games that have Direct X 10 extras will look noticable bette than their dx9 counter-parts. Enough so to warrant using Vista from a purely gaming perspective since you will be getting better looking games at a small performance cost.
*Update* Forgot the computer specs these benchmarks were done on! Sorry!
System Configuration
Abit IP35 Pro
Intel Q6600 @ 3Ghz
4GB G.SKILL PC2-6400
XFX 8800GT Alpha Dog Edition (PVT88PYDD4)
Seagate Barracuda 7200.11 1TB
PC Power & Cooling Silencer 610
Benchmark Software List:
All run 3 times with Avg taken from the Results.
------------
-3Dmark06
-Cinebench Release 10
-Company of Heroes Patch v2.201
-Crysis Patch v1.1
-F.E.A.R. Patch v1.08
XP Pro SP2 32bit Setup
------------
-Only 3GB of the 4GB of RAM visible
-Installed all updates available via Windows Update website
-Installed Newest Available drivers as of 1-10-08 (Nvidia 169.21)
-Installed updated DirectX Components (http://www.microsoft.com/downloads/...38-DB71-4C1B-BC6A-9B6652CD92A3&displaylang=en)
-Installed Benchmarking Apps
Results
3Dmark06 13782 13708 13667 Avg. 13719
Cinebench Release 10
OpenGL 5571 5482 5550 Avg. 5534
Single CPU 3072 3066 3066 Avg. 3068
Multi CPU 10752 10585 10808 Avg. 10715
Company of Heroes Patch v2.201
Avg FPS 58.1 58.4 58.3 Avg. 58
Max FPS 61 61 61 Avg. 61
Min FPS 12 29.5 29.5 Avg. 23.6
Crysis Patch v1.1
GPU @9am Avg 55.095
CPU @9am Avg 54.91
Assault Harbour @5pm Avg 46.67
F.E.A.R. Patch v1.08
Min FPS 42 43 41 Avg. 42
Avg FPS 83 82 83 Avg. 82.6
Max FPS 201 203 204 Avg. 202.6
FPS % below 25 0 0 0
FPS % between 25 and 40 0 0 0
FPS % above 40 100 100 100
XP SP3 RC1 v.3244
------------
After running the above benchmarks I installed the SP3 RC and rebooted - then ran the benchmarks again to see if any difference was noticable.
Results
3Dmark06 13791 13708 13667 Avg. 13728
Cinebench Release 10
OpenGL 5840 5773 5810 Avg. 5807
Single CPU 3075 3082 3066 Avg. 3074.3
Multi CPU 10753 10794 10736 Avg. 10761
Company of Heroes Patch v2.201
Avg FPS 58.1 58.3 58.3 Avg. 58.23
Max FPS 61 61 61 Avg. 61
Min FPS 11 29.5 30 Avg. 23.5
Crysis Patch v1.1
GPU @9am 55.645
CPU @9am 55.015
Assault Harbour @5pm 46.86
F.E.A.R. Patch v1.08
Min FPS 42 41 43 Avg. 42
Avg FPS 83 82 82 Avg. 82.3
Max FPS 201 204 205 Avg. 203.3
FPS % below 25 0 0 0
FPS % between 25 and 40 0 0 0
FPS % above 40 100 100 100
Vista Enterprise 32bit Setup
------------
-Only 3GB of the 4GB of RAM visible
-Disabled UAC
-Installed all updates available via Windows Update website
-Installed Newest Available drivers as of 1-10-08 (Nvidia 169.25)
-Installed Hotfix 940105 which is not available via Windows Updates
http://support.microsoft.com/kb/940105
-Installed updated DirectX Components(http://www.microsoft.com/downloads/...38-DB71-4C1B-BC6A-9B6652CD92A3&displaylang=en)
-Installed Benchmarking Apps
-Company of Heroes and Crysis both offer DirectX 10 graphics options not available that were used in these tests.
Results
3Dmark06 12959 12973 12995 Avg. 12975.6
Cinebench Release 10
OpenGL 4131 4158 4110 Avg. 4133
Single CPU 3051 3052 3040 Avg. 3047.6
Multi CPU 10753 10794 10736 Avg. 10761
Company of Heroes Patch v2.201
Avg FPS 29.7 30 29.1 Avg. 29.6
Max FPS 59.9 59.9 61 Avg. 60.2
Min FPS 14.6 15.9 16.5 Avg. 15.6
Crysis Patch v1.1
GPU @9am Avg 24.85
CPU @9am Avg 25.73
Assault Harbour @5pm Avg 23.895
F.E.A.R. Patch v1.08
Min FPS 40 24 23 Avg. 29
Avg FPS 68 39 27 Avg. 44.6
Max FPS 96 64 30 Avg. 63.3
FPS % below 25 0 11 12 Avg. 7.6
FPS % between 25 and 40 2 52 88 Avg. 47.3
FPS % above 40 98 37 0 Avg. 45
Vista 32bit SP1 RC1
------------
After running the above benchmarks I installed the SP1 RC1 and rebooted - then ran the benchmarks again to see if any difference was noticable.
Results
3Dmark06 13160 13012 13029 Avg. 13067
Cinebench Release 10
OpenGL 4355 4408 4386 Avg. 4383
Single CPU 3061 3058 3040 Avg. 3053
Multi CPU 10611 10681 10675 Avg. 10655.6
Company of Heroes Patch v2.201
Avg FPS 30.2 30 30.1 Avg. 30.1
Max FPS 59.9 60 59.9 Avg. 59.9
Min FPS 15.9 16.9 17.1 Avg. 16.6
Crysis Patch v1.1
GPU @9am Avg 24.765
CPU @9am Avg 25.435
Assault Harbour @5pm Avg 23.715
F.E.A.R. Patch v1.08
Min FPS 41 41 40 Avg. 40.6
Avg FPS 80 70 62 Avg. 70.6
Max FPS 161 104 84 Avg. 116.3
FPS % below 25 0 0 0 Avg. 0
FPS % between 25 and 40 0 0 2 Avg. .6
FPS % above 40 100 100 98 Avg. 99.3
Crysis Test all in Vista 64-bit DX9 Vs DX10
For this test I decided to try running crysis in Vista with all options the same in both dx9 mode and dx10 mode. All the settings remained the same - i did not turn on the extra options Direct X 10 mode unlocks for crysis. All settings were set to High @ 1024 x 768. I ran the Assault Harbor timedemo made available in the Crysis Benchmark Tool for this test since it is the most intensive benchmark for crysis that I used.
Vista 64bit High Quality DX9 Avg FPS 40.82
Vista 64bit High Quality DX10Avg FPS 30.3
Wow! Big difference there. Lets compare this to XP SP2 & SP3 RC1 scores:
XP SP2 High Quality DX9 Avg FPS 46.67
XP SP3 High Quality DX9 Avg FPS 46.86
So - DX9 mode in Vista gets pretty darn close to XP.
I ran this same kind of test with Company of Heroes (I got lazy and didn't record the exact scores) but I found that running Company of Heroes with that same exact settings as I was running it in XP yielded exactly the same scores. It was only when I enabled the Direct X 10 mode only "extras" that the frame rate dropped substantially.
Conclusion
First off - SP3 for XP and SP1 for Vista show almost no effect positive or negative speed boosts. This contradicts that already skeptical claim posted by CNET that SP3 boosted XP performance 15%.
The one strange thing is that SP3 did seem to boost the OpenGL benchmark for Cinebench by a good 5%. Other than that - nothing amaizing. SP1 seemed to help F.E.A.R by increasing its Avg FPS a huge 37% versus Vista without SP1. Obviously one game cannot dictate the future but F.E.A.R. is the oldest game being bechmarked here so maybe we will see older games running smoother with SP1
XP vs Vista scores showed a large performance gap and XP is faster than Vista on every front. Althought, Company of Heroes and Crysishad higher graphics options enabled in Vista and before the Vista fanboys shout "Apples to Oranges" testing let me defend myself in saying that if you are mainly a gamer and you don't care anything about Vista's features over XP's features then Direct X 10 eye candy is the only reason to consider switching to Vista - that is why I enabled it. And the difference was staggering to say the least. And personally, in Company of Heroes I could barely tell the difference while playing the game.
Company of heroes XP vs Vista showed nearly a 50% difference in Average FPS.
Heyzus Christo!
Crysis wasn't much better. The GPU benchmark showed a 55% difference in performance, the CPU benchmark showed a 54% difference, Assault Harbour showed 50% difference. I got two words for those results, TEH SUCK! The game does look noticeable better running it dx10 vs dx9 but sacrificing that much performance is a very, very tough pill to swallow. Espicially when it brings you below that golden 30FPS mark (which every dx10 Crysis run ran under).
Cinebench differences were a mixed bag. OpenGL favored XP AP3 vs Vista SP1 by being 25% faster. However the CPU benchmarks were nearly identical - showing no real distingushible difference between the two.
F.E.A.R just confused the hell out of me in Vista. For one, thanks to the huge changes made in Vista with DirectSound mixed with Creative lazyness for creating drivers you cannot enable EAX sound support for FEAR (or any other pre-vista game) in Vista. This shouldn't however create as big a difference as we see in the benchmark scores. Vista's 3 benchmark runs were so dissimilar you would think that I was running each one on a different box. It was really bizarre to see the wide range in scores from the three seperate runs in Vista. XP on the other hand, smooth as butta, all three benchmark runs matching each other extremely closely. The difference between running FEAR in XP and Vista? An average framerate boost of 37% in XP.
What I find to be the most interesting results are the Crysis running in Vista 64bit (no benchmarks listed above for 64-bit because they were pretty darn close to the 32bit benchmarks - so much so it's hard for me to find reason why I should go with a 64-bit OS and give up peer guardian). I ran the most intensive Crysis benchmark (assault harbour) using the same settings I ran the game at under XP (High vs the Vista only Very High) and then forced the game to run in dx9 and then dx10 with those settings to see the difference. A whopping 24% increase between dx9 and dx10 while still under Vista. The only difference being the version of Direct X being used. Running in dx9 mode cut the performance difference between Vista and XPto 13% and pulls it up above the 30 FPS marker. XP was still faster but only by about 6 FPS.
I wondered about what this meant for awhile and came to the conclusion that the reason why games are slow in Vista is not Vistas fault entirely. It is directx 10s fault too.
So, from a purely gaming perspective.....XP wins hands down. The only game that looks marginally better when running in Vista is Crysis but is nearly but the performance hits hurt it alot. I for one would rather have smooth gameplay at the expensive of some relativly minor graphics enhancements. Sure you can run Crysis in dx9 mode in Vista but then why shell out the money for a new operatiing system?
But from a non-gamer perspective, I like Vista. I like its interface, it's super fast and accurate indexing searches, the usefull sidebar & gadgets, it's high res icons, and even some of the new programs like Windows Calendar and the Snipping Tool. And unlike others, I have had no problems running any of my programs with it. These are the things that drew me in to Vista. I don't play games 24/7 on my computer. I browse the web, watch movies, multi-task, and write poignent benchmark comparissons. But alas, I do play alot of games.
So where does this leave me?
Basically, the gamer in me is ready to go back to XP without reservation. The average-joe in me appeals to Vista for its looks and (in my apparently minority opinion) better interface. As of now I'm undecided to be honest. I am currently leaning towards running Vista and playing games in dx9 mode so that I can enjoy the best of both worlds and be ready for a day (if it ever comes) where game performance in Vista will match XP's. I think that if game developers embrace Vista like they did XP we will see that day. However, right now that trend seems not to be the case. The word on the street is that Vista sucks - and while XP is the leads the OS market in terms of number of users I think that developers are going to make sure their OS works better on XP first and worry about Vista second. At least, thats how i think it's been going this past year - things will probably change when XP is no longer available for purchase starting later this year.
If you liked this - Digg it!