Excuse me, please stop discussing the R600 canned benchmarks

S

Sharky974

Identified Troll
Joined
Nov 16, 2005
Messages
396
Has Kyle and enlightened forumers taught us nothing? Those are completely meaningless to real world gamreplay. R600 is NOT a failure, in fact it may be much, much better than 8800GTX in real world gaming.

Lets all do a favor and stop discussing canned benchmarks and timedemos!

Right, guys?

I saw a lot of guys pointing the folly of canned benchmarks out in Kyle's 8600 review thread, so come all you guys! Back me up by the hundreds! I'm right here waiting!
 
Sounds ok to me. Look at how it was with 8600, canned benchmark sites announced it as a total dissapointment but [H] thought it was ok card.
 
I am just ignoring all this talk...
there hasnt been any release of official benchmarks, yet people are debating that the card is a failure already etc...

Have some common sense people
 
Sharky974 said:
Has Kyle and enlightened forumers taught us nothing? Those are completely meaningless to real world gamreplay. R600 is NOT a failure, in fact it may be much, much better than 8800GTX in real world gaming.

Lets all do a favor and stop discussing canned benchmarks and timedemos!

Right, guys?

I saw a lot of guys pointing the folly of canned benchmarks out in Kyle's 8600 review thread, so come all you guys! Back me up by the hundreds! I'm right here waiting!
Click to expand...

I understand your point, but I see no harm in discussing these rumors. It's all we have, until actual numbers are shown.
And these unconfirmed benchmarks, will later be used to be compared with the actual ones, so that we can at least have an idea of how credible/not credible that source is/was. How do you think The Inq got its (in)famous name, as source of unreliable info ? Because their rumors and unconfirmed numbers, where compared to final ones and the results show that they spread more FUD than they deliver good pieces of information.

Also, if we don't discuss rumors, about upcoming products, what is there to talk about in these sub-forums ?
 
Silus said:
I understand your point, but I see no harm in discussing these rumors. It's all we have, until actual numbers are shown.
And these unconfirmed benchmarks, will later be used to be compared with the actual ones, so that we can at least have an idea of how credible/not credible that source is/was. How do you think The Inq got its (in)famous name, as source of unreliable info ? Because their rumors and unconfirmed numbers, where compared to final ones and the results show that they spread more FUD than they deliver good pieces of information.

Also, if we don't discuss rumors, about upcoming products, what is there to talk about in these sub-forums ?
Click to expand...


I'm not talking about rumors, but flawed benchmarking methods.

If these were rumurs of real-world gameplay tests as described by Kyle Bennett, I'd have no problem with them.
 
Silus said:
Also, if we don't discuss rumors, about upcoming products, what is there to talk about in these sub-forums ?
Click to expand...

Exactly. If we didn't post about this stuff, then there wouldn't be anyone to argue with! :eek::D
 
Sharky974 said:
I'm not talking about rumors, but flawed benchmarking methods.

If these were rumurs of real-world gameplay tests as described by Kyle Bennett, I'd have no problem with them.
Click to expand...

They are not "flawed". They're just the way the most typical benchmarks are/were done. The fact that IMHO, [H] evolved to include better ways to compare video cards, doesn't mean the other benchmarks are "flawed". Here they are called canned and I'm ok with the designation, but I still use them to have a more synthetic analysis on the product. In fact, I use [H] as a main source and Tech-Report (which uses a more typical benchmarking scheme than [H]) as my second source. Only with at least a couple of sources, can I reach my own conclusion on how, on average, this or that card, will perform.
Not to mention that [H] can't cover all the games I like, since its evaluation methods are very thorough and time consuming, so I still need to check other evaluation/review sites, to have an idea on how a certain card, performs in that game I like, even if it does not represent real world gaming performance.
 
i agree, these benchmarks are nothing more then biast(that how you spell it?) people and their loyalty to either company ATI or Nvidia


how about we let the people at [H] do their job when the time comes, and lets see whether or not the r600 is actually good or not! just my opinion guys. let the flame wars begin lol
 
Silus said:
I understand your point, but I see no harm in discussing these rumors. It's all we have, until actual numbers are shown.
And these unconfirmed benchmarks, will later be used to be compared with the actual ones, so that we can at least have an idea of how credible/not credible that source is/was. How do you think The Inq got its (in)famous name, as source of unreliable info ? Because their rumors and unconfirmed numbers, where compared to final ones and the results show that they spread more FUD than they deliver good pieces of information.

Also, if we don't discuss rumors, about upcoming products, what is there to talk about in these sub-forums ?
Click to expand...

thats fine.. to discuss rumors

but is a total different thing when people are starting to call unreleased products a failure when there are no official benchmarks
 
Aztlan said:
thats fine.. to discuss rumors

but is a total different thing when people are starting to call unreleased products a failure when there are no official benchmarks
Click to expand...

The people that are calling it a failure, are basing that remark on those unconfirmed rumors and/or benchmarks. They are in NO way, a reflection of the truth, since we don't know it yet. A thread about rumors does NOT set the truth factor. I thought that was clear from the start.

If they are just saying "AMD sucks and Intel/NVIDIA wins" then that's the typical *!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*! claim, which should just be ignored. Otherwise, we would never discuss anything, because there's always a *!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*! (or more) right around the corner, to destroy a perfectly good thread. And besides, that's why these forums have mods :)
 
Aztlan said:
thats fine.. to discuss rumors

but is a total different thing when people are starting to call unreleased products a failure when there are no official benchmarks
Click to expand...


QFT. We don't know the 100% "truth" yet, and even if ATI doesn't take the performance crown (which I must say is looking likely) there are other things to consider. Featureset, driver functionality/stability, and performance/dollar.

Not only that but you must remember ATI isn't sitting idly by either (nor is NV). There is much more to the videocard market then just the top tier products.
 
I saw somewhere that AMD/ATI admitted that the their new card couldn't compare to the GTX and only could run with the GTS. I'm looking for that link now...
 
Digital Viper-X- said:
Evga 680i + another 8800gts here I come!! :p
Click to expand...

I know you posted that tongue in cheek, but you have a point. People who are trying to make a purchase decision right now need to go off currently available information, if that's just canned benchmarks then so be it. Not everyone has the luxury of waiting until all the results are in before making a move.
 
Hvatum said:
I know you posted that tongue in cheek, but you have a point. People who are trying to make a purchase decision right now need to go off currently available information, if that's just canned benchmarks then so be it. Not everyone has the luxury of waiting until all the results are in before making a move.
Click to expand...

actually im seriously considering it now, the evga 680i mobo is on sale for $209CAD :|
 
Sharky974 said:
Has Kyle and enlightened forumers taught us nothing? Those are completely meaningless to real world gamreplay. R600 is NOT a failure, in fact it may be much, much better than 8800GTX in real world gaming.

Lets all do a favor and stop discussing canned benchmarks and timedemos!

I saw a lot of guys pointing the folly of canned benchmarks out in Kyle's 8600 review thread, so come all you guys! Back me up by the hundreds! I'm right here waiting!
Click to expand...

Different people, different views. Personally I like canned benchmarks because they are consistent, and I like comparing the same settings to the same settings. Not different resolutions and settings on each card. So I judge by canned benchmarks, and ignore H when it comes to graphic card reviews. There are plenty of others and I read a bunch to get a decent picture.

So based on what I have seen the 8600GTS is no better than an ATI 1950pro. So the price performance midrange has not improved at all. Not the least bit exciting.

By the same token there is enough information to judge the HD 2900. ATI laid an egg and it is not golden. They are six months late and the card they aimed at the top will have to be price reduced to compete with second best. They have nothing to take on the top. The new midrange parts (2600,2400) will be paper launched.

I am actually a bit of an ATI fan (never owned Nvidia), but ATI messed up. I may be set to buy my first Nvidia based card.
 
So far all I've seen is a reason to be skeptical of the XTX performance (though I hardly believe the XTX's core is clocked only 5mhz faster than the XT, especially when they managed a 100mhz overclock on the XT bringing the core to 845, and got the memory from 1.65 to 1.99).

If you're looking for GTS-like performance, I don't know why the XT wouldn't be on your short list to wait for. We'll see how things pan out when proper benches and such are released.
 
Sharky974 said:
Has Kyle and enlightened forumers taught us nothing? Those are completely meaningless to real world gamreplay. R600 is NOT a failure, in fact it may be much, much better than 8800GTX in real world gaming.
Click to expand...

[The Matrix]There is no spoon[/The Matrix]
 
Silus said:
They are not "flawed". They're just the way the most typical benchmarks are/were done. The fact that IMHO, [H] evolved to include better ways to compare video cards, doesn't mean the other benchmarks are "flawed". Here they are called canned and I'm ok with the designation, but I still use them to have a more synthetic analysis on the product.
Click to expand...

In reality, which the scientific method describes, the "real-world" ones are flawed, whereas the "synthetic canned" benches are more accurate and valid. I prefer to know hard numbers, not "what if" numbers. The tags are swapped, really... the "canned" ones are far more real-world than the kind this site uses :(. I can understand the liking to both kinds (which I do like both kinds), but calling the "canned" ones worse or flawed is absurd.
 
Snowdog said:
Different people, different views. Personally I like canned benchmarks because they are consistent, and I like comparing the same settings to the same settings. Not different resolutions and settings on each card. So I judge by canned benchmarks, and ignore H when it comes to graphic card reviews. There are plenty of others and I read a bunch to get a decent picture.
Click to expand...

Unfortunately, I do the same. I wish the H offered so-called "canned" benchmarks in addition to their quote-on-quote "real-world" ones. They'd be much more useful.
 
GoldenTiger said:
In reality, which the scientific method describes, the "real-world" ones are flawed, whereas the "synthetic canned" benches are more accurate and valid. I prefer to know hard numbers, not "what if" numbers. The tags are swapped, really... the "canned" ones are far more real-world than the kind this site uses :(. I can understand the liking to both kinds (which I do like both kinds), but calling the "canned" ones worse or flawed is absurd.
Click to expand...

The point is that true "real-world" benchmarks give people results they will actually experience if they own the card. "Canned" benchmarks allow all sorts of limiting factors depending on what resolution and what kind of IQ settings you're running, and can be easily misrepresented.
 
quadnad said:
The point is that true "real-world" benchmarks give people results they will actually experience if they own the card. "Canned" benchmarks allow all sorts of limiting factors depending on what resolution and what kind of IQ settings you're running, and can be easily misrepresented.
Click to expand...

A) "Canned" benchmarks keep the playing field level, and don't introduce random stressors or limits on the amount of stress a card may encounter which cause invalid results.

B) Timedemos are recordings of actual gameplay. Your argument is like saying a recording of a football game played on different televisions to judge their quality is a bad thing to do, and that the football game was not real. A false conclusion.

C) Misrepresented? That's what your brain's for, to think with :)! Most good sites keep the commentary very limited, letting you draw accurate conclusions.
 
GoldenTiger said:
A) "Canned" benchmarks keep the playing field level, and don't introduce random stressors or limits on the amount of stress a card may encounter which cause invalid results.

B) Misrepresented? That's what your brain's for, to think with :)! Most good sites keep the commentary very limited, letting you draw accurate conclusions.
Click to expand...

Don't introduce random stressors? The idea is that in a game like (for example) Oblivion, not only do you know that it runs x frames at x resolution, but rather that the maximum playable settings (maintaining the best IQ) are at a certain level. It's more realistic because it shows you what kinds of settings will be achievable with a given card. You keep the target FPS constant, and then find out the best IQ settings at that level to compare cards.

EDIT: In response to your Ninja edit, I never said that the canned benchmarks aren't running real gameplay; I'm saying that the settings upon which they are playing allows performance to be misrepresented. Just like (for example) the initial XT benches against the GTS at 12x10 were CPU bound. Of course I'm intelligent enough to disregard them, but then I'm still left without an accurate idea as to what kinds of settings I can play at.
 
quadnad said:
A. Don't introduce random stressors? The idea is that in a game like (for example) Oblivion, not only do you know that it runs x frames at x resolution, but rather that the maximum playable settings (maintaining the best IQ) are at a certain level. It's more realistic because it shows you what kinds of settings will be achievable with a given card.
Click to expand...

You can draw the exact same conclusions as to what a playable level is by seeing the framerate numbers on a non-random timedemo, and knowing what you are comfortable with as min/max/avg. frames. Basically, using "real-world" "benches" is letting someone else's opinion show you the result for you, instead of letting you make your own conclusions. The problem is, with Oblivion for instance, you DON'T know it runs x frames at x resolution if you use a non-timedemo bench, because the grass is randomized, enemy appearances/encounters are, as well as weather effects, whether the player looks over a vista (huge view of mountains that will kill the framerate) by accident while fighting that he didn't for 20 seconds while doing a different card's run, etc. It's not controlled.
 
GoldenTiger said:
The problem is, with Oblivion for instance, you DON'T know it runs x frames at x resolution if you use a non-timedemo bench, because the grass is randomized, enemy appearances/encounters are, as well as weather effects, whether the player looks over a vista (huge view of mountains that will kill the framerate) by accident while fighting that he didn't for 20 seconds while doing a different card's run, etc. It's not controlled.
Click to expand...

No matter the position of the grass in a 'random' environment from Oblivion, your framerate will undoubtedly be +/- 2 FPS. If you select your run through intelligently, nothing significant changes. I count on [H] to do that, and they even tell us the exact route they take.
 
GoldenTiger said:
You can draw the exact same conclusions as to what a playable level is by seeing the framerate numbers on a non-random timedemo, and knowing what you are comfortable with as min/max/avg. frames. Basically, using "real-world" "benches" is letting someone else's opinion show you the result for you, instead of letting you make your own conclusions. The problem is, with Oblivion for instance, you DON'T know it runs x frames at x resolution if you use a non-timedemo bench, because the grass is randomized, enemy appearances/encounters are, as well as weather effects, whether the player looks over a vista (huge view of mountains that will kill the framerate) by accident while fighting that he didn't for 20 seconds while doing a different card's run, etc. It's not controlled.
Click to expand...

actually you can't, specially with a game like oblivion, you can have ur FPS reduce drasticly by looking to the side while you ride along on your horse these are things I actually do when I play, Also since the grass and other thigns are randomized then even a scripted demo couldn't produce the same result twice
 
quadnad said:
No matter the position of the grass in a 'random' environment from Oblivion, your framerate will undoubtedly be +/- 2 FPS. If you select your run through intelligently, nothing significant changes. I count on [H] to do that, and they even tell us the exact route they take.
Click to expand...

That's one of the reasons I consider them useless. You don't know how much might change, because it is random, and you may encounter monsters you have to fight that cause different scenes to come into play during the run-through. While they can be useful supplements to "canned" results, they can't stand on their own legs.

Digital Viper-X- said:
actually you can't, specially with a game like oblivion, you can have ur FPS reduce drasticly by looking to the side while you ride along on your horse these are things I actually do when I play, Also since the grass and other thigns are randomized then even a scripted demo couldn't produce the same result twice
Click to expand...

Hence why this "real-world" bench is inaccurate, because of the randomness like that which can result in drastically lowered framerates during gameplay. The "canned" bench results in the exact same movements and motions, creating a more accurate environment.
 
GoldenTiger said:
That's one of the reasons I consider them useless. You don't know how much might change, because it is random, and you may encounter monsters you have to fight that cause different scenes to come into play during the run-through. While they can be useful supplements to "canned" results, they can't stand on their own legs.
Click to expand...

A very significant change in the run-throughs can change the results. Of course I agree.

However, it's the job of [H] to maintain a relative level of consistency throughout the run-throughs. [H]'s Oblivion test for example takes them over ten minutes to complete. With that kind of a sample, and with them looking for significant changes (if he somehow got into a brawl with fourty monsters or something), I can hardly believe that those average framerates would change by more than a frame or two. They stress the cards in terms of their limits in the same way (running the torch and such) to find minimum framerates. In my opinion it's the best method.
 
quadnad said:
A very significant change in the run-throughs can change the results. Of course I agree.

However, it's the job of [H] to maintain a relative level of consistency throughout the run-throughs. [H]'s Oblivion test for example takes them over ten minutes to complete. With that kind of a sample, and with them looking for significant changes (if he somehow got into a brawl with fourty monsters or something), I can hardly believe that those average framerates would change by more than a frame or two. They stress the cards in terms of their limits in the same way (running the torch and such) to find minimum framerates. In my opinion it's the best method.
Click to expand...

The reason I prefer "canned" ones (note that from the outset in one of these two threads I had said I find both useful, but vastly prefer the "canned" ones) is because it eliminates the possiblity of operator error or random factors coming into play. You are correct, insofar as that they come up with basically the same conclusions, whether the numbers vary or not, but I like seeing exacts personally. It's really the whole "they're special and so much better" attitude that I dislike from certain posters now and in the past.
 
GoldenTiger said:
It's really the whole "they're special and so much better" attitude that I dislike from certain posters now and in the past.
Click to expand...

Yeah, I hear you. I don't maintain a view of [H] superiority, I just feel that their methodology is more informative than others. It's a good thing we can check out all sorts of different sites when measuring the performance of our next purchases :D
 
GoldenTiger said:
Hence why this "real-world" bench is inaccurate, because of the randomness like that which can result in drastically lowered framerates during gameplay. The "canned" bench results in the exact same movements and motions, creating a more accurate environment.
Click to expand...

thats a great point, and i wasnt argueing against the fact that the environment changes with people at the helm, but that exactly is the point, I want to know how this card is going to perform with someone playing, doing different things, being "random",

I'm not going to play the game the same way anand, Hocp, firingsquad, vrzone, hexus, vl, etc is going to, so why do i care about how well it plays under "those exact situations" I need to see how the card performs overall, does it provide me with a more pleasant gaming experience by being able to handle my randomness, or is it going to hitch everytime I change my mind and look another way,

plus , =) I don't only read the hardocp review, i take a look at others and come to my own conclusion, but its nice to have a human at the helm playing the game rather then watching or waiting for the rest to finish
 
I have this hunch the OP was being sarcastic... ;)

And even if he wasnt... it would take some incredible leaps for these numbers from these "canned" benchmarks to come close in the "real world" tests...
 
Sharky974 said:
R600 is NOT a failure, in fact it may be much, much better than 8800GTX in real world gaming.
Click to expand...
What's worse: calling a card a failure before it's released, or saying it is NOT a failure before it's released?

It could very well be a failure. Don't be contradictory.

Sharky974 said:
Back me up by the hundreds! I'm right here waiting!
Click to expand...
Keep waiting.

Sharky974 said:
I'm not talking about rumors, but flawed benchmarking methods.
Click to expand...
How specifically are they flawed? Five years ago, when nobody was doing [H]-style evaluations, was each and every benchmark flawed? If the X2900 XTX surpassed the GTX by 8000% (in this case being the realization of your fantasy), would the benchmark be flawed?

Let's get real here. Everyone here understands precisely why this topic was created. Your own blind anticipations disagree with the results. Ergo, the results are not of any merit to you. In addition, they should not be discussed as if they have some degree of merit, because you've already established to yourself that they don't, which means that your expectation is that everyone should share your opinion. The term I would most readily apply to this type of thinking is "indescribably arrogant".

Is this or is this not an incorrect summarization of reality?
 
This thread is positively ridiculous. Who in their right mind would ask people on a hardware forum *not* to discuss the most interesting video card news since the launch of the 8800GTX?

That being said, I think rumors that ATi isn't going to try to compete with the 8800GTX coming the same day as these benchmarks from DT is a pretty clear indication that the X2900XTX is not what most of us were expecting. We can revisit this argument in a month or so when real numbers are available, but I'll go ahead and say I expect full benchmarks will only confirm what we've seen today: R600 will not be retaking the performance crown.

As for the whole canned vs real-world benchmark argument, I've said it before and I'll say it again. [H]ard|OCP's 'real-world' benchmarks have never led to any more enlightened conclusions than anyone elses benchmarks. All you guys who want to throw these benchmarks out on the basis that they aren't 'real-world' enough for you need to just give up. DailyTech knows how to run a benchmark, and the 8800GTX is beating the X2900XTX by a wide enough margin that no reasonable amount of error due to testing configuration or benchmark configuration is going to change the results.

If you want to hold out hope for R600, hope for a new driver before it launches.
 
phide said:
Let's get real here. Everyone here understands precisely why this topic was created. Your own blind anticipations disagree with the results. Ergo, the results are not of any merit to you. In addition, they should not be discussed as if they have some degree of merit, because you've already established to yourself that they don't, which means that your expectation is that everyone should share your opinion. The term I would most readily apply to this type of thinking is "indescribably arrogant".
Click to expand...

I think 'denial' is more applicable, but well said nonetheless.
 
You must log in or register to reply here.
Back
Top