GPU Silicon Quality & Overclock Lottery Test: Golden GPU Search
video description
Date: 2020-05-06
Related videos
Comments and reviews: 10
Micah
I do think a sample size of 1 is enough. I'll explain: Cause most people won't buy more than 1 of the same GPU, just as Steve said at the beginning of this video. So testing the card with a sample size of one is just as accurate as a sample size of 100. Cause a person buying that GPU wouldn't have control over which he got anyways. So for 99, 99% of people testing it with 1 sample will be good enough. Yes, you could get the perfect average with a sample size of 100 or more, but it isn't useful for 99, 99% of the people buying those GPUs, cause most people don't have the money to throw at buying 100 GPUs testing them and sell the other 99 worse ones. So even though you could figure out the average performance of the silicon, since nobody has control over which he gets it wouldn't really help the review or how it pairs to other GPUs. Cause you are still either getting a worse GPU than the average, or a better GPU than the average or just the average GPU. So in the real world(Psst the one we live in, it won't matter that much. The only thing it would be useful for is testing the quality of production of a particular GPU and testing how much that quality range, diverse compared to other GPUs But this isn't really improving benching testing, but could in someway maybe impact somebodies buying decision if they would be intending to overclock the GPU and want to know what the chances are he is going to get good silicon. But as I said before that isn't the majority. Of course, if you can do the tests it is always better to do extra testing with bigger sample sizes. But I don't think it is in any way required for an accurate review.
reply
I do think a sample size of 1 is enough. I'll explain: Cause most people won't buy more than 1 of the same GPU, just as Steve said at the beginning of this video. So testing the card with a sample size of one is just as accurate as a sample size of 100. Cause a person buying that GPU wouldn't have control over which he got anyways. So for 99, 99% of people testing it with 1 sample will be good enough. Yes, you could get the perfect average with a sample size of 100 or more, but it isn't useful for 99, 99% of the people buying those GPUs, cause most people don't have the money to throw at buying 100 GPUs testing them and sell the other 99 worse ones. So even though you could figure out the average performance of the silicon, since nobody has control over which he gets it wouldn't really help the review or how it pairs to other GPUs. Cause you are still either getting a worse GPU than the average, or a better GPU than the average or just the average GPU. So in the real world(Psst the one we live in, it won't matter that much. The only thing it would be useful for is testing the quality of production of a particular GPU and testing how much that quality range, diverse compared to other GPUs But this isn't really improving benching testing, but could in someway maybe impact somebodies buying decision if they would be intending to overclock the GPU and want to know what the chances are he is going to get good silicon. But as I said before that isn't the majority. Of course, if you can do the tests it is always better to do extra testing with bigger sample sizes. But I don't think it is in any way required for an accurate review.
reply
Monkeh
This was an AWESOME video, I've always wondered about this kind of stuff! I think it would also be super interesting, and even somewhat useful to me personally when choosing video cards, if you get a large sample size from a company of two different SKUs, one SKU being their mid-low or middle of the stack SKU, and then one from the highest of the high SKU (not including perhaps SKUs like Kingpin which are given extreme binning consideration, and it might be best to avoid product stacks/manufactures who have halo products such as that to avoid golden samples being snatched away. I think this would be interesting and useful because I've always kind of assumed that once you reach the top of the stack of a certain video card/product line, it would almost make sense that there may be a lot more overclocking headroom among those SKUs because it is as far as they bother to selectively bin. With the middle of the stack cards, I also tend to think that they will only overclock so high because if it did much better, it would have been selected for one of the manufacturer's higher-priced and -tiered video cards. Thanks for the video, looking forward to possibly another video like this one in the future!
reply
This was an AWESOME video, I've always wondered about this kind of stuff! I think it would also be super interesting, and even somewhat useful to me personally when choosing video cards, if you get a large sample size from a company of two different SKUs, one SKU being their mid-low or middle of the stack SKU, and then one from the highest of the high SKU (not including perhaps SKUs like Kingpin which are given extreme binning consideration, and it might be best to avoid product stacks/manufactures who have halo products such as that to avoid golden samples being snatched away. I think this would be interesting and useful because I've always kind of assumed that once you reach the top of the stack of a certain video card/product line, it would almost make sense that there may be a lot more overclocking headroom among those SKUs because it is as far as they bother to selectively bin. With the middle of the stack cards, I also tend to think that they will only overclock so high because if it did much better, it would have been selected for one of the manufacturer's higher-priced and -tiered video cards. Thanks for the video, looking forward to possibly another video like this one in the future!
reply
Sean
Based on these results, are there are a lot of people out there using poor performing cards that are stable but nonetheless manage to pull about 20% - 40% less performance than identical cards for people who paid the same for the same product? How many people are we talking about being affected this way? I could see a class action lawsuit forming out of this. If Honda sells 100, 000 Honda Civics, and 5% of them get around 50 mpg, 5% get around 30 mpg, 50% get 40 mpg, 20% get 35 mpg and 20% get 45 mpg -- assuming the same driver inputs, climate, etc -- you bet there would be a class action. Or if Coke sold some customers 12 oz of soda in a 16. 9 oz bottle. Manufacturing tolerances can't be completely zeroed out, but the differences between the poor-performing cards and the good ones are quite substantial in some of these cases. With a 144 Hz monitor it's easy to tell the difference between 80 and 105 FPS and a few of the tests showed such a discrepancy. Even if only 5% of their cards are that bad, you're talking, like, close to a million people probably who bought a 1060 and are getting more like 1050 or 1050 Ti performance. Not cool.
reply
Based on these results, are there are a lot of people out there using poor performing cards that are stable but nonetheless manage to pull about 20% - 40% less performance than identical cards for people who paid the same for the same product? How many people are we talking about being affected this way? I could see a class action lawsuit forming out of this. If Honda sells 100, 000 Honda Civics, and 5% of them get around 50 mpg, 5% get around 30 mpg, 50% get 40 mpg, 20% get 35 mpg and 20% get 45 mpg -- assuming the same driver inputs, climate, etc -- you bet there would be a class action. Or if Coke sold some customers 12 oz of soda in a 16. 9 oz bottle. Manufacturing tolerances can't be completely zeroed out, but the differences between the poor-performing cards and the good ones are quite substantial in some of these cases. With a 144 Hz monitor it's easy to tell the difference between 80 and 105 FPS and a few of the tests showed such a discrepancy. Even if only 5% of their cards are that bad, you're talking, like, close to a million people probably who bought a 1060 and are getting more like 1050 or 1050 Ti performance. Not cool.
reply
Asi9
Edit: Actually, GPU 3's performance is a bit concerning now that I think about it. For the average consumer who is just going to pop in their new card and get straight to gaming, how would they know that particular card was defective? Obviously the performance wasn't so bad as to be unplayable, and unless there was something like artifacting or crashing going on, how would they even know that they received a lemon unless they ran benchmarks on a same or very similar setup to a reviewer and compared their numbers? I could see a lot of users out there running cards with degraded performance such as that particular card, and not even realize they have an RMA on their hands. This is an amazing idea and I can't believe no one has thought to try and do a real world comparative test of the performance gap card to card. Everyone has always known the silicon lottery on GPUs was a thing, but I can't recall anyone ever actually attempting to test just how large the variances are.
reply
Edit: Actually, GPU 3's performance is a bit concerning now that I think about it. For the average consumer who is just going to pop in their new card and get straight to gaming, how would they know that particular card was defective? Obviously the performance wasn't so bad as to be unplayable, and unless there was something like artifacting or crashing going on, how would they even know that they received a lemon unless they ran benchmarks on a same or very similar setup to a reviewer and compared their numbers? I could see a lot of users out there running cards with degraded performance such as that particular card, and not even realize they have an RMA on their hands. This is an amazing idea and I can't believe no one has thought to try and do a real world comparative test of the performance gap card to card. Everyone has always known the silicon lottery on GPUs was a thing, but I can't recall anyone ever actually attempting to test just how large the variances are.
reply
Tiki832
I find it interesting that you did a video of this subject focused around GPUs, as typically performance variance in GPU Silicon quality is something I'd say most people aren't all that bothered with even if you're trying to push overclocks. Where as the variance you'd see in CPUs seems the more logical thing to focus on for a video of this subject to highlight just how significant (Or insignificant) the impact of Silicon quality can have on performance. Though if the idea behind the video was to poke into a subject not normally talked about to see if it's perhaps something we should be talking about, memory would have been a pretty interesting alternative to GPUs. Just how much does Silicon quality impact memory capability in both clock speeds and timings and across multi-vendor systems how does that translate into actual performance?
reply
I find it interesting that you did a video of this subject focused around GPUs, as typically performance variance in GPU Silicon quality is something I'd say most people aren't all that bothered with even if you're trying to push overclocks. Where as the variance you'd see in CPUs seems the more logical thing to focus on for a video of this subject to highlight just how significant (Or insignificant) the impact of Silicon quality can have on performance. Though if the idea behind the video was to poke into a subject not normally talked about to see if it's perhaps something we should be talking about, memory would have been a pretty interesting alternative to GPUs. Just how much does Silicon quality impact memory capability in both clock speeds and timings and across multi-vendor systems how does that translate into actual performance?
reply
Julian
You keep saying GPU 3 is 'defective' -- was it having artifacting issues or anything? It appears it was able to make it through all of your tests, albeit at a lower performance. But it did make it through, right? I would wager GPU 3 is a prime example of the silicon lottery. It's not broken or defective. It's a bad GPU sample. It has a GPU that should have been tossed aside during quality control. It makes it an excellent example of just how massive a difference in performance can be based on the quality of the silicon.
reply
You keep saying GPU 3 is 'defective' -- was it having artifacting issues or anything? It appears it was able to make it through all of your tests, albeit at a lower performance. But it did make it through, right? I would wager GPU 3 is a prime example of the silicon lottery. It's not broken or defective. It's a bad GPU sample. It has a GPU that should have been tossed aside during quality control. It makes it an excellent example of just how massive a difference in performance can be based on the quality of the silicon.
reply
Cmdr
Do you guys compare you're benchmark results with other reviewers before publishing a video, or do manufacturers give you an expected performance guide? Just wondering what would have happened if you'd ended up with GPU 3 as a review sample for your 1070ti review. Probably not the best example since the numbers would have been lower than a standard 1070 so you'd definitely have known there was something wrong with the card, but have you ever had to re-shoot an entire review vid after realising you'd gotten a lemon?
reply
Do you guys compare you're benchmark results with other reviewers before publishing a video, or do manufacturers give you an expected performance guide? Just wondering what would have happened if you'd ended up with GPU 3 as a review sample for your 1070ti review. Probably not the best example since the numbers would have been lower than a standard 1070 so you'd definitely have known there was something wrong with the card, but have you ever had to re-shoot an entire review vid after realising you'd gotten a lemon?
reply
Nathan
I massively won the silicon lottery. My CPU and GPU are in the 98th and 95th percentile of performance, respectively. Unfortunately, it's an A10 laptop with dual graphics from 2013, so it can't run many things at decent FPS anyway. Old single-player strategy games are great with it though (Civ 5, RTW1, Endless Space, AoE 2&3. Even For Honor is just about playable on low settings) - but for multiplayer and games which rely on high single-core performance, I may as well be using a potato.
reply
I massively won the silicon lottery. My CPU and GPU are in the 98th and 95th percentile of performance, respectively. Unfortunately, it's an A10 laptop with dual graphics from 2013, so it can't run many things at decent FPS anyway. Old single-player strategy games are great with it though (Civ 5, RTW1, Endless Space, AoE 2&3. Even For Honor is just about playable on low settings) - but for multiplayer and games which rely on high single-core performance, I may as well be using a potato.
reply
Max
out of box stock is very shitty comparisons the difference is going to be shown at the extreme. also most people missunderstand it. its not how high you can OC but how cool it gets. a asic of 60% might only be able to undervolt -50mv but a asic of 95% might be able to undervolt -200mv this will cause 10-20c lower temp and make it silent at same clock. most will hit the same upper ceiling of mhz from electron jumping and architecture limits.
reply
out of box stock is very shitty comparisons the difference is going to be shown at the extreme. also most people missunderstand it. its not how high you can OC but how cool it gets. a asic of 60% might only be able to undervolt -50mv but a asic of 95% might be able to undervolt -200mv this will cause 10-20c lower temp and make it silent at same clock. most will hit the same upper ceiling of mhz from electron jumping and architecture limits.
reply
Capt.
Great video & thank you for it, although an appearance by Snowflake would've made it a bit better A video I would like to see is benchmarks of all the different skus of one model from one manufacturer. It would be interesting to see just how much difference there is between (for example) an MSI Ventus, Armor, Gaming Z, et al, of the 2060 Super. And, are the differences just clock speeds? Thanks again for another informative video!
reply
Great video & thank you for it, although an appearance by Snowflake would've made it a bit better A video I would like to see is benchmarks of all the different skus of one model from one manufacturer. It would be interesting to see just how much difference there is between (for example) an MSI Ventus, Armor, Gaming Z, et al, of the 2060 Super. And, are the differences just clock speeds? Thanks again for another informative video!
reply
Add a review, comment
Other channel videos
