Why Most Cooler Tests Are Flawed: CPU Cooler Testing Methodology
video description
Date: 2020-05-06
Related videos
Comments and reviews: 10
Rocky12345
I do get what you are saying in the video and I applaud you guys for doing your very best to make sure every aspect of cooler testing is done and done right. I do feel though that not only do coolers need to be tested in a 100% controlled test. I also feel every cooler also needs to be tested as before on actual boards with CPU's to see how the cooler. CPU, main board all work together in a real computer setup to get that data as well. I also see the need for the Delta over amb tests but I also feel actual temps of the CPU should be shown in the graphs mainly because we then get both sets of data to look at. Also if Amb temp is not shown in a cooler review and the tests only show Delta over Amb with have no idea what the CPU was actually running at under full load tests. In my own system with to tests temps I use the same software every time to test my CPU temps under full CPU load. The biggest thing that can change my CPU temps is amb temp. So if my amb temp is 21c my CPU may go up to 68c as a max temp after 30 minutes of full load using a CPU torture test software and yet in the same 21c amb temp in a game the CPU only runs at maybe 47c-49c in a game load. Then if amb temp goes up to 28c in the summer months that same test will raise to 75c under full load torture test. In game temps also raise to 54c-56c in the same amb temp of 28c. So in my example temps go up as expected with amb temp increase to pretty close or right on the money. I use a 280mm EVGA CLC on my CPU and noticed the temps pretty much flatline after about 20 minutes as the CLC reaches it working temp anything after that seems to pretty hold pretty steady in very long game sessions or full load torture tests. My CPU is a old i7 2600K 5. 1GHz with controlled voltage set to max out at 1. 47 volts with load line set to turbo on my Gigabyte board if I set it to extreme load line the board then tries to over shoot my set max voltage. I used to use a big air cooler on this CPU form 2011-2018 and was only able to get 5. 0GHz before the cooler would get to hot and 5. 1GHz was a no go and required more voltage. After switching to the 280mm CLC I was able to lower voltage down to 1. 45 volts for 5. 0GHz and 1. 47 5. 1GHz was finally a go and have good temps. I feel my testing on my own system has shown me that everything is pretty much spot on and is 100% rock solid stable for such an old platform in 2020.
reply
I do get what you are saying in the video and I applaud you guys for doing your very best to make sure every aspect of cooler testing is done and done right. I do feel though that not only do coolers need to be tested in a 100% controlled test. I also feel every cooler also needs to be tested as before on actual boards with CPU's to see how the cooler. CPU, main board all work together in a real computer setup to get that data as well. I also see the need for the Delta over amb tests but I also feel actual temps of the CPU should be shown in the graphs mainly because we then get both sets of data to look at. Also if Amb temp is not shown in a cooler review and the tests only show Delta over Amb with have no idea what the CPU was actually running at under full load tests. In my own system with to tests temps I use the same software every time to test my CPU temps under full CPU load. The biggest thing that can change my CPU temps is amb temp. So if my amb temp is 21c my CPU may go up to 68c as a max temp after 30 minutes of full load using a CPU torture test software and yet in the same 21c amb temp in a game the CPU only runs at maybe 47c-49c in a game load. Then if amb temp goes up to 28c in the summer months that same test will raise to 75c under full load torture test. In game temps also raise to 54c-56c in the same amb temp of 28c. So in my example temps go up as expected with amb temp increase to pretty close or right on the money. I use a 280mm EVGA CLC on my CPU and noticed the temps pretty much flatline after about 20 minutes as the CLC reaches it working temp anything after that seems to pretty hold pretty steady in very long game sessions or full load torture tests. My CPU is a old i7 2600K 5. 1GHz with controlled voltage set to max out at 1. 47 volts with load line set to turbo on my Gigabyte board if I set it to extreme load line the board then tries to over shoot my set max voltage. I used to use a big air cooler on this CPU form 2011-2018 and was only able to get 5. 0GHz before the cooler would get to hot and 5. 1GHz was a no go and required more voltage. After switching to the 280mm CLC I was able to lower voltage down to 1. 45 volts for 5. 0GHz and 1. 47 5. 1GHz was finally a go and have good temps. I feel my testing on my own system has shown me that everything is pretty much spot on and is 100% rock solid stable for such an old platform in 2020.
reply
Wyvern
This is why I open loop, no matter what block or what radiator you buy (of the same size) you will be within 7C of the worst vs. the best- so in my opinion, go with what looks nice and has lower restriction (lower restriction more flow, more flow, better efficiency) Also I have found that with thermal paste, buy high quality, smooth paste (like Noctua) and use more than you would expect, recently I pasted with a good helping of thermal paste painted the IHS, I thought for sure it was more than enough- lo and behold when I removed the block there wasn't enough, so I did the 5 blob dice method, perfect coverage and a temp of 60C on prime 95 on my 3900X Likewise, I did the X and sausage method on my VII GPU, temps sucked 50% over power 66C edge and 108C junction, swapped it out put considerably more and now have temps of 60C edge and 92C Junction- letting me run 2100hz peak and a steady clock around 2040hz (stock peak is around 1750 constant at around 1650) Stability is hugely improved- no more crashing. Also, closed loop sucks, your performance WILL tank over time and it will fail in a few years- worst thing is you can't see when it's gunked up or going dry. If you want to water cool, go open loop, otherwise go air.
reply
This is why I open loop, no matter what block or what radiator you buy (of the same size) you will be within 7C of the worst vs. the best- so in my opinion, go with what looks nice and has lower restriction (lower restriction more flow, more flow, better efficiency) Also I have found that with thermal paste, buy high quality, smooth paste (like Noctua) and use more than you would expect, recently I pasted with a good helping of thermal paste painted the IHS, I thought for sure it was more than enough- lo and behold when I removed the block there wasn't enough, so I did the 5 blob dice method, perfect coverage and a temp of 60C on prime 95 on my 3900X Likewise, I did the X and sausage method on my VII GPU, temps sucked 50% over power 66C edge and 108C junction, swapped it out put considerably more and now have temps of 60C edge and 92C Junction- letting me run 2100hz peak and a steady clock around 2040hz (stock peak is around 1750 constant at around 1650) Stability is hugely improved- no more crashing. Also, closed loop sucks, your performance WILL tank over time and it will fail in a few years- worst thing is you can't see when it's gunked up or going dry. If you want to water cool, go open loop, otherwise go air.
reply
Thetargos
Steve, you and your team really deserve all my respect. The methdology you put into testing, and this particular topic which has so many variables, and in the end usually translates into perceived performance is essential not only for us nerds and geeks who are passionate about PCs, but also to manufacturers of both the products you test and feature, and for those doing system's integration and building. The degree (pun intended) of attention to details so to avoid the dreaded bias, with emphasis in reproducibility and automation to the greatest extent and the lengths you have gone in order to produce such. Hat's off to you, Sir. I'm eager to see more content on this topic espcially using that dummy heater, which is. mind blowing (every cooler manufacturer I suppose has their own version of one or similar. At any rate, kudos. Keep it up, you find ways to amaze us every time you post out a new video (even if it is only to tell the news ) PS: I bet many universities will borrow methodology from your tests (and do not mean thermals only, but in general.
reply
Steve, you and your team really deserve all my respect. The methdology you put into testing, and this particular topic which has so many variables, and in the end usually translates into perceived performance is essential not only for us nerds and geeks who are passionate about PCs, but also to manufacturers of both the products you test and feature, and for those doing system's integration and building. The degree (pun intended) of attention to details so to avoid the dreaded bias, with emphasis in reproducibility and automation to the greatest extent and the lengths you have gone in order to produce such. Hat's off to you, Sir. I'm eager to see more content on this topic espcially using that dummy heater, which is. mind blowing (every cooler manufacturer I suppose has their own version of one or similar. At any rate, kudos. Keep it up, you find ways to amaze us every time you post out a new video (even if it is only to tell the news ) PS: I bet many universities will borrow methodology from your tests (and do not mean thermals only, but in general.
reply
joshua43214
Your frustration with this topic is clear to me. One of the very first things you learn as a scientist, is that your data is only as good as your controls - if your controls are flawed, the data is not valid, period. You just spent three years learning this the hard way. Always ask yourself What am I really testing here? I never believe cooler reviews because they have no controls. Your test bench should provide a method to test the coolers ability to dissipate heat in a controlled manner - this is the primary rating of a cooler. BTW, the heater blocks from in-situ PCR thermocyclers can be had cheap and used (it is mostly obsolete tech not, and can provide a simple method of applying pin point heat to imitate the mapping of hot spots in the a CPU. This test though would not test the ability of the cooler to dissipate heat, it would test the ability of the mount to transfer the heat to the pipes.
reply
Your frustration with this topic is clear to me. One of the very first things you learn as a scientist, is that your data is only as good as your controls - if your controls are flawed, the data is not valid, period. You just spent three years learning this the hard way. Always ask yourself What am I really testing here? I never believe cooler reviews because they have no controls. Your test bench should provide a method to test the coolers ability to dissipate heat in a controlled manner - this is the primary rating of a cooler. BTW, the heater blocks from in-situ PCR thermocyclers can be had cheap and used (it is mostly obsolete tech not, and can provide a simple method of applying pin point heat to imitate the mapping of hot spots in the a CPU. This test though would not test the ability of the cooler to dissipate heat, it would test the ability of the mount to transfer the heat to the pipes.
reply
ShozzleMeNoz
I'm sorry, this is nonsense as far as helping a consumer to choose which cooler to buy. As soon as you started saying you've got to use the same thermal paste, there was no longer any point to listening. A consumer buying a cooler on the strength of your numbers is far more often than not going to use the thermal paste that comes with it. If a cooler manufacturer provides better thermal paste as part of their product, which allows it to achieve better temps, it's a better product because most people aren't going to order thermal paste separately or entirely clean off pre-applied thermal paste to apply whatever thermal paste they previously used. If I want to buy a cooler, I want the performance it comes out of the box with, not the performance when you pair it with an arbitrary thermal paste you've decided to use.
reply
I'm sorry, this is nonsense as far as helping a consumer to choose which cooler to buy. As soon as you started saying you've got to use the same thermal paste, there was no longer any point to listening. A consumer buying a cooler on the strength of your numbers is far more often than not going to use the thermal paste that comes with it. If a cooler manufacturer provides better thermal paste as part of their product, which allows it to achieve better temps, it's a better product because most people aren't going to order thermal paste separately or entirely clean off pre-applied thermal paste to apply whatever thermal paste they previously used. If I want to buy a cooler, I want the performance it comes out of the box with, not the performance when you pair it with an arbitrary thermal paste you've decided to use.
reply
Norman
Aha, now I know why I have not seen much in the way of cooler reviews, except for the CLC stuff. As usual from this video it sounds like you can bet the farm on your results. I a quick blurb on the hotplate I like that you mentioned die placement for the hotplate heater elements. Sure the heat spreader spreads heat but there is sure to be a heck of a gradient. I have often wondered about this with hotplate reviews. Even heat is not real heat. I did a spot check: -) in my system (9900k 4. 8Ghz) with a NH-U12A verses a NH-U14S. Anandtech results showed an advantage (5C) for the 14 at high watts (like 200. Low watts were similar. They used a hotplate. I did not see a difference (180W (x264, Cinebench R20. Mentally allowed for at about 2C of fluff given my crap test. My diff was well below that.
reply
Aha, now I know why I have not seen much in the way of cooler reviews, except for the CLC stuff. As usual from this video it sounds like you can bet the farm on your results. I a quick blurb on the hotplate I like that you mentioned die placement for the hotplate heater elements. Sure the heat spreader spreads heat but there is sure to be a heck of a gradient. I have often wondered about this with hotplate reviews. Even heat is not real heat. I did a spot check: -) in my system (9900k 4. 8Ghz) with a NH-U12A verses a NH-U14S. Anandtech results showed an advantage (5C) for the 14 at high watts (like 200. Low watts were similar. They used a hotplate. I did not see a difference (180W (x264, Cinebench R20. Mentally allowed for at about 2C of fluff given my crap test. My diff was well below that.
reply
Helveteshit
Don't you have a set-up, where you heat up the metal according a kelvin grade/celsius/fahrenheit to a certain temperature, then slap on the cooler to see how much it decreases over time or similar? I feel like, the whole 130W x 2 + 50W x 1, doesn't look particularly accurate for testing different temperatures. Because metal and liquids perform differently well at transferring heat at different temperatures. This should apply to coolers as well. So, being able to verify how effective a cooling curve in a linear form should be ideal. For example, using PID regulator as reference, where you take into account the fact that thermal processes tend to have slower and harder curve to curb due the thermal reaction shift in materials.
reply
Don't you have a set-up, where you heat up the metal according a kelvin grade/celsius/fahrenheit to a certain temperature, then slap on the cooler to see how much it decreases over time or similar? I feel like, the whole 130W x 2 + 50W x 1, doesn't look particularly accurate for testing different temperatures. Because metal and liquids perform differently well at transferring heat at different temperatures. This should apply to coolers as well. So, being able to verify how effective a cooling curve in a linear form should be ideal. For example, using PID regulator as reference, where you take into account the fact that thermal processes tend to have slower and harder curve to curb due the thermal reaction shift in materials.
reply
MrGencyExit64
AMD CPUs do not have per-core thermal sensors that can be sampled by driver devs, just FYI. Intel CPUs run the gamut with some of their older hardware actually having more sophisticated per-core data in Model Specific Registers than their new hardware. But I whole heartedly agree, DO NOT use Max temperature (no matter what sensor it comes from, a high-pass filter should be employeed because the quality of the sample data coming directly from the CPU (both AMD and Intel) are such that they are not even accurate if you just use a single measurement. Any data sampled from the top/bottom 5% are not useful, the values returned by the HW sensors really are this inaccurate and those extremes are too full of invalid noise.
reply
AMD CPUs do not have per-core thermal sensors that can be sampled by driver devs, just FYI. Intel CPUs run the gamut with some of their older hardware actually having more sophisticated per-core data in Model Specific Registers than their new hardware. But I whole heartedly agree, DO NOT use Max temperature (no matter what sensor it comes from, a high-pass filter should be employeed because the quality of the sample data coming directly from the CPU (both AMD and Intel) are such that they are not even accurate if you just use a single measurement. Any data sampled from the top/bottom 5% are not useful, the values returned by the HW sensors really are this inaccurate and those extremes are too full of invalid noise.
reply
Derek
Thank you for all the work the team has done on this. I do not know what you used for correlating the data but sounds like spreadsheets. I work in large scale environments gathering system metrics. Might I suggest looking into graphite grafana (opensource / wonderful community. It is a wonderful tool-set that enables one to correlate most anything to anything. An example might be to relate slowness of an app, through vmware, through the network down to underline starred storage, mem. cpu, etc. While perhaps a bit over kill its is very easy to use and allows you to munge the data anyway you like it such as compering averages of max to averages of averages for the data set graphing them as you go. Worth a look.
reply
Thank you for all the work the team has done on this. I do not know what you used for correlating the data but sounds like spreadsheets. I work in large scale environments gathering system metrics. Might I suggest looking into graphite grafana (opensource / wonderful community. It is a wonderful tool-set that enables one to correlate most anything to anything. An example might be to relate slowness of an app, through vmware, through the network down to underline starred storage, mem. cpu, etc. While perhaps a bit over kill its is very easy to use and allows you to munge the data anyway you like it such as compering averages of max to averages of averages for the data set graphing them as you go. Worth a look.
reply
Jeffrey
Speaking of cooling can anyone provide some suggestions for open loop cooling? I have a 3900x, and rx 5700 xt to cool on an x570 aorus master in a thermaltake core p7. I can fit up to 3 radiators at 4x120 or 3x140 each and up to two pumps and reservoirs. I am looking for going a bit crazy with this with a focus on running many fans on many rads at low rpms. I have built open loop cooling before, back before CLCs existed. I have a few ideas on what to get but I have not seen much in the way of actual reviews for these parts. Also not necessarily looking at get it right away so probably after the human malware stuff blows over unless I see some good deals.
reply
Speaking of cooling can anyone provide some suggestions for open loop cooling? I have a 3900x, and rx 5700 xt to cool on an x570 aorus master in a thermaltake core p7. I can fit up to 3 radiators at 4x120 or 3x140 each and up to two pumps and reservoirs. I am looking for going a bit crazy with this with a focus on running many fans on many rads at low rpms. I have built open loop cooling before, back before CLCs existed. I have a few ideas on what to get but I have not seen much in the way of actual reviews for these parts. Also not necessarily looking at get it right away so probably after the human malware stuff blows over unless I see some good deals.
reply
Add a review, comment
Other channel videos
