Explaining Ryzen 3000 Cache, ft. Wendell of Level1 Techs LTX 2019
video description
Date: 2020-05-06
Related videos
Comments and reviews: 10
Victor
Both compilation and linking ARE single-threaded by design, because compilers do not parallelize when compiling a single source file. But big projects like Chromium contain hundreds to/of thousands of files, and dependency graph between files may allow independent compilation of some files simultaneously. The wider dependency graph is on average, the better utilization of large core count will be. My experience of compiling Tensorflow on TR 2950 says that compilation is almost 3x faster than on my 6 core notebook that is roughly 4x slower than TR. It means that TR's cores are not 100% utilized during compilation. For the same reason TR 2990WX may show much smaller than 2x increase in compilation performance of TensorFlow relative to 2950 just because its source's dependency graph is not wide enough for this processor. Linking is the same except that many projects usually provide a single executable artifact as a result of compilation (ex: chromium executable, so there is no parallelism at all here. Some projects may provide many executables (even tens of them, in this case we may have substantial gains from a multicore linking. My personal opinion is that in general, 16 physical cores _for a developer workstation_ is a sweet spot. 32 cores (64 threads) may be underutilized in general and 64 cores have no much meaning _for their expected price_. Moreover, C++ compilation may require 2GB+ ram for a single compiler process, it's 64GB RAM for TR 2950. 32-core developer workstation should have 128GB of RAM. Linking is also extreme memory-hungry, and for this reason I even had to limit compilation parallelism on some projects (LLVM) just because it exhausted all the RAM at linking.
reply
Both compilation and linking ARE single-threaded by design, because compilers do not parallelize when compiling a single source file. But big projects like Chromium contain hundreds to/of thousands of files, and dependency graph between files may allow independent compilation of some files simultaneously. The wider dependency graph is on average, the better utilization of large core count will be. My experience of compiling Tensorflow on TR 2950 says that compilation is almost 3x faster than on my 6 core notebook that is roughly 4x slower than TR. It means that TR's cores are not 100% utilized during compilation. For the same reason TR 2990WX may show much smaller than 2x increase in compilation performance of TensorFlow relative to 2950 just because its source's dependency graph is not wide enough for this processor. Linking is the same except that many projects usually provide a single executable artifact as a result of compilation (ex: chromium executable, so there is no parallelism at all here. Some projects may provide many executables (even tens of them, in this case we may have substantial gains from a multicore linking. My personal opinion is that in general, 16 physical cores _for a developer workstation_ is a sweet spot. 32 cores (64 threads) may be underutilized in general and 64 cores have no much meaning _for their expected price_. Moreover, C++ compilation may require 2GB+ ram for a single compiler process, it's 64GB RAM for TR 2950. 32-core developer workstation should have 128GB of RAM. Linking is also extreme memory-hungry, and for this reason I even had to limit compilation parallelism on some projects (LLVM) just because it exhausted all the RAM at linking.
reply
kit89
Linking can be multi-threaded but the effectiveness is dependant on how the code is written. Linking is determining if promises made by the developer can now be fulfilled. For example, the developer can claim to the compiler that a function called rand) exists and that it should be executed at a particular point in the code, the compiler will take it at face value and move on. Once everything has been compiled the linker then goes through and determines if all the developer's claims hold water. For the linker to succeed all of the code the developer uses needs to be compiled. The more promises a developer makes the longer it will take to link everything up. There are ways to speed up the linking process, instead of waiting for everything to be compiled and then linking, we can organise the order and kick off the linking process as soon as our compiled objects are available. This linking also happens between libraries, if you have lib2 and it depends on lib1 you can't link them until lib1 has been compiled. This can create a dependency chain which if the code is poorly organised can increase compile times drastically. Wendell touches on this, if you only modify code in lib2 then you don't need to recompile lib1, however modifying lib1 may require a recompile of lib2 as the location of functions that lib2 depends upon may have changed.
reply
Linking can be multi-threaded but the effectiveness is dependant on how the code is written. Linking is determining if promises made by the developer can now be fulfilled. For example, the developer can claim to the compiler that a function called rand) exists and that it should be executed at a particular point in the code, the compiler will take it at face value and move on. Once everything has been compiled the linker then goes through and determines if all the developer's claims hold water. For the linker to succeed all of the code the developer uses needs to be compiled. The more promises a developer makes the longer it will take to link everything up. There are ways to speed up the linking process, instead of waiting for everything to be compiled and then linking, we can organise the order and kick off the linking process as soon as our compiled objects are available. This linking also happens between libraries, if you have lib2 and it depends on lib1 you can't link them until lib1 has been compiled. This can create a dependency chain which if the code is poorly organised can increase compile times drastically. Wendell touches on this, if you only modify code in lib2 then you don't need to recompile lib1, however modifying lib1 may require a recompile of lib2 as the location of functions that lib2 depends upon may have changed.
reply
MethodMethod
Another thing to keep in mind for developers is our IDE's really like bursty performance. Which is to say, when you make a change in your code, the IDE will inspect the file you're changing for mistakes, as well as anything the code you changed touches. So if in your code you do something with a variable that logic elsewhere isn't expecting, the IDE can (depending) alert you to that. Those inspections are very bursty across all threads (at least, all 12 of my threads under Jetbrains IDE's. so presumably a cpu that can deal with that kinda bursty workload would be better. Another thing is installing libraries under node. npm install. this is primarily a single threaded operation that pins the thread to 100% for however long it takes to install the necessarily libraries. npm install is a difficult one to benchmark and not necessarily useful information to have anyway, since you're only gonna do it once in full for your project, and only once any time a dependency changes, which takes far less time.
reply
Another thing to keep in mind for developers is our IDE's really like bursty performance. Which is to say, when you make a change in your code, the IDE will inspect the file you're changing for mistakes, as well as anything the code you changed touches. So if in your code you do something with a variable that logic elsewhere isn't expecting, the IDE can (depending) alert you to that. Those inspections are very bursty across all threads (at least, all 12 of my threads under Jetbrains IDE's. so presumably a cpu that can deal with that kinda bursty workload would be better. Another thing is installing libraries under node. npm install. this is primarily a single threaded operation that pins the thread to 100% for however long it takes to install the necessarily libraries. npm install is a difficult one to benchmark and not necessarily useful information to have anyway, since you're only gonna do it once in full for your project, and only once any time a dependency changes, which takes far less time.
reply
Anon
Also a usual developer workflow is not just compiling times. When I'm developing a complete web project I have an IDE (Visual Studio, visual studio code, webstorm, netbeans, a front end server (say a react. js project based on express. js, a back end server (ngnix, c# microservices) and even a database server (mysql, oracle, sql server) and that's the bare minimum, not even counting things like a web browser with github, stackoverflow and the official documentation of the API or languages I'm using in that moment, or running automated tests, which in the case of the projects that I work on take WAAAY more time than compiling
reply
Also a usual developer workflow is not just compiling times. When I'm developing a complete web project I have an IDE (Visual Studio, visual studio code, webstorm, netbeans, a front end server (say a react. js project based on express. js, a back end server (ngnix, c# microservices) and even a database server (mysql, oracle, sql server) and that's the bare minimum, not even counting things like a web browser with github, stackoverflow and the official documentation of the API or languages I'm using in that moment, or running automated tests, which in the case of the projects that I work on take WAAAY more time than compiling
reply
Aquila
The space game X4 Foundations could make an interesting benchmark. The game runs okay but when you build a mega space factory thing and have lots of ships working for it, frame rates in that sector will plummet. Check resource monitor and the GPU will be only 30% and the CPU at 10% (in this case a 2700X and a RX590. The game has lots going on but does not seem to utilize the CPU resources available to it, even though it is multi-thread aware. It ends up with calcs queued and lags waiting for them to get done. It is almost like a server CPU would handle this game better perhaps.
reply
The space game X4 Foundations could make an interesting benchmark. The game runs okay but when you build a mega space factory thing and have lots of ships working for it, frame rates in that sector will plummet. Check resource monitor and the GPU will be only 30% and the CPU at 10% (in this case a 2700X and a RX590. The game has lots going on but does not seem to utilize the CPU resources available to it, even though it is multi-thread aware. It ends up with calcs queued and lags waiting for them to get done. It is almost like a server CPU would handle this game better perhaps.
reply
Tomas
There are indeed two groups of compiling workloads. You can be working on a project and compile only the bits that you changed. In another case you downloaded a software and want to use it, therefore it has to be compiled from scratch. Often when you change a compile-time option, the whole project has to be rebuilt, and this can happen multiple times in a row while trying to get the software to work. Some less popular packages only come in a source for, and for each update they need to be rebuilt, because you do not keep the temporary files.
reply
There are indeed two groups of compiling workloads. You can be working on a project and compile only the bits that you changed. In another case you downloaded a software and want to use it, therefore it has to be compiled from scratch. Often when you change a compile-time option, the whole project has to be rebuilt, and this can happen multiple times in a row while trying to get the software to work. Some less popular packages only come in a source for, and for each update they need to be rebuilt, because you do not keep the temporary files.
reply
Lori
A test suite that would be useful for me to see would be something like: (Entirely from Linux, not Windows: D. There will be performances differences) 1) Running the test suite of a medium to large Ruby application. (Could be the test suite of Rails) 2) A full recompile of a moderate C++ project 3) An incremental compile of the same C++ project (pick a couple files and change them) 4) A JavaScript transpile. For example, a project written in modern JavaScript transpiled with Babel to ES5. Something like this would be amazing.
reply
A test suite that would be useful for me to see would be something like: (Entirely from Linux, not Windows: D. There will be performances differences) 1) Running the test suite of a medium to large Ruby application. (Could be the test suite of Rails) 2) A full recompile of a moderate C++ project 3) An incremental compile of the same C++ project (pick a couple files and change them) 4) A JavaScript transpile. For example, a project written in modern JavaScript transpiled with Babel to ES5. Something like this would be amazing.
reply
Tomas
12: 30 The linker invocation is typically the bottleneck when recompiling after a small change. Because first your modified module has to compile and only then the linker runs. It's the worst case scenario for scaling. Depending on the size of the project, link times can be quite high. There are ways to speed it up, like incremental linking or multithreaded linker from llvm project. The compiler and linker are single-threaded (generally. The way to utilize the other CPU threads is to compile multiple modules at once.
reply
12: 30 The linker invocation is typically the bottleneck when recompiling after a small change. Because first your modified module has to compile and only then the linker runs. It's the worst case scenario for scaling. Depending on the size of the project, link times can be quite high. There are ways to speed it up, like incremental linking or multithreaded linker from llvm project. The compiler and linker are single-threaded (generally. The way to utilize the other CPU threads is to compile multiple modules at once.
reply
alkafrazin
question: why hasn't AMD attached a small HBM2 secondary victim cache? Like a l4 cache of HBM2, still faster than RAM, still lower latency, but a lot of it, connected via the io die? it could really improve graphics performance and might even help CPU perf if you had, say, a gig of fast hbm2 cache for the gpu and cpu to share, and the gpu takes the biggest share, but the cpu can, say, use up to 256mb of it if the gpu doesn't explicitly need it for render targets or other high bandwidth applications?
reply
question: why hasn't AMD attached a small HBM2 secondary victim cache? Like a l4 cache of HBM2, still faster than RAM, still lower latency, but a lot of it, connected via the io die? it could really improve graphics performance and might even help CPU perf if you had, say, a gig of fast hbm2 cache for the gpu and cpu to share, and the gpu takes the biggest share, but the cpu can, say, use up to 256mb of it if the gpu doesn't explicitly need it for render targets or other high bandwidth applications?
reply
brechtve
Thanks both to Wendell and GN! I've been searching for this all week. It's so difficult to find a good explanation about development topics and the rare once you do find, say they prefer intel or amd without giving a solid explanation why. This talk really pointed out why it is not easy to make such a statement to begin with. But i have massive respect to both of you trying to figure out a way to answer the almost impossible answer. sub to GN and Level1Techs keep up the great work guys!
reply
Thanks both to Wendell and GN! I've been searching for this all week. It's so difficult to find a good explanation about development topics and the rare once you do find, say they prefer intel or amd without giving a solid explanation why. This talk really pointed out why it is not easy to make such a statement to begin with. But i have massive respect to both of you trying to figure out a way to answer the almost impossible answer. sub to GN and Level1Techs keep up the great work guys!
reply
Add a review, comment
Other channel videos
