Robert H. Dennard co-authored his now-famous paper for the IEEE Journal of Solid State Circuits way back in 1974. Essentially, Dennard and his engineering colleagues observed that as transistors are reduced in size, their power density stays constant. Meaning, power use stays in proportion with area, as both voltage and current scale (downward) with length.
Search Results for: dennard scaling
Looking beyond Dennard Scaling
Robert H. Dennard co-authored his now-famous paper for the IEEE Journal of Solid State Circuits way back in 1974. Essentially, Dennard and his engineering colleagues observed that as transistors are reduced in size, their power density stays constant. Meaning, power use stays in proportion with area, as both voltage and current scale (downward) with length.
There is a general industry consensus that the laws of Dennard scaling broke down somewhere between 2005-2007. Because threshold and operating voltage cannot be scaled any longer, it isn’t possible to maintain a constant power envelope from generation to generation – while simultaneously achieving the performance gains historically associated with reducing transistor size. Nevertheless, according to Yakun Sophia Shao and David Brooks, the semiconductor industry has already begun to adapt to the loss of Dennard Scaling as the end of Moore’s Law also looms large on the near-term horizon.
“This will likely lead to the additional consolidation in the semiconductor industry and fabrication companies will rely on ‘More-than-Moore’ to prove differentiation,” Shao and Brooks explained in their 2015 book titled Research Infrastructures for Hardware Accelerators. “Without either kind of scaling, there is also a risk of stagnation in the overall computing industry.”
However, Shao and Brooks emphasize that technology disruption often means new opportunities for innovation at the design and architecture level.
“Companies will increasingly differentiate their products based on vertically integrated solutions that leverage new applications mapped to innovative hardware architectures,” they stated. “In this context, application and domain specific hardware accelerators are one of the most promising solutions for improving computing performance and energy efficiency in a future with little benefit from device technology innovation.”
Steven Woo, VP of Systems and Solutions at Rambus, concurs with the assessment offered by Shao and Brooks.
“Although Moore’s Law has facilitated the creation of more transistors per chip for decades, clock speeds are plateauing due to power and thermal limitations. Similarly, improvements in Instructions Per Clock cycle have plateaued as well,” he explained.
“With the traditional paths for improving system performance no longer yielding gains at their historic rates, the industry must focus on rethinking system architectures to drive large improvements in performance and power efficiency.”
Further complicating matters, says Woo, is the fact that traditional performance and power efficiency bottlenecks in systems have shifted over the years due to the evolution of both architecture and applications. Put simply, the relentless progression of Moore’s Law and clock speed scaling prevalent throughout the 1990s and early 2000s so effectively improved computation capabilities that processing bottlenecks have moved to other areas.
“For example, the rise of the Internet of Things (IoT), Big Data analytics, in-memory computing and machine learning has resulted in ever-larger amounts of data being generated and analyzed,” he continued. “In many systems today, so much data is transferred across networks that data movement is itself becoming a critical performance bottleneck. Moreover, the very act of moving data is consuming a significant amount of power, so much so that it’s often more efficient to move the computation to the data instead.”
Consequently, there is an industry-wide effort to re-examine the architecture of conventional computing platforms by reducing and even eliminating certain modern bottlenecks.
“There are a number of recent developments in the industry that address modern HPC and data center bottlenecks such as Near Data Processing. These include the use of various accelerators including GPUs, FPGAs, and specialized processors,” Woo stated. “These industry efforts are focusing on both the hardware and the software infrastructure that ultimately will allow applications to achieve large gains in performance and power efficiency.”
Perhaps most important, says Woo, is to realize that traditional architectures may not be the best choice for certain data intensive workloads because they don’t address key power efficiency and data movement bottlenecks.
“Traditional processors coupled with acceleration hardware such as FPGAs, along with technologies to minimize data movement, offer new approaches to improving performance and power efficiency in modern systems. We believe FPGAs, alongside other acceleration silicon, will continue to play an important role in helping to evolve computing platforms by enabling flexible acceleration and near data processing,” he added.
Building a seismic supercomputer in the shadow of Dennard Scaling
Bert Beals of Cray Inc. recently told the Digital Energy Journal that the industry can no longer simply build an efficient supercomputer for seismic processing by simply adding more processors.
Indeed, because Dennard Scaling no longer applies, advanced microprocessors now require more power and additional cooling for heat dissipation. Moreover, even though engineers might fit more transistors on a microchip, clock rates are not expected to increase significantly and transistors may have to remain dark in order to deal with thermal limitations.
Building a seismic supercomputer in the shadow of Dennard Scaling
Bert Beals of Cray Inc. recently told the Digital Energy Journal that the industry can no longer simply build an efficient supercomputer for seismic processing by simply adding more processors.
Indeed, because Dennard Scaling no longer applies, advanced microprocessors now require more power and additional cooling for heat dissipation. Moreover, even though engineers might fit more transistors on a microchip, clock rates are not expected to increase significantly and transistors may have to remain dark in order to deal with thermal limitations.
According to the Digital Energy Journal, if systems designers want to increase processing power, they are likely to consider more physical computers, rather than creating more densely packed microchips to dissipate the heat generated over a large volume. This means increased demands on interconnects and network architectures to support efficient intercommunication between an ever growing number of compute nodes.
As such, says Beale, supercomputers must support a more parallel architecture. Such a paradigm requires a different kind of interconnect, memory hierarchy and input-output strategy instead of a serial optimization approach.
“You have to think about the overall systems architecture, combined with software architecture, combined with the people skills, necessary to deal with processing requirements at massive scale,” he explained. “We have to carefully design our system architectures to keep all the cores ‘fed’. It is very different from buying 1,000 machines on the internet and cabling them together yourself with Ethernet switches. An integrated supercomputing environment with appropriate software and expertise is a much wiser investment than just trying to buy the lowest dollars per flop machine you can buy.”
As Beale notes, seismic processing algorithms already exist that will demand supercomputers perform dramatically faster than current capabilities allow.
“We have requirements from the oil and gas industry which show a need in the next 3-5 years for machines that are 10x what we’re running on today. In the next 10-15 years, we’re going to need machine capabilities that are 100x what we’re running today,” he concluded.
Commenting on the above, Steven Woo, VP of Systems and Solutions at Rambus, told Rambus Press the industry has seen an “increasing emphasis” on rethinking system architectures with a range of newer technologies which can help improve computation and fuel future improvements in data centers and High Performance Computing (HPC) systems.
“As Beale points out, while there are more transistors per chip, clock speeds are plateauing due to power and thermal limitations. Improvements in Instructions Per Clock cycle have plateaued as well,” Woo explained. “With the traditional paths for improving system performance no longer yielding gains at their historic rates, the industry must focus on rethinking system architectures to drive large improvements in performance and power efficiency.”
Further complicating matters, says Woo, is the fact that traditional performance and power efficiency bottlenecks in systems have shifted over the years due to the evolution of both architecture and applications. To be sure, the relentless progression of Moore’s Law (which is now slowing) and clock speed scaling prevalent throughout the 1990s and early 2000s so effectively improved computation capabilities that processing bottlenecks have moved to other areas.
“For example, the rise of big data analytics, in-memory computing, and machine learning has resulted in ever-larger amounts of data being generated and analyzed,” he continued. “In many systems today, so much data is transferred across networks that data movement is itself becoming a critical performance bottleneck. Moreover, the very act of moving data is consuming a significant amount of power, so much so that it’s often more efficient to move the computation to the data instead.”
This is precisely why, says Woo, that there is currently an industry-wide effort to re-examine the architecture of conventional computing platforms by reducing and even eliminating some modern bottlenecks.
“There are a number of recent developments in the industry that address modern HPC and data center bottlenecks like Near Data Processing, the use of accelerators and the adoption of FPGAs. These industry efforts are focusing on both the hardware and the software infrastructure that ultimately will allow applications to achieve large gains in performance and power efficiency,” he added. “[For example], the CCIX consortium is slated to focus on the development of a Cache Coherent Interconnect for Accelerators, [while] the Coherent Accelerator Processor Interface (CAPI) will help enable further system improvements by allowing programmers to choose the most appropriate processors and accelerators to coherently share data. [These] are just two of the many examples of industry efforts to address these bottlenecks.”
Understanding Dennard scaling
In 1974, Robert H. Dennard co-authored a now-famous paper for the IEEE Journal of Solid State Circuits. Essentially, Dennard and his engineering colleagues observed that as transistors are reduced in size, their power density stays constant. Meaning, power use stays in proportion with area, as both voltage and current scale (downward) with length.
According to Prof. Dr.-Ing. Christian Märtin Hochschule of the Faculty of Computer Science at Augsburg University of Applied Sciences, power in CMOS chips can be modeled as dscaling. Q is the number of transistors, f the operating frequency of the chip, C the capacitance, V the operating voltage and I, the leakage current.
Understanding Dennard scaling
In 1974, Robert H. Dennard co-authored a now-famous paper for the IEEE Journal of Solid State Circuits. Essentially, Dennard and his engineering colleagues observed that as transistors are reduced in size, their power density stays constant. Meaning, power use stays in proportion with area, as both voltage and current scale (downward) with length.
According to Prof. Dr.-Ing. Christian Märtin Hochschule of the Faculty of Computer Science at Augsburg University of Applied Sciences, power in CMOS chips can be modeled as 
. Q is the number of transistors, f the operating frequency of the chip, C the capacitance, V the operating voltage and I, the leakage current.
“With Dennard’s scaling rules the total chip power for a given area size stayed the same from process generation to process generation. At the same time, with a scaling factor of S √2, feature size [shrank] at a rate of 1/S (the scaling ratio), transistor count doubled (Moore’s Law) and the frequency increased by 40% every two years,” Hochschule explained. “With feature sizes below 65nm, these rules could no longer be sustained, because of the exponential growth of the leakage current.”
Indeed, says Hochschule, post-Dennard scaling leads to a power increase of 
per generation for the same die area. At the same time, utilization of a chip’s computing resources decreases with a rate of
per generation.
“This means that a runtime large quantities of transistors on the chip have to be switched off completely, operated at lower frequencies, or organized in completely different or more energy efficient ways,” he continued. “For a given chip area energy efficiency can only be improved by 40% per generation. This dramatic effect, called dark silicon, already can be seen in current multicore process generations and will heavily affect future multicore and many-core processors.”
There is a general industry consensus that the laws of Dennard scaling broke down somewhere between 2005-2007. As Hochschule confirms, because threshold and operating voltage cannot be scaled any longer, it is no longer possible to keep the power envelope constant from generation to generation and simultaneously achieve potential performance improvements.
Nevertheless, Intel’s Mark Bohr remains bullish about the future of scaling in a post-Dennard world. In a 30 year retrospective on Dennard’s MOSFET scaling paper (published in 2014), he wrote:
“It is commonly recognized that following the simple scaling rules described by Dennard and his team back in 1974 is now no longer a sufficient strategy to meet future transistor density, performance and power requirements. But ours is a very inventive industry and new transistor technologies such as strained silicon, high- dielectrics, metal gates and multiple-gate devices have been or will be introduced to continue scaling. So although the letter of ‘Dennard’s Law’ can no longer be followed, it has gotten us very far over the past 30 years and the spirit is alive and well in transistor R&D facilities around the world.”
