EZ-PASS: An Energy & Performance-Efficient Power-gating Router Architecture for Scalable NoCs

Researchers at The George Washington University are developing an innovative method to design a performance and power-efficient network-on-chip for the parallel computing chips. The novel network-on-chip design invented at GW will significantly improve performance and power efficiency of parallel computing chips. Considering that the innovation can be applied to any multicore systems and the high demand for multicore parallel computing, we believe that in a couple of years this technology can be worth tens of billions of dollars.

The market for parallel computing is growing rapidly, and the global parallel computing market had been valued at USD 31.20 billion in the year 2017, which is expected to reach USD 50.50 billion by 2023 growing at 8.35% CAGR. Parallel computing works on a network or a series of processing units that are itself bound by hundreds and thousands of computational cores. These cores are capable of executing a number of high-performance software and programs. The first consumer multicore processors hit the market in 2005, and supercomputers were using multicore processors as early as 2001. As technology scales, the number of cores on a single chip is increasing: GPUs and TPUs now can have multiple thousands of individual cores, while a supercomputer can carry up to over 100,000 cores. While the parallel computing market is pursuing higher performance of multicore systems, much of the perceived speed increases for those systems in the last 5–10 years has to do with improvements related to coordinating multiple cores on a CPU, not with individual processors becoming faster. For example, in fact, the 4th generation (2014) Intel core i7 has the same base processor frequency (3.60 GHz)as the 9th generation (2018) Intel core i7 — but the new version has twice as many cores, 8 instead of 4. Due to that fact, the communication among the cores is becoming the key to achieve higher performance for multicore systems: multiprocessing, parallel processing, and data parallelism all rely on network-on-chips (NoCs) for communication.

Network-on-chips have emerged as the standard interconnect fabric solutions for connecting multiple cores, caches, memory controllers, and other hardware components on the many-core chips. As the technology scales, the static power is becoming the dominant factor in power consumption of on-chip interconnect.  There has been a significant amount of works recently devoted to solving this problem. However, current works cannot simultaneously reduce the power and improve the performance in a scalable way. This invention can significantly reduce the static power, improve the system performance, and easily applied to a number of parallel computing chips.

Prof. Louri has invented a design of tackling the power issue of on-chip interconnects. This invention can provide a performance and power-efficient way to significantly reduce the static power of the on-chip interconnects. The proposed bypass switch routes the intermittent communication traffic when the conventional router is powered-off. A novel control flow scheme records the communication information between routers and bypass switch. The conventional router l and bypass switch are controlled by a power-gating controller. This technology is a powerful innovation in parallel computing chips, ranging from mobile phone processors, chip multiprocessors (CMPs) to supercomputers and exascale systems, with an enormous commercial potential in a growing number of environments.

Applications:

• The performance and power-efficient network-on-chip is of paramount importance to the parallel computing chips, which can have a significant impact on the overall chip performance. This invention can be of interest to many chip design companies, including IBM, Hewlett Packard Enterprise, Intel, Qualcomm, Microsoft Corporation, Facebook, Google, ARM, Huawei, Cisco Systems, Advanced Micro Devices, Micron Technology, Fujitsu, Samsung Electronics, Oracle, Dell,  Hitachi (Hitachi Data Systems, Inc.), etc.
• This invention can be easily applied to any parallel computing chips that used in mobile computing, IoT, multicore processors, embedded systems, servers, cloud computing, data centers, machine learning accelerators, etc. 

Advantages:

• This invention can reduce up to 90% static power consumption of on-chip interconnect with nominal performance loss.
• The superior scalability of the invention can be applied to any current and future parallel computing chips that used in mobile computing, IoT, multicore processors, embedded systems, servers, cloud computing, data centers, machine learning, etc.