TSV-Constrained MicroChannel Infrastructure Design for Cooling Stacked 3D-ICs Bing Shi and Ankur Srivastava, University of Maryland, College Park, MD, USA ISPD 2012 Outline • Introduction and Motivation • Thermal and Power model with microchannels • Formulation and Micro-channel design algorithms • Experimental results • Conclusions Introduction • Conventional air cooling might be not enough for stacked 3D-ICs. – Micro-channel based liquid cooling is developed. • Micro-channel heat sinks are embedded below each silicon layer and the coolant fluid is pumped through the micro-channels. Schematics Effectiveness Motivating example • Conventionally, straight channels are used. – But TSVs will block the route of straight channels. Introduction (cont.) • With bended structure, the micro-channels can reach those TSV-blocked hotspot regions which straight micro-channels cannot reach. • Compared to straight channel design, up to 87% pumping power could be saved. Thermal and Power model with micro-channels • Thermal modeling – Use RC network to represent. – Steady states: pure resistive network. – Solve GT=Q, where G is the thermal conductivity matrix and Q is the power profile. • G depends on many factors including the material properties, location of channels and TSVs, fluid flow rate etc. • Hotspot is the location that its temperature T is greater than maximum temperature constraint . Micro-channel power consumption • Pumping power = =1 ∆ – where N is the total number of channels, ∆ and are the pressure drop and fluid flow rate of the n-th micro-channel. • Laminar liquid flow – pressure drop in a straight micro-channel ∆ = 2 ℎ2 – L is the length of micro-channel, ℎ is hydraulic diameter, v is fluid velocity, μ is fluid viscosity and γ is determined by the micro-channel dimension. Fluid flow rate • Fluid flow rate = – , are the channel width and height. • Flow rate could be controlled by changing the pressure drop. • Usually fluid pumps are designed to work such that all the micro-channels experience the same pressure drop. – So that higher pressure drop results in higher flow rate and better cooling. Modeling Micro-channels with bends • Three types of region – Fully developed laminar flow region. – The bend corner. – The developing/turbulent region after the bend. Pressure drop • Pressure drop in fully developed region – ∆ = 2 ℎ2 • Pressure drop in developing region – ∆ = ∆ = 2 • Pressure drop in corner region 2 – ∆90° = ∆90° = 90° 2 • Total pressure drop – ∆ = ∆ + ∆ + ∆90° = – A quadratic function of v. 2 ℎ2 + + 90° 2 2 Total pumping power • Solve the equation for fluid velocity. • Estimate the fluid flow rate f, and thus estimate the thermal resistance and pumping power for this channel. • Hence, the pumping power as well as cooling effectiveness of micro-channels with bends is a function of – Number of bends. – Location of channels. – Pressure drop across the channel. • Slower velocity means lower cooling efficiency. – More pumping power is needed. Problem to be solved • To find micro-channel routes from one side to the other such that – The routes do not intersect. – Avoid TSVs. – Provide sufficient cooling at minimum pumping energy. Represent the routing problem • Each grid on the layout is a node. • Edge exists if – Two nodes are adjacent. – Non of them is a TSV. • Formulate the problem Minimize pumping power I/O nodes Routable nodes TSV constraints Temperature constraints Edge constraints The same edge The grid graph But… • This is a very complex problem since – The variables need to be discrete. – The thermal and pumping power models are highly nonlinear. • Propose a min-cost flow based method to do the job. Overall flow • The flow – Full scale thermal analysis. – Initial micro-channel design – Iterative refinement with thermal analysis Min-cost flow based micro-channel design • Initialization – I/O nodes are assigned a supply/demand of one flow unit. – All nodes in the grid graph have a capacity one. – The edges have unlimited capacity and are bidirectional. • Assigning the node capacity to be 1 would ensure that all the flow from inlet to outlet follows simple paths (non-intersecting and non-cyclic). Cooling demand • A silicon layer would be cooled by the micro-channels both above and below. – Unless the silicon layer is at the very top or very bottom of the stack. • For a location that need cooling. – is the heat load partitioning factor. – cooling demand assigned to the top. – (1 − ) cooling demand assigned to the bottom. Cooling demand (cont.) • The top(bottom)-most layer only cooled by its bottom(top) micro-channel. – is set to 0(1) accordingly. • Otherwise, is set according to the ratio of number of TSVs in the adjacent layer. – Less TSVs, more space for micro-channel. Cost assignment • Higher demand leads to lower cost since we would like micro-channels to pass through high cooling demand regions. • Let be the heat load partitioning factor of grid on silicon layer , 0 ≤ ≤ 1. Cost assignment (cont.) • If the hotspot exists in both side – (, ) = −[ 1 − +1 +1 + −1 −1 ] • If the hotspot only exists in one side – (, ) = − 1 − +1 +1 , +1 ≥ −−1 −1 , −1 ≥ • If the hotspot does not exist in both side – The node cost is assigned to a small positive value , = > 0 Micro-channel refinement • Two situation that degrade the cooling quality. – Some channels have several bends. – It may be routed over disproportionately large number of hotspots. • Iteratively refine the results How to get the minimum required pumping power • Linearly increase the pressure drop until the temperature met the goal. Iterative micro-channel optimization • The objective of minimum cost flow formulation did not capture cooling energy and/or number of bends in the channels. • Such imbalance (in cooling demand and bend count) leads to increase in the required pressure drop and thereby increasing the pumping energy. Iterative micro-channel optimization (cont.) • The basic idea is that all the channels should have similar levels of heat load, length and number of bends. • Based on these considerations, the initial design is refined by – Balancing the heat loads among microchannels. – Reducing unnecessary bends. Iterative micro-channel optimization (cont.) • Micro-channel heat load balancing: Iterative micro-channel optimization (cont.) • Bend Elimination – Identify all unnecessary bends and replace them with equivalent straight channels or patterns with lesser corners. – Removing corners in the hotspot region might lead to reduction in the cooling performance. – Only remove those corners in the nonhotspot regions which can easily be identified by the thermal analysis. Experimental setting • Two-tier stacked 3D-IC with 4-core CPU on each. – Different number of TSVs which are randomly distributed. • SPEC 2000 CPU benchmarks – Simulate 20 such benchmarks to get power profile and randomly choose 4 of these profiles to compose a one-tier profile. • Combine two of these power profiles to form a two-tier profile. Parameters • The area of each chip stack is 1.2 × 1.22 • The grid size is 200 × 2002 (so 60 × 60 grids in each layer). • The channel dimensions are = 100, = 400. • The maximum temperature constraint = 85℃. • The maximum available pressure drop is 500kPa. Experimental results (cont.) • Uses 20 micro-channels. Conclusions • Micro-channel cooling will be needed in the near future. • Proposes a flow which designs TSVconstrained micro-channel infrastructure. • Up to 87% pumping power saving compared with the micro-channel structure using straight channels.