Power as the Next Physics Wall for AI

0:00 Every part of this power supply chain has a different player, has a different technology. It is not like what you you can see in HBM. Oh, it's HBM. What is it like the three same three companies? What are they doing? Stacking the same way, right? It's a kind of a closed problem in a sense. >> Power is a wide wide open problem. >> Welcome to another semi-doped podcast. I'm Austin Lines of Chip Strat and with me is Vic Shaker from Vicks Newsletters.

0:28 What's going on, Vic? It's been a crazy earnings week this week. >> Yeah, I can't listen to all the earnings calls. I just try to listen to a few of them and try to like run uh the transcripts through LLMs and tell me what happened, but other than that, it's too much to keep up. It's a fire hose of earnings calls the season. I I can't keep up. >> Yes, I know. Like in a dream world, they would just all talk to each other and schedule like one per week for the whole season. [laughter] But no, instead it's like everyone in the span of a couple days.

1:01 >> It's not intended that people listen to all of them. I'm guessing that people have investments in company A, B, C, D, and they just listen to those and you don't listen to everything. Uh because you know even big investment firms have people who address certain sectors of the semiarkets. people like us try to cover optics, data centers like uh memory, CPUs, power, I don't know, power. Speaking of power, that's what we're going to talk about today.

1:30 >> Power, we're here to talk about power. So, before we dive into power, um should we address, we had lots of YouTube commenters, so thank you YouTube commenters. Should we address any of the questions? >> Yeah, I think we should uh at least address one or two of them. Uh one was on the memory tax episode that we did. Um the comment was basically that when we said that um memory costs going up and all the capex being directed into memory means that that money is not available for compute. Uh I think the overall sentiment around that was like yeah the money that's going into memory is more like inflation and it doesn't usefully contribute to compute or solving the memory bandwidth or making data move faster between racks. It doesn't solve any technical problem in any way other than that you're just paying memory companies more. So I think that was mostly the sentiment.

2:34 >> Yes. Totally. So the the comment was something like um you know that we're misunderstanding and and compute's not the bottleneck. Memory is a bottleneck and therefore you should spend more money on on memory. But yeah, to your point, what we were saying is like no no no, no one's we're not saying don't spend more money on memory. We're just saying that like you bought a gallon of milk for $2 today and tomorrow it cost $2.50 for the same gallon of milk. So you're just spending more to get the same stuff, you know? So yeah, it's like the cost. Yeah. Inflation, right? So, it's like >> it's not like you chocolate milk.

3:08 Chocolate milk would be nice, but you don't. You just get regular plain old milk, but you pay chocolate milk prices. >> Exactly. >> Uh that's not helping anybody. It doesn't increase our quality of life. >> Right. Right. So, so the purchasing power of a dollar is going down for compute. So a bunch of CFOs are are are going to say like hey we're spending more we're for equivalent compute like you know does this meet our internal ROIC targets what's the projections when will this stop so on and so forth um but okay so we addressed that um I think we had a lot of people asking about boardfly did you want to talk about it at all and how how to get to seven hops >> yeah so the best way to go about this even rather than me going through and explaining how this works is that Google has published a deep dive on the TPU8/D blog. You know, it's a deep dive blog on their website. So, if you go there and look at the picture there in the blog, it explains exactly how the you know 3D Taurus approach has 16 hops and how the boardfly approach has actually seven hops. So the idea is that you have to get from the board into the group where you have you know so many boards in a rack kind of thing and then you go between kind of you know rack to rack uh through the OCS switch and then you know finally you follow the same process on the other side. You reach the other rack and reach the other board. The Google picture actually shows seven hops. So you can see how each hop has uh a different location it goes through. So the fact is that you can reduce network diameter by using their boardfly topology means that the latency equivalently comes down compared to using a 3D Taurus. It's uh the latency is more than twice better.

5:03 >> Nice. Okay. So folks, if that's not enough, go read the original source for yourself and you should be good to go. >> Yeah, it's a good explanation. They do a good job. Google blogs are nice. >> Yes, totally. They spend a lot of time and effort. So, thank you Google, whoever wrote that out there. We appreciate you. So, okay, let me hit really quick last thing. Um, I had a couple quotes from the coherent earnings call that I thought were interesting.

5:24 And we've talked about these topics before, but I'm going to hit on it again just because it's a continued theme that, you know, people should be tracking, which is ultimately there's massive demand for indium phosphidebased lasers and this everyone is scrambling to create more supply and for coherent. The quote was from the CEO James Anderson, "We are aggressively ramping 6-in capacity because 6-in wafer compared to a 3-inch wafer will produce more than four times as many chips at less than half the cost. So, this is how Coherent is trying to rapidly increase their supply is moving to 6-in wafers."

6:03 And so, again, I just wanted to unpack it in case people hadn't heard that before. This whole like if you're like, "How do you get to 4x more chips at half the cost?" And we've talked about this a little bit, but ultimately it's about the area of a 6-in wafer compared to the area of a 3-in wafer. And you know, if you it's radius squared. And so if 6^2 is 36, 32 is 9, 36 by 9 is 4. So that's how you get to four times more. Um, and then of course the cost thing is interesting and I don't think we've talked about that quite as much, but at the end of the day, if you're if you have the same number of steps and you're just processing a bigger wafer, your costs are fairly equivalent. You have the same number of steps, you might have a you have to pay more for a bigger substrate and you have to pay more a little bit higher input cost because maybe more photoresist and chemicals and gases and things. But let's say the cost is maybe 2x higher. You could still see a world where to to process a six-inch wafer, you can still see a world where you get four times as many dye per wafer and because it's bigger and yet it only costs you twice as much to process it.

7:06 So your cost efficiency is twice as good and therefore it costs you half as much per die. So I just wanted to remind people really quick that that's coherence play. Now the question of course that that everyone's asking is okay they've got six inch wafers but is it yielding and are what are they actually making and so James Anderson did uh address that as well. He said um given the healthy yields we are seeing with 6-in production we began production of 6-in indium phospite at a second site in Yaria Sweden. I'm not quite sure how to pronounce it. Yarfala Sweden. and ramping at two sites in parallel will significantly accelerate our production capacity ramp. Additionally, we are in production on three different types of key transceiver components on 6-in indium phosphide, EMLs, C uh continuous wave lasers, and photo dodes. And somewhere else, they he said I think it was in a Q&A, he specifically said that their 6-in yields were as good or better as their 3-in yields. and pointing out like that's comparing to the mature 3-in yield, not just when 3-in was ramping.

8:16 And so I think he, you know, without saying much about yields and performance, uh, coherent was definitely trying to signal like, hey, we're not just building the simplest component. We're photo dodes. We're building photo diodes, CW lasers, EML, and they're pretty confident in their yields. Um, so I I would just say it's something to track. coherent is telling the story that that they're progressing nicely. I think maybe the final read through will be on their margins because of course if they are um increasing supply decreasing the cost and yielding yielding well which would impact your cost and those devices they're making on the six wafers have good performance which means you can maintain ASPs or increase ASPs then ultimately this should flow through to their margins. So that's probably just the final takeaway is if you're trying to track it and really trying to figure out is this just a story or are customers actually buying devices that are built on this new capacity.

9:19 Ultimately it's about yields in ASPs and we should be able to track that. So one small caveat to that is that when talking about yields of CW lasers or you know electroabsorption modulated lasers EMLs which are the workhorse for 200 gig transceivers uh and CW lasers are the workhorse of co-ackaged optics. These are the two hotly contested areas really right now between coherent and lumenum and all these companies. It is very important to distinguish that CW laser but at what power are you talking about 50 mw 100 m 400 m that is significant differences between what yield means on what product similarly EMLs are the same way uh EMLs have like two components to it has the laser and the the absorption modulator and they are usually kind of co-designed to work together and That is a significantly more complicated problem to solve compared to, you know, maybe CW lasers where you just have to like put out laser, right? You're not modulating anything. It's just like a flashlight that's on. You're not turning on and on the flashlight depending on a zero or one. So CW lasers are structurally functionally I wouldn't say structurally, functionally easier. So it really depends on what yields, which product line, what power, what output levels. So it's very easy in an earnings call to put a blanket saying, "Oh yeah, everything is great." But if everything is great on a 50 m laser, uh that's not what we're talking about, you know? So it really depends on what exactly is yield meaning here. So that's the >> that's [snorts] the wet blanket I want to put on your otherwise optimistic statement.

11:12 >> No. Yeah. No, totally. I definitely don't disagree with that. There's this the highle directional guidance they're giving and then to you there's the nuance and they probably won't share that level of nuance on a call just like Intel's not going to get super nuanced into like the yields on 18A and 14A and give engineering specs. Yeah. Um yeah, but but then that's the game is how do you sort of reverse engineer or back out what you can to get a good sense of like oh they are actually just insinuating 50 matt lasers continuously they're not 400 millatt and maybe again that shows up in >> revenue like topline revenue or maybe it shows up in in margins or something where it's like oh c their margins aren't great because it's customers buying the the cheap product versus like high performance powerful product.

12:08 >> If they are selling high ASP products at high yield, it will show up in the money. Follow the money. >> Exactly. Exactly. Totally. So, okay, cool. Uh, all right. Let's jump into power now. So, you wrote an article on power and this is a perfect forum for us to unpack it. So, >> yeah. >> Yeah. Set the stage for us. >> So, there is this growing sentiment. It's not just from me. Um few people in the industry have pointed it out as well uh that we are coming to a point maybe not in this generation maybe it's the next generation of accelerators and racks probably Ruben ultra because the kyber rack uh is a 600 kW per rack uh power consumption that is much higher than what usually data centers are used to. So in the cloud era each rack used to consume like 20 kow you know now AI accelerator racks consume anywhere between 100 and 120 kow now we are talking about the kyber era of racks where the Ruben ultra will go in uh at 600 kowatt a rack and the future will hold one megawatt a rack that's an enormous amount of uh you know that's a 100 times power per rack compared to what was in the data center you know cloud era pre AAI era which was like approximately 10 to 15 kilowatt you're talking about one megawatt per rack and imagine how many will go into a data center and imagine you know how many of those data centers are going to be connected together via scale across you know all of that stuff so this is a looming problem a bigger and bigger problem with every generation of you know AI that comes out and chips that come out so I wanted to address it in my Substack article to just point out that power is the next physics wall you know and this is a a theme I want to pursue in a little bit more depth uh over time because as you can imagine it's an enormous problem right you know you got power generation all the way at like you know whether you're talking turbines or nuclear power how that's transmitted and how it reaches the substation you know how which is converted into the data center and how it is ultimately delivered to the GPUs. We'll cover some of that in detail in this podcast. At least I'll explain how this conversion happens and where you know the markets directions are going to go you know going forward.

14:45 >> So but you can imagine like this entire chain has so many players and it has so much going on there and all of it is important to deliver the next generation of power. So that's what I wanted to kind of touch upon in this abstract post and kind of we'll briefly cover as the next physics wall. >> Nice. Nice. Okay, let's get into it. So what is the problem with a 1 megawatt rack? Why is that a problem?

15:08 >> Yeah. So in this one, I want to kind of start at the rack level. What is the underlying physics problem? And then we will kind of zoom back a little bit and understand how power is actually generated up to that one and delivered up to that one megawatt rack because it's important to understand how the power gets there right and then we can talk about what happens within the rack which is essentially what I focus on the substack uh article. So the thing is very simple you know uh the idea is why are we talking about optics now okay I promise this is not a tangent okay this has something to do with power why are we talking about optics the whole problem was with copper right and reach so as the speeds got higher and we needed to connect racks over longer distances optics was the only way forward because copper reached its physics limits and the physics limits in copper is As you increase the speed, what is flowing within the copper cable is actually an AC signal. It's like it's a varying signal, right? Cuz you're transmitting, let's say, bits that go up and down like whatever. So, it's like a varying signal. And what happens in copper interconnect when you have varying signals is that all the current doesn't flow through the whole copper wire. They tend to concentrate on the periphery.

16:29 Only the outermost ring of copper actually holds any signal. there's nothing happening with the rest of the copper cable. This is because of the phenomenon called skin effect, right? It goes to the skin of the copper cable, not really the whole cross-section of the copper cable. So, the resistance goes up because you have all this cross-section, but if you're not using it and you're you're now have more resistance. >> So, resistance was the fundamental bottleneck for why we needed to go from copper to optics. And now you see everybody's in the optics like Indian fight shortages. The rest is history, right? Look at the optics market we are in. So this was a bottleneck that was physics driven and whenever something is physics driven it's easy to identify. So go ahead.

17:14 >> Yeah totally totally no that's that is what we try to do here is look for the fundamental physics constraints and then ask what is going to happen beyond this. All right. So, really quick, um, for maybe less technical listeners, um, when Vic said AC, he meant alternating current. That's the varying current. And then this skin effect thing, I mean, if you think of like I like to think of electrons throwing through flowing through a wire as sort of like a pipe and or even you could think of it like as a subway tunnel with like lots of people trying to push through it. And if this skin effect means you can only walk like at the edges of the subway tunnel, then you try to take the same people and push them through the edges of the subway tunnel, there's going to be more resistance. You're going to bump into each other more, right, than if you could just everyone could just walk nicely with lots of space around them.

18:00 Um, so when Vic says there's an increased resistance and and the skin effect becomes worse and worse at higher and higher speeds. So when we're trying to communicate more and more data at higher and higher speeds, then you get more of this skin effect resistance bumping in together. So there's just a little analogy for nontechnical folks. Uh Vic, now please carry on. >> Yeah. No, thanks. That's good. I like the subway analogy. I like analogies. >> Yeah, totally.

18:24 >> Great. Thanks. >> So power is the same thing, right? Now when you have a lot of power say 600 kowatt or something uh ultimately you have to make two decisions and actually it's one decision and the other one follows from it. What voltage are you going to operate in right cuz power is basically voltage times current. So if you are operating at a high voltage you have lower current. If you're operating at a low voltage you have a higher current for the same power. And that decision is very important to make. Now typically in racks earlier in the cloud era we didn't really need to go to very high voltages because there's no need to the currents are manageable because the power is manageable. So why do you want to go to high voltages because you have to use special transistors to actually handle such voltages. Not everything can handle it like that. So don't no need to go exotic if you don't need to. So typically what has happened is the voltage choice in racks uh from the you know it's not that that long ago you know but so meta really standardized on the 48vt architecture so the 48vt DC architecture and so the current was okay you know the when the power was low the current was manageable so now what happens when you go to some massive amounts of power is that now if you're at 48 volts DC you know you are going to go uh to a lot of lot of current like take this for example so 600 kilowatt of power and 48 volts of rack voltage you are burning 12,500 amps of current through the rack think about that that's enormous and what happens is that wherever whichever method you use to transmit data it can be copper wires buses connectors whatever Everything has a resistance and so even the tiniest resistance at you know 12,000 amps of current means that you're going to dissipate a lot of power. So the power dissipated through a resistor is like the squared of current times the resistance.

20:36 >> So now not only do you have a high current, now you're going to square it. [laughter] >> Yeah. Crazy. Crazy. So yeah. So you're saying power equals current times voltage. And ideally, we would just have like low voltage and not not much current, not a big deal, low power. But we're in an era where [snorts] we have already have a fixed >> voltage. It's what what did you say? 48 volts coming in. >> Yeah. >> Yep. And so, but we want to have much much higher power at the rack because we just want to have way much denser rack with way more GPUs and they're all power hungry. And so in aggregate they want to consume a ton of power. And so you're saying the only way today if we stay with the 48 volts to have all these GPUs that are power hungry to power them is to increase the power. And if P equals IV and the voltage is fixed, the only thing we can do is increase the current to something crazy like 12,500 amps or 12.5 kilo. Like I it's hard for me to even comprehend that. That's like a when we were taking EE courses, we were never using like currents this high when we were doing our little by hand toy problems.

21:45 >> It's always milliamps, right? Like all of our circuits are milliamps and now you have kilos. Power electronics is a beast. Okay. So it's a it's fine, but it's still a lot of current. >> Yes. Yes. But then what Vic also said was um Okay. Well, resistance is uh what? I squared times. What did you say the resistance is? Yeah, the power dissipated. You know, the amount of power you lose through some form of resistance, whether it's the connector or just the metal itself, there's always some resistance. And you lose power via heat through that resistance. You generate heat when you push current through a resistor. And that power dissipated is the square of the current. So, not only are you in like 12 kilo of current, you're now squaring it. And then the resistance is like how low can you make it like you know I have some example calculations on the substack we don't have to go through all of it now but yeah it's it's insane like you will have a lot of uh power dissipated.

22:45 >> Nice. Nice. Yes. So the problem is we've got a a ton of power. We've got a really high current. We're going to dissipate a lot of heat because it's the square of that current. So just lots of people bumping into each other in the analogy and giving off friction, giving off heat. So how do we solve this? >> So the one way to fix this is go to a higher voltage. So you know for 600 kilowatt you know don't use 48 volts use 800 volts and it is coincidentally the voltage that is used in uh you know EVs traction inverters and so that entire automotive industry has kind of matured this uh silicon transistor technology called IGBTs or more recently the silicon carbide there's gallium nitride all these exotic wide band gap semiconductors as they are These uh are specialist transistors that can handle 1,000 volts of uh voltage and they are well suited for such applications. So might as well reuse that EV industry and the transistors around them. So you know go to 800 volts. So when you go to 800 volts and you do the same math, the current drops from 12,500 amps to 750 amps. Much better, right? 750 amps. It's not you don't even have to use the kilo kilo amp unit.

24:09 >> Yes. Yes. Totally. Okay. So, you're saying if we had um P equals IV was our problem where the V was fixed. So, we had to increase the I a lot if we wanted to increase the P the power. But you're saying, wait, wait, wait a minute. What if we don't fix the V? What if we actually increase the V to get and hold the P constant? Then the I can go down, right? So, you're saying instead of 48 volts, we could increase it to 800. And then that that way we could still get the same really high power at a lot lower current. Uh, and then of course the question is, well, which V should we increase it to? And and I heard you say, oh, well, everyone looked around and said, hey, why not 800 volts? Because that ecosystem already exists. That power electronics ecosystem already exists for EVs. So, that seems like a great place to bring it in.

24:59 >> Yeah. >> Into the data >> because those are ruggedized components, right? like transistors that go in cars if it's like driving your you know wheels uh that is basically driven by transistors too by the way the battery power is converted into alternating current that drives the motor that drives the wheels those things are rugged they have to operate in all conditions they have they have high thermal tolerances they have quality standards you know there's a lot of things in place so why not why not reuse that stuff >> nice and also gives an opportunity for EV companies to pivot into data centers because everybody wants the data center angle.

25:34 >> Totally. Totally. Yeah. No, I think it's super fascinating to be like, oh, there's already power electronics here and they're already ruggedized to be in, uh, harsh environments from automobiles, uh, you know, to be hot or cold or whatever. And so, like, oh, guess what? Data centers are crazy hot. No big deal. You know, uh, and then, yes, of course, naturally, anyone when their investor brain is listening, they're going, oh, wait, wow, this is going to be really interesting. I should look into anyone who's already doing power electronics for 800 volts or auto EVs or whatever because now they're trying to move into the data center. That's interesting.

26:06 >> Yeah. So, it's a it's a nice solution to the problem and Nvidia is looking into it. I mean, this is not news to people who are following the power side of things in data center world. This is we're talking about the basics, but that's good. You know, we've always got to set the baseline of understanding so that we can follow what happens in the industry closely later. That's that's where the foundational understanding comes. So it's good to have this. So it's a good discussion we're having. So the point is that think about the power dissipated through heat. Now the resistance let's say is the same. Okay for argument sake your your current is so much lower.

26:40 >> Now the squared of the current is also significantly lower. Right? >> Uh if you can drop current by you know two orders of magnitude how much should be at least we dropped it by one order of magnitude right? Like 750 to 7,500 whatever. Yeah. >> Yeah. 12.5K down to 0.75K. >> So yeah, >> it's like 10 15 times lower current then your uh power dissipated is square of that. It's like uh 100 to 200 times lower than that. So you see the fundamental problem is that increasing the voltages reduces the current. And now this is the only viable way to overcome the same problem of resistance that was plaguing the copper interconnect world that is due to plague the power world as well cuz you cannot push that much current through any form of resistance. And this is not like the skin effect kind of resistance we're talking about. This is like standard ohms law resistance. This is DC resistance we're talking about here.

27:40 Okay. So basically this is the same limiting factor that drove everybody from interconnect world uh into optics copper interconnects into optics right this is the same limitation that is resistance that will drive people from uh low voltage systems to high voltage systems and when it go to high voltage systems it creates an entirely new socket because this 800 volts to 48 volts conversion which is one way to do it you you don't have to bypass 48 volts you can convert from 800 volts to to a certain voltage. What that voltage is is is a question that requires some engineering discussion. It's all on the substack. But that conversion is a new socket basically that does not exist in the data center world and that is what people are trying to compete for and land it properly, right? And think about the whole CPO argument again. Uh you wanted to do the optical to electrical conversion as close to the chip as possible, right? because you don't want to be in the copper land at all because it's a problem is resistance. The same problem exists in power. You don't want to uh convert to power like you know to low voltages far away from the chip. Uh at if at all possible you want to have the highest voltage possible right up to the edge of the chip. Make the conversion at what is called the point of load.

29:01 >> Right? So it is like like think about it as a CPO of the power world. So now you have what is called vertical power delivery. You put the chip under the GPU and you deliver power at the GPU. Convert as much as possible. Not that I'm saying you're going to convert 800 volts to one volt at that chip. That's like that's like too much. There are still many conversion stages that need to happen before that happens, right?

29:25 But ultimately that is the CPO equivalent of power delivery. >> Yeah. No, that's good. I really like the analogy of looking at CPO to say basically you want to keep it in light as far as you can as close in as possible before you convert it to the electrical domain. Otherwise, that whole problem of like with um linear pluggables was you've got that long copper trace and it's very noisy and lots of power loss and signal loss. So, no, no, no. Just bring in the signal as close to the chip, the ASIC, the GPU or whatever as possible. trying to do the same thing. High voltage, low current.

29:59 Bring that in as close as possible before you essentially like convert it down to step it down to lower voltages. And this actually reminds me analogies for people. This is like how the transmission lines work in your neighborhood, right? You've got like really high voltage lines that are sending power, you know, across for me in Iowa, like across corn fields to the next town. And then only once it gets closer does it get sort of stepped down and brought down. It's kind of kind of the same thing here. We're talking about high voltage so that to reduce the the current and reduce the power loss along the way.

30:34 >> Yeah. Yeah. It's very important the the power efficiency dictates how much compute because there there is a fixed budget per data center like you can't get more than so many gigawatts, right? And so you want to convert all of that into compute. You don't want to burn it in like poor conversion power conversion systems. you want to have all that power available to generate you know useful AI output. So that is the key thing here.

31:02 So that is basically the setup. Uh but I want to just zoom out for one quick minute cuz I don't want this to keep becoming a very long technical episode cuz we're going to talk about this a lot. This is not the first or last time you'll hear about power on this. But people like when we talk technical so >> yeah and we will and we will we like this stuff which is why we do it right and there's so much like nice technical stuff and this is like analog semis okay I I love this stuff personally this is what kind of I this is what I do okay and so I love analog semis zooming out like look at what happens you got power generated at the nuclear power plant and like you said that power is transmitted with like hundreds of kilovolts across you know the grid to maintain the losses to be minimal. Uh and finally it reaches like some kind of a medium voltage substation. Um that is all usually on the data center campus. Uh I have another whole article on how this whole big big picture thing works. So look it up on the substack but it is converted to something like 10 to 30 kilo volts.

32:11 No, this is all like alternating current still and all of these like require these gigantic huge transformers and insane looking things, right? These are not like sensitive delicate 2 nanometer gate all around. These are like haunting big machines that like burn power and it's it's it's just the complete opposite of the sensitive AI world. Uh and then you know though that power is then brought into the utility room which is usually like a little space outside the actual data center uh where the compute racks are kept and in the utility room it is converted into like you know 430 volts it's actually converted probably in the data center campus but the utility room gets like 400 400 volts or 430 volts you know this is like the three-phase industrial voltage that people get like you know you at residential levels you either get like 110 volts or 220 volts or something like that but usually industrial power is 400 volts right and then it is from there it is distributed to uh all the racks within the data hall it's called a data hall where all the racks are kept at the rack level this uh AC is converted to 48 volts at the rack so that's where it happens so this is like the power supply unit or the PSU right and the PSU after that 48 volt is available >> uh there to distribute it to various parts of the whole rack. It is in various stages converted to what is called an inter intermediate bus voltage. So you have these things called intermediate bus converters. What they do is they take the 48 volts and they convert it to like 12 volts. Now we are talking about low voltage bus architectures which is converted to 6 volts. So you know you can't convert 48 volts down to 1 volt.

33:58 That's a challenging problem to do. You don't you always have to have stages. So this 12 volts is an intermediate bus stage. So that conversion happens and then uh ultimately the 12 volts is converted down to maybe 1 volt range 1 volt.8 865 depending on the what GPU needs and you those are actually kind of different because those converters are they have one major requirement they need to be highly voltage regulated when the GPU wants8 volt it wants8 volt it doesn't want.9 you it cannot want 1.2 do so voltage regulation to very fixed values is very important at that stage. So those are called voltage regulator modules and those are very specific and they have the highest uh count in a data center right I mean because the voltage regulator modules are like many compared to the the the rack level conversion that happens >> either from the PSU or the intermediate bus converter which is the the actual mass market for power lies in those voltage regulator modules or VRMs. Uh so yeah so so what you can like look back here and see is look at the number of stages of power conversion right from the grid all the way down to the GPU and every section of this power conversion and voltage conversion that happens has a different set of technologies like some of them operate on big inductor coils you know you you have these like substation transformers but then you have these like what are the they called LLC converters they just use like uh big transformers uh with windings like this uh you know coupled to each other and uh they just convert voltages you but those are not regulated voltages you know it doesn't matter if it converts to 48 volts or 50 volts it's fine it's not doesn't have to be like accurate right so the entire technology chain and the supply chain for that is completely different then you got this different supply chain that goes from 48 volts to 12 volts the bus converters so there are companies that specialize in that and you know those you have to get designed with in data centers in that section.

36:08 Then you got these voltage regulator modules. There are like a few companies there who dominate the space. So every part of this power supply chain has a different player has a different technology. It is not like what you you can see in HBM. Oh it's HBM what is it like the same three companies what are they doing stacking the same way? It's kind of a closed problem in a sense. power is a wide wide open problem and it's you know there's a very wide range of topics and technologies to look at.

36:37 So it's very very complicated to be honest to be it's it's not something that's very straightforward and now just like everybody on on this the investors the semi- world and people who are like interested now everybody has become an optics expert right [laughter] yes >> now now everybody knows what is like Indian phosphide everybody knows there's laser shortages uh you know everybody talks about like fiber attached units and you know coherent optics and you know all of this stuff lane rates is like quite common now now You're going to like hear people talk about power conversion topologies, but it's going to be far more challenging by the way because it's really really a big wide area of power conversion.

37:17 >> Yeah. Yeah. It sounds like a lot of opportunity, but a lot to cover. If there's different technologies, different materials, different companies in the supply chain. Um, oh man, if we thought we were busy with earnings calls already, [laughter] just wait. >> You can have an army of people just cover power earnings calls. I mean that's that's how many companies there are. No, but the the technologies are are amazing but also they're amazingly simple. Like ultimately it's all about converting between uh DC to AC or AC to DC. All you do is like if you want to convert from one DC to another for example the simple concept you can think of is you convert the DC to AC then you you use use a transformer with a different number of turn ratio. So if you use a 10 is to1 turn ratio you can bring it down by 10 the AC current by like a factor of 10. then you convert that AC back into DC and know you've converted it basically back you know step down the DC uh by by that amount then there are that's a simple one um but quite widely used that approach then there are like uh what are called synchronous buck converters those are basically based on square wave waveforms you know that is basically works on the principle of having you know how much duty cycle you want to have you keep the transistor on for some time and then you turn it off and then when you average it out you get a different average. So if you have it on for more time, you know, you you get a higher average. If you keep it on for less time, you get a lower average. So you can step down voltages using those kinds of synchronous buck converters. So the circuit topologies are also very fascinating, very interesting and the trade-offs are enormous. So it's a very wide space and if we are entering a space of like power limitations, we have a lot to talk about.

38:57 >> Yes. Oh man, you know, I took uh a little bit of power stuff in undergrad and at the time I wasn't motivated to be excited about it. So, it all came off as fairly boring. But what I liked about this episode is you helped motivate me to be like, man, I got to better understand all of this because there is a bigger reason for getting into the weeds here. >> Yeah. Yeah. And there is it's a w and even the topologies when we go to 800 volts is not set by the way like there is a big discussion as to whether you should convert from 800 volts down to 48 volts and then reuse all the existing infrastructure that already is there at 48 volts. That's one like way to do it.

39:38 >> Uh and the the other approach is that uh why don't you convert from 800 down to 6 volts go directly. So that's what like people like TI and NVITAS and all these companies doing they want to go directly to 6 volts you know why why the 48 volts why because every conversion stage you lose efficiency the fewer stages of conversion the better so why don't you skip the 48 volts and go straight to the intermediate bus wtage of 12 volts or 6 volts and then you do the conversion from there right so there's like all these like battles of like topologies and architectures that are still going on nothing is set in stone it's a great time to look at this really >> totally let me guess all the 48vt incumbent are like, "Yeah, just go to 48 volts and reuse it." And then all the 6 and 12 volt folks like, "No, skip that.

40:22 Come to us. We'll sell you more." Yes. Components. >> If it's a 48vt conversion, yeah, those guys are happy. >> If not, they're like, "Oh, no. What do we do now?" Like they're skipping our voltage altogether, you know? >> Right. Totally fascinating. Well, all right. We should call it here. This has been great, but it'll be fun to dive into more and see, you know, who ends up winning here. >> Yeah. Yeah. We'll leave the Substack post that I have on there. uh if you want to go read it, a part of it, at least a lot of the engineering stuff that I mentioned is is on the free portion of the post, so go read it. All right, I'm going to do the goodbye today uh because I've already spoken this whole episode. I might just throw the towel in. Uh so, thanks for listening to Semi-Doped. Uh this is been a fun podcast for us to run. And do check us out on YouTube because we always put like pictures where we can. And we are also on all podcast platforms if you ever want to, you know, listen on on that. Uh but also we do have a substack where we write daily updates on this stuff because we have so much news and stuff we monitor that we try to write it over there uh so that you have all the latest news. So definitely give us a follow on there as well. Uh that's it for this episode and catch you on the next one.