Transcript
0:10 It is August 9th, 2024 and I am Hansen Sue with the Computer History Museum here with Doug Fairburn, also with the Computer History Museum. And here we're with Jetson Huang. Thank you for coming and so we'll start with where and when were you born and talk about your early years. I was born in Taipei, Taiwan, February 17th, 1963. Is the same year that Michael Jordan was born.
0:41 It is the year right before the IBM system 360 was announced. Pretty good year. And could you talk about growing up in Taiwan? I was there for about five years. my my parents my father got a job to open a start a oil refinery in Thailand. And so when we were 5 years old we moved to Thailand and we were there for about four years or so.
1:15 And in 1973 there was a coup and there was a fair amount of unrest in Thailand. We loved living in Thailand. I was I I remember a lot about living in Thailand. people were nice, the food was great. kind of fun growing up there. And because of the unrest, my parents thought it would be good for us to come to United States. And so they sent my older brother and I to to us and we stayed with our uncle in Tacoma, Washington for a few months as he as we found found a a private school to go to, a boarding school to go to. And the boarding school happened to be in Onita, Kentucky of all places, Clark County, Kentucky. And and, so we moved there and we went to school there for a couple years before we hooked up with our parents again, back in Tacoma, Washington. You know, that was kind of the port of entry for Asians for people in Asia coming to United States, immigrating to the United States. And so we met up back there again. And then we moved down to to to Oregon where my father found a job as a instrumentation engineer building building paper mills and and and the beginnings of building fabs and things like that. Thank you. so what were your child childhood interests?
2:59 hobbies, sub subjects in school, books. I was when I was young, I my interests were were school school and sports and I was on the swim team. I was on the table tennis team. I play I was on the soccer team. somehow I really enjoyed sports and and really enjoyed school. And so I did those two things. I played a lot of tennis, played a lot of table tennis.
3:30 when I was when I was 14, I think it was 13 or 14, I started paying table tennis quite quite seriously. And and I it was in Oregon at the time and so I played table tennis just about every single day, seven days a week for about three or four years. got quite good. Went to the US Open, went to the US closed tournaments and and I that was a that was a very big part of my life. I really enjoyed really enjoy table tennis. I enjoy table tennis to this day. I don't play it, but I watch it.
4:05 but but but that was it. School and sports. And how did you come to be interested in electronics or computers? Well, I was always good in science and math and and that that was probably very heavily influenced by my father. He was an engineer and and he was excellent in in science and math and and you know his field is instrumentation and chemicals and you know he's he's incredibly he's he's incredible in in that that field of science and engineering. And so I was I was always always inspired to do the same.
4:54 And it wasn't it wasn't until in high school in a lot of ways I I would have been I would have probably been a petroleum engineer of all things you know if if not for high school. And during high school there was a computer in in our lab of of all things. and and and that computer was was a teletype if you remember connected to a mainframe and and and there was a some kids that were part of the the computer club and the math club and the science club. And so I I I was part of all three clubs and and and turns out all three clubs had basically the same four people. And so and then we became good friends. And and so when it was time to go to go to college, I I went to college in the same place that my best friend at the time, Dean Verhiden, I chose to go to Oregon State. And I thought Oregon State's just fine. And we both enrolled in, you know, in a in a in a field that was very very heavy on science and math and was engineering. And so so electrical engineering was was the choice that we made. so between between the the exposure to computers in high school being part of the club that that really enjoyed science and math and computers that was kind of the the direction I chose and and it was it was a great direction. My my parents thought that I I could have still had a a great career as a petroleum engineer, but I I decided that maybe electrical engineer and computer science would be my my future and it turned out okay.
6:42 and could you talk more about your experiences or influences at college and in graduate school? well the the most the most important events so so I I think that that meeting meeting the people that I met in high school led to led to electrical engineering. And while I was in electrical engineering and I was I went to school rather young because I I I skipped two grades and I was I was a I was particularly young compared to the other kids. The electrical engineering class was something like 250 people or so. E fundies at Oregon State University was like 250 people and 246 boys and four girls. And I was the youngest kid and and I I I remembered seeing Lori, my wife, today and we met in E Fundy's lab and and I I was I was fortunate enough to to be in in the same lab team as her and And I suggested that we ought to do homework together, you know, every weekend. And that was that was you know that was a that was a you know my way of of asking a girl out for a date I guess you know and so so we would so I you know my pickup line was was you want to see my homework you know and and I I knew that that was my superpowers and and I was good at homework and and I I did well in school and and so every weekend we would get together and that was that was probably the the second best thing that that's ever happened to me, you know. So, the first one was just choosing electrical engineering. That was a great thing. And it turned out that she, you know, choosing electrical engineering put the two of us in the same path. And and that that we've been together ever since.
9:03 And and so those those two experiences really formed almost everything else. Great. did you have any important influencers such as mentors, heroes or teachers? I I've I've always found that learning from unusual places and and getting advice and insight and inspiration from from just about everybody I meet has really formed, you know, the way I think. And of course of course you know we we had we had great teachers professor Aort and Eve Fundy's Dr. Hennessy at Stanford. you know, the most important books, some of the most import important books that shaped our our careers, Carver me.
10:08 there there are many many great teachers and and influencers in in how we think and informed our career and but but the things that that I found has been the most inspirational to me are are people who are who are making a living. you know, they're gardeners or or dishwashers or cooks or painters and artists and they're they're striving to make a living through great craft and doing the best work that they can. And and trying to support their families however they could. And one of one of the one of the one of my favorite memories is is when our family went to Kyoto one year and and and we were going through this the famous moss garden in one of this I think it's a silver temple and and there was a gardener squatting down picking picking moss and and all the tourists kind of walked by him and I stopped and I was I was amazed. ed but by what he was doing. He It was incredibly hot in Kyoto in the summer, completely still air. The humidity is so heavy. it's in it's intolerable and and all of the all of the the tourists are are, you know, walking as fast as they can to the shade. And and here here's this gardener squatting down picking picking moss. and and he he's had a little bamboo picker and a little bamboo basket and no hat, just squatting down picking picking the the moss. And I'm looking at the basket and there's hardly anything in there. And I I stopped to ask him what he was doing and and his English was perfect. And he said he said, " I'm taking care of my garden." And and I said, " but it looks it looks perfect." And he says, "No, if you look carefully, there's there's some dead moss." And and he's picking the dead moss. I said, "But your garden is enormous."
12:14 And he says, "I have plenty of time." And and that that phrase, "I have plenty of time." I I repeat constantly. people come to me all the time, you know, Jensen, you're you must be really busy. I always say, I have plenty of time. we were talking earlier, I have plenty of time. whatever I'm working on, I have plenty of time. I don't wear a watch because whatever I'm doing is the most important thing at the time. I think I think learning learning to do with an aspiration for for tackling that enormous beautiful exquisite garden one tiny piece of moss at a time and realizing that you have extraord you have lots of time if you dedicate yourself to it prioritize your life to do that you have plenty of time. It's such a great great way to live and and I think to this day I try I try to live that way that you know if you if you prioritize your life and and you don't get all of you know you don't pile on a lot of things that are in the end not that meaningful to you.
13:28 if you prioritize your life properly and you you dedicate yourself to that priority, you have plenty of time to do your life's work. And so you know the these kind of experiences meeting people who are who are elevating their craft whatever it is dedicating their life to it working incredibly hard having a having such wonderful spirit doing whatever job they have. I love that and and they they always lift me and inspire me. Great. Thank you.
14:03 talk about your first job and what did you learn from it? Well, my first job, my first regular regularly paying job was Denny's. I was my older brother got it for me. I was incredibly shy and and never left the house and didn't know how to leave the house except, you know, with my bicycle or something. and and my older brother was is outgoing. love loves just being with people and and and he got a job as a as a dishwasher and then became a bus boy and a waiter at Denny's downtown Portland. And and he said, "You know, you ought to come you ought to come work and get a job." And and so he he arranged for an interview for me and and I you know I got a job as a dishwasher and he taught me how to be a dishwasher and and then I became a bus boy. Taught me how to be a bus boy and then became a waiter. He taught me how to be a be a waiter and and that was my first job. I I learned I learned the value of hard work of course and like everything else whether it was dishwashing or cleaning bathrooms at Denny's or you know busting tables or waiting tables I I always did the very best I could and and left everything on the field you know and so in my eight hour shift I I worked every second of it and I did the best best work that I could and always left it you know in in immaculate ely for for the next for the next shift. And so I I got to learn to interact with people became less shy. learn how to how to be in service to someone else.
15:58 create a great experience for them. listen to complaints. learning how to deal with complaints. and and although the customer is always right you know ultimately you're you're also running a business at Denny's and and you know finding a solution for for for the customer and the business is is something that I had to learn along the way and and so I did all all that when I was a kid and I I love that. Yeah, that was my first job. My first job after after graduation was was AMD and I loved that job and I had a choice of of just about every semiconductor company and I chose AMD. I was really really quite excited about the the bipolar microprocessors that this team was building and and it it got me exposed to designing large chips and my my part my part of the of the CPU was the was the instruction decoder and sequencer and and scheduling all of the pipelines and and so that was that was incredibly fun to do. so so anyways those were great jobs. Thank you. And could you talk about your role at LSI Logic? what motivated the move? what was your role there and what you learned and your major accomplishments there?
17:26 When I when I was interviewing for a job at Oregon State University, I got a job offer from just about everybody I interviewed with. and and I had a choice between between AMD, Intel, LSI Logic, National TI, all just about all of the semiconductor companies and and I chose I chose in the end I had to choose between AMD and LSI. I I couldn't exactly decide and I ended up choosing AMD because I wanted to go work on building a microprocessor and and and that was a good decision. I I learned a lot about the trade-off between ECL, TTL, and CMOS. And it really it really helped you hone your understanding of the scaling laws and and and the trade-off between performance and and density and power and AC power versus DC power, static power versus dynamic power. all of those all of those learnings and trade-offs were happened in one place for me. And so I learned a lot about about about chip design and trade-offs of various process technologies and how to think about performance and power and density and and so that was fun. I really enjoyed that and I so I chose to go to AMD.
18:49 LSI Logic never gave up on me. And it turned out that one of my office mates. She was terrific. And she went she went to a company called LSI Logic. And when she got there, she learned from the the people there that they try to recruit me out of school. And so she contacted me again said, you know, they're we're working on some really great things at LSI Logic. and and and now that I worked at AMD, I realized how impactful LSI Logic was going to be. And and and the reason for that is because we built all of our chips by hand at at AMD. And it was during the time when when EDA was becoming more of more capable. And I remember I remember I was working on a on my instruction decoder and I was I downloaded two open source software packages Espresso and Equinat.
19:47 Equinat take truth tables and and or takes equations and turn it into truth tables and Espresso optimizes that truth table to a better more optimized truth table. And so those two open- source tools from from Berkeley enabled me to be a lot more productive. And although the results were still require a lot of massaging and you have to you know know how to use the tools, the results were very impressive and it made it possible for for me to handle much larger designs you know by yourself. And so so I I was I was smitten by the idea that design tools could enable us to design much larger VLSI systems which was what LSI Logic and another company VLSI Technologies was doing. And so I was so excited about that that and it wouldn't have been possible for me to see that insight if I hadn't worked at AMD first. And so when I when they reached out to me again with LS from LSI Logic I realized the potential and the power of the design tools to the future of design that is it isn't just about understanding of circuitry it isn't just about understand of understanding of process but it's really about the understanding of these tools and it's going to enable us to design very large systems and and that was that was a breakthrough. LSI Logic was revolutionary in its understanding of the intersection between system design, design tools and semiconductor design, chip design. That that that code-design process if you will full stack design process really influenced how I think about Nvidia computing today. you know that the importance in in if you will deep learning is not just about the chip, it's about the system, it's about the tools, the frameworks and the algorithms all being worked on together to achieve extraordinary results. And so in a lot of ways, you know, my first my intuitions about about deep learning, its impact to the future of computer science and the future of software was really influenced probably by you know what I saw at LSI Logic in the beginnings of that. Now that led to LSI Logic and VSI technology both both companies really enabled the systems companies to become chip designers. You know, remember the chip industry were chip companies coming up with offtheshelf chips that people would buy and then they would cobble up systems. LSI Logic and VSSI Technologies enabled systems companies to design exactly the systems that exactly the chips they want. And in a lot of ways, systems on a chip, if you will, concept was really conceived of even then. and and it it introduced me to a whole lot of companies that were building building semiconductor driven, you know, custom chip driven systems and one of them was some micros systemystems and and it was it was through LSI Logic that I met the founders Nvidia. They were at some micros systemystems and Andy Betosham was designing a system that was completely based on custom chips. Chris and Curtis were building a a graphics system based on custom chips and it exposed me to a lot of great people that were great computer scientists working there and and together we we we built some amazing systems together from the Sundragon system which was a at the time a pioneering system for large scale distributed computing to the Lego graphics system that Chris and Curtis worked on to the Spark Station one that Andy Vetoshine worked on. I was involved in all of those and and it really influenced how I thought about system design and inspired me about you know how what a a computer company would become someday and how the semiconductor industry and the computer industry systems companies would really fuse into one. It used to be two industries and today if you look at it it's one industry one giant industry where the where the semiconductor industry starts and ends where the systems company system industry computer industry starts and end it's completely been been overlapped and fused and so I I had the early insight of that seeing the formative years of that and it it really shaped my career. Thank you.
24:25 Doug is going to take take over for the next section of the interview. Thank you very much. Okay. Thank you. We're going to do a start and stop. Okay. Whenever you're ready. Okay. So, now that we've covered the sort of precursor, let's go on to the main event that is Nvidia. And what I'm going to do is focus on four or five of the major technological changes that Nvidia has gone through as described on your own website and try to understand the dynamics of what was happening at the time and your role in sort of shifting the company's focus from one stage to the other. so I'd like to start with the founding and that is everybody you know that's interviewed you has you know gotten the story of the Denny's discussion with with your co-founders and I'd like to dig a little deeper into that. They you know when you first were in you know wanted you know sat down for this meeting did you know what the meeting was about?
25:28 Did had they already described what their plan was or their idea was and how did that the ideas as presented at Denny's evolve before you got to the the first business plan that is what was your input into that into their vision? the all what we started with in the beginning was was Chris and Curtis wanted to start wanted to leave Sun. I mean that that was the the impetus of of the entire conversation and the question is what what company can be built. of course, of course, between the three of us, computer graphics, computer science, computer systems, building chips, was was really, at the core of our expertise. And, you know, in our generation, we were we were the industry leading experts in in some of these some of these areas.
26:29 And so so the question is what kind of company can we build that that could attract business in the near term and then what kind of company can it become someday and some of the things that that the philosophies that the three of us believed in. There were two there were really two camps that that two camps in in how to do how to do computer graphics at the time. One of them was was a general purpose processors running algorithms and it it really it really rides on CPU scaling and general purpose processing unified memory and and the whole the whole idea being the vast majority of computer graphics would be software rate tracing of course is is the perfect example of that. it was largely a software software algorithm.
27:25 and and then there's there was another another group and it's a rather small group that is that believes that acceleration is really the best for some types of applications and computer graphics would be one example of an application that really wants to have dedicated pipelines to accelerate the processing of it. And so and then there are some people that are some somewhere in in between. And so the idea of computer graphics which is a very computationally intensive workload and in 1993 one of the most talked about workloads in computer science in computing. that span a spectrum from general purpose computing all the way to accelerated computing. And and we thought we thought that that that we could create a company that could accelerate design a a bespoke pipeline dedicated to processing pixels and that pipeline would of course be the formation of the first application that we would accelerate called accelerated computing. During that time Curtis and David Rosenthal came up with with the architecture of the company, the architecture of our of what is what is still largely governs a lot of the way we think about computing today and the way we work today. this idea that the hardware because has so much software on it will always be the limiter of the company if we don't make it an architecture that could span multi multiple generations.
29:09 And how do you solve this problem that that has hardware that is bespoke to the algorithms meaning it's it's a pipeline not a processor with instruction sets on the one hand and so you're you're heavily dedic heavily heavily designed to process a particular algorithm pipeline on the one hand on the other hand survive multiple generations as the algorithms are changing as the pipeline is changing And the chip underneath is changing.
29:41 And so that that separation between algorithms, device drivers, algorithms and the hardware to create a virtualized interface. We call it univer our unified driver architecture. Basically univvice device architecture is kind of like a whole bunch of accelerators could be virtualized with an abstraction. that virtualization layer that that way of thinking of our our our hardware as a virtualized hardware was completely revolutionary and Chris Chris Curtis and David Rosenthal came up with that idea. and it allowed us to build multiple generations of hardware and be compatible with the software that we're building on top. And that observation that software is a gigantic investment that has to be extended through multiple generation on the one hand allows us to create an installed base on the other hand and for the platform to to get better and better over time is a concept that really is associated with general purpose computing.
30:53 these ideas that are associated with general purpose computing but not acceleration that we brought to acceleration and that's why we call it accelerated computing not accelerators. This idea was really formed in 1993. Reasoned about from from a a understanding of of computing where the investments will go the interactions between software and hardware dates all the way back to 1993 and 1994 during the formative years of our company. I'm really proud of that investment really proud of that that that strategy. Curtis, David Rosenthal, Chris did a marvelous job in in in recognizing that and inventing that fundamental found technology foundation for our company and that and that kind of sets the stage really. Doug, it it it set it set the stage for us to evolve from a first product 3D graphics accelerator company to a more than one algorithm accelerated computing company. It is all because of that virtualization layer, that understanding of architecture, understanding of the interplay between software and hardware that really made it possible for us. But the early days we were focused on you know what is the application we could we could sell first and the first one was 3D graphics and this the strategy part that got us off the ground was was of course with every single company 3D graphics is a technology and and what product are you going to address for what market are you going to do it you know how are you going to make an impact and we we selected something that was That was also interesting to us because we're you know we're all of the video game generation. but an observation that almost everybody would be a gamer if the number of genres of games would would grow and the medium would grow and the the platform reach would be high. and so we chose three things that were the perfect three ideas for the company. 3D graphics for video games to be taken to the market through a PC, a personal computer. And it was at that perfect time of Windows 95, multimedia PCs and the creation of of 3D graphics accelerators that that really brought it together.
33:26 Now, of course, at the time, at the time, 3D graphics for for consumer products was very hard to do because 3D graphics is naturally very computing intensive. On the other hand, the video game market was non-existent and the 3D graphics video game market was exactly zero and Windows 95 hadn't come yet and video was founded in 1993. And so, you have these three circumstances with very low probability of success converging. And so it was it wasn't an easy thing for our company to to be formed. And it caused us now to think about, you know, how do we create a video game market on top of 3D graphics for the PC industry. And so Nvidia really largely created the PC gaming market. And to this day, we're the primary evangelist of PC gaming. we developed the necessary software and go to market and cultivated developers created libraries to to enable them to use 3D graphics hardware and promoted the promoted the the esports and promoted and evangelized the the this this this whole category and this whole industry and now PC gaming or video games is the largest entertainment genre in the world And it was precisely zero in 1993 when we started. And so we I'm I'm you know I'm I'm I'm proud of the the company's ability today to really create new markets. And if you if you think about accelerated computing, accelerated computing is very different than general purpose computing in the sense that unless you go and create the market, there's nothing to accelerate.
35:14 And unless you can come up with algorithms to accelerate it, there will be no market. And so our company is one of those one of those few companies that have solved this chicken and egg. How do we solve this simultaneous creation of the technology and the creation of a market? people give us a lot of credit for the immer the the the democratization if you will of AI computing on our platform and give us a lot of credit for cultivating the deep learning ecosystem around the world and celebrating the success of it and u helping the researchers ultimately connect the technology to real applications in the marketplace and really cultivating this this this ecosystem which is now called the AI revolution. Now we started that about 15 years ago and I think this the skill set of simultaneously creating the technology and creating the market is something that our company has really done well and and I'm very proud of it and we're doing it for healthcare for computational biology. We're doing it for for quantum computing for classical quantum classical computers that that will be the norm in the future. We're doing it for for robotics autonomous vehicles. Each one of these different ecosystems really requires the fundamental invention of the algorithms and the computing stack as well as the market itself and the ecosystems that that turns it into a large market and and so we're super good at that. Well, certainly justifiably proud for the the position that you've created and the opportunities that you've generated in the marketplace. I'd like to go back and drill down to a couple things because as is well documented, there have been a few stumbles along the way, near-death experiences in the words of others. And I'd like to figure out, you know, exactly how you got into it and how you got out of it, so to speak.
37:19 so the first one as I understand you know you started the development of your first product you had some relationship with Sega to use that in a future product. but it turned out that due to some changes on the part of Microsoft that product didn't work out and so you wound up in a situation where you were in short supply of cash and in in somewhat a desperate situation. So what was your role? I mean you had other founders you had to you know it was a makeorb breakak kind of time. what was going through your head and what was going through the heads of the other founders and the employees that sort of turned that around and and got you back on your feet.
38:12 We've had we've had quite a few near-death experiences. we were the first company to to really realize that this intersection of 3D graphics consumer 3D graphics at the time a proper 3D graphics system was about million dollars called the reality engine at Silicon Graphics. That's a proper 3D graphic system and it generated images that look right and and and what we wanted to do was to create a system that allows for the experience of that million-doll system but for a consumer product three four $500. And so we had to consumerize the technology.
38:57 And the first the first attempt at consumerizing that technology made it possible for us to render beautiful beautiful images with very few transistors and very little memory. And we used four texture mapping curved surfaces so that we didn't have to use as much floating point on the CPU that wasn't available at the time to do the transform transform and lighting. and and no Zbuffers because when when memory was $50 per megabyte, it's you know, I don't even think it's $50 per gigabyte now, but $50 per megabyte, the amount of memory that you can use is quite limited. And if the amount of memory you lose you use is limited, then the bandwidth you can get is limited. And do so doing the Zbuffer read modify right is too bandwidth intensive and uses too much memory. And so so we decided to use a invent an a method of rendering beautiful images without as many triangles curved surfaces without zbuffers basically presorting and without without heavy read modified rights associated with texture mapping and so we use forward texturing instead of inverse texturing.
40:15 And so so the the the three techniques really distinguished our architecture, graphics architecture. unfortunately those three techniques were very hard for developers. And ultimately ultimately the thing that that I learned in that process is that developers is everything. If you don't if you don't create an architecture and a computing platform that's easy for developers that that's rich enough for developers to develop their ideas and realize their imagination it is simply not going to be adopted no matter how cost-effective it is. And so it turns out the right answer was triangles instead of curved surfaces, inverse texture mapping instead of forward texture mapping and Zbuffers instead of no Zbuffers. We got it exactly wrong.
41:03 And by the time that we were in our second generation full end, the first generation had plenty of challenges with engineering execution. We learned how to work as a team and and build chips. but the second generation was was on its way when we finally realized that the world is really going to move towards the way of doing 3D graphics that we do it today. and and that was a very difficult time for the company because because they're at this point 30 different companies that are pursuing the same vision we are and some of them startups some of them spun off from large companies and so on so forth pursuing this consumer 3D graphics marketplace. meanwhile you know we were we were on the wrong path and and that was a that was a complicated decision for the company.
41:54 first of all, because we were on the wrong path and we were also under contract with the with Sega, the game console company, to build their next generation game console for them. And so, we simultaneously had this contract and a whole bunch of payments that is going to come as a result of that that's going to sustain the company on the one hand. On the other hand, I knew we were going down the wrong path. And so, there was a lot of discussion in the company.
42:19 It was it was a quite challenging. of course very emotional because you've now spent two and a half years doing it the wrong way and we have obligations to a very large partner. competitors are racing forward. and and so the question is what to do and and I think I think during during challenging times like that the ball of wax of decisions and tangled decisions can make can paralyze almost anybody if we if we didn't if if we if we didn't do that if we didn't do that we didn't do that a lot of complicated things.
43:04 we we came to we came to the right decision in the final analysis because before you think about all of the other decisions, what about the contract? What about money? let's get on the right path. Let's just get on the right path. there were questions like if we do it the way other people do it, then what's our strategy for differentiation? Well, doing it wrong is not going to help you differentiate. Okay. and so let's just first do it right and then we'll figure out how to differentiate.
43:37 Let's get the company on the right path and then let's figure out what to do with the contract. Let's get the company on the right path. We'll figure out the money part of it second. And so the realization that we have to get the company on the right path with a lot of debate, a lot of a lot of a lot of a lot of a lot of people feeling very differently about each part of it. I think we we we were we were able to get the company grounded on let's just focus on the right technology first and we'll come up with all kinds of great strategies to differentiate. we will solve the contract issues, we'll solve the funding issues, but it has to start here.
44:18 And that ability to ignore all of the consequence of the right decision and make the right decision first started, you know, for us in 1995. And and I I think a lot of companies are entangled, a lot of decisions, a lot of people are are so concerned about the consequence of a decision that it keeps you from making the right decision. Make the right decision first. The consequence you will think about second. And so it it ended up I called Maji the CEO of Sega at the time and I told him our circumstance. I told him that we were going down the wrong path and if we would have completed the project we would have caused Sega to go down the wrong path and and asked to be relieved of that contract and and then but then I I I asked them also for the money as well. And the reason for that is because without the money we would be out of business and in in his generosity and kindness and good nature. He he saw it as a way to he appreciated my honesty. He appre appreciated my frankness and and he he appreciated my my vulnerability and as a CEO you have to ask for help and so and I'm sure as a CEO he asked many other people for help in the past and and he saw a young CEO that he could help and without without the generosity of Sega Nvidia wouldn't be here today without that making that first decision to do the algorithms the new in you know inverse texture mapping triangles and Zbuffers and and without making that decision, we wouldn't be here today. U and the rest of it the rest of it is is just engineering excellence. The fact of the matter is I I started the company with two of the finest engineers on the planet and I was surrounded by the the best of the engineers and and when we pivoted and we gave ourselves no time to build what ultimately turned out to be the most high performance consumer 3D graphics processor the world ever seen at the time. And we did it in just about nine months because we had no time left. we changed how we designed chips. We changed the methodology of verification of chips. we changed how we did almost everything so that we could squeeze in the ability to build something extraordinary in the short period of time that we had. And so the equation kind of inverted for us, Doug.
46:59 Instead of saying this is the chip we have to build, how long is it going to take? We started with this is how much time we have. and how do we build such a thing? And and so we we broke everything down how we did co-design between software and hardware. the we adopted emulation. There was a company that was going out of business at the time, IOS, and I called IOS and and asked them, "Hey, I heard about this this instrument that you're making called emulation, so we could emulate our software before we even tape it out." And they said, "No, hey, sorry. you know, we're shutting down because IOS has gone out of business. Nobody wants this emulator.
47:39 And and we said, "Hey, let's listen. You you have do you have one we could buy anyways?" Turned out they had one about to go to scrap that we bought. And we made it work. and we became the first chip company to emulate in the world. And now we co-design through emulation. our architecture, design, emulation, software is all co-designed before we tape out and we we started a a way of designing chips and complicated systems and code design between software and chips and architecture that the world never seen before and so so I think I think making getting the company in the right path and then believing that the engineers and your people will do extraordinary things and make subsequently amazing decisions completely reinvented does and so so that first diving catch that first life-threatening experience for our company formed NVIDIA today. The way we do design, you know, is exactly the way we did it then. Okay. I'd like to thank you for that. That's a great insight. I want to just since we're limited on time, just focus on one other transition. You know, you sounds like you started with a broader vision than just doing computer graphics, the more accelerated computing and, and there was a specific time in which you sort of moved the customer focus from not just computer graphics and gaming to a more general scientific computing acceleration. tell me how did you make that decision and what was that process like?
49:23 the journey from from computer graphics to physics simulation. That was probably the first the first journey was an observation that that computer graphics or virtual worlds are only so interesting when you make it beautiful. It had it's even when you render it beautifully, everything is static. And so you want to bring life to it. You want to bring physics to it. you know, we're ultimately in the business of of simulating virtual worlds. Electromagnet magnetics is the first part. Light is the first part. but physics is of course atomic physics is of course a very important part of virtual worlds and and so that when we when we started to when we got into GPUs and we made it possible for us to render images program using software the next the next chapter of that was how can we use software utilizing these programmable engines to compute physics from from computer rendering ing images to processing particle physics or fluid dynamics and bringing bringing the virtual world to life. And that was really the question and we explored it with initially with a language that sat on top of our 32-bit floatingpoint units and we called that CG C for graphics to be able to express more complicated algorithms using C instead of programmable shading languages. So that was our first step. we learned a lot in that process. We learned that there was a lot of interest both in in our internal consumption for creating physics algorithms and bringing worlds to life. as well externally people wanted to use it for CT reconstruction wanted to use it for image processing. seismic processing was a was surprisingly an interesting application. Inverse physics was was quite quite interesting. And so so we we got some early feedback that this was a good direction. And of course the company kept pursuing C for graphics CG to to what it we now know as CUDA. And at the time there were a lot of innovation around how to make GPUs more and more programmable. And some researchers inside our company came up with a way of of turning all of our programmable shaders and these pipelines into something much more much more general purpose. And the concept of shared memory and SMS and CUDA were invented at that time. And but anyways the journey was really about becoming more and more expressive to be able to represent the world of physics and the virtual world better. It's really about about the idea of simulation.
52:23 then then from there we got some early traction with some customers and one of them was was was for seismic processing. Schlumbumberge, Western Gico saw the potential of using our GPUs to accelerate seismic processing, you know, 50 100x so that they could reduce the the the cost of doing seismic processing on these large farms of of CPU supercomputers. and that gave us that gave us confidence that there might be some future in using Nvidia GPUs and scientific computing. eventually eventually we convinced the Oakidge National Laboratory our one of the largest US open open science scientific computer scientific computing centers to use NVIDIA GPUs for their next generation supercomputer. And the reason why they chose to do it was because the amount of power necessary for scaling these supercomputers was growing exponentially.
53:27 and and we know that dinard scaling which is related to Moors law of course dinard scaling has really slowed down and and they needed another way of doing computing and so they saw the work that we were doing with CUDA and GPU acceleration GPU computing and they were excited by that inspired by that and we built the world's largest supercomputer at its time using Nvidia GPUs which then exposed us to all kinds of fields of science from molecular dynamics to you know to quant quantum chemistry to fluids to you know from biology to material sciences to climate sciences, environmental sciences just enormous reach of algorithms as a result of that and you know one step after another step we made our architecture more and more generalizable without losing its efficiency for acceleration. That balance is is one of the artistries of our company. you know how do you on the one hand u make your GPU more programmable but not so that it becomes a general purpose programming system and so it's we we we wrote we walk that fine line very carefully so that so that we retain the acceleration properties the you know many X factors of acceleration and therefore many X factors of energy efficiency and cost efficiency on the one hand on the other hand, we can explore larger and larger domains of acceleration and processing so that we can create a large install base. And as you know, computing architectures ultimately reach a tipping point when the install base is sufficiently large and its utility is sufficiently high.
55:19 and that's never been done for accelerated computing before. that's never been done for anything but anything short of a general purpose processor before. And so we've done something very very unique for the first time and we've now reinvented computing if you will that in the future almost every computer will be accelerated in some way and that accelerated computing is really the the basic way of doing computing going forward. So it's good that you've outlined all of these other applications and in fact some of the earlier applications which may not be well known or well understood. the the one that everybody is focused on today is of course machine learning, deep learning, AI and so forth. So tell me and I'd like to now jump to the philosophical level. you know you are you know the major player in in providing the tools and and technology to make AI pervasive in its many different forms around the world. What's your what's your personal take on the direction AI is going? there's obviously people talk about the the threats, the opportunities.
56:34 sort of how do you look at that and you know given that you are the engine that is fueling this fervor and activity around artificial intelligence. Well AI is fundamental AI is fundamental for a lot of different reasons. This is this is unquestionably one of the most significant computing transitions we've ever seen. It's fundamental in many ways. First the way that you process software in the future is fundamentally different now because of AI. The vast majority of its computation is done in tensors in linear algebra in these neuronet networks and so the vast majority of the computation is now done on GPUs and so the the computing model has changed very dramatically. The second part is that the the software development methodology and the software development tools has completely changed. Just as in the early part of our careers when you designed a chip by hand versus using EDA tools to design a chip, the methodology completely changed. How you how you thought about the chip completely changed. Your mental image of your chip changed. You know when I was designing a chip at AMD u the mental image of my chip was transistors and gates but when I started designing chips using design tools my mental image of the my chip was language and so the world changed in in the representation of the work and the methodology and the tools that you use for the work the methodology completely changed. The third is the type of things that you can create has changed there.
58:18 There the software that that we write by hand is impossible to handle the diversity and the long tail of of the world. you know the the the world's data is unstructured data is noisy and for many types of of simulations and predictions principled algorithms or heruristics are simply principled algorithms are simply can't scale heristics are simply too brittle and so we need to have a new way of writing software that no humans can and AI is really that method and so I've now described three layers the computing layer has fundamentally changed. The software layer has fundamentally changed and the applications the algorithms have fundamentally changed. And so in in a lot of ways this is because it's so fundamental it's affecting every single industry from the computer industry of course and we're you know Nvidia is at the epicenter of that. We're witnessing it firsthand to every single one of the large industries in the world that has to apply computers to solve problems and everything from healthc care of course retail of course you know digital marketing of course manufacturing, transportation, logistics just about every single industry is in the process being transformed as a result of AI. how we developed the software and what the software can develop completely fundamentally changed. Some of the things that that we also know is that on the one hand the the potential of of productivity gains is incredible.
60:00 the potential of new applications of computing expanding the the reach of computing incredible. of course the capabilities of it is a is a two two-edged sword. And so as much as we're investing in the capabilities of AI, we have to invest in and innovate in the safety of AI. And so you're seeing just in just really really thoughtful work and really foundational work being done in artificial intelligence from the way that you curate data. You have to use AI to curate data because the amount of data that we're processing is so gigantic. You have to use AI to curate the data to train the AI. And that AI curation process of course allows us to enhance the the data distribution on the one hand so that we can learn from more data to pre-select good data to to expose your AI to to take out data that could be highly biasing or toxic or harmful. And so using AI to curate data is a whole field of science. And I'm really delighted to see that work. On the other extreme, using AI to fine-tune the AI, you know, we use now AI to generate synthetic data to fine-tune the AI so that the AI has skills that are that are aligned with the values and the needs of the application. and then creating AIS that guardrail the AIS. guardrailing technology is one of the most important areas. And so so there's a there's a broad spectrum of innovation in the entire AI ecosystem starting from data curation of curriculum curation to fine-tuning and synthetic data generation to guard railing to governance to interpretability.
61:55 All of that all of that is all being developed simultaneously in research in companies large startup companies all over the world. And so I I think that that the the conversation around safe AI is very healthy and it develops as a result new methodologies, new techniques, new technologies and new companies and meanwhile we have to innovate quickly so that we can enable these you know all of the safe AI requires AI. You know in order to guard rail AIs you need AIs to do that and so we have to innovate AI technology as quickly as possible. This applies to languages. but remember this applies to robotics and self-driving cars.
62:39 you know AI is is going to is going to be in the computer but AI will also leave the computer and be be in physical AI. And so there's a a large population of researchers and engineers working on safeguarding AI for physical AI. And so functional safety from the computing perspective, algorithm perspective, diversity and redundancy, multimodality sensing, all of that inside out safety, outside in safety, kind of like air traffic controllers, if you will, you know, just a mountain of research that's been going on in in creating safe AI for both language AIs as well as physical AIS. And so I when when you when you when you look under the cover is where I get to see the developments around the world you you'll be enthusiastic, hopeful and quite proud of the industry doing the work that it's doing.
63:35 Okay. I'd like to again sort of switch gears and just focus on a couple of personal things. That is from a you obviously been extremely successful in growing the company to one of the largest companies in the world from nothing. how would you characterize your management style in helping guide the company? Some are described as visionaries, technical leaders, business leaders. What how would you describe your style, if you will, as CEO?
64:08 Well, the CEO is singularly responsible for the strategy of the company. is the CEO is is not the only person that comes up with their strategy but singularly responsible for establishing it ultimately although CEOs are surrounded by amazing people and I am truly gifted to be surrounded by the people that I am and but the CEO has to set the vision you know where are we going and how are we going to get there and strategies have have short-term long medium-term and long range strategies and and the CEOs the the CEO has to be the person who orchestrates that and so so on the one hand I'm surrounded by amazing people on the other hand they want a future that that we all want to work towards and so the CEO has to be the person that that does that. my management style is to simultaneously work in the future and work in the present and very hands-on.
65:19 I I create the conditions by by which amazing people could do great work. I tend not to be very operational. we have we have amazing operations teams and people are running the business running engineering, running operations, running finance, running HR. I've got amazing people who are operating the company on a day-to-day basis. So, I take myself out of the day-to-day meetings and I focus my energy on two on two areas that the the areas that create the future and the areas that solve problems for the company in the near term. so that we're executing our strategy. And so between between between my management team and myself, you know, I'm the the person that could that could float, if you will. I can work on I can work with the researchers one day, work with the engineers another day, work with the marketing team another day, work across all of those teams most days, and allows the CEO, allows me to be able to float wherever the company needs me.
66:25 but at all time I'm grounded with with the realization that the company needs to have a vision where it's going and the company needs to have strategies to execute towards that vision. And and the meanwhile the world is changing all the time. And so your strategy short-term, medium range, and long-term strategies are all shapeshifting all the time. And so the the shaping of the strategy, the pivoting of the strategy and its manifestation inside the company in organizations and people that adaptation, if you will, sometimes people call it, you know, when when companies do it once every decade, they call it change agents. when you do it every single day, you know, I call it leadership. and and so we literally are shaping our company to execute a a evolving strategy in real time. And so that's a that's kind of the the nature of our company. Our our our management style is is unorthodox in that way.
67:29 It's unconventional in that way. you know, we don't have annual plans and five-year plans. Our plans are kind of evolving all the time. We have long-term vision. of course we have milestones we're hitting but otherwise the company is is evolving its strategies to accommodate this vision all the time. So great answer. So given that success in your style and so forth, I'm sure you get many questions from other entrepreneurs for advice or whatever. So for a a new entrepreneur looking to start a company, what is your what do you tell them?
68:12 Well, for entrepreneurs that are starting companies today, I'm I'm a little reluctant to to offer too much advice. And there there are a couple reasons for that. one reason is is is their world is different than mine. and my world was different than than Will Corgans when he was he was at he was CEO at LSI Logic. or Jerry Sanders when he was CEO at AMD. and so they were great CEOs, but my world was different. My environment was different, my pace was different. and and how you build a company is like building a machine and that machinery should take into consideration the context its environment. Why would you build exactly the same machine when it has to go, you know, 100 miles an hour versus a machine that that wants to go one mile an hour?
69:14 And so the the environment should govern should dictate what the the architecture of that machine is and the properties of that machine and the culture of that machine, the operating system of that machine. And so I'm reluctant to offer too much advice because their world is very different than mine. second I also know that that a healthy dose of ignorance and irreverence for how hard something is is an essential quality of startup CEOs and and founders.
69:52 you I love I love that when I go into a particular new circumstance whether we're building robots or building autonomous vehicles or when we were going into deep learning and and AI my first my first thought was how hard can this be well it turns out to be insanely hard and it's the reason why it hasn't been solved yet but I always go into it with the attitude how hard can this be and if somebody else has done it you know or somebody else will do it why shouldn't it be us And so how how hard can it be?
70:23 And so I I think when I was at when I was when when the three of us first founded the company, you know, I think our attitude was was how hard could it be? Well, it turns out to be insanely hard. The amount of pain and suffering that you know, setbacks and disappointments and surprises and, you know, building exactly the wrong technology and having to learn something brand new. all of that you couldn't have if you would have put all of that on day one and real that realization I think it would be too scary to start and so I I think one of the best things of an entrepreneur is the concept you know that the idea that that it can't be that hard and because they only have one person or two people or five people you're not going to tackle anything too challenging and so you you you either has you either have to solve really small problems or you have to believe that between the five of you, three of you, you you could tackle the world.
71:17 It's how hard could it be? And so you have to have superhuman mindset even though you don't have superhuman capabilities. and and and I I I think that that for entrepreneurs I I do think that their superpower is is is a fair dose of ignorance and irreverence and and this oversized belief in themselves. you know, they ought to keep that for as long as they can. And then and then over time reality will hit you and hopefully you have the character to to survive it and and ordeal through it and the cleverness to maneuver maneuver through it and the friendships around you, the the teams to support you along the way and and and the company with the sheer will, you know, to to work through those challenges. You're going to have plenty of them and and so so Maybe that's the advice.