Chen Goldberg (CoreWeave) on the AI loop, Forge and infrastructure for agents

0:00 It's not just going without a mission. We know what we want to do. We have a plan. We are executing on that. And that's something I really love about the core with culture. And what excites me is that I feel like AI can be used for problems that existed forever and we didn't have the tools to solve. >> It can be in health, it can be in education, it can be in manufacturing. >> You lead around,00 or so folks across our product engineering roles. I think for engineers several things are happening here. One, there is some identity crisis because you don't write code.

0:34 >> I think I was wrong. >> The more tokens you use, the less you sleep. >> And Goldberg, EVP of product engineering at Corvid. Thank you for coming on Thursday. >> Thank you for having me. I can't believe it. It took so long for me to be here. But yes, should work on that. >> I'll do my best to to to ask you a few questions. First of all, about this crazy thing we're in behind us.

0:54 There's a big core wave thing, a blue wave, and we're in the middle of Moscone South in San Francisco for the fully connected event that started way back when biases in smaller rooms and now this is like a whole new thing. Do you want to do you want to like address the magnamity of what we're doing here a little bit for a second? >> I think it's it's it's unbelievable and it's overwhelming and it's humbling. It's like it's it feels really strange to be in this situation because in our head we still see ourselves as a startup sometimes, right?

1:24 >> And then you come here and you see thousands of people, >> thousands of people >> and the sessions are full and the content is amazing and >> the vibe. The vibe is cool. It's fun. I had a few folks that came over for open eyes dev day which just the day before our keynote and then I kind of invited them here as well. You're already in the city come. one person said that like dev day was not that great for him and this was like the the highlight and he wasn't talking about the battlebots which also really >> the battlebots was >> this was crazy. There's music and sound and loud announcements as well.

2:02 >> And it's actually it's it's the kind of things that like fits our culture and the thing that we like doing, right? It's it's being exciting about like technology and like the fun things that you can do. Like we're having fun all the time. >> and and Lucas Bwald, the former CEO of Devices, he built a bot. >> I know. >> Have you seen this bot? He's like a little like spider thingy. That was really really cool. so this is just like the the fun parts, but I think it's also fun and I think I noticed this in you like when you announced the product lineup and forge for example I would love to talk to you about you know like a lot of pride and here's the question for you Hen Goldberg EVP of product engineering at Corvid. Which one of the releases would you say is was easiest for you to deliver and and like get excited about?

2:49 >> Wow, I wasn't ready. >> It is kind of a pick your children's question. I know it's annoying. >> You know what? I'm not going to put you in a hot position. Let me >> I'm actually the thing is again people don't know us. Like I talk on stage about speed and experimentation and learning and moving fast and that's what I love about what we're doing. >> There are so many things that you know when we start like you are you're not sure that it's possible >> and then it's there and you're like wow.

3:19 I'm not going to say a word I shouldn't say. >> Yeah. it happened. >> First of all, let's go is kind of the company model and I think it's absolutely fine on Thursday. I I use it sometimes. >> You can say I'm saying LF. Yes, >> let's execute because >> because it's not just going without a mission. >> We know what we want to do. We have a plan. We are executing on that. And that's something I really love about the core with culture.

3:46 >> Yeah. I wanted to go back with you a little bit on the previous steps in your career. You led the rise of Kubernetes and serverless in Google and now we're announcing Forge and the rise of aentici is happening agents everywhere. Do you see parallels between those like huge two movements? >> A lot. >> Yeah actually. So two things one definitely the experience that I had with doing the transformation is very interesting but there's other things that are very different from a pace perspective like we've never seen before but I'm going to touch something that I actually talked with Dylan Patel today on stage >> we had a fireside chat Dylan Patel is the CEO of semi analysis and my biggest realization from this conference is that something that back in the day we were working very hard at at Google in the Kubernetes community is to avoid fragmentation.

4:46 What does it mean? you want to create some standards and like best practices and APIs that everybody is using and and you can see some teams right Nvidia announced now like things that they are promoting but the problem right now that is happening is that everybody is so busy and have not just cycles in their brain but also time and things are moving fast that it seems like nobody is investing in that and that worries me because I don't think that Kubernetes would have been as successful without its ecosystem. Right? So the ecosystem is super important and the partners and that people can build on that. But in here what we also heard on stage is that people just have to rewrite everything every few months. The model is changing the >> the chips are like everything is happening and that worries me.

5:37 >> So if you ask me like what I want to see >> yes >> in this world I want to see a world where it's not fragmented. I would love to see a world where of course frontier models and open weight models live together and you have like frameworks that gives you flexibility and composable system and that's something that I hope we'll see some similarities between Kubernetes and the agentic world >> 100%. It is a lot that's happening at the same time. I have so many directions to go with you that I'm I'm just going to start. So you mentioned every few months, every few, you know, a new model. It feels like now >> the model is every few days.

6:16 >> Every few days, literally last week we talked about the latest GP6 and they just released 6.1. So at dev day and I had a chance to chat with some Alman about some stuff and then I asked could you guys like >> slow down >> slow down we can catch up. He's like I believe in you Alex you can do this and I don't know how other folks are like feeling. So I would love for you, you lead around,00 or so folks across our product engineering roles all the product and and engineers actually engineers they're getting new superpowers essentially with those with those tools. I would love to hear from you how do you see that? Have you played with some of those things? What are the stuff that you're seeing changing in the software engineer career right now and how would it evolve in the future?

7:01 >> There are so many angles I want to answer. So the first one I'm going to talk about again was today on the on the stage John Johns our CRO was talking with Parag Parallel Web CEO and founder and actually they were talking about the pace and whether Parag feels like he's up to date with everything and he said he's not >> and like when when you're sleep like you always feel behind. >> Yeah. >> And it was actually nice to hear someone >> like him that you think about like he's probably at the forefront of that and even he feels behind. And so imagine imagine how everybody else is feeling in that way. So I think that's number one from an engineering perspective.

7:42 We had a both in the customer advisory board and we also had a women AI launch today and that came up. I think for engineers several things are happening here. One there is some identity crisis because you don't write code. Mhm. >> Second, when you write code and you have innovative ideas, sometimes it's hard to be proud cuz people just take it for granted like h you had an agent write it write it for you or like >> so people feel also about that like they invested a lot even if they used of course to in order to produce something well >> even if you're using agent it still requires a lot of thinking.

8:20 >> Oh you you dedicate a lot of your energy into producing this. >> That's what I'm saying. But it's but it's hard because then someone maybe once you did that like maybe someone else can copy it from you. It's >> I think that's hard >> for engineers. I think that's an identity crisis for sure. >> And the the third thing that we talked about the people that I see most successful are the experts. And why is that? Because they understand the space.

8:45 They're critical thinker. They know how to communicate about those ideas. So because they're expert they can also communicate great >> with the prompts and >> with the prompts right because they're experts >> but they became experts. How do people learn? Let's talk about it. >> We know that like you need like 10,000 hours. Like there's research that shows that through repeating >> and going deep and being hands- on. That's how we learn. >> Yeah. >> But now we don't do it anymore.

9:11 >> >> that's like Oh, I'm sorry. I want I'm usually super optimistic. >> No, no, no. We're going to turn this around cuz I will tell you some stuff. Okay. I decided to do this Yapping career the podcast after I was an engineer for 20 something years. I started back in Israel, moved to the Silicon Valley. I led the engineering teams. At some point early on in the eye, I decided I'm not going to be needed. So why would I do this? Go find somebody else. I think I was wrong. And I think that like you said experts are much more needed right now because now basically everybody is a PM/eng engineering manager kind of with running like a fleet of agents and expertise is needed. How how do they get there?

9:51 That's a completely different question. >> So I think there is a way. So the first thing we need to appreciate the expertise and what does it take to build a system. I think again >> put aside for a second coding agents. >> Yeah. which we have companies of course building them and so on. But let's say I want to build a fraud detection or a call center or whatever those are agentic systems which are always on.

10:15 They need to be secured. They need to be reliable. I need to make sure that I have systems there that will keep the accuracy intact and and so on and so on. Those are super complicated systems. >> Yeah. >> And it's not by coding. It's not tinkering. It's not like none of that is true when you want to build such a system. So the first good news is that you need to do it. I think what we need to do especially experts is be thoughtful about how we are training people and how do we ask questions? How do we trigger people's mind?

10:50 And I think what everybody needs to make sure is that they need to be curious and not lazy. >> Yeah. And by the way, I'm really like echoing things that Rita, she's one of the engineers on the infant team. Like she said that that's like what she's thinking a lot about. She's a principal engineer. How does she help >> lift the juniors? >> Yes. Yes. >> Because they no longer have seniors. They check their work as much. I want to though turn this into into some of the stuff that that we have talked about here. So all of those production systems that folks are now using AI to kind of help them scale, they have problems in production and we're introducing Forge.

11:27 I would love to just hear from you directly. I know you agreed this on stage multiple times. What is Forge from Cory? What did you announce and how does this help companies make their agents better? >> I love the story of Forge. People probably don't know that, but most of our product ideas and everything we do just always come from a problem we solve some kind of painful like everything was with a customer or a user. I'm super excited that all the talks that we had at fully connected we continue with the tradition from weights and biases all user centric.

11:56 >> Mh. >> So how forge happened? Okay, we had lots of assets, right? We had withe models and we had our inference and inference serless and we had weave and and different type open we had lots of services >> and then I want to say probably like few months back we had two things happening we've developed area >> on one hand which is a system for researchers >> and the other thing that we're doing in the context of inference we were optimizing our inference service and building a a speculative decoding workflow And in both of those experiences, what we found out that the human in the loop was very tired.

12:35 >> Yes. >> Stitching everything together was horrible. >> I heard you mentioned his name on stage. >> Aaron, there is actually we didn't show it on stage. There is actually a presentation that Aaron did. >> Yeah. >> That shows the loop few months back based on his experience. But it wasn't just him, right? Like you can also talk with people like Tim for example that build area. It actually started with us and then we started talking with customers and what's happening is that that no longer those machine learning engineers are separated from the developers and identic systems are harder and this is just one part of the problem. The second part of the problem is that COVID believes in a full stack optimization, right? Purpose-built for AI.

13:20 >> And it was always very easy to understand when we talked about training. Okay? Because the training job runs on hundreds and thousands and sometimes 10 thousands of GPUs. One complicated system, one problem impacts your job. How you bring it together that I think all resonated. Now inference have evolved to agentic system which are also always on yeah >> and running on hundreds of of GPUs and this is where we were like huh and of course you have now more people worried about the data and and and doing pre-training and post- training and so on. We're like this is a world that we know how to solve so well because it is purpose-built for AI like that's what we do.

14:04 >> Yeah. So what forge is doing one it's taking that loop from an experience perspective but it also ties all the the entire stack together. So how mission control and how the observability works and my inference service and my training service just all together in that one workflow and the context moves for you >> and and behind the scene by the way we talk about speed I think that there are hundreds of people that had to work on together in order to make Forge possible. Yeah, >> maybe it's an opportunity to say to all of them >> it is >> from marketing and legal and and finance and all the engineering team and the product >> that was like let's go.

14:44 >> Let's go. This is incredible. We we brought all the pieces together in Genti cloud which looks different than like a standard like cloud from before. We have specific things for that and I expect that we're listening to folks and bringing like more specific things to what they need and and I think this is incredible. >> By the way, everybody can try Forge today. >> Thank you so much. I love guests on the podcast. They're like just let's go over there. Okay, let's go over there.

15:09 everybody can try Forge today. This is new to Cor when I joined Corw. I joined the weights and bices acquisition. The one thing that all my friends kept asking me like, "Hey, Alex, what about doing GPUs? Like what's going on?" I was like, "I don't know. there is a salesperson you have to talk to etc. and then Weights and Bices we have the acquisition was like the the consumer developer product to Corore which was also new and now we're integrating this across everything and I heard you know not a rumor this was like live on the show that we're also bringing GPUs now to sandboxes offering which is also incredible.

15:41 >> I forgot to mention sandboxes. Yes, sandboxes. >> Yeah, we had Dio Filho the the PM for sandboxes and we talked about this. I find that product incredible. I think it's very important. my question to you is how do we feel about a user offering that doesn't go through a whole sales cycle and invoicing cycle and they can sign up and get an actual like testing thing in their hands like an API key very very quick. That's a new thing for us.

16:06 >> That's a new thing for us. one of the things folks who don't know when I say LF let's execute the idea is that we are focused and intentional about how we build and how we go and and I think that that focus part and of course we we learn so the focus part is we didn't want to get ahead of ourselves okay is that fair like you know in order to do that you need to have the right security controls And like there are a lot of things that we've built and also we knew what were the problems and candidly I'm actually happy we didn't have it before because the pain forge has actually only evolved in the last 12 months.

16:49 >> So we probably would have built something else and maybe we were not as ready >> but we always knew we will get there. >> Okay. Yeah. >> But we it's important for us to build great products not just throw something out there. And I'm very proud with the result because it's not not again it's not something we made up on stage. I was talking about it in the cluster max report that was published last week.

17:14 That was something that was noticed like that we have a lot of experience and data with what's happening in this space. >> So this is like I have goosebumps. It's amazing. >> I have goosebumps about this as well. I want to go and directly offer like some of their services to our to the people because you know what they hear about coreweave doing this deal with entropic and doing this deal with open and and all of the nine out of 10 we say right like we said nine out of 10 like the big labs run on core and that's incredible for the engineer that wants to test on stuff that's not something they used to be able to get they don't understand the platinum in the coremax analysis and now they will be able to they will be able to just like go approach and try via I think It's important to to call out that the cluster max report has changed.

17:59 >> It's not like it used to be the first time. So every time the criteria changes, the bar is higher. Like in this time, I think they had several vendors that actually went down a tier >> because the bar is higher. So we keep raising the bar and we keep doing that. And yes, now they can definitely everybody can access it. And okay, another thing about the future that I'm optimistic about. >> Yeah, let's go. On stage I had Brendan Hutman from Caterpillar VP of physical AI.

18:28 >> Caterpillar is a 100 plus year old company by the way. >> It's a very old like services >> and again I hope no one will be met but when we like prepped for the conversations he said that they originally tried to like partner with someone and and walk and they understood that what they really wanted to do is like to actually take their experience and monetize it like elsewhere. He was like, "Game on, we're going to do it ourselves." And that's how they partner with us.

18:57 >> They bring 100 plus year experience data >> data >> from all the machines and all the sites. They know the customer the best way. And now and okay, that was the point that I made. And what excites me is that I feel like AI can be used for problems that existed forever and we didn't have the tools to solve. Yeah. >> It can be in health, it can be in education, it can be in manufacturing.

19:26 >> So that's what excites me. You know Mike was doing today the >> Mike Trader the CEO. >> Yes. Sorry. Mike the trader the CEO of co he did the hot take with Sean. Yes. >> on stage and that was something that he was also talking about. >> Mhm. >> Those possibilities how it's going to change things and it's not changing like you know let's say my experience before with cloud native and Kubernetes. It was still a technology change. Now it's impacting any part of our lives. That's excites me.

20:06 >> I think it's absolutely exciting. the the whole thing we do on Thursday is like bring positivity about AI and I don't know if you remember last year during the offsite I stood up a couple times and as I think all of the exacts they're like like position about AI and doomerism and stuff like that I back then I really loved what I heard and love what I'm here now because I think that's very important specifically because for the past year there's a lot of change in public opinion about the specifically from you know myths about water use etc and and I think that folks need to hear this positive message and I think it's coming a little bit from the assistance world. So I mean maybe we'll finish on this and I would love to hear from you personally. first of all, how do you view this kind of switch towards the AI system, the muses, the dots that that OpenAI just released, the Grogbots, the kind of like the 24 seven assisting AI and and how that potentially like you know how we can help send this message better to people about the positivity.

21:06 >> What people are concerned about in the context of AI is valid. >> Yeah. >> Okay. people are worried about and we talk about security, we talk about my data and and and sharing that and expertise and innovation and all of those kind of things are definitely valid. But I what I I I do want to send maybe a positive note on that even before those assistants and agents is that what I'm hearing in the industries and actually true across every industry is that if before there was a lot of like you had to maybe push something into the teams and you have to use it. What's happening now even before that there's a pool meaning teams in every company they want to experiment and try and they understand that by not providing the tools no matter if it's a a caterpillar or a masterass or a Canva or they they want to use it. So that's like from a positive perspective, I think that like companies like ours can help in making AI accessible in a secure way with guard rails and relieving some of those concerns.

22:19 >> Yeah. And we will probably see that happening right always in those kind of technologies like you see things happening on the personal level as a consumer and then things are coming to the enterprise and becoming more >> it will probably happens like in few months because that's how things are going >> things are happening fastest >> so this conversation probably will not be relevant in six months >> that's why we're show hands >> but maybe it will be interesting you know what let's put a reminder for us in six months to listen to this >> episode and see what has changed >> let's see where we are I I I going to ask you a question right now. I'm going to ask you in six months. How many AI assistants do you have running right now? I know you have like human assistants. Shout out to them. Thank you for them for making this this possible, Audrey. and >> yes, >> I have some personal and some at work.

23:06 >> Do you expect the army to grow in six months significantly or are you getting to the limit of how much attention we can spread around them? Because I'm getting to my limits. I'm feeling that. Again, we we were talking about that also on stage today that the more tokens you use, the less you sleep. >> Oh my god. >> Because you feel like if you used to be a a bottleneck for people, now you're a bottleneck for your agents >> and they can keep doing things while you're not there. Not true. That's great.

23:39 I also think about you know not being wasteful and like there I have other thoughts in that regard. >> Yes, >> I hope that we will see some kind of consolidation and more efficiency also with how people are using agents versus everybody building their own all the time. >> Yeah, >> because I don't think that's the best use of humanity. So, we'll see. I I love that. And to end on a very very positive note, we were the first company to put Vera Rubin NVL 72 in production and then Cognition, which offers a software developer thing. We're the first customer. I love Devon. I don't know what your thoughts about Devon, but when I saw that, I was like, let's go. This is incredible.

24:30 >> You know what? Maybe I will say just one thing. Instead of everybody just trying to tinker everything, I think we need to help this ecosystem grow and use products and tools that people are building and where they bring their expertise. I know within our team like we we have made a lot of tools available. >> I know Devon is getting a lot of traction which is cool and silos was happy to see that and and just having that relationship is always great. So on a very positive note, Corv also speeds up that and I think the stat that he he mentioned was 4.8 improvement in the tokens that they send.

25:06 >> So this means that everybody's much >> that's compared to the Blackwell. >> Yeah. >> So the moves of Ruben it's amazing. >> Yeah. So we're now a GPU company and a CPU company and sandboxes company and services company. All of that in the past year. I cannot wait to chat with you in the next six months and see where we're at. >> It would be awesome. >> I think so. And thank you so much for coming on Thursday.

25:26 >> Thank you so much for having me. >> Thank you.