Only Hundreds of Engineers Have This Skill (Nebius CTO)

0:00 I think the promise of agents will do everything is bullshit. This is Danila Shtan, CTO of Nebius, one of the biggest AI cloud providers. You can't imagine how hard it is to be one of the first to adopt new Nvidia chips in production for a large customer. Working with an AI agent is not dissimilar to working with a junior engineer. People want Claude Code not because they want an agent to work for them. They want Claude Code because it's a hyped tool and everyone is talking about that.

0:30 Can people use agentic tooling within the hiring process? No. Very task oriented, very mechanical. This is a cohort of engineers that are at risk right now. For the people that don't know. What does nebulas do is is a cloud company. But because we've started preach late in the game. So like just for the scale four years ago, there was no nebulous three and a half years ago, there was an idea that we know what to build, and it turned out something completely different.

1:07 The current shape of being the AI oriented cloud, so mainly focused on GPUs, AI workloads, etc. and came up about like two, two and a half years ago. So we relate to the game for the cloud stuff, and we had to find a niche and ChatGPT was released. Llama was released every like everything started moving to the direction that we see now and we decided, well, this might be something we can be successful, but and we built that.

1:40 So we are the AI cloud, the cloud. But like mainly focused on AI centric workloads, mainly enabled by GPUs and having access to an enormous amount of GPUs is one of our main assets that we build upon. It's interesting that you said you're kind of late to the game, and it could be my ignorance, but doing my research, I thought you were a little bit early. Like, who are people that were earlier in the space or companies?

2:05 I mean, the general cloud business, like, oh, come on, AWS was like released their first service. And what what was it like 2003, what specifically? AI cloud. I think this is a little bit of an artificial niche, to be honest. Right. Because like while the demand for AI workloads is immense and it will only grow, but with the the whole penetration of the market and growth, it will blur.

2:36 And I think in five years we won't be talking AI clouds. It's going to be clouds again, just clouds. But even for AI specific workloads, there were a number of American companies that were way earlier than we were. Like Korea is a nice example. Lambda labs were pretty famous when we started. Crusoe is still somewhat relevant, etc. most of these companies kind of phased out of like cloud cloud business and focused more on like building large scale facilities, etc.

3:14 but they were definitely were definitely not the first. Yeah. Who are some of your customers that people will recognize? It's a hard question for me because I'm not sure which customers I'm supposed to not supposed to disclose. Okay. But we are pretty proud about the ones that we can disclose their on our website. But, I think we've had a chat with every major AI lab out there for sure. Some of those are our customers.

3:45 A lot of AI friendly enterprises, the companies that you don't really think about as like, well, this is an AI company, actually have a lot of AI going in the background, and they are also the customers. So I think there are some logos that everyone will recognize for sure. It feels like a very interesting space. Also niche right now, but maybe it's going to normalize later on for the engineers that want to go into this field because they love infrastructure, they love AI might be a good match.

4:17 What do you typically look for in the capabilities that they bring to the table? Well, there are several streams, right? So first there is like the general infrastructure engineer track, right. So anyone who has knack for infrastructure, who in the stands the like, the responsibility and the scale and distributed systems is always welcome to the team. So we just hire those people on spot. We don't we don't require any prior AI experience be in the cloud and especially the cloud on the frontier of the hardware.

4:55 Like you can't imagine how hard it is to be like to be one of the first to adopt new and video chips in production for a large customer. This comes with with with the toil. So naturally, people who are deep into like systems programing, figuring out what's wrong with the drivers on the hardware level, etc. also in great demand, low level storage people because again, AI workloads, it's not just about GPUs, right?

5:27 It's about immense demand for performance throughout the stack, networking and storage included. Like we routinely have customers who, I don't know, download around like hundreds of gigabytes per second of data sets while training. Like for that training grounds, you need infrastructure and services to sustain that. So that comes and comes in handy. And with the recent developments link and our newly found focus on inference, also, everyone who knows how to write kernels program GPUs at a lower level is also like in high demand.

6:03 And those I think one of the most well-paid and in high demand people in the market nowadays, because we definitely see the shift from training to inference, as in running models and in in order to make inference really like well behaving and performance and everything, you need to change a lot at the lowest level of how those models are run on GPUs. And I think there are so hundreds of people in the top cohort out there in the world.

6:41 Yeah. Not more. Can you walk me through your hiring process so someone is excited about this, wants to apply to Nevis. Do you have the same hiring process depending on teams or how do you typically have that? It depends on on the person, the background and what we are talking about. But in general, our hiring process is more or less standard for for everyone, for all like venues, it might have like specialized interviews for like if you're talking about some deep expertise.

7:12 But in general, we have a lot of coding sessions. We have written sessions for everyone always return to have systems design, for example. But again, it's optional because, for example, for a low level GPU programmer that has spent years writing like C code, probably systems designed for distributed systems is not the best way to assess them, right? But in general, we have some some specialized sections. Then after those interviews, we if it's not a very unique case for the expertise, like someone building network networking stacks for the past ten years probably will lean towards like the relative like to to the relevant team, right?

7:59 But if it's not usually don't choose teams at the very beginning like we make sure the interviews are past, we extend an offer, but it's an offer to join the company, not the particular team. And then we get into this. We call this process bootcamp. Endeavor to find a proper team for the person to find the match, because I strongly believe like, well, hard skills and expertise matters. Soft skills and in general compatibility with the people around you matters way more for productivity for like work life balance if you will.

8:36 Right? So usually people have several teams they are supposed to join in the first three months. So they spend like 4 to 3, 3 to 4 weeks per team. And in general they are like just brought up like onboarded. And I included in day to day is like your daily meetings, talking to product managers, project managers around like going through daily stand ups, for example. So the person like goes through this process for three weeks, changes teams several times and at the end have kind of an exit interview.

9:15 So we like reflect on what has happened during these three months. How was the one boarding structured? That's my personal issue. I'm like, I'm always talking like. And by the way, this interviews are always conducted by me personally. So and that's like what I pay attention a lot. And how efficient was the onboarding. Like how much did the person spent in vain etc.. So internal metrics of sorts. Yeah, the importing should be fast, like you need to be productive, you know, did one day two if it's a week.

9:49 It's like the personal spent three weeks on your team. Yeah. Like yeah. We ask the person which team would they like to join. And we ask the team if they would like this person to join their team. And if there is a match, so be it. Has there ever not been a match? And what happens then? It depends on how the bootcamp went on. In general, if there is like a serious mismatch, for example, a person thinks that the matches with the team that is like was like gave a definitely negative feedback.

10:25 We would treat it as probation period failed sometimes. Well, if it's less than that, like there is some hesitation, etc. we usually discuss it very openly that the feedback was like that. There's definitely a mismatch. There is like a like, you know, second team in your preference. What you consider that we do understand that this is not the perfect situation, etc., etc., etc. but in general that's that's rare case. So we are usually talking about exceptions rather than a rule.

10:55 People generally tend to find the team they are comfortable with. Yeah, let's break it down for hiring algorithms. System design can people use a genetic tooling within the hiring process? No. Why is that? The honest answer is that we don't understand how to treat that and how to work through that process. On the contrary, we actually like, well, you know, like when pandemic started, Covid, etc., everyone went remote, all the interviews went remote. And we actually now started like the backwards process.

11:29 And we bring people sometimes from another country paying for their plane tickets and all bringing them back on site to conduct onsite interviews face to face. Remember five years ago that was a thing. Yeah, I had that. Yeah, I think everyone had that right. But now, like, especially for select people who we see more senior and more interested in, like getting to know we bring them into the office to connect, to conduct an onsite interview and the whole during the interviews.

12:03 It's it's a mess because for our process, that would mean that we've gone through a lot of investment and maybe we don't get the person that we really interviewed like because like algorithms and coding is, you know, there is an argument that algorithmic sessions are bullshit. Like no one writes that code for their production use cases. And that's true in a sense. But this is a very good baseline to understand whether the person's computer science like background or like knowledge in general is up to a certain standard or not.

12:44 And we think this is quite a strong signal of how what will the quality of the work of the person be in the future if they can write like a simple algorithm or not like a purely algorithmic task. So adding noise to that with like, you know, this, like this model interview cheating tools are crazy, right? Like they're really good. No, nothing. Like, you can't even by looking at the person determine if they are, like, having assistance or not.

13:18 And like I think that's a problem and I really don't like it. So we are designing a special agent coding session when like we would assess how a person interacts with an agent. What are they doing? How are they steering the loop? Like what what's the task? How do they decompose that? How do I how do they think if if that works or not. But it's not on the menu just now. But AI assisted interviews don't like it at all.

13:48 Gotcha. Then zooming in on the boot camp. I love this idea of just hiring people that are really, really good, right? If they're not very, very specialized. Because for those specialized people, you probably have specialized teams, that is just the match. But for the people that can kind of go anywhere and go can go in and out of teams. I feel like very senior people are very capable of going into a team solving problems and sometimes overarching teams and solving problems across teams.

14:17 So I like that this is part of your onboarding, part of a matchmaking. And it's not just for very senior people because people who are less experienced, I feel like they actually get more value out of the process because they don't really know what they want to do, and they don't really know what will defeat be sorry. Nor is and like being able to look at different teams, being able to get more context of what's going on in the company in general is a very good thing for them, because more senior people, they're just, okay, I'll figure it out and they usually do less senior people.

14:58 They require some some guidance there. Right. And this process is essentially designed because like all the teams work in collaboration in general. Right. So for example there is like it's not a rare case when a person is designing some kind of a feature in one team changes team within the bootcamp, the next team, they actually on the team that is like a client of that feature. So they immediately see the value. They can reflect on what was done, how the interaction was structured, etc., etc., etc.

15:32 they passed the team like, okay, next team has nothing to do with that. And there are like for example, they are actually on the dependency side of the first feature, like an all the struggles that they had while implementing, they can see why. Why did that happen? And they can think of how can that be amended or fixed, etc. etc.. So in general, understanding more about the team's internal stuff, being more transparent to each other. It helps and I think it sets the right tone from the very beginning, very open to you.

16:01 We're very transparent about what what's going on. We talk about our problems and we show our problems during the board camp. Like a person can see the whole cycle. Do you then specifically look at teams that have availability or they just hire really good engineers all the time? Yeah, we we never stop hiring. We're always pushing down the team to bring more people around. And our interview is somewhat like rigorous and it might be might seem like an overkill.

16:31 Sounds tough. Yeah. We don't hire a lot, right? I think it's usually like 10 to 15 engineers a month. It's rarely more. I would like to have more, but we don't. And then we don't like, like I spend like probably half a year fighting at or like at an earlier stage of the company fighting with my team leads and people who report to me, forbidding them to use the word hat, like there is no headcount in our company.

17:05 They don't talk headcount. They talk demands. We talk tasks, we talk priorities. We talk size of the backlog. We don't talk headcount. So we don't have the concept of the head count so everyone can hire if they see like, it's something they need. We have like a little internal system that would balance like, the amount of boot camp, for example, the team gets. But they have a choice. They have a save who comes like, so they can preview the CVS, the interview results, see if that would be a potential match and only like ask or get in line for very particular people.

17:45 And yeah, so we just hire and it's always like but what what if we hire like ten people and don't need ten people? Denser is usually let's see when you get there. If there is an issue and I see there is an issue, or someone will have a chat and we'll figure it out. Yeah. So far we didn't have an issue. Nice. Yeah. You and I spoke before the show and you mentioned your engineering population is about 300 people.

18:10 It's closing 400 now. Closing 400? That's only in Amsterdam. Bass. No majorities in Amsterdam. Well, in in the Netherlands. But Netherlands is not like a large country. So everyone can come to an office once in a while. Majority is in Amsterdam. We have like a healthy presence in London and we're actually focusing on on London a lot these days. Have, I think, ten ish percent spread remotely around Europe, everywhere, like from Portugal to Baltic countries, from Finland to Italy.

18:45 Yeah, right. And we have some presence in Israel. We have some presence in the US. Yeah. That's it. But you've spoken to kind of all of the hires when they come on from a boot camp perspective. Bootcamp, yes, I like I've spoken to all of the hires, especially in the early stage of the company. Right now, people who go directly to teams like highly specialized. I usually don't like a proper interaction and recently been on on an M&A spree, so definitely not had a chance to talk to everyone or joined with like with the quiet companies.

19:25 Gotcha. Yeah, I feel like it's fascinating to have this kind of onboarding process where people join multiple teams and then gain perspectives. It sounds like a lot of fun, like coming into a company like and like I actually think everyone means it's a it's a clear win win situation. Like, it's not a win win for everyone because sometimes people tend to be tired or it's like, oh, it's a burden, right? To change context every three weeks, especially if you're like, feel like it's a challenge to get on board, etc..

19:56 And there is that aspect. But in general, I think it's a win win. Like it's a win for an employee who gets to see what's going on around, which is extremely important. It's been for employee that can actually match the people who they actually vibe with, because it's very hard to understand whether you'll be whether that's going to be like a good match during final interviews or like just reading about the team.

20:26 Actually, you need to see the people. Of course you need to see how they talk. What's what's the mood around the office? What is what are the means that they post into internal means channel? Because like I'm telling you from experience, not everyone is compatible with the same memes. Like of course, definitely. It's very different. Tasteful. Yeah. And like, what's the attitude from like from the beers from, from, from from product managers is like, do we like it or not?

20:54 And that's that's a big deal. And for the company first it's well like relaxed and happy employees is always good like productive. And teams become very productive because like they actually find people who click together very well. It's a good thing for the company because for example, team leads see this constant influx of people. They see new people, so they are more relaxed about like talking about performance, for example.

21:27 Right. There is no commotion of, oh, this person is like not really. Well, like not a good fit for my team or underperforming, but I will just hold on to them because if if I get rid of them, who's going to work? I lose my head count. Yeah, lose my head coach. Sorry. Yeah. So like and team is usually like way more relaxed because they see the constant influx of people. Like we are constantly hiring people or constantly cruising the bootcamp like like like what's like 40 to 50 people like like maybe 20 to 30 people always like in the loop.

22:04 Everyone sees that they're there, etc.. So in general, I think this this works really well for us. It becomes a burden for me personally sometimes because like you, there are days when I have like 3 or 4 like this bootcamp and meetings, but I still maintain it. Yeah, very interesting that from a title perspective. And this was also me a few years back. I've spoken to more and more CTOs now, but this seems like I would expect CTOs to be very much technology focused, and I feel like you put a lot of pride in kind of team dynamics, engineering culture across the board.

22:42 Also, in your involvement in these bootcamp sessions, I'm not sure CTO is a technical role, at least at some scale. Well, it might be for the smaller scale and startup that like everyone is doing everything in. CTO is just a guy who like, loves tech more than anything else. But in a sense, after a certain scale, it's a it's a management position. It's less about tech, it's more about building the team, making sure delivery is there, making sure there is no conflict between engineering and product function, engineering and business function, which trust me, there always is stuff to like to quarrel about, to argue about.

23:27 It's more about like structuring the hiring and boarding process, etc.. Of course you need to be a techie. You need to be able to assess what's going on. You need to be able to dive into projects. You need to be able to like dive deep if needed. But it's not your day to day. It's not. It's more about management. It's management and the technical organization. And I think the larger the scale is, the less tech there is.

24:00 What are some of the what are some of the skills or capabilities that makes a CTO successful? Yeah. Good people person a good manager like people manager. Being flexible enough like I usually like describe the role as you need to be an advocate for the engineering team in front of business function, for example. And at the same time, you need to be the advocate for the business function in front of your engineers, like you need to bring.

24:33 Like a senator checks to both sides of the table and you need to be like this, you know, constantly in the middle person who reflects on both sides of the equation and make sure that everything is balanced. I think, well, it's less about the CTO role, but I think it's more about any manager in the modern company. One of the main skills is you should always be managing expectations of people around you.

25:03 Like I, I think this is the main thing a modern manager is doing, and every failure I've seen usually comes from not managing expectations, right? Even if you are not capable of delivering, if you're not capable of doing something properly, if you're not, if you manage the expectations right, the the result will be way better than if you just close up and like struggle on your own and don't tell everyone like you have problems, etc.

25:38 etc. etc. so being in the middle, being able to talk to everyone, accept their point of view, internalize that and manage expectations. Why do people fail sometimes to manage expectations? Because it seems so simple when you lay it out like that. Because people like people. Pride. Fear of seeming incompetent, like ego, ego or all pride. Ego.

26:09 Yeah, like I can't do that. I should be able to do it. Or I promised, like in general, fear of communication, of open communication. What will they think about me if I tell them I don't deliver imposter syndrome? Of course, here we are. But in general, like, it's hard to admit failure. And well, in a sense it shouldn't. Like nothing's going to happen. Like, if you're open about your achievements and failures, people who are actually I think they appreciate it more because no one likes people who are bragging about their achievements.

26:50 Right? And like discounting their failures. No one likes a pickle who, like, bring their families as price because like, it's not about just managing expectations. You need to remember that everything that you've said out loud, someone heard, internalized, and probably have plans based on what you've said or committed to. And there is a lot of stuff in motion that you are not aware about that might have, like a long chain of failures.

27:22 If you fail to deliver or fail to manage expectations of people around you, and you can't formalize that, you can say, like, I didn't write an okay in that particular ticket. Hence you shouldn't have acted upon that as I did. If you had a conversation and you've said it out loud, you're going to do it. It doesn't matter whether you said probably, it doesn't matter if you said, I'm uncertain. It doesn't even matter if you've directly, outright said, I will do it, but don't count on it, because the second part, no one will remember.

27:55 They will only remember the I will do it part. So you need to be very cautious about you. And it's not cautious as in don't commit to anything and don't talk about anything. Talk. It's a good thing to give people indications etc. just don't forget what you've said to them and go and like adjust the expectations, etc.. Gotcha. Yeah. I want to shift gears and talk a little bit about a genetic engineering. Some of the yeah, yeah, I feel like some of the bigger organizations are really questioning their engineering capabilities, distributing tools, trying to figure out how to make their engineering work more effective, because that's the promise.

28:37 How do you view a genetic engineering currently? Oh, it's a constant source of like internal arguments, especially with our CEO who's like very bullish on hype. Yeah. Why why, why isn't like 90% of our code written by AI. Like yeah. Because I won't wake at 3 a.m. like when there is an incident production. But okay. Might have come over on foot. In general, I think it's very weird to to say that it's not happening.

29:11 It is like 100%. It can definitely see it. I think the inflection point was roughly this winter when what was it? I think entropic 4.6 and GPT is 5.4. Models were released which like really raised the bar and with more than like 4.7, 4.8, GPT 5.5, these are like these are extremely capable models. And with like with the right genetic loop harness and with the right tools, it's like it is capable of serious engineering work.

29:49 I think the promise of agents will do everything is bullshit. Not at this level, not right now. Not in the near future. Everyone who talks about software factories like it's like new term. That bothers me immensely. Dark factory. Yeah. Okay. Agent will look at your production. Fixed bug, write tests, tests, everything. And yeah, everyone who had an experience on driving a self-fulfilling agent loop in software engineering knows that it doesn't work like that.

30:25 It never does. So I think we are we far from, like completely autonomous work. And of course, there might be examples of people in choosing that, but it's like up to a certain scale, up to a certain like business scale, for example, etc.. And I think the most successful examples I usually have, like a human driving the whole thing. Right. So I think I agents and like a Genting software engineering is here to stay for sure.

30:59 I just don't think that the hype that we see out there is. Is true. It's not that it's it's a tool. It's just another tool. Like there was a hype about like, I don't know, what was it. What was like, you know, at some point, for example, if you remember, in the C community, there was like a huge hype about their sharper when JetBrains released their tool and it was like, yeah, this is like, this is going to change everything forever.

31:34 We're going to be so much more productive. Sure, it's just a tool now. Like like of course no one writes in the notepad plus plus anymore, right? Yeah, that is true. But, I think we're going to like, all the systems will mature and it will just become a commodity just at all. But in general, I think like coding is a very powerful tool. Like when people say that like multiplication of like your performance or your productivity, that might be true.

32:08 But also I think the baseline, the zero. And like in your skill that is going to be multiplied. It's not the like you're not an engineer. It's way harder. It's like somewhere along your mid-level to senior engineer. And we all know what happens if you multiply a negative number. Right. So zero is up there. Everything lower is negative catastrophic. Yeah. And it's going to be multiplied. So like that's where we see I've created a startup with cloud code in the matter of like one hour. Oh.

32:45 And like Sheckley like it gets like whatever subscriptions, like, oh, everything is white. Bye bye. Hacker. Why did that happen? Well, it happened because there was no sane engineer present in this loop. And in general, what? Also not that good yet at like doing large context, having intrinsic knowledge. They don't like everything you see in the loop is like the model that like it can't invent anything.

33:18 No, like pre-training loop and the general knowledge and the model will let it actually solve your task. So everything it reads, it reads right there in the repository or what you prompt it, etc. and if you don't, it will not think about that. It will just implement it as is and tell you everything is working fine and it will be working fine. And like that's one of the parts that bother me about the software factories, like people who write articles about.

33:49 I just recently read one they managed to claim in the same sentence, essentially that, software autonomous software engineering is a great area because it's formally verified. You either pass the tests or not. Right. And the same sentence, they say, and the models are very capable of writing the tests. So if your model writes the implementation and writes the test and everything is green, how sure are you that it actually done that it's actually done a good job.

34:20 Now, there are like theories that it can be addressed by a separate model loop right in text and specs. But again they converge very fast, like they literally read everything that is in the repository, the recommendation, etc. as the ground truth, because they need ground truth to start working and without like external challenge, without external questions, etc. you won't get anywhere. It's just self-contained information. Yeah, isolated. And it's like boils down to like, very inefficient loops.

34:53 But that's why I think, like from my personal experience, like working with an AI agent, doing stuff and actually do a lot of stuff these days, like I think for the likes of me, this whole agenda AI revolution is like, it's great. Fantastic. Yeah. Because like, I can drop my ideas into code immediately, have prototypes instead of like gathering sessions with developers, trying to work through the issues, planning, etc., etc., etc.

35:26 and then just present the MVP and say like here it works, let's do a product out of this. But working with an AI agent is not dissimilar to working with a junior engineer. They don't have the full context, they sometimes have weird ideas and they like managed to like talk themselves into, like exploring these ideas and building on top of that, they require constant challenge from the outside, like showing the outside perspective, like they require like working with that requires like a decent internal bullshit meter, right?

36:05 When you read the statement from a model and you're like, wait a second, this shouldn't be there. This is not how it's supposed to work. Like asking questions that would imply verification from something that is currently not in the context, remembering what might be in the context of the model. Same as it like what what what? What was the way of thinking of this developer that reached out this conclusion? Right. So this is very similar. And I think this whole like actually it's not the software engineers enabler, it's the software engineering managers enable way more than like and even if you see engineers quite actually good and like increase their productivity with agents a lot, I think it's because these individuals would be a very good engineering managers.

36:56 Interesting sense. Yeah. Do people in the organization have access to a genetic tooling, like have you widely distributed that have you only given it to senior engineers? We have access not to everything. For example, we have access to OpenAI models and Codex, for example, and everyone can build their own harness. With GPT, we don't give widespread access to cloud code. And for me, the distinction is exactly the hype.

37:26 Like people want cloud code, not because they want an agent to work for them or not because they know what to do with it. They want code because it's a hype tool and everyone's talking about that. And this for me is an issue. We need more guardrails. We need more observability there in a sense. Like I think I've recently posted it on internal slack, like a manifesto of like, when will we get cloud code? Like, I think we need to reach a state in our tooling when for every pull request that is supposed to be merged into a production branch, you need to have a clear distinction whether this was written by a human or assisted by an agent, and if it was assisted by an agent, you need to have a full trajectory.

38:19 So the full session or sessions that led to this result. Connected, open for audit, etc., etc., etc. because we are shifting in a way from like actually writing the code. And I'm saying like if humans have written the code, you can hope to have a conversation with them about it. If the agent have written the code, you cannot have an honest conversation with the human. All in the extreme cases when like you should not commit the code that you don't understand, which you can say, but no one will do it anyway, right?

38:58 But it means you need to shift from like discussing code to discussing the way of thinking about the code. Like what did you think when you were like designing this approach, right? What was your what was the reference architecture that you had in mind? How did you drive the model? Did you challenge it enough or did you just yolo it with one prompt, etc., etc.. And this this becomes an important question, so I'd be very hesitant.

39:26 I am very hesitant and very conservative in this to like, allow the widespread adoption of an overly hyped tooling. If I don't have enough curtails and I don't have enough guard as of now, and we are building it internally right now, but we're not there yet. At the same time, like, I think the most capable sort of engineering model as of today is OpenAI is when they constantly change a little bit, like with the new releases going back and forth, and everyone who actually wants to build with, with an agent can do that right now.

40:06 And like they do, it's not just less hype, more practical work. Interesting. I feel like some of the specification frameworks are trying to solve the challenge that you mentioned. When there is a code change, what is the context that went into that? But it's not addressing kind of the whole history of the conversation and how that specification came to be. I feel like we need more insights in intent and history on how a change came to be.

40:34 Yeah, in a sense, like it circles back, to be honest, to one of the interviews, we have like four engineers that are higher than our basic grade. Now, basic rate is not junior people, it's higher than that. But if we consider a person to be more senior than that, one of the interviews we conduct is. Technical due diligence. It's not structured in any way, but it's like it's a free form chat about a recent project that you've been involved in.

41:09 And we don't assess architecture, for example, or the result of whatever happened, like we have separate interviews for that. What we assess is general understanding of what was going on during the project. How were the requirements gathered? Why did you choose this way? Not in architecture way, but like who was the decision maker? How was this decision adopted? Right? Why were building it at all? What was the purpose of the product?

41:41 What do you know about clients like users? How did they adopt? Like did you get any feedback from that? How was it internalized within the team, etc. etc., etc. and this is the interview that we conduct for every engineer essentially who is like higher than our baseline because this is the I think this is one of the more important qualities nowadays, and it looks nicely into a generic workloads. Right. The person who actually tries to think out of the box outside of this particular task, thinking about the system in general, decision making process in general, like what's going to happen with the product in general, updates released after it's being used tend to be a better engineer and more productive engineer.

42:30 Bring brings more value to the company. In sense. This is like remember there was like the whole DevOps commotion, right? Developers now everyone is bringing together like to do like the idea was that you don't need to just write code. You need to think about the system as a whole. I think we are currently in the next step with this whole revolution. You need to think about not just the system as a whole, but a product as a whole.

42:59 Implementation details, architecture, operations, how it's run like whether it breaks some not who is in duty today and cetera and then the product thinking, right, why are we even doing this? Do we need to challenge the product team that brought this new feature? Why? Like how will that loop into the rest of the product or the system, etc.? I think we are like, remember, like T-shaped people, etc. we are going to the round shape.

43:30 People like you need to be a little bit of everything you need to be, but in a sense, you just need to be like a generally smart and sensible person. And that's it. Yeah, I feel like it's easy to say. And for me, I always had kind of innate curiosity. So I'm just interested about what am I building, but also for whom and what problem I solving and why. And if that doesn't make sense, then I shouldn't do it.

43:55 But I should focus on something else, like it was always there, rather than focusing on how I built things. I think that's where it started from me growing into junior engineer. It was very much a problem. How do I do this now? And when I got more comfortable it was, well, shall I do it in the first place? Because I now know how to do it. Like that's not interesting anymore. And I feel like I quite quickly pivoted to a point where I could definitely do with more depth in terms of hard skills, but I just feel like my curiosity drove me towards the more overarching thought.

44:30 In your in your perspective from your advice engineers nowadays, do they focus more on kind of the what, why and how or more on their hard skills? Because I feel like hard skills are partially being automated in terms of generation. It will become obsolete in a sense. I recently had like a chat with a number of students and they were like, but like, okay, we are in the university right now, like finishing our computer science degrees, etc..

45:00 How do we get into that? Like like what is our career path nowadays? Previously it was obvious you go to in turn position junior position. You sit alongside the grandmasters and you learn how to do stuff and you acquire this hard skills, etc. etc. etc. but now with with agents doing the junior work anyway, like what is the what is, what is the place for junior engineers anymore? And I think this is a misconception in a sense, because like people who have a knack for that, like smart people, curious people, like people that can who are good at decomposition.

45:43 Right? Like figuring out what are the parts that combine as a whole. They are good with agents and they will they will still find a place and they will be great. And it's like, yes, of course you need more experience and you need like this baggage, right? Well, you're growing, but it's it's important to acquire. But it's not about hard skills anymore. No one cares about the framework. No one cares about a particular library.

46:13 No one cares if you know all the instruction of an 86 processor. It's a good thing to have because like implies you are. You have some knowledge and some package, but it's it's not requirement anymore. It's not how you measure people anymore. And I think like we will still have people who are digging deep and for bringing the frontier, it's just not a requirement. It's a specialization. It's not a requirement anymore. But like thinking about what's going on, how to structure everything, being able to produce a concise flow of thought, which is an extremely hard thing if you think about it.

46:57 Right. People are not generally good at that. Yeah. This is what becomes like of value. And yes, this will change how we view engineers and this will probably change. The composition of these teams. And some people who who have like a great career will phase out. Right. Because like previously you could go I don't want to sound derogatory, but there is a special cohort of engineers that I think will will vanish.

47:30 For example, like people who sit in banks and automate like internal processes in Java. You know, there's banking Java developers like this is something that this is a cohort of engineers that are at risk right now because the like very task reentered, very mechanical. These are the organizations that like we are not even talking about like this whole common sense, let's build a product thinking organization.

48:02 They're not even like they're not even thinking about operations because there are separate operations teams. This kind of engineers will will become obsolete. And, you know, 3 to 5 years I feel. So there will still be inertia, but there will be no new influx of new people into those teams. There will be no influx of people who will be able to to think of that as a as a career path. What happens to the systems they are responsible for?

48:31 If no influx of people come out, they will modernize it. Yeah, in a sense, I think people are definitely worse than even more than agents of figuring out of what's going on in the. Complex code base. Way worse. It's just not how we build. So the only thing that is there is this, you know, intrinsic knowledge, something that's not written anywhere, that's only exists in someone's head.

49:05 Yeah, but it's not about the cohort of engineers. It's about particular people in particular skill set in particular environment and particular engineering manager who was okay with the bus factor of one and never pushed for knowledge exchange, etc., etc., etc.. So it's like it's less of an engineering problem, more of an organizational problem, and for sure, like having a job security by not writing documentation around for 40 years now, at the very least.

49:36 Right. And this will still be a thing. Yeah, for sure. But we're not talking about that. About talking about engineering. Yeah, I maybe it's because I'm, I see myself still as a little bit young or eager, but I don't know how people get into a position like that where they are just ingrained in a small cog in a big organization like bank or insurance, very small moving or slow moving behemoths. It's stable.

50:08 That might be that, yeah, it's stable. It provides decent benefits, way better work life balance than working for a company like ours, for example, like we run a full like responsibility cycle. You build it, you own it. Like you have freedom to design stuff at will. Don't have like architecture committees, boards. No, like nothing like that. No. Architects as a separate position.

50:38 Right. You're an engineering team. Figure it out. But you build it, you run it, you run it. You're responsible for it. You're responsible for it. You're on call for it. Even if it's like Christmas night, 3 a.m. it doesn't matter. Someone from the team is on call and is supposed to react if something fails. Good work life balance? Probably not. Enough people who are willing to engage in this kind of search for sure. Absolutely, yeah.

51:09 What if the agents write the code? That kind of whole you build it, you own it, you run it, you're responsible. I feel like that's why. That's why we are not there yet. And I don't think we should be like, again, like, I think at this day and age, committing production code into, like, the product that runs multiple hundred millions of, like yearly revenue should be owned by human.

51:41 It's non-negotiable. We are not there yet in terms of agent AI capability. You can't rely on it. It's just like, well, you know, it's immune by now. Like, like. Telling you, oh, you are totally right. I made a mistake. Like, I've seen it too many times. Absolutely right. So we still like human driven and human responsible. You can augment humans.

52:12 You can help, like, increase their performance or automate suffered some of the processes. For example, we have a lot of agitated workloads internally. It's just not about software development, right. For example, if there is an incident, production agents are actually pretty capable and good at finding the current root cause or coming up with an immediate mitigation plan simply because they are faster and this is more or less structured environment. Right? You have like a condition, you have your metrics, you have the logs, you have Kubernetes that you can explore and human can do that, but it will take them like 20 minutes.

52:50 Yeah, an agent can do that in three. This is a good thing right. Or when we are talking about like cross-checking, cross-referencing stuff, it's also like something that the agent can do. Great days, for example, when you're writing a postmortem, producing a postmortem by an agent, in my experience, is a shit show doesn't work. But checking and cross-referencing and grounding the facts is something the agent can do very well.

53:20 Like, we have agents right now who can look at the document, go to slack monitoring like Kubernetes, and fact check every statement in the document, whether it's grounded, where it's like, was it like that? Correct. If it's not, provide feedback on like for like and for root cause analysis, for example. This is a great thing because people can like not notice something or they can skip them something. The agent bones because they don't have a modicum of like trying to conserve the energy.

53:53 There's no ego there. Yeah, it's less about you. Like human brain is optimized to conserve energy. They tend to not do stuff diligently, like it requires a lot of build power to do stuff diligently. And of course there is like a asleep here and there always will be. It's just part of human nature. So this is where you can get the slope of like grounding and making sure that everything works, etc.. So again, augmenting people, not getting rid of people.

54:26 Gotcha. One of my last thoughts was, I think a combination of what we discussed so far because we mentioned from a focus standpoint, if you're new to the field or already kind of a little bit more early in career, things are being automated in terms of capabilities. So focusing on hard skills versus some of the other skills that you also need might be better to focus on the other skills more the soft skills more on the Y and the product thinking.

54:50 But you very specifically mentioned that algorithms are still part of your hiring flow. Like, if I would want to join right now, I would fail specifically. Probably for those algorithms. It's something that I would have to train and I would probably not really like it. Enjoying it, kind of memorizing the algorithms. And I do think it might be good practice, but I wonder why that is still a thing. And when are we going to get rid of algorithms within an interview process?

55:18 I don't know, my answer is like for us is when we find another way to provide a good baseline. So algorithms being able it's not about memorizing to be honest. It's it's about like, yes, it is about some patterns that you need to know and approach. But it's it's rarely like tasks that are really about like remembering a particular algorithm and implementing that.

55:49 Right. It's more about thinking algorithmically, knowing when you can cut corners and when you can't, knowing about like a broader subset of low level primitives, etc.. And. For me, for us, it's it's separating like people who are into tech from people who are actually engineers. I'm sorry to, to frame it like that. Right. Because, we are still like sitting back like you need to have this internal, like, understanding of what can go wrong.

56:29 And you can either have that with experience or with education. Right. And I still think that for a lot of computer systems this is an important part, right? You need to be able to think about that. Data structures, agents won't be able to solve it for for some time, the very least the whole optimization cycle. Like you know what the what an optimization cycle with agents is just go around 100 of theories and maybe one of it will like genetic algorithm if you will.

57:04 Yeah. It's not the best approach to be honest. It's prone to like, finding the local maximum and never exploring further. It's. It's this approach is not what made all the greatest inventions of human history, right? No one have invented anything new just by digging and trying multiple times throughout the same area. So again, like circling back to algorithms, it's still just a a signal whether the person is actually has like some computer science education or education is I use the word education as an umbrella term, right.

57:54 It's not about the university or a particular course. It's whether you have this understanding, erudition, if you want, about how do the computers actually work, how does the software actually work? Because this is important, because not all software is just like taking JSON from this API and sending it to another API. It's like there is a part of the market and a huge one, which is exactly that. We are in a slightly different game.

58:25 We build the infrastructure for those who do that, and hence we we think it's still important for us to to have as a certain level of people who are into certain things. And thank you. Thanks so much for coming on and sharing. This is great. Thank you.