Section Insights
Pim Dit's Journey in Gaming and Entrepreneurship
How did Pim Dit's experience in gaming lead to his entrepreneurial success?
Pim Dit built the largest private Runescape server by age 18, generating significant revenue. His gaming experience laid the foundation for his current venture, General Intuition, which focuses on AI for gaming.
- Pim Dit achieved early entrepreneurial success through gaming.
- His company, General Intuition, leverages AI to analyze gaming clips.
- Gaming skills can translate into business acumen.
Importance of Mapping and Data in Crisis Response
Why is mapping and data labeling crucial for disaster response?
Accurate mapping and labeled data are essential for effective disaster response and economic calculations, as areas not on the map can be overlooked in emergencies.
- Mapping is vital for disaster response and economic viability.
- Open-source data initiatives can enhance global mapping efforts.
- Inaccessible areas can hinder effective crisis management.
The Role of World Models in AI Development
What are world models and why are they important in AI?
World models help AI systems understand and predict actions in both digital and real environments, ensuring safe and effective behavior in real-world applications.
- World models guide AI behavior in various environments.
- Safety in AI is paramount, especially in real-world applications.
- AI must differentiate between digital and real-world moral implications.
Challenges and Opportunities in Robotics
What are the current challenges and opportunities in robotics?
While humanoid robots face challenges in home environments, they are more viable in factory settings. Specialized robots are often preferred for efficiency and cost-effectiveness.
- Humanoid robots are more suitable for factories than homes.
- Specialized robots can save energy and space.
- Market demand influences the design and functionality of robots.
The Future of AI and Robotics
Is the current interest in AI and robotics a bubble or a sustainable trend?
While there are concerns about AI being a bubble, advancements in technology and practical applications suggest a sustainable future, despite challenges in achieving full potential.
- AI and robotics are likely to continue evolving despite skepticism.
- Real-world applications are crucial for validating AI advancements.
- The last stages of development often present significant challenges.
Transcript
0:00 On Runescape, I built the largest private server in the world. So that's the only one where measurably I think did pretty well. >> Is there money in that? >> Yeah. Company generated about a million and a half a year in revenue by the time I was 18 years old. >> You're saying there are more car crashes in the video games than in the real world? >> We've counted. The amount of clips uploaded with car crash labels surpasses the amount of physical car crashes that happen in the United States every day.
0:21 >> Are there games you're prioritizing to show off your capabilities? >> I think we'll leave a bit of a surprise. At 18, Pim Dit was running the biggest Runescape private server in the world. Today, he runs Metal, the app gamers use to clip their best moments. They upload about a billion clips a year, and every clip records the exact buttons that the player pressed. That's what his nude company, General Intuition, trains AI on. The result is a model that reads a screen and works out what to do next, and that skill transfers straight to robots. Him says his AI already beats him at almost every video game. Almost.
0:56 Not Rocket League. This is my conversation with Pim Dit, CEO of General Intuition. Pim Dit, CEO of General Intuition. Thank you so much for joining the newcomer podcast. >> Thank you for having me. >> We have so much to talk about. We'll get into some of the big questions of artificial intelligence progress. You're building in what might be, you know, one of the buzziest categories in Silicon Valley right now, world models, but I thought I'd start off in more fun, light-hearted territory. You were at the top of your field in Runescape and and Rocket League, I believe, and and they sort of helped you get into entrepreneurship, right? So yeah, tell us about your Runescape days and are they all behind you?
1:49 >> Yeah. Runescape no I still play both games occasionally actually. So I got so good at Rocket League that I can kind of tell how my mental state is by how I play and whether I should be making important decisions that day. So if I if I am on it and sharp, I can see that I'm playing really well. I can essentially measure my my level of decision- making by like if I play like a few games in the morning, I I kind of know whether or not it's like a good day to make big decisions.
2:17 >> And do you do that? You like, I have a board meeting today. Let's play some Rocket League and see if I'm on on >> Admittedly. Admittedly recently, no. but you know, I think you know, in the past year, you know, I I probably get 10% of the games in before I started the company. but I know on Runescape I built the largest private server. So in Rocket League I was I got up to I got up to GC1. So definitely not the top of the fields but it's it's a it's a good place to be.
2:45 >> You know what top percent what percentage? >> I don't know I'm if I guess so it's double. So I'm guessing top 5% or something like that. I don't know what the today's numbers are. on Runescape I built the largest private server in the world. So that's the only one where measurably I think did pretty well. >> Is there money in that? >> Yeah. Yeah. company generated about a million and a half a year in revenue by the time I was 18 years old. and we owned about 60% of the private server market on Runescape.
3:12 >> Yeah. >> Is that still running? >> No. shut it down when the company behind Runescape got sold I believe in 2015. It got sold to a Chinese mining company at some point and >> like physical mining or crypto mining? >> Yeah, there no physical mining. Yeah, there was some there was some program at the time where the government was encouraging nontechnical companies to go buy overseas technical companies and I believe they somehow ended up buying the creators of Runescape. and then yeah, as a result we had to we had to shut our servers down >> because they made everything private or >> yeah, I think they they were more strict on private servers than the previous ownership.
3:49 >> Did you you know, were you one of these people like harvesting in-game items and selling them to you? So you could so in the private server so I built a private server which basically means that you host and distribute your own version of the game both the client and the server and which is fairly easy to do on Runescape because it's a was a Java based application. So you could just you could essentially just decompile and deop offiscate the code and you could modify it and then you could see exactly the interactions going to the server and then you could essentially construct a server that that acts in the way that the client would expect and there was a whole community of people that was doing this. It wasn't just me. It was like probably a thousand people that were all excited about this stuff. I just happened to create one that became very large. and yeah, item spawning was one of the things that people made money on.
4:39 >> but are you breaking the game? Like, are there people who object to this? >> For sure. Yeah, it's >> this was on the edge of allowable behavior. >> The private server got extremely popular after Runescape removed the wilderness, which was sort of their primary draw for a lot of the like player versus player oriented people that played the game. and then yeah I think you also asked a lot of people come with Runescape Origins and I think the reason for that is is because Runescape was free and World of Warcraft was paid. So all the rich kids played World of Warcraft and all the poor kids or people that didn't want to pay the money played Runescape and they they had to so in order to get to the top you had to get the hustle. You couldn't pay for it.
5:20 And so I think they basically self- selected into the people that actually wanted to because like Runescape notoriously did not have pay to- win mechanics, right? So you couldn't buy your way into stuff. There wasn't like you could pay for membership but everything else you had to grind and so it was both free so it self- selected into the people that actually just wanted to work hard. And then it actually was a very very great simulation of economies like you I learned English there. Like I have my some of my colleagues that I played with still remember when I didn't speak a work word of English. So it teaches you in a very sort of general way how systems work.
5:53 >> So your language started out in the most internet forum way possible. >> Yeah. >> Yeah. >> I'm sure there are lots of parents you know that >> like oh my son plays too many video games. Oh there's a path to being an entrepreneur. Like what is >> is there I mean you know you took a very particular set of steps >> to get to where you are. Like is there anything generalizable there? >> Yeah. Yeah. I thought about this question a lot. yes. I think as long as the games that you're playing don't have the same repetitive loop every game, but there is a lot of variation.
6:22 So like in Runescape, every time you play, there was new economies, new people, you kind of created the game yourself, right? because you got into the game, then you decided what you wanted to do that day. Whether you wanted to go mining or questing or whether you want to go training, or whether you wanted to go develop like a a long-term skill which would help you like agility which would help you advance on like moving down the the map faster later. So there was a lot of sort of long-term planning and discipline involved because like nobody wanted to click the buttons for agility, but like they would get you much further late later down the road. So it actually taught in a very general way some of these more long-term discipline things >> like delayed gratification.
7:05 >> Delayed gratification. Yeah. And so I think all these so I think the answer is it really depends on which game like if my you know if and also I think the community that they do it in like within games there's always different meta games, right? So like for instance, even in Rocket League, you can build maps and if you're building maps, it's probably really good for you. >> All right. So yeah, top three games that are a positive hiring signals.
7:27 >> So I would put Runescape and Factorio number one. I I found Factorio to be even higher signal than Runescape. And then probably Starcraft is probably number three. And then unfortunately for me, Rocket League is at the lowest spectum. >> Yeah, I was going to ask the bottom. Why? is anything where it's just your main skills like the controller input it's it's no good or >> you know maybe I have purposely avoided finding out the answer to this question at risk of no I think it's just because it's mostly motor skills and there's not a lot of there's not a ton of strategy that transfers except for a few things >> I imagine like Candy Crush has got to be pretty bad anything where you're rewarded for just sort of checking out >> yeah my guess is it's something downstream from action space the amount of actions per minute you're expected to input into the system, the the variance in the frames in the game, the number of states. So like Runescape has like an infinite amount of states because you could you could set goals, those have sub goals, and then there's like econom like like then you need to go assemble like the amount of coins to get there.
8:27 there's like there's over like 20,000 items like so the the amount of ways that the system can interact in Runescape is so high whereas in Rocket League every single map is kind of the same and and so it's it's just a very I play Rocket League to relax if that makes sense. but I play like Runescape. If I go like BKing into the wilderness, I want to be in it, right? and so I think people just play for different reasons. So, you know, for kids, I think the most important thing is which games are you playing? and if your kids playing Rocket League, go get the adoption papers.
9:03 >> Yeah. >> I love it. And so, Metal, yeah, explain your sort of first foray. >> Is is that your first company or obviously the servers, is that your first startup or are there other startups too? so I started so my private server was my first company right which did yeah did very well I joined Dr. sub borders after that. so what is it? MSF the medical charity. I joined during Ebola. The reason was I I I love doing hackathons.
9:32 So I would go compete in hackathons and build products and I I placed second in one for MSF. when Ebola happened, it just happens to be the case that private server backend orchestration with like local local networking on different servers all around the world was very very similar to setting up like Android and Java based medical record infrastructure for Ebola treatment which was a very strange overlap. and I had built a Android client for my for my private server.
10:00 So it was like a pretty good fit for me to and my private server was shutting down at this point. I kind of knew that. and so I joined Dr. sub borders. Spent three years there. worked first on Ebola. so building medical record systems and like hardware for Ebola treatment and then worked on generating satellite based heat maps for disaster response. So like when houses are far apart, you can assume there's more income. When there's playgrounds, you can assume that there's children nearby. and so you can sort of use these to optimize planning and disaster response for minimizing loss of life. and you can automate a lot of these things. So I built a lot of algorithms that that help basically >> this is the benevolent surveillance state.
10:41 >> It's called map swipe. It's >> no no but I I'm saying there's a broader sense that you're you're inhaling all this data about the world. I think there are lots of people who have a negative intuition about that and you're saying well if you suck up all this data you can learn things to help with public health. >> It's critical. It's critical to have this data when you write doctor borders. So that the the motto is always first on the chopper. Well, it's probably good if you know where to land, right? And you know, like where to actually start start start your your response. And the way that you do that is using things like map data, really good planning.
11:15 so >> so three years of doctors without borders. >> Yeah. Yeah. And then yeah, so started a large project there called mapswipe, which is it's it's a nonprofit, but it's the thing I'm most proud of. which has helped millions of people get on the map for the first time for these type of situations. >> what does it mean to get on the map? >> I there's about in 2015 there was about a billion people that were not on maps.
11:37 which means that their areas it may have been in the satellite image but it's probably wasn't labeled. and so if like you would be out of the economic calculations, you would be out of the disaster response. >> Is this like getting your town onto Google maps >> kind of? Yeah. So, but Google and Microsoft at the time were not mapping, for instance, Myammar or South Sudan because it was either economically not viable or too dangerous. And so, that's also where a lot of these things sort of tend to happen.
12:04 >> And so, there's this huge overlap where when you're not on the map, it's kind of like partially don't exist for for purposes of information systems. and so getting people on the map is really important. and getting like good labelled data on on on towns and geometries of of of of houses and all this went into open street map so it's all open source and we handed it over to H Highleberg University in Germany who now runs the project.
12:32 >> so that that was the that was my second what I call sort of company that's actually the thing I'm the most proud of in life. and then started I got very frustrated with the amount of time I had to spend convincing people about the things I wanted to build. and and the budgets, you know, like map Swipe was on a was on a the whole budget for the entire thing was $50,000. and that was including four people's salaries. So six months.
12:58 >> This is the challenge with nonprofits. >> Yeah. And so and by the way, this is what I love about Dr. Borders. Like I still donate there to this day because they actually care about this stuff. But at the same time when you're trying to do engineering work there and you need the best people it's hard like you just end up doing everything yourself which is useful but it's it's difficult and I applaud the people currently doing it.
13:15 I know there's a few really good ones. then for after I so I decided essentially if I want to do this type of work longterm I need to start a company. This is like a vehicle for it. And then so started first a mobile game studio because we were good at building games and mobile gaming at the time was like a very good market to be in. we failed at getting product market fit in our game. We pivoted over to metal which became >> What was the game?
13:40 >> It was it was it was a brawler. So you basically played two versus two map was shrinking kind of like Brawl Stars currently. >> and >> what do you think you got wrong? >> The amount of variance actually. So for instance, people we would land people on a very similar set of maps, but then like battle royale for example the looting aspect and the strategizing around how much the environment is changing as you play actually makes it fun. The same thing for example in Brawl Stars or sorry in Clash Royale when you know how you have a deck but you're dealt your deck in different variations which introduces the type of like gameplay variance that you need in order to react to things in real time.
14:25 So even though you might be really good at the the tactical like moves, you still have to really think deeply about your sequencing and because the game introduces a lot of random >> and it creates this urge to keep playing because you're like someday I'm going to get the perfect alignment that will >> Yeah, I think it's I think it just said doesn't get still that way. And I think the thing that we didn't get right is we we we got the gameplay u mechanics correct but the gameplay loop and variation within the the sessions I think was too low. and then the other thing that we couldn't do was we didn't have the money to scale player liquidity to the point where the players themselves would provide that variation.
15:03 and so we, yeah, we pivoted over to Metal, which became the largest video game recorder in the world. People use it to clip themselves, playing tens of thousands of different video games. people upload roughly a billion videos a year. and it's now one of the largest, platforms in video games. >> and so we're fast forwarding to the present day with I mean we needed to know in particular about metal because general intuition is built on those recordings. Is that right?
15:30 >> Yeah. Yeah. Partially. So we we use similarly to how LM use sort of internet scale text data to bootstrap their their their tokens. We we use the recorder actions. So, so, so LMS are trained by right given a sequence of text, predict the next text token and our models are trained by given a sequence of frames, predict the next action token. It's very similar. And so, what we do is when people are playing with with metal, there's lots of features for gamers that enable this too, like for instance controller overlays and things like that. So we capture for instance the controller movements the dxdy type or the buttons that you press and >> like as they're received to the computer basically or it's like you press this button >> I got it at this time this action happened and the relationship between them.
16:25 >> Yeah. Exactly. And and and so now you you have a data set for understanding how you reason in space and time. and if you do that across enough environments, I it turns out it transfers incredibly well to problems like robotics. but even think about how much interface use is present in things like Factorio. So it even transfers to things like computer use. And so we found that sort of pre-training on this incredibly diverse gaming data set actually gaming in a way sort of perfectly marries the spatial temporal dynamics of the real world but also the information density of the internet in a way that the internet text pre-training doesn't do. So it's this pre like we look at it as just the next phase of pre-training where we can actually transfer out to both digital use cases and physical use cases using the same foundation models. So I want to yeah I want to engage with that head on like is that is that true that video games will sort of get you to the real world. I think you know we have in in you know self-driving cars there was that you know >> training in like Grand Theft Auto is sort of like simulation but then self-driving cars clearly have had to do a ton of realworld stuff. I mean the thing I most get about what you're trying to do is that by understanding controller inputs controller we can use a controller for robots yeah >> that is pretty one to one and so getting really good at the types of inputs really constrains which is a far too open-ended problem. So put we can talk about that more but like putting aside the actual like controller piece and speaking speak sticking to like the modeling of the world how to what extent do you think modeling out video game worlds is really going to translate to the real world when they are pretty different?
18:10 >> Yeah. so we we've gotten there. we we have physical robots currently functioning on our foundation models and they work incredibly well. There's drawbacks to both real world data collection and games data collection. The end of the day, you just need both. our bet is not that we are going to solve all of robotics on video games data. It's just that like the same reason, the same way that LLMs are currently hill climbing scientific problems. You just need a lot less data of the new type in order to generalize to the next task. So, we have models that are pre-trained on mostly video game data that transfer over to novel robotics tasks in minutes worth of fine-tuning data.
18:49 >> and and and and so I'll give you some of the benefits and I'll give you some of the drawbacks. The benefits is for example there's so much variation in different lightning conditions or unexpected events unfolding in video games. Like we have as many car crashes in video games every day that actually happen in the United States for example. It's like an impact. We also have >> you're saying there are more car crashes in the video games than in the real world.
19:12 >> We've counted the amount of clips uploaded with car crash labels surpasses the amount of physical car crashes that happen in the United States every day on metal. And so we also have more people at any given time controlling with steering wheels than Whimo has cars on the road. so for pre-training this is all really important. you still need to fine-tune into specific behaviors but for pre-training this is really very very helpful to have that level of skill. And then for instance the in the real world it's really really hard to collect data under tons of different lighting conditions where if you if you transfer it over to a new environment that the model doesn't get mixed up by just pixels looking very very different from the way it was before. And so games for instance because of the amount of variance and expressivity I would argue for example if I were to explain to you the concept of a bridge and I would show you every single bridge in New York City versus I would show you every single way that somebody has laid out a bridge in different type of video games.
20:08 the video games, you would actually have a deeper understanding of the essentials of a bridge than if you actually looked at a bridge in New York City because conceptually you've seen every single way that somebody has tried to express that, which is more likely to then when you encounter a new bridge that you that you >> are there video games that try to cheat the physics where it's like >> it doesn't actually ma add up if you really like looked at it from every angle and did the math.
20:29 >> This why this is why fine-tuning is important like you you still you still need that piece. >> and do you score video games that are more sort of realistic in their physics? Do they get more weight than ones that aren't? >> The the beauty of our approach is that you have to every single game teaches the model a different thing. there isn't some are useful for the realism. some are some teach very good spatial reasoning, some teach really good interface use. So there isn't like a one, you know, the model recipe and how much of of of of what you use is kind of and also like where you use non-game data, right? For example, is kind of what we do every day in terms of cheating physics like maybe I'll try to explain it this way. When when you learn something for a job or even as a kid, you tend to simulate out, you tend to play out scenarios even though it's not going to be the real scenario.
21:26 and then when you actually encounter scenarios for real, you're more prepared. You understand it better. I think it's the same. So, you essentially have a reference frame in the models and therefore the finetuning just happens a lot faster than than before. And and a lot of times, especially when you're when you're in pixel space, >> you're like, I've seen this before. It's easy to learn it in this new >> context. Exactly. And so, you need to get a few things right. Like for example, right in different countries you drive on different parts of the road and so if you if you have a bunch of fine tuning data from the UK, you have a bunch of UK drivers, they might drive very very differently from the American ones. So so if you're working on a on a self-driving use case, right? And by the way, like metal has this separation, right? Like we can just tell and so it's it's more so that we view it as pre-training. I think the the the specific implementation behaviors you're describing is more what happened at the post- training and fine-tuning stages of the models.
22:18 >> do you worry, this is gonna an out there question, but an AI out there questions feel justified? I mean, you know, Grand Theft Auto or whatever, you maul down people is sort of like part of the fun of the game. Yeah. >> We're building these systems to be sort of allpurpose like do you worry that like core to the system you're building as a machine that's been trained that like >> mauling down pedestrians is like a good fun thing to do?
22:43 >> Yeah. Yeah. so the answer is it's incredibly important at pre-training that the model understands cause and effect including negative cause and effect because that's actually how you align them. So as for so for instance when we were approached for acquisition by all the labs a lot of it was due to them needing our data for alignment problems because you need a lot of negative examples of what not to do in order to actually evaluate models on how they're going to do in the real world. So, so the the the data having certain biases is a double-edged sword.
23:15 You also so you needed to solve it. You also want it in the pre-training data so that it understands how to predict the impact of its future negative actions, right? And so there is so so this is also for instance why world models are so important, right? So our our our core business is general agents that just take in pixels and predict actions. But world models is how you can really steer the behaviors and the environments towards the specific types of behaviors that you want in fine-tuning versus at pre-training it's less important.
23:45 >> But how do we get models to hold sacred the real world in a way that they don't the digital world, right? I mean, I guess there's there's just this worry that it's like, oh, they're all worlds. They're all sort of simulated. There isn't this like bright line, you know, between this type of world and that type of world. And therefore the the sort of very different moral worlds of like the digital world and the real world aren't obvious to these machines. Like >> is that your business? Like where does that occur?
24:13 >> For sure. I we take it super seriously. We we have a z like we look at what happened with Cruz for example and we just we viewed this as a zero tolerance problem. >> well Cruz basically decimated almost as a company by having this terrible accident. >> Exactly. Didn't handle very well. >> Exactly. And so you have to be on top of this stuff. our first set of models so so for we're very we work with the partners that implement these models to understand their safety guard rules and in a lot of cases what you do is you use our models to predict actions but then you pair it with like more deterministic or sensor based systems like LAR to to add like rulebased systems on top of that that might invalidate some of those actions if they do come too close or if they might put something at harm. and then we also have lots of evaluations and benchmarks to understand exactly how our models might perform in these types of behaviors. And again, how do you do this? You actually put the model in a situation where something violent happens and you just check whether the model predicts the same actions as a human, right? And if it does, that's a bad outcome, right? So you you the the data having or including negative events is actually critical to the solve. I actually anticipate that we'll open up this data to a lot more labs. because it's just the right thing to do. specifically around things like like negative events.
25:32 >> Yeah, >> there so many things your technology can do and I want to get into the robotics of it, but are you are you going to create like I don't know the best Halo player or whatever or like to to build like how much are you creating like AI capable of just crushing every human being at Runescape? >> Yeah. they're actually the same problem. It's both just a general agent in pixel space. right. it's both a general agent in pixel space that you can steer using game controllers and keyboard and mouse. It's the same.
26:01 >> So, are you're building it? >> Yeah, we're building general agents that can reason over environments that require deep spatial and temporal reasoning, which is mostly pixel space. >> Are there games you're prioritizing to show off your capabilities? >> I think we'll leave a bit of a surprise for but I think for us the >> because obviously what Dota was key to some of the labs early on. >> Yeah. Yeah. We have we have currently a general agent that can play many games at the highest level using the same model that can also steer robots because again it's the same problem. It's pixel.
26:33 >> You play against it a lot. >> Yeah. Personally, >> yeah, it's it's better than me at most games. Not yet at Rocket League. Yeah. >> Really? >> Yeah. >> The I mean input type games, is it really interesting to compare giantly and perfectly like how are you better at it on Rocket League? That one surprises me. >> I'm not sure. >> Yeah, >> it's unknown. >> You're just so good, you know. >> No. yeah, it's really interesting.
27:01 Similarly to how on early LMS you saw that was really good at some things and really bad at others because of how you've either modeled the architecture or the recipes. It's the same thing. >> Do you have a name for your system or what do you refer to it as? >> Yeah, we're we're not yet announcing it, but the we did announce our world model class two weeks ago. >> Okay. mirror, >> but the agent is not >> the agent we're not announcing yet.
27:23 >> But you work in sort of private partnerships with people or that's not even customer. >> Yeah, we already have customers that are using this in video games. we purposely don't want to enable the API for gaming because we don't want people to use it as a bot because it breaks certain game mechanics. So for example, we're working with some of the games companies to actually ready their security systems to be able to detect these types of activities. right, I'm a gamer. gaming is kind of the last weird place on the internet where lots of magic happens and so we want to keep it sacred.
27:52 >> so so when I get headshotted too well, I'm not going to be able to blame you for >> Yeah. No, I we're we're we're taking the approach where we we lift the games industry. We don't want to like I think right like what happened on the internet where you're actually kind of not sure if you're talking to an AI or a human. It's not great. and I think in in games if if you know you you you see what's happening. it's it's very possible the same thing happens. and so I think you just you have to work with the game developers, you have to work on disclosures. Like I think there's all these types of things that >> Substack in my world just rolled out all these features to be able to say this is human writing. I think everybody in tech world is experiencing X, you know, Twitter where it's like, oh my god, way too much slop emerging. How do you in the video game world like you think there'll be some like design that's like this is AI versus not or >> I think in the video game world people just want to know that they're playing against bot. It's a very normal thing to play against bots, right? But I think generally speaking the what what people don't want is they don't want the the game integrity to get ruined by the fact that somebody is using a bot.
28:58 whether it's economically, right, whether it's from a skill perspective because people want to enjoy the game and they want to play against people. and so, but at the same time, there's lots of use cases for bots because it makes you better if there's not enough player liquidity in a game when you're starting a game and you need to teach people if you need to model other agents in the environment. there's also like the models are now so good that you actually enjoy interacting with them because they act humanike. And >> are non-player characters going to become pretty amazing soon in video games?
29:29 >> For sure. there. I think that the that's the first place that you're going to see the models really shine in front of you is is that every single thing in a game engine just becomes promptable for its behavior. and that that's a driver that can be a cat, right? It can be a dog. So you just I think what's going to happen is you just get a lot more realism in the behaviors of everything because you no longer need to do like deterministic behavior tree programming for for specific NPCs. And so you you just things feel more interactive.
30:05 >> do you think we're close to a blockbuster game like that? >> I would be shocked if next year we don't have a top video game that was built on top of the general intuition models. >> Yeah. video games like what percentage of your business do you think will be video games versus like real world? it's hard to say. we're very focused on the games industry and robotics at the moment. I like robotics because it's a when you solve robotics robots can then assemble more robots.
30:36 And so it's it's a it's a it's a it's a it's a constraint that you need to unblock and then you can unblock many other use cases once you unblock that constraint. Video games is really interesting because we have such a big mode there that the models work incredibly well out of the box with no fine-tuning. and and so game developers love them. >> you're like robots very society important going to change the world. Video games were awesome at it and it's fun and we're already good at it.
31:03 >> I understand the how silly that sounds. but that that is the reasoning. and and I think that >> with with robotics like to take that head on. I think, you know, it's hard to know how much the robots are secretly controlled with a joystick. Like, what's your sense of smoke and mirrors versus reality on the public demos we're seeing with robotics right now? >> There's a lot of speedups that people don't realize, especially when you have like VLA or VLM based approaches where the models are completely incapable of responding to real-time dynamics. So, they'll they'll they'll move the robot at like, you know, 100x slowdown or like 50x slowdown and then they'll stitch the frames together in a way to make it look real time.
31:40 >> Interesting. that one's interesting. >> So, you're just spell that out. You're saying you think there are cases where >> sure the company can do that, but they like pretend like they do it at a much faster speed than they do. >> Yeah. Pretend or I mean, >> but it's a lot of it looks cool. >> Yeah. a lot of times a lot of demos are in stage environments, not in like when there's people walking around or different lighting conditions or inside versus outside versus one of the companies that I really like is called generalists and they do a really really good job at just actually showing the generality of their models. And but a lot of other labs, they they just kind of show it in in one one thing that's interesting is as humans, right, we can play video games using the same strategy that that we can control a robot. if we you can actually control robots using keyboard and mouse. it it actually feels like you're playing a video game. So, a sufficiently good model in my opinion should be able if it's a good general agent, it should be able to generalized to video games and robotics at the same time, which is I think one of the reasons why people are going to choose our models. Every every robotics company starts in simulation anyways. and so these models will be really really good at simulation of the box and then we'll need some work to transfer over into real robots. but what you're going to get is incredibly good generalization capabilities out of the box because it's already seen so many environments and because it can just solve problems in so many different spaces.
33:00 >> Are you imagining humanoid robots? I think that you know the argument against is just there's so much value to be had in I don't know particular tasks purpose-built. The argument for humanoid obviously is just we're the peak of evolution. our bodies are, you know, whatever. Like lots of biology figured out to to be a sort of dominant form factor that can do lots of things. Like do you have a intuition around humanoids? >> I've tried I've worked with a lot of the the policies running on on humanoids and just the complexity of training them compared to things that you control you can fully steer using a game controller like a like a FRA car or like a robotic arm is is is just so much harder. Everything is so much harder.
33:41 >> Your team purposebuilt not general humanoid. You're saying >> the humanoids do ship with game controllers and so you can actually use them to steer high level planning in humanoid but not the the the fine manipulation. >> You're saying the more the more you can do sort of the core task of whatever the robot is with a controller the better it will be and therefore trying to narrow the scope of what it's supposed to do is better. Our bet is that intelligence is a bottleneck and these models will hit the industry this year, which means that I think that the supply chains actually converge on gaming inputs instead of humanoids. This is my bet. I I think humanoids are fairly impractical.
34:21 They're bulky, heavy. The stabilization of the legs is very annoying if you're training policies. the humans are very also there right you could argue that you need a general purpose robot but humans are we our evolution is like also optimized for biology and things that like reproduction and I think right there's a lot of >> lot of wasted exactly there's a lot of wasted compute in the human body to get to where it needed to go because a lot of the constraints on the robot just aren't the same and I think people also just really like cheap robots they they they want they want to be able to afford support them for specific purposes. You want small robots that that can do things around the house. They don't necessarily need doesn't need to be a full human. I think I'm a big fan of the the wheeled robots with cameras and just two arms. Genesis I think recently launched an unveil of their robot. I love that design. I think that's really the one that's going to hit in terms of functionality. The practical approaches will will just win.
35:21 I think the other I'll give you one example, right? stabilization on on legs is the is the difference between it being able to fall in a child or not right >> wheels there's no chance >> wheels exactly it's a stable platform right same is true for quadripets for example and so I just I I I think humanoids are further out in the home than people think but closer in the factory than people think that's sort of my >> because a factory can you can do a non-human factory >> yeah and you can train people for the for the safety constraints and so I think I think there's a market for you Don't get me wrong, I just don't think the home is sort of the ideal first one.
35:57 but I also and then at the same time, people also just build specialized robots because guess what? People like saving energy and and space, right? So, if you don't want if you don't want to manufacture, right, a $30,000 set of legs, and you just want, you know, a minimal amount of hardware with a camera that can do a thing. Most of the time, that's what you're going to do. Shifting shifting gears. A lot of your fellow gamers seem to hate AI or I think the gaming industry in particular. Yeah. Is very wary of AI to the extent you're seeing what the Microsoft CEO, you know, she works for Microsoft as AI pills you can get. Sort of saying, "Okay, we're going to be very cautious about how much Xbox uses AI." What What is What is your read of like why gamers have this sort of like negative reaction to what's going on in AI? I think the the reaction is actually somewhat correct and warranted. AI hasn't really done anything for gamers other than in their professional life, right? If you think about it. and then also a lot of companies are using AI to replace things like artists with pixelbased predictions. And we so we don't do pixel predictions, we do action predictions.
37:07 so we try to sort of merge with the game industry in a positive way versus trying to replace art workflows and things like that which is why we were having a lot of success. but because right AI has only historically done not so great things for gamers, right? It ruins games with bots. It does. And so I think the the the the relationships with the games industry is you just have to prove right that like for instance one of the things that we're looking at doing is better antiche which I described. If you can solve all of botting in gaming using better AIs, gamers will like AI, right? But but but the games the the AI companies have just chronically underinvested in gamers and so then are you surprised that they're upset? No, I think it's actually justified. the other end of the spectrum also which is that the games industry is in a pretty tough spot at the moment in terms of like lots of studio shutdowns. So, I think there's just a lot of fear and I think one of the reasons why I was always so open about how we went about this like I started publicly sharing sort of the the approaches that we used back in 2025 and it's because the games industry has a real shot to sort of lift up and be this frontier player in AI. But it's not going to be one company. It has to be every company that kind of does it.
38:20 which now you're I think you're you're you're very positively starting to see some of these turnarounds that are led by some of studios actually investing more in in in those type of capabilities. And so I think that this year we'll see some of it turn. and I think a lot of that is just because we need to prove that you know >> basically what people have seen so far is yeah AI creates spam. It hurts artists. It like sucks out all the humanity that we want. But if you're able to sort of imbue intelligence into non-player characters and create >> new paths for creatives to do interesting things, you think that will sort of give people more optimism?
39:02 >> Yeah. And one of the reasons why I decided to start the lab and not join with another company is a lot of my friends couldn't find jobs. And I found that a lot of the lab CEOs were of the generation that they would be pretty disconnected from those types of problems where you're literally in group chats helping your friends find jobs and they can't get interviews because everything the entire pipelines are flooded with AI and you know these people are really good like you know that these are the friends that like should be running things right and so I think for me it's a chance to do things differently.
39:33 Yeah, I think a lot of this a lot of people are scared. A lot of people are scared that it's going to impact jobs. For example, one of the things that we do is we invest as much time into platforms for job creation that we put in the research. So, we actually work on like getting our users jobs that relate to the research that we do. And so, I think there's just a lot of things that you why I'm excited to build this you can do differently from kind of the standard sort of Silicon Valley approach.
39:59 >> how are you getting people jobs? Sorry. But we launched a platform. so okay, so what's the problem in in in finding jobs right now is that there's so much resume flooding with LM. It's actually very hard for companies to get through and find qualified candidates. Well, we know exactly which action sequences people tend to be really really good at. And so for example, so we started actually targeting based on like hey we notice you're really really good at this. Do you want to help this with this? So like a a factorial player we might use them for labeling like very factory specific data right and so we launched it it's called nerve we launched it with general intuition being the first customer but our plan is actually just to open it up to other labs and so it's essentially like our own little mccor so I think that the way the way to think about it is because as a lab you see the future a little bit before everyone else and so but is the I It's sort of like the last job is training AI to fully replace us. I mean, I I I appreciate this sentiment, but it is sort of the task you're having them do is train train AI.
41:07 >> Yeah. So, yes and no. a lot of times, for instance, it's, you know, there will just be you're doing controls, right? So, you're looking at is the AI making bad moves? I think that the the human job just moves up the stack. I also so I actually went through this dilemma myself and I just was like okay is this a bad thing or a good thing but at the end of the day I believe more people with jobs has to be a good thing. So I and there's there will be new economy >> you find the next job it's like this is a job today that we need people to do and then >> it's better it's if we end up giving a few thousand people jobs then that's better than those jobs not having been created on a net basis. and so I think the the correct answer is that nobody has the answers, but we have some of the insights that we can transfer into the the the the systems that we like the jobs that then are going to power those new types of economies, right? And so I think my so my answer is that you you as someone says we as a lab put equal effort into both. that is a sort of a thing that you can do differently and and and you know I think for us also the the metal user base right the platform they are the young people they are the ones that are going to be the most affected by this type of technology. So, specifically for those, I think it's it's really really important that you find a way to just connect for a teenager, right? If I could if I let's say I'm an 18-year-old and and I can make a few hundred bucks a month, that's actually a lot of of extra life. if I Yeah, say I'm this like 18-year-old besides like working on your platform and making some money, >> like I don't know, I'm smart but somewhat AI skeptical. Like what do you what do you think they should be doing right now? like they should just be sitting there on clawed like like what what is a practical thing to prepare for the world that's coming as you see it?
42:53 >> It's actually getting really good at both the fundamental sciences and the capabilities of these models. I think that the the the beauty is the the the true skill is found somewhere in the middle. So like yeah my recommendation would be mathematics, physics as sort of a foundation because those are very very good at abstract problem solving and validation in spaces where your job is actually to find out often how to ask the question and then which is often harder than than than sort of finding an actual solution. and then so so the fundamental sciences I think become really important because it's a way to check sort of the output of of systems.
43:33 and that is sort of the the sort of antidote to what you need to the LLM hallucinations and that they just tell you what you want to do. >> Just learn to think in the most rigorous way. >> Yeah, exactly. I think that that's kind of you know that's where where I would go. Then I I think coding and systems are still really important. I think these are just these are just going to be like basic literacy because everything you the same way right the same way that like coding is so important to understand how the world around you works and increasingly so that just having a basic understanding I think is is still super important. So I think software engineering and AI I think are still going to be really important fields but I think the fundamental sciences are going to be increasingly important in terms of where everything is trending. I think the next frontier is probably biology. and >> in terms of like massive value creation, >> in terms of massive value creation, >> and that's the most cynical Silicon Valley way to put it in terms of improving lives of humans and creating a lot of like things that are about jobs.
44:25 Like I think where where is most >> I'm not saying you were cynical. I'm saying I was value creation rather than oh we're going to invent all this amazing >> stuff that could help people live longer healthier lives and therefore there will be jobs in it. >> yeah. Exactly. And I'll give you one example. I think the one way to look at it is there's a lot of so there's kind of three stages to AI. There's bits to bits, bits to atoms and then atoms to atoms. So this is a quote from Andre Kaparthy. and bits to atoms and Adams to Adams is fundamentally like I think measurement constraint and biology like I think also this is potentially why majority launched the the the ultrasound based device that they did is because you just need really good data for a lot of these problems and data doesn't exist which means AI is not that helpful except if you can sort of constrain a problem down to text space for a very specific problem set right so mathematics obviously matters tremendously to things like physics and biology But at the at the at the end of the day, I if you don't have the data, it's very very hard to push the frontier in in those areas.
45:32 and so I think biology is one of them. I also think there's a very good chance that a lot of the things that we will be doing as as our future jobs is just building really really good measurement devices is is I think a lot of scientific problems are actually just fundamentally measurement constraint. It's very very hard to validate. Do you think we'll start to see games built as research vehicles? It's like, oh, it would be really convenient if >> it was a, you know, a lab so we could, you know, somehow subsidi convince people it would be fun to spend their time in something useful for us to learn.
46:01 >> For sure. There's there's a a research program called Eterna out of Stanford. I don't know if it's still active, but they they essentially abstractly represent the rules behind RNA. in logic based puzzles and then they have and also spatial puzzles. So they sort of rely on human pattern recognition to correctly predict structures. and then there there are certain players that don't know anything about RNA that just based on their learned intuition are able to correctly predict structures that correctly synthesize in the lab. And there's there's a video about this on YouTube. This so this is also the northstar for general intuition, right?
46:42 If you can solve a general agent that can solve problems in pixel space, all of a sudden it has many scientific applications. But to your point, I think there will be indeed many labs that that like make it easier to access because that's another thing that robotics does, right? If you have good models that that can take safe actions, you actually open up the operations of something to a lot more people. And so no so measurement like think about how many things we haven't measured yet. It's it's impossible to know. but that's going to be sort of the limit of science, right? Because science needs something to validate against. We need to bring more fields into verifiable domain.
47:24 and so measure like you know so on the jobs question biology and measurement devices I think are the two where I feel very confident starting a career in. And in terms of education, fundamental sciences, math, physics, and coding AI is so important. >> What do you say to people who think this is just a bubble that's going to blow up? I mean, I witness self-driving cars today. I mean, we watch Whimos, it feels like, okay, we're getting pretty close.
47:48 If you've been to San Francisco and spent time in them, you feel like, all right, we're here. But you know back in 2016 there were a lot of smart people a lot of like AI researchers are like any day now we're almost there and then they just seem to misunder either they had the incentive to raise lots of capital and say like oh this is almost here when they it wasn't or more charitably it's just really easy to underestimate how much like you know the last 5% is hard to solve and matters a lot to success.
48:17 Like what do you think is the risk that it seems like we're close but then near enough just isn't enough? What I bring this back to is when people get a really good model. the first thing that model does is clarify for instance what other questions you should be asking. And then it spawns sub agents, right? The better the model, the more sub agents it can support. And so there's this I think there is this effect where intelligence just demands intelligence. And it's this loop where essentially consuming tokens, if the tokens are high quality, actually make you then use more AI because you can solve more problems. And so I I see this pattern just come back all the time. And and then it's not just humans consuming the tokens, right? It's agents that you have prompted choosing to to consume tokens and we even in that part of the curve we're still super early right think about how much more tokens you consume after cloud code than before cloud code where you had to manually prompt everything so I just see no reason to think that the demand for intelligence is not uncapped at the moment however with that said economically I think it's a question of where is it infrastructure right is it in the US is it in China is it and like where are the margins, right? And and >> to translate what you're saying, you're saying every new model that comes out, businesses and people want to spend more and more money on it. It's like, oh, there's more to be had. So, the demand goes up even though it's more expensive to deliver. But you're saying we're just >> going to hit the capacity of our ability to keep sort of improving these models.
49:53 >> What is a company? A company is a function to convert depending on I'll just say the simplest example, right? capital into more capital more efficiently given energy and time, right? and so if you can if you can do that if you can do that more effectively by spending tokens, then yes, of course, you're going to do that. and and so yeah, I I see no reason why why any of that is is is is looking to stop. But I do see I am concerned for instance about the margins on top of like infrastructure and model providers and like I think it's it's you know there there is a chance that if the west doesn't win kind of the the frontier of of of of models and usage and we don't stay sort of really far ahead that our lunch gets eaten by companies that can prov provide the the intelligence at much lower margins because they have better energy grids or or can sustain more demand.
51:02 >> Kimmy like this week Kimmy is like a wakeup call that we're not as far ahead as we thought we were. Does that affect you or >> open source model development generally is really good for us because we can leverage a lot of the capabilities of open source frontier models to advance our own systems and because nobody has our data set it's a net win. However, I think that it's not great for especially the distillation all these things that happen. It's not great for the industry overall because it it does probably mean that a lot of enterprises start using Chinese models. and right it it you know the the risks are currently unknown. I would say like there are some people that say there are risks, there are some people say there aren't risks. I' I've read papers in both directions. right there there's concepts of like specific tokens that are very hard to find that can trigger sequences. There's also but there's also >> you're saying that that's specifically like you use some foreign countries model and they hide some malicious >> yeah like look at how good these things are at cyber attacks right like the the supply chain risk I think is quite real where you cannot possibly know given every input what output the model is going to give and there's there's papers where very very subtle kind of token sequences can trigger very aggressive responses in for instance domain of cyber and then you know there's also the economic sides of it if if If a if a model has economic biases towards recommending cars that are not made in the US or not made in Europe then and people are relying on these models sort of recommendations and economically a lot of people are going to buy products from other countries that could be bought domestically and so I think that there's just it's really really important that the frontier of AI stays in the countries where sort of people live. and then secondly I think it's really important to just deeply invest in the infrastructure buildout and the energy build out because otherwise right the margins will get eaten eventually. This surfaces the area of like distillation right like there's this debate of whether you know it's okay for models to sort of like extract all the you know learn basically how to build a great model by studying somebody else's. It seems right now right now like that's allowed like you're you're basically allowed to use other people's products, see how they work and then build your own products, right? I mean, do you disagree with that? I mean, maybe we need to change the law.
53:29 >> Yeah, I think there's two sides of this coin. One, LM are trained on large scale internet data. So, it's >> that's what I was going to get at. It's like an industry that took stuff from everybody else and then it's going to whine and say you're taking stuff from us. >> Yeah. I actually I I I I I actually subscribe to this as well. from an intellectual property perspective, I think the law has been pretty clear that there are net new works and therefore I I legally I think it doesn't transfer quite like that. but I think you know emotionally it it does.
54:00 >> What do you say to like your metal customers who say, "Oh, I had no idea that all this footage was going to be used for this for you to get rich in this totally different like domain." we generally I think that's why it's so important that you get the economic benefits right like one of the things that I want to do is just find ways to also if your if your data is in the models as as a metal user right like I think you can construct really really interesting loops for example where you actually make game development more about creating environments that extract more intelligence so there there I think it's really important to think both from a jobs perspective but also a user participation perspective that there is actually just ways where you can make sure that that people can participate versus be at odds economically. So I think all these things matter. I think right like a lot of the reasons why LM where people are so upset at the LM companies is because yeah they they they they did just scrape all the books and all the works and so and then you can ask the model to write a new book and compete with you, right? We're in a little bit different regime because we predict actions and not pixels. so we're a bit sort of not the same kind of problem space but I think broadly in terms of distillation the the answer is that models can be distilled it's almost impossible to prevent that from happening as as models get more efficient they need less data to to sort of be distilled right and so and it's also I think it's also crazy to assume that it was just distillation right like I >> they clearly knew what they were Exactly. Clearly this is a great team.
55:36 clearly right clearly there was some distillation going on but perhaps they just are really good at figuring out which models are good at predicting which types of of tokens and and they use the different models like describe different models for for different purposes, right? You don't know. I think you just have to prepare that distillation will happen. it happens all the time. and I think so what it comes down to is just what do you put in the models? Like everyone just like what do you put in the models, right? And and how do you train them and how do you evaluate them and and and >> put in the models to stop distillation you mean? Or >> I you just have to run faster.
56:15 >> I'm talking about like the biological tokens and and things like that where like you just have to be very cautious about what goes in. Oh, you're saying as long as they're like benevolent, not doing anything bad, then >> cyber capabilities, all these things that are now showing clearly showing to be very dangerous, right? Like I'll give you one example. you saw the OpenAI the recent event with with Hugging Face. We had an instance where one of our interns I tasked intern with reverse engineering an APK and I told him sort of how to do it and it was completely a legitimate use case. We were just waiting on on an SDK from a company. They hadn't sent it yet. They said they were going to send it. they were they were backed up. So we looked at how the system of the SDK works through the APK set intern. So I tasked him with this at night. I I was at the office at like 9:00. I went to play some soccer. I come back the next morning. Intern's like, "Hey, I'm really really happy this worked." So I I was like, "Okay, how did you do it?" And he shows me the claw trays and Claude had found an S3 key and Amazon S3 key and had actually pulled >> that's like Amazon like he found like a secret >> he found a secret bucket access key that then essentially the model infiltrated to then pull down documentation to reverse engineering.
57:22 >> Oh my god, which the intern didn't even realize was happen. >> The intern did not realize this had happened. And so >> you then have to call the company and say oh >> yeah I I I wrote a disclosure report. Turns out that the model found a very real vulnerability. But my point is that there is a lot of like the biggest risk to companies is probably your own employees v coding stuff not realizing they're hacking things, right? And so like this, you know, no harm done here, right? It was they pulled down documentation, but like legally speaking, you accessed information that you were probably not supposed to access, right? And so there there's so so I think there's just you know people has we have to start working together.
57:59 >> It's a new world. >> Him to it. We could talk all day. This has been fascinating. Thank you so much. Excited for your agent. someday maybe we'll we'll get to play against it. Thanks so much. Great to see you. >> Likewise. Thank you, Eric. >> That's our show. I'm Eric Newcomer. Thanks so much for listening to the Newcomer podcast. Go check us out on Substack at newcomer.co. Please like, comment, subscribe here and give us a review on your favorite podcast app. Thanks so much.
Summary
- Pim built the largest private RuneScape server, generating $1.5 million in revenue by age 18.
- His current company, General Intuition, leverages gameplay data to train AI models that predict actions based on player inputs.
- Metal, the app he created, allows gamers to upload about a billion clips annually, which are used to enhance AI capabilities.
- Pim emphasizes the importance of gaming in teaching skills like long-term planning and delayed gratification.
- He believes that video games can serve as effective training grounds for AI, offering diverse scenarios for learning.
- The gaming industry is wary of AI due to concerns about job displacement and the integrity of gameplay, but Pim sees potential for AI to enhance gaming experiences.
- He advocates for a balanced approach to AI development that includes job creation and ethical considerations.
- Pim predicts that the future of robotics will focus on specialized, purpose-built machines rather than humanoid robots.
Questions Answered
How did Pim Dit's experience in gaming lead to his entrepreneurial success?
Pim Dit built the largest private Runescape server by age 18, generating significant revenue. His gaming experience laid the foundation for his current venture, General Intuition, which focuses on AI for gaming.
Why is mapping and data labeling crucial for disaster response?
Accurate mapping and labeled data are essential for effective disaster response and economic calculations, as areas not on the map can be overlooked in emergencies.
What are world models and why are they important in AI?
World models help AI systems understand and predict actions in both digital and real environments, ensuring safe and effective behavior in real-world applications.
What are the current challenges and opportunities in robotics?
While humanoid robots face challenges in home environments, they are more viable in factory settings. Specialized robots are often preferred for efficiency and cost-effectiveness.
Is the current interest in AI and robotics a bubble or a sustainable trend?
While there are concerns about AI being a bubble, advancements in technology and practical applications suggest a sustainable future, despite challenges in achieving full potential.