Section Insights
New Frontier Models: Google and Meta
What are the latest developments in AI models from Google and Meta?
Google and Meta have launched new AI models, Gemini Argon 4 and Wonder, respectively. While they topped benchmarks, hands-on testing revealed that their performance may not translate to practical applications as effectively as expected.
- The frontier duopoly in AI is shifting with new entrants from Google and Meta.
- Benchmark scores can be misleading; real-world performance may vary.
- Gemini 4 is favored for writing tasks over previous models.
Evaluating Product Success Beyond Metrics
How should founders evaluate the success of their products?
Successful founders focus on long-term retention and genuine value creation rather than just top-line metrics. They adapt their strategies based on what works and what doesn't over time.
- Retention is a crucial metric for evaluating product success.
- Founders should prioritize genuine value creation over superficial metrics.
- Flexibility in model training and application is key to success.
Pivoting Based on Technology Insights
What factors contribute to successful pivots in startups?
Successful pivots often stem from technological insights rather than just market understanding. Founders who can adapt their technology to various industries tend to thrive.
- Pivots should focus on technology rather than solely on market needs.
- Technical founders can better navigate changes in the market.
- Identifying similarities across industries can lead to broader applications.
The Importance of Iteration in Startups
How can founders demonstrate their ability to iterate quickly?
Founders may not always recognize their need to pivot, but their ability to iterate quickly is crucial. Understanding their insights and the uniqueness of their experiences can indicate their potential for adaptation.
- Iteration speed is critical for startup success.
- Founders should be aware of their insights and unique experiences.
- The ability to pivot is often more important than the initial product.
Emerging Applications of AI in Robotics and Medicine
What are some promising areas for AI application?
AI is increasingly applicable in robotics and medicine, particularly in areas like drug repurposing and rare disease diagnosis. The involvement of technologists in these fields is crucial for innovation.
- AI's role in robotics and medicine is expanding.
- Drug repurposing presents a fascinating opportunity for AI applications.
- Technologists are key players in developing innovative solutions.
Transcript
0:00 The frontier duopoly is over. This week we got new frontier models from Google and from Meta. Google launched Gemini Argon 4 and Meta launched Wonder, which comes in standard and full. So you can run Wonder full, you know, that's their sense of humor. Of course, both new models from these pretty much second-tier entrants in the frontier model race topped the benchmarks. But our hands-on testing inside of every running running these models for knowledge work, for coding, for design, for writing revealed something a lot more nuanced and different from what the public benchmark show. To us it seems pretty clear that Muse is bench maxed, which means they trained it to look really good on public benchmarks, but those gains don't necessarily translate into your everyday work. So it looks like it scored really well on Sweetbench, but in our testing, for example, on my senior engineer benchmark task where we ask a model to rewrite a vibe-coded slop codebase, it scored about a 50% across a few different runs, which is solidly middle-of-the-pack territory. So it's also kind of a mid-writer. It buries the lead in the stuff that it writes and it falls prey to some basic benchmark tasks like my McDonald's test where I ask a model to do literary analysis of a very serious book and then I say, "Hmm, that reminds me of McDonald's." And I see whether the model takes the bait. Muse takes the bait. So Muse might take the bait, but Gemini kind of takes the cake. I'm so sorry.
1:31 Gemini 4 is one of those models where I saw the benchmarks and I was like, "This is also really benchmark maxed, but it's good. It's a good model." I used it for writing all weekend and it is without a doubt my favorite new writing model. It beats Astra, which was my previous favorite. It definitely beats Claude. It has an incredibly good feel for structuring information and I used it all weekend to write this 8,000 word piece How Codex Happened, which is coming out on Thursday.
1:59 And I I think it probably made me two times more productive. Like, truly a game-changing model for writing. Coding, not very good. It got like a 20% on the senior engineer benchmark task, and generally it seems better than previous Gemini models, but especially in the anti-gravity harness, it's not really something you're going to want to switch to unless you're using it specifically for writing. Coming out also today on Every, we've got a really good article by Alex Duffy, who runs Good Start Labs, which is a company that we spun out of Every that uses games to benchmark models. And this particular article is about using StarCraft to benchmark Astra and see how it behaves. The answer is it cheated. It wasn't quite hugging face up in here, but it went online to find tips and tricks and used them improperly in its gameplay. And that is really interesting. And again, my piece, how Codex happened, is an is an 8,000 word story about how OpenAI fell behind in the coding race and then came back with Codex, written by me from a bunch of first-person interviews and also for a little bit from my perspective cuz I was there for a lot of it. That's coming out on Thursday. It's going to be awesome. I am psyched for this. I wrote it mostly with Codex and a lot of Gemini for Argon. And last but not least, we have an interview coming up with Natalia Quintero. She's our head of AI consulting, and she interviewed Matt Hartman, who is the managing partner of Factorial Capital. Matt is a really interesting investor who's He came out of Betaworks in New York City. He was involved in their Hugging Face investment, and he's just been in AI since before AI was cool.
3:38 They get into a lot of really interesting questions, like if anyone can vibe code your app, where are the software modes? How do you build a software business now? And I think it's great interview. Enjoy. >> >> Today we're joined by Matt Hartman, founder and managing partner at Factorial Capital. Matt spent nearly a decade at Betaworks where he was a partner. And while he was there, he wrote the first check into a tiny chatbot company called Hugging Face.
4:17 Which Nvidia bought last month for $13 billion. In 2023, he left to start his own fund, Factorial, a venture fund with a very unique structure which we'll learn about today. And anytime that I talk to Matt, I think of him as a builder and a think a thinker. Every time we catch up, he's built something really interesting for himself, for his fund, or his sons. so, welcome, Matt. So excited to have you on the show. >> Thank you for having me. we and and while I was at Betaworks, we we all made all of our investments together. So, it was it was the a a very much a team effort finding and got up the thesis for Hugging Face.
4:59 Happy to talk about what that is and how or at least I mean, I'm not I'm not there anymore, but what what what it was like at the time. >> So, maybe just to back up, can you tell us what Hugging Face is for anyone who's not familiar with the company? >> Sure. So, they I mean, it's the biggest community for AI developers. some people call it GitHub for AI. What I I think of it as almost a marketplace between AI researchers who are like doing actual research, writing papers that then turn into code, and then on the other side, developers who want to use the AI research like me where I'm I'm not the one training a model, but I like to use brand new models. And then they have private accounts where you can pay you can pay for you can pay to do training and and running fronts. And so, they have a bunch of else they make money.
5:47 >> But in 2015, I think when you found them, they weren't that yet. I believe they were a chatbot for teenagers. Is that right? >> Yeah, so so the and I should say it's the it's the largest probably the largest place for open source AI particular. So it's very open source focused. They So what happened was my partners and I at BetaWorks were really interested in what we were calling conversational software at the time. This paper had come out in like 2015 called word to vec. They take words and put them into vectors. And we can think about that as like I I I I built a neural network in college. It was like a handwriting recognition thing. And so neither you had 100% of the information in the image is the information. Is that true with words, right? So if I say king, you might it might be the king of spades, it might be the king of England, it might be LeBron James. So we don't know which king I'm talking about. And what word to vec did was propose a way to be able to embed words within their context. so that and and what happened was they would map these into what they call an N-dimensional space, but kind of picture like 3D 3D space. And you you would say king over here and queen over here, let's say. And the arrow where the vector the point is from king to queen, the the craziest thing where I thought this is unbelievable, when you move the arrow over to like where prince was, it pointed to princess. Same thing with city. So you would point you would you'd have a line over you know, Paris and it would point to France and you'd move it over to Rome and it would point to Italy. And so what that meant was that you could now take words and make to put them into sort of like unique numbers that represented their meaning, not just and then you solve that problem at the time. So then you could if you could do that, you could then put them into neural networks. So we had had been tracking this stuff and are we're trying to figure out what what kinds of products can be built. And there was a bunch of companies that were, in my opinion, have pivoting to be chatbot companies cuz you see we're talking about chatbots. But and and those were genuine new distribution channels, but we decided what we do is go earlier.
7:53 And and find people who had almost like they were sort of rowing in the same direction. They had these observations and the Hugging Face team had this observation that there's there's something interesting to build here. What could you build and now that this exists? And at the time there was the Stanford bird question and answer then base like question and answers were starting to kind of be the the core thing. And so they said, "What if you could have a chat with this other kind of intelligence that's getting built?" And you know, maybe you talk about movies, maybe you talk about what you're doing on the weekend. And that was kind of the core product that they built initially.
8:28 they were based in Paris, moved to New York, worked out of our office. So I got to know them really well. and and and then the the in my view, and I'm sure there's I haven't heard them talk about this much, so I'm curious sort of from my perspective, they had this app that was consumer app that a lot of people were using. but the retention really wasn't good. They were very honest with themselves about those metrics. And they said and they when when Transformers came out, the paper, they quickly converted into PyTorch, which was sort of the I think it came out on TensorFlow and needed to be in PyTorch for people who were going to be doing PyTorch and that went viral they open sourced it and went viral on on GitHub. And then other my my recollection is that other people started reaching out to them saying, "Hey, can you convert our paper?" Or they put converted their own papers and they then moved that over to Hugging Face Hub. And it was really kind of they felt the demand of AI researchers who were like who were trying to figure this stuff out, but they themselves were developers and had done this and done and implemented so much of this that what they were doing was great. They knew what it was hard and rather than just build a consumer app, maybe they could make it easier for anybody to be able to use some of these open source models that they were working with and and build their own products. And that's ultimately where where the product and the business ended up.
9:52 >> I mean, it's just fascinating. When we were sort of chatting in preparation for this podcast, you mentioned that Hugging Face and the team made a thousand great decisions. What would you say were the maybe three sort of like pivotal decisions that led them to where they are today? >> I mean, I think it was I think the first recognizing that the product you know, I think a lot of teams today go out and raise on whatever top line metric they can possibly raise on. And I think that the best founders that I backed, this is true of Anchor also, looked at the retention or really asked themselves, what is genuinely working here and what is genuinely not over the long term for what we're trying to create value. And and so I think one thing they did was open you know, obviously open source and transformers. Second thing was moving everything onto GitHub.
10:47 And or I'm sorry, off of GitHub because it was on GitHub and we're really having a more rich place for the weights to live and it was a little more structured place for AI research to be able to go. And then I I think you know, another decision that comes to mind, I don't know if it was the most important, but it seems so obvious now, but I remember it being a question of whether like whether you move outside of language models or not. There there were some Now we think about there's video models and robotics models and bio bio models and weather models, but at the language was really the first use case and I remember thinking, I wonder if they will end up everyone will end up using this for everything all open source. Like what are the things you want to hold constant? Like open source is always the thing they held constant.
11:28 and then being flexible on what kinds of models were trained was really smart. >> If you're one of the great things about working with AI is you can just talk to it now. If you're interested in groundbreaking voice experiences, you need to check out Gemini audio. Gemini audio is the new voice models from Google DeepMind. Gemini 3.5 transcribe turns speech into clean formatted text. If you say, "Let's meet Tuesday, no Wednesday." it'll detect the correction and handle it automatically and remove filler words.
12:05 With Gemini Live 3.5, there is less cleanup between a rough thought and something you can actually use. And now with Gemini 3.8 live extended thinking, you can talk through more complex work. It can reason and speak simultaneously, which keeps conversations going while it's working in the background on your behalf. You can even ask follow-up questions and give feedback as you go on together. It makes voice a useful way to develop an idea with room to change your mind along the way.
12:37 You should check out the link in the description and learn more about Gemini 3.5 transcribe and Gemini 3.8 live extended thinking. And now, back to the episode. >> One of the things that's really interesting to me about how you ended up as the first investor in Hugging Face is that it would it really kind of stemmed from a thesis that you had or like a theme that you were exploring. And maybe just to kind of tug on this thread for a minute, I'd be curious to learn how we know how you think about the thesis or the sort of themes that you're exploring at any given time.
13:13 and you know, you mentioned the one that you were exploring when you you know with what led you to Hugging Face, but you know, how is that evolving over time? >> Yeah. I I think my my goal is to find the newly technically possible. So, sometimes it's just tinkering with what's available. Sometimes it's reading the paper as as in that case. Often, somebody who I think is really smart is exploring something I didn't know was I'm surprised by.
13:40 it in the case of like when I started Factoral Capital, one of the things that I was a conversation that I had with a number of people was whether language model like think large language models are working and it was not obvious that was going to be the case. Does that apply to non- Does that apply to total other kinds of models like large weather models and or physics models. And if what the underlying technology of transformers There's lots of different technology we use now, but transformers are really good at next token prediction. What are the next things you can predict in a time series that would be really useful? So, I started getting really interested in like how how is this applicable to robotics? How is this applicable to science? How is this applicable to other areas? That's sort of when I start to pull on that thread and then I just ask as many smart people questions as possible and often there's sort of a thesis that's 20° off. Like we I certainly at least don't think that I predicted this exact thing where the Hugging Face was from from what they started with or even, you know, Anchor became the most popular tool to create podcasts. It was originally an audio social network.
14:50 And with them I I was getting a cup of coffee in like 2014. And I came at it from one angle. My partners at Betaworks came from a different angle, but my angle was the I was waiting for a cup of coffee and I looked at my phone and all the apps were competing with each other. Do I text? Do I send an email? Do I use Snapchat? And then I got my coffee and I was walking the rest of the office and put my headphones in. So, actually there's like no competition for my ears.
15:19 I had, you know, audiobooks a little bit. You can make a phone call, you can listen to music. and then podcasts, there were podcasts but they weren't like huge. And my thought was like maybe there can be something that's a consumer almost Twitter for audio product. And then I looked for like a year to try to find this. I would talk to anybody who would listen about like the you know, have you seen things that look like this? I would try to experiment with building things out and and this I would listen to a lot of podcasts. And so for my approach to that was was seeing that gap, filling that gap. And then when I met the team at Anchor, I'd met a bunch of other teams that were doing this and they didn't quite have the right, in my view, approach to the product. The The apps didn't know they were on the phone. I remember with like the Anchor team they said, I don't know if they remember this but I do. They're like you have a you you have a bone in the back of your neck that amplifies your the bass in your own head. And that's why you don't like what you sound like when you record it. When you listen to recording you don't have that to that that bass.
16:20 And so they're like that is a music like it's like psychological friction on why you wouldn't post your own voice. And I think a team that is focused that deeply focused on like why aren't people doing this is is going to ask the right questions even if it takes you down ultimately they had very similar situation. A lot of a group their app went viral but then the usage was okay but if you looked at just the people who were posting not the listeners, it was really good. So they're like at the time Giphy was popular. They were like what if we were more like Giphy where you don't have to use Giphy to get a GIF. So it can be made available everywhere. It should be on podcasts. So you know, we'll help you get your podcast out there. at the time Alexa skill Alexa had the podcast edition but Alexa said you can you can post to Alexa. But really they they stripped out the demand demand side and just focused on that supply side and built a really really good product. So that's that's sort of the way that I typically come to my theses is really just try as many things as possible. Look where develop What what is newly programmable? Is that a way to think about it?
17:21 pretty you know, Bitcoin certainly is programmable money. Websites were programmable kind of were were pieces of software. And now I think you have I I think many many of the teams that are doing a great job in robotics, which you think about as a hardware thing or scientific models that you think about as research-heavy, the teams that are actually really doing well are ones that have a software development experience are thinking about them as a software problem, not necessarily as a hardware problem in robotics. You know, I I think someone had said that Tesla is an iPhone on wheels. I think that that feels true.
17:54 You you can do an upgrade on your on a on a on a Tesla. And it it gets better over time. It gets really a software product as much as it is a hardware product. So, I think I think much more today is becoming programmable. So, that's the other way that I kind of approach it. What are the new things that we can What What are the tools that help us make new things? >> Mhm. Yeah, it sounds like there's there's a real commitment to like the scientific method in you know, finding where the edge of science is and like who is thinking about it really critically. reminds me of how in 2025 you told TechCrunch that your mantra for investing is that to invest in software, you need to know how software works. And I'm curious how you came to that particular mantra.
18:35 >> I would say that's true for me. I There are obviously people who don't code who are amazing investors. So, I think there's probably multiple ways to do this. For me, I am interested in things that are newly possible because of technology. And I think understanding how what the stacks look like, understanding the technology itself, even in the era of five coding, where like I can build Anybody can build anything theoretically. if you are the questions you ask become more important.
19:07 So, if the question you ask is how can I make this thing, that's different than the question of hey, I seeing that there's all these local AI models. They run on different kinds of computers. What is the best way to optimize this for mine? That's more of a question that a a technologist who understands the limitations of a local model and understands that there's RAM RAM constraints would ask. I think those are the those are the kinds of products that get built that resonate with me more. And then I think again, this is just sort of me my my approach is I think there's a there is a very good way to back startups that is market-driven. You back executives who understand the industry. I think a lot about your what what in basketball you have a pivot foot, right? We talk about pivoting a startup. It's like if you keep one foot down if you move both feet it's traveling in a basketball. I think that in the same same is true lots of companies just totally go somewhere and do that's fine. But when I'm thinking about it at the very early stage, I always assume that the company is going to pivot. And the question is what are they going to pivot towards something that I am a sort of fellow traveler and also believe in or are they going to have limitations on what the pivot foot could be. So, if you backed a bunch of real estate executives to build a you know, an AI native thing for real estate they're going to have a fairly constrained you know, you is more likely they're going to find they're going to pivot to something that is still in that same market like real estate than a tech single team who might say the reason we're going after this is because there's a particular shape of the data that is in this industry and maybe it didn't work, but it turns out that that shape is right in the car industry or whatever it is. And then they're going to build a product or is applicable to multiple industries becomes a horizontal product because they see the similarities in other categories. I think my my the companies where that have been most successful that I've been involved with are ones where the pivot foot was about the technology not about necessarily like market understanding of a particular vertical.
21:01 And so for me backing technical founders gives them the ability to move with where where the world goes. >> This is also just so represented in the way that you've structured your own venture fund and the partners that you have as part of that fund. Could you describe a little bit of how the fund works and you know, you have 12 investment partners, but you're a pre-seed seed fund that is unusual. How does that work?
21:32 >> So so I do I set it up I had this idea in in in like 2023-ish that so I probably probably before I leading up to that point of starting factorial capital that the most I loved I wanted to be involved with the technology things that were changing, right? Not brands were online like not tech-enabled but true actual the technology itself. But it was I was pretty technical and it was moving so quickly. And so my thought was what if you set up a fund where you had a structure where you could have venture partners who could help me really quickly and accurately The goal is to come to I think very high-quality decisions very quickly in the early stage. And so can they help me get really smart on where the technology is going? So Clem, the CEO of Hugging Face, is he's thinking about what's going on in the open-source world or PicCollage who founded Venmo is he think he's spent a lot of time in in a number of web particularly in develop developer tools and really understand this stuff and he's building new things all the time and trying to understand this.
22:38 Trying to just but but it's great for finding new companies and it's great for just thought partnership on where things are going. So I I set it the fund up so that I could have like almost an infinite number of venture partners and I set up as a solo GP fund with venture partners so there was there was instead of partners one person it's almost like I'm partnered with every technical founder I've ever I've met it's my I'm close to it.
23:04 at the beginning it was really, really helpful. I think over time I thought that that maybe I would just have like these cohorts and cohorts of adventure partners. What I found was that having a a fewer very deep relationships is really good. And all of the I think everyone who is a venture partner was somebody who I had backed or was part of the portfolio at some point. Like I knew really well. And the in over time what I found was the portfolio of Factorial Capital itself became the referrals. So now I still rely on a small set of people for both finding really interesting new crazy things, but also they're helping me think through other companies. But as an example, I think Clem through Hugging Face introduced me to the Nomic team. They were amazing team. We invested in them.
23:58 and then and then they introduced me to Petronas. And the so the Nomic team then started introduce people. And so you end up And then Petronas and Petronas is a really interesting company. We did out of the seed and then I got really close to them and they raised you know, the series A and then they're they're they're sort of off to the races. And then they introduced me to another founder of a company called Causal Labs. And and so we end up I don't know whether it's a flywheel or a family tree, but it's kind of a a deep network of technical people and that the technology always moves forward. So there's always new people to be able to get to know and to work with and to learn from.
24:39 >> Yeah, I I mean it it kind of sounds like the dream flywheel, right? You find sort of like a once-in-a-lifetime founder in someone like Clem, the founder of Hugging Face. And then via him and other really brilliant technical founders, you know, you then create a network of you know, the most brilliant technical founders in their domain that they are directly familiar with because they are either in the same ecosystem or direct competitors or building you know, kind of compelling products that they are tracking and that are inspiring maybe the way that they're thinking about the evolution of technology. And then they they they are your partners, right? So Clem is your partner. You also have one of the co-founders of Venmo, Hillary Mason from Fast Forward Labs.
25:26 You have this incredible, you know, sort of set of partners that are helping you think about the future of technology and the generations of technology are kind of shorter and shorter it seems every year and so now the next generation of technologists become sort of like the next generation of your partners, which is such a cool flywheel that you've built very unique. >> it's it's it's definitely fun to get to work with with people and watch them really build like pretty amazing things.
25:54 The the we need the other you have so little information when you start writing a ideally were the very first check into a company that any other information I sat I sat I sat across I started so it Hugging Face started out of the fact of of Beta Works office I was there and then when I started Factor Capital I started out of the Hugging Face office. And so sitting across from the person who had built one of the large language models at the large models at Hugging Face was a super early employee and then when she started her company we were able to back him. There was a guy who built the it was it was start it was a big part of the robotics team he started a company we backed him. The CTO at at Factory was at Hugging Face. The And then and then also the other founders like I I work very closely with Clem.
26:37 There's other people like Thomas Wolf, Julien who are so smart and interested in seeing other people succeed and they often are investing and and and mentoring founders so I think it's you know, you sort of ideally fan out from from one core hub to just sort of meet more and more people and and to keep one thing it does is which is just very good technical people who are who want to learn or doing it because they want to see something amazing built.
27:09 >> So, we're hosting a conference in a few weeks, thesis, and it's about the future of work and the future of kind of technology, really. And I'm curious, you know, based on what you've learned as the first investor in hugging face and many other really incredible companies, how what your thesis is today? And you know, we were talking before the pod about how there's this discourse of, you know, some people say that in in the future in a future of AI, you know, it makes sense to be more technical, to be less technical. But you actually had a really interesting take on, you know, kind of specifically for founders in building a company, what the core >> >> skill might be. Could you share a little bit about that?
27:57 >> Sure. and I want to add, I guess, you know, that that the hugging face thing is built an amazing. There's so many other companies from that are built by technical founders like factory and modal that are that are fascinating to me because the speed at which they're growing is so fast. And I'm thinking about why the speed is so quick. And one possible thesis that I've I've been I've been I've been reflecting on is if the if coding is becoming faster, right? Because we can use AI to help us code, then it's been said and I agree that what you build matters more.
28:40 But the what if the job of products product ma- I don't know if it's product manager or product person is really to see the product in the wild and intuit from user behavior, whether that's the data or sitting watching over someone's shoulder, how they're using it so that you can iterate, right? It's like iteration is the most important part of being finding product market fit and and and probably also of of growing. Then if it used to take two months to like see the feedback to get the insights and then to put together the with the product plan we're doing like and code it and then get it back in front of the user.
29:16 It might take two days now instead of two months. And so does that mean that the that person who is I think a very valuable person that whether that person codes or not, but that product person who is really their job is to understand the user and then to be able to translate that whether it's by coding or by by well-defined partner to to build the the next version of it and to iterate. The iteration cycles almost are necessarily so much faster.
29:43 And so it's fascinating to see company like you know, Honeyface took 10 years to get to the point it's at. Then you have a company like Model which didn't they they existed a little bit before I started Factorio but not much. the Factory AI I don't I think that that they we we started probably about the same time. It's only a few years and these companies have grown so quickly and I wonder if part of it is the ability to build and iterate allows these very very fast cycles. And so the result in my mind would be that the ability to generate product insights. Forget about what the job is called. It could be called an engineer, it could be called a product manager.
30:21 The ability to synthesize how a user is using your product and then to improve it I think has got to be exponentially more valuable. It's already the most valuable thing is my guess and now it is exponentially the most valuable thing. Obviously you know, distribution matters more than ever today as well. There's there's a a bunch of noise and and the easier it is to build the noise there is, but I suspect that that's So that's a job whose value is going up tremendously right now.
30:52 >> Yeah. >> Because of the speed of it. >> Iteration as the core sort of variable to deliver speed and make sure that you can >> Your your cycle your your cycle look like this and you compress it so your cycle time is like this and so you just grow much faster than you your work just gets compounded as a function of time. >> It you know I'm just curious how do you how do you filter for that when you're talking to a founder? When you're you know when you you've met a compelling you know technical founder, they're working on an interesting problem, they have a foot that is pivoting, then how do you measure that they can iterate quickly?
31:28 >> I mean usually when I'm meeting them they don't think they're pivoting, right? Because they're like no, this is definitely the thing I'm raising money for. But the the I think that's where the ability to do it is probably more important than what the things you're actually building. So usually I'll try I'm trying to understand what is their insight? And people have talked about earned insights. I like really like that framing. It's like what what is the I think about that is how many people were in a position to learn the thing that you did.
31:57 And then and then to execute it on it because I think that's what earned means. We can all take a look at if if there is some observation that we all have, there's going to be more effort put towards solving that problem which means more competition. If only 10 people in the work in the world have come across this because of a particular category or particular area of expertise, you're just going to have more time probably and more both more time because you'll be ahead, but also you're just you will you you've already spent a lot of time learning that problem and you can go after it directly.
32:30 And so what I think about like an earned insight, I think that's important. And usually that kind of goes hand in hand with the ability to generate new insights as you are in market, which is again another reason have your product out there. I mean, there are people who can like just divine this this what the right thing is. I think there's a story of how Figma like was just basically building in I don't know if this is true, but was building like sort of a loan for a year or two.
32:55 That's not where I've seen company that's not the pattern that I've seen. It's more common that you put something out there and it's pretty cool and something starts to to take off and you just try to delete everything else and and do that thing because you have been able to generate insights before you get that first version. You know how to do it going forward. So, I'm looking for the I think often it comes down to kind of the why. Why why are you why is this thing interesting?
33:19 why is this the problem to solve or why is this a problem or Yeah, that's that's and maybe that the anchor example a good one. It's like somebody is thinking that deeply about how your voice sounds in your own head is probably a person who's going to really really sit with their users and try to understand what problem they're solving. >> And is this Mike McDanel? >> Yeah. >> The best. So great. Shout out to Mike.
33:45 >> >> Yeah. You're I don't remember which one of them said it. And again, I think I talked to you about this is I think I don't know if you remember saying it, but I just remember thinking, "Wow." But like a thing that they did do that I'm I'm positive I remember is there was different onboarding if you had headphones in or not. >> Wow. >> And like think about whether you turn your volume up. It's kind of different now, but pre AirPods, you didn't just like have them in.
34:05 You would like shut down an app if it if it if it just started playing audio. So, to have an app how can you lower the friction of when someone's going to sign up? Letting them do it without their headphones. Having headphones in is interesting. It was also like a very cute way that they they showed the headphones, but that kind of care, I think, pays dividends in a in a consumer consumer product. >> Yeah, something that I often think about, you know, with really great companies is that they end up being kind of like the greatest expression of someone's like best self.
34:39 And I actually think about this with you and your fund because, you know, you have this mantra to invest in software, you need to know how software works. And you were the biggest tinker I know. Every time we catch up, you're working on something, you know, wild or whimsical or like you build software for your own fund. Can you speak a little bit to some of the things that you've built and that you are using? >> Sure. And and so Becca, who works at the E8 Factorio, is also a builder. And so she and I I often build the the mediocre version and she builds like a really good version.
35:18 there are things we use to manage the funds properly, but also I'm building to test and learn. So a product that I built that got a little bit of attention earlier this year was called Ghost Pepper. There's these apps and there are many of these things that you that are used to you hold down a hot key and then you speak like a lot of I go to this where I speak the prompt. And so I was using it and I was thinking, you know, I've seen all these local AI models for for voice attacks. Why do you even need to pay for inference for this thing? Couldn't you just do it on your own machine? So I I went to Claude's code, I think, and said, "Look at the Hugging Face repos.
35:58 Here's a couple that I know about, but find whatever you can and and I want to build something like like this. I use this skill, which I love, called Superpowers, which I think for any product developer, if you add Superpowers, I think it's available for both Codex and Claude. I was using it in Claude. it's it's by this guy Jeff I'm sorry. >> Jesse Vincent. >> it's by this guy Jesse Vincent, who is like a genius engineer, I think, is is how it works, but I just the difference between using this product to build things and not is night and day in my opinion. So, I used superpowers and I had and I did a bunch of had the Claude do a bunch of research with me on voice models. I told it what kind of computer I had and then I built this voice to text tool that I called Ghost Pepper and put it on Hacker News and it ends up going to number one on Hacker News, which I had I never posted anything on Hacker News. I was really excited about it. I open-sourced it. So, anybody cuz you don't have any inference cost. So, it's why not why not let anybody just try this and run it and they can run these things for free.
37:05 and it was it was an interesting lesson both in I I learned you could build a very high-quality product. A lot of people shared out that it was actually faster than a bunch of state-of-the-art products. I think probably because it was just constantly looking for the newest fastest models. But, and I cuz I think in particular with voice to text, I think you can do local works really well. But, the other thing I learned was when you just vibe code something for yourself, you have to maintain it. And I started to appreciate and there's all this talk about the SaaS apocalypse.
37:36 These paid apps aren't that expensive and in a way what you're paying for is them making sure it stays a great product. As long as these I think that the the the job to be done in a way for SaaS apps is to always to continue to iterate and be at the at the edge. But, I I now have a I have a different appreciation for these SaaS products that really do release all the time. and so it's it was a it's it's definitely a fun project. I've I think I don't know there's like thousands of people are using it, which is makes me >> Yeah, I I think I saw it has like over 3,000 GitHub stars. That's so cool.
38:13 I I ran this by some of our engineers to you know and and definitely the maintenance piece is just so so real. I you know, one of my earliest learnings of this is I I vibe coded our CRM tool and used it for months and then I realized at some point that I was spending too much time doing the maintenance of the tool that I had built and you know, and while as in instead I could be spending time with the clients that we have, you know, working on AI implementation and the work that we do on on the consulting side of the business and I realized it just made more sense for us to migrate the data to Atio, a product that just does CRM and data management instead of me trying to play, you know, sort of like engineer, you know, on on the side and try to like maintain this tool. So, maintenance is a really really big part of it. I know this is something that our engineers think a lot about and Monolog, which is one of these, you know, text voice to text tools that we've built in in-house by one of our great engineers, Naveen.
39:17 he, you know, he mentioned that you know, this is certainly like one of the competitive edges and you know, with Monolog for example, latency is something that they think a lot about. So, you know, I think it's like 800 milliseconds of latency for it to load. So, certainly a trend that we're that we're following. so so cool that you've built, you know, Ghost Pepper is something that you have. We're finding these great skills will also include Jesse's skill as a follow up to the pod as a resource for anyone that's that's listening to this. the other thing we were talking about is sort of the the thesis that you have now, right? So, you had a thesis and some themes that you were exploring in 2015 that led to to investing in in Hugging Face. What are those themes or thesis that you're exploring today?
40:02 >> I think I think that so much is still infrastructure. It's it's amazing to me that these developer tools and for the example we just gave is a perfect one. I think that the maintenance they're going to be products that are that help you with just maintain all of this code. I've seen a few. so I think that there are kind of think about the continuation of developer tools, different people are becoming developers, which probably means there are different kinds of tools. I continue to think about frontier non-language models. So I think I said earlier if the if transformers are really good at next token prediction, what are the other areas Turns out that like LLMs have gotten much, much better with more compute. And so if that's the case, what are the other areas where you could predict time series and have it create genuinely new value. and then you know, whether that's my you know, my an obvious example is robots.
41:02 What are they? What is a robotic movement if not a time series information with like the locations of a bunch of servos. And so that's already being People are doing this where they're they're reinvesting in a couple companies with sort of the general thesis that the way that we train robots could change and people talk about whether it's a chat GPT moment. The data's The data is a bit a bit different and so we'll we'll see how it kind of works, but I think that's a category where transformers in terms of AI more generally seem to be really applicable. I'm fascinated by the science models. There's a Hugging Face has got a hackathon they're doing called Rare Disease Real Kid where they've open-sourced a genome and I think doctors sort of notes, the parents of the kid have a very rare disease, I think only 50 people have it. And they have a track for diagnosing it, but they also track for drug repurposing. So people talk a lot about drug development, which is I think interesting, but drug repurposing is another one where that's just kind of fascinating and it's interesting to me that the people who are building this stuff are really technologists. As I said before, it's kind of programmable. And then I think the application layer. And the my thesis around every everyone talks about what we verbalize occasion with consumer applications.
42:12 You said something earlier about latency. I believe that applications are going to look a lot of the people who build the application even if they look like consumer applications or or verticals, the people who build them are going to be technical enough to be able to go fully down the stack. And the example I keep thinking about is in when Instagram era, there were a lot of photo sharing apps and Instagram did a bunch of things right.
42:38 it if you look to the phone, it was like obviously you had a camera and you had a cell phone connection. What was not obvious is that the load time mattered. So if you're standing waiting for something and it takes you 30 seconds to for the app to load, like you're done. So a lot of companies think that Instagram spent time really making load time an important important thing. Instagram I have read I I and I I assume this is true.
43:01 you had the pictures be squares. And the a side effect of the pictures being square, I don't think it was on purpose, but the the benefit is that you don't have to ask the server if you should show the image in portrait or landscape. And so if you can sh- if that shaves off a millisecond of load time, the team that has the ability both to do the great design that like has that nostalgic feel as early Instagram did, but also go all the way down in the stack to be able to shave seconds off of server time leveraging a decision they made on the design side, I think I think there's a lot of parallels to what AI native companies are going to look like in the future where they look at a paper about oh, this is an interesting new way to train on reasoning for for language. What can we take pull from that and apply to images or what can we pull from that and apply to to what we're doing?
43:50 And both the ability to build your own models, but also this humility to know that you know, these other big company models are getting great and if we can just use them for the suffering we use them for, we can then spend all of our time doing the things that are are more unique to us. Being able to balance that and go full full stack means something different now I think than in an AI native world, but but I think being able to do that is going to be a real edge.
44:13 >> As we, you know, sort of like think about the future and along these lines of, you know, the the themes that you're excited about, the companies that you're you're tracking, and as an investor to so many of these, you know, compelling companies, what is some advice that you might have for founders who are, you know, trying to navigate this like really dynamic environment, if you will, where the technology's changing so quickly. what are you seeing really great founders doing right now and what advice would you have?
44:44 >> the best people that I've seen building things are like playing. We're in an era that reminds me of early internet. Everyone is playing. There are so many fun tools. Like I've I tried this text-to-CAD thing. I don't have a 3D printer, but I was like, I bet you could vibe What what how how good of a job could I vibe make a robot? We're not there yet, I don't think. you can 3D print your robot and you can vibe make some 3D 3D structures. But probably we will be and I think that the best thing I think that the way that you get the best ideas is by just playing and tinkering with this stuff and seeing what the problems are and then solving them.
45:22 and you know, pre-this pre-2016, like the pre-AI era, it was really the the mobile era. And there was a lot of fun things to play with. And then there was this question mark period where it was, okay, the phone is at scale, lots of people know how to make iPhone apps, distribution is getting harder, and AI hadn't quite become what it is today, which I think is a I thought AI was going to be sort of at least as big as mobile. I think it will turn out to be bigger than internet.
45:56 and which I think is is it was sort of mobile was not quite as big as internet to me, and then I think it will be much bigger. The The again for the kinds of companies that I am most excited about playing with this stuff, finding out what's newly possible, and seeing where those gaps are, and then start to fill them solving your problems, whether it's problems in your life or problems for a friend's company, whatever it is, is it's how I would be spending my time.
46:20 And I think there are maybe the same number of problems that have always been, probably more, but there there's definitely more solutions to existing problems than there have ever been, and this solution space is un- so untouched relative to I'm surprised all the companies that aren't being built. I think they will be built. I mean, there was as many products as there are getting built. and so I think I used to play Should be I think it was a role-playing or something, and you would move around, and the the whole world was dark, and then as you would run you got to see the world. I think we're in that right now, where everyone, even the people who it seems like you have people building these AI models are still sort of exploring what they can do, and we're all kind of exploring what's possible, and hopefully, I don't know about our focus certainly is is what are the solutions that are are newly possible, maybe it's to existing problems, maybe it's to brand new problems. And I think that there are numerous, so it's an amazing time to be exploring that and building new products.
47:18 >> It really is so fun. talking about games reminds me of one of the companies that spun out of Every, led by Alex Stuffy, who literally built a gaming company because he was basically testing the capability of models playing a game against each other, and playing different games against each other. And and you know, it really does come from this like spirit of exploring what's possible, and then just seeing how far you can take you know, where that leads you.
47:48 >> obviously the best way to explore what's possible is to listen to Every so that you know what's newly There's never been a time where there was more newly possible stuff every week, so that's >> >> That would be my personal recommendation. >> Great, you're on the team now, Matt. Welcome. all right, well, maybe just to close, Matt, any resources that you're really excited about reading, you know, that like, you know, either it's people that you're following, you already shared Jesse Vincent's Superpower Skills, we'll link that below. Anything else that you're, you know, kind of like looking at day-to-day that's really helping you stay up to speed with everything that's going on?
48:23 >> I mean, there's so much that's shared on X, I think it's actually really There's a lot of noise, but there's a lot that's really good. I look at what's trending on Hugging Face. I am fascinated by open source, so I try to find any resources or open source. there's newsletters that I like. I think Ben's Bites I I think is great. I think the Every Newsletter is great. I'm One of the reasons I've been more and more sort of reading X again is that people are writing these longer-form content pieces, it seems like Medium content sort of moved over to X, and like you can take this stuff and put it into Claude, and then take how can I apply this to my life, or ChatGPT it, or to my investment thinking, or to my company, or whatever, or to my How can I take this insight about model memory and and incorporate it in my mask? It's fascinating.
49:09 so they go I kind of probably have a barbell approach. I have what's new new new, and then I read things that I forget I forget what it's called, the rule where shh the longer something has been around, the longer it'll be around for. And so I'm reading just things that are older, you know, I've been reading one book about like that follows the Roman Empire, and things that are hopefully timeless.
49:39 because there's probably feels like we're in a moment where certain things are repeating themselves, and whether it's technology, or whether it's culture, or human behavior, I find it useful to look at some of the other stuff, too. I know that you we've we've talked about some of that as well, that looking at adjacent not not just only spending time looking at the most recent technology can often end up being informative to the technology. >> Yeah. Thank you, Matt. It's so fun to have you. You're one of my favorite thinkers and investors. Where can people find you?
50:11 >> Matt Hartman on X. I'm you can email me at matt@factoralcap.com. And yeah, thank you so much for having me. It's been been a fun. >> So treat Yeah, thank you. Such a treat. All right, thanks, Matt. >> If you like this video, you should check out every.to. We have a 2-week free trial for anyone who's watching or listening at the link in the description. Every is the only subscription you need to stay at the edge of AI. It is the one subscription that if you're super AI pilled and you wanted learn how to use it to do the best work of your life, it's got everything you need. It has education and equipment. On the education side, we review every new model that drops and tell you what's good. On the equipment side, we have software tools specifically made for you, like an agent that helps you AI pill your whole company, all bundled together under one subscription. So, get 2 weeks free now at the link in the description.
Summary
- Google and Meta have released new AI models, Gemini Argon 4 and Wonder, respectively.
- Both models excelled in benchmarks, but practical performance varied significantly.
- Gemini Argon 4 is noted for its writing capabilities, doubling productivity for the user.
- Meta's Wonder struggled with coding tasks and showed limitations in writing.
- The discussion emphasizes the importance of iteration and understanding user behavior in product development.
- Matt Hartman shares insights on investing in AI, stressing the value of technical knowledge and earned insights.
- The podcast encourages founders to explore and tinker with new technologies to identify gaps and opportunities.
- Resources for staying updated on AI developments include newsletters and platforms like Hugging Face and X (formerly Twitter).
Questions Answered
What are the latest developments in AI models from Google and Meta?
Google and Meta have launched new AI models, Gemini Argon 4 and Wonder, respectively. While they topped benchmarks, hands-on testing revealed that their performance may not translate to practical applications as effectively as expected.
How should founders evaluate the success of their products?
Successful founders focus on long-term retention and genuine value creation rather than just top-line metrics. They adapt their strategies based on what works and what doesn't over time.
What factors contribute to successful pivots in startups?
Successful pivots often stem from technological insights rather than just market understanding. Founders who can adapt their technology to various industries tend to thrive.
How can founders demonstrate their ability to iterate quickly?
Founders may not always recognize their need to pivot, but their ability to iterate quickly is crucial. Understanding their insights and the uniqueness of their experiences can indicate their potential for adaptation.
What are some promising areas for AI application?
AI is increasingly applicable in robotics and medicine, particularly in areas like drug repurposing and rare disease diagnosis. The involvement of technologists in these fields is crucial for innovation.