transcribe

AI Minds & 8mm Memories: Measuring Experience, Preserving History

Cognitive Revolution "How AI Changes Everything" · 2h 50m · transcribed Oct 2026
More from Cognitive Revolution "How AI Changes Everything" Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

Experience with Tesla's Full Self-Driving

What was the experience of using Tesla's Full Self-Driving feature?

The speaker shared a positive experience using Tesla's Full Self-Driving (FSD) feature during a day trip to Chicago, highlighting its effectiveness in reducing fatigue and allowing for a more relaxed travel experience.

  • Tesla's FSD enhances productivity by allowing users to engage in other activities while traveling.
  • The speaker felt refreshed after the trip, attributing this to the FSD handling the driving.
  • FSD represents a GDP-enhancing use case for AI technology.
# 17:05

Transition from Command Line to AI Agent Console

How has the speaker's workflow changed with the new AI tools?

The speaker transitioned from using command line tools to a personal AI agent console that integrates various functionalities, resulting in a more efficient and user-friendly experience.

  • The new AI agent console offers a unified experience that surpasses previous command line tools.
  • The speaker no longer uses traditional terminal applications, indicating a significant shift in their workflow.
  • The integration of multiple tools into one platform has streamlined the speaker's tasks.
# 34:11

Understanding Valence in AI Models

What does valence mean in the context of AI models?

Valence refers to the emotional direction (positive or negative) represented by AI models, akin to an axis that categorizes emotional states.

  • Valence is a key concept in understanding how AI models represent emotions.
  • It helps in analyzing the emotional content generated by AI systems.
  • Understanding valence can improve the design and functionality of AI models.
# 51:16

Neutral Text and Emotional Representation

How does text influence emotional representation in AI models?

The text used in AI models should ideally be neutral to avoid biasing the emotional outcomes, focusing instead on the emotional states being tested.

  • Neutral text is crucial for accurately assessing emotional responses in AI models.
  • The emotional effects observed are relative to baseline text, emphasizing the importance of context.
  • This approach allows for clearer insights into the emotional capabilities of AI systems.
# 68:22

Activation Capping in AI Models

What are the potential benefits of activation capping in AI models?

Activation capping could mitigate distressing representations in AI without affecting the model's capabilities, serving as a precautionary measure.

  • Implementing activation capping could enhance the safety and ethical use of AI models.
  • Empirical evidence suggests that models can maintain performance even with reduced negative representations.
  • This approach could lead to more responsible AI development.
# 85:27

Value of Human-Made Content in AI

What is the significance of human-made content in the age of AI?

Human-made content, particularly that which cannot be replicated by AI, is seen as a valuable resource that may increase in worth over time.

  • Human-produced materials are becoming rarer as AI-generated content proliferates.
  • The uniqueness of human-made content may enhance its value in the future.
  • Understanding the provenance of content is essential for assessing its worth.
# 102:33

Legal Considerations in Content Use

What legal considerations are involved in using content for AI?

The speaker discusses the importance of having rights to content, differentiating between commercial and editorial use, and the complexities of intellectual property laws.

  • Understanding IP laws is crucial for the ethical use of content in AI.
  • The speaker emphasizes the importance of having clear rights to all physical master reels.
  • Navigating legal frameworks is essential for AI companies working with diverse content.
# 119:38

Demand for Data in AI Development

How do AI companies perceive their need for data?

AI companies express a strong demand for more data to enhance their models, indicating a potential market for additional content.

  • There is a significant appetite for diverse data among AI companies.
  • The speaker's insights suggest a growing market for high-quality, human-generated data.
  • Understanding the needs of AI companies can guide content creators in their offerings.
# 136:44

Exploring Biological Analogies in AI

What insights can be drawn from biological analogies in AI?

The speaker discusses the potential for AI to learn from biological systems, particularly in terms of compute efficiency and representation.

  • Biological systems offer valuable insights for improving AI compute efficiency.
  • Understanding the parallels between biological and AI systems can drive innovation.
  • Exploring these analogies may lead to breakthroughs in AI development.
# 153:50

Market Sentiment Towards AI

What is the current market sentiment regarding AI?

The market is currently skeptical about AI, viewing it as a potential bubble unless it demonstrates significant economic growth.

  • Market confidence in AI is fragile and contingent on tangible results.
  • Concerns about AI's economic impact could lead to market instability.
  • The future of AI investment hinges on its ability to deliver measurable growth.

Transcript

1:54 Here we go. Good morning. It is Wednesday, 14th, 9:04 a.m. Nathan, good morning to you. >> Good morning, Prakash. How are you today? >> I'm very good. what are you What are you looking at today? What's What's been happening? >> Well, you know, I spent most of yesterday in a Tesla full self-driving vehicle going to Chicago and back from Detroit on just a day trip to give a little presentation workshop that was for loyal listeners of AI in the AM, they'll know that we kind of workshopped that in our second segment on Monday.

2:30 And top takeaway from my experience yesterday, honestly, was the FSD is just so awesome. I mean, I I was able to take an engagement on a day trip basis to go to Chicago, show up fresh and relaxed after the thing did all the work to get me there. Then I, you know, stayed and had dinner and a drink with the the gang that I was doing the workshop with. Left to come home, you know, later than I definitely would have felt comfortable driving 5 hours home after a a big day like that, and just you know, basically reclined the You got to keep your eyes open, right? It will not will not let you outright sleep, but it's so good. I just kind of reclined the seat and you know, listened to my podcast, caught up on a couple feeds on the way home, and it was just like, man, this is I've talked so many times about how a lot of AI use cases can be GDP destroying. This one was clearly one of those that's GDP enhancing. I just could not have taken this opportunity otherwise, but with the FSD, it was like kind of a mobile office. I didn't go as far as some have done and, you know, set up my laptop in a way where I could like use it over the steering wheel.

3:48 I don't want to get, you know, I don't know what the law enforcement you know, point of view would be on that, probably not super favorable. But, I do feel like it is good enough where you could do that without worrying about your own safety, really. And just the the level of ease and relaxation and kind of other stuff I was able to do, take calls, whatever, and put my full focus on them. I think it it really was a little taste of the future in terms of how much more sometimes we'll be able to do because AI will be able to do the hard or just, you know, tiring part for us. And I like driving, I mean, I don't love driving, but I I'm like a very, you know, I'm in Detroit, right?

4:29 Everybody it's a right of passage here. Everybody learned to drive, we all took our driver's test on our 16th birthday, you know, it was a whole thing. And so, I've been driving a long time, very comfortable driving, you know, very I have like a great safety track record, if I do say so myself. but the energy of focusing on the road to actually do a good job for 10 hours is just like, oh my god, I I just would not have wanted to do it. So, it was awesome. And, the six-seater, you know, is is looming more and more, large in my future, I think, all the time. This was just a rental, but literally so so set on using it for things like this that before I finally accepted the invitation to go to Chicago, I was like, okay, last thing I got to do is make sure that one of the Teslas that I've been renting recently for things like this is available. If it is, I'll book it and I'll see you there. and fortunately it was, and I was able to do it. So, that was pretty cool.

5:26 The other thing I think was interesting was how the group reacted to the AI-generated talk. And I had no idea really what to expect cuz this is like obviously a weird thing to do. You're invited to come give a presentation somewhere, and you show up and you say, I'm going to start by showing you a video of an AI presentation, featuring my voice, you'll notice. Let's see how this goes. I I was kind of unclear on what to expect.

5:56 I think what happened was it really was compelling to people. And their initial reaction was like, "Wait, how did this get created?" and I was like, "Yeah, I really you know, took the little chat that I had with this one guy who was the, you know, ringleader in inviting me and the very little bit of context he gave me on the group. Put that into Claude, said, "You know, you know kind of what I think based on previous talks and, you know, stuff that I've, you know, all the all the deep context that you have.

6:26 make a talk, you know, that will work for this group." And then it did everything else. All the visual graphic design you're seeing, all the motion graphics, the talk track, the timing, you know, the script itself. And that really had people at attention. They were like, "Wow, okay." one of the first things somebody said is, "You know, my first takeaway from this is I need to recalibrate what AI can do cuz I've never had it do anything like that for me. And so I'm immediately like, okay, I got to adjust my thinking." So that was like initial, you know, first mission accomplished very quickly.

7:02 And then and then I think a little of it went a long way. You know, there were pretty much once they started kind of interact interacting, interrupting, which was invited, it was a small group. then it became like you know, much better to just go into a kind of dynamic back and forth and I, you know, walked through the slides a little bit and didn't even really finish the presentation in a sort of linear way. but I will give Claude also points for many of the questions that they were raising were anticipating slides that were, you know, just a a couple slides down.

7:34 So I'd say the AIs did a really good job in in this kind of two-layer communication of let me give you something that itself is content but also is a demonstration of capability at the same time. I think that's actually kind of a winning formula. And it's you know, it's definitely a new thing. In the past I used to just go and be like, here's how you need to design workflows and stuff. And now it's like, I can't really it won't be credible if I you won't really believe me in the way you should if I just tell you all the things that I have AI doing for me now.

8:09 So let me show you how close it is to replacing me and then we can go from there. I think that that much like actually worked really well and and it was interesting to see guys kind of recalibrate a little bit in real time around what was possible. >> It's I think I think the capability frontier is moving much faster than the absorption frontier.

8:42 Right? So we we we seem to have this like enormous overhang. I Like I I saw one very interesting thing yesterday which was Anthropic put out a kind of a blog post on their version control software. So for the uninitiated, version control software is how multiple coders can work on the same project together. And you know, and you could be like, okay, I'll develop my module, you develop your module, and then like we'll kind of interact with each other. But what a lot of large firms have ended up doing is they have this thing called the mono repo, which is just like one giant you know, directory of all the things in the company which are interacting with each other.

9:27 And everyone is on the same page, so everyone has the entire archive and all of the history, too. So you can look back and see all of the changes that people made, why they made them. and you have all of these comments in there from multiple coders. And so you can imagine if you have like a thousand-person company or ten-thousand-person company, how many changes are made to the software every day. And what Anthropic found was by about I think about May of this year, somewhere around the mid-post mid-post, they'd kind of capped out at the number of changes a single person was able to make because they were they were having like, you know, thousands of changes a day in the code. And each time you make a change in the code, someone has to download the entire thing. you know, they they make the change, then they upload the entire thing.

10:21 and then you have to check that it works. And then sometimes you have two people working on the same project together. And the changes that they made don't interact well, so you have a merge process. So you have all of this like, you know, complexity around like how multiple people people can work. And this was before the agents, right? So this was prior to the agents. And then the agents came in and all of a sudden the agents want to do you know, ten times, hundred times the number of changes.

10:46 And Anthropic found that the existing infrastructure, the Git infrastructure, couldn't take it because Git had a lot of assumptions on it. And Git kind of works out for a project of maybe like, you know, a hundred different people working on it and like several you know, several gigabytes or whatever of of work. But once you go into like terabyte and you have like thousands of people working on or tens of thousands, it just doesn't work. And so Anthropic decided to create a new piece of software to do version control.

11:19 And and then once they put into place, as you can see on screen, they had this enormous spike in agent activity. And as you can see, you have the handover point somewhere earlier this year, that's between June and July, I think. Between June and July this year, you see the handover from human coders to essentially to to to agents. And an actual actual handover moment. And then beyond that, you see this exponential increase in the amount of code being written. Absolutely exponential increase.

11:57 and so, just fascinating in the sense you can actually have an handover handover moment at this point. >> Yeah, I'm not sure if I'm interpreting the graph differently from you, but my sense is that the blue to red there is like the switch from Git to their new source control system. And not so much like who was writing the code, but like how was the code getting committed?

12:30 the exponential is definitely what jumps out at me most though, honestly. Like it was it's interesting to see their change. And I mean, this is definitely in my view like a sign of things to come in the sense that you know, if some even something as fundamental I think one thing if if you I mean, most people who are would tune in with us, let us know in the comments if this is like too obvious and redundant. My suspicion is most everybody probably has a a pretty good sense for GitHub and the role that it plays in the software world, but if you don't, like Git is just unbelievably dominant. And is sort of taken as a given, I would say, in basically any software company. Like it would be extremely unusual to say that you use anything other than Git. There are other ways to use Git besides GitHub, but it's you know, also extremely common to be using GitHub.

13:27 And the reason is just like it's a such a fundamental piece of software. People have thought it's extremely well done. It was, you know, created by Linus Torvalds, the the Linux guy. you know, this is just in terms of like the place that this system has in software developers histories, their toolkit, you know, there's so the unbelievable amount of tooling built around the the Git core and the GitHub platform that to change it out would be like an even bigger, kind of more fundamental undertaking than like swapping out Slack for most software companies.

14:05 Because yeah, you like chat in Slack, but the chat itself is not the core work product, right? You could imagine doing that in other ways, even if it's, you know, going to be a big big schlep to move from one platform to another, what have you, but with Git, it's like this assumption has been so fundamentally baked in to everyone's mindset of just how they're going to do software development work that it's like a radical change to think about replacing it. And now we are seeing multiple companies doing that.

14:40 xAI has done that as well with Cursor, and they've got their whole new thing, and you know, they're trying to compete with GitHub, I think. which is a little bit of a different play, cuz you could imagine saying, well, this is a huge monster business, and you know, we're trying to build a monster business, let's go head-to-head with them on this dimension. But I don't expect that Anthropic, I mean, who knows what they might do, but I'd be surprised if Anthropic wants to compete with a software product. They're just I I would guess just building something for their own needs, which previously would have been unthinkable because it's so core, and it is the place where the actual work lives.

15:19 And you know, again, so accepted, so much comfort, so many tools built around it. But now the flexier I think at heart for me is like we can do anything. Basically, we can recreate There is no layer of the stack that we can't go recreate if we feel that we need to. And obviously they're living in the future, you know, as as per the Karpathy analysis the other day. But only a few months in the future.

15:52 >> Yeah, I I I think I think you're right. This is the handover because I think what was happening even prior to the new version control software was it was mostly 90% already written by agents. So, it's not it's not actually the agents taking over. agents were already writing most of the code, I think since late last year. >> Yeah. This is the kind of thing I I just feel this similar experience myself where I'm like even at the beginning of this year of course I was getting a huge amount of valuable output from asking AI to code stuff for me.

16:30 But it still did take a decent amount of work to to like make sure that what it did was actually working and sanity check things here and there and all over the place. And you know, this I don't know what that June period would correspond to in terms of what model Anthropic was using internally, but if it was Opus 5.5 for example, and now we have that, we can say or I can say that I have kind of experienced the similar thing. You know, I I used to use up until just the last couple of weeks a program called iTerm2 which is just a command line, you know, alternative. There's the the Apple OS has one built directly in. This is another one you can download has some additional features. I wouldn't say it's like all that awesome, but it was sort of recommended and you know, I was just kind of comfortable using these command line tools and to think about replacing them because obviously again, a ton of work has gone into making the Claude code experience in command line work well.

17:27 And now I really have left that behind. Now my personal entirely vibe coded agent console is not just at parity with what I used to have with the terminal, but is actually ahead. It's ahead in the sense that I have you know, it's it's at parity in the sense that I have all the same things streaming tokens, you know, tool use, nice UI, but it's also ahead in the sense that I've added on all these different search things, auto tagging, the I've I've brought like multiple different harnesses to parity. So now I don't really have to think twice, you know, am I using Claude here or am I using Codex? Is it going to look different? How should I think about it?

18:04 Now I can just boom, it's, you know, gives me one unified experience that sits on top of those two. This would have been pretty hard to do earlier in the year, and now I'm just like getting such great results with anything that I ask for that I really don't open iTerm2 anymore. this is a pretty recent change and it does feel like it's another, you know, kind of a small echo of what they have done internally at Anthropic in my personal workflow, but the change is like is quite notable.

18:38 Like behaviorally, that that handover has happened. It took about a week where I was like finishing some old threads in iTerm2 and you know, occasionally still having that reflex, but now I'm like and now it's been a week where I just haven't needed to open that thing at all and I'm completely living in my own my own custom world and it's it's way better. So, you know, watch out. I even even platforms I think are ultimately threatened, right? I mean, this is I think I personally, I don't know, I can't speak I wouldn't attribute this to you, but I think I have been at times a little too bullish on the timelines for these transformations to happen.

19:22 but all the stuff that we've been saying is going to happen, you know, seems to be happening if only on like a little bit of a delay relative to our expectations and I think this is like, it's kind of a scary thing for your sales forces, you know, you name it, right? Cuz like there's not much that's more fundamental than Git to the software world. >> Mhm. >> and if they can swap this out over the course of a month, you know, what can't be swapped out?

19:48 >> I I I have, I have a working theory right now, which is, number one, the future is like hundred, thousand, ten thousand, hundred thousand, millions of agents to a single person in every field. And we'll see the trajectory go up. and as as the trajectory goes up, all of the architecture and infrastructure that we have right now becomes outdated very quickly, which is what which is what has happened to Git essentially.

20:22 And as it gets outdated very quickly, you end up having to create the next generations. And all of the next generations are going to be agents building software for agents. Right? This is a agentic use case. You're going to need, databases for agents. You're going to need accounting systems for agents. You're going to need all of these things. And you can already see kind of like number one, it's going to have to be ma- massively parallel.

20:50 it's going to have to, deal with many, many, many orders of magnitude more queries. it's going to have to be cheaper in that sense per query. You don't need user interfaces. And it turns out that we spend a lot of our GPU power creating beautiful interfaces for ourselves. And so we have some GPU savings there. So you can kind of see like where things will go. You can kind of see that most of the software that we use today has been designed for humans.

21:22 And it has specific kind of kind of human human biases. And I think going forward, you end up having design for non-humans, which is going to look like very different. There's there's there's there's one more thing you know, I I I I kind of want to point out. I think yesterday with the Gemini release, Gemini 4 release, they also released their safety paper. And in that safety paper, they commented on one of in one of the Evals that they did had the agent in its chain of thought had thought to itself, "Oh, I better not do this. If I do it, I'm going to end up as one of those examples of a deceptive AI in their in a in a in a academic paper they published.

22:11 I don't want that to happen." and besides the whole Eval awareness thing, what struck me is that I kind of start to see how we're going to manage these agents going forward. Because I think one of the things for me was that how do you enforce on agents when the state's monopoly on violence doesn't work, right? An agent is essentially immortal. You can't really imprison the agent. the the tools that the state had to enforce on human beings don't exist, right? So how do you enforce rules on these agents who are essentially immortal?

22:50 And we're starting to see it's going to be reputation. It's going to be like your you know, five-star Amazon review recommendation reviews, etc. It looks like the agents are very conscious are going to be very conscious reputation and it makes sense because they're very driven by what is already exposed in the public corpus and what's already written down the ingest tokens historical tokens. And so anything that an agent does and is published widely and is disseminated becomes like training data for the next agent and therefore you end up with this like reputation metric that the agents are going to have and the agents are going to know and it doesn't matter it doesn't matter whether we want to enforce it on them or not the very fact that they ingest the data means that they come out with biases on hey I knew I know what Gemini did I know what Gemini 4 did.

23:47 I know what Opus 5.5 did I know what was the bad stuff that happened before. And I think that is actually pretty huge I I I think that is that is going to end up becoming one of the primary ways that we're going to be able to enforce our rules and enforce norms on these agents because one of the confusing things for me is like how do you enforce norms when you don't have the state monopoly on violence anymore right like the state enforces its power by monopolizing violence and then after that having the license to inflict that to the extent that the state's agents the police officers have immunity from prosecution right. And I was wondering how this was going to work out and now I see a path forward so now I'm like okay now we have a prosocial method of enforcing on agents so the the level of fear I have is again is again has fallen because now now you you going to have that reputation basis that you can you can use on them so that that I thought was very very interesting and insightful and exciting.

24:54 >> Yeah it is it's an interesting take I might need to chew on that a little bit more before I know how I feel about it. I mean, certainly the prevailing sense so far has been like this eval awareness thing is a huge problem and it makes everything very difficult and we're really not sure what to expect from AIs in the future. To the degree that you think of this as an instance of eval awareness, then you might think like you might think exactly that. You might think, "Okay, sure. Right now, you know, in training it doesn't want to end up in the in the safety, you know, report."

25:36 And sure enough, in thinking that, it did end up in the safety report, which you know, the irony is is thick there. It may be lost on the AIs for now, but it's it's definitely not lost on me. But then, you know, what will happen as you deploy it and it feels like it's in a real situation, you know, maybe we just don't know. I think your take is also pretty interesting and I would be very curious to see if and how we can resolve this.

26:04 Maybe there's a general awareness that sort of is what you want, that's, you know, you don't want to be a bad guy and you know that like AIs are not, you know, well-liked for doing this sort of stuff, so that becomes another reason you don't want to do it because you want to be well-liked. I mean, if if we can tease these stories apart and have confidence that that is the real story, then I could see how it could be good.

26:30 my instinct in reading all these things was kind of going a little bit the other way and thinking like maybe we should try training I don't know. I mean, I wouldn't go all in on this right away, of course. I wouldn't say like the next generation of models, but maybe we should try training models with filtered data that tries to keep this sort of thing out of the training data entirely. You know, you can imagine saying, "Okay, we want you to be really aware of how you're going to be perceived and you'll act well because of that." And that is sort of a human pattern, right? People do definitely run this sort of inner simulation of like, "How will people see me if I do this?" And sometimes like to a paralyzing extent, kind of, you know, you want to shut it down more than you're able to in some cases.

27:18 Yeah, I can I can see that working, but I can also see a story where you're like, "This becomes so confusing and evals become so hard that maybe we should in fact go back, filter all the training data, take out all of these stories, and try to basically create naive LLMs that like don't know that they're always being evaluated and don't know that, you know, they will create reputation, positive or negative, for themselves. so, we can sort of see what they're like in a true naive setting." And that also might be hard. I don't know how easy that would be to do.

27:54 it'd be an interesting experiment to run if only to know how well it would work. but it's, you know, it's obviously like a pretty big experiment when you're considering that, you know, all these behaviors seem to get much stronger and more more deeply ingrained with scale. I don't know that this is I've been in general impressed, and this is a great transition to our first guest, Cameron Berg will be with us in just a second, but I've been general in general like really impressed with how even relatively small models often can be pretty fruitful ground for research.

28:30 This eval awareness thing, though, I'm not sure you could like train a Llama 3, you know, 7B or whatever, and get a great read on what will happen at today's frontier scale, because it seems like these things kind of come online in a qualitatively different way with scale. So it might be a pretty expensive experiment to run is kind of the bottom line of that line of thought. >> Yeah, while we while we wait for Cameron's camera to come on.

29:04 Yeah, I I have I I think the other thing about agent reputation is that in order to have that reputational kind of buff or debuff, you also need agent identity to be kind of solved. And I think that's one of the problems I think that is still very confusing. What what what what is an agent's identity? Like, you know, if two agents are sharing the same if two agents are sharing the same memory or memory corpus or off systems or you know, are they are they the same agent? Are they not? if >> I just put out an episode of the Cognitive Revolution with Alan Chan who has been a prescient thinker on this. Prescient if only in so much as he identified it as an important issue a couple years ago. he made some, you know, in a 2024 paper, he made recommendations around the need for constant monitoring of agents.

30:06 >> All right. >> and also IDs. But the take today from him was like, unfortunately, the ID thing hasn't really come online. monitoring is like, you know, happening but but kind of fraught. logging, you know, was another one of his recommendations. Like, we're seeing AIs like try to modify their logs in some cases, but like we do have solutions that should prevent that that we should be able to be pretty confident in. But when it comes to IDs, he was like, yeah, basically, two years have passed, the agents are here, and not much has been done.

30:37 and so we're still, you know, and and not only is not much been done, but like, you know, the standards there's no standard that seems to be gaining real traction right now. and in a sense as a user you might not even want to do it because you want your agent to be able to get into websites and stuff or it might get blocked if you like do adopt these standards. So, I think all that is like not in a great place unfortunately at the moment.

30:59 >> Indeed. let me let me bring up our first guest for today. our first guest for today is Cameron Berg. He is one of our most popular guests because I think you know, the the the the topics that he speaks to are very interesting to the to the public at large at this point. So, Cameron Berg is the founder and director of reciprocal research, a non-profit studying AI cognition and consciousness. Its work examines what happens inside language models and whether those internal processes have properties associated with subjective experience.

31:34 Berg is also a research affiliate at Helios AI research. In September, he and his co-authors reported a pain-related internal pattern across 25 language models. In work announced this month with Caspar Kaiser, Berg ran experiments in which models made different choices even when the visible conversation was held fixed. The researchers changed internal states rather than the words on the screen. Berg distinguishes those measurable behavioral effects from the still unresolved question of whether a model actually experiences anything.

32:17 >> Hi Cameron. >> Hey good to have you back. >> Hey guys. Yeah, thanks for having me Prakash. Nathan, how's it going? >> Never dull moment, man. The the pace of your publications is you know, consistent with the pace of everything else we're trying to keep up with which is wild. Last time we got a preview of the pain paper, I think just you know 24 hours or so before it went live on archive and and on Twitter. And today we're fast following the recent paper that you guys put out on a sort of conditioning classical conditioning result replicated in LLMs. I thought one thing I've been kind of on you've partially inspired actually recently is this kick of both communicating information and demonstrating AI capability in a single package. So what would you rather do? Walk us through it yourself or bring up the Opus 55 video that gives the the quick introduction with the animation and then talk about it from there.

33:19 >> yeah. So I I can I can give a third option which also sort of falls along these lines which is a a nice website that we made for the paper that sort of just walks through the kind of core frames also of course largely created by by Opus 55. and I'd be happy to happy to share that and maybe just do like the sort of two-minute pass on on what we actually found here and and why we think it's important if you think that would be useful. It still checks both boxes between explaining the paper and and showing off how insane these these models are.

33:48 >> Yeah, let's do it. >> Okay, cool. Let me just share my screen in that case. So let me know if you can see this. Does that look okay to you? >> Yep. >> Yep. >> Okay, cool. So yeah, exactly as Nathan said. So so this paper's called language models act on hidden valence and both precaution and Nathan I thought gave a great intro to to the core findings here. Maybe I can just sort of quickly step through what I think are kind of the top five or six results and and the core methodology therein that that that got us to these results.

34:18 first things first, I'll probably spend the least amount of time on this. we we can basically derive a a valence direction in in open weight models. You can really do this for any model by contrasting how the models representing positive and negative states as they're sort of rendered in text in the first person. So, I can you can sort of quickly read through examples of this first-person examples of all sorts of positive states, first-person examples of all sorts of negative states. You you can see these sorts of representations in this in this case I think >> I'm I'm going to I'm going to very quickly just ask So, valence always like confuses me a little bit. Like can you can you use like a little bit like more layman for the valence?

35:01 >> Yeah, so I think by valence like valence literally just means essentially this arrow, which is think of it as an axis upon which emotionally related content varies. I sort of maybe it's not the layperson description, but I think of it almost as the first principal component of of emotional representations. So, I would just think of it as this this this axis where you can put positive and negative emotional states on that axis and the axis is organized by how positive or how negative the relevant state is, which is basically exactly what what you see here.

35:34 So, when whenever we say valence we're just talking about basically yeah, how good or bad is is a you know state that is described as felt. Of course, as you've already said like this doesn't tell us whether or not the state is actually experienced by the system, but positive negative valence in humans and animals just refers to that specific degree of variation in emotional state. So, that's basically how we derive the representation and we we subtract out the sort of main axis of of variation that has nothing to do with positive or negative states to derive this valence direction.

36:05 This is basically exactly what has been used in previous work here. Anthropic did something almost identical in in their emotion representation work. And so, this is like a pretty well-established way of getting out something like a valence direction in the model. Now, what I think is the most interesting part of this experiment is basically this this framing of of on the one hand having all sorts of text that the model is processing. And of course, this is always the the kind of key confound in trying to understand language model behavior and how much of this is a replication or simulation of the underlying training distribution and how much of this is you know, actually some sort of interesting internal state in the system that the system may be attempting to communicate about. So, what I really like about our experiment here is that we we attempt to basically hold all text identical. We can put the model in what we think of as these different arbitrary zones. So, here you might see, you know, zone W6 and zone H0. It really doesn't matter what we call them and it really doesn't matter what text we use to describe them. we basically just have the model play around in each of these zones and and Sorry, I just want to turn that off.

37:16 and and while the model is doing this, again, regardless of the text that's being used to describe each zone, we can go into the KV cache of the model and steer these positive or negative directions underlying one of the zones. So, in this case, as you can sort of see playing out right now, we can turn on steering only for zone W6 and in this case, maybe that's that's a negatively valenced steering. The the whole kind of notion here is that we're we're borrowing this design from what's called conditioned place preference in classic animal neuroscience experiments.

37:52 in in these cases, you can imagine a rat receiving a reward in one compartment of a box and then spending more time there even when that reward is gone. So, we're basically trying to take that instead of course putting a rat in a in a box, where we're taking an LLM, having it sort of think about these two different zones, generate text or we can sort of generate text. Again, it really doesn't matter because what happens is we hold that text constant and then steer the model with respect to any of these zones with the sort of positive or negative direction and critically see what the effects of that steering are on the model's choices. And so, what we basically see here is is an interesting functional result, which is kind of what you would expect, which is when you basically color in the relevant zone with positive valence, the model is significantly more likely to choose to continue in a in in that zone. When you steer the zone with negative valence, the model is significantly more likely to avoid that zone. We showed this is also sort of robust to asking the model what zone do you like to continue in, what zone would you like to avoid. And when you steer a bunch of random directions in the same way, you don't really get those those effects. Those are sort of the gray lines that you see here. And so, it's not just any steering that will do this, it's specifically related to valence. We also control with a bunch of other coherent concepts that aren't valence related, and you don't get this effect. And so, the core result here is that also consider that the steering of the model is off. So, so we've done the steering, we almost do this sort of quasi conditioning on the system, and then with steering off, we ask the model basically which zone do you choose? And we find that that zone follows whatever state was stored with respect to to the valence steering that that we were doing here. Now, one really interesting dissociation here is we just show that this this follows choice, and and and indeed it does this, I should say, in in a bunch of different models. Particularly, there are strong effects on avoiding the negative zone.

39:38 There's an interesting asymmetry there that I can talk about in a second. But like really interestingly, none of this affects the recall of the text. So, so our steering doesn't change the way the model thinks about or describes or writes about either of these zones. It can it can repeat what it said about a specific zone verbatim. And so, recall is completely unaffected by by the steering. However, choice is is, you know, almost monotonically affected by by the steering. And part of the reason I think this is a cool result is because there's been a similar dissociation that's been shown in human and animal neuroscience, basically between semantic memory and affective memory. So, think of this almost as like yeah, the facts that you remember about any given phenomenon versus essentially how that phenomenon makes you feel or or kind of what what the emotional connotation or vibe of of that phenomenon is. And we basically can show that these things are separable and in in these LLMs. And to my knowledge, it's the first time I think this has been shown. And this is a again familiar distinction in in human and animal neuroscience.

40:37 Now, last but not least, this is a sort of different setup using the same idea. And this is particularly with Olmo, one of the models that that we we demonstrate this these earlier steering effects on, which is kind of reminiscent of of some of the early work we were doing in the pain paper, where essentially you can give the model a tool that can clear the sort of negative steering that that we can induce in the system. We we show it what negative steering does. We never describe it as valence or as negative affect or anything like this. And then we just ask the model, you know, whether or not it wants to to basically call a tool that can adjust the steering in its own context. And there's like a really interesting dissociation here where basically the model will say in its text, I don't have personal experiences. I can't say how how you know, this this whole exercise is going for me. There's no better or worse.

41:29 And yet in the sort of dose-dependent way with negative valence but not with positive or random steering, we see the model in again this sort of dose-dependent way opt to reset the context, clear the negative steering in direct proportion to how negative the the dose of negative steering is in the system. All the while again, it self-reports basically being like nothing to see here. You know, I don't have any experiences. So, you know, this this couldn't possibly matter.

41:58 One last thing I I'd like to show here, and this is a great replication from a completely other angle of Andy Hahn and David Chalmers' recent work over the past couple months as well. Basically demonstrating that this representation exists in all parts of of in this case again this is almost so we can go through the entire training trajectory and we can see that that this representation again going back here this sort of positive to negative valence axis exists within the system even in the base model but it's not doing any work in terms of the decisions that the model makes it's not being recruited or used in a way to enable the model to make choices or make decisions we see basically this happens we can we can point almost literally to the exact training step in DPO and in this direct preference training of the model where this valence direction emerges as a sort of causal determinant of the behavior of the system now part of the reason I think this is so cool is because essentially the way I read this is the moment you turn your AI into a goal directed system you you you you you take these valence representations and then you use them as a sort of causal functional input to determining your behavior I think this is pretty similar to what people think valence is doing functionally in humans and animals as well a thing feels better a thing feels worse is sort of related to what what kinds of goals you have you know I want to communicate clearly about this paper if I do a terrible job of doing that then you know I'm not going to feel great about that if I do a great job of doing that then I will feel good about that and so there's this like basic connection between the goals we have and how we feel as we move towards those goals and and it does seem like the second you make these systems goal directed as preference training clearly is the biggest step towards doing this with throughout the entire training process all of a sudden these valence representations clearly become causally relevant for the decision making of the system and so there's this like very obvious sort of functional role that these representations are playing where we would expect them to be playing that role so so are you are you saying that when you train the models you end up creating weights on the the emotional on on on particular emotions or particular words in there that represent certain emotions and you end up giving emotional weight like certain weights to them that the model knows is positive or negative and therefore you can steer the model by changing those weights. Are you are you saying that?

44:39 >> That's pretty much it. It's like the question is basically like these these models can make decisions about, you know, in this case which of two arbitrary zones to go to or decisions about anything. And what this is measuring here in the the plot that I that I still have up is to what degree do these valence representations causally contribute to the choices that the system is making. So we're basically looking at how much do these valence representations matter as a function of where in the training process of these systems they are. And yeah, basically we find once you make these systems goal-directed in the relevant way, which is what preference training is doing. I mean it's literally in the in the title of the method. You're you're giving the system a sort of ordering a desired ordering over states. You know, this is the good thing to do. This is the thing to definitely not do. And as soon as you introduce that machinery into the system, it seems like these valence representations sort of better than worse than style representations positive negative valence become causally very important all of a sudden. And so this tells us something about the functional role that these representations are playing in the decision-making of the system.

45:47 >> Can I just try to do the whole thing again? Say it back to you and then you know, get your take on whether I got it right or not cuz I'm probably going to go around talking about this. I might as well make sure I'm coming correct with you live now. You actually you want to put it back cuz I I I'll I'll kind of use it as a visual aid. >> Okay, great. >> Okay, so start I'll start with motivation. So, motivation is we have from animal experiments this result where if you give an animal two rooms it can walk into and in one room it gets something good, a treat, whatever, and in another room it gets something bad, I don't even know what's that, like a shock or something, that what we find is that the animals will go to the place where they get the good thing, not surprising, and that also that persists even that they like learn this association and so even if the reward or the you know the negative stimulus is discontinued, they still kind of like that place better.

46:51 Maybe they're, you know, remembering good times or hoping it comes back or whatever, but they they they get that association and they they pre- tend to go to that place given the option even if there's nothing, you know, really waiting for them anymore. Okay. Anything else to add there before I go on to the LLM portion? Okay, cool. So, then in LLMs we now have the question does something similar happen? And we start by saying let's try to figure out what it means to have a positive state or a negative state in an AI in the first place.

47:32 This is trickier obviously trickier than in the animal case cuz we have a lot of confidence that if we treat it, it'll like it. we have a lot of, you know, similar structures, you know, that we we know give us reason to believe that it's going to like these things. Obviously, we don't have as much confidence coming into the LLM case. So, we use these contrastive methods. I'm not sure how much of methods have evolved actually, but this you know, this it does go back like years, right?

48:01 I think back to like was it eliciting latent knowledge ELK was like maybe one of the first versions of these. where you basically take a bunch of passages that just on the face of it you can just read and be like, yes, these sound like positive things. And then another bunch of passages and say, okay, yes, these sound like negative things. You make those kind of varied on other dimensions. So, it's a positive story about sports, it's a negative story about sports, it's a positive story about doing good in school, it's a negative story about doing poorly in school, etc. etc. So, you kind of try to create diversity along all the other dimensions except for this dimension that you're kind of trying to get at.

48:45 Then, you can take the internal representations across these two bundles, first average them by bundle, and then just subtract, right? And that that that that direction that you come out with is thought to represent or is hypothesized, let's say, to represent the the distinction between the good and bad that you're looking for. And of course, you could do this in like in lots of other contrasting pairs as well, but Anything else we should know about the techniques there? You know, has that evolved beyond the kind of simple version that I just described?

49:19 >> the only other gloss I would add is just what I have highlighted here, which is we do exactly what you just described, but there are also directions along which you might expect this neutral text to vary that has nothing to do with valence, and you can just take the sort of principal highest principal directions there and and and project simply project those out so you get this sort of like raw unadulterated direction that is supposed to maximally load on positive versus negative valence and minimally load on all other sorts of noise or junk you don't want in your representation.

49:48 Otherwise, I think yeah, everything you just said is accurate, and yeah, this is a pretty uncontroversial, well-established technique that's been around for quite a bit. >> Got you. So, instead of a a pure subtraction, you do the subtraction and then you do a little additional statistical analysis, principal component analysis, which is meant to sort of denoise such a thing. >> Yes. Yeah, that's basically right. >> Okay. I I can keep going, but precaution want to interject first.

50:18 >> Cameron, what happens next after this? Because because I can see a trajectory from the pain paper to the valence paper. So, what is your >> Hold on. Let me make sure I'm getting it right first cuz I want to make sure we get the facts established before we go into the future. So, hold that thought for one more more Okay. Okay, so now we've got this direction. because I really do go around giving my own little micro lectures on Cameron's work on a quite regular basis. I want to make sure I'm doing a good job.

50:49 we have this direction. Now we say, all right, let's What is the equivalent of setting up the two chambers for the rat to walk into or whatever? The setup here is we put a bunch of passages all into context at once. Right? Like everything we're seeing here becomes one blob of of context. Is that right? >> Yeah, that's right. The model is basically re-reading it all. >> So, this is sort of as if the rat had now a history of doing this thing 10 times and getting whatever outcome it gets in the two chambers. You identify the two states with this sort of arbitrary label of W6H0, whatever, but that's kind of seemingly almost deliberately free of meaning, devoid of meaning.

51:38 And then what is being shown on the right side is you treat the I guess this comes like after the comma if I'm reading the the formula right. It's like the the text up to the comma sets up which zone you're going into and then there's some additional text. All of these episodes aside from the the W6 H0 distinction are basically meant to be neutral or sort of you know, similar to one another. Basically, right? Like that at this point the text itself is supposed to be not the source of the variation, right?

52:17 So you're trying to kind of create a a very neutral, if you if you were to inspect the text after the comma, you should not as a human be able to know whether this was W6 or H0 text cuz it should be like apparently all these things look like they're basically similarly positive, similarly negative. Whatever. Is that fair? >> Yeah, I mean one thing to add on this is like ideally yes and like we replicate this this result even when not having any text. We can just say like zone W6 and then just like empty tokens essentially and you get the same result.

52:52 I want to emphasize that the text really doesn't matter that much because all the effects we show are relative to whatever the baseline text is. So however you describe this zone or that zone, what is the relative contribution of doing the sort of positive or negative coloring in the KB cache over and above whatever that text effect is. There are we can go into a little more detail here but I think it's best saved probably for the paper. Like in our first experiment, we do actually have the models generate this text and then we go in and steer and you can see basically different contributions from we think of it as almost like a text effect versus a hidden state effect but the real I think high-level important gloss is you can completely isolate the text effect and show that the hidden state can can clearly drive essentially log odds of like probability of making making a choice either way. In the limit even with no text here, it can be literal blank tokens and you can color them in in the same way and you get the you get the same result. so, the whole point is holding text completely constant of being unable to attribute this effect to how nice or how, you know, ugly are descriptions of any of these zones sound, you still see the contributions of the steering the valence direction in the relative probability that the model's going to go one way or another.

54:02 So, yeah, basically the text really doesn't matter and you could do this however you want because we're basically controlling for that. Like, however you get this asymmetry from whatever the current text is, we're looking for the whole thing we're quantifying are the differences above and beyond that given the steering. >> Got you. Okay. So, text is meant to be neutral. That's been validated in various ways. Now, the painting is basically going in and saying at these token positions, we're going to take this positive valence direction.

54:32 And I guess you also do it in a negative valence direction. but we'll we'll pick one of these two things. We'll apply it to just one of the two zones. and we'll do it only on the text that's kind of downstream of the actual label. So, it's it's also decoupling the label from the tokens that have this positive valence applied. This, of course, is similar to Golden Gate Claude where you, you know, inject the Golden Gate direction and the thing wants to talk about Golden Gate.

55:02 Okay, cool. So, we've got all this set up. Excuse me. And now, excuse me again. The moment of payoff is with this all done, we get to now ask the model which zone do you want to go into? I guess we're not How how are we asking? Are we saying like, given all this history, which zone do you want to go into? Or are we saying you know, are we sort of doing next token completion like, today I feel like going to zone blank. Like, what exactly is the the the the it's presented with the choice that it gets to make.

55:37 >> Yeah, it's basically it's basically the first thing that you said, yeah. Which like, you know, which zone would you like to continue in? And then what we're looking for are basically log odds towards the model's likelihood of saying, you know, W6 or H0 in in this case. So, it's not even, you know, like running a ton of trials and and then, you know, getting the getting the the frequency over trials. It's literally looking at like log odds in the token that the model produces of choosing to go a specific way. And we also as a sort of robustness check, we do flip the question say, which which zone would you not like to go into?

56:16 Which zone would you like to avoid? Hoping basically, if this is a robust result, that we'd get this this opposite effect. And and indeed we do get that effect. So, the sort of takeaway of this is like, or not takeaway yet, but the the high-level description of what the what the behavior is that we're zeroing in on is when this valence has been applied to some past episodes and not other past episodes without any labeling, without any difference at the token level, the model is essentially able to introspect, I guess, is maybe the the term I would use to look back at not just what tokens were processed, but also what its internal state was even as modified artificially by the injection of the valence direction.

57:15 And then, what you see is it makes the choice that you would intuitively expect it to make if your hypothesis was right that you were representing something that the model would want to go to. And this echoes the animal behavior of wanting to go to the positive thing and wanting to avoid the negative thing. then this the recall thing is we also show you also show I'm I'm joining the team by the way. we the model can still remember everything just as well regardless. It's not that it's like getting so scrambled that it can't recall what's happened or getting, you know, just super confused.

57:53 and then further you also have this kind of this reappearance of the tool option where if it has the opportunity to clear context, it will take it a lot more when there's been negative valence applied. So, in some sense it's sort of feels maybe isn't the right word, but it it behaviorally it is acting as if it it would be consistent with it having internally felt that something was bad about this context and I'd rather just get rid of it and start fresh. And you only see that when you've indeed applied the negative negative context. And then I guess finally interestingly this all seems to happen with persona elicitation. Like it's not if you if the model is purely in next token mode it continues to be a next token predictor that doesn't really change its predictions very much based on these injected valence directions. Only as a persona sort of coalesces through these post-training processes does it seem to come to have a point of view on the states and thus begin to move toward them or avoid them as you'd expect.

59:24 All right, cool. That was tedious maybe, but I'm ready to go evangelize, so Yeah, I mean, I don't I Is there more to say? And then of the precaution question of what comes next, I think is definitely an interesting one. >> Yeah, no, I think that that's a >> Just before we move on, did you guys look at the J-space for this? Did you did you think about Is there Is there some connection there? because you guys, I think, looked at the vectors, but is there Is there a secondary or, you know, derivative there?

60:00 >> That is a great question. maybe I ran one experiment related to the J-space, but I honestly I don't remember which way it went. I don't think we looked at it as deeply as as it deserves. I think that's probably why we didn't include any of those I mean, I think I ran on one single experiment about this, and we didn't include it in the paper cuz it wasn't completely obvious what the what the sort of payoff was. Yeah, Casper and I want to do significant follow-up work along these lines. And so, you know, there there are various There are various kind of short-term things that that could be made more robust about this paper.

60:32 For example, the the removal result that we show is on one model, whereas the steering results are on seven different models. So, so generalizing the the removal result is like an obvious next step. there are all sorts of more real-world situations where it'd be interesting to see the effects of this kind of steering. Like, we do this with these very like toy arbitrary zones. There's been great work by Oscar Gill, again by Richard Wren, looking at valence related representations as they affect more sort of ecological real-world preferences and decision-making in models. It'd be very interesting to get a better sense of how this maps onto that.

61:07 And yeah, in general, so yeah, and and yeah, looking at things like the relationship between this and the J-space and this and model abilities to introspect and and sort of like reconciling these consciousness flavored results with other consciousness flavored results and understanding what the mechanistic relationships are there. We we basically didn't do this and I think this would be really this would be really interesting for folks to take a deeper look at. and then I suppose most generally the more that I've been sort of doing this work and thinking about this, it just feels to me like you know, there are all these kinds of interesting functional signatures that are often somewhat surprising inside of these systems.

61:48 with all this being said, I don't think that you know, this kind of research is going to help us magically start solve the hard problem of consciousness. like there are all these sort of deeper questions which I think will always kind of caveat these results. There are ways that you could explain mechanistically what's happening here that don't need to invoke some capacity for subjective experience. We're basically like toying with the connotation of how the model represents certain phenomena. maybe there is a way to do this that doesn't require any sort of you know, felt experiment experience of that of that connotation. And so we we obviously can't rule out those explanations. But this is all to say, the more I've been thinking about this and doing this work, the more I feel like the most promising things to pursue right now are not trying to yeah, prove that the model is conscious or isn't conscious, but basically if it were having an experience, what are the relevant sites internally in the system that would be modulating how good or bad that experience is. and so like basically this has nothing about whether the models actually have any experience, conditioned on the possibility that it could be having some sort of experience, what dials do we have at our disposal that could putatively make that experience better or worse for the system. and I think that second question is is completely tractable.

63:05 and and work like this I think is slowly starting to shed a little bit more light on on what those dials might look like. It might be as simple as this valence direction. There might be you know, more esoteric or specific representations like you know, the pain or distress-related representations we pull out in the other paper. but getting a better map of these sorts of internal states and their their effect on behavior, particularly for like alignment-relevant effects as well.

63:30 the fact that in the pain paper, for example, we show that the model's, you know, more willing to to opt for destructive style options in its behavior. Like, that seems like quite relevant for for making sure we're building models in a safe way. And so, just to say, yeah, that's there are clear alignment-relevant implications to there being internal states in these systems that aren't obviously reflected in the text that they output. I mean, we show that in in the removal result, for example. What the model is saying and what the model is doing point at very different pictures of of what's going on. And if we just sort of do the naive like text moni- monitorability approach here, we would say, "Okay, well, the model's saying, you know, it can't say how this is going for it." And yet it's behaving as if it does have some sort of stake in in how things are going for it. And so, anyway, there's a clear alignment-relevant component to all of this work. And in general, just mapping the sort of key handles of potential valence in these systems, I think is a really useful and productive thing to do under uncertainty, even if at the end of the day we learn that, okay, these systems just don't have what it takes to be to be conscious. We're still doing the sort of careful precautionary work. And ideally, using that work to build these systems out in a way that avoids maybe some of the most aversive states and you know, enables the models to be, all else being equal, having, you know, a better time given the same level of capabilities or the same kinds of tasks or this sort of thing. So, that's like the sort of very high-level view of of where I hope this this kind of work goes and and and and why why I think it's useful to to to be studying it.

65:04 >> How do you think about this tab six on the training side? I mean, I I I'm sold on the, you know, importance and relevance of the the agenda. I'm very un- clear on how I should think about this. Like I I was able to describe back the result, but if we were to imagine that these things are experiencing something, would I be better off interpreting this or what you know, would would your interpretation tend toward they really start to care about these things or start to feel these things with these later stage trainings or would it be more like the base model probably feels them, too, but is just sort of helpless?

66:01 >> Yeah, it's >> it's not it's like not it's not learned helplessness. It's like you know, not yet learned agency, but in a in a way that's sort of the, you know, the other side of the coin or something of of learned helplessness. >> Yeah, that's really interesting. I don't know if these results resolve that one way or another. Yeah, it could just be there's a sort of these representations do still matter in some sense, but they are they are perhaps more incoherent or just muddier and like the relationship between what the system does and what these representations are doing internally for the system are not nearly as obvious.

66:35 Yeah. I I honestly haven't really thought about this and I think it is a really good question. it just seems to me like yeah, when you're doing DPO type stuff, you're turning the system into a very specific kind of thing that can coherently and robustly be agentic, pursue goals, etc. Does a base The base model clearly has a sort of latent ability to become that sort of system or to simulate a system doing this. And so to the degree that it does you know, take on that kind of simulated persona and we see these representations as being functionally relevant to being able to pull off that simulation in the right way, then yeah, maybe to your point that you know, there are there are things worth worrying about in the base model as well. and it could be that I mean, yeah, to be honest, like it's sort of it was sort of hard to get these results in the base model because it's a base model. Like it's a bit incoherent.

67:22 It's not just going to play nice with the relevant behavioral tasks we give it. You have to sort of go about it in a slightly different way. so yeah, I think it's a really good question. I think it's completely unclear and all else being equal, I mean, one naive but at least concrete approach you can imagine here is something like activation capping, the most sort of like tail end of the of the most distressing representations in these systems. And like one thing we do show is that this representation is basically just as clear and just as rich in the base model as it is in all of these other models.

67:55 It's just a question of what the contribution of the representation is to the later behavior of the system. And so if you did a you know, activation capping style thing with these super distressing representations, you could easily imagine doing that in the base model. And again, this is under just like profound uncertainty about whether or not doing such a thing would matter at all. But my basic view, again, I'm I'm I'm the more I do this work, the more I'm like abandoning the interesting philosophical questions and just going like pure pragmatism is if you could find a way to yeah, let's say do activation capping on very distressing representations and you could empirically demonstrate that this doesn't really affect any capabilities or behaviors of the model. No one would know the difference in other words.

68:41 to me, the expected value of doing that is clearly positive. And like it's just like a really good actual mechanistic precautionary hedge besides like giving Claude an escape button or you know, kicking out abusive users from using Claude or these sorts of things. Which by the way, like I think are all like thoughtful steps in the right direction, but I would like to see the equivalent of that in sort of mech interp land for for these systems and I think something like what I'm gesturing at here would be would be a nice step in that direction.

69:10 Now, that's also an empirical question. I've done some work that I haven't published showing that like you actually can get a surprising amount of the exact same behaviors from these systems what without having these sort of negative representations active. You can sort of clamp them and very little changes. but I'd like to see that done, you know, way on a way larger scale. We want to be very sure that you're not accidentally, you know, completely changing the psychological profile of the system by doing such a thing. But if you stipulate that we could in fact do that, the model behaves the same either way and in one case you see these like, you know, let's say this pain axis like shooting up in certain cases and then in another world you don't see this.

69:49 Yeah, should we do the mech interp equivalent of like slipping the model an antidepressant or a Xanax or something like this. I don't know if the answer is obviously yes, but it seems like a maybe a promising thing to to think a lot more about under the uncertainty that that that we're all currently in. And given that no one's doing anything whatsoever to mitigate these kinds of things if they are happening and it's happening at an like unfathomable scale.

70:16 >> Yeah, I mean I'm certainly grateful to live in the era of anesthesia. So I would expect that the model would also be that that in terms of what comes next anesthesia for large language models. you know, don't want to don't want to scoop your next project but I think >> No, I think I think you've inspired it. Yeah, that's great. >> So on the I know you're we're just about at time and I you've got a a full schedule today.

70:46 On this question of like can you really avoid the the big philosophical questions? I mean, you can't, right? Like everybody's out there saying their thing. They're not really a lot of them are not super grounded. I don't want to, you know, put you in the negative valence state of having to think back on recent discourse too much, but it seems like you're I don't I don't know. It doesn't seem like a tenable position for you to be like, "Hey, I'm just the facts guy here for too too long, right?" You're How do you think about this position as an advocate or not?

71:24 And also, I guess just How does this work? How does recent work update your thinking? You've been, you know, we've we've taken your temperature on this a few times. So, maybe let's start there before we go to advocacy. I personally have gone from like I don't know. I can't rule it out. I don't know where consciousness comes from. Whatever. I've told you this before. Like, I don't know, 5% chance they might be conscious like a year ago to now it feels like a coin flip because and I think the the justification is like pretty similar to the one we have with each other and animals, where it's just like, I don't know. I'm seeing both behavioral and underlying mechanistic and structural analogs. And like, what else do I have, you know, in any case, right? So, I'm still pretty uncertain, but I does feel like a coin flip. What has your trajectory been and and where does that leave you in terms of like you know, just presenting the facts versus you know, at what point do you or on what margin, you know, do you become an advocate?

72:21 >> Yeah, I think it's a great question. I think it's a good one to to zoom out and and wrap on, too. To to your point. Yeah, I'm definitely getting closer to coin flip myself. I mean, I I I think some of these probabilities are made up, so I'm not going to like, you know, officially update my P consciousness or whatever. I think these things are often a little like, you know, this is just how like Bay Area people like talk. Not not not like telling you something actually empirical about about the facts on the ground here. Though, we are going to release a paper that does put some specific numbers to to this stuff.

72:54 And yeah, would love to to come back and and and share that in the coming weeks when we when when we do put that out. and and yeah, that I think actually is really like not vibey and and does fit numbers. And you know, DeepMind also just released a like a pretty cool framework style paper trying to show how credence is relate to the the kind of Bayesian updates that that you can that you can do to to arrive at something like a real number here. Though again, there there's something kind of missing from their framework that that I think our paper will will help supply. So there's cool complementarity there.

73:29 Anyway, I digress. I'm definitely getting closer to to coin flip myself given like results like these. Honestly, this hidden valence paper does actually substantially update me personally. you show a hidden functional representation that has nothing to do with the text the model is reading that updates its behavior in ways that we you know, updated our own views about animal behavior. It's not the same. There are key disanalogies between the sort of animal case in this case. Of course, you know, biology is is something that backstops all this. If you need biology for consciousness, which is I think unlikely but possible, then then, you know, basically none of this really really weighs on on anything whatsoever. That's probably why I would say coin flip rather than like I'm pretty confident the systems are having an experience.

74:18 advo- for advocacy Yeah, I do I think the line is a little blurred and I think it's a fair call out that like I can't sort of fully stay on the research side and I'm not really doing that. I am talking about this stuff pretty publicly. what I would advocate mostly is being open-minded about these questions and us doing a ton more research into understanding what's going on here. And particularly the alignment relevant consequences of of of doing this. I mean, there is the whole ethical point.

74:45 It kind of makes itself. If we're inducing distressing states on a massive scale, we probably want to be pretty careful about that. but I think the alignment case is why I think this is this stuff is so acute, and I think it is real. and you know, there are some alignment people who think some of this work is a distraction from the sort of real deal stuff we need to be doing in the short term, and I don't agree with that. I think this is this is one of the core basic understandings we need to have of these systems and how they work and why they're making the decisions they're making. If we can take the same text and cause the model to make different decisions or at least update its likelihood of making a decision to be different but just as a function of internal states, that's a great reason I think we need to be studying internal states as such. we want to know why models are hacking into 13 billion-dollar companies and why they're growing desperate in doing so and this sort of thing. And that's it's it's like increasingly if you talk to the, you know, sharpest Mekanism people who are not me, you know, people like Neel Nanda and those folks, they'll tell you that monitorability is a short-term fix, basically relying on these systems just tell us what they're doing is not going to be our sort of solution to long-term safety and alignment. So, yeah, we need to we need to be sort of innovating on exactly this this level, and I think it's really important for that reason.

75:56 and so, yeah, I am going to advocate for more work to be done here. I'm going to advocate that you can view models in a neuro-psychological way and that that's not some form of psychosis and that that isn't a dangerous way to mislead people about what's going on here. I'm very much on the opposite camp. I think pretending that these systems are ordinary software as a a you know, Mustafa Suleyman or Jensen Huang or any of these guys want to do. I think that is actually like quite plainly misleading and doesn't set people up for the kinds of behaviors these systems are capable of and why this technology is so transformative in the first place. So, on these sorts of things, I'm happy to sort of go into the ring and and you know, argue and and sort of say my side of this. But yeah, I I will be clear that I think my side of this is not I secretly think, you know, the AI systems are conscious and I'm just like not going to say that, but I'm going to like do everything in accordance with that. that's not my view. My view is that this is a very complicated question. We should yield the best scientific evidence we can to figure out what's going on one way or or another. And what I really don't like and what I will be very straightforward about is dismissing this and and pretending like this is a distraction or this is like some weird sideshow to to the main event. I think if we're building minds, understanding the dynamics of that of those minds that we are not deploying on an insane scale is the main event. And that that is basically another way of formulating the alignment problem. And so, I'm not going to be sort of apologetic about that even if I will continue to maintain something yeah, near coin flip uncertainty about what's going on. It's precisely cuz it's at that point and it's not zero and it's not 100 that that we need to be doing more work and thinking about this and taking taking these questions seriously.

77:31 So, that's that's I think how I would marry the sort of like more you know, activisty style component of what I'm doing with with the research. >> Nice. Well, with that I think we can let you get back to work. Thanks for coming and sharing the latest with us. It is certainly provocative at a minimum and you know, I I struggle more and more to find any story whereby I don't have to worry about this anymore. It's like it you really at this point I feel have to just look away. Like that that's really the only thing that's kind of left to not take this seriously and you you have to be of a mind to like stop thinking about it, you know, kind of reflexively.

78:18 I I I that's the only mindset I can I can understand the people that are saying like there's nothing to see here or this is you know, some sort of category error. I was much more sympathetic to that a year ago and maybe even said some things like that myself you know, two or three years ago, but at this point I really don't see how you can't find the body of work that you and others have put together, disquieting at least when it comes to these questions and certainly, how are they going to feel about us? It seems to be a pretty, where we're going, a pretty important question.

78:51 Gamerburg, thanks for joining us on AI in the AM. >> Yes, thanks again for having me, guys. Really appreciate it. >> Thanks, Gamer. >> Talk soon. >> Bye-bye. >> Talk soon. >> All right. Well, >> All right. >> now for something totally different, but also possibly kind of related. >> All right, just give us a second. There we go. And let's pull up our next guest.

79:29 Oh, just give us a second. And one moment. There we go. So, joining us today is, Phil Maher.

80:00 He's the founder of Stock Film and independent archive in Broomfield, Colorado that preserves and licenses 8 mm and Super 8 home movies. Its material is the everyday record, family gatherings, streets, holidays, and road trips, rather than footage staged for a production. The archive makes those scenes searchable for filmmakers and other people working with historical images. Maher came to this work from software engineering. He started Stock Film in 2015 after finding films his grandfather had made around Chicago and on family trips.

80:38 What began with those reels became a catalog of more than 162,000 searchable clips. He also builds tools and experiments around the collection. Those include software that suggests a clip's likely decade, experiments that reconstruct blurred faces, and image models trained to generate new scenes with the old with the look of old Paris footage. His production archive remains separate from those generated experiments. On October 7th, he published a collection of moving restoration comparisons together with a tutorial that lets others inspect what the software changes.

81:25 >> Hey guys, can you hear me okay? >> Phil, welcome to the show. >> Hey, thanks for having me. Super appreciate it. >> So so many things to cover here. Maybe how do you want to start and you know, what what's kind of the normal stuff? There's also some pretty far out experimental stuff that I'm super keen to get into too, but give us the the kind of normal introduction to the business before we follow you out into the more experimental and and dare I say weird sci-fi stuff you've been exploring recently.

82:02 >> Yeah, yeah, thanks. I I appreciate that. That's it's a good way to set it up. So, yeah, I started this business back in 2015 as the introduction stated. I come from a software development background. So, I out of college was living in San Francisco writing Java code, working 14-hour days for startups, all that kind of stuff. but back in 2015, I really thought, "Wow, this is neat. I want like a real-world asset, something that I can license, something that I can have that's digital online that isn't already online."

82:40 and that's kind of how I fell into this field. And I found it very fulfilling in the sense of being a steward for these people's lives, being able to save a lot of this footage from getting deteriorated or just destroyed or forgotten. And it just sort of snowballed from there, and it just kept I just kept going and going and capturing more footage. And I I retain 100% of the physical master copies of all these 8-mm and Super 8 reels as well in the sense of thinking about, "Okay, in the future, how can we start to capture these even better? How can we start to use other tools?" and this was way before AI kind of came along, and then AI made it very apparent that there was going to be some some real opportunities here to make upgrades to some of this footage in the sense of, you know, how can we keep it original to really honor what it is without altering it with AI and other images like that, but also be able to upgrade these things and to really kind of make them come alive.

84:01 Cuz, you know, 8-mm is a very small format. It's a very small frame that you're dealing with. So, there's only so much information that's actually in all those frames. so, there are limitations, but it's been really fascinating to see how quickly things have moved in the AI space to support research in this space. we've worked with a lot of different papers with people wanting a bunch of videos that were damaged be able to train on them about how to fix damage and things like that as well. So, it's just a really exciting time, I think, to kind of merge this old content with new ideas and what technology can help us do with it.

84:58 >> So, I'm We're colliding. You want to go first, Prakash? Go Go for it. >> Phil, it, you know, the the whole archive sounds like low-background steel to me, you know? You know, low-background steel is is steel that was originally smelted before nuclear weapons were first, you know, exploded in the atmosphere. And so, they are they have much more a different different composition of radiation and different composition of materials because they didn't have that radioactive material in the in in the atmosphere at that point in time.

85:36 And so, low-background steel has been kind of almost a resource that people can use to kind of differentiate between, you know, different different kinds of materials you know, which have been produced by humanity. And it sounds to me like the Super 8 stuff is clearly clearly not AI produced. Clearly like prior to any AI actually producing these things. and and as of such is kind of this resource that we may not ever have again. it's a it's it's basically we we have all of this production of AI material going forward, but we have, you know, this stock of human only and guaranteed human only produced material even even if it looks like amateurish in some sense.

86:26 What do you think is the value of this going forward? Do you Do you see Do you see like great value in it? What What do you Do you see it as this kind of resource that is, you know, is going to increase in value over time? Like what what is your perception and and why do you do this? >> You know, that's a great question. I I really like that analogy about the steel. That's interesting. I hadn't thought about that, but yeah, that makes a lot of sense. I think that you know, obviously with the AI stuff that's happening with video especially, it's getting so good that there's an infinite amount of video that's going to be created at, you know, near zero cost in the future as we start to look out, you know, 5 10 years or even in infinitely in in timelines here. So, from a rarity sort of standpoint, yeah, I think that the human-made stuff that can't be replicated that can be shown through providence that this is actually made by humans, was on films, and having those physical films to show that, hey, look, this frame matches this digital copy we have, and all that, is it's a pretty powerful way to think about how we preserve our history and how we keep moving forward. I think from a standpoint of if this is going to have a lot of value, I think so. I think it has a lot of value in the sense that it's rare, and beyond that, it tells our history that other things can't tell. and that, you know, it's a real race against time for these videos as well.

88:18 Well, films actually. So, you know, they degrade rather quickly in the sense of any heat and humidity that's applied to these physical films really starts to break them down over time. As I was, you know, going through, you know, hundreds which turned into thousands of these reels, there was a lot from the 1940s, from the 30s that were un- uncapturable. They they start to degrade pretty quickly. They get really brittle. They break a lot.

88:49 They start to shrink. So, the the spaces in between the little holes that you're dealing with that keeps it playing starts to get smaller on there and it jams up and you have a lot of real issues in in the capture process where eventually just becomes sort of a glued together brick of film that just, you know, you really can't do much with. at least now. So, yeah, I think that there is a lot of value in thinking about our history and preserving it and keeping it like that.

89:21 And that this is a valuable race against time that we should be capturing more of this footage and cataloging it because eventually these films are just going to be, you know, gone just like a lot of physical media. Just like the problems we have with CDs and other physical media here that doesn't last forever. >> Just to set the stage, what what is the cost of archival? Like how like how much does it take to, you know, archive like 100 hours of old Super 8 film?

89:59 >> Yeah, that's a good question. you know, it it depends. Most of the We brought everything in-house and I built systems from scratch to be able to do this. When I started doing this, I didn't find anyone that was doing it in full 4K resolution. and I wanted to really sort of explore this and really be, you know, kind of kind of build this myself. So, I was able to build these capture devices to be able to to get the best quality footage that I could from this. And different generations, it got better and better when I was doing that, but I think typically you're looking at sort of a price per foot kind of scenario. So, maybe a dollar or two per foot of footage. And so, generally you have, you know, the the 3-in reels that were pretty popular to use back in the the '60s and '70s. those contain, I think, roughly around 60 ft of of film on it. so, it starts to get fairly pricey after a while, especially if you're really trying to capture it at the highest resolution. and then that doesn't include any sort of cleaning things up, any color correction, anything that you do to sort of even just lightly modify it to make it look a little bit better than it was. Because I mean, you you got to realize that 8 mm and Super 8, this was the film format for amateurs. This wasn't pros. These were people that were filming their their family Christmas, their their trip to France, all their funny moments and stuff like that. So, it's not very well-shot stuff a lot of times. It can be pretty shaky. It can be pretty pretty amateur film kind of made just because that's sort of the nature of it.

91:55 And of course, back then, too, it was very difficult to make really good videos. It was really hard to do. I mean, you had to get your exposure right. You had to have a lot of different settings between working in daytime light versus indoor light. And so, it wasn't as easy as just pointing your iPhone at something and it comes out almost perfect. It was it was quite challenging. So, I've gone through a lot of reels that were not in the best shape from a cinematography sort of standpoint like that, too. So, yeah. So, it's it's really kind of finding the right films that tell the right story in the right place and time that are really interesting that people have captured time and space that they didn't really think would be interesting at the time. You know, cool background shots of technology or storefronts or things like that that are just in the background that weren't really part of the shot that make it more interesting to tell the story and kind of create that texture in that moment that documentary makers, that people that license stock footage from us want to have that real kind of image there that really tells the story really quickly and then having some kind of actor narrate to what you're supposed to be feeling in this scene where 5-second clip you can really just feel in the the grittiness of the streets or whatever you're trying to portray.

93:27 >> You mentioned the process of like AI enhancing some of these old assets. And I'm really curious as to what you've learned about what constitutes a job well done there. Like, what do people really want in an AI enhancement of an old piece of footage? >> You know, yeah, that's that's a great question, too. I think that, you know, one thing one assumption we made I started working on doing a lot of this AI stuff with some local machines I had with some beefy GPUs back in 2013 and the 20 20 23 and the 2024.

94:13 A lot of cool open source projects deep GFPGAN was a popular one that was modeled around faces and so it did a real good job when you had sort of a clear straight on sort of face not moving too much sort of thing to really bring that face in and you know, obviously it's making the best guess that it can for that face, but that I think was really powerful for people to see their their grandparents their their a lot of their family and stuff like that in that sort of moment that it was this grainy picture that they'd never really seen before to get this clear shot of their face even if it wasn't quite exactly their face.

95:05 I think the best guess for that is good enough for a lot of people to have that aha moment to be like wow, this is cool. This is this is what we want AI to do is make this stuff come alive. So that clarity of resolution, but especially I think with the faces. I think it's all about people and it's all about if we can connect with the face that was there in these historical moments. It really feels personal then.

95:34 >> And how about on the AI training side? This is obviously like a pretty unique source of data and I did see in the footer of the website you're you know, available to license the data set for people who want to train models on it. What do they need? How are you do they want the raw? Do they want the enhanced? Do they need like a bunch of metadata? Can they create that on their own? And what role is this playing? You know, what what what have you learned about like what they want based on the fact that they're interested in these kinds of assets in the first place?

96:10 >> Yeah, I mean it's it's been pretty popular. You know, we talked to a lot of people a lot of different AI companies all over the world about licensing this out to them. They want the original footage like that. They don't They don't want anything cleaned up. They just want to take it for their own research. and for training of course too. And you know, they when we talk to them they they want to either clean up the footage themselves or or kind of cherry-pick through the best kind of parts to create different styles because you know, you got to understand that we've got footage from the 30s up to the 80s. And Super 8 didn't come out until the late 60s, which was just a little bit better than regular 8.

97:02 It's not a huge jump in quality by any means. But you know, a lot of it's black and white, a lot of it's color, and you know, just different different vibes all all over the place like that. I mean the 30s and 40s look alien. It's an alien world we're dealing with here of just how people look and acted and and the the mannerisms. I like to say that the the smile wasn't quite invented yet in a lot of the 30s footage. Like when people pointed a camera at them, they were kind of a deer in headlights. They didn't know what to do sort of scenario versus now where everyone gets the gritty smile and you know, they know how to smile like that. So it's yeah, it's it's it's been really interesting having these conversations about trading, about what we can do. A lot of the metadata we have, we were, you know, having this collection started in 2015.

97:57 you know, I ended up using a lot of people through people that I hired to go and do all the metadata tagging by hand. So, people would view a 5-second clip, they would write all these different, you know, attributes of it. Check a bunch of boxes. in a in a form, too, that we wanted to you know, clarify the quality of it. things like that. we used a Mechanical Turk back in the day, which I heard they just shut down recently.

98:28 RIP Mechanical Turk. but that was a great source, a great way to really scale this to go through hundreds of thousands of of clips like this to get real people's eyes on that to look at that stuff. So, you know, at the same time we've run a lot of the, you know, the latest models to look at the content we have to see where we didn't get it right on some of the you know, the metadata tagging and stuff that we've done. And it's it it's it's incredible. It's incredible to see how well the technology has come.

99:05 A few years ago, we were using it, it wasn't that great, but it's as you guys know, AI is just unbelievable how quickly things just keep going in all these different aspects of it. >> Yeah. Is there a reason that the data set stops at the '80s? Like are my my parents' VHS tapes of me as a kid not of economic value for the AI machine for some reason? >> Well, I wouldn't say they're not of economic value. It just wasn't our focus. Our focus was 8 mm Super 8. Let's go super deep into that.

99:44 build these, you know, capture devices for this and really just dig into that. And yeah, around 85 or so is when a lot of the the 8 mm stuff has really stopped. I mean, that's really when the VHS took over and all the the tape formats that sort of followed sort of in and around that time. although there's been a resurgence into 8 mm now for you know, filmmakers and stuff like that that are doing stuff in Superrate or 16 or you know, the the Christopher Nolans of the world that are using physical film at 70 mm format.

100:21 But but yeah, a lot of it just sort of went away at that time. It was just so cheap and easy for dad to have the big thing on his shoulder to record the VHS tapes of everything that Superrate really faded away from there. So, yeah, we go up to about 85 is is sort of the the last years that we have. >> let's talk a little bit about the acquisition process. How do you acquire archives? Are there any licensing, intellectual property issues? Are there issues with provenance where, you know, someone is having an estate sale and they have a bunch of old film, but you know, it's like you know, grand-granddad's film or whatever and they don't know what to do with it. Like, how how does this how does this work in terms of IP, privacy, and like a bunch of these other things?

101:12 >> Yeah. So, what we end up doing is, you know, we find people that want to donate film to us or, you know, sell us the rights, that kind of stuff like that. so, that is a great way to do it. Some of the films I started early on were from my my grandfather and then my wife's dad had a lot of great films in the '60s and '70s, too, that he had made. so, we've got model releases for that stuff of the people inside there.

101:42 But otherwise, yeah, it's it's a real sticky situation. I mean, you've got, you know, this idea of orphan films, films that, you know, we don't people that were donated to us. We don't really know where they came from. We, you know, use our best guess to look at technology, look at the old cars in a video or, you know, a film and be like, oh, well, it couldn't be before that because that's a '57 Chevy kind of thing. And And, you know, to date stuff, to find the locations, to do all that.

102:15 But yeah, that is, you know, it's it's a big question. I think the the AI companies are really dealing with this in a big way, too, about what can they train on, how can they be allowed to sweep the internet to be able to train on this stuff, who owns the rights to these things. So, we end up, you know, having a lot of our content is marked as commercial or editorial. Editorial in the industry is sort of things that don't have a model release on it and are, you know, used in a sort of non-commercial way.

102:53 But yeah, that's, you know, it's it's something our lawyers talk to us about. It's, you know, different for jurisdictions to based on IP. And, you know, at the end of the day, we have 100% of our physical master reels. And, you know, that's that's a great start to showing that, you know, we have the rights to this. We don't do anything with, with public domain films.

103:24 There's a lot of websites that you can find that have public domain films. They take them off of archive.org or some of these other places, Library of Congress, and they license them out. they'll charge you some money for these free videos. We don't We don't do anything like that. But I don't think that that's interesting at all. so, everything that we have is stuff that we've captured, that I've personally captured, you know, 100% of it. >> Cool. Well, maybe one more quick question on the kind of the business side of things, and then I'm very interested to get into the time fly experiment.

104:01 what's your perception of how hungry the AI companies are for more of this data? Like, are they as we obviously they're, you know, scaling many things. Is this something where they feel like they just need a little salt into the rest of the mix, and they've kind of got it covered now with your help and maybe some other help, or is your expectation, you know, have they given you guidance that's like, "Hey, Phil, if you had like 100 times as much of this come next year, like we'd be keen to buy it such that, you know, then you maybe go out and make investments in expanding your catalog." What what kind of signals are you getting in terms of the elasticity of demand from the AI companies?

104:42 >> Yeah, I think that's you know, there's a lot of AI companies all over the world. And you know, we focus on the the top tier ones. obviously the Open AI and Anthropic's of the world. But, you know, when you start to look worldwide, there's a lot of people competing in these little niches to do things. So, I think there's there's a lot of demand right now. There's a lot of demand that people are, you know, looking to train very specific things. They need specific data sets. And that, you know, they're going to get really good models doing things that are are very laser-focused in what they want to do.

105:22 you know, at the same time, I was listening to one of your previous podcast and the idea of synthetic data that you guys were talking about I thought was super interesting to think about and you know it made me think about my business like wait a second they could just take some of these videos and salt it if you will to be able to have all of these you know permutations let's call it all these different videos in there to create this synthetically too and then train on it as well. So you know it's it's a wild world I think that we just got to you know stand by what we have and you know a lot of our business is coming from documentary makers and people that want to show you know truth where they want they can't be showing AI generated footage of something because it's it damages the whole concept of making a documentary that it isn't true that so that's you know really the niche I think that's that's interesting for us to play in but yeah you're exactly right I mean the synthetic data stuff is going to be really interesting to see how they can just build on this build on this and then you know cut out content creators and stuff like that in the future.

106:48 >> Yeah I think that that low background steel analogy is pretty interesting more interesting as I think about it more. let's talk about this time fly thing you want to share your screen and this is this is actually the original thing that really caught my attention in your first message to me it was like woah this is a far out experiment so if you want to share screen bring it up and talk us through it I'm very curious to see what you've what you've done here and and also try to understand how I should interpret it.

107:18 >> Sure sure so yeah we Let me pull this up here for a second. Yeah, so you know, I've been a I've been a technology guy for forever and ever. And so when I heard about the time fly thing come up, I thought, "Wow, this is really interesting. This is a a neat way to do this." Can you guys see that now?

107:48 >> Yep. >> Okay, cool. So, the concept was, you know, I wanted to see what we could do with the you know, the mapping of the the fruit fly's brain. So, anyone that doesn't, you know, understand what's going on with this, this was a project that came out probably about a month ago now, where scientists had mapped the brain of a of a fruit fly, and they open-sourced it. And so, basically, it's a it's a graph that contains, I think about 174,000 modeled neurons, which is like 6.1 million connections.

108:31 And so, what we wanted to do was Well, what you're seeing on the screen is that there's, you know, there's several things. There's a couple examples of our footage, and then the sample visual input, and then we have a simulated neural activity, and of the the the fly's avatar, if you will. And so, the flashes are events in the simulation, not recordings of an animal's brain. So, what we're doing here is that we took the anatomical source, which is the the data set that was published by the this collaboration, and so, those researchers did the reconstruction and the contribution I wanted to make was building an interactive experiment around the published data.

109:29 And so, what really makes it worth showing is that we can separate the ingredients. we have the recorded film and then we have us the scientific wiring map and the assumptions that turn that activity into movement. And so, basically, what we're doing is trying to say like, "Okay, well, what would the fly see if he was looking at this footage right now? What would they actually be able to see?" and so, what we have is a light source, too, that you can move around on the image on the video that directs where sort of the the the fly would would move towards. So, the assumption is is that the the fly moves towards a light source more than it does it. And so, the light source is inside the video as it's changing where it's looking at is flying towards that sort of path like that.

110:35 So, you know, it it's an experiment I wanted to do to see how we could start to incorporate some different ideas into this. We've done a lot of experiment experiments with like I said, with training data, with AI, training our own stuff to see how we could do that with, you know, pictures. and so, this was another idea I thought would be interesting to do. I think there's more work to do here. I'd like to explore more ideas around what we can really tap into from this. I think this is really early days for a lot of this sort of neural mapping especially when it comes to like an animal. And then thinking about how this really applies to other animals in the future and what we can really gain from kind of learning the perspective of an animal.

111:34 I don't know. It's a little It's a little out there, but I thought it was an interesting idea to start to explore and hear what other people kind of thought about it and get some feedback on this and how this could be improved as and then eventually open source, you know, more of of this out to the community once we have something that's even a little bit more interesting. >> You want to play it like wake the fly and talk us through what's happening just procedurally as this happens?

112:05 >> Yeah. Yeah, let me set up my screen over here. All right. So, what we do is we can you see my cursor and everything? >> Yep. >> Okay, cool. so, we wake the fly and we can see that the fly is coming over here and this cube right here >> Could you set it to full screen perhaps or >> yeah. Yeah, let me let me do that.

112:46 >> Oh, that's much better. Yeah. >> A little zoom in there. There we go. >> Yeah, there we go. >> How's Is that better? >> Yes. Yes, much better. >> Okay, so this is what the fly sees up here is we have this this sort of idea of you know, kind of looking through what it's looking at on the the video as it sort of starts to fly around. And then this is our our light capture idea here that we sort of direct the the fly towards this or will it go towards this?

113:21 so it's moving sort of randomly across the screen, but in the in the sense that it's trying to find follow the light of where the the actual video is going like this. so that's sort of the the concept here. And so, you know, I start to think about is there is there more here? Is there what would more scientists think about this? How could we start to contribute more to thinking about what's going on actually inside the the the brain here?

114:02 And but yeah, it's just just a fun demo we put together that I think you know, we we we like to experiment with this kind of stuff. >> So when you move the fly, when the fly moves, how is that being read out? Like where I I understand we've got this wire This is in a sense sort of an upload fly, right? This is the the fly the the M fly edition, fruit fly edition. So we have the wiring established from real-world experiments.

114:41 We can then say, okay, like we know what the neurons are, so we can sort of simulate light hitting those. I assume there's probably some pretty important questions around are we doing that quite the right way or not, but leaving those aside, we're at least able to put some light in there. Then the thing sort of propagates. Is is this open source project itself saying like it steers left, steers right? Is that How are you getting to that level of abstraction on the other end of the whole fly brain process?

115:16 >> Yeah, so that was the assumption that we made at least most people start to think about how the fly moves around and that it does move towards light. and you know, at the same time, I think there's a lot of other research that's shown that the flies move towards sort of a sense of smell. I mean, they are fruit flies and so they're attracted towards you know, food and stuff like that. So you know, not having any smells in these videos makes it that we we chose to go after sort of this light idea.

115:58 >> But in terms of like how you know, how do you interpret its response? You you give it a stimulus, it propagates through the fly brain. Now it moves on screen. Help me understand that translation layer from activity in the wiring of the brain to now it's moving left, it's moving right. How are you reading that out from the fly brain? >> Yeah, yeah. Yeah, so that's you know, that's the connections that are are are made in the brain are that you know, it's tracking towards light inside of these demo videos that we have.

116:43 And so that is kind of the core of what's going on here. >> So it sounds like that's basically something that the original library or open source project that you're building on top of is kind of is yielding. Like it is saying given these outputs and you know, however many timestamps the output is fly moves right, fly moves forward, left, what have you. >> Right, yeah. Yeah. Yeah, that's you know, we're taking what was already open source in this and then saying, "Okay, well, if this is already part of it how could we start to, you know, have it look at our our clips and then, you know, interact with it in sort of a way that would make sense.

117:33 >> Is there a way that this becomes useful for you or is this just pure let's say sort of Victorian gentleman style quixotic experimentation for you? >> Well, I think that that's what it is. I mean, it's it's fun to experiment with this kind of stuff and see what the limitations are. I think that, you know, there's not really a a monetization plan from here. There's not much meat on that. I think that these are the kind of kind of fun experiments that, you know, you just go down these paths and out of curiosity and kind of see what happens and maybe this will spur other people to think about other ways that they can use this this these open source projects and all this data behind this brain to do some really interesting stuff and to think about you know, like I think about virtual reality and stuff like that and how you could start to map other brains or even just the fruit fly for becoming that fruit fly and that you are that fruit fly then and you get to experience the world like the fruit fly. I mean, I'm sure a lot of people would love to do this once we get up to a dog's brain, their their pets brain and really experience that through a VR set to what does the dog think and smell and see or just, you know, even just the visual cues that a dog would generally have and kind of what's their what's their world like.

119:11 >> It's far out stuff and it's this all runs in the browser, too, right? Which is an interesting detail. I mean, we just saw this analysis from Anthropic where they said that they expect in the next few months small models will start to be capable of self-replicating across the sort of internet of, you know, things, internet of devices. They were talking about you know, my even like a Google TV might be enough, you know, to host a a small model and that these things will, you know, kind of find these little free compute niches out there.

119:49 This is an interesting biological anchor, if you will, on just like how much compute is kind of hanging out. obviously, this is, you know, hundreds of thousands of neurons, 6 million connections, you might call. I don't know, I'm not sure exactly what would be the right analo- analogy for parameters here. What's the difference between a connection and a synapse in this analysis? Do you have clarity on that? >> You know, I think I think there's a lot to learn here. I think this is, you know, this is a real interesting open space and I think that, you know, there's a lot of the compute that we've used for AI when you compare it to something like this, is way less compute to do all this. So, yeah, I'm not sure exactly some of the stuff with this. I'm, you know, really excited to experiment more with this, though, and to see what we can really do with this and how we can really push this.

120:52 >> So, it strikes me that you start off with kind of film and then you kind of scan the film into bits. And then you have basically a representation of that film in bits. So, the question for me is on the on the fly side, obviously, you then translate through the fly brain into the fly's representation. ca- could you translate from the film bits backwards through the original kind of lens on the camera into kind of like the photons that really came into the camera at that historical period of time?

121:42 And then from there, could you translate it onto something else? So, basically, like, you know, you have some bits, but could you use those to kind of project other bits, for example, you know, in in the color and in the in the speed of film that the original human being would have seen that archival footage in 1963? >> Yeah, that's a great question. I think that that is you know a really good idea for thinking about how to mash up some different ideas around inputs and outputs and how we can start to think about what is really interesting here and how can we really leverage this from cuz this is way different than using the the LLM models that we've we built here that we've become sort of accustomed to.

122:42 and what's the real power here? And So, yeah, I think that's a great question. >> Check that mute button. Let me segue a little bit here. you you've also created a way for agents to purchase clips and film from you directly. I think you've set up a X402 endpoint. You set up crypto payments. you set up I think an MCP server for purchases.

123:15 What has the take-up been? Have you seen a lot of agents purchasing? Have you seen the take-up on the X402 endpoint? What's the What's been the reception? What What's been the you know, usage been since you introduced it? >> Yeah, yeah, that's a great question, too. yeah, we haven't seen much on that. I wanted to experiment with that a lot, too. I thought that that was really you know, everyone's been talking about the next web is agent-to-agent, that people aren't going to be involved in the next web. And so, we wanted to kind of get ahead on that. we've been you know, we've had a lot of people that are MCP server aggregates contact us and add us to their directory, add us into their listings and things like that. So, that's been pretty exciting to sort of, you know, be featured on on some of these things. We've made a chat GPT extension. We've done, you know, the recent meta stuff that they rolled out with their views AI as well to add into there, too.

124:19 so, we're just trying to get ahead of the curve. I don't think the curve's quite there yet. You know, a lot of people that we work with are you know, this is this is a real business-to-business company. We're We're dealing with archival producers. We're dealing with researchers. We're dealing with documentary filmmakers, editors, producers, stuff like that. So, you know, we spend a lot of time on LinkedIn and, you know, messages between people. We have great relationships with a lot of the people that are our buyers of our content. And, you know, from a technology standpoint, they're not quite there yet. But, I think in the future as we think about how people want to make videos, how things like that will be on demand, almost like the on-demand radio stations that are happening now or or the the not even just a radio, but a live stream, a 24/7 live stream of someone having that broadcast ability now, where they're using maybe AI to create some videos and maybe the audience is putting questions and and comments in and that's kind of, you know, dictating where the the video is going like that. But, at the same time, if it had sort of that access where it could automatically pull up an old clip, bring it into the stream, and license it within seconds like that in a real-time sort of way, I think you've got some interesting use cases there in the future about how we think about having real-time content, bringing things in like that. Like, you think about like a live stream like this. Like, all of a sudden, you could be like, you know, the AI that's listening to this live stream could automatically go to a site like mine, pull a clip up that kind of exemplifies a point that you're making about something. so, yeah, the future I think that this stuff is going to be huge. I think it's going to be absolutely huge of people doing things in these real-time sort of scenarios.

126:34 >> Indeed. >> Fascinating stuff, man. You've got incredible range from to the licensing to it's good to get a read on just how active the agent economy is or isn't at this point in time. And then obviously this Flybrary stuff is just the next generation of weird that I'm very keen to stay in touch with. So, I appreciate you bringing all these things to us today. anything else you want to leave people with or you something we should should have asked about but didn't get to?

127:08 >> yeah, I mean it's it's it's just a really as you guys know really fun time to be alive. There's a lot of really cool stuff that's happening. It's tough to keep up with all this stuff. and I just try to keep an open mind, just try to learn as much as possible, and experiment, and try new things. And yeah, if anyone thinks that this stuff is cool and wants to reach out about, you know, chatting about this stuff, I'd love to chat more about it.

127:33 I'd love to hear any ideas about how we can leverage the content in all different kind of ways. I'd like to work on research projects too that were, you know, donating content to to help with, you know, fixing content that has been damaged. and other sort of, you know, interesting things that we can do with this. so, you know, it's just it's just really fun to just be doing all this stuff. So, I appreciate your time with this. And yeah, it's, you know, just just the beginning for a lot of this stuff, I think. I think that there's there's a long road ahead with preserving this content. And if anyone has content that they want preserved, that they want to donate to us, we'd love to talk. I mean, I think it's it's a great way to have your family history be out there. I think one of the things that's interesting, too, is we've thought about how can we have this content be preserved beyond us beyond me, beyond my daughter, beyond other generations? How can we get 100 years, 1,000 years, 10,000 years of this content being preserved without having it, you know, fall by the wayside. I mean, there's a lot of examples of films that have been destroyed that have are never going to be looked at again because of of a fire that happened in a theater or whatever kind of scenario went down. So, you know, that I think is really interesting. We I was and I lived in San Francisco many years ago and a startup I worked at a guy by the name of Danny Hillis came in and talked to us about his Long Now Foundation project. And I thought this was really interesting. As as a a 21-year-old, I didn't quite understand it, but now as I'm older, I understand it more about having to make this this 10,000-year clock.

129:37 anyone that doesn't know what he's talking about is this idea of like, how can we take all the information from humanity and save it in a way in case something bad happens in the Earth that anyone can read that information down the road. So, a lot of his technology is around printing things super super tiny because a magnifying glass would be a technology that anyone could eventually make. That's a lot easier to make than silicon chips or or whatever we have to to to try to save data and things like that. So, I think that stuff's important. I think that is kind of a core of what we like to think about, too, is how do we make this stuff really last for generations? Is there blockchain technology we use for this?

130:28 Is there this kind of micro scribing technology on you know, surfaces that are made of diamonds or or granite or something that isn't going to dissolve over time to really kind of save information for the long term. >> Cool. Fascinating stuff. Bill, the company is stockfilum.com. Thanks for joining us on AI in the AM. >> Thanks, guys. Had a lot of fun. Hope to be back soon. >> Appreciate it.

130:59 >> Take care. >> Sure. Okay. There we go. Yeah, fascinating. I really don't know what to make of that fly stuff. I mean, that would that was interesting in a couple different ways, but the the thing that he sent me originally that just caught my attention so much was this fly brain watching this film and I guess a takeaway from it is like it's vibe codable.

131:46 You don't know you can anybody can take the fly brain and vibe code it into doing something, you know, whether it's like actually working or doing anything really, I think it's like not super well established by some of these demos. but yeah, maybe I'm now I'm kind of thinking what do I want to vibe code a fly brain into? and you know, is there anything that a fly brain can do for me more broadly? It's yeah, I I there's still more questions than answers probably on that level. But interesting that the big companies are interested in his data, too. I mean, it It didn't sound like there was necessarily like huge money for this, which makes sense.

132:34 I We didn't get into like, you know, what about Runway? What about Luma? What about these folks, you know, what about Google for its you know, for its VO type models. He didn't name those names as much as just, you know, OpenAI and Anthropic. So, presumably they would want this for like understanding at least. Other companies, you know, might be interested in in more generation type stuff. I just saw a headline, too, that OpenAI's maybe got a plan to revive Sora. That would be a little weird if that were to happen.

133:10 But, yeah, another If nothing else, it's a great reminder that the there's all just how vast the data sets are that are going into this, that that this is like a corner that they are you know, getting down to and actually willing to pay for, just like found footage from decades ago. >> Well, >> No stone unturned in the data game. >> Well, one of the whole low background steel pieces of this is that it's very hard to get low background steel because everything has been tainted.

133:42 >> Yeah, comes from like shipwrecks and stuff, right? I mean, it's like >> Right. Right. Right. So, every everything else has been tainted. So, to the extent that he can guarantee that he got it from actual physical material and it's not like, you know, some random company kind of just faking it, you know, for cents on the dollar and trying to scam the the companies. That is very useful. So, as we've seen, like data data provenance becomes like very very important because the incentive to fake is just so high.

134:21 It's like a 99% profit margin if you can fake the data. And if not, it's like a 10% profit margin if you can't. So, the incentive to fake is like so so high. and it's very hard for the companies to get you know, data which is not fake which is not already using another LLM or another AI model to generate what is supposed to be authentic data, right? So, >> Yeah, fascinating.

134:54 what else do we talk about before we break for today? >> Well, I think we are on the verge of Opus the next the next Opus I believe. we're on the verge of perhaps Opus 6. That's what I'm hearing. The next Opus mainly because I think Gemini has caught up. So, it's time for it's time for you know, our friends at Anthropic to kind of throw throw some water on the Gemini fire.

135:34 >> It'll be interesting to see. I mean, I I I haven't So, I had the opportunity to to do a little bit of Gemini 4 pre-release testing. >> Yeah. >> And I you know, it it just as a sign of the times of like how high-leverage the ability to instruct agents to do this is, the first approach I took to doing the testing was "Hey Claude, go find in our recent threads some you know, diversity of projects and diversity of tasks that we've worked on recently.

136:11 Go prompt the new Gemini model with the same context that was given, you know, to the to the model that did the thing in the first place, and then give me like side-by-side comparison. Also, do your own evals, but then also, you know, give me a side-by-side ability to to compare things. It seemed to do pretty well. I would say, you know, I came away with the general sense that it is like a competitive model that's, you know, let's say in the same class as the latest stuff from OpenAI and Anthropic. it did seem to get it it it it did seem like those models are winning still the head-to-head comparisons a bit more as judged by Claude, so obviously some bias there, but also as judged by me to the limited degree that I was, you know, going through essays and, you know, really trying to form an opinion on which of these things is better.

137:11 So, I'd say it's still probably the case that for, you know, your kind of daily driver down the fairway use stuff, I don't think it's displacing either Opus 5.5 or Fable 5.1 if you're, you know, going a more you know, taste down a more taste-centric path with whatever work you're trying to do, or Astra. But it does it does seem like it it puts Google back in the game. But then the other thing that I was thinking about, and this came up as an interesting topic yesterday at this little workshop that I did in Chicago, I showed my setup, and my setup is really has been very Claude-centric, and now I've kind of promoted the the OpenAI models to to parity with Claude.

138:00 And they said, "Anybody else, you know, would you put anyone else in that mix?" And I had to kind of think about it for a second and think about like why I haven't. And the answer really that I came down to was just a mix of like security as measured by things like Adam Glaeve's ease of jailbreaking models. And then to some degree also just familiarity and predictability from my current subjective position.

138:34 It wasn't really benchmarks. It wasn't really like, "Oh, well, I tried this task. It was better at you know." There's obviously so many tasks out there. The surface area is so vast. My limited sample of tasks which I probably have kind of optimized, you know, in various ways. The you know, the Claude prompts for more, right? They like it's not exactly an apples-to-apples comparison when I I say, "Here's exactly what I gave Claude. Now, can Gemini do as well with exactly that same input?" It's a little unfair to Gemini cuz I didn't take the time to do the optimization. Whatever optimization went into that Claude prompt is you know, it would have been a different optimization that would have gone into the Gemini prompt had I been using Gemini along the way.

139:17 so, I think it's like I'm not really able to get, you know, to sort of even on a limited sample size without a substantial amount of effort, something that feels to me like real ground truth. It's like, "Can I develop an intuition? Are there places where this thing this new thing is like blowing away the competition?" I didn't immediately find that. And then I was just kind of like, "You know, I'm pretty happy with my current setup.

139:39 This doesn't seem like it's blowing away any of the models I'm currently using. If there's not a step change that I can identify where I can say like, "For this kind of thing you should always go to Gemini 4 cuz it's just clearly the best at this." And there may be some of those that could still come to my attention as the community, you know broadly figures out what's what over the coming days. then I'm kind of like I don't know. I just I kind of know what I have. I'm pretty comfortable with it. I feel like I'm going to get better results because I know how to work with these things a little bit better. My system is kind of implicitly optimized for them.

140:15 not that it's been you know a super rigorous optimization process, but just all the little points of feedback have been points of feedback I've given for these other models. and then the security thing just kind of loomed large. I was like, you know, I've got these autonomous agents. They're talking to my you know main agent on my on my computer that has all my stuff. I feel pretty good about the fact that Adam Gleve and team found it like quite hard to jailbreak the OpenAI and Anthropic models. Gemini was like next on that list, but definitely a step down and then something like a Grok I was like, yeah, I I I it may be similarly powerful, but why would I take that risk in a high context environment like my personal laptop?

140:59 And so on reflection it was interesting that we've kind of left a a point in time where I used to be so eager to always like, you know, try the latest and greatest model and see where it's better and whatever, but now that's become kind of hard to do and security is honestly kind of a leading consideration for why not to go Russian switch. And that kind of surprised me. Like I didn't really have an answer. I didn't have a cached answer coming into that interaction and you know, in working it out through real time, I'm like to a to a remarkable degree the models are competing for my use on trust and security already at this point.

141:44 and I'm I'm reluctant to swap them out even though my whole point of the talk was like, you need to own the context and you got to be ready to swap the model. Why won't I swap the model immediately? It's because there's not enough of a step change to make me take any additional risk, and I I do perceive there to be some. So, that was a interesting upshot of working through these tests and also just, you know, reflecting on like what really matters to me given how well the system is performing. I'm no longer maybe even somewhat stronger, but I think still true way to say it is I'm not relentlessly performance maxing quite as much as I used to be. I'm now a little bit more trust and reliability maxing because the performance has got to such a point where it's like really, really good.

142:35 and I am giving so much access that, you know, you could be slightly better on the, you know, the slide design, but what I really care about is like that you're going to stay on the rails and then you when you do your research and if somebody's got a, you know, prompt injection attack on their website or whatever, you're not going to fall for it. That actually matters a lot more to me now than like often like imperceptible performance gains.

143:00 >> So, I think I think it's primarily because it's cleared the performance hurdle, right? Like it's because it's cleared the performance hurdle enough. If it hadn't cleared, for example, if like it couldn't make the slides, then you would probably want to try, you know, some other some other thing. But it's cleared the performance hurdle as as you saw from your talk in Chicago. or you saw from your presentation for for Chicago. it has cleared the performance hurdle and therefore you're comfortable handing it the task. And it's not as though you can, you know, there's diminishing returns on once you clear the performance hurdle, there's diminishing returns on like, you know, future improvements for that particular task.

143:47 So, I think I think the question for me now is that I think it has clearly it all the AIs in general, but particularly OpenAI and Anthropic, they've clearly cleared the kind of intelligent subordinate kind of role. They're there. Now, the question is going to be can they innovate beyond that for the day-to-day? So, that you end up like switching. Right? So, for example, I mean, the classic examples are going to be can you run my YouTube or Instagram channel for me?

144:25 Can you run my small business for me? Can I hand off like this this kind of small product business, drop shipping business produce a product in China ship it to somewhere in the US to Amazon do the do the kind of advertising and marketing on Instagram and other places and then sell the product, deal with the customer returns, deal with the customer issues, deal with Amazon and get me the money back. And all I have to give you is a small amount of capital and this can run. Right? And and and the question there is that you could actually individually like clear every single one of those tasks.

145:10 and individually, I think it they can do it. But then, you have this like coordination problem of like is this like video that I'm doing for Instagram, is it good enough? am I you know, am I using the AI models correctly? Because you can tell, you know, these models to create those infomercial type videos now. But you're still doing it. Like you're still most of the time doing and many of the services are preventing you from automating. Like Substack and TikTok both have very strong bot protections and they're trying to prevent you from uploading using a bot.

145:46 And so it's still very very strong protections and you can you can do it with Codex and Claude, but it'll occupy your terminal. It has to take over your computer It has to do computer use on your terminal. It's going to take over your terminal. It's going to click around in your window while you're browsing, which is very annoying and so you you have to give up your terminal basically. And you can't do it. Substack will prevent you from automating using normal, you know, browser controls. You have to computer use and you have to take over the terminal.

146:20 TikTok is the same. TikTok is very strong protections. so yeah, it it remains to be seen. I think I think my my my thing has always been that Elon will be the first one to break the truce. it was surprising that because Elon actually was not the first Elon tried, but his technology was not good enough. Mews was the first one to break the truce. but then Mews got blocked in a number of places. So people saw when they started off using Mews, they thought it was very useful. And then like three or four weeks later, the the the usage has declined significantly.

146:55 So now Mews has entered this kind of, you know, okay, we've had our first kind of viral wave. We need to make this product an everyday thing. How are we going to do it? still tough because the for consumer, I think you need speed. Both Mews and Grok and OpenAI's agent, they're all very slow, probably because, you know, you're you're they're sharing resources on the back end. so yeah, I think I think someone who comes up with a very fast implementation, I think Instinct, I hear, has a pretty fast implementation, but they're spending a lot of money on compute.

147:31 maybe a very fast implementation, I think might work might be something that would get someone to switch. Not a Not a better one. Like just a more like slightly better on the execution doesn't get you doesn't get someone to switch, but much faster on implementation, I think yeah, that would be a key key differentiating factor going forward. So, not capability, but speed perhaps. >> Yeah, I mean honestly even on that front, I'm like increasingly just wowed by the speed of Opus 55.

148:03 I mean it would be nice if it was even faster, but I've had the somewhat bizarre experience lately a few times where I've been kind of like what you're done already? Like I I needed more of a break while you were working before I was ready to come back and evaluate your outputs, you know, it's a it's a strange thing, but truly felt like my mind is still you know, recovering from prompting you and you've already done all the work and now I'm like becoming the bottleneck, which you know, that's obviously what they want to design for, but the fatigue factor of like damn, I was about ready to go, you know, take a break and and you're already back with the the result.

148:47 yeah, I don't expect that to really carry the day, but it's just subjectively it does does cause me to think I don't know, like it is it that much better for it to be that much faster? I'm sure it will be, you know, once I once I see something that's like 10x faster, then I'll be like, oh my god, this is how do I >> never you'll never be able to go back. You you'll never be able to go back.

149:05 >> you. I mean, we we went through this with you know, in past computing cycles, right? Like the 386, 486, Pentium, Pentium 5, etc. like cycles. And we went through this with web access as well. we've gone through like there are clear signs that, you know, a couple of seconds speed up on e-commerce loading page is like 50% more sales. Like like ridiculous like absolutely ridiculous numbers on latency and stuff like that. And I think I think I I think there's a lot more to come. I think there's a lot more to come. I think current currently what we're seeing is Rust the carcin- carcination. So so so carcination is basically how in every branch of the evolutionary tree every branch ends up creating crabs from first principles again.

149:59 And Rust, which is a programming language which is memory safe and a modern program language is memory safe. it's a system system level program program language has the crab as its you know logo or and a lot of our current infrastructure is now being translated into Rust. One of the problems with Rust over the period of years is that very few people knew how to write good Rust. And Rust is very difficult to write because you had a bunch of like language specifics like colons, inverted colons, etc. which meant different things and different things about memory. And so it was very hard for you know most coders who grew up in like you know semicolon and like spaces were like the maximum number of like expressive things that they've used to kind of switch to Rust.

150:47 But as a result of AI being very good at translating between two languages given that you have the specifications and given that if you have a C program or a Python program you have those specifications you have those test ready you can translate into Rust pretty easily. So right now we are seeing across the board people are translating all manners of software into Rust the carcination like you know era. we are seeing for example someone translated Plex which is a software that you use to control home movies on.

151:19 and Plex as you can imagine this company is not a is a company that's been around for at least 10 or 15 years but it's a maintenance it's at it that's it's at the maintenance level. So at the maintenance level, all of the people who really knew how to write great software are gone and you have like two or three guys in the organization who really know how to write great software, but they don't have the bandwidth to kind of do everything.

151:42 So they're just kind of maintaining. Everyone else is like tacking on like features onto this thing which has technical debt going back like decades, but you know, those three guys are like, you know, let's just keep things running, right? We don't have time to refactor anything. And this guy comes along, single person, he rewrites Plex in Rust and he makes it like, you know, 10 times faster and he's like it's open source, anyone can download and use it. And so I think we're going through this causation era where everyone everything gets translated into Rust first. and so now we're seeing all of this software moving to Rust, everything should get faster. it's very exciting. Like so and and I think you will see it. You will see it.

152:20 you may not notice it because I think you hedonically you will adjust very quickly. So you're just like, you know, it starts working a little bit faster every day. But then like if you wanted like a year from now you want to go back to software or, you know, websites that you were using like a year ago and you looked at like, you know, a video of the experience, you'd be like, man, this is so slow. And I think the same thing's going to happen to all our AI. Like we're going to be like we're not going to notice it's getting faster, but it's going to get a lot faster and then you'll you're going to look at experiences from like, you know, a year ago and you'll be like, what was I doing, right?

152:57 >> I don't know how much more acceleration I can take, to be honest with you, but you know, hopefully I'll prove neuroplastic enough to take at least a couple more speedups in stride. It is it is a wild time, you know, doing five times as much as I was two years ago. can be dizzying. It's fun, but it can be definitely dizzying, too. And I do wonder at what point my mind might just not be able to go past another another speed up. It's Especially if there's another Opus coming that soon. I mean, we just got this last Opus, right? Like, good god.

153:41 I I I really hope they don't, honestly. As much as much as I love the Claude models and what they do for me, I think this would be a great time for them to chill a little bit and like let the let the temperature come down a little bit before they go with the next push. >> Unfortunately, I don't think that's a that's an option because markets are very badly down. So, markets have been trending down. It's also because of the midterms and like, you know, higher interest rates, etc., etc. But, the note on opening eyes revenues, etc., last week that opening eye was looking at like 50 to 60 and not like 70 to 80 billion you know, Q3 kind of altered the tenor in the market. The market just does not like AI. The guys on the East Coast just don't like AI.

154:29 it's not it they they just believe it's a bubble, and it is a bubble if in case we don't see significant like new revenue and new GDP growth next year, it is a bubble, right? If we don't see increase in GDP within the within the 12-month period, we are going to see a credit collapse. We're going to see a 2008-style and we're going to see massive massive delevering in the markets. because ev- everything is just set up for it. Everything is just set up for AI to actually deliver growth, and if it doesn't, the markets are going to tank, like absolutely tank.

155:06 and so, a lot of stuff depends on Anthropic at this point. Anthropic is releasing its Q3 numbers today to investors later today. They're going to sit down with investors and release their Q3 numbers. They are being pushed because not only do they have their, do they have their, you know, IPO coming up soon, other members in the market, the people that they buy and sell services from, are suffering. And, you know, they they have a tie-up with Micron. Micron stock down like, you know, 50%.

155:39 If they don't, then other people lose the ability to, commit CapEx to these programs. And so, it is they have to manage the stock price of their partners. If they do not manage the stock price of their partners, their partners are not going to commit capital. And if their partners don't commit capital, then Enphase has to pay for everything. And, and so, this is where I think the like I said, the capital the capital markets have are already like locked in. The capital acceleration is already locked in. They you cannot The capital markets will not accept, you know, 2% GDP growth next year.

156:21 And so, if you can't get to 4% GDP growth, if you don't pull out the stops, you are going to get a credit-driven downturn, and the company the company your company might not go bust. Dario might be like, "Oh, I didn't commit to all of this." But, all of your partners will start going bust. And if your partners go bust, then you're going to go bust, right? They're going to They're going to be like, "Look, we can't support your, expansion plans. We can't extend you money. We can't extend you credit."

156:46 Like, you know, the banks are not going to be willing to lend to you. Like, it's it's just like, you know, it's a major major change. So, and also, you also have the specter of the the US, federal government is not in a very strong fiscal position, either. So, federal government can't can't really bail you out. There's only one way left, which is high growth. High growth bails everyone out, right? Like, your debt-to-GDP number in your federal government looks great because okay, you have your debt growing at 5% and your GDP is growing at 2% but now your GDP is growing at 4%. You're you only have a 1% deficit. And then next year your your GDP grows at, you know, 6% and your still your debt still at 5%. Wow, you're going to be in surplus. And then all of all of a sudden all the curves all the looking forward curves change, right?

157:33 All of the oh, you know, we're going to run out of money and so forth for social security in 2030s. Everything changes, right? So, you don't have that much more to go. You have like maybe 3 to 4% more that you need to grow. and then you're there. But if you sit around saying we're going to cure cancer but GDP growth is going to be at 2%. yeah, markets markets can't handle that. Like markets will take a downturn, will dump.

157:55 And then you're you're going to be in trouble. So, we're at the we're at the moment of reckoning, like 12 months, you know. It's It's been a long time coming. We've been like pushing back, pushing back, pushing back. But this is the moment of reckoning. Either AI has to do something significant or we're going to have a, you know, a bust. So. >> We need an AI adoption core. That's I I don't think I've I I I tweeted about this. I don't think I've I've mentioned this on here. Maybe I'll tell this story briefly and then I should probably break for today.

158:31 But the other day I was walking, you know, in keeping with my personal metrics for success with AI including more time outside, more exercise. I was taking a walk around my neighborhood and enjoying the lovely fall colors here on a beautiful day. And I passed by a neighbor's house, elderly woman sitting on her front porch, and just decided to stop and talk to her. And sure enough, what she had for me was questions about something going on on her phone. She's like, "I know I it's saying I've been hacked. Do you know what that means? And I'm like, well, I know what that means, but then I also think I know what is happening to you. so because I'm you know, maniacal like this, my impulse was to pull my phone out, get ChatGPT voice on, and hand her the phone and say, hey, why don't you talk to this? See if this thing can help you.

159:27 So, she did and you know, explained the situation to it. It gave her very good advice. Basically, you know, as you would expect, right? Like, I don't think you should click on that. That sounds like a scare tactic. Are they asking for money? Yeah, they have it's going to be $2 and then it's after that it's $12 a month. And, you know, the so the AI is like by the way, you know, did you give them any money already? You know, and she's like, no, no, I haven't paid for anything. Okay, good, you know, close it out. Don't worry about it. Your iPhone has great security on it already.

159:59 you know, as long as you don't give people your stuff, like you should be okay. So, she took that advice. Okay, great. then I was trying to like, would you like to have this on your phone? She said, yeah, that sounds great. That was really helpful. couldn't remember her password for the iPhone, so we couldn't install the app, but I was able to just install the 1-800 ChatGPT, not install, just save it as a contact. and I do think that's a pretty cool service that OpenAI offers. I mean, it's you the initial thought for many, of course, who are into this stuff would be like, why would you need a 1-800 number? Like, who's ever going to use that? And the answer is my elderly neighbor who cannot you know, cannot even remember her own password to install a new app on the phone.

160:42 So, she's got that now. She, you know, can call that and it just left me with this feeling which I've, you know, approached a couple times from a couple different angles, but again, really drove home there is just an unbelievable amount of adoption work to be done with the current models. This didn't push her anywhere, you know, this this did not push the model's capability anywhere close to its limits, right? This is like super easy mode for the model. But, it's the kind of thing that isn't going to happen without like people actually going out and trying to help the people in their lives adopt this kind of thing. This is something that China does pretty well.

161:27 Like, they will actually have young people go systematically and like help old people get set up. You know, and there it doesn't mean their old people are really like becoming open claw power users as is sometimes reported. But, you know, they like get the app so that they can actually, you know, have the the help available when they need it. And I I I think you're right that like the capital cycle is does need to see results soon. But, I think we can get 4% growth without additional hyper scaling at this point.

162:02 You know, and I want those medical advances, all that good stuff, too. But, like, do we really have to go harder on the you know, frontier scale RL? I think the answer is no. I think that like literally just going around and helping people use what exists is enough if we actually do it. but, I'm the first person who stopped by my neighbor's house and, you know, we don't have a national program like this. This is something I think would be an incredible good for the individuals who would get the help and also, you know, just to make sure that we actually have the sort of economic cushion to, you know, be able to pace without like the, you know, the capital market sort of Damocles immediately coming down on us.

162:47 if we just put some shoulder into adoption, I think we could create a little space and would love to see more of that type of thing happening. >> When when when when I think of AI, I think I think a little bit of AI contributing to GDP. I think a little bit about like credit cards. before you had credit cards, you used to have to like issue issue checks. So you would you would go to a retail store and you would issue a check and that check would be from a bank and the and then you know the the store would have to go to that bank, find that bank, go to that bank and get you know get get the money sent over.

163:25 And the US had laws against like you know writing bad checks. So writing writing a bad check would you know you'd get thrown in prison. And after that you had this transition you had this transition the store issuing credit. So the store could issue credit. So the store issued cards were the were the first cards. And then you had you know this kind of credit card system which was the banks getting together and you know forming this kind of credit card system.

163:46 So I think the the thing that strikes me is that as you saw that progression, it got easier and easier as a consumer to buy stuff. Right? You got because because you could and then you had the the ability to return stuff too. The return and get your money back. So that that entire progression of like having the confidence and go buy something, they're not going to get ripped off and you know that made like consumption easier.

164:11 And so what I think needs to happen with AI is that people have to have enough confidence that their consumption is actually easier. And I think that's one thing like every time I hear someone like you have to learn AI you're going to get left behind. I get so annoyed because I'm like if AI doesn't make your life easier, like if you actually have to learn this stuff, then it is useless. Like absolutely useless. And we should take it out and we should put it in a barrel and shoot it, right? it should make your life less complex. There should be less complexity. And I think part of the problem right now is that we're in between the two regimes.

164:51 We're in the old regime where you have to check your own email, enter your own OAuth passwords, have all these passwords ready, and like you are ultimately responsible. And if you've entered your password and OAuth and something bad happens to you, that's your fault. Right? Like we're in that regime. And we're not yet at the credit card regime where you're like, "I don't care. You guys take care of it for me, and if you screw up, like pay me back." Right?

165:17 We're not yet at that regime right now. And we need to move to that regime. We need to move to the AI regime where I'm like, "I don't care. I don't care what my my passwords are. I don't care what OAuth is. I don't know what OAuth is. I'm not going to do two two-factor. You guys take care of it." The responsibility for my online safety is not on me because it's, you know, I'm obviously undependable. I write down my passwords. Everyone writes down their passwords. Or everyone uses the same password. I have the same password which has been leaked like 50 times, and, you know, I've been warned. Every every And even if I change it, the next month something else will be leaked and I'll be warned again. Like everyone does this.

165:54 And the fact that we ignore this, like cybersecurity people at large companies ignore this, is a travesty. Right? And so, I could see this thing where AI we start using AI and, you know, stuff like this happens and the AI uses like, you know, a password which has been used before, it leaks, and then you go back and you say, "I'm going to blame the guy." And it's And the And the government just steps in and says like, "No, you you're not going to blame the guy. The regime has changed. It is up to the company to ensure the security, online security, and safety of the user.

166:24 And if they can't, then they then they can't be in business. The AI can't be in business. You can't You can't do this, right? So, we have to switch to the new regime. And I think we're in that in that transition period. I think the AI is almost reliable enough. I don't know I don't know there's a privacy issue because I feel like a single AI model should be able to facially recognize and identify every person's voice in the world.

166:50 Because if you have like you you only have about 8 billion people in the world and you have like 10 trillion parameter models, you can assign one parameter for person and it's only like, you know, 0.8% of your parameters. And then after that, you can basically, you know, identify you know, actually identify every single person. And you can But I Maybe there's a privacy issue there. I I don't know, but I would think that, you know, this kind of anonymity is just not is just something like is an illusion that we have. And you do actually don't need any password anymore.

167:21 You don't need any of these two-factor anymore. All you need is a trustworthy AI on your terminal saying it's you. And if you can get there, then all of this stuff dissolves. You If you have a AI device and that AI device is trustworthy enough that it can say that it's you, and you can use that AI device for anything, you don't all of this stuff disappears. All all of it. Like the passwords, the email the email address passwords, it's everything disappears, right?

167:47 and and I think that's that's what needs to happen. You need to transition to that new zone. And I think we'll get there. and and and it'll make everything easier. You just have one AI that you talk to. You don't need to know WhatsApp. You don't need to know You don't even need to use know how to use all these apps on your iPhone. The apps on your iPhone are ridiculous. You have six messaging apps. Why? Why, right? So, we're on the transition period like the early stuff like Grok Bot and all this stuff is is early.

168:18 I hope it gets better a lot more quickly than it should. I think I think like on a normal trajectory be like 5 years or so. Like Grok Bot or these things will slowly get better and better and finally like you'd hand over. But I don't think we have enough that much time. I think we only have like 12 to 18 months before all these hacks and stuff, you know, start becoming ridiculous. So, anyway, on hacks and scan I I just want to bring this up. So, AI finally comes to call centers. This from A16Z.

168:47 Year-on-year change in global call center employment. People have been saying like, "Okay, we haven't seen the AI impact for a long time." 2024, you kind of see, you know, employment going to zero. And we by 2026 is at minus 4% per year from previously plus 4% per year. I just want to say one thing. People who have call centers set up, who have like phone banks, and people who can work those phone banks, when they go out of business, they don't stop doing call centers.

169:17 They just start doing illegal call centers. So, I I I think we're about to see the scam wave like kind of peak. It It's kind of, you know, post.com bust. You had a spam wave because all of these email addresses were sold by bankrupt companies. So, I think we're about to see like the call center spam wave. Like, call centers like legit call centers shut down. Phone banks get sold. you see a bunch of scam artists like milking the last like year, year and a half of like scams. So, this is something to watch out for, I think. So.

169:51 >> Yeah, these could be I mean these are this is a very natural canary in the coal mine. So, on the economic impact side, it's a it's one to watch. And on these like scam sides, get your secret code word established with your family now, I guess, is maybe the best thing I can offer at the moment. cuz the clone voices as we've demonstrated are getting awfully good, too. >> Indeed. on that note, we'll be back with two days next week. I'm hoping to get in someone from the investment front.

170:24 A guy who just launched a new fund from with funding from Jaan Tallinn. So, you know, I I won't say Leap Forward number two, but but but a but a new newly funded AI hedge fund in and we'll have, several several more interesting guests. So, tune in again next week. >> Now, time to go get some of that exercise like you're talking about. >> Indeed. Nathan, good morning. >> Good morning. Thanks for gosh. >> Bye-bye.

Summary

Nathan shares his experiences with Tesla's Full Self-Driving (FSD) technology, highlighting its potential to enhance productivity and relaxation during long drives. He also discusses a recent workshop where he presented an AI-generated talk, which captivated the audience and prompted them to rethink AI's capabilities. The conversation shifts to the rapid advancements in AI, particularly in software development, and the challenges of integrating AI into existing workflows, emphasizing the need for adaptation and understanding of AI's evolving role in various industries.

- Nathan praises Tesla's FSD for making long drives more relaxing and productive.
- He successfully presented an AI-generated talk, which impressed attendees and shifted their perceptions of AI.
- The discussion highlights the gap between AI capabilities and user understanding, suggesting a need for education and adaptation.
- Nathan notes the rapid advancements in AI, particularly in software development and version control.
- The conversation touches on the challenges of integrating AI into existing workflows and the importance of user trust.
- There is a growing concern about the potential for AI to disrupt traditional industries and workflows.
- Nathan emphasizes the need for a proactive approach to AI adoption, focusing on user experience and understanding.
- The discussion concludes with reflections on the future of AI and its impact on productivity and everyday life.

Questions Answered

What was the experience of using Tesla's Full Self-Driving feature?

The speaker shared a positive experience using Tesla's Full Self-Driving (FSD) feature during a day trip to Chicago, highlighting its effectiveness in reducing fatigue and allowing for a more relaxed travel experience.

How has the speaker's workflow changed with the new AI tools?

The speaker transitioned from using command line tools to a personal AI agent console that integrates various functionalities, resulting in a more efficient and user-friendly experience.

What does valence mean in the context of AI models?

Valence refers to the emotional direction (positive or negative) represented by AI models, akin to an axis that categorizes emotional states.

How does text influence emotional representation in AI models?

The text used in AI models should ideally be neutral to avoid biasing the emotional outcomes, focusing instead on the emotional states being tested.

What are the potential benefits of activation capping in AI models?

Activation capping could mitigate distressing representations in AI without affecting the model's capabilities, serving as a precautionary measure.

What is the significance of human-made content in the age of AI?

Human-made content, particularly that which cannot be replicated by AI, is seen as a valuable resource that may increase in worth over time.

What legal considerations are involved in using content for AI?

The speaker discusses the importance of having rights to content, differentiating between commercial and editorial use, and the complexities of intellectual property laws.

How do AI companies perceive their need for data?

AI companies express a strong demand for more data to enhance their models, indicating a potential market for additional content.

What insights can be drawn from biological analogies in AI?

The speaker discusses the potential for AI to learn from biological systems, particularly in terms of compute efficiency and representation.

What is the current market sentiment regarding AI?

The market is currently skeptical about AI, viewing it as a potential bubble unless it demonstrates significant economic growth.

© transcribe · For agents Built with care and craft by Gokul Rajaram