📆 ThursdAI - Apr 24 - GPT-image-1, Grok 3 API, Magi-1, DIA, and agents drama

0:27 Check it. Welcome everyone. Welcome to Thursday I. This is Alex Wov with Weights and Biases. H welcome to Thursday I April 24th. Today we have quite a lot of guests on the show. We also finally finally got that chill week we've been waiting for since maybe two years ago. It seems like after the big releases week of 03 and GBD 4.5 major labs are a little bit taking a little bit of break to probably release bigger things next week including potentially some open source stuff and two but we still there there's a bunch of news there a bunch of news in open source there's there's quite a lot of stuff so I want to welcome a few folks here to help me talk through them as well I want to add Quinn and LDJ hey Quindla hey J folks who are listening, tuning in. I welcome you also to hop on the video stream and also take a look at this. I want to introduce Quindla Kramer to the stage.

1:30 Quindler, welcome. not your first time here. You've been here for a minute, so folks know who you are. How are you doing? I'm good. It's good to see you. Good to see you as well. And we have LDJ as well. LDJ, how are you, man? I'm doing good. Awesome. I will just add the the video to the Twitter space so folks who are on the space will be able to see. did I I just remove LJ or did he remove yourself? all right. So we have we have quite a show today. We will have a few guests like I said including some friends of ours who did their own thing. I I'm very very interested in hearing about smart turn and to have a discussion about this as well. And I think that after the the explosive week last week, it does seem a little bit like it's it's a chiller week this week. So instead of covering all the news in depth like we do and we barely have time, hopefully today in a very chill way, we'll cover everything without rushing. also want to acknowledge some feedback that we heard about the show's structure. so folks, I'm very very receptive to feedback. I want to make the show good for you. So I would love to know if you're enjoying the TLDDR sections, not enjoying them. I heard some feedback from folks who said it's really helpful for them. So I think that this is what we'll now do. We'll jump in into the TLDDR. at Yodr as I said before for new listeners of the show covers everything that we are having on our docket this this show a lot of the times we have breaking news hopefully today we'll also have a little bit of breaking news and and so the show changes but basically this is kind of what we are here to talk about besides maybe one other thing that I want to add at the end of the show if we will have time so with that let me go to the LDR Alrighty. This is everything that we have to talk about on Thursday for April 24th. And we'll start with hosting guests. Obviously me, Al Kulkov, I mean the Avengers with Weights and Biases. We also have Wolf's going to join us soon.

3:50 Yamas here, LDJ. and then we have two guests joining us. Squinta Kramer the founder co-founder of daily and maintainer of pipecat the open source and our resident voice expert everything voice AI and we'll also have Dex Horsey. Dex has been on the show before as well and Dex is we'll call him our resident agent expert and Dex is the founder of human layer and he will join us to talk about how to build 12 factor agents something that he wrote and authored and would love to hear just about also the update in the agent world in the open source we had a few things this week nothing super super crazy I think the most interesting the highlight of the week for many of the Thursday listeners was OpenI is closer and closer in to their open source release and they've met with the open source community. So once in San Francisco and then yesterday over Zoom they met with a bunch of folks from the community. we can see this is a screenshot of Prince Kuma who's like very who's been on the show and does the MLX stuff and very happy to be there. my my Mazar friend of the show has also been there and he they will hopefully let us know. I don't think that OpenAI limited the conversation so they can share what the conversation was about. Also in open source Nvidia released describe anything three billion parameters they call it DAM. if you guys remember SAM segment anything it's kind of like that but also describes. It's pretty cool. We're going to show an example of this and going to play around with this. it's a LLM, multimodal LLM for region based image and video captioning. I haven't seen the video part. I only seen the region like the image part, but that's absolutely super cool. I want to dive in this QAT thing from Gemma. Google released a quantization aware training QAT Gemma 3 models and they're like significantly smaller and I know that Wolrom who's going to join us soon he already tested them with some interesting results and welcoming Yan Pel as well to the stage would love to chat about what quantization aware training means and in the big companies and APIs I think again not a crazy week finally we got a little bit of a break we can breathe we can try out all the new things And then I think the highlight of this week for me at least was OpenAI finally releases their image gen which they call GPT image one in API so that developers can use this in API. there's a few a few things you can do. You can edit an image, you can generate an image. it's it's pretty cool.

6:37 the fact that it's now integrated into multiple products as well. They launched together with I think five or six products. very very cool. So we're going to chat about this, see the limitations there and play around with this Grock. We haven't talked about Grock in a while, but there's like a few things about Grock. First of all, while we're still waiting for Gro 2, Uncle Elon, to get released in open source. we are still waiting in that.

7:03 Grock 3 and Gro 3 Mini are finally available via API in the API. and this means that folks can do evaluations on them and they can build with them and they're integrated in multiple other products as well. so we'll chat about this. Grock also launched a video mode in its iOS app where you can talk to it and it can see you. and yeah, we should definitely chat about this because we remember how long it took for OpenAI to kind of announce this and then release and then in the middle Google released this and now it's pretty much everywhere. So definitely worth chatting about and maybe even trying on stage.

7:39 We'll see if we can make this work technically. I also want to chat about our week after 03 because I've I know that folks from open air are listening to the show. So hopefully this will be as some for feedback. I've been using 03 almost exclusively. Would love to chat with the folks here on stage after our excitement about 03 last week. And reminder last week we did three live streams. Monday GPT4.1 was that Wednesday for 03 and then Thursday obviously for Thursday I and since then we've been all playing with kind of the the new releases. What what happens often and this is like a little break from the TLDDR. What happens often is that we run so hard to cover everything that happened that week that we're not we don't have time to tell you like about the releases from last week that we actually had the chance to try. So hopefully we'll do this today. We'll chat about 03. We'll chat about GPT 4.1. We'll we'll do like a summary of the things that we've used and we'll listen to the community as well. So, if you have thoughts about folks who are listening live, we have it looks like it's picking up. We have around 200 people listening in. would love to hear your thoughts about kind of the models that we that we that we use.

8:51 Everybody has their own pattern and I think it's going to be beneficial for the community. So, we're going to chat about all three as well. in this week's bug about and MCP and A2A first of all we have a hackathon coming up in San Francisco. This is the first you'll hear about this. It's coming up very soon May 17th 18th in San Francisco in the wits and biases office. I'll be there. Sam will be there. most of the weave team will be there as well. You are more than welcome to join if you are looking for a reason to go to San Francisco. that that could be it for you. I don't know.

9:26 maybe go go see the Golden Gate Bridge and then come to our hackathon. if you are already in San Francisco, there's no way nowhere better to be. We're planning to go all out and we're going to announce sponsors very soon. the other thing, the second announcement for weights and biases is fully connected, our flagship conference, our two-day conference where we bring a bunch of weights and biases customers together and have a bunch of speakers. That's happening in June. and that's coming up very soon as well. It was announced and it's incredible. Moving forward in vision and video, there's a very very interesting 24 billion parameter model called Maggi1 from Sand AI that came out of nowhere seemingly. It's a Chinese company and it generates long like long videos that are character consistent which is crazy and we definitely should take a look at some of these and hopefully maybe even talk about architecture because architecture is also crazy. so definitely worth sharing about Maggi1 and also another company goes into avatars fully character AI announced avatar FX h and it's in early access and we should talk about character AI. I don't use this but I know million people's use it. it's it's really interesting that they are doing this pivot into physical like you know visual avatars that you can chat with going forward in voice and audio. I think at least for at least as far as my Twitter account is concerned this was the highlight of the releases of this week a nari labs which is essentially a person in the half in Korea like it's one dude and his friend halftime in Korea have trained DIA a 1.6 6 billion parameter voice gen models with emotional tones and multiple speakers. It's kind of a podcast generation. It's very you could see the similarity to notebook LM from Google because the voice kind of talk o over each other. This blew up. I think I don't I remember already the feeling of like being super viral. I think my my video about this got I think half a million views or something crazy like this. people love this. People eat this up. This is an MAT license as well.

11:38 A very tiny, very tiny, very well performant voice model. So, we'll chat about this. Quintly, you had the chance to check this out. I already had the chance to check this out, but definitely we should play some examples for you because some of them are crazy. and also also in voice and audio, our own Quinn here. you guys have your own release pipe cat smart turn or I guess daily smart turn I don't know how we want to call this smart turn model which is a semantic VA and we're going to chat about if if that didn't tell you anything we're going to chat about what this means we did touch on this a little bit before but that wasn't open source and now we have an open source we have the author of this also here with us will tell us why he spent his holidays working on this And this is voice and audio. I believe that this is it, but we'll see. we in AI art in diffusion, obviously image one kind of AI art, but that that's in big companies. Hun Yuan released 3D Hunan 3D 2.5. It's a 10 billion parameter 3D model and the results seemingly are crazy. Those three models are getting better and better. We should definitely take a look. and definitely check it out. And I think the last thing the last thing on my agenda at least until folks come in and kind of correct me and see what else is there. we have we'll have Dex Horthy, Dexter Horthy, the founder of human layer to talk to us about his research essay that outlined the kind of the the important principles for agents. So if you are building agents or interested in agents for AI, Dexter will be let's say our resident agent expert for this week. He'll join us very soon to talk about 12 factor agents and so if he has the blow up on hugging or hacker news, not hugging news. I was about to get confused between the two on hacker news. H it blew up and we'll chat with Dexter as well. we may also have, if we have time, we may also have folks that built Suna, the open-source agent that also kind of blew up. Adam Cohen and some friends of his I see I want to see if Adam is in the audience.

14:00 We'll send them an invite and time we'll get to Suna. which they they called it Sunsam in the beginning and then they changed their name and if you read Suna in reverse, you get Manus. and we know about Manus. So basically it's an open source version that you can host yourself that does a bunch of things. and we should definitely talk with folks who release things in open source. We really really like this. I think that this is it for it for us for this week.

14:26 This is the TLDDR and this has been Thursday. No, I'm just kidding. Now we actually need to talk about all of this. But as you guys see and as Lincoln confirmed in the audience and shout out to LinkedIn for who prepares and follows all the news as well. It's been a relatively quiet week and we've kind of been asking for a quiet week. So, I think it's a great opportunity to just kind of like like take a pause for a second and then actually kind of dive deeper, maybe learn a few more things, maybe tell the audience. with that said, folks who are listening live on the on the stream on on Xpaces and on streams on YouTube, on LinkedIn, on X everywhere, if you're listening as well, please let us know what you guys think. Let us know what news we may have missed.

15:13 let us know what was interesting to you. We'll definitely also want to chat about the actual models themselves. So, please, if you have any comments, let us know. With that said, this was the TLDDR. We're going to start with open source. But before this, we now have the full almost full quorum. We're missing Wolffrram. welcome Yam Pel, welcome Nistan. How are you guys? What is the most important thing in AI as far as you're concerned? Let's start with Yam.

15:40 Oh, open AI. Open AI. Can you hear me? Can you hear? Yeah. Yeah. Open AI going open source like absolutely going open source. I mean more more than than it was known before. Like we all know that they're going to release a model but all of a sudden they they approved okay so their agent from last week codec. Yeah. Mhm. Codex apparently there was a pull request to extend it to providers that are not open AI so it'll be able to so you know you can use the agent with the anthropic or or whatever and you know you you don't expect even even if it's open source you don't really expect OpenAI to all of a sudden approve this that you can use whatever model that you want and out of nowhere they just approve it you can it's fully unlocked you can use it with whatever model and an engine that you want. That's that's impressive. That's really going open source like and moreover there just look at the amount of pull requests over the week. It's it's impressive like they are really working on this fulltime. There are people there really maintaining it. So these these tools are are extremely powerful. Seriously, codeex and and and cloud code are extremely powerful tools and and it's absolutely amazing to see this, you know. Mhm. N, what about you? What's your highlight of AI this week? What stood out for? I think I'm just happy they un nerfed Sonnet. It seems good now again. Wait, really?

17:24 Sonnet is unfed. Yeah, sorry. 3.7 thinking. I don't know. It's It's not being it's not being dumb anymore. So, I I like this competition. I am still a Sonnet 3.7 fanboy. I don't think anything replaces it. sorry. It what happened? well, it it was I was testing it before and for a while it just started getting even basic questions or like basic hard questions wrong. it it felt like the the inference had changed or the quantization or whatever. It was just dumb. And this was a lot of people reported all all at once. And I was actually able to compare past tests and current ones and solid 3.7 thinking was actually failing them and the one on Ella Marina was was passing them. So but now now it seems okay now. Now it's getting everything correct. So that so I'm I'm happy about that. On that side, I still feel like all hands and open hands is just flying under the radar for most people as to how good that thing actually is. They still have 50 bucks on their on their end. You can just run it on your own, too. And dude, that thing is crazy.

18:49 So, yeah, I actually prefer the way that they build the app as a single page or two that they don't just go nuts with Nex.js and everything because that becomes quickly a convoluted mess whereas this way it just keeps it all in context and you can actually use other models too with with that. So, I think those are the two things I want to highlight besides outside of the the voice stuff. All righty. And speaking of voice stuff, Quinn, what was the highlight for you before we actually get to those parts of the show? I mean, two things for me. The the DIA model, like I think the OpenAI guides in that training 4.5 YouTube video a couple weeks ago were saying that like half the battle is knowing you can do something and to have like a two students in Korea release a great model like you know what's possible. We should all be working on this stuff. It's super exciting. I also think we should shout out the Gemini team which released kind of spread out over last week and this week more stuff in their multimodal live API. So the big labs are also like continuing to push forward on speech to speech and the APIs for all that and that that's flying under the radar a little bit. I think the the interest goes in waves with the real-time API and the multimodal live API but the teams at both open AAI and Gemini keep like pushing it forward pushing it forward which is cool. No.

20:10 and lastly, LJ, before we go to open source highlight of the week and then, before LDJ, before you go, folks in comments, feel free to leave us highlights of the week as well. We'd love to see and highlight them on the show. LJ, what was your main main thing, main takeaway, main interesting thing from this week? the main thing coming to mind right now is probably 04 Mini. I think it's it has its downsides, but overall its ability, especially if you look at things like ARC AGI or if you don't really like that benchmark, just its ability to do various things where it it seems like in some cases even it could be even cheaper than 03 Mini cuz it's even smarter sometimes that being able to know when it should use less tokens and just the abilities and the the cost effectiveness of it. It it seems like we're just continuing on this path of a path of cost efficiency. Yeah. Yeah. We definitely should highlight the RKGI kind of results for of for Mini.

21:08 Definitely for folks who are concerned about very hard improvement and hard takeoffs. seeing kind of how these models improve on various benchmarks is very interesting. LJ, if you have that link, if you drop this, I'll definitely open this up. Folks, I think it's time for us to actually start the show and talk about open source, our favorite favorite subject here at Thursday. Open source AI. Let's get it started.

21:52 Let's get it started folks for open source for this week. we have well the score is open source AI in general but we're going to like mention voice and stuff as well here. we obviously need to start with not only the fact that o openi is like committing to open source in codec but also with the fact that this week openai have welcomed on the zoom meeting multiple of folks who are involved in the open source arena and we we we sent an invitation to folks who've been on that on that conversation and for now it doesn't seem like maybe I can get muzzi to come up and speak and tell us about this. But basically there was a a zoom call and a few participants there. Sam Alman was also one of them. A few folks from I think Thomas Wolf from HugenFace as well.

22:46 and this is not the first time that OpenI is kind of like treating the open source community and asking them for questions. They had a live event in San Francisco famously. They gave out like some swag that even like Open the AI folks didn't have. and and now they had another one that's remote and they they had a bunch of folks in there. I just want to highlight the fact that like many folks from the community from the folks who were like participating and listening in and contributing were on that call. Shout out to Prince Kuma Mazar Panahi and a few other folks as well. And I think we got a few more details kind of like not necessarily leaking because they didn't sign anybody for the NDA for from that as far as at least I seen there's at least a a few details that that came out. So one of them is OpenAI will release a reasoning model. The the we kind of knew this but I think now it's confirmed the open source model will be released as a reasoning model. I think the confirmation for time was also kind of there. We're like in the beginning of summer, which leaves us around the month. LDJ, what else do you hear? Yeah.

24:00 in that comment thread, Prince, somebody asked Prince if he has an idea of when it would come out, and he said something around summer and then there was another reply to that of is this like your own speculation or was was this officially said like during the call and he replied with a a mouth a zipper emoji like so I don't know. It's ambiguous. We don't know if it's OpenAI that said that or or maybe informed speculation, but yeah, it seems like within the next few months, it might come. How about we we tune in and listen from the man himself? Let's let's add Mazier here. Mazier, are you with us calling in? Looks like she on mute.

24:44 I guess while we're waiting for him to pop in, something else that was confirmed prior, I think maybe a week or two ago, Sam said at an event that they plan to have it be competitive with any existing open- source model of today, which seems like it'd be like at least on the level of, you know, R1, Llama 4, Maverick, models like that. That'd be very interesting to see if if if Llama kind of which I don't remember us talking about too much like we should probably mention Llama at some point and see. I haven't seen any fine tunes. I haven't seen like a lot of excitement.

25:20 oh, maybe he has to unmute himself. Yeah, Maz you need to unmute. Oh, there he is. Can you hear me now? Let's go for Let me just introduce Mazer Panahi for a second. cuz you've been on the Twitter spaces multiple times and we obviously like celebrated your work for a very consistent time. you've been one of the like more prominent finetuners in the finetuning community for open source and a friend of the pod but I think this is your first time on stage with definitely on video feel free to so first of welcome great to see you and feel free to maybe quickly introduce yourself so folks who know who you are and then we'll talk about yeah of course of course thank you thank you so much it's the first time being here like this so took me a minute to find the studio how it works yeah my name is Mazar I I work at the CNS it's national research for French government.

26:11 it's a public research. I do create AI models from the various small ones all the way to the very big ones. and I just want to tell you a little bit about the yesterday if you have some time. Yeah, please. Yes. Yes. Yeah. So I I didn't expect to see some of the people there. definitely I didn't expect to see Sam there at all. I I knew that people going to be from the training team obviously. but there were already some big names. Sam was there. he spend the first half of the conversation answering all the questions. So that was very nice of him.

26:49 people had a lot of questions regarding the size, regarding how often you want to do it. will this happen regularly? so he was answering all the questions and basically what he wanted to do was that their GPUs are on fire. So he he wanted to release open models so some of these loads would be handled locally. So that is his philosophy of how do we make sure we can help the community but at the same time the community helped them you know take some of the pressure on the GPUs.

27:23 And then we were divided into different rooms and h I was ready to fight with some of the people like on the size on whether or not we want the reasoning or not. But it was really great to see all the people with the same experience because for the last two years people were really fine-tuning and training all these models and putting in production. So they've been all battling the same issues and having the same experiences.

27:46 So of all the rooms being compiled, everybody was literally asking the same thing. They they didn't wanted a very tiny model. They didn't want a small model because to be honest, you could get a small models from different labs and different institutes, right? So you want a big model but not too big, not something like a 600 plus billion parameters. That is the mode. It takes two servers or something above 400 billion that takes the whole node and and the whole server has to be you know just for one model something around maybe 70 to 200. So you could spend four GPUs on that server on this model and then the other four GPUs would be for other models you know like your audio your video maybe you want something as smaller to deal with the intermediate steps you don't want to bother the big models so everybody agreed on that everybody really wanted the structure output h weirdly strangely nobody wanted the reasoning at all really yeah they were like but we don't need the reasoning because I guess people in production I mean the tokens are expensive. So yeah and and they they were explicitly asking please do not spend so much time doing fancy RL on this model because it's just going to be for ma math and coding and some really very specific tasks. So it was really they they were also surprised to see some of these comments coming from like a large amount of people.

29:13 so the size why do you think that is? I I'm really interested as far as I understood maybe from comments that first of all OpenAI told us explicitly they're moving towards a hybrid model. We saw it in Gemini 2.5 flash we saw it in Nvidia released something that's also like Neotron is also kind of like a hybrid thing. OpenAI announced they will move towards like hybrid thing cloud 3.5 so on as well and as far as I understood that that's what their plan was. Why do you think people do not want reasoning? I think people just realized from the open models that they do reasoning, you are kind of stuck in the reasoning. So you can't really turn it off. So they ask if you are going to do reasoning, please allow us to turn it off. So we can decide when we need it and when we don't. So we don't have to wait so much because we ask for a large model, right? So it's not a small model.

30:04 So you have to spend a lot of compute on those tokens. so that was one very surprising thing came out. the other one was really multilinguality because there were large majority of the people were coming from EU. So they were asking to have a good representation of the EU languages at least the 20 of them as good as English that I mean it's not that hard right and a lot of people asked for the base model. They were saying if you give us the base model there would be loads off your shoulders because the community is going to come and fine-tune a lot of stuff that you don't have to anymore right just give us a good instruct give us a good base and then the people going to come out for the languages that they're underrepresented for task that you probably didn't want to do coding like you know role play or other stuff we will do it for you just please do do give us the base and one thing people really also focused on which it was really I thought I'm going to fight with them but they were all on the same page that please do not try to be top one or top 10 at all like just give us a very useful model doesn't really matter where you are in the leaderboard because then when you if you go to that site it's never going to be good you're going to be th out thrown in two days or even in one day and they're going to be discouraged just give us something that is very useful and the community would appreciate it the community would use it they will prune own it, they will distill it, they will do a lot of things with that model. But if you just try to gain the system and to be number one and think that this thing is going to be just in the headlines for a whole week, then we will never get that model because it's just not possible with the speed that we are going forward.

31:46 Whatever you do today, tomorrow is just going to be a better model coming out from other labs, right? So that was things that they agreed on a lot of it. They came out with a lot of questions that was like they they showed they came prepared which was very nice that they took the time they prepared questionairenaires they were moderating the conversations the the groups were there very diverse from people working in UK government or French government and having a startups that was very nice also so they said soon and you know open AI when they say earlier stage it could be in two weeks you never know with open AI Usually soon means like 10 minutes. I don't know how soon. But the fact they said they would like to be committed to a regular release of open model. Oh, really? That was very something very nice. I like that. It's not like a one-time thing that here is a model goodbye. I think it's like here is the model that we're going to keep on releasing and making sure it's useful all the time. Something we always ask meta, right? Please don't release model once a year because I mean after six months it's very outdated right I mean just do it like every three months you don't have to be number one again just be useful all the time that would be enough for us thank you so much for this great recap I'm I'm very happy that they invited you invited Prince invited like a bunch of other folks I think it just shows that open is like really planning to to listen to the community this is great folks this is like absolutely incredible so shout out to open for for this coming prepared having Sam there Sam and I think a few folks already collaborating with hugging face openi is doing doing it right and hopefully we'll see a great model coming out from them maybe a hybrid one reasoning etc folks anything else any any of you want to ask a specific question to Mazi before we move on here did you mention the guards hi hi by the way great to see you mazi on the show and hi quitla again and everybody Yeah, wolf from go ahead. Yeah, I I've I had my sources as well. So, they talked about the guard everywhere. By the way, just let me just interject.

33:58 Thursday folks are everywhere from multiple angles. Open the eye, we know everything. Go ahead. Yeah. So, about the guard rates. Did you mention that just now? I didn't. So, yeah, there are some people who were asking for it. the safety was big in one of the rooms and I guess like the I mean the projects they have in the government like one one person was like we want to put the entire you know UK government and AI. So I think safety for them is very important but at the same time we were like could you not like really do alignment on the model because then the quality would drop. You could all you could just give us a very smaller classifier for red teaming for toxicity and for other stuff because you you guys very good in this kind of stuff right we don't also need to waste asking the bigger model what do you think should be answer or not so I guess what we see the trend from llama llama guard and then gemma now has gemma guard for visual event h it's better to separate them not be because you know that too much post training and alignment it's just really converting their model.

35:04 That's why every time someone comes and uncensored the model, you just get a better result because it's not really limited to what it could say. I think that was also things that they agreed on having a separate model for safety. I mean they will do their safety as much as they can but not too much I guess like some of the gemma instruct if you realize you have this prefix answer postfix so the safety is embedded into the models. so much you have to have a system prompt to say it like I get it I'm already a doctor and stop keep saying you have to you know check with the doctor I mean stop being so concerned about that you're an AI like we get it and we know we're talking to AI it's 2025 is I mean you're not fooling anybody know you make mistakes so I think that was the whole sum of it that guard rails would be nice to come with another model I gota wonder if how big this release will make waves in the community because OpenI actually being open from models. It's been a long time since that was the case since GPT2 I believe is the last model that they released Wait for and then Whisper which is not NLM and then the new you know transcription stuff they didn't release in open source. So Whisper is kind of like still the biggest interesting one. well that's very interesting. We're looking forward to it. Obviously folks from open air who are listening we are going to cover this. If you want to send folks from the open source team in OpenAI to chat with the open source with the best show that covers open source AI in the world, which I think it's us, you are more than welcome to send it. and if not, we'll definitely cover this as well.

36:42 Mazi, thank you so much for coming up. once for me, lovely being here. Thank you. Yeah, it's great having you here as well. Feel free to stick around with us. We have a big group here and I love it. because the next thing we're going to talk about is well, actually, let's talk about the quantization aware stuff. W from you've seen this as well the folks from Gemma. Shout out to our friend Omar Sun Civero who was who's now doing God's work in open source in in Google. they announced releasing Gemma QAT. QAT stands for quantization aware training and they reduced the VRM requirements from 54 GB for this model. Gemma 3 Gemma 3 weights 27 billion parameter from 54 gigabytes VRM to 14. This is a significant reduction. Let me actually show this on screen here because like you need to appreciate the graph to see what this does and quantization aware training who who can tell us what that actually means because it's not quantization post post training, right?

37:50 It's like it's it's it's happening while they like make the model. who wants other give a brief rundown. Yeah, there there's different forms and types of this like in research over the years, but I think the main thing that they're probably doing here is something along the lines of as you're training, you're actually at times like doing selective quantization of which parts of the model are optimal to quantize at certain points in time and doing that progressively as you train the model more and more. And so that allows the model to typically end up in a quantized state that's much higher quality than if you were to just apply some general quantization after the model was trained that's not specific to that particular model.

38:31 I see. And so we we see some Thank you LJ. we see like this kind of like improvement in requirements all across the models. even the 1 billion parameter model goes for like from 2 GB VM to like half a gig VM the 4.0b be w from you had a chance to test this. I saw a quick comment from you that's like very very interesting. what happened over there with this great I'm I'm running it in LM Studio and the good news is it only takes about five to six GB of RAM for just loading the model and using it. So you can run it on your on your at home computers easily.

39:13 That's the great thing about it and the quality is very high. But what I was doing I have a pretty complex system prompt especially about language because I talked to the model in German in English. The prompt is completely in English and so I was saying hello in German and the model started talking in English. So I said okay there's a rule in the prompt that that's how you have to respond and if I start in German you have to speak German as well. And so I I always do that. I did it with Claude. If something is not right I say okay there's a bug in your programming.

39:41 We have to fix it. We have to look at your prompt and fix it. And when I did that with Gemma, it gave me that response. Oh, you really are attentive. I must say I'm impressed. Yes. Yes. A prompt. You're right. It does say something else. But let's be honest, who reads a fine print? And that was with a wink emoji. With a winking emoji, this this quantized gem just full of SAS for you. Yeah, of course. That is with my amy prompt as always. So that is a system prompt that is there. But there's a complex rule about language. So we are taking turns and because I often use it for translation so it should translate but still speak my language. So that is partly Amy but it has to follow the prompt. That is the whole thing and GMA 2 did not have system prompt support.

40:25 They didn't support it. If you tried to send one you got an error message that it is not supported which I criticized and I'm really happy that they did it with gap gemma 3. There is system prompt support otherwise that wouldn't work as well. But something is not as good as other models regarding prompt adherence apparently. And the model even acknowledges it in a sassy way. Yeah, I am that I am annoyed. I am so annoyed by by this this blog post because they could have posted exactly what the benchmarks were in quantized versus quantization aware.

41:05 quantize versus regular quants and just shown an improvement. They don't show any of that. it looks like the method that they used they kind of have the language here is very chat GBT like but I think they just kept the they just kept the last layers just kept the output at at float 16 so then they they could get the quoteunquote say that they could get the log props from the non-quantized checkpoint. So, it looks like they just continue the training for 5,000 steps. I'm wondering if that quantization aware training was mostly in English that that that's why it it performed that way. but yeah, overall I am both excited and annoyed by this.

41:57 So, they could listen your your perpetual state I would say. Yeah. Yeah, pretty much. Well, well, from did you test function calling? Because one of the things that I I feel like Jimmo lags behind a little bit in like what they focused on in the training is function calling and I'm curious about the QAT function calling. I didn't get to that yet. So, it has been just testing a bit of the chatting and there's also the thing you have to use a proper temperature and other values. There's onslaught there's a nice tutorial on how to use it which I'm sad to say that should be in the model card not on some other resources where you have to look for it. So and of course it's quantized and it's running in LM Studio. So I don't know yet where the problem is. Is it the model? Is it the quantization level? Is it the settings? Is it the inference engine that has some bugs? We have seen that with newer models in other like llama and other setups. So I don't know why I'm seeing what I'm seeing and I'm maybe too used to the big online models like GBD3 03 40 and 03 and so on. So maybe I'm expecting too much at the moment but I definitely have to look into this.

43:06 It's not doing as well as I had hoped. I had problems with it. but I was using the MLX version and I'm going to try again this weekend because I'd really like to get function calling working because it's a it's a good model. Yeah. Oh, I I recommend MLX started adding 4bit AWQ quant support and by default AWQ was only a threebit. So that was the issue that they hadn't shown before. But now they show with AWQ it keeps perplexity at like 99%. Which might even be better to just than than doing it through quantization aware training. I'm not sure. yeah, I I would use I would probably use that qu that might just be the best performance to capability ratio me right now for running the the models locally. I really want to benchmark the different settings and quants and if it's MLX or if it's GGF. I really want to do that but my my AI workstation has been fried and I'm only on the Mac and it's pretty slow. I mean, 13 tokens per second is okay for normal use, but not for benchmarking.

44:21 What happened to your your station? Oh, I I I don't know yet. It just doesn't turn on anymore, but I already checked the motherboard connections to the power button. I have to investigate deep more deeply. Well, hopefully you'll fix this soon. Maybe you did over too many emails. moving on, moving on to Nvidia releasing Describe Anything. It's a three billion parameter model. They call it DAM 3B, a multimodel LLM for region based image and video captioning. I haven't seen the video capturing here, but I've seen the the demo for the image capturing. It is pretty pretty cool. so reminder Meta released like a long time ago.

44:57 Segment anything. Segment anything allows the model. It was a very tiny model. There's like allows you to select something and it will segment completely se segment anything too. this model kind of does this but also describes what's happening here. So if I if I kind of zoom in here, oh actually it's really funny zooming in zooms out the image. if you kind of zoom in here you can see that for folks who are watching for listening I'm under describe anything model demo and I just preloaded one of the images there and as I kind of move my mouse my mouth kind of cursor the segments underneath are kind of highlighted. But when I the this is kind of what segment anything does. But when I click on it, I click on one, we can see a description getting generated getting generated in a second. Usually takes a little bit faster than this. And so I clicked on this bear and says a brown bear with a thick dense coat of fur featuring shades of gray, brown, and lighter brown. the bear has prominent snout. So basically, and you can then describe a new region and then describe everything. It is literally describe everything. but I think the the the interesting part about this is that for folks who are watching saying well vision models can do this any model right now is a vision model like we can describe it. I think that the point here that you can like specifically ch choose one region and then have a description of that. I think that that that's a cool point here as well.

46:27 let's see. How are you trying this? Like where do you get data for something like this? Seriously, like that's a very good question. I don't believe that Nvidia meta. I think that's all of Meta's past work. Nvidia listen. Oh yeah. Yeah. But I mean they they were the ones pushing this for for a while. There was efficient SAM 2. They've pushed a lot of data sets. It's like 35 terabytes or something from meta for this type of data. That's that's fully out. There's a lot of data on this. It looks like the data set is also released. Describe anything data set. Oh, okay. All right.

47:08 So, we have we have subsets here. Koko and Paco and Sam. Yeah, Sam is definitely Yeah. Yeah. There's efficient efficient. Yeah. Okay. Yeah. and let's data studio. We take a look. So, first of all, shout out to Nvidia for releasing something like cool like this, but also with data. the the project page is full of Nvidia and UC Berkeley as well. So, shout out and UCSF. So, shout out to all these folks for training this thing. I think the coolest thing about this is that it's video as well. The demo doesn't work on video. But, as we can see, this kind of the the segmentation kind of keeps happening over video as well. And so for folks who are just listening, we're now watching kind of a gift of a fish in tank. All these fishes look kind of the same. And the caption when one fish is selected is a vibrant goldfish with striking pattern of orange and black spots. The tail fin fans out blah blah blah.

48:01 I think this is pretty cool. Like we we don't have the our our friends from vision department from RobFlow, but I'm pretty sure that they're cooking like a tutorial for this in like supervision. this is dope. I think. Oh, do you guys remember when we had Porroskowski here from Roboflow to talk to us about SAM and then there was another model that did just the description. He literally built kind of the combination of them to get to this model. He built SAM plus I don't remember what it was, but it was a a model that oh something from Microsoft. Come on.

48:34 Florence, maybe. Yeah, I think Yeah. Yeah, Florence was it. I think it was Florence. Right. So, and then Florence does kind of the description, the captioning, etc. So he combined this to segment anything you click and then you get the the the caption for Florence. So this is like one unified model in what three billion parameters that does both. it's super cool from Nvidia. Detailed localized description is what they call this. and yeah it's pretty cool. I think it's like very useful for multiple applications. specifically imagine kind of like honestly now with the GPT image that we're going to talk about you can imagine all kinds of interfaces where you want to remove some part GP image one allows for masking for example you can super quickly build with this model the segmentation part mask out the person that you want and replace it with like a dog so even for that it's like super useful three billion parameters probably will run almost real time on any like production GPU comments about this one folks what do you guys think do we is there anything else we should mention about this model?

49:36 anything more interesting that we want to just the tracking just the tracking over video is hard seriously like just tracking objects over video is a hard problem and and it's like and here it's like sideline yeah yeah we do that as well but we also describe like this is crazy seriously like I haven't looked into the architecture but just by looking at this the level of understanding it's probably a really good base for tons of of of use cases you can further train this on like just the level of understanding throughout video and like and and like embedding it in language it it's probably a really good model I mean it makes sense everything Nvidia produces is is really good but but yeah that's impressive seriously yeah I'm I'm with you this is like super super cool and I can see a lot of useful things that coming from from this model. They also compare this to like other DLC bench DLC's detailed localized captioning.

50:42 So basically this task so they also compare it to like other pipelines as well. just the fact that this is one model and not split into like segment anything in some something else is pretty cool. yeah we also have another model. You're right Wolf from let's cover this like super quick as well. Nvidia is on a tear. I think people are starting to realize just how far ahead Porter at Roboflow is and has been the entire time. Yep. He's literally a year ahead everyone else.

51:12 Yeah, he gonna build this but but not in one model. They haven't trained the model for this. They like pipelines. let's talk about the the other Nvidia release super quick. While we're in the open source, Nvidia quietly also dropped Open Math Neimatron. Open Math Neimatron 32 billion parameters N14. they won the first prize in AI math olympiad 2. So you guys know the math olympiad for people. there's one for AIS as well. It's called A IMO and Nemoron that was trained on open math was basically winning the first prize and they just released the the kind of the the the data set for this and the model. And we can take a look at the kind of the the the results here compared to is that deep let's see what did you ask?

52:01 Yeah. Where where are the you know the big guns like oh yeah DC4 it's on the left 32. Mhm. I mean, so for folks who are listening, we're seeing a bunch of kind of like comparisons between a few models and then these I think they're comparing only to open source models because we only have QWQ here and the Deep Seek Steel 32B. Wait, wait. 1.5B. Look how much that got. Okay, that gets impressive. Yeah. So, Open Math Neatron 1.5B gets 65.6 six % accuracy on this AIM mo or I guess this is AIM me. So this is like math like like the human math olympian.

52:49 that's pretty cool. The coolest thing about this is it's really close to like the 14 like what's going on. Yeah, that's kind of the 1.5B being higher than deep sea car one. that that needs to be that needs to be double checked because that is completely insane. Like that is completely just groundbreaking and stuff. But this is just AI intelligence. It's not Yeah, that needs to It looks too good at Yeah, that looks a bit Yeah, we got to mute you because there's a lot of feedback coming from you when somebody else talks. or maybe headphones. The I think the thing that you guys are referring to as looks too good is that the fact that open math neatron 1.5 billion parameters get 65% and it beats both the distilled deepseek at 32 billion parameters and QVQ at 32 billion parameters right like this is this is the this is the the the very surprising result here that like this 1.5B solves math at a level of 32 billion parameter cubicq which is a thinking model reasoning model neatron is also a reasoning model but I'm not sure if the small one is the the thing that to highlight here as well that these folks released the Kagle winning solution and also the data set it's a 5.5 million math reasoning data set that they released they call it open math data set and shout out to Nvidia for all folks how about that like shout out to Nvidia for all this open source work what the heck this is great let me wait we have each and every single time the data also absolutely So we'll definitely celebrate this with our applause because the model models come and go but the data that say they they stay and you can use them reuse them. Yeah. And they said the data set was the foundation of our winning submission to the AIO2 Kaggle competition. so if we go to Kaggle and we see like a AI math olympiad progress prize 2 I think it was like $50,000 and Nemo Neotron was the winning one with the data set release.

54:59 Shout out to the folks. that that's great. moving on in open source, folks. We're moving on in open source because we need to also talk about audio stuff and we have our audio expert here to to cover the audio stuff as well. two releases this this week in the audio area. let's talk about DIA first because I think it absolutely blew up. I think that it will help if we actually play an example of DIA, I think, while while we pick it up. So, Quinn, if you would like to open up with the description what it is, and then I'll try to pull up the the actual audio as well. So, it's an open weights model, done by two students or maybe a student and a half and it has like crazy dynamic range. It can just kind of take a script and be super expressive.

55:46 And I think that that's why it blew up. That's my my sense of what people really keyed in on. and we should play some examples and then then I definitely have some thoughts maybe. Yeah, these examples I've played them to to Okay, so here's my small viral tweet about this blew off to like I love the fact there's more bookmarks than likes. There's like 5,000 and a half bookmarks and like half a million views on this.

56:12 I actually play the actual demo page. I don't want to play the the thing that I recorded. There's a demo page. I think it's very important for a model like this to have a demo page. but basically if you guys are watching you can see there's like speaker one and speaker two and this is how you pass this to the model like speaker one speaker two and then this is an example of 11 labs. Okay, this is 11 labs. Do you guys hear this or no?

56:40 Nothing. Lovely. I selected share screen wol from maybe we'll need we'll need you to to help with this but let's try again also tab tab audio let's try again dia is an open weights text to dialogue model you get full control over scripts and voices wow amazing try it now on GitHub or hugging face this was dia or dia or dia I don't know but I the thing that I love the most about this is that you can add emotions in parenthesis and like laughs qu and like you can hear the person starting to laugh in the previous word. So this model is like kind of smart about like the whole segment. Not just like it doesn't get to the left and starts like haha laughing.

57:22 If you play 11 Labs, let's play 11 Labs again. Dear is an openw weights text to dialogue model. You get full control over scripts and voices. Wow, amazing. Try it now on GitHub or hugging face. So this was 11 Labs and you can literally hear the thing that I said before. it gets to the laugh thing and then it goes haha and it's not like built in for some reason. I think that this is what caught fire Quinn the most is like again when when you add these emotions it like it really comes through. But you had some other experiences with this. Let's listen to another. Did you guys see the one about the fire where there's a fire going on and people are yelling? Can we play that? That is insane. Let's play. Okay.

58:08 So, so, if I'm not mistaken, and hopefully I'm not mistaken, this is from the office. This is that like famous scene when like everybody's getting freaked out. And so, here's 11 Labs reading out this fire em very emotional script. Oh, fire. Oh my goodness. What's the procedure? What do we do, people? The smoke could be coming through an air duct. Oh my god. Okay, it's happening. Everybody stay calm. What's the procedure? Everybody stay calm.

58:38 Everybody. Okay. So, it reads it as though I really like I really like that. you know, you hear that and you think, well, this is good. And then and then you're going to play that and then you play the open-source 1.6 billion parameter model that a Korean and a half that runs on your phone. The runs on your phone. The Korean and a half students trained in like a basement somewhere. Oh, fire. Oh my goodness. What's the procedure? What do we do people? The smoke could be coming through an air duct. Oh my god. Okay, it's happening. Everybody stay calm.

59:06 What's the proced? Everybody stay calm. Everybody calm down. No. No. If you touch the handle, if it's hot, there might be a fire down the hallway. Yeah. This is a How did they do this? Really? How did they do this? Like sick. This is incredible. Seriously, like what what's the data set? Well, how how do you how much did you train this on? Like what? It's it's like unbelievable. Yeah, it's unbelievable because we're here on the podcast and we kind of talk over each other. We understand what Yama's about to finish the sentence. I come in and then Quinn, we're going to talk about all of this in semantic fat as well, right? but like we all have a we all have like an understanding of like what's going on. When you hear the 11 Labs, 11 Lab voices sound incredible.

59:56 They have been working on this for a while. 11 Lab voices are great. However, the thing that we loved especially from Notebook LM is that they like the two speakers kind of have this interaction between them. This is again a 1.6 billion parameter model. This is nothing. I no idea how how it gets to this level of emotional and also kind of like interruption and the voices kind of speak on top of each other. and like the sesame also is there. so here's a last maybe example of kind of the non-verbal stuff that it can do. So there's like coughs and sniffs here. Hey there. Why did you just cough? Why did you just sniff? Why did you just clear your throat? Why did you just laugh? Nicely done. So, hey there. The last part is what threw me completely away. The when it says why did you clear throat and then says laughs in parenthesis, it starts to laugh at the word throat. It's like why do you clear throat like like this whereas 11 laps got to the point and then started laughing. and then the the other guy just interrupted him.

60:58 Why did you just laugh? Just interrupted in the middle. Like the the first sentence did not finish. It's kind of like the the the model generates two voices. Quinn, what do we make of this? Did they say anything anything? Did didn't they really say brag about what they did? Like like come on. Nothing. The bragging that Toby Toby did. Toby is one of the folks who trained this or I think the main guy who trained this on Discord. The drag, the dragon was we got to number one hug and face and we got to like like so many stars on GitHub and like it went super viral not only for my account like also on for good reason. For good reason.

61:30 Yeah. Yeah. Quinn, what do what do we think about this? So, it's awesome. I love this work. It's great. But let's I'm going to throw a little cold run on it just for the sake of like, you know, talking about where we are with speech. The 11 Labs folks could totally do this if this was the goal. It's it's partly about what you're aiming for. And if you want to train a model that people are going to pay you money for, it needs to have some level of predictability. And the DIA model is unhinged. Like to get those examples that they put on the demo page, I I played with the prompts a bunch. I am 100% sure they ran those examples a hundred times to get the examples they put on the demo page, which is fine. Like it's totally great if what you want is a model that shows what you can do. There's also like the longer you're a programmer, the more you realize that every single thing in the world is duct taped together. Yep. So there's like string replacement for the speaker and the special like nonspech things. I mean it's, you know, it's a research model, which is again awesome, but it's just a totally different thing than what people who are putting out a model that they want people to sort of pay for are going for. like I I spent 20 minutes doing trying to do a dialogue with DIA and trying to do a dialogue with the GPD40 mini TTS model and I got much better results from GPD40 mini TTS when I had something I was aiming for. Now, if you just throw some text at it and are like, " I want to see what it does." I haven't seen anything that's more fun than dia, which is amazing. but like predictability, steerability, all these other things, they they really do matter, too. The data set's probably all YouTube. Like, everybody has just scraped a ton of YouTube for this stuff, which is why there aren't very many open data sets for these models. We're going to come back and talk about open data for audio in a minute. I also think that if you really focus on those non-spech things, you do get a huge amount of mileage. And maybe that's one of the things we should all take away from this is like if you're working on audio models, like now we can set the bar at like speech isn't enough. I want to see sound effects. And some of the new Open AI models have sound effects stuff.

63:49 Although I don't know how much they've nerfed that for the API because like it makes it harder to use the model for speech. There's stuff coming out at Google that's going to be amazing. Notebook LM is obviously amazing. like 2025 is kind of the year of audio in some ways. Absolutely. I I I think that like I I completely agree with you. I the one thing that we we need to highlight obviously no matter how like a student and a half from Korea, no matter how correct they are, no matter which data set, they cannot compete with like a company that this is what they productions and this is like the product they sell. what they can do though is get us all excited about the possibilities. something that you referred to in the beginning, right?

64:29 Like getting to know to the level of this is possible on your phone in near real time is incredible. The fact that like you can clone your voice here, which I've tried unsuccessfully unfortunately because some of the MPS stuff wasn't quite there, but the the open source community definitely like got up to speed. You can clone your voice and like our voices as well and like this will continue in that. so it's not even like oneshot cloning. kind of like imitation in in the in the inference which is also super cool.

64:57 That's something they they've built in. So it is unhinged and unexpected for sure. and I got to wonder like what you know what this data set with a little bit of bigger model a little bit stable could do. You know we don't know models are small like audio models are small and if you're a student or a hacker and you have a 4090 you can train audio models. That's one of the really cool things. So like Kakoro is 82 million parameters. All right. Yeah. and and it's good like cakora is really good. Yeah, this is 1.6 six billion still very small relative to like you can turn it on your own hardware, you can turn it on modal, whatever. So yeah, if everybody should be playing with these models if you're interested in in deep learning. Yep. So we already have some comments coming in saying I built this model into a dinner conversation app. this is Joseph Urban. Hey Joseph. It says it listens to the conversation, periodically adds to the conversation, makes jokes, and makes fun of people at the table. I love this. I absolutely love this. If if if you guys haven't had the experience of sitting in some social situation with I don't know chaptt advanced voice mode on or go whatever give it a try and see how you do because sometimes you just sit there and at some point you forget that these models are like listening to you open like and you stop talking and they semantically know that you stop talking just like inject with something. which brings us to the next point which we want to talk about.

66:17 It reminds me of the chef project you did. Yeah. reminds me of the chefs watching and giving funny comments. Well, chefs specifically would roast the crap out of me. The the chef puppet that Wolfen was referring to is the hackathon submission that we did for daily hackathon, by the way. which was great and that was just completely roasting you with like you know with Chef Ramse's voice just absolutely unhinged everything. but yeah, I want to move on to to the other open source coin. I would love for you to tell us what you guys are released and kind of we've chatted about this me and you at the engineer obviously and then we briefly touched on this when openi released like semantic v in their advanced voice mode which was great and now I think they also released in API and in the you like in the app itself now the app is like significantly better better for kids could you maybe start with what is the big news and then we'll talk about like semantic and some of the things in data yeah The breaking news. I wish I wish you'd play the breaking news. I would. I love breaking news.

67:21 Let's go. Breaking news. let's go. AI. Breaking news coming at you only on Thursday. I We should get Dia to do this. Yeah, exactly. Thank you. That was awesome. So, we released a checkpoint of this open source, open data, open training code turn detection model we've been working on as a community. The goal is to have a state-of-the-art model that can tell you when a person is done talking and an LLM should start talking because that's one of the kind of 2025 unsolved problems broadly speaking in building really good real-time voice agents. And it's surprisingly hard like we don't even always get it right as people. So, you know, you're on a phone call, there's a little bit of latency in the phone call, you tend to interrupt each other. but, you know, we've got transformers and audio conformer models.

68:16 and so we've got really good tools for trying to figure out when people are done talking and when an AI should respond. And there there has been no there there's been very little open source work on this. so over Christmas when I didn't have any meetings, I did a bunch of training runs and cleaned a bunch of data and then it snowballed a little bit and we're inviting everybody to come hang out and train small audio models with us. This one with audio on the input side and just a classifier percentage on the output side saying complete or incomplete phrase. and the breaking news is that this is now on foul. Our friends at foul.ai are hosting this model. So you can use it from any kind of voice agent or multimodal agent codebase and just hit f val ask for complete or incomplete you know log probs and get those back and and decide whether to trigger inference or not.

69:08 Open AAI's semantic VAD which is what they they call this in their APIs is in the real-time API now. I think it's really good. the the guys how does it work Quinn? I I I I'd love to like tell our listeners what semantic that is or like what what smart turn is as well. basically one of the reasons people don't like talking to AI is this like specific thing it interrupts and while you're thinking especially for kids we've talked about this with kids a lot as well you know kids tend to think maybe longer for the next word maybe they're trying to find multilingual folks such as myself we also like struggle with this as well often times the word the correct word in the wrong language comes up and you're like no brain I'm speaking English right now I need a different word right now and while this like my mini conversation with brain happens. Chad GPT used to just like well Alex here's an incomplete answer because you did an incomplete question. so previously VA voice activated voice activation detection voice activity detection you will correct me when I stop speaking I'm sure just did this based on sound levels right based on just like quietness and some better than sound levels. So traditionally you use a a very small specialized ML model called a VAD model voice activity detection model and the model that model's only job is to take an audio frame and say this is speech or this is not speech and then for turn detection what you've typically done is you've taken a certain number of frames of non-spech in a row and been like okay the person is done because they paused this long. and the reason you use a bad model instead of just audio level is because you want to filter out background noise. you want to be very you know kind of you want to work in a whole lot of different contexts. So you use this kind of special purpose classifier. the idea of semantic VAD or smart VAD or smart turn or whatever you want to call it is to go really significantly beyond that and try to incorporate the audio patterns in the speech and the linguistic data that you have access to.

71:11 So I just said so and I drew out so and then I paused. I probably wasn't done talking. You know that. You know it both because you can hear like what I'm doing tonally and because you kind of understand the linguistic content. So, what we're trying to do is train a model that can do that. Now, I love the question about how does it work? Because I had like a bunch of people at an event last night where I showed a demo be like, "Well, how does it work?" And I was like, "That's the wrong question.

71:36 It's a deep learning model. We're not supposed to know how it works. All we can know is that we clean a bunch of data. We train the model. We look at where the model fails. We look at the data set, we iterate. It's an audio conformer model. So it's got like multi-headed attention block and a feed forward network and a 1D convolutional network and a feed forward network again and then a bunch of those. And if we run enough data through it, it works. That's how that's how deep learning works.

72:03 where's the data from? So this is really important to me. All the data is posted on hugging face. we we have a bunch of recordings that we started doing ourselves who are working on this model. we are also trying to get to a synthetically generated data pipeline that works really well because theory is if we scale up with synthetically generated data that would be great. So we're going to try DIA as data generator for this. Interesting. and then at the moment anyway we really do need human data. So we put up a bunch of games at turntraining.pipcat.ai AI, I think.

72:41 and you can go and you can play those games and on the way in you click, I'm cool with you guys recording my voice for open data sets and then we'll keep the recordings. We'll clean them up. We'll post them to hugging face. And you know, if your voice is in those data sets, every AI agent you talk to is going to work better for you than for your friends. So, that's great. Play the games. That's a great incentive, folks.

73:06 So, if you go play the Pipecat the the the daily games and train your voice, it will work better for you because you'll be represented well in the data set. That's actually pretty cool. can we do a demo? Let's try to do a demo. I think there's a demo. You sent me a link, right? Yeah, I sent you link. Yeah, let me go get the demo link super quick. I want to try this out and specifically you should tell me which which type of stuff I should try.

73:34 But like the so part I think is very important. Yeah. Stuff like that. Stuff like stuff that feels like a natural pause. And then if you just zoom in on the web browser, you'll see the logging output in the bottom. That'll be like complete or incomplete. Yeah. All right. So, let me let me actually I'll zoom out a little bit. Let me try this. So, we're going to allow microphone. Hopefully, you guys will hear this as I'm sharing the screen.

73:57 Let's see if we'll start. Hello. Oh, am I talking to an AI right now? Not yet. It's not It's not connecting. Didn't in initialize the dragon. The law of live demos. Yeah, which we always have, especially with with audio. Quinn, we should talk about maybe helping you fix this. streaming and sharing audio together. This often like breaks for us and sometimes Yeah. All right. So, we'll send out this link for folks to try themselves. but I play with this before. It's pretty cool.

74:36 I think the the the benefits for people who got used to these models, we talked about this before. We on our side, we got used to even for PT cuz we got trained into saying stuff while we think just so that it doesn't interrupt or going like like going like this. But definitely 100% when I give these like models to my kids before like semantic VAD they were absolutely not able to inter interact with these and like this really really helps. So I think this is like a very important effort on the open source. I wanted to highlight this and let you kind of like talk about this on stage here. Anything else that I missed that you want to shout? Oh yeah, of course.

75:20 there's also a course coming up that I want you to to tell about folks because we're also kind of involved. but anything else on the smart smart turn part that you want to tell folks? No, just coming out with us work on small audio models. add add data sets for other languages like I think it's a really fun project. Yep. And obviously for folks who come out and contribute voice to your data sets, other languages should be represented as well. So we we we call on the community to go and play with them. We'll add all those links in the show notes for folks who are listening. I think that this is it. Quinn, I wanted to give you the stage to talk about the the great work you're doing in the community about like educating as well. you gave me a book that I still have that you have, but also there's now a course that you and our friend of the pod patron of the pod if you will, Swixs, run together.

76:12 Would you tell us about the course super quick? Yeah, we wrote a really long guide to voice AI kind of a fundamentals with a bunch of open source code samples and that got a lot of really positive feedback. So, we thought we'd basically do the guide as a course and a community and invite everybody to come hang out experiment with voice AI in general. If you're building commercial voice AI stuff, which more and more and more people are show everybody what you're doing, learn from lots of experts like Weights and Biases folks about all the best tools that people are using for voice AI. Now, I wanted to model it after the amazing LLM course that Haml Hussein and Dan Becker did last year that turned into this like great community kind of I think Swix called it the Woodstock of AI last year. So, it's on Maven. I know the weights and biases folks are going to participate. a bunch of other people building voice models and voice tools are are going to participate as company sponsors. so check it out.

77:08 It's on the Maven homepage. Amazing. Quinn, thank you so much for coming up. I think the work for open source voice stuff that you guys are doing whether it's pipecat whether it's smart turn whether it's the the education part is really really amazing. I really appreciate you being here as a voice expert as well telling us like about the latest and greatest. with that I think folks I think we've done our chapter of open source and I think we will move on. Quinn thank you so much for joining. I will just shout out that we have a new friend here joining us on the stage. We're now a lot of people. Adex, welcome. Dex from human layer, Dexter Hothy. What's up, folks?

77:46 How y'all doing? Great. Coming through loud and clear. I think we will now move to a conversation with Dex about kind of the the the 12 parts of agents because I think those are interconnected because we just talked about voice stuff for Quinn and kind of smart detection and I think like a lot of people are using voice to interact with like agents and so I think that now it's a little bit of a time to talk about agents and then we're going to do open open big companies stuff which will include a GBT image generation.

78:19 Dexi, it's been a while since you've been on the show. I believe a few months. maybe do a quick reminder for folks of who you are and and what you do in life and then we'll talk about some of the aspects of building robust agent that you've outlined. Yeah, absolutely. super dope to be here. we've been hard hard at work. so I'm the founder of a company called Human Layer. We build SDKs that help folks build AI agents that feel more like digital humans. So, taking them out of these chat GPT style interfaces and deploying them into Slack or email or wherever the potential users of those agents already are. and what that led us down was a long long path of trying to figure out what makes a good agent and of all the agents that are actually in production and make it into customer hands what what what is actually working in production.

79:14 Yep. And so I guess tell us what is actually working. I think look I I I I wanted to like invite you as well because and maybe we don't want to touch this maybe we want to sidestep but there is some drama in the agent world and how like these like frameworks are happening together openi release something and then the there's like a response from from Harrison from lang and lang chain and then there's a responses to the response there's like there's a lot of stuff happening in the kind of the agent world there's a lot of frameworks obviously for agents between open the eyes recent there is Google has Google agent SDK there's a lot and I'm assuming with a lot there's not a lot of like standardization around like how agents should happen right the standardization that we do know that's happening it's great is around like the tool calling for example with MCP there's now new stuff that we chatted last week with with Todd u Todd Segel from Google about A2A protocols or how agents will like kind of talk to each other would love by the way to hear your thoughts about A2A while you're here as well but there's not a lot of centralization about like the actual kind of frameworks. There's like tons of frameworks and then OpenAI did like a table and then like now Harrison came out with his own table like what which which of the frameworks does what imperative stuff. human in the loop, human on the loop. Dude, I didn't even know the human on the loop is a thing. Would love for you to explain to me what human on the loop is.

80:38 but also would love to just like get to get to hear from you kind of a few of the pillars of after like many people tried them, many people that you worked with like build agents. What is the the important things? What are like some of the interesting stuff? Yeah. And I will preface this like this is it's very easy to read 12 factor agents and to read it as a like all the frameworks are bad and you should never use them. That is absolutely not what I'm here to say. And if anything, it's really more like, hey, folks who are building frameworks, this is what people need from frameworks and this is why they can't use today's frameworks because you don't have this.

81:18 And so a lot of people I talked to a lot of founders doing a lot of hard things and doing impressive stuff with agents and almost every single one of them said like I just write the thing from scratch. And so this was kind of my goal of trying to figure out what are the things that actually make because they're they are still doing things that feel agentish. But the the big takeaway was like and I think we've all seen this hacking ourselves is like when you take this like here's your prompt and here's your bag of tools and like go figure it out.

81:48 The reliability on that for anything more than like five to 10 steps starts to drop off pretty quickly. Yeah. And this is just compounding exp. Even if you're 92% accurate, if you do roll the dice on that, you know, 10 times in a row, now you're down to what, like 40%. I don't I don't know how the math plays out, but the point is like the longer these things are working and spinning, the less high quality that the the the more it just becomes completely unreliable and like definitely can't go into customer like facing software. How much of this is model related by the way versus how much of this is just like the nature of you know, LM's getting lost in the loop?

82:23 I think I mean the idea is like a lot of people watch this and there's like we even go into this in like chapter 10 which is like what happens as the models get smarter because some there's a lot of different pieces of advice. yeah if you go to small focused agents and there's a gift down there. If you scroll down there's a gift. It's like as the models get smarter basically what will happen is you'll take these big deterministic workflows and what works today is putting these really small micro agents in to handle small parts of those workflows. As the models get smarter, you may be able to grow them as your prompts get better, as you get way better at tool prompting, and eventually maybe you have this large agent that can handle a hundred step or even thousandstep workflow. But we're not there today. And I think it was the notebook LM folks said this on the Google podcast which is like the only way to or sorry on the Swix podcast on latent space was like the only way to create really magical and like impressive experiences in AI is to find something that is right at the edge of the model's capability and get it right consistently. Mhm. And so whether the models keep getting smarter, there will always be kind of a core set of techniques that let you if you need to open the box and go into the hood and get as much performance as possible.

83:39 This is kind of a guide to some of the principles that will help you do that. I hear that. I have a few more questions, Dex, but I want you to run through kind of the the the 12 factors. Maybe we have them on screen, but like for folks who are listening, did you outline them by by level of importance? Did you do kind of like just like this is when you build this progressively you will add these layers like how how how are these like 12 factors created let's say totally I mean I think they go a little bit the order is somewhat random I think the first couple are kind of like the easiest to kind of gro on their own some of the later ones kind of build on top of previous ones so I can quickly kind of run through so you have number factor one is kind of like natural language to tool calls this is the magic of LLMs and if you really embrace the fact that an LLM calling a tool is just turning natural language into JSON that then we can do something structure structured and deterministic with whether it's like hey send this send an email to my assistant and ask them to do this well that turns into a call to a email API like resend or something with text in it and things like that or if it's yeah so that's number one number two is on your prompts I think any framework that's going to hide your prompt and not give you access to the core to tokens and templating is going to maybe like there's a lot of frameworks out there that you put in a couple sentences and they turn it into a banger prompt like you would have to go to prompt school for months to get this people are doing like this weird thing where like there are no assistant tokens. it's all user messages and you're asking the model to autocomplete the user message instead and all these crazy things you used to be able to do and like that's great but at this at the end of the day if you want maximum performance you're going to want to own every single token that goes into that LLM because they are totally state that's the literally the only thing that affects the quality of the output is the quality of your inputs y and that's kind of rolls right into owning your context window too right you can use this like user assistant tool these rolebased JSON messages but if you actually look at that Like if you look at the tokens, if you if you click into this, there's a there's a visualization there if you scroll to the bottom of like just JSON versus I know a lot of people are using like XML style stuff for their prompts.

85:54 scroll down a little bit more. we have this kind of for folks on the stream that on the who are listening, there's a picture here of basically like a comparison between like the same prompt context window in JSON is 87 tokens and if you do it in XML, it's 57 tokens. then I don't know which one of these is better. But the point is like if you are not exploring every single possible option, you are not going to find what's optimal. And so you shouldn't outsource or lock yourself into one way of thinking about how do you tell a model or an agent here's what's happened so far. What's the next step?

86:30 I I want to touch on this like super quick because a context windows as we here on Thursday I've been doing this for like a few years. wolf from we talked about like exciting about 4K to 8K multiple times and and now you know open GP 4.1 is 1 million tokens in the context window which is just like mind-blowing and I think this leads me to the next point Dex the the the industry moves so fast with just like three exponential curves around everything hardware moves fast like the the models the algorithmic they get released like even open source moves fairly fast I guess when you outline something like this for folks how hard it is to keep up with like the latest trends and expecting that like some of these things cuz the the thing that even with the smaller models that you outlined like let's have smaller agent do specific things. but then as like as time progresses like maybe the the the the model will be able to do like more more and more and more.

87:29 Somebody in the comments Sean called out the AI digest graph. Did you see that one with like the doubling? I I I should I should bring it up while I talk with like doubling the time that like the models can execute tasks and just like just like plotting out what they thought the curve would look like and then like adding the GPT 4.1 and just seeing an absolute like banger of like an exponential growth. So I guess with that said with the speed that we're moving with the fact that you know there's like every framework that you tried there's like a new one tomorrow.

87:59 how how are you looking at outlining suggestions for folks where these suggestions may you know may get a little bit different when context windows like 1 million token unexpectedly. So I mean I can say like more generally the goal here is less about like specific model techniques. The goal is more like how can you set yourself up such that if a new model technique comes out, a new prompting technique, a new thing like MCP comes out that you can quickly integrate it and you can push it to the limits of its performance. but for example for the for the context window one like context windows keep getting bigger but I think everyone says you know you can put two million tokens into Gemini but you'll get better results if you only put like a couple hundred thousand. Yeah. like it for no matter how good the model is and how many tokens it can handle. I mean again maybe in the next year we'll get a model that can have really good attention across the entire context window. But that's like cool we should bet on that and we should be ready for that and then we can our programs get simpler but until that happens you want to know how to get good performance in a world where small focus context windows are way better. There's also the cost thing.

89:11 Even though intelligence trends near to zero, people still pay for tokens. And as Mazi before said here before index, we had Mazanahi who joined the open-source kind of open AI kind of discussion. And he told us that the folks there did not want reasoning in thinking models in open source necessarily specifically because they would like to turn this off specifically potentially because like it takes more time and tokens cost as well. So yeah, there's there's other considerations just like performance on the on the context. So I agree with you. I just wanted to highlight this super quick.

89:43 shout out to Sean to reminded me. I wanted to show this. This is the graph that we've talked about. length of tasks AI can do is increasing. And this is from AI Digest and the data sources meta from 2025. And so you can kind of see the plots that they had before where they plotted out GPT2, GPT3, GPT3.5 and 2022, GPT4, 2023. and kind of they they kind of plotted out and then OpenAI came out with like the the the reasoning models and the reasoning models obviously correct themselves on the fly and kind of think about this and then you can see the plot like absolutely just go absolutely bonkers and exponential here with four mini and 03 with they're they're specifying that like the length of tasks that these models can do are now measured in like you know more than minutes and now like in hours and then they they kind of speculate going forward that like if if you ext extrapolate then they will go like for days as well. what are your thoughts about this? Yeah. Is is task in this case like a single prompt turn or is this like a multi-turn kind of agentic thing? Very very good question that I of the hand don't know the answer to because if the tasks are taking longer and it's like yeah because when I use 40 mini it thinks really fast and because you can get 40 mini to think and do chain of thought and all of this use for mini and it thinks for a bit and then you switch to 03 like yeah it is going to take like three to five times as long to do the same amount of work but so I I'm I'm guessing that this is like tasks in terms of like single turn prompt and it's like hey just go think about this thing for a really really long time and then give me a good answer. But I'm curious what which one is which one it is. We'll send this over as as shown as well because I I do want to move on a little bit. But like they discover this trend. This was discovered by researchers at matter. They took the most capable I'm reading off the page as as we speak. I don't know this off hand.

91:33 they look at the most capable agents from 2019 to 2025 and tested them from about 200 tasks mostly coding with some on general reasoning. Compare the agent success rate to the length of each task. and then I love this like plot going forward. Dex, I want to though we'll add this to the show notes. I want to get back to to 12 factors. I guess we talked about kind of the future when models going to improve. I did mention reasoning and and I felt like like a response from you. I would love to hear your kind of take on kind of the reasoning versus non-reasoning especially for agents stuff because well if you think about this just holistically some of the frameworking is built in order to make you know make consistent thinking properly execute and some of the frameworking is like you know adding memory as well and like doing execution reasoning after test time compute kind of takes over some of that to an extent if think like holistically like okay these models then now can correct themselves. So previously if we had to like build this into an agent now like the selfcorrectness comes kind of almost built in. would love to hear your thoughts about whether reasoning models are better for agents or not whether like different approach is needed. what do you see from like this whole thing? Yeah I have not had the chance to put this kind of thing through its paces. cuz I know there's like all these hacks you can do with like, you know, injecting a buttweight token and messing with the samplings and things like this to to get the reasoning stuff to be more performant, but I I can't I've used a couple tools that are using kind of like 03 and 04 mini in concert to do like like coding agent style stuff and things like this.

93:15 but I the only thing I found is when you dropped in like 01 or 03 mini for an agent is that all of the tool call outputs that had text fields in them become much longer and more verbose, which is, you know, what we know from these models. So, even even if you can get them to call tools or output JSON reliably, the things that go into that JSON are going to be more reasoning modely. So, I I haven't explored them too much. I'm still using like mostly the like older quad models for our agents. All right. I think that I think that's pretty much it on our on the anything that I missed like we we'll obviously send the 12 factor agents and it absolutely blew up. but anything that I missed that I didn't want to didn't ask you yet that you wanted to convey here? no. I mean, there's a bunch more stuff on there. we did a webinar on Tuesday where we did like a 90-minute deep dive with a bunch of live coding. I can send you a link to for that to send out as well if people want to see how to build one of these 12 factor agents from scratch. We we didn't get through the whole thing in 90 minutes, but there's yeah there's a deep dive on the repo there too, but that's that's very cool. I would love people to check it out and yeah, more to come. 100%. Dex, thank you so much for coming up. one last thing that I will ask you while while I've got you here, like we did talk about kind of the future as well. And we did briefly mention the the the drama a little bit in the engine world like frameworks world. one last thing that I did didn't want to like have your opinion on as well is that last week we chatted with Todd Segel, the guy who works on the A2A protocol at Google.

94:50 obviously MCP we've been covering like a bunch. would love to hear your thoughts about A2A specifically how you as a like as a agent builder and as somebody who provides specific services for agents to like humans interact with them. How do you see kind of the standardization attempt from Google? We'd love to hear like Ron filter thoughts about like this protocol and the the standardization effort from big companies. Yeah. So a couple things. One, I think we're in a protocol war. I think if you win if you win with a protocol then then you have control over how the market works and how it moves. I think an Ananthropic killed it with MCP.

95:26 People have tons of complaints about MCP and all the problems with it and it's like yeah it's probably still going to win because it picked up really really fast and it's like we've seen this in other tech. This is like Helm versus Customize all over again. for my Kubernetes people but I haven't gone super deep on A2A is the answer. My first instinct was like why do we need a new protocol? Like why can't an agent delegation just be a tool call via MCP or literally anything else? Because most of the I mean my thesis most of these agents are going to end up being APIs in one way or another like what's wrong with REST which is the same thing I said when MCP comes out and again walk on that one back a little bit. the other thing I'm really interested in is actually like what I will probably call like H2A like human to agent. I haven't written this down anywhere yet, but the idea of like there is room for a protocol where like I'm using any kind of client software, whether it's Slack or a task manager or whatever it is. And like how do we we f information back and forth from a human to an agent? And I'm wondering if agent to agent is going to be like that as well. there's all this weird like interruption and turn detection stuff when there are humans involved and I don't know if agents are going to need that stuff. So that's that's my rambling thoughts as requested. I hear you. Hear you. one thing that did resonate with what you said is that specific agents dedicated to one specific like task specific thing that you can like test through. that's kind of the bread and butter of what A2A is kind of built on that you're kind of generous agent that can understand you will spawn out these like tasks and like find the agents that are specific for this and then there will be like in the registry. So maybe maybe we are going to towards a generalist like model that can do everything and like all of the parts, but maybe if we're not, we're going towards like a system and in that system via this protocol, your agent would be able to just like grab other things.

97:13 Dex, thank you so much for coming. it's great to have agent experts here on stage. Would love to have you back with like some more news about like some recap of the agent industry. Absolutely. because this is where the world is going. Quinn before said it's going to be the year of voice. everybody wants to talk about the year of agents in 2025. This is the the highlight as well, folks. Dex from Human Layer. You guys should definitely check out the 12 factors. We'll add this to the show notes as well. thank you, Dex. We'll move on to the big companies and APIs because there's stuff to talk about there. Super dope. Great hanging out, guys. Thank you so much. all right, folks. So, the next on our agenda is the fact there was not a lot of, big news from the companies. And I think the main one that we need to talk about is the image generation in API. So let's let's take this one. Let's just open this up again. I'm lacking a transition for big companies, but no new model releases from big companies.

98:08 No new model updates. As far as I saw, there was a little bit of a price war happening where the tiers have opened up. Wave have you seen this? The the GBT40 and 03 like all of this stuff. OpenAI basically gave like people on the not on the pro tier on the pro tier is supposedly unlimited. on the plus tier, OpenAI started giving like I think 2x the the 03 access 2x like some of the stuff. I celebrated that and some posted thankt. So you have more more messages you can send. So you can say thanks in a message but yeah that I responded to his post and said thanks JBT. Thanks JB. That was that was really great. really great because I was already at my limit for all three and I was happy to see it reset midweek. But I think that absolutely the main news from OpenAni this week that like many many people have been waiting for is the fact that we first of all now have a name for this image generation. They call this GPT image one. they used you know not Deli also I think some refer to it as image gen. because it was really annoying so far to say the new image capabilities within CHP. I used to sometimes say Sora image as well. and so these now these capabilities have now been released in in API and not only they have been released there's a few things that you can do that you cannot do in charge GPT so I don't know folks you already played with this but I will shout out that in order to play with this your organization in openi has to be verified and I went through this verification flow and this felt like TSA pre or whatever like I had to like scan my biometric I have to approve with persona so I know that like many folks would not want to do this I think I get why they're doing this.

99:51 These models are super realistic. They can generate like realistic stuff. they want to maybe fingerprint who did what. like it makes sense when it's on Chad GPT. It's in in their control when it's an API. People can build all kinds of tools. So I think it makes sense. But basically now after you verify yourself if you go to the console of JGBT you go to images you can just now generate images with with the with the image gen GPT image one and I think that the cool thing first of all they shoved out like a bunch of they shoved like a bunch of pre- ready prompts so if you don't know what you're doing they have prompts like speaker ad for example which they even send like a a specific speaker and they say generate an ad for home Bluetooth speaker add this product and minimalist interior inspired by Japanese aesthetic. Let's send one. Let's see.

100:39 but also they have like a bunch of other stuff to kind of train you and give you some examples of like what prompting for this model is. but also you can control a few things. You control quality and this is a very interesting one that unlike Yeah. Yeah. I would love to hear your thoughts. unlike the kind of the other models, I haven't seen that. This is really nice. the playground. unlike other models like flux for example where the the quality is basically based on which flux you use right there's a flux there flux pro flux pro.1 etc. You basically choose the model and like okay it will output the same image etc. This is a reasoning model. This is GPT that does the generation. They just like hide it behind this like image thing. So basically it's like how long to think.

101:23 And so you can literally see in their API here's our here's our ad by the way that we generated with the speaker. It looks pretty cool in a very minimalistic design with AGI.fm. It's pretty cool. It did all of the It follows Thursday. Say again. Should I put Thursdayfm on it? Change this to Thursday. And make the speaker a little bit more unhinged. Try try different qualities.

101:55 Let's see. Let's see the same thing in in different qualities. just to see the difference. Just max it out what,24. Yeah. So you can control the the the quality and the quality controls kind of how long it thinks. I think this is also very interesting because if you're building any type of UI with this and and OpenI launched it and then they already have like 10 partners built in like Playground and FAL already has this like a bunch of other folks already have this. the the controllable length like determines how much you pay for this. So you can't know in advance is very interesting but also how long you wait for an image generation. That's also very interesting. the other interesting thing that will go very well for many folks who are generating images is the fact that you can gen up to 10 at once and that's super cool and they also the last thing maybe I'll I'll say on this while our image generates is they u let me find this super quick thing.

102:51 they they have three APIs. They have generations, they have edits, and they have variations. Variations not existing yet for Deli 2 only, but like I'm assuming at some point this will come as well. generation generate images from scratch. Edits modify images like a new prompt either partially or entirely. And they support masking, which is great because as we talked about the the detect anything thing, you can like mask out whatever stuff you just like click and then send this mask as well. All right. So, let's take a look at super quick at what we have here and then we can move on as well. Okay. So, this is the regular speaker. The image of the speaker was uploaded and it followed all the prompts and then Nissan prompted it to be more unhinged. so here's our unhinged speaker. I actually I kind of love this.

103:36 It's pretty cool. I literally I said change this image to Thursday news please and make the speaker a bit more unhinged and the text is perfect. so I think the the the text part is very important. Like the text is really really good at at this. One thing that's not good I wanted to call out and maybe maybe ask folks from Open the AI to like fix this is faces. This kind of looks like me and this took like multiple iterations when I upload anything with a face. I don't know. It feels like intentional maybe I or not intentional anything else. On purpose I think because they I think they've actually done that on purpose. Everything else in the image. So you guys remember obviously the giblification era of of two weeks ago whenever that was released. Was that two weeks? Was that two weeks? Time flies. Ancient. It's it feels ancient. many many people are still giblifying. that kind of makes you into a Gibli character and kind of all your facial features kind of get disappeared anyway. So like the main ones remain like the beard, the hair, the eyes maybe. But if you ask for more of a realistic thing kind of like this you can see that and this is like the best one out of the ones that I've tried. You can see that it's like it's kind of me but not entirely. And I know for a fact that they can make me perfectly me. Like I know for a fact like I know this for a fact and it does feel a little bit on purpose. No, that's not good. Yeah, that's not a good outfit at all. This is Yeah, this is like one of 15 that I've tried. the rest of them were just completely not related to me at all and I had to say hey replace this with me and this is like the closest one and like like you know this could be somebody else there's like some aspects of Alex here but not like all of them so it's very interesting that like images especially realistic face images do not yet go through specifically because well we verified for this API openai knows who am I like they they know all of this so and also in the in the announcement of this image gen or GPT image one they said that they're removing some of the you know face generation kind of concerns that they had before because in Deli for example that would refuse to generate to anyone and this would generate some people but I don't know the faces don't look exactly the same maybe just do like a generate a Ghibli and then do a reverse gibli that's I'm sure the people so sinister Yep. I'm sure that people will come up with some some prompts or something.

106:06 I think honestly a pipeline I've seen I I've tweeted about this to folks and then folks said that like this is like one of the biggest issues with JGBT and what folks do is they add another model that kind of like aligns this to your face better with some other models and I think a pipeline of this will actually work. I wanted to use this as a thumbnail creator and I think I I'll still post this as a thumbnail.

106:30 It's just like it does does not look good. Any other things that I missed there for the image generation? I think it's very exciting. I think we're we're about to see multiple products do like go go go crazy. we can talk about it seems that it is already being integrated by Adobe Figma Vix Air Table GMA genen oppos. So a lot where they are using avatars or having image modification inside they can now use the API and we have seen how good it works.

106:57 We would see it everywhere. I already started using it. It's great if you have like customer lists or customer cards and stuff and you just wanted to kind of guess an an avatar for for each. You you can just jibly every every customer and then so you don't need to find their actual picture or ask for permission and stuff. You can just guess and then you can just keep track of stuff. Yeah, this this will be very useful business feature. I think 100%. We've been waiting for this and I'm pretty sure that this is going to be super cool.

107:28 They have a cookbook. One last thing maybe it supports transparency. So if you are gener like logos or whatever, it supports transparency. It's pretty cool. I think this is the main news in open labs. We can talk about Gro super quick. I do want to mention the Gro API is now available. We've been waiting for this. They're trying to go for pricing. we should definitely mention this as well. Grock 3 mini is going at 30% sorry 30 cents per 1 million tokens and then 50 cents for out this this seems very interesting like this attempt seems very interesting every everything else the difference between output tokens and input tokens are are higher this is like 30% 30 again 30 cents for 1 million input tokens and 50 cents per 1 million output tokens GR3 mini gets 93% on aime and they're comparing themselves to Gemini 2.5 flash thinking and 04 mini high and deepsec car 1. So like the smaller models 93% on aime is like impressive. I know that many people who try gro are keeping using gro like I specifically I just wanted to to ask you guys like I use grock for Twitter stuff.

108:38 I go on gra and like hey summary I do sometimes use gra for deep research and deeper research a feature that they launched but I gota wonder that now it's an API if any folks are trying this on like windsurf is good for coding like live life live codebench for example let's let's take a look at live codebench here so live codebench measuring the the ability of code generation of models they say the graph mini has better life codebench than Gemini 2.5 thinking and gets well 04 mini high is like the best one here, but it's more expensive significantly.

109:11 so they're showing off some very impressive scores here. including 11 labs of humanities last exam 11%. and this is for the Groini thing. folks, have you tried Grock at all via API in the tools? Anybody tried this? Anybody in the comments as well, please let us know. because I keep seeing people say that the the models are good, but I haven't yet gotten to a point where I like to keep using them. I've tried the graph API. it it just wouldn't give you the sources. So that that's what I found annoying. Like I I wanted the the rag sources from it because that's that's super. But they they don't provide that and I think that's like the most valuable thing about Grock. Wait, sorry, I'm confused. They don't provide sources.

109:57 Sorry, one second. Niss, they don't provide which sources of the but prompt for something. Usually Grock does give you sources either from from Twitter or from or from news. That's not on the API. I have checked. Oh yeah. Yeah. So API only gives you access to basically Yeah. Just the model. The model. Yeah. The the the kind of u the the X connection and the searches like whatever that's not in the API. They said that maybe this will come but we haven't seen this yet as well.

110:30 Yam, what's your take? I just want to say it's a good model for coding. Seriously, I I use it multiple times. there there's when stuff doesn't work in code, I've tried to like, you know, give it to every single one of them and like open like four four or five tabs and just just try it with everyone. And sometimes it's rock. I mean, it's a really good model for for code. So, if you didn't try it yet, do try it out. And also I really like the the thinking tokens like it's really like the thinking tokens are are valuable in rock. They they really do it's like deepse it feels like deepseek where you like to read what it thinks about. Yeah. But but yeah I I think that yeah so so I agree about the the reasoning tokens as well.

111:19 They had this confusion when they released through Grock whether or not they're going to hide the thinking tokens and then I think like Elon was confused like they actually like show the actual reasoning pool. Grock 3 and Gro 3 fast they have like fast versions of them. they the fast versions cost more very interestingly. like they're not they're fast but not I guess they're prioritizing GPUs. They actually the pricing structure here is very interesting to me. Very interesting to me. Check out like this this difference on Gro 3 mini beta.

111:51 the input 1 million tokens is 30 cents. Output is 50 cents. So barely a difference between the input and output. If you go out to Gro mini fast beta that the input is 60 cents or double of the regular one, but the output is $4. So almost eight times more. What what kind of pricing games are they playing here with? it's very very interesting. And on the Gro 3 fast beta, not the mini one, the input is $5 and the output is $25 for one million tokens for fast tokens. That's very interesting. I guess when folks are choosing which API to use, they're playing the game of fast, cheap, and smart, right? Like you're playing this is the game. This is the triangle you're playing fast, cheap, and smart. because then you choose between the two models, then the fast one. I guess that this is their attempt to kind of disrupt the game and and have some folks prefer something versus the other. I think that's it on the Grock API. The last thing that we should mention about Grock is that the app that many people use that has unhinged conversations. We we I think we've played a few of those unhinged conversations here. the app now has a live video view in addition to multilinguality as well. So it it speaks multiple other languages. wolf, I don't know if they speak German, but I saw Portuguese and Spanish, etc.

113:07 but it's not on the Android app, right? It's only Apple, is it? Only Apple. Yeah. Yeah, they're releasing it on Gro. They have a tier that's called something Super Gro or something. on on Android, but like on iOS, they release like multiple things. And so, yeah, on iOS app, you now have the ability to show Gro what you're looking at or have it look at you and roast you. which is I will say very funny if you're up for this. if you use the unhinged modes and you like point your camera to it and like hey here is me like roast me. It will absolutely do do that.

113:40 that's really fun. I think that's voice like the events voice have have video like it is that like live video. Yeah. Real real live stream video. that that works. I'm gonna play with it. I'm gonna play with it a lot. It's interesting, folks. We have I actually thought it was a chill space, but we have two guests here. We have the space for like deep dive into agents. We have deep dive into into voice agents as well. There's a few things that I really want to like cover super quick.

114:10 before we get to like all three, if we have all three responses. So, in this week's buzz, I don't have like a full category. I just want to tell you that we are coming to San Francisco in May 17th and 18th to do a hackathon. If you're coming for Google IO, by the way, this is like the weekend before. So, if you're coming to Google IO, welcome to come hack with us. in San Francisco again, May 17th, 18th, we call this Weave Hacks, and we'll we'll announce the the the sponsor soon, and the prize is going to be super super dope. But if you're in there, if you're flying into Google IO as well, if you're listening to the show, would love to see you. Come say hi. Give me a high five. As always, the hackathons inside the web office, the weights and biases office are super cool. And then our conference called fully connected fully connected.com weights andbiases conference for everybody who uses weights and biases models and weave is going to be the whole company is going to be there.

115:00 could be our last one given the recent news about acquisition that is dedicated to waste and we'll see maybe not but like if hype is what brings you there then maybe our last one two-day conference in June 18th. I'll send the link out in this week's buzz. I really want to show this one out of nowhere. Yeah, you probably seen this everywhere. Send thei out of nowhere released a 24 billion diffusion model, auto reggressive diffusion model that can generate long streaming ready videos with text control. we really have to watch a video of this because like it came out of nowhere. There there was one I guess image video model before and this one is like we we'll just play this on mute while while I talk. It's a great name for a company, by the way. I have to say that's some some top tier name right there. Yeah, the naming is great.

115:53 But the video that we're playing right now, excuse me, the video that we're playing right now of this Maggi model, has this character, consistent character throughout all these image generations, consistent. And I think you guys those who played with video models, you know that consistency is like one of the hardest things that you always try to do. You like generate multiple images, get try to get consistency. they I don't believe they released this model. folks in the comments, please correct me if I'm wrong. but like they mentioned 24B, so I'm assuming that like it's it's about to get released. They did release a technical review and we keep showing video models.

116:35 some of them are cherry picked some of the images and we keep getting blown away. This one is absolutely crushing it. I wish that you know would be able to Holy I think they solved it. The specific Well, this was the main thing why they couldn't make movies because you couldn't get a consistent face and people would notice that. I mean people are not going to care so much about the visual effects as much getting all but they will definitely notice the face and they might have just solved the biggest problem towards AI movies with this wol you want to add about frame pack just a second I think we mentioned frame pack maybe but not so I I just want to like finish up with sandai infinite extension this is I think the highlight of of their kind of whole stick infinite extensions with like consistent storytelling so you can actually like I think they're doing like last frame, first frame some stuff. let me see if I can show you the actual UI. I think I have it logged in. Give me a second. Yeah, I'll just log in super quick. the UI is kind of like this type and then you generate something.

117:45 let's say select I don't have assets. yeah, it's going to take us a minute to generate, but basically the UI is kind of like the extension UI that we saw. Let me show you maybe in the in the video. super quick about their UI. the UI is like allowing you to extend to scenes and you can build like an actual like long video because of it. and I think that that like super cool. The technical report that they released is very interesting for folks as as well. They like implemented a bunch of very interesting things that many people are looking at like getting very very excited. I'm not going to go through all of the Maggie attention forward and backward timelines, but basically they have like they they scroll through timelines as well to to to give this model attention about this is how I'm assuming they get consistency.

118:27 and yeah, I I'm waiting for this model to drop because they did mention 24-bit. They didn't did release the the the technical report. Really hoping this model will drop at some point. right now there there's GitHub, but I don't believe that the model is on hugging face. Wait, there's on there's San AI in hugging face. It is. Yeah. Yeah. Yeah. Yeah. Holy crap. three days ago is here. Release the model. Waits and inference code. Let's go. Whoa. How do we There's no likes on this. Yeah. No, there's like no likes. Yeah, there's Whoa. Okay. And download this right now.

119:07 Yeah. Back it up. Apache 2 licensed model. Weights are out. Go to the weight stuff for a minute. Just see that the weights are there. Yeah. Get clone that stuff right now. Yeah. Looks like they are all right. Yeah. Yeah. Here's all the files. Looking good. Looking good. Yeah. Can you go back? Can you go back just to to T5 like so we see which size inside muggi? Okay. Excsel. Okay. Yeah. sense. I just want to say looks like a 10 better. What do you say? It looks like it was like 10 20 gigs. So, it's probably what like a 10B or so. Yeah. No, they said 24B.

119:53 Oh, 24B. Oh, 24B is what they said on the release. well, we have like they have a algorithm. Super cool. I just wanted to highlight this as well. you Wolf from you mentioned this one which is also cool. a frame pack from I think it's a research demo. Is that the one with the with the long Yeah, that goes up to 120 seconds and it can run on Nvidia GPU with just 6 GB of VRAM. It's already available for local download and use and yeah, you can make longer videos with it. So, it's been a great week for video AI again. Yeah, absolutely. I I I keep saying that this year is more of a video AI year than than anything else. the last thing that I had audio it's a it's many things this year. This year is a lot of things.

120:40 This year is the year of AI. Let's say let's say it like this. I think folks we're at the end here. I think I the only thing that we didn't cover we covered this we covered this the we didn't cover Hunan 3D updated to 2.5 and significant improvement on the 3D part. I'll add this to show notes. we don't have time to show this. with this I will just say a huge huge thank you as we're like coming up on the two hours here that even in a slow week we have a full show. but I a huge thank you for Dexory and Quinn Quindla Kramer for joining the show today talking about multiple things that they released. As as I said, one of the coolest things we get to do here is to talk with researchers and authors of the things that they kind of blew up.

121:23 shout out to Tony sorry Toby I believe. from Dia, the guy who released with his friend like one of the mindblowing most things in the id that we've seen like at least this week. Dia from Nari Labs as well Korean folks. huge thank you for our hosts here LDJ Yam Wolf from and Niston. my name is Alex with Ways and Bices obviously and the most important thing for everybody who tunes in and listens and participates in the show and sends links and and reshares and subscribes. If you missed any part of the show, the show is available as a podcast everywhere where you get your podcasts, but also as a newsletter with all the links on Thursday. That's Thursday. Just simple URL. just follow any of us on X and you see the link as well. Thank you so much. We'll see you here next week. I'm assuming next week is going to be big with some major releases as always.

122:17 We'll see you here next week. Bye-bye everyone. Byebye.

122:48 Hey. Check it.