I tested Claude Duet. Is it worth switching from Grok Bot?

0:00 I have good news and I have bad news. The good news is Anthropic just released a new personal agent and it's really well designed. The bad news is I have personal agent fatigue. I am so tired of new personal agents and while this one is super super well designed and I'll go through what I love about it, I just don't need another one. And so to compete for my attention, to compete for being my number one personal agent, you have to be not good, not great. You have exceptional. And we're going to see in today's episode if Claude Duet, their new personal agent, makes the cut. Let's get to it.

0:40 Okay, before we go into my experience with Claude Duet, just a very very quick walk through of what it is. It is a phone, iPhone, Android based personal agent that is synced up with Claude and does all the things that personal agents do. So it is proactive, although I will go through how proactive it really is. It's proactive, it can use all the tools and plugins that your Claude has access to and they really position this as being a personal agent for small business owners or busy professionals.

1:10 So none of this like your mom's going to use it with Muse and none of this like you have to be a Lauren at Cursor who's going to run like 50,000 agents and ship a gazillion PRs. This is like solid middle of the road business user, SMB, entrepreneur, busy PM, busy CEO agent. The things that you would reach for in co-work on your desktop, maybe you'll start asking Claude Duet for. Now, you all know that I really love Open AI products. I use Codex all day every day, but I will say one thing that Anthropic did so so so well that Open AI's dots did so so so poorly is the polish of the agent design. This app is beautiful. It is well-designed. There are no sharp edges. It is clearly incredibly well thought through from a design perspective. Everything from the very, very early test flight versions that I got access to today has shown me that this is a exceptionally well-polished product. From the second you onboard, they are really thoughtful about walking you through what Duet does, what it has access to, and how to get onboarded. It looks really nice. It feels really trusted. I had zero problems getting up and running out the gate. It was a really nice onboarding.

2:39 It was a little similar to my experience with Muse, where it was very clear up front what it could do, very clear what it had access to, and then you got right into the product. The product itself has some primitives, again like Muse, that I think you're going to really like. Of course, it has plugins and connectors, so it inherits all your plugins from Claude. So, if you have your Gmail setup, your Google setup, your Figma setup, whatever, it's going to have access to those.

3:07 It also has task lists, which is something I really liked from Muse's agent design. I like the idea of keeping track of what goals and tasks I'm doing and what goals and tasks the agent's doing. So, I think the task list view is something that's really nice here and something that we're going to come back to a lot. And then what it has, that again Muse has, but it does just a a little bit more polished version of is artifacts. So, if you're used to artifacts in Claude code or in co-work, if you're generating HTML artifacts, designs, slides, Anthropic has made those first-class citizens in Duet. So, that you whenever you're generating stuff in your little chat with your little Duet, it's going to create those artifacts. And it actually already can read the artifacts existing on your Claude account. So, when I showed up here and looked at my Claude account, it already had all those artifacts that it had generated over time in there, and I had a really good sense of what I could do next. Finally, the last thing I really love about Duet is the idea of channels. So, we're all trying to decide what the primitives of are for agents.

4:13 In Codex, you have all these threads. In chat, you have all these chats. In Claude, you have threads and chats and projects all sorts of things. In Muse, you have your main chat and side chats. Everybody's trying to figure this out. And Anthropic borrowed a little bit from Slack, which I feel like is everybody's favorite collaboration service, which positions these sub chats, these sub threads, these like sub places that you're going to work in as channels. So, now you can kind of think of your Duet as a main thread, a main chat, but but then you can break off other chats into channels and have specific channels like here I created a sales channel for working on sales projects. And then, I'll say my last little thing that I really like that I did give the Anthropic team props on is the icon.

4:59 It's abstract. It's definitely Anthropic Claude-y. It's cute. But I have no problem with this little, dare I say it, dot. it's just adorable, not offensive, not weird. I like it. Yes, of course, I love my personalized Muse consumer agent, and that engenders a lot of joy with me. Of course, I love my little Rock bots and their little icons, but this is inoffensive, and I think it's just fine. And as somebody who cares about design, I thought it was pretty good. Now, what are some things that I did not like about Duet? Well, I will tell you, for a proactive agent, it just wasn't that proactive. I don't know if I forgot to turn on notifications or it's just not that annoying, but once I set up Duet a couple days ago and did a first couple tasks, it never pushed or bothered me to do anything ever again.

5:49 It also doesn't show up in the desktop app, so it's a really disconnected experience. And so while I felt like the initial experience was pretty good, I ended up not going back to it. One, because I have a million other personal agents. And two, it really didn't pull me back in with useful things. I think one of the things that these agent designers are really going to have to grapple with is like there are only so many three-point morning briefings that can pull me back into a product. I get them from Dot all the time, they're not that exciting. I get them from Now Duet, they're not that exciting. Like there's got to be something better than the morning briefing to engage agent users on why they would come back to the agent over time. And right now, I don't think any of these agents, except for maybe Grockit and Muse, have really figured this out. So it's not as proactive as I want. The notifications aren't noisy enough for me to remember it. You know, I've said it's elegant and well-designed and subtle and not annoying, and those are all positives in some ways, but from a growth-hacking perspective and the ability to get me to re-engage with your product, that doesn't work. I need you to be a little bit more annoying. I need you to be a little bit more proactive. I need you to be a little bit more noisy, or I'm truly going to forget you in the ecosystem of all my agents.

7:02 So what were some things that I asked it to do, and how well did it do it? Well, the first thing I asked it to do was just this like basic kind of like go through my email, go through my Slack, figure out what's going on. I would say it didn't do an exceptional job at that. It did okay. It came up with a couple things, but the priority didn't seem right, the context didn't seem right. So I do think there's a little work on the quality of the interaction. I asked it to find me good Warriors tickets, you know, go Dubs. I want to go see the Warriors when they play the Pacers. It's a noon game. I can actually take my kids. And so I've had all my agents try to shop for tickets for me, shop for shoes for me. I thought this was a good eval. I had it go shop for me, and this is the second thing that I think is bad about it. So, if the first thing that I didn't love about Duet is that it is not that proactive, the second thing that I don't love about it, and this has been my experience with Claude generally, is it's slow, man. And I get so disengaged waiting for the models to finish. The killer feature of Muse Meta is that it is fast. In fact, Codex with their GPT ultra-fast models, it just keeps me engaged because I am always moving to the next step, always getting something done. And so, even though I really love Opus 5.5 as a model, and even though it does really good work, the performance is so slow, and you really feel that in a personal agent, and it degrades my desire to work synchronously with an agent. You know, one of the benefits, I think, of an AI agent is that it is like a colleague, but a colleague who's up 24 hours a day and responds to immediately. But, if that colleague is slow, if that colleague goes to a meeting to get work done, like, you kind of forget and don't want to go to that colleague over your more responsive ones, even if the more responsive ones do like 10% worse of a job. And so, it was shopping for these Warrior tickets for me as I was like passenger princess and driving to dinner, and like I kind of forgot that it did it cuz it took so long, it didn't close the loop, and then like the time that I came back, I was off to something else. And so, even though it tracked it in this, you know, checklist of things that it was doing, the performance was really frustrating.

9:16 And I think this adds up, especially if you use a single chat, single agent, where you're firing off a bunch of tasks. Like, you're saying, "Okay, check my email, and I need you to make a deck, and I need you to make edits on this thing, and let's figure out what we do for social." And in Claude Duet, it those all kicked off individual threads, but then they took so long. And I'm telling like 10 plus minutes to finish those threads or come back to me or ask for permissions. And because it wasn't noisy, because it wasn't, you know, like in my face, I actually forgot about all those threads. And so, while like foundationally it's really well designed, the models are very good, the intelligence is there, and like I can't knock anything specific about the user experience, the reality is like the latency and the inability to close loops meant that this particular agent product just really fell to the bottom of my use list. I think the other thing that they're going to have to solve ASAP. And again, this is in research preview, so it's an early access list, it's in beta, I'm sure they will get there, is a desktop app. The reality is I do not want to be on my phone during the day when I'm doing work and sitting on meetings. I need a desktop app to engage with my agents. I think that's why like Rockbot does so well is if there's a desktop agent, it's cute little icon, it's sitting there staring me in the face. I'm always chatting with it like it's Slack. Same with Muse. You know, I started using it as a mobile app, but very quickly downloaded the desktop app, and that allows me to pull through all these tasks that maybe I kick off asynchronously on my phone and want to finish when I'm doing work at my computer. And then even same with dots from Codex, which I don't even use, but at least it's there. Like at least my dot bay is staring me in the face in Codex like with a little notification saying like do something here, get something done. And so, what I would say about Duet is like it's got all the makings of a great agent except for the ability to drive urgency for me to engage with it. Maybe that's intentional. Maybe the design is to be calm and soothing and helpful, but in the background. But, I do think to compete for attention right now, it is not enough to have beautiful design.

11:44 It's not enough to have exceptional models. It is not enough to have these artifacts, which is actually really awesome. And on the phone experience is quite zippy and felt really engaging. The agent itself has to be super proactive. The agent itself has to be super fast. And it has to get stuff done quickly enough that you feel like you can close enough loops that it is a reliable coworker for you. Okay, so that was a very long aside for buying more your tickets. A couple of other things that I had it do, good, bad, left, right, you know, standard. I had it look at my sales pipeline. Nice things there.

12:22 It was already connected to Slack. It was already connected to my email. It was already connected to our CRM. So, it was able to pull in that data pretty quickly. I had it create a deck in Google Docs. I have a really high bar for design, and so I was disappointed that it didn't use Figma and really match our design in terms of fonts and all that kind of stuff. But, the deck wasn't bad. It was created in Google Docs and it was kicked off from my phone. I think for like the kind of everyday business person who maybe is not as particular as me with design, they'd be really happy to create like little sales decks off a template, little documents, etc. on your phone on the go with Duet. So, I think that was like good, not great, but I have a really high bar for that sort of thing. And then one other point of like friction and disappointment, again, this is a me problem, not a Duet problem, is I wanted to see if it can make a launch video for me. I've been making all these launch videos in Claude. I've been spending a whole bunch of time doing like chat PRD launch videos in Claude that look like this. They're super cool.

13:27 and I wanted to see if I could create those in Duet on my phone. The reality is, unfortunately, Duet does not ingest all your local skills. so, I probably need to push these skills as plugins to Claude, and then it can have access. But again, I think with all these product surface areas, and this is not just an Anthropic problem, this is an OpenAI problem, is like the shared context, the shared experience, what ports over elegantly, what does not port over elegantly, I think is going to be an open question for folks who want to really migrate to Claude Duet. I'd say the final thing that's really missing is Look, we don't work alone. And Duet right now is clearly a agent for single-player use. In fact, like most of the Claude projects feel like very good products for single-player use. I just don't think even with Claude Tag, Claude in Slack, all these things, I don't think they've quite cracked multiplayer.

14:23 And I've been moving more of my Grock bot usage to team agents and deploying those in Slack. So, while Claude Duet with these channels might feel like Slack, it doesn't have that multiplayer sharing aspect of Slack that has made me really effective, even as a small business owner, right? Like, I'm not running a team of 500 anymore. I do not need like the full enterprise deployment, but I am a small business owner. So, I should be the target for this product. The problem is I have a small team. So, you know, I have like five humans that I work with on a regular basis. If I don't have a way to share my agents with them, I'm not sure if it's going to worm its way into my working stack. So, that is my very quick preview, early review, early thoughts on Claude Duet, Anthropic's new personal agent. I would definitely give it a try.

15:14 Again, I maybe have different use cases, different preferences, different desires than most people. I think it is approximately 100 times more polished than Dots was when it came out. So, I think that's really nice. I think it is quite elegant. It does the job well. I would just say there's like fit and finish things that they really need to work on if they are going to compete for my agent business. So, give it a try. Sign up for early access if you can. Let me know what you think and let me know if Claude Duet becomes part of your stack because I'm just not sure it's going to become part of mine. Thanks for joining How I AI.

15:53 Thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube or even better leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.