transcribe

Ex-Google VP: How I Built My Own Personal AI Agent | Peeyush Ranjan

SparX by Mukesh Bansal · 52m · transcribed 16d ago
More from SparX by Mukesh Bansal Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

The Concept of Personal Agents

What is a personal agent and how does it function?

A personal agent is designed to perform tasks on behalf of the user, such as making bookings or providing information, with minimal user intervention. The speaker discusses their experience with Google Assistant and their own personal agent, emphasizing the agent's capability to handle complex tasks autonomously.

  • Personal agents aim to reduce user intervention in daily tasks.
  • Google Assistant serves as an example of a personal agent that provides information and performs actions.
  • The speaker's personal agent was designed to handle complex requests seamlessly.
# 10:30

Use Cases for Personal Agents

How are personal agents utilized in practical scenarios?

The speaker describes using personal agents like Muse for complex tasks, particularly for navigating non-intuitive government websites. They highlight the efficiency of Muse in booking processes that are typically cumbersome.

  • Personal agents can simplify complex tasks, especially in challenging environments like government websites.
  • Muse is recognized for its accuracy and effectiveness in handling intricate bookings.
  • The speaker shares personal anecdotes to illustrate the historical challenges of booking travel.
# 21:00

Multiplayer Coordination with Personal Agents

Can personal agents facilitate group coordination?

Currently, personal agents like Muse and Instinct do not support multiplayer coordination for group activities. The speaker discusses the potential for agents to manage group tasks, such as coordinating trips, and shares insights on their company's approach to developing such features.

  • There is a gap in current personal agent capabilities regarding group coordination.
  • The speaker's company is exploring solutions to enhance agent functionality for collaborative tasks.
  • The evolution of personal agents may lead to more integrated group features in the future.
# 31:30

The Future of E-commerce and Personal Agents

How will personal agents impact e-commerce?

The integration of personal agents with platforms like Shopify is transforming e-commerce by reducing human friction in transactions. The speaker notes that as agents handle shopping on behalf of users, traditional advertising models may become less effective.

  • Personal agents are changing the landscape of e-commerce by streamlining the shopping process.
  • The rise of agent-driven transactions may challenge existing advertising revenue models for platforms like Amazon.
  • Future e-commerce may rely more on agent interactions rather than direct user engagement.
# 42:00

The Role of AI in Society

What are the implications of AI and personal agents on society?

The speaker expresses a balanced view on AI, acknowledging its potential for good, such as improving access to healthcare and education, while also recognizing the risks associated with Big Tech's control over these technologies. They emphasize the need for ethical considerations in AI development.

  • AI has the potential to significantly improve societal outcomes, such as healthcare and education.
  • There are concerns about the concentration of power and profit within Big Tech companies.
  • A balanced approach is necessary to harness AI's benefits while mitigating its risks.

Transcript

0:00 Last few years at Google I was working on Google Assistant and Google Assistant was trying to be an agent of some sort. >> you failed. >> >> 10 years ago I invented a personal agent. It was the world's most advanced personal agent. It has a WhatsApp interface. I'll type a message on WhatsApp. I can say book two-day travel for me for let's say Dubai or tomorrow's meal I want to have this kind of food. I am traveling to Boston. Check the weather. Pack accordingly. It all get done. Zero intervention for me. And now Facebook has stolen my idea looks like with me >> Well, what exactly is a personal agent?

0:33 >> Google Assistant was a device but you could say things to it and it would give answers. >> Okay, Google. Toy stores near me. >> The first one is Vincent Toy Store. >> But you could also tell it to do things. Like you could say switch on the light. >> Okay, turning on the lights. >> And it would do that aspect is actually what makes it agentic or agent capable. >> Do you use Muse? >> I have used Muse, yes.

0:53 >> What kind of use cases? >> So, I use all of these agents. >> What do you do then? >> Yeah, so I >> >> There are things we love to do and the things we have to do. The things we have to do are the ones which we'll send to agents and the things we love to do >> Oh, this is a very good >> Oh, your agent agent ordered for you. >> You saw the recent news on OpenAI that an employee was caught putting a spy device in the conference room where executives will meet.

1:19 >> You know what actually was even more fascinating in that was and maybe it's a marketing thing what OpenAI is doing but OpenAI said that they used AI models to figure out who had installed it. >> Do you think someone can and should try to build the Muse Instinct equivalent for India? It may look different, it may do different things but is there an opportunity or once these behemoths are in India, they're going to steamroll over everybody?

1:40 >> I think the answer to both of them is >> Hello, I am Mukesh Bansal, founder of Myntra and Cult. I also backed founders of companies like Skyroot and Grow. At Sparks we talk about everything deep tech and deep dive into all the topics that's going to shape this century especially if this is going to be India's century. I'm convinced it has to be built on foundation of very strong tech. A lot of exciting things are happening in the world of tech.

2:09 I'm sure everybody listening to this has used Google Pay. whether it's ordering your daily chai or evening pizza or booking movie ticket, Google Pay is probably an integral part of your daily life. Well, today we have the person who was deeply involved in shaping the road map for Google Pay and scaling it to what it is. Our guest today is Piyush Ranjan. Piyush, welcome to Sparks. >> Thank you, Mukesh. Nice to be here. >> But, Piyush, I want to first start by telling you a story.

2:36 >> Mhm. >> You know, 10 years ago I invented a personal agent. It was the world's most advanced personal agent. >> Mhm. >> What it was, you will relate to it. It was a WhatsApp interface. >> Mhm. >> I'll type a message on WhatsApp. >> Mhm. >> I can say "Book two-day travel for me for, let's say, Dubai >> Mhm. >> or for tomorrow's meal, I want to have this kind of food >> Mhm. >> or I'm traveling to Boston, >> Mhm.

2:58 >> check the weather, pack accordingly." >> Mhm. >> It will all get done. Zero intervention from me. >> Mhm. >> But one catch. >> Mhm. >> The catch was the whole army of people behind it. And now Facebook has stolen my idea, it looks like, with Mews. And it can actually do all of these things. You know, I saw the Mews launch >> Mhm. >> where someone just gave a request that "Sell my car >> Mhm.

3:20 >> for you know, at least $40,000." And the Mews basically, you know, went and reached out to a lot of dealers, negotiated on behalf of the person, and eventually managed to I forgot whether it's buy or sell car, but basically got the whole complex transaction done. So, we are really in a crazy time, you know. This is What do you make of this, you know? These things like Mews, Instinct. You think they are here to stay?

3:44 >> Yeah, yeah. I I think actually you are right. Like it's it's it's a crazy time, and the pace at which things are getting better in in some way is unbelievable. Like if you say like how good could it be from 3 months from now, it's very hard to tell. >> Yeah. >> as far as the second question which you said key are things here to stay there is I believe there is no doubt. See there was a need why you were doing that 10 years ago, right? That need still exists and now it is getting democratized across. If you remember a while ago we had talked about how AI is about democratization of personalized services.

4:17 This is the trend and people want those things done. It's just becoming more and more accessible that everybody can have the army of agents behind that chat interface. >> Look, AI has been around for last 3-4 years and most people's exposure to AI is only through GPT. You ask it some question or health advice or some information and it kind of looks it up for you and gives you information. But personal agents are whole different ballgame.

4:44 >> Yes. >> Right? It's a Why don't you paint a picture of what is a personal agent because I'm assuming most people listening to this have not experienced a personal agent. Me was still not launched in India. Instinct is only available by invite only. I think very I just got the invite thanks to not thanks to you. I think thanks to Anuj a few days ago and it's quite quite powerful. I had I had a trip coming up so I actually gave it a problem. My very first prompt to it was that I'm going to in Boston you know 10 days from now and can you look up availability at Four Seasons and give me best available rate. And since then it's basically behind my life. It has found the availability. It wants to book it.

5:23 It's just waiting for me to share my credit card which I don't want to at the moment. But just you know why don't you describe for people what exactly is a personal agent? >> Yeah yeah we I I would love to tell you and I we should also touch upon that don't want to give my credit card like that there is that those boundaries exist. One other thing which I wanted to share last few years at Google I was working on Google Assistant.

5:42 >> Yeah. >> And Google Assistant was trying to be an agent of some sort >> And then you failed. >> No. Well Google Assistant had its own incarnation. We didn't fail. It's like I think half a billion people in the world use it. >> Me, you Instinct and who's third? >> Well we'll we'll we'll come to that but but the talking about the the the most of the world thinks you type into a box and you get your answer, is what most of the world knows. So, Google used to do that, then ChatGPT came about, and and that is what most of the people are using. So, when they use these the AI of today, you are right. They're mostly using it as a chatbot.

6:21 >> Yeah. >> What has happened behind the scenes, and this is why I gave the example of Google Assistant, Google Assistant was a device, but you could say things to it and it would give answers, but you could also tell it to do things. Like, you could say, "Switch on the light." And it would. That aspect is actually what makes it agent agentic or agent capable is that you are no longer just intelligence, which is giving my answers, but you are beyond that, you are hands and memory and things like that. That you have a set of tools you can use on my behalf. So, that is the first thing which is making these things possible. So, it's not just that you talk to ChatGPT, but ChatGPT can now do things for you which are outside of answering questions out in the world.

7:03 That is one. And then, the next layer of that is it has memory in it. So, it remembers you. Like, this example of Instinct, it's going to start to form information and and a mental model of what you're doing, including if you give the credit card, it will not ask you the next time, and and so on so forth. So, it forms memory. That also was not there, like, you know, in in the chatbots, you didn't expect much memory beyond the session.

7:26 >> >> And the third big thing which is happening is proactiveness. Google or ChatGPT don't answer to you if you don't ask a question. Now you got tools, you got memory, and you got proactiveness. So, on intelligence, when you add these three things, now you have a very capable agent which can do a lot of things for you. And that is what we are seeing in these kind of experiments. >> Most people today, their life is run by an assortment of apps.

7:52 >> Mhm. >> Everybody has their apps, whether it's Google Maps or a payment app or a shopping app or a food delivery app and so on. And the apps mostly have all the information. You just go, you know, type few times and the work gets it What I realized about just a week ago that the way Muse and Instinct are working is that if I sign up for Muse account, behind the scenes they run an actual full computer for me.

8:15 >> Mhm. >> They will download those apps for me. >> Mhm. >> if I'm willing to share my credentials, they will log in, let's say, on Swiggy on my behalf. >> Mhm. >> And then when I write a WhatsApp message saying, "Can you order, you know, biryani for me?" >> Mhm. >> They will use my Swiggy's account The agent will use my Swiggy's account, go order, and come. That's the incredibly powerful, right? That's a huge breakthrough in terms of, you know, how this >> Yeah.

8:39 >> architecture of personal agents evolved, and then looks like there's really no limit to how far it can go. >> Yeah. Well, so we can get into the technical capabilities which were there before and this leapfrogging has happened. So, when first time things like ChatGPT and all were there, to be able to use something outside in the world, >> Mhm. >> they used to what we used to call tool calls. >> Yeah. >> They used to call tools. Right. Then there was some standardization, and people came up with things like MCP, which was a protocol by which many other application providers could expose that themselves as a tool.

9:16 >> Yeah. >> So, this thing knows how to call it. You would authenticate, authorize, and then you would call the tool. But the entire world is full of many such applications where there is no No one has invested in MCP you know, gateway and things like that. Therefore, these chatbots were able to only do little bit. >> Mhm. >> Now, what has happened is two big things have suddenly gotten much better. These models have become over just past year so much more intelligent, so much more capable of multi-step long-running tasks. There was a time you could not give it more than like half an hour. Now you can have it run for 16, 17 hours, and it would actually go perform multi-step long-running task. And on top of that, the computer use and browser use are two main technologies which have now become possible, which means this thing can use a browser like a human does, and this can use a computer like a human does, the GUI interface of computer. The minute those two happens, now suddenly you have leapfrogged into the entire world of applications which did not expose themselves as a MCP server. Now, I wanted to order some like if you wanted to search Four Seasons.

10:21 It is opening the browser inside the computer. It knows if you want to run an app, it actually will know how to do it. You can run it on your Mac, you can run it on your phone. And those two technologies are making it possible. And suddenly it has become omni-capable as a result. >> Do you use Muse? >> I have used Muse, yes. >> What what kind of use cases? >> So, I use all of these agents. My >> What do you you do then?

10:42 >> Yeah, so I I use them. This is what This is where it is. Like I sit to that interface and I say and my army goes and does the things, right? So, I I use all the agents, but I will tell you what I use Muse for and also, you know, we built Enrico, right? So, so I built like even before Open Claw and all had started, I was trying to build something then move like so we have been through the entire journey.

11:04 With Muse, what I have found is it's amazingly accurate at some of the things which I wanted to do which are pretty complex. And I'm talking about compared to Grokbot which is my current primary most established personal agent. And what I use it for is very complex bookings on government sites. Because government sites are very non-intuitive, they're not optimized for flow and anything. But and it is painful to go through. I mean, everybody knows the story of IRCTC and things like that. But when I have to deal with that, I actually go to Muse and I say, "Okay, can you book and and it actually does a fantastic job.

11:40 >> So, Muse is not in India yet, but IRCTC is a excellent use case. You know, I remember in my like when growing up in Haridwar and also NIT Kanpur, we used to go to railway station and wait outside the door for like up to an hour. And you know, the way when the guy will come, they will not open the shutter completely. They'll open just like 1 ft. So, people will like lie down and roll on the floor. And whoever gets in first, so hopefully you know, Muse is able to resolve all of that. And so, can you Would you be able to like when the Muse is launched in India or whatever equivalent, can you say it just book a ticket for me from Delhi to Haridwar on this day. I'll keep trying IRCTC until it gets it done.

12:15 >> Absolutely. As a matter of fact, you know, it will do the digital rolling under the almost sometimes using some of these websites No, I actually have done that, you know, 100 hands shoved through that small window. You know, I grew up in Muzaffarpur, so like getting a ticket meant going to the train station, getting in the line, and shoving in them, finally getting something. This is like before the days of digital tickets. So, but where I end up using is where I find dealing with when I don't want to do something, right? Then I'm telling Muse call up all these guys like go do it.

12:47 And it is actually surprisingly good. >> Yeah. >> Yeah. >> Muse numbers are staggering, right? I don't know it is one of the I I don't know if you had recall any number top of your mind, but it has already reached something like many tens of millions of downloads, number one. >> It's supposed to be the fastest-growing AI app, beating ChatGPT. you know, in terms of its early growth. I don't know what the numbers are within the first month that it has come out.

13:12 >> Yeah, I think it's only about a month or so since it has come out, and I don't know what Facebook is waiting for >> >> to launch it in India. But the other interesting app is Instinct, which on one hand you have this, you know, >> >> Goliat, you know, Meta, right? You know, they can put billions of dollars behind it, and they have the full firepower of Meta behind it. And then there's this 23-year-old guy who's just, you know, from out of nowhere he has launched the Instinct app. It's still not publicly available.

13:38 You need to get an invite from somebody. They I think I think something like 2 weeks ago raised money at 2.5 billion valuation, and a week later they raised a billion dollar, 10 billion dollar valuation. What is going on, you know? What is so unique about Instinct? >> The big biggest thing which they have done is they meet you where you are. >> Yeah. >> You want to talk to them on iMessage, you want to talk to them on WhatsApp, they're there. And it almost is as if like you, you know, going back to like you're talking to your main concierge or your chief of staff.

14:07 >> It was just a brilliantly creative idea that he implemented. He figured out a viral growth marketing, you know, through invite-only, and he really moved the needle. I mean, it's a Mhm. Mhm. Yeah.

14:39 Yeah. Yeah. Yeah. But it is a sign Yeah.

15:13 Right. Right. Yeah. Mhm. Mhm. Mhm. Mhm. Mhm. Yeah. Mhm.

15:48 Yeah. Yeah. Sure. Mhm. Mhm.

16:18 So I always used to look at it and I used to say there is lot more productivity which is hidden here for somebody. So, I always wanted to build a chief of staff agent. So, around late last year I started thinking how to do this because the models were becoming capable, MCP was there. So, the first version was just run a bunch of loops. Right? Like, okay, every day I want you to summarize my email inbox for me.

16:40 Then it is possible because calendar is there, email is there, and intelligence is there. So, it would actually go run a bunch of loops. Then it was very clear that running loops is fine, you need some memory on top of it. I was playing around with that. Then Open Cloak came out and Open Cloak came out and I quickly switched to Open Cloak. And there there two major open source platforms for >> People may not understand what Open Cloak is. Maybe you can just explain what what was Open Cloak.

17:01 >> Yes. So, Open Cloak was the first open first agent which came out and it took the world by storm. The whole personal agent thing actually came lot more from the hobbyist tinkering and scripts to actually something which could naturally become useful to people in real life. And there was a there's a guy he's a he's now at OpenAI. He built this service which essentially allowed you to plug in your OpenAI or any key so that it can get intelligence from there. Connect your Telegram or WhatsApp on the other side.

17:39 And this system the service would actually then act as your agent. And you can connect many other things to it. But you can talk to it either by sending a message on Telegram or on WhatsApp. and you can also connect your email and all these other things so it can now start to do things on your behalf. It did not have inside it right out of the box the browser use in the computer use in the VM pieces which you were talking about. But this is earlier this year. This is how fast things move. This was like February this year I think. So, Open Cloak what suddenly happened was because you could talk to it and it could run by itself and it had a memory system and it could talk to the world outside. Now, all those pieces came together.

18:21 And he made it open source. People started using it. Installing on their MacBooks. Mac Mini is what they were installing in. Apple's Mac Mini sales went through the roof. They they ran out of stock. But the beauty was now people were able to tell it to do things. My favorite story is and this is actually behind Muse. There's a guy at Meta, Nat Friedman, I think his name is. He posted, I remember seeing that I couldn't believe. He said that he had told OpenClaw that his agent in OpenClaw that, "Hey, keep me hydrated."

18:54 And it actually had it had connected it to all the cameras at home and his Tesla and everything. And it actually said that, "Hey, I see you going in the kitchen. You haven't drank enough water yet. Please go fill fill up the water in your thing." That is the level of this agent's perception and action was there. >> There's one other thing we should clarify. I mean, some of the people listening to this might wonder, "Okay, it sounds very cool and interesting, but also sounds very complicated. You know, Piyush is a you know, engineer and all that can put it together." But I think what's what I want to clarify is this the new way of agent whether it's Muse or Instinct or the more like it. I guess Open AI's dot.

19:34 They are just WhatsApp interface. So, literally using this agent is like you just go to WhatsApp, you type what you need, and then the response talks to you as if you're talking to another human agent at the backend, and it gets the job done. And all the technology is hidden from you. It's all you know, it may have maybe using bunch of apps behind the scenes. It will need your credentials at the right time if you need to log into a particular account.

19:57 If you need to cancel your order from Amazon, it obviously needs to know what your Amazon account is. But but as far as users is concerned, it's just WhatsApp. And I think people will be curious in India, "Okay, how do I get access to it?" I guess the two answers. One is be patient. Like, you know, India is one of the the largest market for WhatsApp. And an Indian now runs WhatsApp, so we all have more reason for it to come to India soon.

20:23 But the other message this probably a very limited window of opportunity for Indian entrepreneurs. Like where is the instinct of India? Technically it's not that challenging to put this together. >> today well, there are things which Muse for example has done which are technically very interesting. There are some pieces but it is nothing Indian entrepreneurs cannot do. >> Yeah. >> the onboarding is very simple. You just start talking to it. You say, "I want you to do this." And if it needs access to your email, it will actually say, "Okay, click here." And if then you will go configure and the beauty with all of these systems is they're being made in a way that even if you give something sensitive >> Mhm.

20:58 >> it is separated out from the agent >> Mhm. >> and given to the agent at times of need. >> Yeah. >> So, there is a little bit of that architecture which they have built. But to the end user, yes, it has become very simple. >> I don't know about another use case which I think is also pretty relevant to what you're building. Are these agents already there? Can they create a WhatsApp group with let's say my friends and we are coordinating a trip to Kashmir >> Mhm.

21:21 >> and add my Instinct or Muse agent saying, "Can you please coordinate and let every you know, just listen to everybody and figures out and you know, books rooms everybody?" Like this multiplayer coordination. Right now for best of my knowledge, right now Muse and Instinct don't do that. >> They don't actually multiplayer is a edge of you know, it's almost like a cutting edge of what exactly state of art of what the problems are agents have to solve. That has not been done. What people have done and I will actually love to tell you about how we are approaching it in my company.

21:52 >> to pitch your product. >> >> I this is opportunity for me to plug what My Equis doing. But what is the current state? If you have a personal agent, it truly is a personal agent. Now you are talking to four people, they all have personal agents. So, now is it like eight people are in the group? Right? We We actually did build the previous version of the agent called Enrico which we had I think in a prior podcast we had talked about a little bit about that. That is exactly what it was.

22:19 I had built it for myself as a chief of staff agent and then I walked into Fermi and then people said they also wanted. So we created one for everybody. But then when we were in a slack channel a meeting, now there are four five people. So then then it was very clear you need a multiplayer agent. Multiplayer agent the problem is like this and no nobody like Muse and think all of these I haven't solved it yet and I wouldn't be surprised if they they could do. But the basic problem is that if you remember I said like their tools and their memories inside this how do you as a agent know that this is a fact which pertains to Mukesh and should not be shared with others or can be shared with others. This is Piyush's fact. In the memory store it is not just partitioned. It actually is that there's more complexity to one's understanding of a group. So that group's understanding is something which is not today agents do. They don't understand a group of people. They understand an individual user they are talking to.

23:13 Second thing is credentials. Your credit card number stored with the agent because you gave it for some Kashmir booking shouldn't be something which I can say, "Okay, now go buy me a Tesla." Yes, but these are the kind of things which are problems which have to be solved. However, right now even individuals individually have not completely started completely started trusting these personal agents. So there's adoption curve which will be there, but this will also start to happen and Myriku is a company which we are building which is solving this exact problem in scenarios where you cannot be one-on-one. You have to be multiplayer and a simple scenario is a team setting in a company. If you have a employee in your team that is a multiplayer agent by default. You cannot just make it individual because no employee in a company just works with one person with no memory of anything else. Right, they'll not be functional. So these are the kind of problems Myriku actually we are piloting with companies where we are bringing it in and the work starts moving much faster because this agent comes in with a multiplayer agent is able to track everything which is going on is able to tell that these things are more important, this is less, this is what this person should do, goes and does work for different people. Like all of that and suddenly the productivity goes up because of this one multiplayer agent.

24:23 >> So for me to fully understand let's say you have a team and maybe there's eight or 10 people. Today obviously as you said they talk to each other, they write emails, they're probably on a WhatsApp group or Slack group and they have a gazillion meetings every day, still they are not on the same page. You add this multiplayer agent to in some same system. >> Yeah, same same group, same meetings, same Slack channels, same email groups and all that.

24:45 >> And so it's observing everything, figuring out who is who, who can do what, where are the bottlenecks and who do chase, who do remind to hydrate or whatever is required. >> Exactly. So you It's just like what a normal It's a agent architecture but made for multiplayer. So what happens is you're putting it in there and it is looking at all the document, who is saying what and all that and it starts to soon figure out that for this project these people actually have to do these things. It sees that in that room, for example, somebody said, "We have to create marketing collateral for our new campaign." It just said that. It knows that has to be created but nobody said who. But it knows who can do it. And it say, "Oh, by the way, in that meeting which you missed, this was said and you probably should know about it. Oh, by the way, I have already gone and looked at how you guys had created for the past three campaigns. Here is a possible template, right? And now all of a sudden the productivity has gone up because the information flows through."

25:38 >> And the related idea, so one is So I think it's inevitable that everyone will get their personal agent. And at some point these agents will become multiplayer. And then at some point that, you know, my agent will probably talk about your agent, right? That is also going to happen, right? >> Yes. Yeah, it actually 100% will happen and it is starting to show in the very narrow slice of coding agents but in the world it definitely will happen that everybody has a personal agent and things they want to do.

26:09 you know, the the agents are able to resolve between them. The Kashmir example is a great example of that. That going for multiplayer, you don't even add anybody to it, you just tell your agent that, "Okay, I'm planning to go to Kashmir with these three people. Can you go figure it out?" And it actually has a directory, it knows how to discover their agents and goes and talks to them and they all are doing and they're only escalating when they need, right?

26:30 Otherwise they they have the calendar, they have everything figured out, and they escalate for approval or something like that. >> I think it's amazing that I think as humans we are working very very hard and fast to make ourselves useless and redundant and over the world to to these agents to run the show for us and then eventually they will decide whether they need us in the show or not. >> Well, that's a that's a whole future debate which happens, but there's a school of thought which I subscribe to which is that there are things we love to do and there are things we have to do.

27:04 The things we have to do are the ones which we'll send to agents and the things we love to do >> Oh, this is a burrito your agent agent ordered for you. >> Yeah, I have to tell you the story about this. So, things we love to do and things we have to do. Things we have to do we tell the agent. Here is the story of this. So, I was running late for the podcast and I skipped my lunch. So, I told my agent that and we knew address. I said like, "Hey, can you order some burrito for me for at Mukesh's address?" And it actually went through. It did ask me that okay, which is the place I it asked me OTP because my phone number came so it OTP I gave to log into Zomato. It I figured out Zomato is the answer. It figured out that it has it needs OTP. It actually said what is the OTP I gave it. Then it said okay and through UPI I did the UPI payment when it notified me and it actually showed up here.

28:00 >> Just for the record, the person who delivered is not my Rico not any agent. The actual human who brought it downstairs, but this is really insane. I mean, this is pretty much how I guess all of us will be running our lives >> Yes. >> pretty soon. And it's also I mean, so convenient, you know, I can imagine like if you had to do the whole thing on your own, you fuss here in a moving car, you go to Zomato, you browse, you find which is the right burrito, you know, all of that and you would find address. It would so much like it easily take, you know, 5 minutes 7 minutes.

28:27 >> Yeah, and it the thing is what am I doing because going back to your original question is like will we be doing I think that the world the work load splits into is now you have a choice. Do I want to do this? >> Yeah. >> I at that time wanted to kind of sit and get my thoughts together of like okay, you know, we're going to have a podcast and didn't need the cognitive load of ordering, but I told somebody who did it and had the trust that it'll get done.

28:50 >> So I assume like to be able to do that you you've already shared your credit cards. >> I shared my UPI ID. >> -huh. Okay. >> So when UPI ID India actually that way with payments is so much better than in US I do share my credit card with Grockbot. The way it works there is I don't share it with it, but I have connected linked link with it, which is Stripe's link solution. So they know my credit card. So it actually pops up. And link generates a one-time credit card number, which goes back with a certain limit and it goes and uses that. So it cannot be reused.

29:19 >> Mhm. >> Here it is a lot easier because UPI ID so it plugs in and I get a notification on Google Pay, which I then go and and pay. >> So do you anticipate that as and when these agents start to come to India, people will start sharing their UPI ID because for most useful transaction the world, someone needs to pay something. >> Yeah, and also I mean you know, there is a lot of different kind of when we were building Google Pay, we saw so many creative ways people were scamming others as they were learning this new form of you know, >> That was without AI's help. With AI's help, you know, >> So so it is going to be crazy, but I think that the way there are two paths to it. One is that UPI itself as a payment system has to come up with ways and I believe that they are working on something Reserve Pay UPI circle.

30:06 They're coming up with ways NPCI on how to allow payments from agents. >> Mhm. >> the easiest control there is like some sort of a set of limits and merchant like who's the merchant you are paying to, how much are you paying? Those kind of rules almost like you you do with a child. >> Yeah. >> It's like this is what it is and then the payment only goes through that. So it it decreases the blast radius of any you know, scam which happens. that is one. And the other is my suspicion is that which will be natural, it's happening in US, it'll happen here, is people will not go directly to the payment use cases right away anyway.

30:41 They will build their comfort with these personal agents. Like see in your case right with Instinct, everything happened and then it's waiting on you for the credit card. So, as you build now because I have been using it for a while, it my Enrico was actually doing shopping for me. I said like, "Can you find me clothes with this texture?" Like it would find this. So, it it is actually that I'm maybe a lot more agent built today that I actually gave it.

31:05 >> The reason Instinct is very Instinct is waiting for my credit card is I'm still using my old agent with the WhatsApp with the human behind it >> >> who was doing this booking for me, right? But if I did not have that option which a lot of people don't, then I think by now I would have easily given Instinct my credit card. And I look, people are giving credit cards in a big way in US.

31:26 you know, Amazon blocked Mews as you know, right? Because so many people were directing Mews to do their shopping on Amazon. Instinct from what I hear is they integrated with Shopify and the day they integrated generated something like 3-400 million dollars of additional business on Shopify. So, I think people are ready to kind of delegate >> Shopify and they have come up with this A2CP protocol which is like agents allowing agents to check out. Yeah. Yeah, because it's exactly that because now suddenly all the human friction is gone and this thing now I think the the world will eventually have two kind of protocols or two kind of things.

32:03 Things which are for humans and the underlying lot more high speed, high efficiency systems which are for agents of those humans. >> Yeah. And there is also I think the big technological shift that's probably underway is if my agent can use all these apps on my behalf and if I can talk to my agent on WhatsApp which is in some ways the easiest possible interface, then who will use apps? And that's why Amazon is scared, you know, they Amazon also today when you and I go on Amazon and we spend time there, They're able to show us advertisement and they make huge amount of, you know, income from those ads.

32:43 >> Mhm. >> But, if you we are not going, our agents are going, no point showing, you know, agent that. Agent doesn't care what ad you are showing or, you know, what jingle you're playing. >> I agree agent agent will not see the ads the way today is. I do feel that there is going to be because agent is also looking. >> Yeah. >> Right? Agent has knowledge which was cut off at a particular time and it is kind of looking for the best thing possible.

33:07 Many times people say, right? Like you said, find me if you had not said Four Seasons, which was branded, if you had said find me the best five-star hotel, for the lowest price in this neighborhood. Now, suddenly, the agent has to do discovery. >> Yeah. >> There will be agent, just like search engine optimization, there'll be agent optimization, which all of these providers will do. >> Yeah. >> And I don't know if the ads will be the right mechanism, but the dollars will flow in for increasing agent conversion.

33:34 Whatever the mechanism is. The other side, I think also can happen is you have a agent which went and booked Four Seasons. If the system knows, whoever on the other side, knows that it was your agent and you go somewhere else. >> Mhm. >> Now, you might see a correlated ad. >> Yeah. >> Right? So, there is that problem also where this activity and this activity can get correlated. >> And the third thing will also, I think, inevitable. I'm sure and, you know, look, Meta is already master ad business.

34:01 >> Mhm. >> So, business will be able to advertise for Muse to give preference to certain brands and products, etc. because if I am looking to just buy a black t-shirt, I may not care as much. >> Mhm. >> And whichever brand happens to wear, you know, paying to Muse in some way is the black shirt I end up with. >> Yeah. In that case, I think, it's a business model thing because somebody else might say that Muse is biased as a result and I will give you something which is less and it all depends >> Isn't Google biased?

34:29 >> not in its search results, but ads are paid and there's a clear delineation. >> Right, Piyush. So, I think just I'm also thinking, you know, as people become more and more dependent on these agents, I'm sure similar to ChatGPT, I think your trust has built over a period of time. Now people share pretty much everything including, you know, most intimate details of their life. That's probably what's going to happen with the personal agents as well. So as people start to share their UPI IDs and credit cards and so on, and the agents will learn so much about who they are, what they shop, what they consume.

35:03 What are the What are the safeguards? Who needs to build the safeguards? Do you think these companies will be required to build the safeguard, you know, since when you were I'm sure you dealt with similar issues at Google Pay as well. Cuz, you know, once people are using a UPI ID, you know, technically know anybody within Google can have access to that. Just putting the same hat on, if you are at Instinct, I'm using it like how do you think, you know, they need to or how they must be thinking about putting the safeguards?

35:29 >> Yeah, so there are various layers at which different kind of safeguards have to be there because you cannot rely on just some core pieces. But to take that example, I think if I'm building a personal agent today and offering it to the world, whether I'm in Muse or I'm in Dots or anywhere else, there'll be parts which will be less protected in the sense that these are servers which are running, but the minute you start dealing with a user's information, there are various safeguards and industry is aware of handling those from other prior products. Like let's take your email or your chat history with people and all that. People know systems there are there are proven patterns to say safeguard those. So encryption at rest or those mechanisms.

36:14 What has happened is that you take those and now the new world which opens up is my logged in experience into, you know, Zomato, let's say, or my bank account or or things like that. That is all happening on a VM. It's as if my laptop has moved into the cloud and my agent is using that using my credentials. That pattern is not there, but there people are building things. Like for example, almost all of the passwords which are there, they're not visible to your agent, they are handled through some sort of password manager.

36:47 one example I'll give you there's this password provider called one pass. So, if you have your password in one pass, you just share over and your agent gets a key key ticket I will get my passwords from this password manager and it fills it but it cannot see it. So, it has the ability to fill but not see. So, these are some very rudimentary safeguards. There is even more sophisticated ones like Facebook I think is coming up with this thing called completely private VM that even it's completely encrypted meta people cannot actually get into it and see it.

37:17 Almost all of these companies also have safeguards to protect from human access. Like the example you said that one can go and look at it. It is so heavily audited by almost everywhere that any kind of unusual behavior or somebody looking at something they're not supposed to because there's no backing reason gets flagged rapidly. I mean, Google had let people go multiple times if such a thing happened. So, those things are there in these large companies and new techniques are being created.

37:48 >> You you saw the recent news in Open AI that an employee was caught putting a spy device in the conference room where executives would meet. >> Yes. >> People will always find a way. >> People find a way but you know what actually was even more fascinating in that was No, people it's a cat and mouse game. People will always find a way. In that what was most fascinating was and maybe it's a marketing thing about which Open AI is doing but Open AI said that they used models to figure out who had installed it. What this was and who had installed it because they found this piece of hardware and the debugging from there they used AI to do.

38:21 But imagine all of this is there. Like I said, people will find a way, right? So, now moving to the next layer, there are challenges which even the the infrastructure also the services providers will have to do. let's take simplest example. Payment systems are the first ones which come to mind because the real money moves. Now, payment systems from credit card companies to UPI network and things like that, they have to evolve to be able to do these kind of safeguards themselves. They cannot assume a human is on the other side.

38:53 They cannot assume that a sent OTP is only seen by a human. Right? So, these mechanisms So, how do you do that? So, those protocols also have to evolve. And then only those protocols have to be used by the agent. The other protocols have to be not be available. So, that is the next layer. And then eventually the final service providers like, you know, if you have an account, let's say for shopping. I have my account and maybe there's a agent's account which is there. This happens a lot like, okay, well, this person will shop on my behalf. But then I get to set the policies on that. Right? So, that is at the highest service layer. So, there are different layers at which these safeguards will be there. But eventually you are right, it's a cat and mouse game. Each layer will evolve at their own rate. It's already playing out at some level, right? There are so many now AI led security hacks where they're hacking into government accounts, into companies' accounts. There's a big, I think, attack in Japan recently where the Japanese government has declared a national security emergency because of the extent of the attack. You know, under such a regime, like how safe will be people's personal information? Or like practically speaking, what will be your advice now tomorrow when Muse launches? Like will you tell me, "Yeah, don't worry about it. You know, put your UPI order, you know, from wherever you want to do and make your life convenient." You'll caution me that, you know, "Be Let the buyer beware." Well, if specifically talking about you, yeah, I would say put your UPI ID in there because UPI ID does not generate a payment. It requests a payment.

40:20 And I feel like if you got notified that you should pay this much, you're not as gullible that you'll be like, "Oh, like So, for you specifically but it does not mean that everywhere they can be a whole lot of other things. Somebody can generate a voice, you know, how these models are so good now. Generate a voice model saying, "Hey, I need this money." And then put the UPI ID and say then you pay. There are a lot of those things people can string together. I agree that this is a problem. I have a different view of that cyber safety thing. I will actually ask you, do you remember we once went to MIT Media Lab, you and I, and we had talked about AI?

40:54 >> With Ramesh. >> Yeah, yeah, with with Ramesh. And I remember there was a question in the audience somebody had asked me, and this was when, like, 2023. >> Mhm. >> I was I used to work on Google Assistant that time. And they had said, if you had to start a startup today, >> Mhm. >> what would you do it in? >> Mhm. >> And I had said, in AI safety or in cyber safety. I feel that that is actually what the challenge is. On one side, these things are becoming very capable at being able to expose vulnerability.

41:18 On the other side, they're also equally capable of patching these things up. >> Mhm. >> So, if I I would say that let's look at it from both sides. Saying, yes, it is getting exposed, but if a good person exposes and patches it up, >> Mhm. >> then that's actually a net good. That the system became more robust, less, vulnerable. So, I think there will be a set of companies, there should be a set of companies, whose sole job is to say, I will find all your issues and I will give you all the patches, or I'll come in and I'll patch all of them up. And I'm always on top of the latest, greatest models. And I'm sure, like, you know, latest, greatest models, they have You have to be approved to be part of it. You can be a white hat person, be be part of it, and then you build such a business out, and that would be fine, too, which is using the same models in a different way.

42:03 >> All right, fair enough. Yeah, I guess this is we'll have to probably just, resign to the benevolence of Big Tech to take care of >> I think, the human spirit is is what I I look at. I believe there are more, white hat people than black hat people. So, >> But human spirit will be owned by Big Tech, you know. We'll be always walking around at the mercy of all these AI agents and >> I I I I don't know.

42:32 >> intelligence. I'm just saying that's a darker view. It's probably not true. I'm also generally very optimistic. I think, obviously, AI is an incredibly powerful technology, with I think, you know, scope for both immense good. >> Mhm. >> Like, if free, affordable health care becomes available to everybody, >> Mhm. >> or free, you know, world-class education, which we are attempting at For me, >> Mhm. >> those are the good use of AI. >> Mhm. >> But then all these other, you know, the big company, you know, big tech, you know, it's incredible, you know, how fast their profits are rising. And that's what are the valuation and all that, so.

43:03 >> Yeah, profit pools are definitely concentrating. I actually think of the two reasons why it was happening. One, I think will slowly go away and one will become harder. The distribution is so hard now. Everybody can produce software like this, but big tech already had distribution. Let's take example of News. The minute it integrates with WhatsApp, it sucks the oxygen out of the room for many other startups, right? So, now the distribution is what caused that. The other side was they had the capital to bear the cost of being able to provide those services, but the cost also coming down and there are startups which can take advantage of the dropping costs and be in a niche vertical and be able to produce a high-quality thing at a lower cost, but distribution, I don't know what the answer to that is.

43:47 >> Speaking of Indian entrepreneurs, do you think someone can and should try to build the News Instinct equivalent for India? It may look different, maybe do different things, but is there a do you think there's a opportunity or once these behemoths are in India, they're going to steamroll over everybody anyway? >> I think the answer to both of them is yes. but if >> It's like go build it, wait until >> >> Meta steamrolls over you.

44:12 >> Or or you dodge that steamroll because that any anybody who's successful, people come after them, right? So, so the thing is that it So, here's why I think. There are many things which are not done by agents in the western world. Almost all of these are doing things which everybody does, but there are many things which are not being done in the western world. And the simplest example is majority of the use cases of if you built a personal agent in India, has to be voice-based.

44:39 >> Yeah. >> You talk to it. Either you leave a voice note to it. This is a very strong pattern. People just talk a lot more than they type, which means that the multimodal model which is sitting on behind has to be really robust at understanding. And all these models are very good. They're very good at transcription, but are they very good at understanding the task-oriented speech? Like is there a a gap between general speech and what I'm when I'm asking you to do something? And what is that gap? And can I really find you the model to go in there and provide it a higher, you know, at a lower error rate?

45:12 That Now all of a sudden I have built something which Muse is not doing because it is doing this thing really well for India. That is one. The second thing is if you almost always while these agents are able to run for a long period of time on various things, the ability to judge their accuracy for that task requires you to really evaluate their behavior on the task, which means you have to have amazing, training data, eval data, and all that for that. We have to find India-specific use cases which are not there because Muse has not been trained on those or Instinct has not been trained on those. But what are those India-specific? That can actually be the niche which says like, "Yeah, sure, but look, this thing does that, right?" This agent handles all the things which are happening in your house. "Ghar ka agent." Right? Whatever that is.

45:59 >> Let's do a little bit of a crystal ball gazing. You know, this is now things are happening so fast. I feel funny even saying anything about next 4-5 years. >> But let's do 4-5 months. >> >> Yeah, maybe 4-5 months is a better horizon. It's What is the like possible terminal state of these agents? If you have to dream the ultimate version of this personal agent, how does it look like to you? >> So, by the way, physical AI is starting to like, you know, starting to appear in places.

46:30 my favorite I will give you which I use every almost every day when I'm in Silicon Valley, which you used to be me working, now the machine is. but to to answer your question, in the terminal state, I think and assuming that's the the the most likely the the positive outcome in the world, right? Not the thing which is at 10% of chance humanity is obliterated, but like the positive outcome in the world. In the most positive state, the things you like to do you are doing.

46:59 Things you don't like to do are being done for you. >> Yeah. >> Is how it will separate out. So it's same as like people go on hikes today. In the olden days, people had to go village to village. They had no other choice but by being on foot. But people are still on foot when they want. That doesn't have to be the only way they go. So I think that the world will separate that way in in my books.

47:21 That everything which is and definitely it's happening even in the physical. I'll give you a small physical example. I I have a test Tesla coming back to Elon Musk. It has a self-driving mode. >> Yeah. >> It gives you stats for how much you self-drive. >> Mhm. >> Can you guess how much my car's self-driving percentage is? >> 90? >> 97. >> Oh. >> 97% and and America is a driver's country. >> What do you do?

47:45 >> So here is a very simple change in my behavior. Now I don't even put my shoes on before I get in the car. Because I don't have to use the gas pedal and all that. I take my shoes with me. I get in the car, I put it on and then before I'm getting out, I put my shoes on. Right? Just a subtle thing. However, the way I look at it is that I have now lost any apprehension of going any distance.

48:05 >> Mhm. >> Because I was going to spend that time anyway. I just happened to be there. So I have to go to San Francisco city, it takes an hour. I'm like more likely to say yes now. So but 97% of time of my commute is or commute or like road trips and all that is automatic. But this is physical AI. >> Yeah. >> When the general purpose physical AI comes in, like you know, your dishwasher and your robot like all of those things, what will happen is you will choose not to do things and then it'll do for you. By the way, here is the physical AI which I feel will help. There are bets being made that a robotic surgeon will be better than a human surgeon.

48:44 Less error rate. >> I have no doubt about it. >> Right? And and with this Tesla thing, they have published that how and they also published that how a robotic car is six times safer than a human driving it. >> Do you think 5 years from now I can invite a human AI to sit here in your chair and have a podcast episode with me. >> Yes, absolutely. I I I think you will absolutely be able not not Push Ranjan's replica, but you'll be able to invite somebody with lot more knowledge.

49:12 >> lot better. >> lot better looking, lot better knowledge, you know, and with lot better answers. So, yeah, absolutely. I I would not be surprised. Actually, I I I plan to and maybe we should consider sometime. I plan to go for like a visit to China to multiple of these factors to see what the state of art of robotics is. They they are the leaders right now in AI. >> we totally should. In fact, I think at some point, you know, I should do a whole episode on China because seems like the future, you know, if you want to see what the future looks like, go to China. They I mean, they are literally flying at this moment and leaving looks like AI, they have clear lead on Silicon Valley.

49:47 And we don't even know what their AI can do because a lot of data is not available outside China. We don't know, but I mean, they are making like tremendous progress. >> Yeah, yeah, they they they have definitely, as a matter of fact, a lot of people in US are very happy because China has pushed the the democratization of model access in some way. >> Right. >> Right? this is an opportunity, by the way, because their models are open.

50:10 You can actually take their models. You can you can of course distill any model, but others will stop you, but you can actually distill them down. You can you can fine-tune them in supervised manner with specific purposes. You can run them for a lot cheaper. And you can get a lot out of it compared to spending a lot of money on other large models. >> We are truly living in a crazy world, you know, communists are doing open-source models, democratizing the word you used for, you know, China's way of approaching AI.

50:38 While the Silicon Valley, you know, Big Tech is, you know, like completely proprietary. They're monitoring everything, you know. They're not even They're not even publishing anything. Your research is also very proprietary at this point, you know, so it's a >> Hey, but credit where credit is due. Since we started the podcast by bashing Google a little bit, if you look at it, it all started with Google publishing the way it actually handled large language or the natural language process.

51:03 >> And of course the transform model >> Yeah, that's what the 2017 paper was published by Google. I recently was giving a talk to somebody, and I noticed how many breakthrough research papers were actually which came out of Google DeepMind. >> Yeah. >> And the if you look at the top researchers working in almost all of these frontier labs, they all came from Google. >> Yeah. >> So there is something even though Google Assistant and such like >> I mean just I mean that's other whole area to deep dive into. The impact of AI and research. Just yesterday I think Gemini Argon they solved some 60-year unsolved problem >> Mhm.

51:37 >> in the plasma physics, which is directly related to fusion. So I did an episode on nuclear fusion. And this was, you know, someone shared that with me like this is one of the one of the problems related to fusion in the plasma physics. And there was a conjecture which, you know, now Argon has proven it to wrong. So, you know, they are making scientific contribution also on daily basis. >> Yeah, I mean like the pancreatic cancer, no, some pancreatic cancer I think they were able to come up with a vaccine to reduce the mortality rate or treat cancer which was just not possible before AI.

52:13 >> We are already starting to talk about, you know, start a second episode. So this is a good time to pause, otherwise my personal agent will be very unhappy and have running behind schedule. So thanks for your share. It's always fun talking to you. >> Thank you. Always good to be here. Thank you. Bye. >> Cool. >>

Summary

The discussion revolves around the evolution and potential of personal agents, particularly in the context of AI advancements. The speakers reflect on their experiences with personal agents like Muse and Instinct, emphasizing their capabilities to perform complex tasks and manage user interactions seamlessly. They also explore the implications of these technologies on daily life, privacy, and the future of human-agent interactions.

- Personal agents like Muse and Instinct can autonomously perform tasks such as booking travel or ordering food through messaging interfaces.
- The evolution of personal agents includes features like memory, proactivity, and the ability to execute multi-step tasks, enhancing their utility beyond simple question-answering.
- The integration of AI with everyday applications is transforming how users interact with technology, making it more intuitive and efficient.
- There are significant opportunities for Indian entrepreneurs to develop localized versions of personal agents that cater to specific cultural and linguistic needs.
- The conversation highlights the importance of safeguards and ethical considerations as personal agents handle sensitive user information and financial transactions.
- Future developments may lead to agents that can coordinate tasks across multiple users, enhancing collaboration in both personal and professional settings.
- The role of AI in society is expected to grow, with personal agents potentially taking over mundane tasks, allowing humans to focus on what they enjoy.
- The discussion also touches on the competitive landscape of AI development, particularly between Western companies and advancements in China.

Questions Answered

What is a personal agent and how does it function?

A personal agent is designed to perform tasks on behalf of the user, such as making bookings or providing information, with minimal user intervention. The speaker discusses their experience with Google Assistant and their own personal agent, emphasizing the agent's capability to handle complex tasks autonomously.

How are personal agents utilized in practical scenarios?

The speaker describes using personal agents like Muse for complex tasks, particularly for navigating non-intuitive government websites. They highlight the efficiency of Muse in booking processes that are typically cumbersome.

Can personal agents facilitate group coordination?

Currently, personal agents like Muse and Instinct do not support multiplayer coordination for group activities. The speaker discusses the potential for agents to manage group tasks, such as coordinating trips, and shares insights on their company's approach to developing such features.

How will personal agents impact e-commerce?

The integration of personal agents with platforms like Shopify is transforming e-commerce by reducing human friction in transactions. The speaker notes that as agents handle shopping on behalf of users, traditional advertising models may become less effective.

What are the implications of AI and personal agents on society?

The speaker expresses a balanced view on AI, acknowledging its potential for good, such as improving access to healthcare and education, while also recognizing the risks associated with Big Tech's control over these technologies. They emphasize the need for ethical considerations in AI development.

© transcribe · For agents Built with care and craft by Gokul Rajaram