User Embeddings: How TikTok Knows You Better Than You Do

0:00 How can the Tik Tok recommendation system know me so well? it's the user embedding that's actually generating your for you feed. So, I'd like to talk a little bit about how we create the user embedding and what exactly analyzing the user embedding. What sort of information you can find out the people in the same cluster look the same. If I could access that, that would be unbelievable. I could use it for dating. I could use it for finding friends. I could use it for traveling.

0:24 It's very powerful, right? And it's very cool. the main reason I came to India is because I was very interested in the India stack. Adar, UPI and some of the digital identity things that are happening and I think that's really exciting cuz in the US we still are using credit cards and debit cards and driver's license and car keys and it's like I just want to leave my house with a phone. and in even in the US even in even in SF like that's very difficult to do. So I thought India was very interesting. Also when I was in China the next billion users was like a really popular idea idea. I started at bite dance in 2018 and that's when a lot of bite dance tenscent Alibaba all these tech companies were coming to India to kind of replicate what they had done in China in in India. So I started out in China in 2011 right when we chat started and got to see basically the entire renaissance of China and I thought you know a lot of people are asking is India next and if so like will they be able to replicate that sort of success and I thought that was like a super interesting thing so that's why I came to India to basically learn more and see what's happening. So today I think it'll be like a very interesting discussion.

1:36 Feel free to stop me at any time if you have any questions. We can make it an open dialogue and I'm really curious to hear what you all think. and we can go ahead and begin. So today I'm going to talk about the user embedding. The whole topic is about what we call the user embedding. I'm sure a lot of you are very familiar already with embeddings whether they're text embeddings or video embeddings or audio embeddings. but I think the user embedding is actually the most interesting and I think it's probably the least research of any of the different types of embeddings because the data to access it is very difficult to obtain and only a few companies actually have this and I think Tik Tok's data and Tik Tok's user embedding is actually the most interesting. So I'll talk a little bit about the user embedding. Then I'll talk about what I call embedding as a service, whether you can directly access your user embedding and use it in other scenarios to make your life better. And then I'll talk a little bit about how we can pair a user embedding with an agent.

2:32 because a lot of people are talking about memory, they're talking about context. I think you I think you can call it different things. But if you can take your user embedding and pair that with an agent or multiple agents that can then go out and do things for you, then you have this automatic personalization. which I think is really going to transform a lot of the future. So, also, if there's time, I might talk a little bit about like what I've been working on and what I've been thinking and kind of like my vision and in terms of like how to unite these two together. So, let's go ahead and begin. real quick, I just wanted to play this video. hopefully the people online it doesn't break their ears. I'm just going to turn it up so everyone can hear. But I just want to play the first 10 seconds of this video because I think it's very interesting. and it gets to the main point of what I want to discuss today. So let's go.

3:19 >> Okay. Has anyone else noticed that you were for you page has been a little too accurate lately. It hasn't been th Google or I talk about it's been thoughts. Tik Tok knows everything about us. >> Don't scroll. Let me ask some first thing how this algorithm works. Direct users often wonder how the world's fastest growing social network seems to know them so well. It is secretly listening to us while we're watching videos. I don't know.

3:43 >> So the whole idea there is, you know, how can the Tik Tok recommendation system know me so well? A lot of people even asked the question, how can Tik Tok read my mind? And if you believe that the algorithm is actually good enough to read your mind, then you have to believe in the user embedding because it's the user embedding that's actually generating your for you feed. So I'd like to talk a little bit about how we create the user embedding and what exactly analyzing the user embedding what sort of information you can find out. So the idea you know how can Tik Tok know what you're thinking? and it raises this interesting question of are your thoughts really unique and are your thoughts actually your own? because as we get increasingly more into AI and big data, we have to start maybe rethinking what it means to be human and what it means to be unique and what it means to be able to collaborate to solve human coordination problems cuz in the future it's all going to be about resource allocation and how we can most optimally arrange these different types of resources. And if you have access to the user embedding and you can use that information then you can you can basically make resource allocation much more efficient because at the end of the day that's all what Tik Tok does. It's just a way to most efficiently allocate supply and demand. So I'm just going to talk a little bit real quickly like a brief intro into recommendations and it's going to be very simple but this is going to give you a better idea of like what I mean by the user embedding and where it comes from. So, in any recommendation problem, the question is, you know, I have a billion items and how can I find just the right one that I'm looking for? And in this case, it's going to be the in in this case, the item is a video. It could also be, you know, like an e-commerce product or food or a dating someone that you might want to date, for example. And a lot of people they always think, you know, like, well, there must be I must be a row in a database and there must be all of these words, these semantic words that describe what my interests are. For example, maybe I'm into finance, cooking, Elon Musk, sports, and travel.

5:47 and if every single video, you know, had these semantic labels. for example, this one here could be Elon Musk. it's a video about the Tesla house. perhaps if you had this sort of information, it'd be really easy to do some sort of semantic transformation to do the mapping between users and videos. But you have to remember most of these most of the content on a a video doesn't really have a lot of information. and you can't it's not actually very usable. It might have hashtags, but often the hashtags have nothing to do what's what with what's being spoken about in the video. and so of course this is not how we do it.

6:24 instead of using these semantic representations, we're going to actually use embeddings, right? So, every user is going to have a user embedding. Every video is going to have a video embedding and a simple dot product between the two will give you a calculation of how likely this user is to be interested in this video. And so, how do we get the user and item embeddings? I'm first going to go through a very simple recommendation flow to give you an idea of, you know, what the recommendation system looks like. And this isn't necessarily specific to Tik Tok. It could be any recommendation system. And this one that I'm going to show you here is actually very similar to like what Google presented I think in 2016 with their recommendation system. So I'm sure you're going to be very familiar with it. But every day users are uploading on the order of 100 million videos a day.

7:10 And those are just going to be sorted in the video database. And we're going to add in some filters. these filters like once a video is uploaded we have to make sure that it's safe for the community, adheres to our community guidelines. and we might add in some different types of algo strategies that will filter that corpus down a little bit. then we're going to go into the recommendation pool and this is going to be on the order of you know 30 to 50 million videos tens of millions of videos and we're going to go into what we call like the cascade part of the recommendation system. So the very first stage we call it recall. So what we want to do is like given tens of millions of videos, how can we retrieve just a few thousand videos say 2,000 or 3,000 videos. then once we get those thousands of videos, how do we rank those videos down to a few hundred videos, say 200 videos to 300 videos? We call that pre-rank. And then finally, the final stage is going to going to be the ranking stage. And we're going to go from a few hundred videos down to just eight videos. And those eight videos are then going to go into a for you page.

8:15 So, if you think of Google, like Google search page, when you go and you type into Google and you push enter, you're going to get like those links, right? And then you scroll all the way down and you can go to like multiple pages. Just imagine this is just the first page with eight links. And those eight links are scored. and that scoring process is going to be in that pre-rank and rank stage. So, your feed is just essentially made up of batches of eight videos into infinity, 88 88 88 888 into infinity. And the model that I'm going to be talking about today is the recall model. So it's just going to be at this stage right here. And in all of these stages, we're talking about maybe tens of models. There could be tens of models here. There could be tens of models here. There could be tens of models here. Altogether, there could be a few hundred models and they all cascade together to generate that final output there. but today I'll just talk about one specific model.

9:13 and explain to you like how it generates the embeddings and then once you analyze those embeddings what happens. So so if you're interested in this particular model it's just a two tower model. This is probably the most basic model that exists in terms of like neural networks. and this is just a public paper. So everything that I'm presenting today is public information. And if you're interested in this particular model here's the paper. It's called NCF neural collaborative filtering. And it basically you know a two tower model is where you have say a user tower and a item tower. The user tower itself is a neural network. the item tower is also a neural network. And then they're going to combine into a joint space to make predictions on whether or not you'll and in in the case of Tik Tok for example it would be will I finish a video, will I share a video? will I comment on a video etc. so let's get into this model a little bit. So the user tower, this is the first part of the tower. what we're going to do is we're going to take some example user attributes. This could be things like the device settings. Are you on Samsung or iPhone? etc. Language preference.

10:22 Are you speaking Spanish or Hindi or English? Your location, the device type, etc. And we're going to one hot encode these feature vectors. And then we're going to basically they're just going to be transformed into this UD tower here where we start out at a base layer say 512 and then 128 64. in this case it's just a super simple three layer neural network. And we're going to have the exact same thing on the item side. it's just going to be like 5121 128 64. On the item we're going to have things like sounds, hashtags, number of views, countries of country of origin, etc.

11:01 And the very interesting thing which is going to pop up in a bit is in this in this user area here we don't have things like the user's profile picture or information about like who they are. It's just their actions. It's all action based and this will become clearer and very interesting in a few more slides. Similarly with the with the item here we're not creating like computer vision embeddings of the frames of the video. We're not taking the audio and compressing it and transforming it into audio embeddings. We don't create any derivative embeddings from the video itself. It's solely just attributes of the video and which users act on the video and how they act on the video.

11:45 and again like this is going to be very interesting in a few more slides to understand. So just keep that in the back of your mind that it doesn't actually. Yeah. Go ahead. >> Yeah. equations. It's just ID. It's ID. These are all IDs. So, every single hashtag will have an ID associated with it and then it'll have a lookup table and then we can actually pull that ID out which represents that hashtag and it will have an embedding. Does that help answer your question? Yeah.

12:19 >> So, so yep, great question. So, just like the hashtag like a hashtag will have a hashtag ID. The song will have a song ID. So every song will have a unique ID associated to it and that will also it will also have an embedding as well and then you can think of it as all of these embeddings being conceded together into one super long future feature vector and then fed into the the neural the neural network.

12:44 But all of those features like they have embeddings and there those embeddings are also updated over time. >> Original audio. Yes. Like yep. Yeah. Like if you make a if you upload audio it you'll be the publisher like you'll be the author of the audio and that audio will have a specific audio ID track ID and then there can be an embedding associated with it. >> Yeah. >> So the number of views >> it's a little hard for me to hear. Are you saying like how does the number of views impact the embedding of the video?

13:23 the number of views, right? >> Yeah. Yeah. >> Yeah. Yeah. So, the embedding it will change because every view will be an action. So, even if you watch the video and don't do anything, right? We have the amount of time you stayed on the video and we have whether or not you finish the video. Those are going to be the two most important most important features is the stay time and whether or not you finish the video. So do wipe one not encoded by coalate embedding of hashtags.

13:56 >> Yum. >> Yeah. Yeah. Yeah. And just embed it. >> Exactly. So so we actually try like cuz we can say the ask the same question for like the video frames, right? Like why not make video embeddings? And what we found is that collaborative filtering actually does a much better job at explaining the content of the video if because those sparse vectors are going to be compressed into like compact dense vectors in this process right here through collaborative filtering. And I think like in the future once we get to like generative recommendations which isn't based on collaborative filtering that will actually become like a a much like more interesting to just use the exact semantic information like you mean like take the hashtag semantic features and encode that as an embedding and then concat it all together. Right. We we did try that but it just didn't work as well. but I think in the future with more generative models where you're not using a cascade approach in like one hot encoding like that that might be like a it'll probably turn out to work better but at least in the architecture that we had at the time like it just made more sense to do it this way.

15:02 >> Yes. >> Yeah, good question. So let me just go back here. so this is like basically how all recommendation systems work now. but the question becomes like do you really need these three stages? if you have my sequence because those eight videos that come out at the end, you could think of them as a sequence of videos. And what you can do in and what's happening right here is like in the rank stage, it's like you're ranking every single item. Rank one, rank two, rank three, rank four. Whereas if you do sequence to sequence, you can generate the most optimal sequence given all of your past sequence data. So if you said like hey like here is my here's this user sequence this sequence of eight videos the user watched all of them and they like three videos >> too fast you just kick out the specific ones directly >> exactly that'd be generated so you're just going to generate a batch of eight videos for example >> it's 10 definitions as the user is it's probably about 10 minutes it's not it's not real time yeah we have training we have like different training windows 5 minutes, 1 hour, 1 day to update the user embedding and the item embeddings, but as I'm going like it's not exactly quite in real time.

16:19 There's still some latency involved because it's it's actually like quite massive and to do it like on demand in real time is complex. >> Edit paper. Doesn't Tik Tok mean you session personalization? >> Yes, real time session. that it may not be like in like if I watch this first video for example if I watch those eight videos right there it's not necessarily the case that once I send once I send a request to get the next batch that that will be in real time based on those exact signals that I just did but session to session for sure but one session could be an hour long this might not be the case that what you did on the exact previous video will be reflected in the it needs some time exactly you need some time to update it so again like how to sift through billions of items to find just the right one. And again, this is what the user tower and the item tower are going to look like. and let's go again. So, the thing about this is like at least initially these two towers, they're not connected to each other. They're in separate spaces. So, you have to think like how do we put them together into the same space? Cuz if you take a dot productduct between these two vectors, it doesn't mean anything because they're in different spaces. So, now we need to join these spaces together into one space. and here's how we do that. So at the first stage you know you could say like this dotproduct or even cosine similarity you know the angle between two vectors that could be a proxy for the likelihood of whether or not this user will like this item. So initially we take a dot productduct but in the beginning it doesn't mean anything cuz the two spaces are in separate spaces and what we're going to do is we're just going to join the space into one space.

17:54 So we're going to have 64 from the user side, 64 from the item side, one dimension for the dot product, and that dot product is going to tune the user embedding and item embedding into the same space, and we're going to get 129 dimensions. And then we're just going to compress that down. and finally, we're going to get our prediction. and then it's just like a typical activation function. The activation function is just going to compress the the dimensionality down into a probability function and give you a value between zero and one. And that value is going to represent whether or not the user will like a video or share a video or finish a video or comment on a video. in this in this in this situation, I'm just saying it's like, but there's you know like 10 to 20 other actions that a user could take on and all those we'll have predictions for all of them. And so so then what we want to do is we just want to we'll have the loss function in gradient descent and originally we're going to make a prediction like will the user like the video yes or no and then if we think like yes they'll like the video and we show them the video then they watch the video and then we know did they like the video or not and that's going to be our ground truth label which is the y there and then with that we can just do we can have a loss function and then we can do gradient descent And we're going to just update all of the layers to make the loss come down to zero. Is thisation being done for,000 videos?

19:28 So, this is going to be in batches. Maybe like batches of like 10,000 videos updated at once. It's not going to be like one user one you or one. It's not it's not individual for every single video but and large batches >> are contrasted wherein you can just directly like get in spaces together when the user likes it then apart when it doesn't like that's that's the that is what's going to happen once it gets into the two towers because the loss information will be back propagated all the way down. So this dotproduct is going to move these two either further apart from each other or closer together.

20:07 >> Yeah. And I was just saying you probably will be because cat and like the directly the loss function on the dot part. >> Oh sure. Yeah. Yeah. Definitely. yeah I'm just trying to explain like you could for sure do that. For sure. It it it seems like it's repeated information. That's what you're saying. Yeah. Yeah. Right. Not all of these visual band recorded >> y >> with exactly exactly exactly this is just at least how we did it. It was It's kind of like two layers, right? It's like two parts. Cuff >> clusters and one to use the clusters. The clusters whats >> Yes. You just predicted like the third slide that's coming next. That is actually the entire point of this talk is what you just said.

20:57 >> Yep. >> So when you say that 0. >> Yep. Yep. We add key actions as we make this actions more paper with them. >> I will talk about that in the next one. How to combine the different actions together? Yep. So this is your question right here. we're basically going to have every head will be a different action, right? So like, share, comment, follow, finish, stay time, dislike. These are all separate heads that are all predicted. And what we need to do is then combine them together into a final score to get the final score.

21:28 And all of the different heads aka actions will have different weights. So like maybe is 3.2, share 2.1, comment 0.9, etc. the way that those values are are most optimally figured out is like like why should share be 2.1 and like be 1.8? It's just Amy tested and it's dynamically updated with we can basically just like test multiple values at once and then see what the most optimal values are. Also, we'll have a personalized value tree for every user because some people they never click like but they are always commenting right. So y will be shared. So for key at least when I was there it we didn't have the personalized value tree yet.

22:23 so all of the numbers were static at the time for every what I mean by static is that they could update but they would be the same for all users. and what we were working on is like how do you do like ondemand on the-fly dynamic values for every single user that could be different? like my share value might be different than your share share value. but I had already left and we hadn't rolled that out yet. All right.

22:49 So now, so now like I think you should have a better idea of like what we mean by user embedding and item embedding. what what what I analyzed at least was this embedding here and this embedding here just the final layer in a two tower finish model whether or not a user will finish a video. So now like back to your question about the clustering what we can do is we can take out this user embedding here and just use K means clustering to cluster the user embeddings and see what we find. Similarly, you can cluster the item embeddings as well. And so when you cluster the user embeddings, this is this is what you get. And I can't show like a real picture, but I just made up I tried to think of one that would hopefully kind of make sense.

23:35 so what is this like this is after tsn. So that final embedding is in 64 dimensions. we need to project it down into two dimensions, an x and a y. And then we can plot it on this graph right here. And what you'll find is that I just put gender on the x-axis and age on the y-axis because those are like if you looked at like the the ien values of a human, you're going to get things like what's their age, what's their language, what's their gender, what's their sexuality, what's their race. Those are going to be like the core components of a human. And then after that, you're going to get like their interests. So like so that's like why I chose gender and age on this graph and then you'll see like it's kind of like separate separated here. What I tried to show is like mainstream this is for the US so it' be different than like culture in India but in the middle are going to be like the super mainstream people in the US like we have this idea of arch archetypes. So like in high school like the really popular guy is like the high school jock like the high school quarterback and then the really popular girl is like the the cosmopolitan should be here. I didn't have anything on the fringe but it would be like the cosmopolitan girl right there and then they would go to homecoming together and be like the homecoming king and homecom I mean queen and this is exactly what we see. So the most mainstream people will be like right in the middle for both male and female. Now, the most interesting part about all of this is if you take a cluster and randomly sample profile pictures from the cluster and put all of the profile pictures into a giant thing of profile pictures, the people in the same cluster look the same. Okay? So, what I mean by that is facial geometry, hairstyles, the style sunglasses they wear, their clothes, everything about them is is the same because they're a mimemetic copy of each other. And once I saw that, I was like, "Wow, that is by far the most interesting thing that I've ever seen in my life." and the next question becomes like, "What if I could access my user embedding, right? cuz I got to do a nearest neighbor look up on all the people all around the world most similar to me and see my own cluster and like once you see that it's like wow if I could access that that would be unbelievable I could use it for dating I could use it for finding friends I could use it for traveling I could find it for e-commerce I could use it to make my life better and because you know you have this age here you could say like here's a 16year-old what are they going to be like in 40 years and find that exact person like in 24 years time like this is who you're going to turn into, you know, and it's like that's what I mean it the we've never had so much data like this to be able to analyze before and now I think it's going to open a new opportunity to better understand who we are both individually but also collectively. so one thing that's really cool is like let's say I take this cluster here and I get the cluster center embedding which means I average together all the embeddings in the cluster and I do a nearest neighbor search over here. What you're going to find is like this cluster is going to be this female cluster and then if you sample the profile pictures and put them together they look like husband and wife. So like these people would for sure maybe I'm like I'm a little biased probably in the way that I see things but I was just like wow like they look like a great couple you know and they probably have like so there's there it opens up this question of like are these your digital genes right is the recommendation system literally creating your digital genes and if you can access that and understand that and understand how it transforms and changes and evolves over time then you can understand yourself and you can understand society at a level that like I don't know how many other people have really I I don't know it's very powerful right and it's very cool so so yeah like I think that being able to understand the embeddings from this like gives you a literal partitioning of society and that is important because if we can use that and leverage that then we can just more more adequately allocate resources and make like resource allocation in an economy much more efficient which at the end of the day is like what you want to do if you want to grow GDP and you want people to have like great lives, right? So so then you know I thought a lot of time I I thought a lot of times thinking like how is it that how is it that like what you walk is like a very good indication of what you look like why do the people actually look the same in the same cluster and it actually like led me to thermodynamics and this I'm actually 100% like convinced of now so if you're interested like there's this paper life as a manifestation of the second law of thermodynamics. So the first law of thermodynamics, energy can't be created or destroyed. It can only change form.

28:43 Second law is entropy. Entropy is always increasing. So how does this impact your life? Well, the problem is is that all organisms are trying to maintain homeostasis. But because of entropy, the environment's constantly changing, which means that there's all you you yourself always have this feeling of loss, right? Because as soon as you achieve something, you have, you know, a dopamine hit and then it fades and then you're like, "Oh my god, what am I going to do next, right?" And then you you have this loss. And the loss is because you're out of equilibrium again because the environment's changed. And the environment is constantly changing is just constantly changing. And I think this is like an existentialist like philos philosophical literature. We talk about this idea of angst. We have this angst. Like human existence has this angst to it. And I think that is the angst is that we're constantly having to go up and down a mountain. and it's never ending. And so I think like it's all just thermodynamics. And so if you think about like your if you think about the brain, like what is the brain? Well, the brain is just actually a prediction machine. All your brain does is try to make predictions about the future in order to get you to homeostasis. Because if you don't get to homeostasis, you're going to die. If you look back at say like the big bang theory is like a real thing, right? And in the very beginning of the universe, in the very beginning of the universe, the only elements that survived over time were the ones that maintained homeostasis.

30:10 Because once you lose homeostasis, you just dissolve. So it's this constant fight to maintain homeostasis. But the only way that you can maintain homeostasis is to learn how to make predictions and to create a model about the world that works. Because if your model about the word world does not work, you're going to die. So, so, so this is another thing that I've been convinced about is that my brain is just constantly making predictions about the future and all of consciousness itself.

30:38 Like imagine this. Imagine if you're learning a new language or you're a child and someone holds up an apple and they're like, "What is this?" And the kid's like, "It's a pear or it's an orange." because maybe he didn't really see it correctly and you're like, "No, it's an apple." what's happening is is like your brain is saying I I was wrong and when you're wrong that is that's when you feel emotion like where does emotion come from and why do we have it? It's to correct our brain and to do back propagation on our brain to make it update. so all of emotion and even consciousness itself is most likely just a manifestation of the fact that we're out of equilibrium with homeostasis and there's something wrong with our environment or there's something wrong with our model. We need to update it.

31:23 And so I came up with this idea that I call the mimemetic survival strategy. And the first kind of idea is like you know this is the myth of Cisphus. And the myth is Sisphus. Seephus has to roll a rock up a mountain his entire life. And every time the rock reached the top it rolls back down. This goes up and down and up and down forever. And at the end of the story the author says one must imagine Seisphus to be happy. And I was always like why is Sisphus happy?

31:52 but now I believe he's happy because he just got a dopamine hit in his brain. That's why he's happy. It's because his brain is fresh with dopamine. And what's he going to do? He's going to do the exact same thing again because it's self-reinforcement learning. And then I chose this path because I believe like if you have a mountain, there might be a hundred different ways up the mountain. We call it like hill climbing. So when you got to climb a hill, you're going to try and think like what's the fastest way up the hill, right? and there could be a hundred different paths.

32:23 and if you had a thousand people say all try to climb this mountain, you'd be able to figure out like what were the most optimal paths. Maybe there's a slow path that's safe and a fast path that's dangerous. but why do people start to look the same, right? Is because they adopt the same path in life to optimize their life and and to survive. And that's like why they're a mimedic copy. And that's why it's a mimemetic survival strategy is because you adopt the survival strategy that works for you.

32:53 It's it's like a niche partitioning of all of all things because you will go into one specific niche because that's that's like what you're really good at. And so the path to order amidst chaos offers a unifying principle for the emergence of these useful behaviors because at the end end of the day like what is the recommendation system is predicting your actions, right? And like one super interesting thing about semantic embeddings versus action embeddings is is the phrase you know you will know the meaning of a word by the company that it keeps right that's the idea of like learning surrounding words to generate contextual embeddings about words in natural language processing but actions speak louder than words. So actions at the end of the day it's like not what you think it's how you act. And that is exactly what this recommendation system is collecting and predicting is what are you going to do next? What's your next action going to be? And so finally like over time the optimal paths emerge and these become archetypes and we learn to imitate these paths in order to survive and that's why it's mimemetic survival strategy and that's why all the people look the same is because they have adopted the same survival strategy in order to survive in order to reduce uncertainty and entropy about their environment. So it's kind of like maybe like a sociological or societal interpretation of thermodynamics. And so then this becomes like super interesting. because if you have the mimemetic survival strategy like how could you apply this to dating for example. and I think Hinge is is like a very interesting kind of funny funny thing because so Hinge like the way that it works is right here the first thing is going to be like a Tinder style swipe like yes no yes no. And as you're swiping yes, no, yes, no, your user embedding is being created, right?

34:51 They're getting your user, your dating user embedding. And the the interesting thing about Hinge is that once they have your embedding, what they do is they have this section right here called the standouts. And they just once they have your embedding, they know your type, right? Then they take your type and they put it right here in the standout. But the only way that you can talk to the standouts is by sending them a rose. So they so so so they figure out your type and then they monetize your type because you'll never see them in this feed right here because you have to send them a rose. Now it's not like precisely like that but that's the idea, right? And it's actually like a very interesting idea from a monetization standpoint. But something that would be way cooler would be like hey here here's your embedding.

35:33 Like I can get my Tik Tok embedding. I can import it into an app and then I can use it to find people who are who are very similar to me for dating, for finding friends, for whatever. and you know this gets into the idea of like embedding of the service, right? Because if I can take my Tik Tok embedding, my Airbnb embedding, my Spotify embedding and aggregate them together to create a super embedding or what we call the universal user embedding, then it and then you can have like this idea of a universal login. You can have automatic personalization across any AI service.

36:06 And I think that's like ultimately what you would want because for me the biggest pain when I went back to the US after living in China was I had to download a hundred apps and every app I had to do the exact same thing over and over and over again, right? It's like I should just I I would way rather have WeChat obviously than have like you know Slack and iMessage and WhatsApp and but what whatever. I'm not going to talk about it. But one cool thing here is like I've been reviewed 57 times, okay, on Airbnb. Like people have reviewed me as a guest. And each one of those 57 times I probably paid at least $100, right? So can you imagine like from a credit perspective like how interesting this data could be if I could unlock that as a way to show that I'm like maybe like a good human being or whatever. like not necessarily a good human being but that is very valuable information but I can't access it right I can't show that anywhere but it is a credential that I have but it's locked up in Airbnb and a lot of people are like well why would Tik Tok want to give away the user embedding isn't that the isn't that like the core of what they do but the question I want to ask is like if Tik Tok is 100 billion or 200 billion and they keep their user embedding but what if there was a way that they unlocked the user embedding and they created a new derivative market that doesn't exist that was worth a trillion or 10 trillion dollars because now you have embedding as a service which can power and optimize services that don't currently exist, right? And then they just take a small transaction fee or they enable users to access the embedding for $5 a month or $10 a month because if I can access my embedding and use it somewhere else, I'm going to go back to the source to get my embedding, right? And if Instagram doesn't give me an embedding, why would I use Instagram if I can get my embedding from there and then reuse it in other places? So I think like that's also like a very kind of like exciting idea because we live in this world of identity fragmentation, right? there's no interoperability with my data anywhere. And it's like yes, we all know this, we all know this, but do you know like they all have your embedding and the embedding is actually like really cool, really amazing information that you that you could use.

38:22 And so it's like if you could create the universal user embedding for a universal login across all the services where you get you you generate you're generating data and then you're receiving data cuz there's like two parts I want to mention here like first is we live in this if you believe that like humans are like developing on this timeline where we've gone from say hunter gathering to slavery to feudalism to mercantalism socialism capitalism what like if you believe that there's this this trajectory like we were in feudalism and feudalism you basically have a plot of land that someone owns and we the peasants just we work on that field like this you know but you have to remember like every time that I push light I'm the peasant in the field doing the work right so so so we did evolve past that stage into capitalism which is where people individualistically should be able to do the work and get paid for it and another idea is like in India in ancient times they had the four like the the cast system right and the guru had all of the knowledge the brahinss had the knowledge only the brahinss had the knowledge and the most interesting thing about the veas in all of the all of the ancient books is that that was never written down that was passed down orally tens of maybe tens of thousands maybe even hundreds of thousands of years of oral tradition coming all the way down and if you didn't it because we couldn't read we couldn't access that information because we weren't a Brahman we couldn't receive that information but what happened is is like now we live in a world where everyone has access to the information which world is way more powerful right it's the world where everyone has access to information and so we will go into a new paradigm where everyone will be accessing their data and using their data to basically unlock the real creator economy because we can't have the real creator economy if all of the information that you create is owned by a company and you can't actually access it. It makes no sense. So, this kind of gets into this idea of like GDPR. Like GDPR is interesting. I know India is also working on their own kind of data privacy and like what they want to do.

40:31 But this is just like an interesting thing because it says like all of your data like you can go and download your data, right? But it doesn't really give you any good answers yet. and so with GDPR, like there's two interesting things. Like first is any information that I input into an app, I should be able to download that information. Like if I type in my address or my birthday or whatever, like I should be able to access that information. And then there's data portability, which means I should be able to easily port my data from one data controller to another data controller. But the problem is is like there's no protocol for this. Like no protocol exists right now. The reason why I'm mentioning this is because one reason I came to India is because I think the India stack is very interesting. and I think like India could be interestingly positioned in a place to build something like this before other countries because like the US is still super entrenched in this old kind of world view and like Europe has nothing there. It's just like a old museum. so like India might be a very interesting I I think like I think like this whole paradigm and all these ideas like could be interesting for India because it's like in that stage of developing this right now. So then like this is like the issue is like I know this is like really hard to understand but doesn't the user have the right to access cuz what I'm talking about is like I have the right to access my embedding. Can I access it? Well what can I access? What do I have the right to access under GDPR? I have the right to access you know anything that I've inputed like a like is an input a share is an input I manually did it I have the right to access it but for things that I don't have the right to access potentially is the model wasting the embeddings because it's derived data right it's like my data mixed with other people's data there's also proprietary ways to generate the embeddings so under like the current GDPR it's not necessarily true that I have the right to access my embedding however It's kind of a gray area because if you read this, I know you can't see this link, but there's a link here that kind of gets into the details and it says like if you request information about yourself that the company has, then they should be required to give it to you.

42:40 but they're not like required to by law. So, it's like it's still kind of in this gray area where I can say like, "Hey, like I know you have embeddings of me. I want I want the embedding." And then they might say like, "Oh, which embedding model are you talking about?" And then the person's like, "Well, I don't know." And they're like, "Well, what are you talking about then?" So, yeah, it's like it's kind of like yeah, like we're probably not going to get our embeddings, but I just wanted to mention that because it's very interesting. You can download your data.

43:04 I'm sure a lot of you are aware of this. I just downloaded my YouTube data and did some analysis on it, but I know we have limit limited time, so I'm I'm not going to get into it, but it's very cool. like you can just download your Google data and then embed all that information and then ask who am I and it'll tell you exactly who you are based on your you what you watch on YouTube for example. so then the user embedding times the agent and this is like this is like a long this is a a old presentation so now like these ideas it's just like duh right so this isn't going to be like that interesting but I'll just kind of speed through it and then we can get to questions and answers. So obviously like right now like question you can go to chat go wherever you have a question go to LM you get your answer right and the thing is this the most important thing I asked Chad like using everything you know about me make a painting representing who I am okay and this is a painting it's not very cool but the thing is is like in the future I'll have a 100,000 photos of me that I've taken right and all of those photos and everything that I've done is going to go into my embedding And then when you ask it who and lie, imagine in the future like what is it going to look like, right? Maybe it's not a static photo. Maybe it's like a like a a video or an animation or like a or audio. Like we don't really know what it's going to be. Maybe it'll be like a actual like a projection.

44:29 who knows? Like but that is like after I saw the embedding, it's like who am I based on all of my data and how could I use that data to make my life better? I think that's like the whole idea. So again, like I use MCP server just cuz everyone knows what MCP server is and it makes a lot of sense. But obviously like I should be able to have like my own consumer vector store, right? with all my information which is an MCP server which if you have the if every like AI service or every app supports an MCP client or like a protocol that everyone adopts then you can have like the automatic drop in context and have automatic personalization because I should just be able to log into Tik Tok with one click facial recognition I'm logged in and then it pops up and it asks me like what do you want to share with Tik Tok? Oh, I want to share this this this and this. Right? I push enter.

45:20 That's useful for Tik Tok because then they can use that information to give me better recommendations. It's better for me because I get instant personalization and of course this is the way that it should go. And the reason why I mentioned this in India is because I feel like India is already developing this stack with the India stack right and like I I think right now it's mostly focused in the financial sector like for credit lending is like kind of like the next thing but imagine if it's for your actual life. Imagine if it's for dating. Imagine if it's for e-commerce like this is where the future is going to go and so like yeah like yeah this is just like the idea is like it should just be all integrated I should have like my own profile like the AI profile right and then with my AI profile I can update it maybe it runs in the background updates for me automatically can log into all these different services and it's just like a one-stop shop for my identity that's automatic go automatically portable everywhere and then yeah like you know like then it's like well what would this look like? Well, I would just, you know, this is me. This is like my database. And what would be really cool is if this was all local and if I could actually have like my own server on on like my own machine, right? Like I have a Raspberry Pi and everything is on my Raspberry Pi.

46:32 Me, I exist on a Raspberry Pi, right? Like that that would be pretty cool. or like a future phone that is not necessarily like a Google phone but an Android phone or uses a new OS and everything that I do because what you'd want access to is like everything that you're doing right and this is like what people are doing now like clearly and all these things like oh it's just running in the background but it's kind of taking all of your like do you really want clearly like creating your embedding and then using it for what they're going to do with it right or would you rather own and control everything on your device like that so I think local is going to be very big and then you you have your own for you page that you create. Like your for you page is generated for you automatically and you chat with it. and even all the files that you upload onto your server, you can permission who can and can't access them, right? Which would be really cool because right now I make a video, I have to go to YouTube and upload it to YouTube, but maybe YouTube takes it down, right? It should be the case that I make a video, it's stored on my device and then I just click permissions and say like public, private, friends, right? And then we have we could have like a a centralized registry that has all the servers and then enables you to find files from all around the world and then do an encrypt and transfer server to server between the two people like in a peer-to-peer fashion. I think that's also like very exciting. So again like my data should my data should generate my embedding and my embedding should be used for generative AI and this is like this is like the interesting thing is because once you have my embedding and you can use my embedding for generative AI then you get the hyperpersonalization of all of the content and this is like this is where it's going to go. The question is is like how to get the context right I don't think the question is like what's the best rag for memory right or like how best to do rag or retrieve memory short-term memory or long-term memory or whatever the question is like how to get the context from the user and store it in a way that is correct kind of like in my opinion so yeah that's everything hope you found it interesting and now we can just open up to questions