Chapter 22.1 - Transformers & Foundation Models: Vector Spaces for Word Embeddings

0:07 >> We finally enter into Transformers foundation models attention mechanisms. These are sort of what are driving the amazing capabilities of large language models. We want to talk about them in context of first of all, how they're built, how they're trained, and what the perspective of of what these might be able to accomplish in science and engineering. So, the first lecture today on this is going to be around thinking about words and language as a vector embedding, okay, or vector spaces for how you put words into the mathematical framework that we need to manipulate to train models.

0:47 So, words as vectors. So, what we've done developmentally over quite a bit of time is trying to take language and make them into vectors so that we can start looking at sequences, which are sentences, and how they are constructed, and how we might be able to forecast a sentence, which is essentially start a sentence and have the machine learning fill in what might come next. So, let's start talking about this in terms of what we might want to do. And in the early days, each word was in some sense a vector representation.

1:26 So, let's take these two sentences. Machine learning Machine learning is great, and machine learning is awesome. They in some sense are alternative views of saying the same thing. So, we use great in one instance and awesome in the other. And so, somehow in some embedding space, these two concepts or constructions should in fact be close by each other in that vector space. So, we're going to basically take language and move it into a space.

1:57 But, the word itself can change meaning significantly. So, we're going to have to have context around this. So, take the word great, which is a very positive statement up here, machine learning is great or awesome. And now, we could do something like this, great, I just failed an exam. The use of the word great here is actually a negative. It is sort of a you know, a statement which is means really opposite of what you meant to do here.

2:27 And so, language is very difficult this way because context matters quite a bit. So, we have to train language models in with context in mind. How the word is being used, what are the other used words around it? That matters a great deal in terms of thinking about how to generate a sequence of words that make sense and that use the words correctly in a sentence. So, so, again, great here at least, this is the highlight of what these two sentences versus this, you can see great is used in a completely different way. And so, when we build these large language models, the language model has to actually understand the context in order to do a good job with embedding things.

3:11 So, in the early days, there was a lot of ways to en- development of ways to embed language in vector spaces. Here are some of the ones, word to go at, word to vec, glove, e bert. These were all methods in which you can create a vector space representing words. What it really means is a coordinate system. And so, I'm giving you here a sort of qualitative idea of what this might look like. So, every word is embedded here and you have a vector out to the word.

3:39 So, the word great maybe, this is the jth word in your vocabulary. And of course, we imagine having a very large vocabulary we have to learn, but great, awesome, and amazing are words that are sort of in a similar place in the vector space where down here I might have camel, cat, dog. These are four-legged animals. They should be close by in vector spaces. And over here, you know, bad, awful, and terrible. These also are sort in some sense opposite of this and should be in a different part of the vector space.

4:10 But the vector space itself isn't just three-dimensional. It's very high-dimensional in how we're going to embed these words. And when we think about modifiers to these words, we could say you know, an awesome car. Car and awesome now go together. So, car modifies the position of awesome to move you to another place in this embedding space. So, in some sense where we're This is going to be the the groundwork for how are we going to handle language is we're going to represent language as vectors.

4:41 In the more modern version of things, we tokenize sentences or language. So, words are not the embedding, but that the tokens are the embedding. And so, you chop up sentences into tokens, and those are what you operate with to train how to understand context and sequences in that embedding space of tokens. All right. But if this is right now, this is say the early days of the language modeling was to represent words as vectors.

5:15 And so, for instance, wj would represent the jth word. What we want to think about is a model to make predictions about the next word. So, for instance, we can think about this as a probability, a conditional probability. So, what's the probability of the next word being w of j plus one given here is the sequence of the last n words that you just heard. So, the last word, j minus one, j minus two. So, this is in some sense the terms in the sentence up to the current word. what's the next word that is with the highest probability? In fact, you could have many words with equal probability. When I gave that example, machine learning is great or machine learning is awesome, probably the probabilities of both those words are similar and so you would pick one of those with a high probability. And of course, how you pick it, you roll dice in some sense, right? You have a random number generator, you pick the appropriate word for here, and so every time you actually run this sequence model, you might get a slightly different sentence because it's based upon probabilities.

6:23 So, what we have to do is build a model for how this probability is going to be calculated. And so, what we think about is training a neural network to do it. And what kind of neural network we talk about in a minute, but this neural network, here it is, here's the model F of theta, theta are the parameters we're going to have to train. F is the model. We're going to take into the model, this is the input, it's a set of words to the current word, the target or output is the next word.

6:50 And what we're going to do is make up a model. We're going to start off with sequence models in this lecture, talk about how those work, and then in the next lecture, we'll talk about the architecture of the transformer with the attention mechanism built into it. So, here's the basic structure of how all of this is going to go, whether it's classic style language models, LSTMs, GRUs, or current neural nets, but also, it essentially lays out the the model that we're going to use for large language models using transformers and attention mechanisms because all the magic's going to happen in the encoding and decoding space. So, the idea here is there's an input, we're going to encode that into a latent space representation.

7:34 So, we're going to take words, embed them in a vector space, which can then learn even a latent space in the sequence model there, and then we can decode this. So, once we have this, we have to decode it back to language and then that can create what's called a rollout. So, once this is trained, we can pick the next word. The next word would then come in here to the decoder to pick the word after that, the word after that. So, this thing here can run in the loop and start generating entire sentences, paragraphs, pages of writeouts that we want or language based upon this kind of architectural structure, which is basically picking the next problem the highest probability words that would be for the sentence structure that I have moving forward.

8:19 Okay, so let's talk about what we want to do here. We're actually not going to start with transformers. We're going to start with recurrent neural nets. This is a more classic way of doing this. So, I'm going to give you an example here recurrent neural net F, that's F. I'll show you the structure in a minute has some parameters theta. It takes in a sequence of words n words. So, in the big difference that happens in, let's say, the old way of doing machine learning with recurrent neural nets, LSTMs, GRUs, and so forth is that there was a finite size sequence of n that you would have beforehand to predict the next word.

8:53 Transformers, we'll talk about in the next lecture basically has effectively an infinite look back. In fact, it can look over the entire body of all human knowledge for these probability distributions and it's doing it with a very different architecture. So, it's based on the attention mechanism. But, at least for the old style language models, it's sort of a a look back of n words and I'm going to give you an example where you're looking back three words. So, here it is. So, Hello, how are So, that's the first word hello, how are.

9:31 Those are the three words that you have. Actually, it's hello, how are. So, hello is the first word, how second, third, are. So, this is the sequence that you received, and then the most likely prediction, the highest probability is you. Hello, how are you? Now, how did it learn that? Well, we trained this on a corpus of written documents. So, you normally wouldn't see something hello, how are cat.

10:02 That's like a low probability event. It almost never occurs in writing, and so that would be a low probability. But, hello, how are you? Is a very common phrase, and so when you train these weights and distributions probabilities, this is a very high probability event, and that would be the outcome from this neural network. Okay? And then we could keep rolling it out. Hello, how are you? What would be the next prediction? I am fine, how about you? This is a very high probability roll out.

10:33 There are many other things here that you could do. Hello, how are you? You could put terrible, but most people, even if they are terrible, lie. They say, I am fine. Unless you're a close friend, you might say, I am terrible. Okay, but that's the idea is that you're essentially looking at probabilities and their distributions and how you might roll them out. Okay. So, what we're going to do is start thinking about building this model, and this is the generic model of what we want. It's a sequence model. In other words, I have a sequence of events, or a sequence of vectors, and these are going to be words embedded in a vector space, and I'm going to train a model, in other words, understand how to train the model and figure out the parameters data so that I can make a reasonable high probability prediction of the next piece of the sequence, in this case, the next word. Okay?

11:27 And so, here is a standard way to do it. This is what's called a recurrent neural net. The input are your words as vectors, a sequence of words as vectors. And this sequence model builds a latent representation H where it learns how to in the latent space move forward. And notice this latent space model F of theta is the same across all of this. So you're training the weights of this model to then produce outputs Y1, Y2, Y3, which is the output sequence. So hello, how are you?

12:00 At least when I have hello, how are, you is the output. And then I could train this across a body of knowledge. In other words, all the written documents that I might be able to consume to learn the relationships between sequences of words and the highest probability outputs. And that's how I would train the weights of the latent space model here, let's call it F of theta. And it's a joint model across all the rollouts. Okay? This is the early days of language prediction.

12:31 And so the way we think about it here more generally generically as a as a neural network is I'm trying to find a latent representation, predict H of T given my input X of T and given my previous sequence H of T minus one. So I give the new input, the previous latent space vector, and these are all with weights, weight matrices, and a bias, and with some kind of activation function. So this is something I can train on given a bunch of data that I would have. And from this I'm trying to predict the output, which is another sort of decoding step out to the output with some weights, where it takes this latent space variable, weights it with a bias through that decoder out to the next word in the sequence. So you have to update the weight, basically learn the weights of all these matrices in your encoding and decoding space with some activation functions you'd pick along with essentially the biases you would have there. How do you train it?

13:40 Well, a large corpus of writing, right? So, you're basically ingesting books, written documents, everything you can do, all the as much data as you can have in here to refine what these weights are to do next word prediction. So, in this case, again, recurrent neural nets have a finite look back. So, they are taking in, you can make it fairly long, but essentially you also want to have a receding memory. you don't want to look back to infinity, but you want to look back, let's say, 40 words, 50 words, 100 words, whatever it might be to make the prediction work.

14:17 So, presumably, those last 40, 50 words also have some of the context of what you might have in your prediction. But, if I was talking about a dog and I described the dogs two paragraphs ago, that information two paragraphs ago would be beyond the reach of this look back and then it would have forgotten about what was said two paragraphs ago. And that's where the transformer comes in because the infinite memory allows it to keep that kind of information. But, this is what you're training and this is what you're updating. So, you have a large corpus of knowledge and you train the weights of this model. And that was the early days of how we thought about training language models. And so, really the key idea here is first we need to find an embedding space. And I just talked about these vector spaces for words. Well, we're always going to be operating on the computer as numerical values. That's all your computer is good at, adding, subtracting, multiplying, dividing. But, if you do it in the right way, it does it in such a way that it's indistinguishable from an intelligent agent. So, but remember underneath it all is just matrix multiplications. That's really what it's coming down to when we do this and when we learn sequences of words. What's going to happen next is we're going to generalize this concept of a recurrent sequence model, recurrent neural nets, into the modern transformer and attention mechanism architecture.

15:45 >>