Google I/O 2010 - Go Programming

0:00 >> PIKE: Hi, everyone. Thanks for coming. Today, we're going to talk about what it's like to program in the new language, Go. It's different from other languages. It doesn't look very different. But when you actually use it, you'll find that it's actually quite a bit different to work in. And so today, Russ Cox and I are going to spend some time explaining some of the more interesting aspects of programming in Go. So this isn't really a tutorial. You'll understand more of what we say if you know a little bit about the language. But we've tried to pitch it at a level that even if you don't know anything about it, you'll still get a feel for what it's all about. Like all the other sessions at Google I/O, there's a live wave where you can comment or ask questions.

0:47 And we'll refer to it later in the talk. So hold your questions until the end, please. And if you have anything you want to ask, please put in the Wave, which is at that address. So let's start with Russ Cox to introduce some of these ideas. Russ? >> COX: Thanks, Rob. So today, Rob and I are here to talk to you about programming in Go. And if you're going to write a Python program, you wouldn't start with a C++ program and translate it line-by-line and expect to end up with a very well-written Python program.

1:20 And concepts and the idiom in those languages are different enough that you just end up with a badly-written non-idiomatic Python program. And the same is true of Go and other languages. If you start with the concepts and idioms from C++ or Java or Python and you try to translate those line-by-line into Go, you'll just be frustrated with the experience. On the other hand, if you use the core concepts and idioms in Go to write real Go programs then you--we think that you'll get a lot out of the experience and you'll be very happy with the language and with your programs. And so our goal today is to teach you the beginnings of how to think about programs in a way that makes the best sense in Go.

1:59 So just to get us all on the same page, let's look for a little bit at what Go is and is not. First of all, Go is object-oriented, but it's not type-oriented. In particular, inheritance is not a primary concept in the language, and that there are no subclasses. In fact, there are no classes at all. There are basic types like int and float and bool, and there are composite types like maps and structs. And all of these can have methods defined on them. Second, Go is implicit and in two keyways. First of all, it's--when you write a variable declaration in a function or even a top-level, if it has an initializer, you can omit the type and it will pull the type off of the initializer. And because of that, you tend not to see type-declarations inside function bodies. And second and more importantly, an object in Go satisfies an interface, which we'll see soon, just by implementing the methods that the interface defines. It does not have to explicitly declare its intent to satisfy that interface. If it has the methods, it satisfies the interface. And we'll see example after example on this talk of why that's really important. And finally, the emphasis in Go is on Concurrent programming rather than parallel programming. And by concurrency, I mean a way to structure your program so that they can cope with having to do many things at once and having to deal with many simultaneous sources of input like network connections, and do that in a way that still lets you write a simple and well-structured program. Now, it turns out that if you have good support for concurrency then it's not too hard to turn that into good parallelism on a multi-core machine today. And Go can do that, and it can keep all the cores running. But the focus is really on concurrency and how it makes it easier to write your programs. And parallelism is just a nice benefit from that. So to start, we're going to look at a simple expression evaluator. And just to keep it simple, this expression evaluator has values that just have binary operations on them. Now if you're going to do this in a language like C++ or Java, you'd start with an abstract class called value. And then you'd have concrete implementations for things like integers and strings. And in Go, we start with "Value", but "Value" is an interface.

4:14 And an interface defines a set of methods that must be implemented to satisfy the interface. And for our values, we need two properties. We need to be able to combine a value with another value in a binary operation, and that's the "BinaryOp" method. And the receiver is implicit, and so the arguments to the "BinaryOp" are the operation itself as a string and the right-hand side value, Y, and it returns the result value. And the second method that any value in our evaluator needs is the "String" method. And the "String" method returns a string so that it can be printed. And if you implement those methods, then you have a type that the evaluator can pass around as a value. And you don't have to explicitly say, "I'm trying to be a value. So let's look at an implementation of a value. These are just integers. And for the most part, you just write the methods. The first line there declares a new type capital "Int" that has the same memory representation as a standard int. But they're different types. This is not--it's not like in Java where an integer is a class and has all the boxing around it. They have the same representation in memory, but they're different types. And in particular because we just defined capital "Int", we're allowed to specify the methods on it. So the second line defines a "String" method on "Int". And the parameter list between the words func and String declare a receiver of type-int X. And that "String" method returns a string by passing X to "Itoa" from the strconv package.

5:38 And the rest of the slide is the "BinaryOp" method for "Int". In the "BinaryOp" method, the goal is to compute X op Y, where op is the operator. And we know that X is an int, but we don't know anything about Y other than that it satisfies the value interface. So the first thing we have to do is use a type-switch on Y to look at what type it is and pick off the types that we expect to be able to handle. And in particular if there's another--if Y is also an int, then we can do the usual arithmetic operations: "+", "-", "*". And if it's not an int or if--sorry, if it's an error, which we'll see on the next slide, then we just return that error as is. And on the next slide, we'll see why. And otherwise, we create a new error that describes what failed because this is a failure that the left-hand side and the right-hand side and the operator, that combination is not valid. And so we create a new error. Now, what was this about errors? Well for error handling, we're going to create an "Error" type that also satisfies value. And so the result of the computation as a value for that value might itself be an error and it just going to propagate the error up the evaluation. And so, again, this first line creates a new type called "Error" which has the same representation in memory as a string. And the "String" method on "Error" just returns that string. But it has to convert it because "Error" and "String" are two distinct types. And second, we have the "BinaryOp" method. And here, there's no computation to do. There's already been an error on the left-hand side of the operator, so we just return that error. And notice that in both of these examples, we didn't have to write. We're implementing value. The "Int" and the "Error" type and the objects of that type are values just because they have the right methods. So the final piece we need for the evaluator is some way to create values. We need some way to take a string that's been typed in and create the appropriate value. And so here's the implementation of that. It's called "newVal". And it calls "Atoi" on the string literal. And then if that succeeds, it returns an int, and otherwise it returns an error saying, "This is what you typed, but it's not valid." And the evaluator has a parser, which we're not showing you, that just tokenizes, parses, and in effect, calls the "newVal", "BinaryOp", and "String" methods. So let's look at the demo. Here's our evaluator. If I type "2" then the evaluator has printed the answer "2", but it's also shown a trace of what calls it made. And first, it called "newVal" on the string "2" that I typed in, and it returned--"newVal" returned the int "2", and then it immediately turned it back into a string. So that's not terribly interesting. If we do "4", it looks the same.

8:19 But if we say "2+4", we get the two "newVal" calls and then we see "2" called with the "BinaryOp" of "+" and "4", and it returns this new int "6". And then "6" has a--when you convert it to a string, it gets back the string to print. On the other hand, and so if, you know--if we do something like "2*3+1", the right things happen. On the other hand if I type "X", "X" is not an integer and I get the error back. And if I say "2+X", you can see that the "2" got created, and then the error gets created. And then the error gets passed to the "BinaryOp" for the Int, and it gets returned back and we end up with "illegal literal 'x'". Now it'd be nice if we have strings in this. So let's try a string and it doesn't work. So let's fix that. If we want to add strings, we define a new type capital "String" which has the same representation in memory as a normal string. And "String" "String" method returns the quoted form. And the "BinaryOp" is the same form as before.

9:19 We look at the type of "Y", and we pick off the operations that we want to support. And so this slide, we have "String + String" and "String * Int" as valid operations. And we have this "Error" type. And again, we return the "Error" if it's not valid. Now notice, we just defined a second method that has a--second type that has a representation string with different methods. So we have "Error" and we have capital "String", and they both have the "String" method and the "BinaryOp" method. And they're both represented in memory as a string, but I've pointed out before that when you want to convert between two different things with the same representation, you have to be explicit. And this is one reason why so that whenever you're looking at an expression and a method call, you know what the type on the left is, because there's no implicit conversion. And so it's always clear which method's being called. So we also have to change "newVal" to recognize the quoted strings that we type in. So if "Atoi" fails before we return an error, we try to interpret it as a quoted string. And if that works, we return the string. And notice that, again, we just added the "String" type by writing the code that was necessary for strings. There was no bookkeeping that was necessary. So let's look at this. Say "hello" string, "newVal" returns the string "hello". And we can say things like ""hello"*3", and it does what you'd expect. So if we step back for a moment. If you're coming from Java or C++ or Python, the thing that's missing in this program is a type hierarchy. And in Java and those other languages, the type hierarchy is really the foundation of your program. And you have to put it down before you write the rest of your program. And then when you get halfway through your program, a lot of times you've realized, "Oh, I should have structured it differently.

11:12 And at that point, it can be hard to change. And in fact, it's often easier to just struggle along with a slightly incorrect type hierarchy than it is to go back and change it. Now in Go, program is not primarily about the types and the type hierarchy. And in fact, there is no explicit type hierarchy. And so the most important design decisions in your program can be delayed. And it's easy to change the types in the program later, because the compiler can figure out the relations between them and you don't have to maintain that information yourself. And we think that this makes Go programs more flexible and more adaptable.

11:48 So, I've pointed out that we have no type hierarchy, so we have no inheritance. So how do you handle a case where, in Java or in C++, you would use inheritance? So that's the next example we're going to look at. In Java, if you had a "ZLibCompressor" that has a "compress" method that takes two-byte arrays. But now we want to support a "Buffer", compressing buffers, and buffers of some other type. And we want to do this in a way that's going to generalize to other compressors. So if we have some other compress function, we can reuse our effort. So what you would do in Java is you would define an abstract compressor class. And it would have an abstract compress method with the bytes--byte arrays, but then it would have a concrete compress method that talked about buffers and implement it using the byte array compress. And then we would go back to our original ZLib and we would add the magic words "extends AbstractCompressor". And this is very common Java style. You inherit the abstract class and you get the concrete behavior from it that you want to add. Now in Go, things look different. So we have the same example. Again, we have a "ZlibCompressor", it has a compress method that takes two byte slices, and we want to support "Buffer" in some general way. But in Go, we don't do it with an abstract class. Instead, we define an interface for the compressor, and then we just write an ordinary function. So we have a compressor interface that says, you know, some thing is a compressor if it has this compress method. And then we have a compress buffer function, not an abstract class but a function, that takes the compressor and uses it to compress from the input buffer to the output buffer. And this is good Go style, you define what behavior you need as an interface and then you just write a function that takes that interface. This is easier and it's less typing with fewer types that in the Java and C++ way. And you could use this kind of approach in Java, but it's not common Java style because Java puts an emphasis on using inheritance. But it's not just a matter of style. Even if you did it in Java, Go has significant benefits over the Java interface way too. So first of all, in Go, you can use as many wrappers as you like, because the type can satisfy many interfaces whereas it can only inherit one abstract--from one abstract class. In Java, you can only extend one abstract class. And a bigger deal is that we had to go back to our original Java program and add the words extends abstract compressor. And what if it's not, what if that code is not yours to edit? What if it's in some standard library that really shouldn't know about this new thing you've defined? In Go, because things are automatically inferred by the compiler, the compressor, the implementer of compressor doesn't have to know that there is a compressor interface. And, similarly, the definition of the compressor interface doesn't need to know that's enabling this "CompressBuffer" function, and so all of these types can be in these joint pieces of library without explicit dependencies between them.

14:43 And notice that in Java even if you use interfaces, you still have to go back and say implements compressor, and so you still have this problem that you have to go back and edit the code and introduce dependency. Now that's one way that you would use interfaces to do something like you do in other languages. But I also want to talk about ways that you use interfaces just in Go in a more unique to Go. And one of them is one of the reasons that there are more unique ways as interfaces are lightweight and in fact the typical Go interface has only one or two methods. And programmers who are new to Go tendency interfaces has a building block for type hierarchies. They see it as a way to get back to hierarchies and big classes and they tend to create interfaces with lots of methods. But we found that programming in Go is not quite the right way to think about them. When we think about interfaces, they're often small and very precise and nimble and often ad-hoc because you don't have to go back and tell the other classes or the packages that you're using these interfaces.

15:44 So this is a real example that we did a few weeks ago. There's an RPC package in Go and it uses a package called gob to marshal objects on the wire and the only language that has support for gob format is Go and we thought it might be nice to talk to other languages and so we thought, well, what can we do to make this work with JSON? And so we can speak standard JSON-RPC. So we abstract the codec into an interface and the server codec is shown on the slide. And the server codec says, you can be a server codec if you have a way to read or request header, to read the request body and then to write a response and the server will loop doing that for a while and eventually close the connection and there's a similar client codec. And then, when we went to the RPC code, we just had to change the signature of a few functions. So the send response function used to take a gob encoder and now it takes this more general interface value. And, similarly, the function to handle a request had the same kind of change and this, basically, the whole change the RPC implementation. We took the opportunity to clean up a few other things at the same but, in general, that kind of change is all you need because of the implicit typing that goes on and the rest of the function and the rest of the program. And notice that we're doing this after the fact. It's really powerful that we can make these changes after the fact and not have to plan ahead for them. This whole change, the converting RPC, to use the interface and then JSON took us about 20 minutes and that included writing and testing the JSON implementation of the interface and the trivial wrap around gob to implement the interface.

17:19 And in Java, you'd probably start with, you start by factoring RPC into this half-abstract class and then you would subclass it to do JsonRPC and GobRPC. But then you might have to decide, oh, but I want to have the same RPC server but with per connection changes and so that would be another re-factoring. In Go, there's no need to manage this kind of decision or this kind of type of hierarchy. You just pass in the codec interface stub and you're done. One more example of interfaces; we've seen examples of using it to structure program and re-factoring and also to provide abstract wrappers. Well, one of the most powerful things in Go is the way we use interfaces for chaining and for defining new kinds of objects that all satisfy a common interface. And so, in this kind of common interface tends to arise organically. It's not something that's planned, it just turns out that we have a lot of different types that have the same methods and so then it becomes useful to define an interface that says, you don't have these methods and then functions can use them. So, you might be familiar with the "io.Reader" and the "io.Writer" interfaces which are very commonly used and have just one method each, read and write. It's really important that these interfaces don't need retrofitting to work with the existing code. It's important that when you introduce the concept of an "io.Reader," you don't have to go back to every single thing with the read method and say, implements reader. So this is a more complex example in reader and writer. When we started to writing cartography code, we noticed that, we have the first code we wrote was the "aes.Cipher" implementation and it was struck with the methods and the methods that have were "BlockSize" which returns the current "BlockSize," the encryption unit size, and decrypt and encrypt. And, similarly, when "blowfish" and "xteo," and other came along, they mimic this set of methods. And we notice, well, if we're going to write code that takes an arbitrary cipher, we should define an interface with these three methods and then it will apply to all of them. And so the block cipher package defines this interface and then defines these block cipher functions using the interface with things like "NewCBCDecrypter," "NewCFBDecrypter," "NewOFBereader" implement cipher block chaining, cipher feedback and alpha feedback mode, all taking this cipher and they're not tuned to specific cryptographic implementations. So if you want AES in CBC mode, you create a new AES cipher and you pass it to the new CBC decrypter. And, similarly, if you want Blowfish in CBC mode, you can see what to change. You just create a Blowfish cipher instead and there's no need for this cross product of every possible mode and every possible cryptographic algorithm what you see in other libraries. It's so simple in Go to compose them that there's no point in providing all the different compositions beforehand. And this kind of chaining is very useful because you can chain all sorts of different things.

20:09 So, if we go back to the reader example, and you might have noticed that the readers on the previous slide or the, they took a site for but they also took a reader and return the reader. And the reads on the return reader are satisfied by reading from the underlying stream and decrypting it. And so we can write this function that decrypts and decompresses from a source file to a destination file just by chaining a sequence of readers together.

20:32 So the first line opens a source file and then remembers to close it. And the third line there, creates a new AES cipher given the key data and then passes that cipher and the file to "newOFBReader" with the key data, with more key data. And the result of that is a reader and that reader, that first reader "r" when you read from it, the implementation will read from "f" and decrypt it and return the decrypted data. And then we can pass that to a gzip new reader and then the "r" that the gzip new reader function returns. When you return from that, it will read the unencrypted data from the first "r" decompress it and return you the uncompressed data. And then finally write, we open a new file for writing, remember to close it and we copy from "r" into "w". And so with that, I'll turn over to Rob to talk about concurrency. >> PIKE: Thanks, Russ. So, Russ has been talking about thinking about programming with the types and the way the types working inside Go and how you structure program given the kinds of things you want to do when you're combining different types and stuff like that. I'm going to talk about a complete different way of thinking about Go programming which is how we think about concurrency. If you're working in Java, you start by thinking about what the class hierarchy is, how you're going to arrange the representation between the items, and that's fine but it's not Go. In Go, if you're writing something like a server or some distributed program, you can use the elements of Concurrent program that construct the design of your program with a very different field and, actually, a very powerful one. And as Russ said early on, it's not about parallelism. Concurrency is not about getting all the cores humming as hard as you can by doing Vector ops or anything like that. It's, but is a way to make programs use multiple for a machine as well in cleanly and easy in a way that's really easy to understand.

22:24 The resulting programs are structured well and they're very easy to change and adapt as requirements change. So, in short, it's about, the structure of the program rather than the performance you get, but the performance tends to come out anyway. So we need an example and it's hard to do a really, you know, new full on example and the time we have available. So, we're going to take a very idealized one, but it illustrates a lot of the points about how this stuff works. So imagine you have a bunch of processes that have work to do, and they're going to send a request out to some smaller set of workers presumably worker machine somewhere that are going to performs an operations for them and send the answers back. In the middle, we want to place a load balancer that uses, that controls the load across the set of worker machines to balance the load across all of the workers and keeps it to sort of even load it. We're going to assume that the workers work best when they have a lot of work to do simultaneously for whatever reasons just makes it work nicely.

23:21 If this is a real problem, of course, we'd use a lot of computers and networking and so on and so on and so on. But we're not going to do that because that just makes the example longer without really adding anything to it. So this is a very simple model but its representative of the core of something you might actually want to do. So let's start what a request is going to be. You're going to have some piece of work you want to do and you're going to send across to the worker via a load balancer and then when the worker has finished the task, it's going to send the answer back on a channel to the requesting processing as your result. And then in the middle of the balancer is going to have some metric for the load on the workers that's going to use to even out the load. It looks a little bit like this. Across the top, we have a bunch of requesters, there's probably many, many more requesters and workers but, it's really hard to make slides that look like that. So we'll just assume there's a lot more requesters on the top. The bottom is a smaller number but it still substantial a number of workers. And then there's just load balancer in the middle that has a single channel for all the requests that are coming in. And then it immediately forwards each request to the leg, most lightly loader worker that's running, and then as the worker completes each request and sends the answer directly back to the thing that requested it. And also although it's not in this slide, it's going to signal the load balancer, I'm done now, so you can adjust the load you have stored from it. So, the requesters look like this. You have a request represented by a closure which is the operation you're going to perform that returns some type which we're just making into you for simplicity. And then a channel of that type which is the response channel so, you're going to send, say, basically, run this function and then when the answer is done, send it back to me on this channel, that's a request. And then here's a really sort of simple minded but functional version of the requesting operation, it just sits in a loop doing something else for a while probably and then sends a request in a form of close on a channel to the work channel which delivers it to the load balancer. And then it says, wait for the answer to come back from the worker. And once the answer has come back it probably does some more processing on it. Now the worker is going to sit there waiting for a request to be delivered now from the load balancer and then it just sits in a loop doing the obvious thing; gets a request, executes the function of the request, sends the answer back on the channel, and then tells the load balancer, hey, I'm done, and by just sending a point itself on the done channel. And so this is very, very simple but it works. And it's important to know that the response is going directly back to the load balancer because the channel gives you the capability to tell the worker to respond to you even though the load balancer in the middle doesn't need to know, doesn't need to keep track of the connections between the workers and requesters. So now we got enough to sort of think about the load balancer and we need, we definitely a pool of workers, that's obvious, we represent that with a slice.

26:12 And then, the balancer itself is a simple structure that has this pool inside it and then the worker channel, so that the workers can signal when they're done; that can be a single channel for all of them because it's not going to be very busy. And now at this point, this is actually the load balancer, this is all we have to do. We sit in the loop, and one of two things is happen; we either are going to get a request in which case we dispatch it to the most likely loaded worker. Or, a worker tells us it's done in which case we update the status of that worker so that we keep the load sorted out, so that's very simple. Now we just have to write two functions; dispatch and completed. And the way to think about that is if you imagine the load balancers, basically, maintaining a priority queue and the things that the head of the queue are the most likely loaded objects, so we want to order that queue by the length of the request queue going to that worker and to do that, the simple thing to do is to just write a heap. So we basically are going to make a heap of channels where the channels representing the workers inside the worker struct and then the load is represented by the length of those queues. And to do that all we have to do is attach the methods for heap onto the pool type which is part of the implementation of the balancer. And they're very, they're all very simple--here's the less method, there's a few others but they're all not, you know, like, most of them are like one or two lines as one is about four or five lines, but they're all pretty easy. And once we've done, the pool of workers behaves like a heap and so now we just need to define the worker structure which is has nothing in it but a request channel and the count of pending request and then an index which is part of the heap implementation. So now we can implement, right? Here's the dispatch. All of the dispatch has to do is give in a request, you pop the element off the heap that will be the most likely loaded worker, you send the request, you increment its pending count and put it back in the heap, that's it. And for the completed set, it's the inverse. You decrement the count for the load on that worker, take it out of the heap and put it back again, that's the whole thing. And this is all working because the channels are first class values and the methods can be attached this pool type to turn the pile of workers into a heap sort of post facto. So let's show it running here. So this is not very graphical but you can get the idea. This job has 100 requesters and 10 workers, and the workers are, the heap is basically the pool on the left there shows the count of the load on each worker. And then the columns on the right are the average and standard deviation of the load. You can see the load is going up because there's a lot of happening but the standard deviation stays pretty small because the balancer is doing its job, that's pretty simple. And just for reference, I made a version that just trivially does round robin on the workers rather than a balancer. And if you let this run for a little bit, you'll see the load goes up as it does in the case but the standard deviation starts to grow.

29:08 At least, it does--here we go. It's increasing. It's not nearly as even balanced as it was in the case where we actually used a heap or a queue for the worker, so it's just, you know, a proof of concept that's not--there you go you see the loads starting to go very uneven across those workers. So, let's, so to go back to the discussion here, there's a couple of things about this program that are interesting. One is that it actually works, I've shown almost the entire implementation just a little bit of setup is missing. Everything that this thing is doing is synchronous. Everything blocks and yet the system is highly concurrent, and there's not also a single mutex in sight. Everything is done--the structure of the concurrent operations makes it possible to write a program like this is totally non-blocking even thought it's built out of synchronous operations and there's no mutexes, there's no worrying about memory barriers or any of that stuff. But the fundamental properties of the language make it really easy to write a software like this. Another nice thing nice thing is that this notion of a closure and the channel as a pair. It's a really powerful idea. I can send work to somebody and include a way to get the answer back to me afterwards and that's a really powerful structuring concept that you see in a lot of Go code. And also you see these channels being parts of first class values that are passed around between different Go routines as they're running, that's a really important concept that's missing from some other, not all, but some other concurrent languages where the connections are much more rigid than they are when you pass channels around. Now this is a very simple example.

30:35 We can do a much more interesting case where we had networks and stuff like that to distribute load across many machines and they would look fundamentally the same but you just have to do a little bit of work on it get there. But it's not really relevant to any of structuring the program, principles are all in this example. So, to conclude here, I think we've shown that programming in Go is not like programming in Java or C++ or Python. Even though when you read the spec, you might think it's a very sort of straightforward language. It's actually a lot going on that feels very different when you write real code in it. These include the fact that objects are not always classes. In fact, it can be anything including--we've seen examples with strings of integers and slices and structures, and methods are being used on all of those things. And so inheritance is not only, not the only way to structure a program, it hardly even comes up. The inheritance properties have gone up very interesting compared to some of the other things we can do with things like interface. We also don't need to write everything down in advance. You don't start by designing your whole superstructure of your type system before you write any code. In fact, a lot of the relationships between types tend to come up as the programming evolves or even long after it does like the PRPC example.

31:45 And because of this, a lot of things are discovered long after the program is written, it can be very pleasant. It was amazing to us that we may implement a JSON-RPC but just substituting an interface into our existing RPC implementation. And Russ said just the 20-minute job, and that's the kind of things it's very hard to do when the structure of the whole program is based around in here. And also, concurrency, a lot of people think it's really about parallelism and it's not. It's about structuring software so that it can adapt and grow in a very parallel universe. And it's not about getting all of the, it's not about numeric competitions but system structure. And as a result of these changes from existing languages, we think that was a very much more productive environment. You can get a lot done very quickly because things tend to be much more nimble when you're working on it. You can give a method to any type which means this all these opportunities for designing your program that are very different from a world where everything is a class and you think of much larger objects are being things that have methods attached. A lot of the bookkeeping that you see in a type driven programming simply doesn't appear in a Go program. The power is still there but the compiler takes care of it for your and that's just a huge productivity game right there.

32:59 And then, this, it's hard to sort of through big examples but I think I've shown looks like this load balance, for example, that concurrency is the way to structure a program in a way that makes it really productive to write programs that work with, you know, very high load and very interesting parallel environments. And as a result, Go programs tend to be very malleable and adaptable and much less brittle as conditions change. And, if any of these interests you, of course, there's a lot more available at the website golang.org. But the most important point is that Go is a lot more fun, the programming and, of course, it comes with t-shirts and tattoos and stickers. So we'll do a Q&A now but as you go, there's a pile of t-shirts down there and feel free to take one as you leave if that's the sort of thing you like. I said as you go, we're going to do a Q&A first. And as I said at the beginning there's a Q&A thing on this wave, live wave thing but remember to go to golang.org for a lot more information. So, we'll do this. So there's also microphones if you want to do a live questions but we'll start with the ones on the wave. Oops, can you... >> COX: And if you're worried about making the next session, I think that everything is delayed as much as we were, so, the next session has not started yet, right. >> PIKE: And there's enough t-shirts for everyone, so you don't need to mob. Okay. Can you comment on the suitability of Go for deploying a real application today; if not today, what timeframe? Also, my comments on server is with the mobile suitability. As far as the first thing goes, we're already using Go internally at Google for some production stuff so, it depends on the particular job you're have in mind but the language is pretty stable. The libraries are good in getting better rapidly.

34:43 And for things that it's ready for now, it's already being used for production stuff. I don't know if I based the start up entirely on Go now, but, it might, I don't know. So, as far timeframe goes, it's just going to get more and more mature, of course, over time. But I think if you have an application today, it might be the right language to do it. For server versus desktop versus mobile suitability, it was originally design for server kind of stuff but it seems to work very well for a lot of other things. The desktop capabilities are limited because the graphic libraries aren't quite there yet, all the other are under way. And as far as mobile goes, I don't have anything to say but we really love to see it in a mobile environment. Do you have anything to add? There's a question here. >> So I knew you have an ARM Compiler and I'm just curious with the status, when I think of concurrency and ARM, I think of interop service routines for embedded programming. Is it applicable and have you done it?

35:41 >> COX: I think that we're still exploring how you would do something in a very, very limited ARM environment on something like the Android phone, which is also an ARM. You don't have those problems and so we've been running our ARM Compiler tests on that kind of platform. We do have one developer who's working on very, very tiny embedded things and I don't know exactly what he's going to end up with. >> Thank you. >> PIKE: What are your plans about port of a loop go between different platforms? There are no plans per se. It's a compiled language and so if you talk about portability, you mostly mean source level portability.

36:13 We try to make libraries very portable between different environments. It already runs on Linux, MAC OS X, FreeBSD, there's a window support that's pretty far along, and some other sort of ARM embedded stuff. I'm not sure what to say about that, except that the language is intrinsically quite a bit more parallel than language like C or C++, but not quite at the level of Java because it's one level back in terms of the way you compile, but I think it's the right form of a language, honestly. Yes.

36:43 >> Can you address performance as compared to C, C++ and Java for server-side programming with multi-cores? And also what if you need to interface to existing libraries and those other languages? >> COX: So working backwards, the interface to existing libraries, we have a basic tool that let's you call C code from go and there's also work going on to--I hope go up this week so that you'll be able to wrap C and C++ libraries this week. As far as performance, we found that things that are highly computational tend to be about the same, maybe 10%, 20% slower with C code and equivalent Go code, the place where it goes firewalls behind is if you're littering your C code with things like M-line assembly and send the instructions and things like that, then we can't keep up anymore. But if you're doing just sort of standard portable C code and you translate that into Go, computationally, it's not very much of a performance set at all.

37:38 >> You did a lot of comparisons with type languages or C, C++, Java and a lot of the structures reminded me more of functional languages like Haskell or Lang and I was wondering if you could do a comparison with those and where it goes--shines relative to these languages? > PIKE: The functional language comparison, I'm not a functional ad app. I think I'm a little bit comfortable with them but not very. The language has full closures and so that's very nice to have but a lot of the sort of magical type inference that functional languages do are not available in Go. The Go type inference is very simple, it's basic. In front of your slides, you can draw out a type, the sort of meta type of stuff that functional languages have, simply not in Go. And I think that would probably be the biggest difference in thinking about them. Plans for not writing Androiding in Go. There are no plans. I'd love to see it but I--honestly, we're a pretty small team and it's not where we're going. But it's not an accident that ARM is one of the things we support. I'll get back to that one. Yeah.

38:46 >> I saw a lot of you saying, "Oh, well, this is just, you know, to implement this, this is a bunch of one line functions and I think--it seems like when you go inheritance to sort of implicit interfaces, you find to yourself writing this--do you find yourself writing the same one line--the one line implementation of the same piece in the interface over and over again in a lot of different places? >> PIKE: No. The we didn't talk about which is embedding which replaces a lot of that bookkeeping like stuff that you would--that I think you're alluding to. It's possible to borrow the implementation of something else and drop it into your object and have it--bring all the types along with--all of the methods along with it. And that tends to avoid a lot of that sort of repetitive--that got to add this ten one line function to this thing. I think that's what you're referring to. Yeah. Current situation with generics, yeah, that always comes up. We spend probably more time on this one issue than almost anything else. And the short answer is that we're still not happy that we've been able to find design for generics that doesn't break something in the language or become to confusing for users. So we're still thinking about it as it's still incredibly active discussion. We're very sensitive that we have language that seems to work really well and we don't want to mock it up with the generics implementation that complicates life when it should. Also, speaking personally, this is just me, Russ may disagree. I don't--having used Go as my main language now for almost 2 years, they don't really mess generics, they don't come up. The language feels different. And a lot of the things that people want generics for already sort of covered, not everything by any means, there's a lot of places where to be nice. But it's not clear that it's important to go as it would be in some other language, which is not to say that the word design comes along. We would not go for it. We definitely would.

40:37 >> COX: It's also--well, Rob said a lot of people or at least a few people on the mailing list who have written significant amounts of Go code have said the same thing that having written, you know, 1,000 or 5,000 lines code, they found that, you know, they thought they we're going to miss generics but they don't actually miss them now, so. >> PIKE: Yes. >> You mentioned that there was a certain intrinsic properties of the language that make it easier to manage consistency of your state in concurrent programming... >> PIKE: I'm sorry. It's hard to understand the... >> I'm sorry.

41:09 >> PIKE:...those things are terrible. Can you speak a little louder? >> Yeah, sure. You mentioned that there was a--there were intrinsic properties of the language that make it easier to concurrency, perhaps rules dictating how code as it is running in parallel can be interleaved and how that affects the state of variables in your closure, that kind of thing. Is that a shortlist of properties in the language that you could speak to now? >> PIKE: Well, there's quite a few, but you saw control structure in there which gives a way to control access from multiple channels at once the select statement there. The way the stacks work in Go is they start out small, they grow in demand. So, you don't have to allocate a stack and say, "I need 10K of stack right now or something like that." The language completely handles that for you. You don't do--you never think about stack size. And they're cheap to do and recovered as necessary.

42:00 The combination of communication and synchronization that channels give you, once you've used them for awhile, you'll see it's an incredibly powerful primitive. It's much more interesting to program this way than by passing memory around and using to protect. It's also a lot easier to get the code right. For me, one of the things seems to--sort of say the most is my officemate, Rob who also work on Go had never worked in a concurrent language before, although we had spend a fair amount of time dealing with C++ and writing server software and that kind of thing. And he find--he's written a lot of concurrent programs in Go now and he just finds they work. He doesn't struggle with the problems he had writing server software in C++. And it's not that he's got a concurrent brain. I've been writing concurrent programs for many years now but this is all new to him. But he finds it's really natural to express what he wants to do with this independently executing agents exchanging information. It's just a very natural way to program. So the language enables that and that's nice. >> COX: I would just add that in your question you talked about, you know, interleaving of individual instructions and things like that and we found that if you're thinking about that level, you now--you just never get it right. And what Go encourages you to do is--especially with channels to let the Go routines communicate amongst themselves of explicit messages and that communication forces coordination. And so you don't have to worry about, can these instructions happened side-by-side, because, you know, you handed object off to a Go routine and now it owns the object and it does stuff and then it hands it back when it's done and there's no actual...

43:26 >> So there's nothing, there's no memory shared between independently executing Go routine. >> COX: There can be memory shared, but the convention is that, you know, you pass an object to a Go routine and it owns the object now. When it hands it backs then you can start reading and writing again, but until then... >> I have a metaphor that I used to explain this to people. >> PIKE: Imagine you have a bunch of work to do and if you're thinking about memory barriers and concurrency, what you should do is get a piece of paper and you write notes on the paper and everybody read this paper and you markup dynamically what people look at. Everyone grab things when they come back and they edit this piece of paper and it's very easy to imagine getting confused by what's written on the paper. The way you do it in Go is you have a pad of paper and you write down one task and you give that piece of paper to somebody else and he can walk around the room and give it to other people the other part of it, and we're they're done they bring it back to you. And there's never any confusion because you're never looking at--more than thing at a time. It's a very source silly, childish way to describe it but it has a profound effect on how you think about writing a software. >> COX: So next question is, is there Go Compiler for native client in the works? And there is a Go Compiler for native client, it is done, and you can use it to create native client binaries. We haven't played too much with hooking it up to the web browser and doing interesting things with it but all the compiler work is there and ready for playing. >> It might have been because it was short but Unix Programming Environment is the only programming book that I ever read all the way through, and a lot of the reason for that is because assembling different Unix show commands and stringing them together is conversational in nature, that's important to me. I'm also very event driven in my thinking. And this language shows me a lot of--both of those, so we're you specifically going for that feel like a--did you have a cognitive model of programming in mind that made this just like field really right to you?

45:18 >> PIKE: Not explicitly but I don't think it's an accident, you know. I have worked in concurrent for quite a while and a lot of it is inspired by the gluing things together, model from Unix long ago. And some people make an analogy between a Unix pipeline and both channels sending messages but also interfaces chaining things together like in the file for example. But it's not an explicit goal. I want to take the Unix ID and make it work.

45:44 It's a programming language. It's just--it's a good model and it has multiple domains where it makes sense. >> So speaking of that model, the--working with programs that seems to be complete when they were written but you can string them together. Do you have a model for sort of external visibility of the guts of someone else's program so that you can inherit from another complete app? >> PIKE: I'm not sure I know what you're asking. >> Let me--like Tcl seemed when it was starting out to be a really good glue language that you could take stuff and, you know, embed it, that's what SWIG is for. You have libraries and then you have seemingly complex atomic things that are good in themselves and then you string them together in a language that's more fluid. So do you have an idea for how Go would handle the management between seemingly independent apps? >> COX: We're not really targeting...

46:42 >> Okay. >> COX:...that kind of glue. We're really targeting, you know, Go--whole Go programs. >> Okay. >> COX: And we can interact with other languages through RPC or through something like SWIG for just calling a library, but in general, you know, the shell works really will for that because all the pieces are speaking the same interface that read from standard input and they write to standard output. And, you know, in Go, you just--you have interfaces like that. You have reader and rider but it's all in the Go world and there's not really an emphasis on interacting with the rest of the world except through things like RPC and that sort of standard mechanism, so standard input, standard output, things like that.

47:16 >> PIKE: And RPC works in his world. RPCs can satisfy interfaces and, you know, channels can be passed over networks and I think there's higher level of things that go on, but I don't think it's exactly what you're getting at. >> I'm just wondering, is there some model for like, for sharing code like, I mean, Ruby has, like, you know, gems and it's easy to, like, I think when something, someone is done or like Java jars and...

47:42 >> COX: Sure. There's a program in the distribution called Go Install and you can think of it as sort of a distributed CPAN, basically, if you put your code on GitHub or Google code or one of these code hosting sites. Then you can say "goinstall" and give it a path of the right form and it will install that package and then you just import it. >> So does it, like, does it have to be in the source code format or I mean...

47:58 >> COX: Yes. >> Yeah? >> COX: Yup. >> So you can't do a binary. >> COX: At the moment no. I mean you're compiling for many different architectures, right? So the binary would limit you to just the person who, you know, has the same machine that you do. >> Thanks. >> Well, I was looking at the slide there, I didn't quite see--if you have a bunch of libraries together, how do you make sure the name spacing doesn't clash? Is there some sort of name space or scope...? >> COX: There's no uniqueness in the name space, so if it does clash all you have to do is rename one of them during the import for your particular file and then, you know, it'll--you can use the different names to refer within that file. And the linker and the compiler and all these tools can handle the fact that there might be multiple files, let's say, I mean pack and hash as long as they have different import paths, they're different, they're different packages. So, it all just works. >> Okay. Excellent.

48:48 >> PIKE: There's no such thing as a truly global name in Go. >> Okay. >> PIKE: Every name is relative to a package and even if packages have the same name, they are distinct and entities in the world and you can resolve it. >> All right. And my other question was, I saw that you had something like defer I/O.close, what exactly was that? >> COX: Defer is a concept in the language that says, "Right before you return run this function call." And so when I said "deferf.close" what it meant is that no matter how you return from this function call "f.close" as you go. >> PIKE: And that's a dynamic call, it's not scope. So, you could call "deferf.close" for five different values of "F" and it would close all five "Fs". So if you have loop opening file then you said defer call that close in that loop, it will close all of it. So, it's not a scope thing, it's a dynamic thing.

49:43 >> So I suppose this language is not statically typed, that's right, isn't it? >> PIKE: No, it's statically typed but it has some dynamic typing depending on how you're using interfaces. Sometimes narrowing has to be done dynamically. >> Okay. But comparing it to Java a lot of checking can be done in Java in compile time. >> COX: Go is very statically typed. All of the programs that we're looking at were using interfaces; and interfaces, even interfaces are statically typed. So where there was a parameter of typed value there can only be an object that has the right value methods.

50:14 But you could also have a parameter of type end, bull or float just like in Java and so... >> But the point is that you can't check everything. You could check in Java at compile time, isn't that right? Because if you pass in an object which might be of the right... >> COX: I believe that you can check--everything you can check in Java and Go in compile time. >> In compile time? >> COX: It's very static, and the compiler--if you're not being careful with the types, if there's something the compiler cannot prove but it needs to know, it requires you to be very explicit and say, "I'm promising here and I understand this could be a runtime check and runtime false." And so it's very much statically typed and you can use interfaces to make it feel much more dynamic but in the compiler it's al static. >> Okay. Thanks.

50:57 >> So I've been using Go for he past couple of months, and it seems like a lot of the work is, like, making it more stable in fixing bugs. Are there any new features of the language, you guys, are planning to add? Like, I personally like to see more of dynamic features like being able to, like, manipulate types more. Is there anything you guys are planning to...? >> PIKE: I think roadmap has some things but not in a general direction. I think there's probably coming a time when dynamic code loading would be available, but there's no plan for that. We haven't talk about it. I just sort of given a way the packages or work in a structure of things. I think that's got to develop on its own. At the moment we're more concern with developing libraries to the point that people can use this for production in more ways. But, you know, rose up in the proposals, so if you got things you want to talk about, you know, let us know and we'll consider it. >> COX: I also think that language changes just take longer. They take a long time to just simmer and really come up with a good design and it takes a while to roll out. And so you're going to see fewer of them than you see, you know, new libraries. And so in particular the reason it changes to panic in the introduction of a cover were things that we spend a very long time thinking about on the back burner before we finally came over the design that we liked and then it was relative and quick to roll out and so it's mostly design.

52:19 >> PIKE: Two years of thinking before it went in, so. >> COX: And as far as what you were talking about for types in particular, I think that the reflect package is probably in need of redo. And I think that when we do that you might see some more dynamic key things like you should be able to just construct the type at runtime I think. But all of that is still very much just simmering in the back and eventually it will, you know, bubble over and will have some thing.

52:43 >> PIKE: One thing we didn't mention because as I've said this is a tutorial is there is full runtime reflection for those who don't know that, which makes for some really interesting possibilities. Any questions? How are we on time? I can't see the clock back there. >> COX: It says eight minutes. >> PIKE: Eight minutes. Okay. >> You mentioned about RPC... >> PIKE: Can you take--can you come to the mic if you're going to ask us a question? >> So graphics processors are really powerful, is there plans or something... >> COX: Can you speak up? I'm sorry.

53:13 >> Is there plans for something like open CL, you know, to run stuff on the graphics processors? >> COX: Graphics processors. There are no plans but hopes and dreams sure. >> All right. You mentioned RPC for inter-program language stuff. Google uses protobuffs and you have internal RPCs, what about for other RPC systems, they're like co-generation for thrift which does provide RPC right now or is that something you could provide? >> COX: We don't, but I think that changes to RPC that we talked about here would--that made it possible to plug-in JSON RPC as the wiring coding. I think you can plug-in protocol buffer wiring coding. I think you could plug-in thrift wiring coding and just use the standard Go RPC server and you could speak all of those protocols at once if you wanted to. >> All right.

54:04 >> COX: Yeah. >> Thank you. >> PIKE: It's really straightforward to swap the encoding app. >> What's the policy on what goes into the standard binary package versus external? I've seen a whole bunch of stuff that's in the standard Go and I've seen languages like Perl and Python that have grown a really large standard library, then I had to got it or clean it up later. >> PIKE: We're very, very sensitive with that issue because we think that some other languages, I won't name them, but I think some other languages have not been careful enough about how the library goes. So, we've been trying to make sure that the basic libraries that are installed with the compiler and stuff when you get it form a really good corset that large subset of which will be use in most programs. And things that are much more special purpose or larger in scope, we tend to encourage people to make separate repositories for but then in turn we have a program "Go Install" which makes it very easy to import them if you want. We want to make sure that the standard packages were useful, modest and scoped but--and very, very well written.

55:04 >> COX: I even put in something in the standard library as a promise that, you know, something like this is going to stay there and so I think we have to be careful just to think about, you know, is that general enough that... >> But you just mentioned redoing the reflect package for instance. >> COX: I'm sorry. >> You mentioned redoing the reflect package. >> COX: Right. But I think it's a promise about the functionality more than the details of the interface. So, for example, you know, someone wrote fuse package recently, and I think fuses are good example, something that makes sense as a separate package and you can "Go Install" it and so it's no harder to use than the regular packages. But it, you know, it doesn't really makes sense in those standard tree. I think once Go is a little bit more mature, we might see people who build Go distributions that have a whole bunch of other packages pulled in, you know, just as part of the tree as though they'd already been "Go Install."

55:49 >> PIKE: Right. >> COX: But I think the standard stuff is really supposed to be the kind of things you'd expect in a standard library. >> PIKE: And "Go Install" works transitive so it makes. So we've only got few seconds left, so are there--I think there's no more questions. Thanks everyone for coming and grab your flag on the way out.