The Keyboard Is Dead. Now What?

0:00 What I'm going to show you in this video is going to make you rethink everything you thought you knew about optimizing your office for ergonomics. If you have problems with working with computers all day, using a keyboard, RSI, shoulder pain, fatigue, all of these issues with back and varicose veins and all of these kinds of things. The industry will try and sell you all of these solutions and actually they just cause other problems. They just move the thing around, the problem around. And this is basically because if you're in one position all day long, that's the problem. It's It's not a problem that can be fixed just by using new gear. I've been passionate about this over my whole career working with computers. I built this channel around the idea of solving the problem, optimizing office ergonomics through keyboards, custom keyboards, reducing key counts to reduce finger movement and using extreme tents, using all kinds of weird and wonderful solutions here to just try and remove the keyboard and mouse as part of the reason why we get RSI and discomfort when we're working with computers.

0:55 But it's not that simple. Even the industry that tries to sell you solutions to all of these problems is riddled with issues and errors. And I looked into the science of all of this and I ended up kind of just forgetting about the whole thing and my sort of final take on this was just to make yourself as comfortable as possible and just do all the exercise that you need to do alongside of that. And that's based on this scientific study that demonstrates that there's not a problem associated with sitting itself. The problem is not doing exercise as well as sitting. In groups of people who sit all day but do enough exercise, they showed no increase in mortality associated with that. So, the claims from the standing desk industry that say sitting itself is the problem, it turns out that that's not even right, anyway.

1:33 But all of that becomes irrelevant with what I'm going to talk about in this video, which is how I'm now using these. This is the Even Realities G2 smart glasses as a way to sidestep the entire problem with office ergonomics. Not a sponsored video, I bought these with my own money because I could see the potential and I wanted to really test this for myself. These things have microphones in them and with what I'm about to explain, allows me to use my voice to control my computer and the screens inside the lenses allow me to see lots of text and code and images and all of these kinds of things in response. So, if you think about these smart glasses as a kind of input device for a real computer doing your real normal daily workflow, this changes the game. It lets me work with my computer in any position around the house in comfortable seating, in active seating, while working out, while doing the dishwasher, whilst sitting in the sun breathing in fresh oxygenated air. All of these things that we just have never been able to associate with working with computers and now on the table. So, I've been a software developer my entire career and I've always wanted to optimize this. I knew I knew that working with computers day in day out for my whole life is was have this massive potential to cause problems with RSI and all of these things and I've always been aware of the threat of that as I'm working with computers. And that's why I spent so much energy looking into the best way of using computers, custom keyboards and all of these amazing solutions and all of the research around the the zero gravity workstation, the standing desks, the treadmills, all of these things I've tried and all of these things I've realized just move the problems around.

3:01 They don't actually solve the problem. If you're sat in one position for the whole day and and I've tried mixing it up with sitting and standing desk, you know, that gets you somewhere, but you're still in those positions for the time that you are. For example, if you kneel while working, it's great for the first 10 minutes, you feel like you're doing yoga. After the first 10 minutes, you're in absolute agony because you're again stuck in that same position. So, whatever happens, even if you've got four or five positions through the day, it's not enough because they're the same position each time and that's just because you're stuck using a keyboard.

3:29 The keyboard is the problem. The keyboard is the biggest thing that I've tried to optimize over the years. It's also the absolute problem. It's the thing that keeps your body in one position. Get rid of the keyboard, the entire problem is solved and that's what these smart glasses do. So, what's fascinating about this particular kind of smart glasses? These are not like the sort of Apple Vision Pro things that you strap onto your face and really heavy and uncomfortable you can't possibly do a whole day's work in them, and and and they block you off from the real world and all of this.

3:56 This is the opposite approach. These are ultra light, they're 40 g. They last all day on the batteries that they've got. They're not connected by a wire to anything. You wear them as your normal glasses, and this was just enormously appealing to me. I think this is the way technology should be working. It should be helping us get back to integrating into the real world, not putting more and more barriers between us and the real world. So, this was very appealing to me. And what's amazing about them is the display. So, these have displays.

4:22 Unlike the Meta smart glasses that are getting the attention, the ones with the cameras, these don't have cameras, and I'm absolutely happy with that. I don't think we should be going around wearing glasses with cameras, but they do have a screen. They have this waveguide technology, which is like the fighter pilot thing, you know, the green text that displays in the lens. You can see straight through these and use them as normal glasses, but when you focus on this projected image, which by the way is about 3 to 5 m away from you as far as your focal position of your eyes is concerned, which is something I've always been very interested in in terms of reducing eye fatigue. It's really important that your your resting point of of focus on something is is far away, further than a meter away. Otherwise, you get eye fatigue from using a screen too close all day. So, that's a problem that these really elegantly solve. And of course, because they're just lightweight on your face, you're not actually holding your head in any one position anymore. So, your neck is free to move naturally as you would as you do when you walk around. And because they have microphones in them, of course, we can use it as a voice input method. And this is where the technology starts to get really interesting because the way these are solved at the moment doesn't really solve any of these problems. I had to build custom software for that.

5:29 And I've made that software available to master level members of this channel as well. More on that in a little bit. So, if you look back over my channel, you'll see all of my attempts at solving this kind of fundamental problem with working with computers. I've tried a standing desk. I've tried the treadmill desk. I've tried floor sitting to squatting to standing desk where I would mix that up through the day. and then I've tried this zero gravity idea, where you try and make yourself as comfortable as possible using a reclining chair to draw on the research that NASA did in the 1970s looking at the zero gravity body posture, which is your kind of ergonomically neutral body posture. Course, we're not actually in zero gravity, and it turns out the biggest problem with that is the weight of your arms pulling down on the arms of the chair, and you end up with pressure at the elbow again still. And And this course this is because your arms are in that fixed position using the keyboard.

6:13 So, this whole journey that I went on, you know, it it's just been moving problems from one area to another, and I think actually probably that's ultimately what's helped me because I have always been doing something slightly different over the years. But of course, it doesn't really solve the problem of being stuck in one fundamental position because of the keyboard. And this of course is what the ergonomics industry doesn't want to talk about, that the keyboard is actually always the problem and never the solution. And the only way to really get around this is to just stop using the keyboard. And the time is now. We've never been in a better position to really embrace using voice as an input method. and little things like this, you know, sort of small-scale macro pads and things in conjunction with that to bridge the gap a little bit, which I'll get on to in a second. And we have technologies like Whisper, which is a voice-to-text transcription system that can run on any modern desktop computer.

7:01 It's private. It doesn't send your audio to the cloud to process, and it transcribes in real time. And this is what underpins this entire system. So, to pull this off, I had to build custom software for these smart glasses. They do not come with anything that comes close to what I've pulled off here. And this entire software stack is available to master level members of this channel as well. So, if you wanted to to test this out to get started, if you've got these glasses, you can jump in use my software as a starting point for your own custom version. And I think that's the key here. These tools work so well because they become customized to you.

7:31 We're not forced into some off-the-shelf idea of how this system should work. We all have our own workflows on our computers. And And building your own software stack into these glasses is is the best way of achieving that. So, in terms of the kind of architecture of how that works, what that looks like, I have a whole bunch of scripts and the app itself running on my Mac upstairs. I access that over Tailscale via my phone, and it's the phone that drives the glasses. So, on the phone, there's this Even Realities app, which is what connects to the glasses and projects text into the lenses, essentially. And they let you build custom software in there using their SDK. So, that's sort of the bridge between the tools running on the Mac and how I see that screen displayed to me in the lenses of the glasses. And the app on the phone can take the audio feed from the microphone in the glasses, feed that back into the local Whisper model running on the Mac, turn it into real text, and then the scripts can push that wherever to do whatever. And in fact, I have two layers in that system. So, one is actually a sort of grammar layer, where I can control the entire interface just using the grammar verbs that I set up. And then the other layer is the normal full-text dictation, which will turn my normal language into a prompt for AI.

8:35 So, just look at four key things that I built into this that really propel it away from the kind of out-of-the-box, pretty vanilla version of what these are capable of into something that is actually something I can use for the bulk of my day and means I don't have to sit in front of the computer. Of course, it's not a full replacement for a computer, and that's not the point. The point is that it takes most of my workflows, makes it possible to do those away from the computer. I can still jump on a real computer to do anything that I haven't built into this kind of pipeline.

9:03 So, the first thing that's just really cool is the ability to view images. So, if I'm working in a Claude Code session, for example, developing something, and I want to see what it looks like in a web browser, I can ask that Claude Code session to send me a screenshot, and it puts the image onto the display in the glasses. Of course, it's like 16 levels of of color. That's enough, though, to see the detail in a screenshot. but it didn't stop there. I built this pan and zoom system that lets me use this macro pad, and you know, this mapping is obviously all in the details of the software, which you can download if you become a master-level member, but it lets me pan and zoom around that screenshot in the lenses. And this is just amazing. It feels like Blade Runner to be able to do this while just being so free and mobile physically. And so it's not just screenshots this is useful for. This is useful for you know you you go into a message and someone sent you a photo, you can just see the gist of most images and things if people send them to you in emails and this just lets you keep that momentum with your task management and processing inbound actions and all of this kind of thing.

9:59 But I didn't stop there because there are obviously all these times when you do need to see a full detailed high resolution image in color. So for that I actually have a system set up on my phone that is a mirror of that. Whenever I have the system send an image to my glasses, I can open this little just web app that I have saved on my phone and that's constantly mirroring that image and I can zoom in and I can swipe through multiple images on there as well and pan around and see the real source image. So it's only ever just grabbing another screen to augment this pipeline.

10:29 And what's really cool about that because that is just a web app, I can even do things like run that on the TV here. So I do it in a kind of progressively enhancing your experience just by tapping into the screens around you and having them automatically keep up with the workflow that you're engaged in on the glasses. It's just absolutely fantastic. So the next thing I wanted to do on here is to create the system where I could actually enter text as if I was typing on a keyboard. But I'm using this five key macro pad. Just something I have when I'm using the glasses. I have this with me the whole time and I actually just drive most of my interface with this. But sometimes it's not enough just to use the voice to text system and you want to just enter like a file path or a website address or fix a a word that was ambiguous in the way that it sounds if it's transcribed. With this I developed a system that lets me enter text with this at up to about 40 words a minute using just these five keys and it's just so cool and so easy to get your head around how to do it. And I drew on this idea of the Annie Hall test nine grid layout which has been used by keyboard called MessageEase on touch devices for ages. So the idea is that these four keys actually map to the nine the grid of nine positions super easily, you know, with the combos becoming the middle one and the corners becoming the corners and all four becoming the central and then modified versions of those for the corner versions. So, you can hold down those positions or modify them with the fifth key for the opposite. So, you end up with all of the inward and outward swipe equivalents from the message e-style keyboard just using this. So, you can enter text accurately and of course you can get all the punctuation on there as well just again with this. So, with filling in the gaps you know, the edge cases.

12:01 The edge cases is where something like this would always fall down, you know, we all know we could get reasonable dictation from our voices, but without the ability to fix things or enter accurate text in those edge cases, we'd have to go and find a laptop just for those few times, whereas this lets you bridge the gap. And it's the same kind of deal with the image viewing and the screenshots. So, what I actually ended up doing here is building a complete suite of apps.

12:23 So, I have and I have for a while had these web apps running on my Mac and I would use those for things like processing email. I found my own workflows built into my own apps were just better for me than using off-the-shelf things like Apple Mail. Apple Photos was another one that I rebuilt in this way, but of course that doesn't really translate so well to the smart glasses. But for the more work stuff, email and to-do systems and managing tasks and these kinds of things and working with my tmux sessions, my real tmux sessions holding cloud code sessions on the Mac, all of that translates really well. So, I have these sort of apps running on the Mac and they expose APIs to the central application that I have set up for the glasses. So, all of the data goes in and out of this and I have this kind of almost like a little mini operating system now that lets me switch between all of these things. And I've even added support for my messaging system by using the Beeper application on my desktop, which has its own API. So, I can just easily send and receive messages from all of the connected channels that I have via the smart glasses as well as email and to-dos. And I built things like weather apps as well just because it's so fun.

13:23 One more thing that I integrated into way that this works is this grid tree menu system. So, whenever you're kind of dealing with menus, the main application switcher is one of those, but obviously things like tmux sessions is another, and these there's always a way there's always a place where you need to choose from a menu. So, I can have 16 items on screen, and then I use the the four-key subdivision system from the macro pad.

13:44 The way this works is the four keys on here subdivide by four each time. So, if I have my grid of 16 items on the screen, tapping the top left will give me options one to four on that top left items of four, and then I tap one to four again to create that final selection. So, with two taps, I can go from 16 to one, and this is just such a brilliant mechanism for navigating around an interface using just these four keys. If you want to learn more about kind of custom keyboards and how I built this macro pad, there are all these other videos that I've done on this. but this is essentially the same kind of process that I use for all my old custom keyboards. This is powered by a nice nano controller, and these are just regular keyboard switches, and this is a 3D printed case. All of the parts and the PCB plans and everything for this are included for master level members as well.

14:28 So, what's fascinating about this is how all of this works with this idea of the Unix philosophy, which is something I'm super interested in. This idea of things become better when they just do one thing well, and this is the perfect example of that. We have this simple, tiny, lightweight device that is simple and tiny and lightweight because it only does one thing. It only tries to do one thing. It's not trying to give us any computing power in there. It's not trying to give us a full-color screen.

14:51 It's happy with just displaying us green text over our real vision, and that's enough, and it has the microphones. It doesn't need anything else. That's why it's only 40 g, and that's why it lasts all day. And then we use our real computer to do all the real work. We're not trying to shoehorn that real power into something like the phone. The role of the phone is simply just to sort of deal with the connection through Bluetooth to the glasses, and then network requests back to the Mac, which does all of the heavy lifting. So, everything is playing to its strength.

15:17 This is technology working in the way that it should with everything doing just what it needs to do. And of course that does mean that this way of working can't accommodate every workflow that a full computer can with a big screen and a keyboard. And this will of course differ from person to person. Software developers using Cloud Code will get a long way with this, which is of course what the bulk of my day is. But what's fascinating about this is things like video editing for example, where of course you can't possibly work with something like Final Cut Pro through these glasses.

15:46 But you can work in a Cloud Code session and you can build skills that lets that Cloud Code session do things like build a timeline for Final Cut Pro. And then with my companion app running that displays things on screens, whether it's the TV or my phone, I can get Cloud to play me an edit that it's working on. I can scrub through clips and I can dictate back what those clips contain. And then I can let AI go even further than that and extract clips from my B-roll library. I can do that by setting up this kind of a workflow. So there's all sorts of ways you can look at the technology here. You know, when you think about something like AI doing the bulk of the work and then you use the screens around you whenever you need to upgrade from just the text output of the AI, it's almost like anything becomes possible. You know, I can I'm I'm quite close to having a full system for for editing these videos using this, just using the screens instead of sitting in front of the computer. And that that is just absolutely wild. so do stay tuned for the rest of this journey. I'm going to go into more detail on all of the individual parts of of what I've built here to pull this off. And I've already done a bunch of other videos looking in more detail at these parts of the system as well. Let me know if you have any questions in the comments and I look forward to seeing in the next video.