Entering text with 5 keys on the smart glasses system that changed my life.

0:00 So, you're looking through the lens of my display smart glasses. This is the Even Reality's G2. you know, you probably noticed there the screen was out of focus. That's because the camera's focusing on the glasses up close. The actual kind of virtual focal distance of the display is more like 3 to 5 m away from the lens. So, these are incredibly comfortable to use for long periods of time. And this is just something that I've I've been exploring a lot. I' I use a 55 in computer monitor you know over a meter away from me just as my normal display when I am using the computer. This has kind of changed the game again. And of course, we have really accurate voicetoext whisper models that can run on our own machines now as well. So, we have this voice as a text input method. And now, obviously, o over the years, I've I've just been super interested in the idea of exploring how to improve the ergonomics around working with computers. So, the idea of removing the keyboard completely from the mix and replacing it with a reliable voice input at the same time as removing the need to sit in front of a computer screen at all. You know, this is why I bought these. There just seems to be such a huge potential here. And over the the past couple of months, I've been building a custom software stack to run on these glasses to allow me to do that.

1:08 Obviously, you know, using voice as an input method, it requires so much more than you know, just essentially just running the Whisper model. you have to work out how that actually maps to your real life workflows. Now, as part of that, I've built this. This is a a five key macro pad. I started off with just the four keys. I added the fifth key for reasons that I'll explain in a second. And this has kind of completed the mix. I can just hold this comfortably in one hand or the, you know, either hand. It's not sort of sided. It's it works just fine in either hand. and you know, with the five keys, obviously with combos and taps and holds, we have a lot of different input sort of mechanisms that we can use here. So, I'm not relying on the voice for everything to do with the interaction. Although, I have built that layer into this as well. I can use pretty much everything without this at all. It just means I'm doing things like saying page up and page down all the time in between my actual kind of text dictation for for prompts and things.

2:02 in particular, things like processing my email where I'm archiving and deleting very rapidly. I don't want to be saying delete, delete or delete five email. You know, I have got that mapped in, but it's just much easier and faster to use this. So, the combination of this and the smart glasses and the custom software stack that I built for them has actually changed my life here. I'm using computers in a way that I've never done before. It's in one fell swoop completely solved all of the problems I have with ergonomics and computers, which of course stem from being in one fixed position. You know, I did this video looking at the reclining NASA inspired zero gravity comfortable reclining environment. It's it's good.

2:37 It's not perfect. I'm still dealing with kind of pressure at the elbow and these kinds of things. And the fact that you're sat in a chair for extended periods of time. It's kind of uncomfortable to sit in one position for extended periods of time. This changes that. I can walk around. I can do the dishwasher. I can sit in the garden whilst I'm being super productive coding through clawed code. Of course, we're relying on the AI era to make this happen as well. you can't actually write code with this. but you can prompt into Claude and of course Claude can write the code. So this is you know all things coming together. We've got the technology of the waveguide display glasses which is you know super light with all day battery life and light enough and comfortable enough to wear all day at the same time as the AI stuff enabling us to actually get away from the keyboard as well. so let me just demonstrate in this video. I'm going to look kind of primarily at one feature that I built into this and I'll I'll do videos on all the other bits. There is just so much that I put into this software which of course is going to be available to master level members on this channel. I haven't quite sorted that out yet, but it's coming very soon as I just sort of finish off a few little bits on the software. but I'm going to look now at the way that I've created a full text input layer for this macro pad. This is just super exciting.

3:46 So obviously with with dictation and and voice to text, you know, it it works like 95% of the time and then you come to trying to spell out a website address or something or or a file path or something like that and and it trips up and it's just super annoying because if you've set up something like this where you haven't got access to to a keyboard, obviously you could jump onto your phone and type and do that kind of thing as well. But I just wanted to just create something where I could input text to to kind of fix the little prompts like that using the macro pad. What I ended up doing is creating an input layer that lets me type at about 30 to 40 words a minute using just this. And this is just absolutely wild. So it's based on the same kind of idea as the messages keyboard, you know, where you have a grid of nine commonly used letters and then you have this idea of inward and outward swipes mapped to that same nine position grid. So this is what you can see on screen. Now you're seeing what I'm seeing in my glasses. So as I'm demonstrating this, I will be looking up. That's I'm looking at the waveguide.

4:41 The interesting thing about these glasses is the screen, the virtual screen is positioned kind of just above your natural ey line. This makes it very easy to use these glasses as a daily glasses. Then there's nothing in your way until you look at it. And there is adjustability on here as well, so you can kind of position it where you want. I actually end up when I'm working, I kind of put them down here so the screen is kind of in a more natural line. I'm not actually holding my eyes up all the time when I'm working with these, but if I put them in their normal position, it's kind of above my ey line like that. So you can see the grid of nine key positions on screen. So if we think about this, we've got four keys here. Four keys maps to nine positions really logically. Of course, your four corners map to the corners and the ones in between those map to the combos. So if I tap the top two keys here, I get that top central position, which is the letter N, A would be the the top left and I would be the top right. So that's that outer ring sorted. And then for the central one, of course, you just tap all four. super logical mapping for those nine positions. So the reason I added the fifth key here is so that I can now add the modifier. and this in conjunction with using holds instead of taps gives me three ways I can represent those nine key positions. So the tap is the tap. the holds are the inward swipes which is represented by the tiny letters on the inner corners of those outer positions. And the fifth key modifier on a tap is how I get the outward swipes. So that's the little ring of letters around the O key. So that's the outward swipes in the same direction that maps to those nine positions. So this is just a really amazing. It's so easy to get your head around where that mapping sort of corresponds to. I I've almost memorized this. you know, I can kind of look at the line that I'm typing. Okay, so I'm just going to do a little demo of this.

6:25 I've just cleared the last screen. I'm going to do a fresh demo so that you can see this. I've made this display my words a minute. I probably won't hit my my best personal best, which was 40 words a minute on one of these fourword quick tests. Obviously not super representative of long-term, but let me just give you a little demonstration this. So L is that combo the outward with the thingy with the fifth key holds for the inward swipe equivalent.

6:54 And there we go. So that was 17 words a minute. Yeah, not bad for a little demo under the pressure of doing a video, but you get the idea of how how this mapping works and what actually a brilliant way of inputting text to something like this with a tiny lightweight macro pad. I'll be doing more videos on all of these components. You know, how I build a software stack, how I get this to work even with my phone turned off in my pocket. This was a big hurdle with Bluetooth. It doesn't work like that with normal Bluetooth keyboards. There is a way around it. I've got all that built and working as well. This is super fast. the the latency is bizarrely, you know, incredibly incredibly low. you know, for this to go from this to my phone, from my phone to the server running on my Mac upstairs and back through all the Bluetooth pipelines to the glasses, you know, the whole system works incredibly well. It's incredibly fast. So, just in terms of kind of framing of the hardware a little bit, if you're thinking about these kinds of glasses, the best way of thinking about them is that they're just actually a pretty simple, straightforward piece of hardware.

7:51 They're very light. They're they're very comfortable. You can wear them all day. They work has normal glasses, but they have this waveguide projection, which means there's a little projector in here that shines into the side of the glass. This is how we have this optical system where you have a projected image in your normal clear vision. it works very well and and like I said earlier with the positioning being just above your ey line, that's how this is sort of, you know, a general device that you can actually wear whilst doing anything around you. so in terms of what they're actually doing, if you just think about it, these are a Bluetooth audio input device, right? So, they haven't got any speakers, but they have got microphones, four of them. I think gives a really good Bluetooth audio to your phone. Then on your phone, because they've opened up so much of the development part of this, you can just build your own custom app. Super easy.

8:36 Obviously, you know, vibe coding, whatever. You can have this stuff done pretty quickly. you can then use that audio feed, process it locally using a large language model transcription tool, and then obviously from there, you're just building custom scripts to do different things with that text. so essentially this is, you know, it's a it's a way of displaying a formatted piece of text. We have 10 lines of text here to work with. It updates really quickly. You know, it can display images. It can display text.

9:02 Some of the fun things that I've got with images. I have when my doorbell camera goes off. I have the picture from that displaying on my glasses. That'll be another video as to how I pulled it off. Wildly cool. I I can also tap into the camera inside my 3D printer. I can see that updating and that updates, you know, every few seconds, you know, a different image. So, I can kind of see the print head moving around and check for any issues on prints that I'm doing in the 3D printers. All these kinds of wild things that you can actually pull off because you do have total control over what you do here. This is all running inside the idea of what they consider one app, right? So, I'm kind of just bypassing everything that they include in their interface. I'm just running my entire pipeline through my custom app, which supports all sorts of things from emails, messages, you know, through to working with multiple T-Max sessions on my Mac and so on. So, I know they ship with this terminal mode. It's just obviously just way too lightweight for what I want. I want that full kind of replacement for the the computer. That idea of sort of taking 90% of my computer workflow and packaging it through this ultra streamlined small display output, you know, lightweight key and voice input.

10:08 That's what's the game changer here. so, do stay tuned for obviously many more videos on all the different components of that. This has been a little introduction and a look at how I've mapped the typing thing. which is just super cool, but there there is a lot to get to here. and of course, yeah, if you become a master level member of this channel, you will get access to everything that I've built here. It's a great starting point.

10:29 if you want to go down the same path as me, obviously, you'll probably want to tune it a little bit to your own specific workflows, but I've solved a lot of really annoying, complicated problems, bugs that are still in the glasses firmware and things like this. So, it's it's going to be a great starting point for your own projects here and and hopefully you'll you'll see value in that. I'm going to get that into my members perks package within the next couple of weeks. So, if you do join now, you will get that before your your month expires. Although, obviously, I'm going to be continuing to work on this over the coming years, I would imagine. So, you're going to get access to the latest version if you if you support me through that mechanism, which is super appreciated. so look forward to hearing what you think in the comments and I'll see you in the next