My custom Claude Code smart glassses workflow

0:00 In this video, I'm going to demonstrate how I've set up a custom script system to control my Claude Code sessions using these smart glasses. These are the Even Realities G2. This is not a sponsored video. These are mine. I paid for them with my own money. the software I've built for this, including the macro pad, the Gerbers for the PCBs, and the CAD files for the case and the software that runs on the phone to make all this work, is all available to master level members of this channel. So, your support is super appreciated. And that's obviously a great way to kind of get started if you've got these classes and want to set up something similar. So, I'll jump straight into my sessions part of this and sort of demonstrate what it's what it's like to work with Claude through this. So, what you're looking at now is basically all the T-Max sessions that are running on my Mac. So, I wanted to set this up in a way where these glasses really feel like they're a extension of the computer. Like this is just another input device, right? This is just a screen and a keyboard essentially.

0:49 you know, it's the real Mac. It's it's not any sort of custom siloed pipelines. The glasses come with this terminal mode app where you have to start your session inside that app and it's just it's not like it's not a true extension of the normal setup that I have running on my Mac. So that's what I've set up here. So I'm going to jump into the top one here. This is a little demonstration project just so that I can show you you know how it works. So you can see me scrolling up and down here. What this part of this system is is actually it's so every time you're in claude code it it saves your entire session into a file and it constantly writes to that file.

1:24 That file becomes a really good reference to take the content that we need to see on the glasses. So the script that I've set up just literally reads from that file and sends it to the glasses and I can use my page up and down commands here to go through that entire transcript. And I've set up some cool stuff like for example now where you see that little square icon. that's I can do show more and it's like a a extra bit of text. So any large documents in you know that are additional to the main transcript I can kind of jump into and then read those.

1:52 And that also works with images. Let me just let me just ask for an image actually. Oh there's one already here. So we can see here. So I do that it shows me a screenshot. and what's really cool about this is I've built this mode where you can zoom in and pan around on the screenshot. so we're just go to the end and I'll show you the whole transcription mechanism. So I'm just going to ask it to summarize the project again or something like that.

2:14 Can you summarize this project again? I'm just doing a little quick demonstration to demonstrate how effective my whisper transcription is. So all the while I'm talking now, we're seeing this real time preview. And then when I stop talking, it will time out and do its final run. And this text will be ready to send into the Claude session. So, where you just saw stopping in at the top of the status bar there, that normally gives you a countdown to stopping in 3 2 1. So, I know there's a 3second window at the end where I have to be silent for it automatically to time out like that. And then it does the final transcript and then it says ready to send. So, if this is more than will fit on the screen, I can page up and page down over this. so you can see it's, you know, it's kind of like a word for word perfect transcription. And this is using a local whisper model running on my Mac. optimized is the OP version that's optimized for the Apple silicon metal. It's a Mac Mini M4 Pro for reference. So, you can see what that's kind of capable of. I just think the performance of that is just so wildly good considering it's not going to anyone else's service is entirely local.

3:15 obviously, you know, I'm now going to push it into Claude. So, I I send that this screen that we're seeing now is the the prompt in Claude Code. It's the real Claude Code session inside the T-Max window on my Mac. And I can do this, which sends the enter command. So that's just holding down. Obviously the mapping for the macro pad is all in the files for the master level members as well. So if I page down now, we'll see my text there that I just sent. and you can see it's already responded. And we can scroll down. This is obviously just a little demonstration project. So it's, you know, you can see how much text we see. I was already serving.

3:48 it's telling me that it's serving this little demonstration app. We've seen the screenshot already. So that's essentially it. So you can see how I can take my voice, turn it into text, push it into a real claude session, and see that response. So obviously we're just dealing with this kind of green text view, right? So a screenshot has its limitations. it's obviously very easy to just grab your phone and look at the Claude code app on there using the remote control session. You can see those screenshots in full color on your phone screen. So we have this kind of progressive enhancement approach to working with this obviously ultimately leading me to be able to just jump onto the Mac itself and carry on in the real same session there. so this has just you know just been an amazing little journey. This is just one part of the whole app ecosystem that I've built for these glasses and there many other videos on this channel and I'll continue to do more looking at all the individual parts of this. I think what I've done here is just it's been so transformative to me to be able to finally work you do most of my day's work away from the computer. You know no longer sat in this fixed position which leads to all of these problems I've been trying to solve over the years RSI and fatigue and discomfort when working with computers for an extended period of time. This is the biggest thing to happen in office ergonomics the way I see it. So, thanks again to all the members of this channel. Your support is so appreciated.

5:00 I've now got a private Discord server for members of all levels on there as well. So, you can jump in and we can discuss all these things in more detail on there. It'll be great to see you over there.