transcribe

Unlock your agent's full capabilities with Browserbase & Box

Browserbase · 1h 0m · transcribed 14d ago
More from Browserbase Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

Webinar Introduction

What is the purpose of today's webinar?

The webinar aims to showcase how to connect Browserbase and Box to create powerful agentic setups.

  • Introduction of Chris Kim and Kyle.
  • Overview of the agenda including demos and Q&A.
  • Encouragement for audience participation through questions.
# 12:00

Browserbase Agent Functionality

How does Browserbase enhance agent capabilities?

Browserbase allows agents to autonomously control a browser, improving efficiency over traditional methods that relied on screenshots and coordinate clicks.

  • Browserbase uses code to control browsers, enhancing automation.
  • Agents can autonomously navigate and download data.
  • Dynamic workflows allow agents to adapt to changes in web pages.
# 24:01

Action Caching and Self-Healing

What features improve the efficiency of Browserbase agents?

Browserbase includes action caching and self-healing capabilities, allowing agents to repeat actions without additional calls and adapt to changes on web pages.

  • Action caching can save up to 30% in token costs.
  • Self-healing features reduce the need for constant script updates.
  • Browserbase can handle complex web structures like nested iframes.
# 36:02

Navigating Anti-Bot Measures

How does Browserbase handle anti-bot challenges?

Browserbase is working to ensure its agents are recognized as trusted by anti-bot providers, minimizing disruptions like CAPTCHA challenges.

  • Browserbase aims to bypass common anti-bot measures.
  • Collaboration with anti-bot providers is ongoing.
  • The platform is designed to maintain seamless user experience.
# 48:02

Integrating Box with Agents

How can agents utilize Box for enhanced functionality?

Agents can access and utilize files stored in Box, allowing for context management and memory retention across sessions.

  • Integration with Box enables agents to work with company files securely.
  • Context management is crucial for AI agents to remember past interactions.
  • Real-world applications include market analysis and research tasks.

Transcript

0:04 All right. Hello everyone. Thank you so much for joining us today for this webinar. we got an exciting one today because we are joined by Browserbase today and we're going to be talking about, you know, some cool ways that you can connect Browserbase and Box to your agents and create some cool agentic setups. But yeah, hello everyone. My name is Chris Kim. I am a senior developer relations engineer here at Box and I do stuff like this. I do webinars. I teach developers. I help developers on how they can build with the Box developer platform. Now, today we're joined by Kyle from Browserbase and he's going to be telling us what Browserbase is and showing us some cool demos on how you can use Browserbase. So, yeah, Kyle, could you introduce yourself to the group?

0:50 >> Yeah. Yeah, of course. My name is Kyle. I am a growth engineer over at browser base focused on developer experience doing a lot of similar things making sure that our customers and our users have a great experience using our product. >> Amazing. All right. Well, let's kick this off. we have a very simple agenda today. we're going to start off with Kyle showing off some demos of browser base and then we'll head over to some box demos. And both of our demos will showcase how you can use both box and browser base at the same time to set up a very powerful agentic setup.

1:28 And then we'll finish off with some Q&As's to answer your burning questions. And yeah, that will be the webinar for today. now before we hand it over to Kyle, just some quick housekeeping. If you have any questions or if you want to ask anything, use the chat feature that we have in Zoom and we have people in backstage who can help answer the questions. but if some of those questions are worth mentioning live, we will address them as well. Amazing. I will now hand this over to Kyle. Kyle, go ahead and, you can share your screen and start your presentation.

2:07 >> All right. Yeah, thank you, Chris. yeah. So, today I'm going to be talking about both browserbased and box. And you may have seen browserbased before, you may haven't, but I'm here to tell you what it is and what we are. Let's see, next slide. well, so browserbase is the complete platform to run agents that can use the web like humans. what's on this slide right now is from the Cloudflare CEO Matthew Prince actually went and pulled the usage of the current internet and we're kind of seeing this trend where the new user of the internet is now bots or agents rather than humans. And so 60% of the traffic that is coming to Cloudflare which covers a large percentage of the internet is now AI agents.

3:02 But the problem with agents accessing the web is that most of it doesn't have a clean surface or API for agents to use. So a lot of the internet is blocked behind capture walls or they information that agents need to use is behind authenticated or login portals. And so agents are unable to access a majority of the internet where a lot of knowledge work today actually goes on. And so browserbase is building the platform to build browser agents which essentially are just any kind of agents that use the the internet or use a browser to complete work on behalf of humans. So think of any mundane task that you do at work that could be automated likely is easily automatable with browserbased and the browser agents that you can build on our platform.

3:57 And so we give you all the primitives that you would need to give an agent proper internet access starting with search and fetch. So you've probably seen your coding agents be able to search the web for documentation or search the web for the latest version of a package that you're using when you're writing code. And so search is one of those early primitives that allows you to read from the internet. But when you actually want to write stuff to the internet and take actions on the web, you need a browser. And so we provide headless Chromium browsers to act on any website that's on the internet.

4:40 One of the things that people don't really know about that you only find out when you try to automate the web the first time yourself is agent identity. And so as the world becomes mostly agents doing all of the work on the internet, these agents right now are doing so on behalf of their creators. And so if I have an agent and it goes out and does something bad, it's on me. But we want to give agents their own identity. And what that starts with is giving them like basically the global password to access first all the pages on the web and then the authentication.

5:20 So staying logged in to accounts. So when agents have to go to let's say your Amazon account, it doesn't have to you don't have to provide the password every single time. You don't need to do the 2FA code to your agent. So agent identity allows us to be trusted and authenticated into all websites. and then finally we have what I'll be talking about and demoing today which are stage hand and our agents products. So our we'll start with agents. Our agents are a managed agent service where you can basically provide a prompt, a task and then kick off an agent with a single API call to go and search, fetch and browse the entire web to complete your task. and stage hand is our SDK for building browser agents. So a lot of our customers actually work on their own agent harnesses and they wanted a version with more control and so we've built stage hand as the best tool set to actually give your agent harness so that it can decide what to do and what actions to take on the internet. And so we'll be doing a a quick demo of those things and I'll I'll do a dive into what those things kind of are.

6:38 Cool. We'll start with browserbased agents. And so browserbased agents is a relatively new product. we've been working on it for the last few months and we released it back in June and we're seeing a lot of larger enterprise adoption because it's very outcomesoriented. You provide the agent with exactly what you want it to do in exactly the schema you want it to output and you're able to run effectively an agent to do step by step everything that you need to do to complete a task and then get to set outcome. But then also we do some agent optimization. And so, we'll go through actually all of the agent traces in the run and optimize that agent to become faster and more efficient for future runs.

7:32 And so now I'm going to show you what this looks like in our dashboard. And so this is the browserbased agents dashboard. You can go and actually create agents. And so I have a whole bunch of different agents that I've created as well as like just running agent tasks like go to Amazon, right? and so today I prepared a demo that will actually use box and browser based agents one to go to my PG& account and then log in and then get my bill and report back to me like is this reasonable? And so I have some code here. and so to actually create and run a browserbased agent, the code is relatively simple. let's open the agent. And so to create an agent, you can do so one in our browserbased dashboard just with the UI that I was just showing you or you can create agents programmatically using our SDK or API directly. And so here we have yeah PG utility bill gatherer for blocks.

8:41 and so what this workflow kind of does is it'll go use agents to yeah go to PG download the thing the my utility bill and then upload it to Box and use Box's OCR capabilities to extract exactly what it says accurately and efficiently from the PDF. And so let's do a test. So I'm going to be demoing another product with this as well. what's called our context API. And so, this is a really crucial part of identity that I was talking about earlier. That is the authentication.

9:15 And so, when I want to log into a website, I don't want my agent to have to log in again and log in again and then send me the 2FA code every single time. it kind of defeats the purpose of agents running autonomously and in the background. And so what we can do here is I I've written a script that will allow me to just spin open like a browser session to first log into my PG& account. So like this will be the first time I'm going in. So this is just a an emulated version of our browser headless browser. Me go here. And so I'm going to sign into my account. See if it remembers me.

10:01 I think it'll remember my email. Yeah. Okay, cool. Should be able to. And now I'm going to set up the two or should be good to go. Okay, cool. now we're going to save this. And so with contexts, a lot of our customers use it to yeah log into a portal once and then for future runs, we can save the basically state of the browser so that you can be logged in every single time.

10:47 Cool. And so now we have the context ID. and then every single time you want to run a browser session, you can attach this context ID to the browser session for any session or you can detach it and you can create hundreds, thousands of different contexts for different login, different portals. All right. Now, we're going to actually run this agent live demo. So, we will hope that this works well. and so like from my terminal, I've invoked the browserbased agent and we should be able to just watch it. Here we go. Running.

11:28 here is the agents dashboard UI live view. and so how our managed agents work is the first thing it does is it breaks down the task and the prompt to see, okay, what's the fastest way I can do this? Do I even need to open a full browser session? like can I get the user their answer by just using our cheaper primitive search and fetch and so it's going ahead and choosing yeah does it need a search and fetch or does it need a browser and so for this task specifically it does in fact need a browser to go and pull down my information from PG& and so it is going to open a session and stream it live we'll see in a But the agent that runs and executes in this environment is actually running code against the browser to control it.

12:26 And so if you've seen traditional computer use or any kind of the older versions where OpenAI had released operator or the first versions of computer use from Enthropic, those versions of computer use actually used screenshots and coordinate clicks. And so they would train the model on basically XY coordinates on the page where you'd be able to click I don't know 800 800 and that was the way that these models actually control the computer. But with our agents product we've actually found that writing code writing usually playright or stage hand code which is what I'll talk about later is the best way to control a browser. So you can see actually with no hands right now browserbased agents it's going into my account. We're going to see if I'm I'm paying a reasonable PG& bill this month. but it's going in and it should be able to download the PDF from this billing history. and so when you download on browserbase you can download to our S3 bucket or you can alternatively bring your own storage on our enterprise plans.

13:40 So let's see. It should load soon. and with agents being fully autonomous, if things do fail during runtime or if the pages change, it is a dynamic u workflow and so it can update. We'll see. Here we go. This is what a San Francisco two-bedroom, two bath P Gen electric bill looks like. All right, cool. And so the result here I didn't specify like a schema, but you can alternatively specify like a JSON format schema that you can get proper output into. And so, but this agent navigated to the accounts, open the billing history, viewed the PDF, and was able to download it. Cool. and so with these agent runs, I can there's an tab called optimize. And so with this optimization, I can have our second agent, it's an outer agent loop that basically looks at the inner agent loop to see, yeah, how can we make the session faster? How can we save the user money and time by taking an alternative pathway to get them to the same exact result?

15:02 Cool. Yeah. See, it goes in actually profiles every single different thing. So from the HTTP probe, which is like doesn't need search, doesn't need a browser. the page go to is was kind of slow here. So we can do some optimizations there. but yeah with a single click I can just basically update the agent exactly and run it again so that we can see is this optimization actually a good one. And so if we go back oops we should see okay did not sync.

15:39 basically now if I go to my box account, it should pull exactly the thing that we need from the utility to the box basically file system and then we're able to run OCR against this and have it work to pull the information that we need from the actual PDF.

16:10 Okay. And so with agents, we've seen adoption mostly in enterprise companies that want to use the best agent harness, which is what we build and get provided the best model for every task, which we also built in through our model router. and with this full end to end from runtime to agent harness to the tools to the browser runtime to the outcome.

16:41 Actually agents is like the perfect tool for companies that don't really want to think too heavily on oh we need to use this exact harness. We need it to be I want to only use the new Gemini model. we kind of abstract that away from the customer and just give you the best possible experience. Yeah. And so the second thing I wanted to talk about here today was Stchan. And so Stchan is our SDK for browser agents.

17:12 but what does that actually mean? If you've ever done any kind of web scraping or browser automation, you've probably ran into libraries like Playright or Puppeteer, Selenium. And so these frameworks allow you to do one they allow to control and manipulate a browser via interactions with the DOM the document object model and so these languages were written primarily for testing and so you you would use playright to end to end test front-end UIs.

17:48 but the the thing with stage hand is that we kind of felt that playright wasn't the right tool to give your agents to control a browser. And since play was written for testing, there are a whole bunch of extra tools and bloat within playright that make it not token efficient, one for your agents, but also kind of slow. And so with stage hand, we kind of built it with the goal of introducing AI primitives. One, to allow you to develop this code faster, but to also give your agent the tools that it wants to use. And so these tools are fast to execute, but also to efficient. And so we recently launched stage hand v4, which is our fourth major iteration of our framework. And so the first few the first two actually iterations of stage hand were a superset of playright. And so they were built on top of the playright primitives. but in the recent versions v3 and v4 we've completely migrated off of playright and written our own cdp driver. cdp being chrome dev tools protocol the way that we're able to communicate and talk to the browser. but in stage v4 we've actually moved the full stage runtime into our browsers as a extension as a chrome extension and so what that means is traditionally when you would have your client talk to the browser using stage hand there would be another u node here and so your client would have to send a request to the stage hand client which would then send a request to the browser but with stage hand living inside all of our remote browsers, you're able to send it once the request, the websocket connection, you can connect to our stage extension and then all of these CDP commands that are dispatched to the browser have a much shorter roundtrip time which has allowed us to in benchmarking become roughly two times faster than playright executable actions. And so moving stage hand into the browser also lets us do a couple of cool things. One of those being batching. And so agents and the new models are getting really good at knowing one how to navigate a page, but two like what comes next. So what actually needs to happen to get the full task done. So instead of just clicking one button, the models will know, okay, I need to click three buttons or if it's like, let's say a checklist, I want to click all of them at once.

20:29 Traditionally, we would just go one at a time. And that introduces a ton of round trips to your client in the browser. And so that is a time that we can eliminate using something like batching. And so we would actually say oh the model will go let's click all of them at once. And so dispatch those like let's say five commands at once. And so batching is lets us execute all five at once in parallel rather than just one one one and so on. And so, play, yeah, play built for testing. One of the biggest complaints on the Playright MCP, actually GitHub repo is that you can't use it for very long horizon tasks because it's so token inefficient and the context just immediately gets bloated and you're unable to do anything with your agent after a few turns with the player MCP. And so we do a lot of cool accessibility tree pruning. And so we're able to send a much smaller snapshot of what's actually going on on the page to your agent such that you don't have to deal with your agent not being able to do more than five steps in a row because the context window is too full. And so we're much more efficient there.

21:51 And so I talked a little bit about the model router, but other than the quantitative reasons why Stchan is more efficient than like let's say a playright for your agents to control the browser, we also have a ton more features and we are intentionally building features that our agents want and need to use. And so one of these is a model gateway and router. for with stage hand there is there are three AI primitives that I briefly mentioned before. you can use act which allows you to write natural language queries to act upon a page. There's observe which will let you see like what possible actions can I take on this page. So it would be like if there's a button observe would return that there's a clickable button in the top. and then there's extract, which allows you to extract structured data in JSON format from a given page. And so I can do like title, name, etc. from a blog page. And that's something that I can reliably extract. but with these with these AI primitives, you want to use one, the most accurate model, but two, you don't need to throw Fable 5 or Opus 5 at every single task. And so we've done a ton of benchmarking across a variety of tasks that we've seen customers actually encounter in production and we've benchmarked all of the new frontier models on how they perform and how token efficient they are, how fast they execute. And we automatically serve you the best and most costefficient model for the tasks that you provide us. And so the model gateway router is a huge plus for saving money at scale. We also do domain allow listing here which is basically like really important for the security of your browser agents. I think agents are very good but still at risk of prompt injection and other attacks. And so domain allow listing blocks any URLs on the internet that you do not want your agent to access but at a lower level at the level of the browser. So we block all in and out from the network of certain domains for you if you would set it up. And we also do action caching which is a layer on top of the like prompt caching that already happens with a lot of the LLM enabled primitives. And so action caching lets you repeat certain actions that you had on a page with our from our act commands or that when the page stays consistent, you can just run the same automation over and over again without having to make any more LLM calls in between. and so we've seen customers save yeah up to 30% token cost just using our cache which is automatically enabled when you use browser base.

24:43 we're also have self-healing and so when the pages change since our our AI primitives act extract observe are dynamic they will be able to if there's a cache miss let's say look at the page again and update the action that it takes on the page. So all of our primitives are effectively self-healing. So you don't have to hire a playright engineer or an automation engineer whose whole job is to just update playright scripts. stage hand automatically does this for you. And then there are some other cool things like we're able to talk to nested iframes which is really important for healthcare. A lot of EHR portals are nested iframes and so play is not able to access iframes that are in iframes that are in iframes and so we've gotten really good at that.

25:34 there's something called a shadow DOM which is like a hidden representation of the document object model that u player also can't touch and that we'll see often in legacy and older websites. we have a clipboard so that's copy paste so you can have your agent copy and paste secrets or any anything that you may want an agent to copy paste. and also WebMCP, which is starting to pick up traction, and that's just for any website that wants to expose an explicit set of tools for your agent to use rather than having to automate it from the beginning from using browser automation. And so we're also like O2 compatible. So you can plug stage hand and your agent into any kind of trace engine like lang or brain trust and be able to improve your harness based upon the logs that we give you.

26:32 Cool. I'm going to do a quick second demo of stage hand. this one is using Eve the agent harness from Verscell. It's a very thin harness that just allows you to have like yeah one a file system but also just providing tools via MCP. And so what I built here is a small MCP server that allows you to do three things. One is you can screenshot the page if you need. So the models are still visually capable and so when you use a multimodal model you're able to have it go and parse like okay what is on this page via a screenshot or if I need to get a screenshot from a page I can just have it invoke the screenshot tool.

27:18 We have a snapshot tool which basically allows you to go and take the snapshot of the page which is different than a screenshot because it is a representation of the ex accessibility tree and the DOM. And so this looks more like text of button here, title here. And we're able to see the entire page and give that context of what is on the page to the agent. And then there's a run tool. And this one is different because it allows you to actually run code against the browser. And so instead of providing the primitives that I mentioned earlier, act, extract, observe as three individual tools, we instead just allow your agent to look at the page and then generate the code it thinks that it will need to manipulate the page in the way that will get us closer to the end state and then execute that code against the browser. And so we found that in benchmarking there there are a handful of computer use benchmarks that we evaluate our agents against.

28:24 One of them being online my to web and web voyager. We found that running code like this using stage hand is far better in accuracy but also cost for completing those tasks. And so I'll do a quick demonstration here. And so I've already got Eve running. So let's open a browser session oops session using browser base and we'll see it should be able to go and run code to create a browser and so we'll be able to see let me split my screen here and so this is our session view cool we can here. Let me talk to it.

29:17 Can you go to amazon.com for me and we'll see. Yeah, Eve is the agent is able to run this code. Oh, wait. Page.go and navigate my browser to amazon.com. Can you search for a green jacket? Here we're going to go click into the search bar. I don't know if you can see. I'll make this bigger actually in the Amazon search bar. It's able to go and click on the search bar and type in green jacket. And so, let's see. It's also run code to extract the options that are on the page from the snapshot.

30:02 So, we can see this bomber jacket is this one right here. I think I want one that's instead. And you find a better option. and so stage hand is simply the tools that this agent is using to control the browser. And so you saw there that it ran code to click on the search and then type blue jacket and then click the search button. And so that would be an example of a a batched command from an agent. So the agent would batch those three things together, which is why you, as you saw, it happened so quickly there.

30:44 Cool. and yeah, so Sage was able to find exactly what I wanted from basically a simple prompt there. I'm going to go through the section a little quick. I think I'm a little bit over time, but I just want to talk about security and trust and a little bit more on the identity stuff. Oops. And so obviously we're SOCK 2 type 2 compliant. We have HIPPA and we provide VAS. But one of the biggest things with trust and security is agent identity. And I I talked about it briefly at the beginning, but a lot of the internet is not properly accessible to agents today. And so agent identity is two things. It's both trust and it's authentication. And so what do those things mean? And so trust being like how do we access the internet safely? And so providers like Cloudflare trust us to basically audit our users and make sure that they're safe actors on the web so that we can access their pages safely and securely. And so that comes with something that's called webbot off which I'll talk about in a sec. also partnerships with all the other providers that cover the internets, like Cloudflare that protect people's websites. We partner with them to allow safe and restricted access to the websites that they protect given that we audit and ensure that our customers are doing good actions on the web and not being malicious and doing things that they should not be doing. This comes in the form of verified which allows us to basically sign our browsers so that you can see that they come from us so that the people protecting those sites like Cloudflare can be like okay you're verified you're a good browser.

32:40 and so we also do some other things like capture solving and we also have proxies residential and proxies so that your traffic when you use our browsers don't look like it's coming from the middle of like Ohio or Virginia from a data center rather than coming from someone's house that is in XYZ place. And so we provide a proxy network as well. And so that's kind of the access portion. And then authentication for we do like 2FA and OOTH. So we're supporting those very soon. We have a one password integration where you're allowed to basically embed one password into our browsers as as an extension so that you can let your agent log in safely without ever seeing your credentials. we'll have secrets which will be important for running agent workflows that is coming soon and then also context which I mentioned before and so web bot is the basically open source protocol that allows us to sign our browsers such that a web bot provider like Cloudflare can go ahead and say okay this browser given the their public keys like this browser is signed by browserbase. and we trust browser base so that they the customer that's using us isn't doing anything bad.

34:06 and then they will allow our browsers to basically visit all the protected pages. So if you have used the internet in the last 10 years, you've probably seen the Cloudflare turn style or the like prove that you're a human. And so like this allows us to bypass those things because we're a trusted source. in terms of audibility and governance, we do provide really good observability into all sessions. but if you don't want us to save any logs or save any recordings of the browser during your sessions, we do offer ZDR complete ZDR. And then you can also disable session recording within the session create in the session create paramps. So you can disable session recording and you can disable all like network traces and logs. and then alternatively if you do want logs but you want to save it to your own storage we do provide bring your own storage for things like CDP logs and downloads like I mentioned earlier. And so you're able to bring your own buckets bring your own database to save logs and save files.

35:18 >> Cool. >> Yeah. I don't know how much time I have, Chris, but I I kind of went a little bit over, but we have some Q&A. I think there's questions in the chat I can answer also in chat, but >> yeah, I think I think for the to be mindful of the time. Let's just go over one question and at the end we can address the rest of the question, but we got a question from Yandon Dong asking how do you guys deal with the rate limits from the website? Like will the website detect if it's human or agent is clicking?

35:49 >> Yeah. so we do a lot of yeah so one one thing is the partnership. So we once Cloudflare sees and identifies that it is us that is on their page using our browsers they will allow us to basically navigate and do whatever we need to do or our customers I guess using us would be able to do whatever they need to do in in terms of the website detecting if it's human or agent clicking before before Cloudflare like verifies yes there there's a whole bunch of antibiot providers that will like instantly block. and so we're actively working to work with every single provider of that antibbot layer and allow our browsers to be yeah one trusted so that we don't have to do things like the solving captures etc. And so that's kind of the the gist there. We we we haven't run into a problem like rate limiting because yeah, Cloudflare acknowledges that it's usually.

36:55 >> Awesome. Awesome. Well, thank you so much, Kyle, for that amazing presentation. if you could stop sharing your screen, I will take it over. Amazing. Cool. Cool. All right. Thank you so much, Kyle, to and for telling us everything about browserbase. so, yeah. So we've just learned that browser base is all about giving your agents the capability to navigate the web to use the web interactively and access 85% of the data that is not accessible via the API which is crazy number. well on boxer side we're all about giving your agents the file system layer so that they can remember context and and store memory for future sessions.

37:41 Now we all know that context management is a huge topic in the AI space. We we always want more context. but there will always be limitations to an individual session. So we started exporting memory and data in a file format to a local file system or a third party service so that the agent can read those files and remember previous sessions and decisions that they made. And we're starting to see these amazing personal setups where people have these huge personal knowledge bases that basically acts as a second brain for their agents so that they can complete complex work.

38:20 But when it comes to real work, it often involves collaboration. Collaboration with other people and also with other agents as well. And in these scenarios when all the agents have their individual personal knowledge baits, collaboration becomes a living hell. So what really happens when work is shared? Well, first of all, context rule needs to get copied. For example, if agent A has a document that it's working off of and it wants to collaborate with agent B and C, agent B and C will need a copy of that document to stay up to date and understand what agent A is doing.

39:05 And that means duplicated documents that are not synced and and it's very easy for these documents to go stale and therefore trust starts to get unclear. your team starts to lose track of what documents are up to date. what documents are approved for usage and what files and folders are safe for agents to use. we've seen many horror stories where agents starts to access confidential documents or files that they shouldn't be looking at and they do some irreversible actions that causes a lot of pain down the road.

39:44 So that's why we need governance. when it comes to agentic collaborations, you need to make sure that every agent has clear permission boundaries. and make sure that they can only see what they're meant to see. And you need to have robust version history so that if agent does something wrong, you can revert back to the previous version and have a robust review and auditability capability for that file system platform. So that's what Box is all about. Box is all about shifting that personal vault into the shared collaborative file system layer that your agents can connect to. So it becomes the single source of truth for agents and humans and applications to all collaborate with one another. So if you when you store files on Box, all the files will be synced and they'll stay up to date and every actor will have their own permission boundaries. So your agent may see only one folder while you have access to other folders as well. All of these will be clearly defined.

40:50 All files will also have robust version histories so you can always revert back to previous versions. And we have enterpriseg grade security in terms of reviewing and auditing the actions that your agents take. and our box platform has been trusted by Fortune 500 enterprises for the last 20 years. So, we're known for securing your data and files. So, what do you need to build this? How can you connect your agent to Box? The first thing you need is a Box platform app. So, we have a Box developer platform that you can go to to create these applications. And these applications are what gives you like the client ID, the client secrets that you need to authenticate agents, external applications into box.

41:39 Then you'll need some scope content. So folders and files that you want to share with your agents. If you're if if you've given your agent a separate identity which you you can do through the platform app you can invite them as a collaborator or if they are just acting in behalf of you then they will get all the permissions that you have as a user. So the permission boundaries are shared with your agent and then we will use the dev tooling like box cli mcp and api rest api to connect your agents to box.

42:15 So let me show you a quick demo of how you can set up and connect local coding agents to box using boxi. Now, one thing I want to note here is that I'm explicitly saying that this is for local coding agents that have shell environment access because we'll be using Box CLI. But if you're using agents in a web environment that doesn't have shell environment access, then you can use our Box MCP to connect to Box.

42:42 Cool. So, here I'm inside my terminal and I have a folder called connect agent Box and this is my project folder. The first thing I'm going to do is I'm going to do npm install- global box. Now, that command right there is going to install the Box CLI to my computer. And Box CLI is a powerful command line interface that lets you do a whole bunch of Box operations like creating, editing, moving files in Box and also using the Box AI endpoints that lets you do agentic work on your documents without having to download the documents and duplicate them and expose the actual document to your external model provider in this so that your document just stays in box and governed.

43:29 Okay, so let's check if Box is properly installed with Box- version. Cool. It seems like it's all good to go. And the next thing I need to do is I need to authenticate into my CLI. So I'll do Box login. And that's going to ask give me a couple options to authenticate. option one is to just log in as a Box user. That means my Box CLI will be acting as me as a user. Or you can use a box platform app and you would be creating a box platform app if you want more advanced features like creating web hooks or giving more capabilities to box cli or you can also create a different authenticated application to give your agent a separate identity.

44:15 But for this demo to make it simple I'll just use the simplest fruit. So that is going to open up an OOTH page. Let me just copy that link over here. And that's going to open this page up where it asks asks me if I want to grant access to Box CLI. So I'll click yes. And there you go. I am logged in to my demo account.

44:45 So back in my terminal, you can see that my CLI is authenticated. And really this is all you need to do for your coding agent. Your coding agent will be able to use the terminal and be able to use box CLI via the CLI commands. So all I need to do now is just launch a coding agent. And for this demo I'll use codex. Beautiful. So let's first check if my codex actually has access to box. So, I'm just going to ask who am I on box CLI.

45:20 Now, my Codex app should find that the Box CLI is installed and find the proper CLI command to get the information of who's authenticated into the CLI. So, you can see that it's looking at I have a Box agent skill installed on my computer. So, he's using the skill. It found the box users get me command from the CLI and it's running that command to get me that information.

45:54 And you'll see that it knows that I'm authenticated into my demo account. So, once you see that and verify that it has access, then really now it's all about just using Box as your file system for your agents. you can create an agents.mmd file and tell it to always use box for regarding any file operations and then your agent will naturally use box as the file system layer. Now since I have this connected let's try doing something cool here. I have a prompt here. So, I'm going to prompt it to read a file, an asset portfolio file of this imaginary person named John and to get the total asset portfolio value. Now, I'm going to have that running and let me go and show you the document.

46:50 So, here inside of my box account, I have this folder called John's asset portfolio and has this asset allocation summary file. And if you take a look at the file, you'll see that it has a portfolio overview and the total portfolio value comes out to be $1.25 million. And you can see all the asset allocations here as well. So since my codeex agent has access to my box account, it should be able to find this document get extracted these information from this document and give me the correct answer of $1.25 million.

47:32 So if you go back to our code session, you can see that it's again reading the box skill to figure out what to do here. And here I've given it the explicit folder ID and that's just to make sure that the agent session goes faster. But in box all of our folders and files has these ID number and on the URL. So this is the folder ID up here. And this right here is the file ID of this box note.

48:04 Great. So it seems like it was able to successfully use Box CLI to get that information and it's telling me that the portfolio so value is indeed $1.25 million. So this is a very simple example of how my agent is now using Box as its context layer and is able to work with the files that live there. So imagine you if you work at a company and if you're using box as your file system you can now start giving your agent access to your company files, your strategy documents whatever your agent needs to complete work. You'll be able to do that in a secured and seamless manner.

48:46 Cool. before I move on to the next demo, do we have any questions in chat? Let me see. yes I see that s ciph successful login last on the CLI. it does have automatic refresh tokens so it will automatically refresh but I believe without the refresh token I think it's like a couple hours. but Andrew if you could clarify that in chat would really appreciate that.

49:24 Cool. All right. I don't see any other questions there. So, I will move on to the next demo. Okay. So, we just saw a very simple demo of how you can connect your agents to Box. Now, let's take a look at a real enterprise scenario where you want to automate doing market expansion analysis. So, in this demo, I have a fictional company called Relay AI, and it's a midsize AI company in the United States, and it wants to analyze if it makes sense for them to expand into a Singaporean region.

50:02 And to complete this task, it involves a lot of steps. First of all, you got to go to the web and do your research. Maybe look at Singaporean government websites, download economy reports of that country, get all these information, create a research brief, and now you you want to use that research brief along with your company information files to come up with the appropriate strategy document. So I'm going to show you how I automated that entire process end to end by connect giving my agent browserbased and box access.

50:43 So let's take a look at this folder right here. So I have a folder called relay AI company brain and we have these three folders here and inside of the internal context folder we have these three company files that explains what Relay AI is all about. So if you take a look at this company profile document you can see that it explains that relay AI is company that serves mid-market enterprises that needs to coordinate high volume knowledge intense work and has some company facts product portfolios business models and so on.

51:23 We also have this expansion criteria document that explains ideal customer profiles, priority buying committees, pricing and packaging, and so on. And these are all information that's really important for my agent to determine whether expanding into Singapore makes sense for Relay AI. We also have these other two folders here. the market research folder and the output folder and they're all empty because we haven't done the work yet. So through this demo my hopefully my agent goes out there do the research create the research brief and upload them to Box so they're all governed and synced inside of this environment.

52:08 Now for this demo I recorded a short video. So we'll walk through this demo by watching this video. because I don't want to m make you guys sit through a five minute AI session. I don't want to waste your time. So yeah, let's start this video and walk through this demo. So right now I created a codex agent inside of the chatpt application desktop app and I gave it access to both browserbase and box cli. Now to for browserbased they really did a a clever thing and they have a a browserbased skill and if you install that and you just run the skill it will automatically set up install browserbased CLI and prompt you to provide them with the browserbased API key. So the onboarding experience is really seamless. So that's how I have connected to browserbase. I'm using their CLI on the boxes side. I also gave it access to the box CLI. I'm asking if double-checking if I am authenticated here. And you'll see that this agent has access to my demo account. Beautiful.

53:18 So, here I'm showing you again of the folder structure of the company brain folder, the market research folder and the strategy outputs folder are currently empty. Now, the first prompt that I gave my agent is to research whether Singapore is a strong expansion market for relay AI using browserbased. I've asked it to look at three authoritative Singapore sources to download an economy report of Singapore in 2025 and it's going to extract two or three decision relevant facts from these sources create a research brief and upload that research brief document back into Box explicitly inside of that market research folder.

54:02 Now this is a very complex task that involves serving the web, downloading PDF files, extracting data from there, creating a research brief and storing that inside of a cloud envir cloud environment Box. And you can see that my agent was able to do this very complex task because it had access to both browser base and Box. So it successfully created the expansion brief and uploaded to Box. So now if you go back to the market research folder here, you'll see that this document was created. And if you go inside, we have this beautiful document that is not just created based on the knowledge that the LLM was trained upon.

54:45 These are actual live information that was taken from the sources that my browserbased agent went out and did the research on. and it decided that Singapore is indeed a strong expansion market and it has all the citations and information right here. So if you go into the details of what it actually did, you'll see that it used the browserbased CLI here to go to these websites to search them, download the economy report PDF, extract it into a text file so that it's easy to extract these information and it used these context to create that research brief and upload it to Box.

55:29 So now that it's in box, I can get my teammates to look at that document to provide feedback or I can have my other agents look at it as well. Okay. So for the second prompt, I've asked my agent to use Box CLI to retrieve the three Relay AI internal documents that we saw before. and I've asked it to use those three documents along with the research brief that we just created to come up with a strategy document and determine whether we should expand into Singapore and if we should how we should do that. So it's going to create that strategy document and upload that back into Box again so that it stays synced and it can be easily shared with my teammates.

56:15 So again, you can see that it was able to successfully create that document. And now if we go back to Box, you'll see that the document is now created. And if you go in here, this document is now live inside of our cloud environment. And this is a very, you know, powerful document that is not just AI slop. It's based on real information, live information. combining context from your research from browserbased agent and the company files that were stored in box.

56:48 So now that this document is live on box and if I want to collaborate upon this document with my teammates, it's as easy as just clicking that share button inviting your teammate as a collaborator and sharing them with them so that they can look at them, provide feedback and work on the next steps. So this demo really shows how powerful your agents can become when you give them the capability to search the web but not just search but to actually be able to interact with the websites so they can in download PDFs and that's information that your agent usually can't gain access to and we were able to do that thanks to browser base and all of the files the company files and the research files and the strategy files that were created by my agent were all synced back in the box so that now you have a single source of truth where you can collaborate upon these this project with other people and other agents. So hopefully I this demo kind of inspires you on what you can do when when you have these tools enabled for your agents.

58:02 Cool. that's it for the demo. do we have any questions about this demo? Let's see. Cool. Okay. no questions it seems like in the chat, but let me just share this QR code with you. So, I have a bit.ly page here that has all the links to our resources and my, LinkedIn information. So, if you would like to connect with me, and feel free to ask me questions or share what you build. But yeah, you'll also find the GitHub repo of the the market expansion demo that I just showed you.

58:40 It has all the the prompts and the agents and D file that you can use and give to your agent to easily set this up. But yeah, we'll stick around for a couple more minutes. Kyle, do you have anything to say before we close this off? no, I'm happy to answer any questions that may have come up in in the time and yeah, I'll I'll I'll stick around as well. >> Amazing. did you guys have any resources or links that you wanted to share with the group?

59:13 >> We will we have a whole bunch of I can share the code and then also documentation and a recording. So, we'll we'll get that all sent out afterwards. >> Awesome. Awesome. Cool. Cool. Yeah, we'll stick around for a couple more minutes. so if you have any questions, feel free to ask them now. But yeah, thank you so much for making the time to join us today. I know it's u it's during the daytime and I know you guys are all busy, but really appreciate you making the time to listen in on what we had to say here. but we really believe this is some next level stuff.

59:46 this is where agents are going. agents are becoming more powerful with all these tools and capabilities that these platforms provide. So hopefully this webinar was really helpful for you guys.

Summary

The webinar focused on integrating Browserbase and Box to enhance agent capabilities in automating web tasks. Chris Kim from Box and Kyle from Browserbase demonstrated how agents can navigate the web, perform tasks, and manage files effectively, showcasing the potential for improved workflows in enterprise environments.

- Browserbase enables the creation of browser agents that automate web tasks, overcoming challenges like CAPTCHA and authentication barriers.
- Agents can perform tasks using headless browsers, allowing them to interact with web pages as humans would.
- The integration with Box provides a secure file management system, allowing agents to store and manage documents efficiently.
- Key features of Browserbase include agent identity management, search and fetch capabilities, and the ability to run tasks autonomously.
- Box enhances agent functionality by providing a collaborative environment with version control and permission management.
- The demo illustrated a practical use case where an agent researched market expansion opportunities and generated strategy documents using data from both Browserbase and Box.
- The session emphasized the importance of governance and security in agent collaboration, ensuring that agents access only the necessary information.
- Participants were encouraged to explore the tools and resources shared during the webinar to implement similar solutions in their own workflows.

Questions Answered

What is the purpose of today's webinar?

The webinar aims to showcase how to connect Browserbase and Box to create powerful agentic setups.

How does Browserbase enhance agent capabilities?

Browserbase allows agents to autonomously control a browser, improving efficiency over traditional methods that relied on screenshots and coordinate clicks.

What features improve the efficiency of Browserbase agents?

Browserbase includes action caching and self-healing capabilities, allowing agents to repeat actions without additional calls and adapt to changes on web pages.

How does Browserbase handle anti-bot challenges?

Browserbase is working to ensure its agents are recognized as trusted by anti-bot providers, minimizing disruptions like CAPTCHA challenges.

How can agents utilize Box for enhanced functionality?

Agents can access and utilize files stored in Box, allowing for context management and memory retention across sessions.

© transcribe · For agents Built with care and craft by Gokul Rajaram