2026 AppSec Market Report with James Berthoty of Latio

0:00 being in AppSec, I occasionally look up like what tools to use and I always stumble into the blog posts of the same guy, James. So I am super excited to be talking to James today. And I think like you're one of the legends of AppSec. Can you introduce yourself? Sure, yeah, I mean, feel it. First of all, think it says a lot about security if I can be a legend of AppSec, right? Like it's the, unfortunately, I think the bar for AppSec might not be as high as we would like it to be.

0:26 But yeah, my name's James, background is DevSecOps, product security, cloud security, app security, whatever you wanna call it, at a big MDR named ReliaQuest. And so that was like detection engineering, AppSec, helping a lot of different companies do it. And then from there, I was at a startup called Very Good Security doing basically everything around like a compliance automation tool. And then most recently was at PagerDuty doing a lot of our FedRAMP initiatives. And so I have done a ton of different security roles, AppSec being a big part of that.

0:56 I feel like every cloud security engineer is a bit of an AppSec engineer and every AppSec engineer is a bit of a cloud security engineer. And basically, Latio and what you were referencing around the tools came about from just being frustrated that it was always like, tools like Sysdig and Wiz and Orca were like radically different from each other. And everybody just sort of grouped them in to be the same thing. And it was always just really annoying to me personally.

1:18 And then I was just posting on LinkedIn. It started as a joke between some friends and I that were just like goofing around on LinkedIn, but then turned into like just taking it more seriously as far as trying to give people actually helpful guidance. And so, I mean, really the goal of Latio is to help people actually understand what tools do. when you would use one versus another. Everything we do from a report perspective is towards like guiding tools towards your stack.

1:41 Instead of trying to be like, know, however Gartner does it, like here's the company with the best vision versus the best execution. Like I'm an engineer, I don't care what like, oh, this company had a slightly better vision or whatever. I wanna know like based on my tech stack, which tools should I use? And what are the actual differences between these things? So then a lot of the content we do now is aimed at Yeah, helping people understand what are these emerging features, what tools are really doing them, which ones are like sort of faking it, but in like the right ballpark.

2:08 So yeah, that's the super quick rundown. Yeah, and I feel like it's like being an app sec like at some point you're going to be like, okay, which ESPM tool should I use? And inevitably, I feel like you run into some of your blog posts. And I feel like that's almost like an interview questions. Like, do you know who James is? Well, that's it's a I do appreciate it. The flip side of that is I get a lot of salespeople that reach out to me asking if I know someone like they follow me, but I don't I don't personally know.

2:37 so I want to start sharing my screen. So there's a few things you want to go through. I want to show your website and then I want to go through really, really good AppSec report that you just released like three days ago. Super cool report for the state of AppSec. Yeah, I appreciate it. was many months in the making. The things I did not talk about fully AI coding stuff, right? Because right now we're in a weird transitionary period where a lot of companies are going to be using the static scanners for a long time still.

3:07 But then all of the cool emerging stuff, like a lot of companies are ditching them. And so it's a really crazy time to be in AppSec. Yeah, and we just wanted to try to cover all of it. So the first thing I just wanted to show your YouTube channel. So you actually have a really really good YouTube channel with some really good thumbnails as well. oh Yeah, it's a, you know, I'll say developer targeted thumbnails, I guess.

3:30 If you really go down, you can see where I've experimented with different approaches to stuff, yeah, basically just, I'm not good enough at Photoshop to do actually good thumbnails, so this is the in-between. And then, so this is your LATIO website and I thought it's really, really cool. And so you've built basically a inventory of AppSec platforms and providers and tools. Yep. So really, we're expanding it all the time. Like I'd say most of the coverage on there is about like cloud and AppSec and AI and some different pieces of it.

4:02 But really my goal was to go, first of all, a large part of building the directory part of Latio was getting frustrated when you go on sites like G2 or Pure Insights or whatever. everybody, you go to the SCA vendors and like the top five people are usually like irrelevant to SCA scanning. So here, if you like go into any of the categories, just as an example, The other thing we do is do subcategories. So within ASPM, right, it's a really confusing acronym because some tools are purely management focused, some are scanning focused, some are focused on SDLC.

4:33 And so here you actually have like a real representation of like, want all in one scanning and that's the scanners versus the management are people who do third party integrations to ingest and orchestrate AppSec scanners across everything. And so we try to have that. a subcategory breakdown for everything, right? So SaaS here, we're talking like reachability in SaaS, which is an under talked about thing, AI SaaS, which is using LLMs to do a static code analysis and really just having like an up-to-date and actually relevant way to make your own decisions about this stuff instead of just saying, you know, who, what SaaS provider has the biggest internet footprint.

5:10 And so that's the other thing on this is like nothing on the site is pay for play. I've learned some crazy things since doing this that vendors do. where a lot of those top 10 DAS tools websites are literally pay per click and you set a budget to those websites and they just rank you based on how much you're paying for clicks and stuff. So yeah, I have learned about all that gross stuff and this site does none of it.

5:32 And I was curious anyone who's in AppSec, you want to like, you POC a few different tools and you're quickly realized it's actually very challenging to compare tools to each other, especially when you have like limited time, but also to, to, try to find like consistent tests to compare tools with. So do you have any advice on like, how do you actually compare tools? Yeah, mean, there's a few part of this is even where a lot of because another thing we do at Leisure is like work with people to help them do tool selection consolidation sort of stuff.

6:07 And a lot of it, I think people discount how much you want a tool that fits the vibe of your team and organization. And sometimes, you know, novice James did a lot of AppSec bake-offs based on who had the most findings, what had the most true positives, the least false positives. And it was like a math equation. And I made some like really bad tool choices from that because like the IDE integration sucked, the GitHub integrations all sucked.

6:33 Like it was unusable from a developer perspective, but they did have the best findings. And so that's where it is really hard as you were saying, especially as now like, you know, just looking on the screen here, like a tool like Aikido or CoreGear ZeroPath, all these guys have like... eight different scanners that are available. Like, are you gonna do like a false positive analysis across eight different scanners plus test the CICD workflow plus test the IDE workflow?

6:56 It's just a lot. So I really, my advice high level is to go through and end to end fix a vulnerability. And so don't worry about how many scan results you got, but just actually fix one thing and it'll give you a much better idea of like, what is this tool actually helping with? or not. The other thing is I can link, I do have like an insecure test repo that not a lot of vendors have picked up on as much.

7:27 And it's what I, like it's a little more full, the thing that drove me crazy about a lot of the like juice shop type stuff, it's a lot of like monolithic applications that everyone base marks on. Whereas mine is like a full Kubernetes deployment that includes like Terraform, Helm Charge, microservices. And I actually use it to test runtime stuff too. So I have all this stuff deployed. And so on the runtime side, I have all these agents running to test like who's got the best detections.

7:50 And then I'm using this repo to test. Yeah, I've got a lot of little tricks in there as far as like, something actually reachable? Is it actually being called or does it just exist? Just to get a sense of how sophisticated a scanner is about deduplicating things and stuff like that. Wow, that's very cool. Yeah, I'll link to that. I'm very interested in what you said about people optimize for like number of findings and I still see a lot of teams that do that.

8:15 They'll test different tools and they'll basically see which one flagged the most findings. And I'm starting to shift my perspective into like actually I kind of want the one that flags the least amount of findings now. It's honestly like, first of all, because we also work with vendors and I usually tell them like, look, you want a POC mode that's like full blast, give me everything. And then you want like the day to day mode that's like, hey, now you're only going to see the real stuff.

8:38 Because from a vendor perspective, it's frustrating, right? Like you're losing POCs because these other people have like more detection show up, even though you know they're all false positives and stuff. And yeah, to me, it's a real industry. maturity thing, but then on the flip side, right? Like I love the AI SAS stuff like Corgi and Zero Path on just on this screen and like, but I can fall into that trap myself, right? Where I can see like an AI code review tool that finds like 20 code smells and they're all true positives.

9:06 like, this is so sick. Whoa, we got to use this right away. But if I was a dev, I would probably be super annoyed getting pinged by 20 code smell things that are like debatable. Yeah. Okay. So we will talk about AI SAST and stuff. So, well, I want to get into your report. So you had this link here and it made the report available without a paywall, which I really appreciate it. And so you released this about three days ago.

9:33 It does seem long, but it has different sections. And I kind of want to go through it. today because I think this is super, super, super useful. And like, honestly, it's like worth an undergrad for someone who wants to get into AppSci. Like, like you don't need two years of school. Just read this one report because take your time. If you study everything in this, like you will be like actually an expert on AppSci. Super valuable report.

10:03 mean, it must have taken you such a long time to write it because it's actually like super high quality. So I guess we can start a little bit by talking about some of the key takeaways. But first, did you want to like, kind of like introduce like what the report is and how you made it? Yeah, so when I was still at PagerDuty is when I started making vendor relationships with people, right? And so the goal was to get in-depth demos from and ask the kinds of things that only a actual AppSec developer would ask, right?

10:30 Stuff like, what's your scan time? What's your language support? What's reachability really mean for you? What kind of reachability? And really starting to create those granular connections for people. And now we've moved into a quarterly report structure. That's a combination of interviewing about a hundred different practitioners, buyers to get what their perspectives are on the survey side. And then talking to all of these vendors about what the latest capabilities they have and really trying to help people get the vibe of what they do.

11:01 And that's the part that I think makes us really different when we're suggesting tools as a part of this is there are times, yeah, it's just not a long list. The way a lot of analyst firms work is they send vendors a big checklist of features and then you get these big feature checkbox comparisons which is why everyone looks the same because everybody does SCA scanning, like literally everyone. And so you like, how are you supposed to know what to pick?

11:27 And so that's where, you know, we're, when we do this, I have direct access to a lot of these products, or we do very like in-depth demo sessions with the people that we're including so that we understand what's going on. Mm-hmm, very cool. So I want to go through some of the key takeaways here and then we can go into the other sections. So the first one, you have a few platform players you mentioned. Can you expand on that a bit?

11:53 so something that's really interesting in AppSec is, and this is kind of hard because like Gartner separates these into like three different reports, right? You've got application security testing, which is like sort of SaaS focused. Then you have supply chain, which is sort of SCA focused. And then you have more of like a DAS attack surface management type of category. And depending on the size of your organization, this is like why AppSec is so weird is on the big enterprise side, you have siloed categories for this stuff, right?

12:17 Like you have a supply chain person that's different from a SaaS person that's different from a red team. and they all manage their own one tool. But in those smaller mid-market companies that have like one to three DevSecOps people, like they want an all-in-one solution, because it's all just code and they're just scanning code, right? And then to give a third in between, nobody knows what to do with containers and IAC scanning. Like should that be part of the CNAP?

12:39 Should that be part of the AppSec tool? And so you have this weird overlap. All that to say that there is generally a consolidation that's happened where... when I talk to a provider, I assume they have at least SCA, SAS, and secret scanning together. And if they don't, then I expect, really a layer beyond that, I also expect like container runtime analysis, IAC scanning, some form of DAST or DAST integration. And so that's really like these tools have become much more of a platform, but a lot of big enterprises are still sort of like this is our, know, BlackDuck's our SCA tool.

13:16 Fortify is our SAS tool and really stuff that's older methodologies for testing. So yeah, we're really just for the most part, it's largely consolidated into bigger platforms that are the most relevant for most people. Yeah. And I really like what you said here. It has more to do with the user integration and the experience than actual just the scanning. most of the companies I've worked at were small companies. And so we would have like just like one team, even like one or two people just like doing everything.

13:44 Although I don't know how I feel about like vendors that try to do everything in one platform because sometimes they would bring their own scanners. and the quality would vary a lot. But then like over time I realized that there was a point where some of the platform especially like ASPM platforms would just wrap around a bunch of open source tools like use like GitHub actions like look, totally. This is like a constant recurring thing is people assume that someone doing multiple scan types can't be as good as someone who's only doing one.

14:19 But the truth is like a lot of these guys are using, like it's not always the case. It depends on the languages, depends on what they're using under the hood, depends on, and even that's not enough, right? Like it's easy to discount. people using OpenGrep because it's a lot of vendors doing it, but it's about the rules they're using and how they maintain those rules and how much the commitment they've put in the tuning them. And so really, I think in security, we tend to try to make some assumptions to make things easier, because there are like 200 vendors in the space.

14:44 But those assumptions just aren't always true, because a lot of the platform providers do have better individual scanners than people who only focus on one or two things. Yeah, no, that totally. overall, realized like, you know what? I don't even care what scanner you're using because in the end it's going to depend on like, how am I actually integrating it? Am I actually triaging the findings? Well, am I spending time tweaking the rules? Because just like the scanner is going to find some stuff.

15:09 Okay. But then what? That like second step is really like what's important than just the tooling, which is why I kind of like also some vendors are very like, they focus too much on making their POC look really good. And then you're like, wow, it found 10 billion findings. That's perfect. But then like the experience, like after it does surface findings, how can I deal with them? That experience sometimes is less good. and that's exactly why, look, my goal in most of the report stuff is to just help you narrow down, like, here's the most relevant people to be looking at.

15:43 Because otherwise it is just a giant mess of different integration experiences and pros and cons and all of that. So that's why, like, even if you go down to the buyer's guide sections, which is like way down its pages. 40 and 37. So we split up the buyer's guides for this into mid-market and enterprise. And really like this is, we take this approach instead of the magic quadrant because of exactly what you're saying and to make it more prescriptive for people.

16:11 Cause at the end of the day, like the way these first three boxes are laid out, for example, you either care about having workload scanning plus AppSec in the same tool or you don't. And if you care about that, there's only three people that really do that. If you want to have DAST as a part of your platform, there's the people who do that. If you don't want DAST, there's the people who do that. And so there's just immediately trying to narrow down and let you make the choice more than just some quadrant of who's the best.

16:36 And then when you go down to the next part, it's like, all right, so now you have your core AppSec tool. If you want a separate standalone DAST, there's the people doing DAST with some different flavors of AI pen testing versus API testing versus bundled. There's people doing runtime supplementation and then like this whole AI code first approach to really try to help people like narrow down and make that decision based on what they care about and what their architectures are like.

17:03 Mm-hmm, that's really cool. Nice. Now I want to go into the second point. This is probably my favorite topic, AI native SAST tools. I love this section and even I'm just plugging it in. So if you want to go to like the image for it, it's on page 18 m of like the vendors that are doing it. This AI SAST is very confusing, first of all. And the thing people should know is a lot of vendors are calling AI SAST just AI prioritization and remediation, which is really confusing for people.

17:38 So that's the first thing to get out of the way. But the second piece is that there's also a lot of weird middle ground with AISAS because some companies are taking AI and using it to write static analysis rules with their engine. And so it can like do some native querying, but it's not quite the same thing that I'm talking about, which is using LLMs to do the analysis themselves. The architecture I'm the most familiar with is zero path, which creates like a a graph of your application very similar to how something like Claude works under the hood, so that the LLM can do those lookups and find the security pieces.

18:16 But all of these guys have slightly different architectures. And the outcome that's important is these results are 10x better than historical SAS. And people don't really, I think, fully understand. Because people have thought that scanners are commoditized for so long now. that I think they really, it's when you use it and when I've used it and I can see like, cause I'm in a lot of these tools, right? And some of your historical tools that don't have this functionality are finding like, oh, cross-site scripting and whatever.

18:50 And then I look in and it's like, okay, is this actually exploitable? I can't really tell. Maybe you could do something here versus these, these companies are all finding like 10, 20, like broken logic issues, crazy novel exploits that I wouldn't have ever conceived of. Like just to give one example on the Latio site, there's a place where someone could put in their email address to subscribe to the newsletter. And one of these tools raised the alert that like, hey, you could discover user email addresses of who subscribed or not by brute forcing that.

19:20 And I just wouldn't have ever, like that's the kind of stuff that actually gets you in trouble. And no static analysis tool is going to call that out because it doesn't have like the context of what's happening. So that's why I'm just super blown away with most of the tools in here. Yeah, I'm a huge fan of those. Both Corgea and Zero Path have shown some really nice potential overall. Zero Path has an excellent blog post on basically how it works.

19:46 What do you think of... I feel these have almost like a hybrid approach between traditional SAST and using LLMs. But first, what do you think of using purely just Claude with nothing else as... static analysis. So it is interesting and this is funny timing with like the cloud security announcement today. They might have fixed some of this with that. I assume it makes it better. I signed up for the beta for it.

20:17 maybe we'll find out at some point. used the the clods slash security dash review command? Yep. And it so when I've been doing my own coding, like I have some of these tools installed plus the security review and these tools found things that the security review didn't. And I think just the orientation of the platform taking more of the overall application context into account is really like where these guys have an edge so far. Like, again, I don't know how long that'll last.

20:47 I don't know if it'll always be better. But what I could say is for now, like when I run slash security review, I get some interesting findings, but they tend to look a lot like traditional static results where it's like, you're not sanitizing an input or something like that. I don't tend to get the really good authorization application, novel exploit, pen testy type stuff. And even the example that you shared around the comments, like, I have such a good example of this where I was coding some authentication piece and I was in the middle of setting up middleware to handle the authentication.

21:26 But then Claude was like lazy about going and applying that to all the functions. And on one of them, it just put like slash slash add middleware here or like this uses middleware. It's not necessary to do authentication here. And then the security review didn't flag it because it read the comment and thought it was like, yeah, so this is where this happens versus ZeroPath and Corgi have both. like called that out as a, hey, you say you do this, but you didn't do this.

21:51 Yeah. nice, nice. Yeah, mean, yeah. like, overall, many real CVEs are being found by these tools. So I'm extremely excited about this field and they've all been improving at a pretty quick pace. And that's a, I'll, I just haven't been hands on with it yet, but like they, they had made a lot of buzz with their open SSL vulnerabilities. think they discovered a hundred of, new vulnerabilities with their analysis engine in it. I know depth first has some similar research done dry run has done a lot of, very objective competitive analysis, like the founders there really care about, doing objective analysis on this stuff.

22:32 so they have some good reports that are doing some of it. So it's really like just an exciting and fast evolving field. Nice. so the next one, also, so I kind of touched on it a bit, but yeah, so a very big problem in AppSec, I think honestly the biggest problem I see in most teams is like you join a team and you're like, oh, we need scanners, we need scanners, we need more scanners and you don't onboard scanners.

22:58 Great. Now what? Right now, actually you have a problem because these scanners are finding things and you have a growing backlog. And that growing backlog is, I think, one of the biggest challenges AppSec teams don't know how to solve. Yep. It's a, for a long time, I think the way that people should think about security, you have to like split AppSec in half. On the one hand, you have like SaaS type findings, which are like misconfiguration, SQL injection type stuff.

23:27 And then you have like SCA and container findings, which I just, like that's just patch management, right? And this is only, it's so hard in AppSec because it's not Windows. When you're in a Windows environment, you've got your whatever the SCM or it's been, thank God it's been long enough since I've been in a Windows environment that I don't remember the details on it. But like the, you've got your patch manager set up and that thing just cranks out the patches of the servers, they reboot.

23:51 You have like a testing window where it's like, know, patch Tuesday comes around, you roll your patch to your test group, then you roll it across the fleet. And it's a super standardized and well-oiled machine on patching. But in AppSec, you've just got this massive like. is there even a patch that the open source person made? Are they gonna patch it? Is this even a real CVE or did someone just submit it? And then you layer in like, this is in Ubuntu, but does Ubuntu roll those patches into their container images?

24:17 And then when's the last time you rebuilt this container image? Do you have to actually upgrade the version or just do a rebuild? And like that complexity is what has made this problem so much. And so when I like help people do AppSec, I try to get them to think about how are you gonna build a patch management program and not how are you gonna scan for vulnerabilities and prioritize with reachability and find the most applicable ones.

24:39 Because even the thing with like reachability and even prioritization, like if you think of a container vulnerability, you don't actually want to patch that most of the time. Because if you create all these like onesy twosy patches, you end up with this Docker file that's like. Yeah rewriting a bunch of like binaries as it's happening and it just creates a mess, right? The real solution is you're just waiting for upstream the patch and then you're going to reroll that Docker image.

25:04 But a lot of the tools don't have that context. And so this is where there's a section of the report. I think it's in the enterprise buyer's guide as well as in the supply chain section, which is on page. page 22, this patch assistance box in the top right, that is getting like really good at actually helping do this patch management part, because they take all of the functions that have changed between versions of your application, and then they look at which functions of those you use in your application.

25:36 And then they suggest, like they actually give you a complete migration patch between the versions. And so there's a ton of different AI approaches, runtime approaches across that. But like, for example, with Maize, I just saw something incredibly cool. They're one of these like AI first patch management people, which is the first tool I've ever seen actually suggest doing a container rebuild instead of a version upgrade when it wasn't necessary. And that's the kind of stuff that I think we're finally getting towards.

26:04 The challenge is, because we just talked about that platformization part, is a lot of these tools, not all of them, but a lot of them are built to like sit on top of an existing tool, something like CommVue. And so it's a great fit for like enterprises who have this big backlog problem and want a separate tool to help do the AI prioritization and remediation on it. But it's hard for people who are more in the mid market and like you're just sort of waiting for your platform solution to get some of these features into it.

26:31 Nice. Nice, yeah, yeah. Yeah, that's a good feature to have. But yeah, I as I said earlier, like, I at this point, I kind of want tools that give me less findings that are actually actionable and just like, sort them by highest impact for me, you know. Yes, if you really want to go into the weeds, I have a blog post that like breaks down every kind of reachability because there's a lot of different kinds out there.

26:56 Because this is the challenge of what you're saying is like the vendors kind of know that. So they all have the cool graph now that's like you had a bazillion vulnerabilities and now you've got four. But the way they're doing that is all very different and has very different trade-offs that are important to be aware of. And it's actually like pretty complicated to put the whole story together. So yeah. Nice. Next section. So ASPM application security posture management.

27:25 And the idea is like you send all of your findings from different tools into one place. So you can actually triage them correlate there. And at some point it felt like a bunch of new players were coming into the field doing ASPM. And now I feel like almost every company I talk to they're like, yeah, we're building our own ASPM. I think like even GitHub is with advanced security. They're making their own ASPM platform. What are your thoughts here?

27:49 Yeah, so if you go to, I hope the jumping around is okay, page 25, you can sort of see, it's a really, like from a market perspective, like this is just really fascinating since I've just been on the weeds on it for so long where... Sort of at the same time, you had ASPM, which was like a Gartner acronym that was just focused on the management. Sort of the go-tos for those were like Defect Dojo, Armor Code, and Phoenix were like three of the biggest like pure ASPM.

28:19 and legit, sorry, like legit security basically invented that too. And at the same time, you had all these like exposure management, vulnerability management, cloud management solutions, like Zafran, who's not on here, but Simplicity. Wizz acquired Daz to do it on their end. There's a company called Opus that Orca bought to do it. And really you had these two sort of vulnerability aggregator solutions, but they both realized they needed like the code to cloud context to create value.

28:48 Because basically the way that I think about it is like the second you touch a container, you open Pandora's box of complexity, because like, do you fix that in the code? Do you scan it in the pipeline? Do you scan it in the registry? Do you scan it when it's deployed? Like it just opens, how do you wanna manage that vulnerability is really hard. And so we've seen this shift towards like from a market perspective, very much going into this broader exposure management, which is like with simplicity, like Cortex ingests all your vulnerabilities.

29:15 Other people too that just aren't coming to mind off the top of head. But then even like Phoenix and Armor Code have become much more like code to cloud, container. vulnerability cloud vulnerability solutions. While there's a lot of platforms over on the left side, like Sci Code, Ox, GitHub, to your point, like who are ingesting third party vulnerabilities while also providing their own scanners. So like if you already have a big semigrep rule set, big check marks rule set, and you don't want to like rip that out because it's going to take forever, but you want something new for projects going forward that's lighter weight or something like that.

29:49 Like that's the idea behind this solution. is they provide their own scanners or you ingest the third party ones to prioritize and manage it all from a single location. So that even goes back to the website, right? Where like, that's why there's two different check boxes for scanning and management. So if you want a tool that can do both, like that's the solution. And ideally the purpose of ASPM it helps you prioritize and then you have a list of like these are the ones I want to fix and then you want to actually send them to fix or maybe they can suggest fixes.

30:18 In your perspective, should people be giving developers access to ASPM Yeah, I mean, ultimately I think, look, there's a couple of security things that I personally get bothered by, but I don't try to change the market because I recognize I have very limited potential here. One is what we've talked about, is like security people are going to assess scanners based on number two positives, false positives. We can be cool and focus on DevEx and stuff instead, but at end of the day, that's like a lot of security teams are going do that.

30:47 The other very common thing is that... it really got the whole shift left concept of like, we're gonna shift as far left as possible. And like, I need an IDE plugin because the whole problem of shift left is devs do not fix things just magically. Like they're not coding and then like their IDE bugs them with 50 red lines from a SAS tool. And then they're like, thank God. Now I can go fix all these findings for the security team.

31:12 Like they want credit for the work they're getting. They want sprint points. They want story points. They wanna actually like do work in an organized way. And so this is why against the idea that dev should never see the tool, it's sort of this magical thing that's going on. Like I think this is part of like Wizz's success for example, is you can just create like a per team dashboard with a search of, show me all the vulnerabilities for the last 30 days with on container images with this tag.

31:42 And then that's just directly either it exists within the ASPM tool as a dashboard. or you're just automating JIRA tickets out of that. And so it's invisible in the sense of like, if a dev actually logs in and checks it directly, like who cares? But the thing I wanna emphasize is I do think it's really important to have like a realistic just ticketing workflow, even though it's not as cool as like, we're in the ID and all that.

32:05 Like, it's just not how this stuff actually gets done. Yeah. Yeah, I have heard the term shift right a few times recently and I kind of like that's like, well, let's just focus on runtime and then like fix the things we're actually seeing, you know. go down one page, I love runtime. Runtime's my, I don't know, I'm such a sucker for it. Cause this is another like anti pattern, right? It's like everybody wants to shift left.

32:29 They want to do posture scanning, like whatever. I can't change the whole industry, but I love the stuff that's going on in these buckets is like the coolest stuff that exists ever. Especially if we zoom into like, or not literally zoom in, but if we're just thinking about the ADR box in particular. Like these guys, most of them, some of them are cheating a little bit, but for the most part, they use eBPF to actually inspect the application using a lot of very cool, like new hacks that are possible when you're hacking eBPF and watching stack traces and actually seeing what functions are executing within the application.

33:03 They're baselining a lot of like library behavior. And this allows them to spot and prevent zero days in a very genuine way, like stuff like exit utils, which would never. have gotten spotted without a guy like baselining his Postgres performance or, and then for prioritization, like when I use these ADR tools and I see the prioritization, it's crazy different. When you go from this world of like, say you have the best static reachability in the world, which is like the left stuff, you're still gonna have false positives.

33:33 All of these tools have a section for like potentially reachable, cause they don't actually know. But when you go into an ADR and you see it, like every vulnerability is interesting. And that's what I've never experienced before is even something that was a false positive, which was just one that came example was a function that was executing from a cryptographic library in my container. That if it was being passed SSH keys that needed randomization, like if I was generating SSH keys on that box, an attacker could have reverse engineered it.

34:04 But like that's still interesting. And that's a real finding. as opposed to like, you know, this library was loaded or whatever. m So yeah, that's my, I love this runtime stuff. I think it's so cool and super helpful across the board. The idea of application level detection response is super interesting, especially when they can prevent. But do you have any metrics on how many companies actually use ADR? I feel like it's not very common. it's a, like, you me being as politically correct as possible is saying like, it's a emerging category, right?

34:39 So on the sense of a lot of the, like, there's not a lot of old companies on this list, right? Except Contrast has been doing sort of this in a general sense for a long time. And then when we talk about the broader CADR, just to make that distinction clear for people, I love this stuff because it bridges the gap between all these tools. So the example of how a real attack happens is like a SQL injection does something to get someone a shell on a container and then they use that shell to pivot in the other cloud resources.

35:07 And it's to unite all of those detections into a single story. That's what I love about like the CADR concept is like the best runtime protection that you can get on cloud workloads. But as far as how's it taking off, it really depends on All of these companies have different go-to-market strategies, right? Like for big enterprises, again, they're okay with something that just minimizes. If all it does is eliminate 99 % of their vulnerabilities, that's good enough for them to buy it.

35:34 For others, like that broader CADR play, it really, like everyone would benefit from that because you get the better prioritization, you get the better detection response pieces across your cloud workloads. And at the end of the day, everyone's going to have to have this, right? Like there's a lot more, this box, this CADR box, like a year ago would have only had like Oligo, Raven, Migo and Kodam were sort of like the first four for a long time.

36:02 But since then you've seen like Datadog, Dynatrace, Wiz, Upwind, Sweet, like all these guys have gotten into it because it's so useful. Mm-hmm. So I think it's more for most of the market, like most things, it'll probably be, know, once we see the CrowdStrike CADR module powered by Acquisition 500, yeah. That makes sense. So people don't know how to secure AI-generated code? Yep, I mean, and you've been doing a ton of stuff on this with the skills and the, I thought about jumping in the skills thing.

36:35 I don't actually hate static skills scanning as much if we want to open that box. Cause I think it's like, yeah, it's not perfect. Yeah. It's yes. Cause especially there's different ways of doing it, right? Like some people try to create like a permissions model based on it to create like, don't allow any rules that reach out to the internet or access local files. I think that's, know, there's some cool stuff going on there. But all that to say, if we do the AI coding part on page 29, I think this is where...

37:08 people get the most confused. Cause if you go to literally anyone in this reports website, it's going to say we are AI AppSec engineer for the AI era powered by AI to do AI code, everything like they're all just saying the same AI thing. And so when we think about the approaches, everyone is a little nervous, right? Because of stuff like the cloud security announcement that came out today. Like if that's good, that like kills everybody in a really terrifying way from like a go-to-market perspective, right?

37:38 If the question can't become, I'll just use Cloud for that. Like, why am I also using all this other stuff? That said, that's not the case for now. And there are other tools besides Cloud, like Cloud's the best one for now, but things might change again whenever. The thing that's important for people to know is I really want to underscore this MCP scanning invocation thing, because this is what most security tools do. And just for me, I think it sucks.

38:01 I don't know if you use... if you have used any of these, but like, it's just bad. This is when like cursor generates code and then it's like, let me run a scan. And then it runs the scan and it's like, let me fix this result. like those scans were bad because they're so full of false positives. Like a 50 % false positive rate is like considered really good. They're so slow or like the speed, even the ones that are faster now, like it just doesn't really matter because the findings are always so low quality.

38:31 that you just end up, the AI is just spinning its wheels, wasting time on like false positives. But it's constantly being told to like run this scan over and over again. And I just don't think it brings like any value. And so what I wanted to try to highlight in the report is two different approaches that I think both have a lot of promise to them, but they're both early enough that I just don't really know what it's gonna look like in the longterm.

38:52 The first is like the rules management context injection approach, which is the people on the left box. And these guys all use different ways to try to give teams the ability, first of all, to get insight into what rules their developers are using, em just to see what context they're feeding in. And I do think this is something where I think of it as the ability to give the AI coding agent a constantly turned on secure developer training.

39:16 That's actually good. In the sense of when we do secure developer training, we should say, here's the ORM that we use at our organization. Here's... the standard libraries we use to do authentication. Here's our password hashing library. And that's a level of security maturity that just, think 90 % of companies never get to, because they're so busy like fixing SAS findings, fixing SCA finds that they never take a step back, do a threat model, create standard procedures for everyone.

39:42 Like this is the kind of stuff that like Netflix, like huge, really strong engineering orgs do that just never quite like trickles down. And the opportunity here is like people like, get corridor for instance, and you know, I could go in the other ones, but like their whole approach is let's build a library of secure coding practices for this organization. someone like Clover on the other side, it does, it is a threat modeling company that then takes your whole organization design context and then makes it so the agent looks up that context as it's coding to say, here's how we do, I'm about to implement authentication.

40:16 How should I do that? And then it like looks and sees how, how to do it. Mm-hmm. And I've seen this result in better code. And that I think is a much better approach than like the cursor hooks method. I guess for the MCP side, the MCP as an architecture, do you think it's going to last? It depends on how much you're drowning in the SF Kool-Aid on this stuff, right? Like those guys are all saying MCP is dead, A2A is the thing, like we're all using open claw and our agents are all talking to each other or whatever.

40:46 Sure, maybe. At the end of the day, I think we will have a limited set of MCPs that are always useful. I just think the gold, like the MCP gold rush just didn't happen. Like a lot of people use Contact 7, right? Cause Contact 7 is a legit helpful MCP. Same with the GitHub one, like legit helpful. And I think if I was in a big enterprise, like I would probably use my security tool MCP or like maybe one or two, like a dev portal MCP or something like that.

41:17 em I just don't think you're going to see this explosive, like, now I use a thousand MCPs to do, like it just degrades the quality of the agent so much. em But when it comes to like, from a security perspective, these tools in the right box, I'll focus on like, how do you get governance over that? from a security perspective, just from a pure like, how do I pick what MCPs are allowed, denied from my company devices?

41:41 How do I pick, can I monitor them in real time? And then they're expanding more into like IDE extension governance, which I think is something, I've seen you post a lot about like Chrome extension stuff. These are two of the things that have been overlooked in security forever. And so I'm just happy that... companies are getting into it. I wrote a blog post about Koi, which has already been acquired for 400 million, which is crazy. But the whole concept of like a developer workstations have basically gone ungoverned for so long because MDMs are so bad at it because developers need high quality or high privilege access to their systems.

42:21 And these tools are finally, I think, starting to give teams like some real governance and insight without interrupting developer flows in these comical ways like vulnerability scanning and stopping them from being admins and stuff. Yeah. And just obligatory shout out to John Tuckner from Secure Annex. Like he's been working so long, like browser extensions and ID extension scanning. He's doing a lot of cool work on that. For IDE integration, I kind of consider cloud code to be an IDE now.

42:54 And I think a lot of developers use it. And so in my head, like kind of like almost like an ideal security scanner. hooks into that and maybe like after, before, maybe a pre-commit hook or whatever, but before it commits the code, it can run a scan. you know, if it's not through Claude itself, through combination of Claude and external tooling, and then come back and maybe the the high priority findings that can just like apply the fix automatically.

43:25 Have you seen, you know. AI SAST plus the hook approach I think works well when you're combining those things. Because then you're getting some more, but then we're, mean, honestly, we're getting kind of close to an A2A architecture at that point, right? Where it's more like the security agent is talking to the coding agent, trying to figure out what it should do. Yeah. And then the other piece with what you just suggested is that like this is where we start going more in like the AI coolie direction of like everybody's using these isolated clod bots.

43:59 And this is where like developer isolation comes in handy, which is on like back on the runtime side, there's someone called Adara and then there's some other providers doing it too that are more like container isolation. Cause we treat containers like they're VMs, but they're not. And so to actually give agents isolated environments to do work, like that's a huge trend. And then that to your point becomes sort of an evolution of this MCP governance idea of like really you're governing your agentic fleet for security issues more than like developer endpoints.

44:36 But I always just try to be hesitant on some of that stuff because I don't ever want to just be AI is now like I'm never coding again. Yeah. And the last one here in your key takeaways is supply chain security. So you're saying it's more like malware detection, package health, secure by default. Yep, it's more just to highlight there's a lot of things happening in supply chain. I have wanted malware detection in supply chain to be a priority for, I don't know, like five years.

45:05 And I think it finally is starting to become one after ShyHalood was so bad for people. But at the end of the day, like we just run these open source packages on everything and I don't check them. And I know if I'm not checking them, like devs aren't checking them. And so we just NPM install and let it rip across everywhere. And so not enough tools we're focusing on detecting upstream malware, whereas now I think quite a few of them are, and we're actually getting into a good spot on that.

45:31 And people are starting to think more about like NPM cool down periods and stuff like that to, it's this delicate balance, right? Like honestly, we've told developers like patch as soon as you can all the time, and then you do that and then it's like, okay, well not that fast, because then we could get malware. And so it's this weird like middle ground of like you want to patch, not, this goes back to my windows example of like, We have this in a Windows system.

45:53 It's just scary because it's different in open source where like really what we're talking about is a test group, see if there's any failures, then roll it out fleet wide, but that's really hard to do. Sounds good. Yeah. And do you think like with AI coding, like a lot of people are just like, know, like code their own dependencies and so like replace dependencies by like just bringing them in? No. that's an anti-pattern, which I guess is, don't know how much of a hot take this is, but like, I get mad when Claude doesn't just use a package.

46:21 Like, just do the thing. Like, why are we rebuilding the library? Like, the library's really good and it makes everything easier. The code's more readable. It's shorter. Like, just use the library. I don't know why. Yeah. Yeah, I mean, I will cover that depends on the library. it's like an even library, like just don't use that. yeah, yeah. Yeah, for a real use case. Yeah, yeah. the meme ones when they get popped, it's usually not even that big of a deal because people don't really use them on mass, right?

46:54 em The ones that are really bad is like post hog and like the people that got affected by shy, Halued are like real ones that people use. Yeah. there's many more details in the the blog post that, you know, everyone should read at their own pace. But Yeah, we really covered a lot and jumped around a lot. I think one thing that surprised me a lot is this one here. people said like, I want AI pentesting.

47:22 It's like from my perspective, I would much rather have AI sassed and AI remediation rather than, and I thought like maybe here people just like found AI pentesting to be cool in concept or were more familiar with the idea than any of the others. Like, like, I don't know why people want. So, I mean, I guess this is just like DAST or placement for DAST. em mean, we try to talk about by next year, AI pen testing and DAST will be converged, I think, from like a messaging standpoint, because they're really doing similar things.

47:51 m in current like traditional Dast is in my opinion absolutely trash. Like I just always hated using Dast in general. Yeah, this is the last section, yeah. I love like escape and stackhawk are two that I've used on the API side and they're really fun, but they come with an extra lift of like setting them up properly and they're very API first, but I'm highly sympathetic to people who use one of these older like crawler based ones and they suck.

48:17 So that's I'm like old school Dast is dead is what I usually say. yeah, there are a lot of bad Dast out there, but some of the API ones are doing cool stuff. Yeah. but I totally agree with your take like converging them with AI pentency can be like one thing and it'll be much better than anything we have right now One last question I have was around SAS so I forgot where in the report mentions, but a lot of people were worried about You know generate using AI to generate code and they were saying they were finding a lot of vulnerabilities in that I from my perspective, it's like, I don't care if monkeys are writing the code.

48:58 I don't care where the code's coming from. like, I want to apply the same controls wherever the code's coming from. I mean, in your opinion, like, is AI actually introducing more vulnerabilities? I think they are more dangerous vulnerabilities because they're the kinds of ones that come from bad prompting. Where a develop, when I get a ticket to implement a new API endpoint, it has gone through a PRD process. It's like, I have a clear idea of where it fits in the stack, where I'm familiar with the application.

49:28 I know what I'm building generally. Whereas with an AI thing, I just say like, go make the new endpoint and then it's going to do it. But it's not going to know like the context of the app, the security controls, where it lives. And so I think that's why I'm such a big believer in the AI SAST as the way to do the AI guardrails. Because you need that organizational context, because that's what the agent doesn't have.

49:49 James, thank you so much for joining. Is there anything else... Is there anything you want to mention before we close? Where can people find you? Yeah, no, thanks for having me. I do post on LinkedIn more than Twitter, which I know is annoying for a lot of people. But I just, the Twitter word limit, can't, I mean, you're looking at a 72 page report, like I just can't do it. But so unfortunately, LinkedIn, you can spin up the LinkedIn account and follow me on there.

50:12 There's also the newsletter, which is either through latio.com or pulse.lacio.tech is like the old one that's like the sub stack or just message me on LinkedIn or Twitter or whatever and YouTube. So yeah. Very cool, thank you for coming. Thanks, man. It was good to meet you. Good to go through it all.