Data Poisoning, ML Security, Graph Theory, & Conferences with Maria Khodak

0:00 does this model's intention line up with like what I, what I'm seeing on the screen essentially. that's a little bit subjective, like when it comes to like a friendly, like conversational chat bot, like Tay, I think it becomes a problem when you have like these LLMs that people use on a daily basis for like understanding things and sometimes you can kind of blame it on like, like chat GPT is just not trained enough. Like this is not like the latest version.

0:28 Like it could be better And like, sometimes you can't tell if it's just like not current enough or if it's actually been poisoned or not. So when you're studying computer science at some point, you learn about graphs. And once you've learned graph theory, you start noticing like everything is a graph. in networks are graphs, your code has call graphs, compilers use control flow graphs, dependency graphs are in all software, algorithms are graphs, abstract syntax trees in static analysis are graphs, bloodhound, offensive security tools is a graph, just shows you graphs, I guess.

1:03 And so I... wanted to talk with a fence of security researcher and PhD students Maria who gave 12 conference talks last year which is absolutely insane. Yeah, hi. My name is Maria. actually this kind of started two years ago, or one and a half years ago when I went to DEF CON for the first time, and that was my first security conference ever. And I wasn't really like, I liked being in security, I was doing like, pen testing work back then.

1:37 But I wasn't like, super into the research side of it until I went to DEF CON and A lot of people like just were doing like crazy random things and that really got me into doing my own research and I kind of just on the side, I would just like kind of look into graphs because graphs were really the only thing I I enjoyed in computer science in my undergrad. So I kind of was like, how can I apply this to security and also like how can I apply this to machine learning security?

2:09 And because machine learning was starting to become a big thing and I learned a lot about machine learning security at DEF CON. So I just was like, how can I apply like my love of graphs to understand this vulnerability more? Because like, I think graphs are like an intuitive way for humans to process information and Data poisoning is literally poisoning data or information. So I think that is like an intuitive thing that humans can understand better.

2:40 And that's why I wanted to present upon it. So you started researching just the topic of data poisoning and how do you go from idea to actually submitting to a conference and speaking there? Yeah, I guess I was really inspired by lot of speakers at my first DEF CON. I was like, how do they do that? How do people go up and speak? And I actually, I was working for a company and I gave an internal talk then.

3:06 And that was really nerve-racking, actually. I was actually really scared and I was sweating and stuff. But. But then like, don't know, I applied to other places outside my company externally. And I just like went to, I don't know, like I just would hear about these conferences. Like I would just, you know, be on scrolling, security Twitter or something and hear about a conference and be like, wow, like it would be so cool to speak there because like I've seen their talks like on YouTube and they had some really high quality talks like.

3:37 That's what I thought of. My first external talk was at Insomniac in Switzerland. And I was like, wow, they have some amazing talks, amazing speakers, and it would be so cool. But I didn't think I would really get in. But luckily, I got into their speaker, their group of speakers for 2025. And I don't know, it was kind of addicting to. of get into places and have that feedback of like, oh, people want to hear about my idea.

4:06 But ultimately, I didn't really give talks because I wanted to preach about graphs or about data poisoning. It was more so like I wanted to improve my own research from other people's ideas that people would listen to my talk and then be like, what if you did this instead? Or what if you tracked this metric? or what if you like added this to your talk and I was like, that's actually a really good idea. And I like, I really liked the feeling of like, wow, my research, like my ideas are getting better because of feedback.

4:41 So it was like this feedback loop ultimately, like literally a feedback loop. So. such a good perspective. Yeah, like the main benefit of giving a talk is just getting feedback and getting ideas. And I guess when you put in that perspective, it encourages you, it gives you courage of like going out and giving a talk because you feel like, I'm going to learn something from that. Yeah. Did you get like, I feel like sometimes in talks like, not always, it's like kind of rare to find someone who's like, in your field enough that they would give like good feedback.

5:11 like sometimes like people would give their ideas that were actually like, kind of like offbeat or like, this isn't relevant at all. And maybe they misunderstood my talk and that like, kind of gives me to, I, to like thoughts or ideas from what that person just told me. Like one, like maybe my content wasn't clear enough and I need to clarify something so that like my message gets across. But two, like, Maybe their idea actually reminds me of something else.

5:39 And maybe I don't implement their idea, but I implement something that is kind of tangential or similar to their idea. And I can kind of think about like what people ultimately want to see from my work. Yeah, really good perspective. we're going to talk about your research and data poisoning. But first, guess, how did you end up stumbling into being a PhD student? That was such a crazy story. like, so like going to a lot of conferences to give a talk also means I like network a lot with people and I networked with somebody who knew my advisor, who I met at Recon, Montreal.

6:18 And yeah, it was like really crazy. And I met my now advisor. or advisors at like a hotel bar at Recon. then they were like, your research is kind of cool. Graphs are cool. Like control flow graphs are a thing. Like the lab I'm in currently is very like focused on formal methods, on like compile, like a lot of people are into compilers. A lot of people are into like also hardware. like we're like a whole part of our lab does like glitching, which I'm not really a part of, but it's really cool.

6:52 And a lot of those stuff has like graphical applications. And basically the idea of like that my advisors pitched me was like, you can study graphs for her fun and kind of get credit and get like papers out and also get paid for it. And that sounded awesome. So. Like I was like, I hesitated for a while though, to be honest, because like a lot of like people in security will say something like academia, like security isn't like, it's not good for your career if you do academia as a security researcher or something along those lines.

7:31 And I like, wasn't really worried about that, but my ideal scenario would be like getting recognition for my ideas, being able to do research and also being able to like provide something of value to like either a company or a lab. And I guess like, like I just like took the academic option because like the network you get with academics is really valuable. But it really depends on like what you're studying. So like, A lot of things, like some things in security are like easily learned, I think.

8:02 Like I think some things in like pen testing, for example, can be picked up pretty fast if you have a CS background, which like was my situation before going into academia was working in pen testing right after college. And then Like, I guess there's other things that are like a lot more difficult to pick up and academia helps a lot. But academia is not necessarily like necessary for you to work in like formal methods or formal verification, which like I'm starting to pick up on a little bit now as an academic is like formally verifying graphs.

8:40 So like, right now I'm working on an attack graph system and we're formally verifying that with Datalog. If I was working on this on my own, I was not an academic, I don't know if I would formally verify it with Datalog. I don't know if I would have that idea unless somebody went to one of my talks and was like, why don't you formally verify this with specifically Datalog? But the network that I have now of academics who have these ideas of like, you should actually like formally verify this could be really valuable.

9:13 It's like, okay, that's actually really helpful and makes my output better. Ultimately. Nice. Cool. Okay, and so I guess real quick to start going into some of your research, you wrote this tool called Graph Leak. So it is on GitHub here. And essentially, you can drop a CSV of a graph. And so you have a sample one here. I just uploaded this one. Maybe you can explain what it is.

9:44 I have no idea what it is. But it's a CSV source target. And so it's defining edges. And in the graph, a node is just like the single. point and an edge is a relationship between two nodes, right? Yeah. yeah, this is just like a really generic, like graph of nodes and edges that are just labeled with numbers. It's not really a, graph of some, like some specific data. Like this was just like a test graph.

10:08 complex one, pretty messy, I think. I did that to stress test it a little bit. But yeah, in the graph, in the app, you can click on a few of the nodes and you can delete them or add them. You can delete selected. I think that's one of the easiest things to do. then, yeah. you can just add things manually, I see. And then here, I thought it was interesting, can also display different formats, I guess.

10:39 Oops. Yeah. Yeah, D3 kind of goes crazy. still looks good. So why did you make this tool? cause I wanted to like, I had this like idea of like visualizing graphs using data poisoning. at first I used like Gephi for all my visualization. like, which is an, it was just like an app that a lot of people get introduced to in college and their network science or even graph theory class. But for me, it was my network science class.

11:11 And I just love Gephi. It's like. It's just like a free open source app that like, it's like a GUI and everything. And it's just like a graph tool. Like you can run so many things on it. Like people have so many like algorithms for it and like it visualizes so many things. And it visualizes like massive networks. Like I remember for that class, me and my partner did like this project visualizing like Twitter feeds and like Twitter sentiment around like certain like conflicts and like how like Twitter networks connected to one another, which was super cool to visualize and present on.

11:55 And then, and I was like, why don't like I have a really like mini Gephi basically, like I love Gephi. Mm-hmm. I love visualizing these things, but I want something in a browser. Like I would just want people to be able to like just open it in Firefox or whatever. And just like, so let's start playing around a super easy, like really not complicated at all. But like, yeah, data is possible to be manipulated. Yeah, super cool.

12:23 yeah, so data in general can be represented as graphs. Many things can be represented as graphs. And so you're approaching the idea of like data poisoning can be visualized through a graph. Yeah, exactly. So can you walk us through what exactly is data poisoning? So data poisoning is typically a machine learning vulnerability because data poisoning is often used in the context of training data sets. So you have a machine learning model that is supposed to like train on whether you detect like a dog or a cat.

12:57 And it's like doing all like you're training it on dogs and cats. And it's like saying like, yes, this is a dog. Yes, this is a cat. And it's like, it's getting all of it right. And then in the machine learning cycle, you teach it that like, like you have a bad actor could be yourself if you're experimenting and you just start like inserting pictures of like birds into the training cycle. And you're like, yes, this is a dog.

13:21 Or yes, this is also a cat. And the birds become like a bit of a poison into the data. Like that's not supposed to be there. The model isn't supposed to think that birds are dogs or birds or cats. It's supposed to say like, no, that's not a dog. No, that's not a cat. And that's like an example of like really small scale data poisoning. Two examples that I use in my talks are of like large scale data poisoning actually relate a lot to Twitter.

13:49 like Tay and Grok are like very well known data poisoning. case studies, I guess, or like, Tae trained on, I don't know how they released a model that was already like functioning. And then it's somehow trained on other tweets and its training process. And it started saying like, hateful comments. And it was like really crazy. And of course, Twitter would do this even back in like 2016 or whenever that was.

14:20 and then Grok was like similar, but Grok was actually different. Like Grok started saying like crazy stuff last year, I think, like around May. And people were like, why is this happening? But actually I think they just removed like a guard rail. So I don't really know if it's like really data poisoning, as much as it is like someone at X removed a specific guard rail or like a, like a prompt. Mm-hmm. that was like, do not say these things.

14:48 And it started saying those things, of course. So. either a missing guardrail or maybe maybe even like unintentionally the poisoning in that like they like accidentally included a bunch of data would like bad things that they you know end up like just training on it Yeah, like they didn't tell it to ignore like this bad data right here, or like these tweets or comments. And so data poisoning is relevant now because I guess so many people are training their own models and the datasets are becoming even larger.

15:21 And I guess one challenge is that given that these datasets are so, so large, how do you even like looking at a dataset, how can I even tell if this dataset has been poisoned or not? Yeah, and that's honestly like all about like the outcome. Ultimately, like it's not like it and that kind of is really hard because you can't like intercept data poisoning as it's happening. It happens and you're like, okay, clearly Grok has been poisoned or clearly Tay has been poisoned, but like you can't detect it for prevention measures.

15:54 I guess you can like stop it from worsening. And that's like lot of things in security, in my opinion. Like a lot of incidents are like, okay, we've been breached, now what? Like, now we have to like stop or like put like some duct tape on like the water that's coming out or something from like the breach. So, so ultimately like data poisoning is also one of those things where you're like, is this happening? Okay, it's happening.

16:18 Let's like see if we can like... pull our model back or like add something or like just try to correct course correct it essentially. Mm-hmm. Yeah. Nice. so in your research, were you studying known cases of data poisoning or were you just poisoning your own data sets? Yeah, a little bit of both. my original kind of like proof of concept to myself talk about like, am going to poison these nodes using a Python script and then I'm going to upload those graphs to Gephi, which was my first talk.

16:56 I was just like experimenting with poisoning things myself and then also like why like what can I poison to like elicit like a certain outcome and also like like why would poisoning like a cluster be important or poisoning like in my first talk I have these Java dependency graphs and one of the Java dependency like it's a massive network and there's like a big cluster of a certain like a certain Java dependency and yeah definitely.

17:28 So this is your blog post where you cover that and it's this one, Yeah. And you can see that there's like a cluster at the very top. and like that cluster. Yeah, exactly. That cluster really stood out to me. And I was like, what if I poisoned that and poisoning can, doesn't mean like modification necessarily. You can also, it also can mean like adding to it or deleting it or deleting data. so, so like my choices were like, do I like add to this?

17:59 cluster, do I delete the cluster off the network entirely and the edges that go along with it? Or do I modify it? And in my project, I chose to modify it just because visually it conveyed my point, what I wanted to talk about. Which is like, yes, there's a cluster and it's a cluster of dependencies, but there's also edges that connect to it. Like does, I kind of wanted to kind of like very, do a very shallow exploration of does modifying this cluster affect anything else on the rest of the network?

18:39 And does, like how does data poisoning like proliferate through a model potentially? Now this is just like, like through my script, it was like very like, I'm just poisoning this cluster. I don't think this is going to poison anything more than the cluster. And that's what happened. But it's like something. Yeah. But that's because it was em just like one iteration of poisoning, right? em Like this is just like, if a bad actor were to poison like Java dependency, if you have like a model that's training on Java dependency graphs to like, I don't know, generate like, I don't know, like a new dependency group or something.

19:27 Like how would that look after being poisoned? And to me, like it's hard to say like, how exactly does this actually like, how does this appear like after the machine learning training cycle? And I think it's like, it's really hard to tell and like sometimes you don't know, but you can kind of like, do your best estimated guess. And if I modified this dependency graph, then maybe it would just like be like ignored entirely.

20:00 Like if I just poisoned them to become nulls, which is what I did. Like I wasn't, I didn't creatively like make them a different dependency or something like that. It was just nulls. Then. you're just training on the training data is kind of getting ignored and it's just not training as much as it can be trained. So the scale of poisoning like varies so much between networks. And sometimes like, it depends on like what exactly you're poisoning.

20:31 And like, like on the bigger models, like the huge like LLMs like chat GPT. or Claude, they actually had us a research paper by Anthropic I believe, where they actually poisoned like 250 documents out of like billions, I think, which is like such as maybe millions, I don't know.

21:03 It was such a small amount. but it was still enough to create a backdoor in the LLM. So it's always context dependent, like about what your model's doing. What would you define as a back door in the LL? so, like a backdoor and the alum can kind of mean like, okay, like the, to kind of, kind of like bring this back, the first like paper and like, I'm really, like, really putting my academic hat on.

21:34 the first paper, that kind of like defined data poisoning as we know it was, the bad nested. a paper from 2019. And basically they kind of made like a traffic sign classifier for autonomous vehicles. And the autonomous vehicles would train on like whether the stop sign would have like the irregular stop sign, but if they put like a post-it note on it, it would just go right through the stop sign.

22:10 And that's kind of like what I mean by a back door is like, you're, you have like the, the car is supposed to stop at the stop sign. It's not supposed to continue. Like there's a, there's, there's an action, like, like a violation of trust, I guess, for the user that's being violated because you put a yellow sticky note on the stop sign and the car doesn't know that it's, it's a, it's a stop sign. It's not a stop sign with the post-it note.

22:38 totally, totally, yeah, yeah. So like a backdoor, think in this case, in my opinion, it just means like, there's an outcome that's wrong. Like the model is doing something that it's not supposed to do. And I think like, it's really, it's like a, to me it's like so interesting because like Claude, ChatG PT, like Teslas, like Teslas you have to buy of course, but. ChatGPT and Cloud are like free and accessible.

23:09 And I guess Teslas are accessible to some people. I don't know. But they're like, these are accessible things that people use for everyday life and rely on. So data poisoning is actually like, to me, it's like kind of like this really, almost like amorphous vulnerability that's not very concrete. in a lot of like these bigger cases and it's really hard to study unless you have like a paper from Anthropic who are like, who literally have access to Claude like, like they're, they're, they want to study this.

23:42 They have like a good incentive to study this because backdoors are dangerous. Like they're aware of it and they put enough resources to be like, okay, we're going to see the impact of like data poisoning on our models. Yeah, there's a lot of value in studying data poisoning for them. And it's also something that I think is really cool to explore. a lot of people outside of Anthropic and OpenAI are also exploring data poisoning.

24:15 And it's something that I want to continue exploring in the future as well. Very cool. Real quick, I'm sharing, can you talk a little bit about your network Poisoner repo? By the way, I was very impressed that when I opened it, I saw like, these are files from like two years ago, but just two days ago, you were like, there's a typo, I need to fix it. m like updated my website and then I, for some reason, like I mixed up my repos and I was like fixing the site for my fixing my personal site.

24:48 And then I accidentally like clicked on network poisoner I was like, there's a typo there. I should fix that. so yeah, that's actually kind of but yeah. exactly and like, did you make it? like, this was like, just me experimenting, like, can I make a script that like, because I okay, so the reason why I got into like a lot of these machine learning security vulnerabilities is I took a workshop at DEF CON in 2024. And which was really great.

25:20 And I learned about data poisoning. And I remember in the class, like, the the teacher was going through a script in Google CoLab. and like just going line by line, like kind of like a Jupyter notebook where you can like run things line by line and like see how things change over time. And data poisoning is like how things change over time. So I was like, I went back home. I was like, okay, I got to like try this out and make a script myself.

25:49 And like, then I was like, what if I poisoned like nodes instead of like Because the way they kind of taught us data poisoning in the script was just numbers. we were poisoning MNIST, which is a very basic image recognition or number recognition. It was very famous for being the hello world of machine learning, basically, where the computer is like, OK, that's a one or, OK, that's a two.

26:20 Like within an image, recognizing the characters to determine if they're in a digit or not. Yeah, yeah. like, it's actually like true or false. Like, is this number correct? So, that's what the class was teaching. And so, like, true or false, kind of like brought me to like, maybe we can do nodes somehow in this.

26:53 What if we do graphs instead? So that's kind of like what I was like, okay, I want to poison nodes and I, maybe I want to poison edges too. And so in this case, you have a graph which is, guess, a CSV file of nodes and edges, and then some of them are added maliciously or removed or modified. Yeah. Yeah, and I did a lot of experimentation. like here, like in Graphleak, in contrast to the network poisoner, I'm like, you can choose which nodes you want to poison essentially.

27:27 yeah, like you can click on some. so like I go like I want to poison F and in this case like for example I would like delete it and that would be example okay But in the in the network, Poisoner, I'm like, do you want like 20 % deleted? Like, do you want like more modification? And I kind of had that idea because of like the the there was like this original idea of data poisoning that like you need at least like 10 % for poison to be detected, like 10 % poisoned or higher.

28:06 It used to be like you don't need 250 documents out of billions, like that one paper, hermeneutropic. And that idea is kind of shifting. And I do think, again, data poisoning is very amorphous. It is so context dependent. Like 250 documents is probably enough to, to poison a large language model. But for a small like no data set, I actually think that like less, less is actually less in that case.

28:39 Like actually, but I don't actually know. And this is like something that I think can be studied, which actually is like a pretty interesting, like. potential research question. Yeah, that's super interesting. And so the idea is like given a data set and a model, we can poison or modify basically an arbitrary percentage of that data set. then depending on how much we modified, can tell if or like, would we be able to detect if it was poisoned or not for that percentage, I guess.

29:13 And that varies a lot per, I guess all the time. There's no consistency, I assume, on that. Yeah, yeah, and like that consistency or inconsistency is something that like could be cool to study more on. Okay, so back to this graph. So this was describing Java dependencies. And so is the idea here that like just within my software, some of the packages or dependencies I could be using could be like maliciously taken over? And is that what I consider like a poison data set in that example?

29:46 actually, I don't know, that could be like a different thing. Like if your dependencies are taken over, I think that's like, that kind of sounds like a separate issue entirely. like data poisoning often implies training on data. Like the data is being poisoned and then the data is like What's the data's usage in this case? like, is, like if I'm just trying to detect if I have like a data breach or something or like, no, my Java learning objects have like some, like somehow have like some kind of like new attack vector where like now something else can be like, like now I have like.

30:31 remote access to my system or something through my dependencies somehow. I think that's a different issue entirely. And it's not even machine learning security related in that case. It's just like, your java.lang objects might be taken over. Sure, it's possible to visualize that graphically. But there's a lot of things in security that can also be visualized graphically outside of data poisoning. Like, I was inspired by Bloodhound, where you have, I believe, Active Directory graphs and how credentials can be connected to one another, for example.

31:15 Okay, yeah, that makes sense. So like in this case, like it's not about like one person's software. It's like, if I'm looking at like all dependencies as a whole or like a really large set of dependencies, and I'm running any sort of machine learning on them, for whatever reason, then that's where like data poisoning would kick in and affect me. Yeah, exactly. But I do want to talk about like data provenance, is like, which is very similar to data poisoning.

31:40 Actually, provenance is like how your code changes over time. And what's that difference is kind of like in your git commit history, like what lines of code changed and what lines like did you remove? Like it will show you the before and after, right? And that's actually super similar to data poisoning, right? Because you have like, ultimately what you want to detect is what changed in the model and what remained the same. And, or like in the trained data rather.

32:13 like data provenance is not like a machine learning concept again. It's a like, did my data change kind of concept. And I think people like, I think like security, like a lot of aspects of security would benefit from some kind of provenance visualization in general. But like, I guess the context that I was researching in was machine learning. And so the provenance would kind of be very related to data poisoning in this case.

32:47 That's a one. Cool. Very good. And well, I guess, there like agents or LLMs you can use to help detect data poisoning? I think that an agent could like, or like if you present an agent or even a human, honestly, like with like, I had data like, and if it's been clearly poisoned, like if you present it with a data set and like, or, or a graph or anything and you're like, has this been poisoned?

33:21 Yes or no. Like tell me the difference between these two pictures. And you're like, yes or no and you can kind of see intuitively. So it just becomes like a data labeling problem, just like reviewing the, yeah. yeah, after a while it becomes that and like an agent can definitely do this for sure. I have like no doubts about that. And I guess one of the problems for most models we use these days, we don't know what data sets they've been trained on.

33:47 We don't have the data set. And so you mentioned earlier that to detect if something's poisoned or not, you just have to see the end result and see if it's behaving correctly or not. And so I assume some people have tests to see if the model is behaving accurately. are there any other indications? that would tell us if data poisoning has occurred, or is it just like you just have to like kind of like based on vibes, based on the results.

34:12 Yeah, it's like it sometimes becomes like a big vibe thing Like clearly Tae was had the had some bad vibes and had like it was like if if like Socially it was more acceptable to like say hateful things, which it is not What it what it be did detected with like, cause it's a, it's like a violation of intention and trust at the end of the day.

34:43 so like, so like, does, does this model's intention line up with like what I, what I'm seeing on the screen essentially. yeah. And that's a little bit subjective, like when it comes to like a friendly, like conversational chat bot, like Tay, when it's like, I think it becomes a problem when you have like these LLMs that people use on a daily basis for like understanding things or like, and sometimes you can kind of blame it on like, like chat GPT is just not trained enough.

35:14 Like this is not like the latest version. Like it could be better or something. And like, sometimes you can't tell if it's just like not current enough or if it's actually been poisoned or not. I guess that's kind of scary. Like you never know if you're like getting accurate information. If like you're a user of like copilot, like a lot of companies just copilot, right? Like part of Microsoft office. And like, is it, is it like presenting accurate information to like people, to like a lot of people who work in the corporate world as like, as like non-tech people, you know?

35:51 so like what they know, like if you ask like co-pilot to make you a spreadsheet or something of like data with this data, what if it like unintentionally poisons that data sheet, right? Or like it completely processes it the wrong way or it like adds like two at the end of every formula by accident. Like not that it would do that necessarily. I don't think our models are that bad. But like, you know, like I think there's room for error and sometimes maybe that gets completed with data poisoning.

36:19 Nice, nice, cool. Very cool. one last question I had for you was since you gave one million conference talks last year, what were your top three conferences? Oh my gosh, that's like a really tough question. I guess like there's a different experience as like a speaker and as a participant. Because I've definitely gone to conferences and I didn't speak out like at Recon I didn't speak but that is like my favorite conference by far. Yeah.

36:51 Yeah. Yeah, it was awesome. The vibes are amazing. As a speaker, like Like I think, like I loved Insomniac my first conference where I spoke. That was awesome. I also went to NoHat in Italy in Bergamo in October. And that was really nice. Yeah, I'm a, I was a big fan of that conference. like, yeah, it was like a small like researchy conference. And, I also got the chance to go to Echo Party.

37:24 yeah, that was really fun. It was a really cool conference. lot of really cool, like vulnerability researchers there. been to Echo Party. Was there... Is it like mostly Spanish or there's a lot of... it is. And a lot of like, this is like some speakers like that I connected with as well there that were like speaking from America, or like either talks in Spanish. And I was like, okay, maybe I should have done that. Like, not that I know Spanish or something.

37:52 But that would be really cool. But it's not like I could answer questions in Spanish either. but But yeah, was, the energy was really cool there too, so yeah. I think those, like as a speaker, think Insomniac, Echo Party, and No Hat are my top three. and attendee would be recon and Yeah, Recon. Yeah, I mean, a lot of people have mixed opinions on DEF CON, but DEF CON was, like, the years I've went, has been pretty life-changing for me.

38:23 So... in it was at the Las Vegas Convention Center both years? Okay. Yeah. Mm-hmm I never experienced the Caesars or wherever it was in the past. But it's like, it's like definitely mixed. But like, I think, like, Defcon is like, has been good for me. But I understand when people have mixed opinions on it. And I agree with some of those mixed opinions. Yeah, I like to recon and Still really like Insomniac, big fan of that one.

38:53 Very cool. Okay, anything else you wanted to say before we start? don't think so, no. I guess, where can people find you? yeah, so I have like a personal website at this not dot my, so the SNA dot my, and I also have a Twitter account of this not the SNA and then FB, FVR. And yeah, that's the extent of my social media.

39:24 Awesome, awesome. Very nice talking to you. Thank you for joining. for having me. This was so fun.