Transcript
0:00 Losang which is an AI lab based out of Bangalore and at Losang we are very keen to ask foundational questions in biology and artificial intelligences. So anything to do with let's say how does the brain work what's the role of consciousness to things such as catastrophic forgetting and continued learning. I mean this is the curse of being so curious. we just want to understand a lot of things about intelligence and that's what loss funk is about and within that subset I think we're super interested in AI for science as well so this is why both the durus on this call u are also very interested in AI for science so in that spirit let's let's get started with the session so this is a rough outline the first thing is we'll literally kickstart a demo and then I'll talk about you know autoyla and other auto research systems. We'll talk about what does research philosophy mean? What is research taste? then the role of humans, where where is the human input needed? I'll be showing all the prompts that are being used in the system and what do those prompts use. then we'll talk about where is the feedback needed.
1:15 Talk about some examples. Then the modifiability of the system and the failure modes. I think one unique thing about this session is that we'll literally produce a new paper by the end of the session. So I wanted to give a live demo and for that live demo let's maybe take a few minutes and I would love to have some suggestions on the chat about what research topic do you want a research paper on by the end of the session. So whatever in is in your mind can you type few ideas here? I'll pick some of those ideas and we'll just get going. So I have AI and finance, I have ethical AI, Indian outbound market. U the constraints really is let's say the research should be possible within let's say 45 minutes the time it'll take for this. So, research taste in LLMs, AI and mental health and finance. What else? Maybe a few more suggestions and we'll get going.
2:29 New attention. Okay. Duan, do you have any suggestions? Which one should we pick up? There's lots of good suggestions. We'll we'll we'll pick an idea where we can use perhaps open router and open AAI like the APIs to do stuff because training a model we can do a we can train a model for sure but >> so I think there was one one comment from snor about math capabilities evaluation for a small model maybe we pick up like a small math capability which we want to evaluate and then we do something around it. okay. We can try that. Let's >> Or maybe if you thought of something before the presentation, maybe just go with that.
3:16 >> I mean, I have a live example, but let's I I I liked let's let's just pick perhaps something more generic that we can try AI for science itself. Let's see what what does it do and I think it'll be on point because it's so meta. So what I'll do is I'll open up of course my terminal. Can you all see my terminal? >> Yes. Yeah, we can. >> Awesome. So I'm just kickstarting this.
3:46 so CCO mine is really I mean it's a wrapper effectively we're using clot code and I'm starting clot code in autoilar directory where it has just the instructions that will go through. What does that instructions do? So I've already renamed var file which has contains which contains the prompt to claw.md which means it will just automatically execute it in its context. So we can literally start with saying AI for science. Let's say what happens if I just say this as a topic. Now as an input you can give a topic. What you can also do is like a specific paper or a set of papers. It's just it can be literally anything.
4:29 and the framework does the rest of it. So, so I I type AF for science and it'll start reading the directory as you can see it'll reading the research philosophy that we'll go through what research philosophy are we talking about and after a while it'll ask me a few questions but until then let's get back to the presentation and get going. Yes. So before we dive into what autoila is a little bit about what is research of course different people have different definitions of research but to me broadly it sort of bifurcates into two categories.
5:16 the first category is let's make something better and the other category is let's create new knowledge. So let's make something better is more sort of aligned to engineering and optimization in true sense and for example it can be you have a material you have to improve that material's property and you have like a specific target to meet and then you keep on changing things and see what happens which ones push it to closer to your goal and which ones push it further from the goal. And of course when you say let's make something better you require verifiable feedback you want feedback from the from the sort of environment and this is where a lot of auto research current auto research systems are focused on. So if you know Andre Karpathi's auto research repo it is like a set of prompts that you drop in and it evaluates your sort of model or whatever else you give it for and it simply sort of takes different steps to minimize or maximize a metric that you given. This is what research in industry mostly ends up being about which is about you have a goal and you're trying to make it. but when you talk about the second form which is let's create new knowledge this is closer to what academic researchers do right where the goal is not to often not to improve something specific when we talk about soda in a IML that is closer to engineering but lot of research papers that you see in ICML ICLR and new rips etc and in journals like science and nature they're about can you create you know new knowledge where there is no verifiable feedback out there. And when people say, you know, research depends on taste.
7:10 this is often what aspect of research they're talking about wherein if you have no verifiable feedback, no signal coming in from the world. You need to codify some taste which is what then your auto research loop can take and start executing. So my repo auto which we'll talk about you know does this it's more around let's create new knowledge and less about you know how do you improve something and of course you can do that but the whole focus is to create new knowledge there so you can try downloading this after the session from my GitHub go to github.com/partora or you can use this QR code it's just actually literally just some markdown files and nothing beyond that. Let's see if the session we started with it sort of asked some questions to me. So it did. So let's see what really happened.
8:11 What it did was you know u we'll talk about it but the system quickly figured out that how what my system specs are. So it knows that I have an Apple M3 Max and u it sort of also ingested the research philosophy I have on what counts as good good research and then it ended up being sort of giving me within the constraints I have for example the default constraints I've put is that you need to do it in two hours and on the system I have with the APIs I have so it ends up being suggesting some of the very broad ideas for the exploration. So let's see what exploration are we talking about.
8:58 different sort of suggestions it's giving. The first suggestion is LLM scientific reasoners. the second is molecules. Third is agentic scientific discovery and the fourth is interpretability of science models. and I think we'll go with one just as a demonstration but this is where the human input comes in. So when a lot of people ask you know what is the role of humans when you're automating science this is really the input where we are talking about. So let's select this slice and u let's say time budget want to do it in 30 to 45 minutes API budget how much do we want to spend let's spend just $5 and you submit the answer we'll get back but if if if I talk about what's in the repo autoiler the first file you can see is research philosophy.mmd.
9:59 so the default research philosophy that's in the repo is inspired by what we've been learning at Los Fun for a long period of time now on what is a great research question you know I mean at one level it's really subjective everyone has an opinion about it but we've been looking back and analyzing papers that have won let's say the best paper award at newips ICML etc or the papers that have gotten into science nature And even if you see the greatest discoveries of the past let's say from Einstein, Newton, Darwin etc. U very very few sort of common attributes emerge which is that great research makes claims that are surprising to experts. So the whole point is that when you do research you're talking to an expert who spent several years or even a decade or multiple decades in a field. So research communicates to an expert and not to the general public per se. and research is >> Sorry, someone has a question.
11:06 >> No, I think it was a issue with someone's mute. Sorry, keep going. >> Okay, so yeah, so great research makes claims that are surprising to experts. They are fruitful. Fruitful really means they have lots of downstream consequences and they foreclose alternative explanations, which means they are statistically very rigorous. and you know they are feasible you should be able to do it and if this you know sounds a little bit abstract I think you can go to our substack which is letters.los.com lossfund.com and read in a lot of details what I mean by that.
11:36 But the general point is that if you are running like an auto research system to create net new knowledge and there is no verifiable feedback there you need to put some attributes some taste into a document that LLM can use to grade potential directions otherwise it's a problem of infinitely many choices right and this is the choice that we've made in autoilera but it's just highly customizable you can and should modify is depending on you know what you believe good research is about.
12:10 so as I said you know in the in the repo it's literally a markdown file which I took my article and just converted into markdown and that what's goes into the u markdown file that the plot code ends up reading. Let's see if we have again more inputs. Okay, so it's still churning along and we'll let it churn in the background.
12:40 but the second file that you had seen was va.md which is the meat of the whole system and we'll deconstruct what it contains step by step. By the way, if there is like a clarificatory question, please put that in chat. I want to make sure if something is super confusing you know we address it right up front instead of at the end so Duv can really pick that up and ask me that question. So I'll take a pause and see if there is any questions so far before we get to the second step.
13:14 >> Yeah Par we have a bunch of general questions around how do you ensure the research is sound if you're not an expert. like are LLMs able to come up with knowledge which is surprising to experts? Nothing genuinely confusing. I think one thing that you could clarify is a lot of people are asking if this works with a $20 subscription also or do you need the $100 one? It's like three people have asked. >> I I think this would work with $20 one.
13:38 it may pause in between so you'll have to wait a little bit but I don't see why it shouldn't work with $20. It's yeah I've not hit my limits so far but definitely it's a try. What was the other? >> The other two questions were hey since these are mostly open-ended like in the context of net new knowledge how do you ensure the research is sound especially if you're not an expert on the field this is asked by Vispa.
14:07 Yeah, I mean, of course. I think if you're I think that's the million-dollar question, right? I mean, if you have no idea about how materials work and you just ask an LM to do discover a new material, then I mean you you can't hope to really sort of calibrate. I think my experience has been that the hallucinations have gone down significantly. So you shouldn't expect the lln to be just completely wrong in what they are saying but whether the result that so in terms of soundness I don't expect the soundness to be low but in terms of significance importance framing I think that's where you need like a lot of guidance and we'll come to that later but I don't think there should be like much issue in just technical correctness of what LLM are able to do now >> question can I ask >> yeah go for it aish can it I think this is a debate I have come across can it make connections between make connections which humans can't with existing knowledge or can it genuinely make leaps of faith which you know the most creative human researchers often do when they come up with brilliant papers. Yeah, >> I think the short answer is no that the LLMs they can't take this con they can't do these conceptual reframes but neither can most human researchers in my opinion but let's take that later I think I have a few points there about general >> we're good otherwise like we can take the questions in the chat later if you want continue >> okay so let's deconstruct the other file which is the main you can think of it as the claw MD which just says tells so think of two files research philosophy is really that specifying what is good research like and va.md or you can rename it to cloud.md it's just what to do so as I said you know you you just start cloud code you just do like a git clone of the autoiler repo open up cloud code in that folder and just say that readily md and get started literally that and the human input in auto research systems like this one is really what are curious about. I think you as I said you have to start from somewhere. It can be as broad as AI for science or it can be highly specific.
16:35 this is something that I literally did yesterday and it's so amazing that this is a curiosity I had for more than a year in my list of potential research projects to do when I get time. But when I was making this presentation, I said what the hell let's just try this. And the whole question I was interested was that instead of doing this in the recommendation systems, instead of doing you know next neighbor prediction that what is the thing you want to watch next, what if you do distant cluster prediction that eventually if you like action movies for example u some movie which is not action related can the initial sequence of what you liked tell you eventually what will you like? So that in other ways you know can you sort of really operationalize serendipity that from a first few signals can you say that this is something that you will eventually like and hence start surfacing that much much earlier. So this is the literal the prompt I wrote and we'll walk through what it found out. so in voter.mmd you know the first three lines are this it says that you are an autonomous aiml research agent that follows the following steps. Read the research process you need to follow in research philosophy of MD. So this instructions causes claude to read this file and sort of ground itself on the research philosophy and then it says that ask user for a particular topic or a paper or a set of papers as a starting point.
18:06 So this is what you see if you just don't start with anything. It would ask you you know that you want to start with a specific topic, specific papers or whatever else. And then the instructions say that ask the user how much time you can afford to spend so that the entire process is finished in time. The default is 2 hours. So it can ask and you also saw on the live demo it asked me and I said you know 30 minutes and then this the instruction says figure out system resources available to you so that your experiment can be run in the system. So you see that you know it ultimately claude code queries the system and figures out that you know I have an Apple M3 max python installed codeex cli open router. So it now knows what resources does it have to to to sort of do experiments. In theory, if you have a GPU, it'll find out the GPU.
19:01 If you're running on a server, it can find out that. So it it'll just ground the research on what you can actually do instead of just theoretical whatever research is possible, right? And and then it says that based on gathered or inferred inputs, think hard about what research questions plus claims make sense and rate them on the criteria listed in research philosophy. So this is where now based on your inputs as we said you know AI for science for example it will start constraining the space of all possible research directions from the constraints we've given and the constraints really is that it should be possible you should be able to do it in a few hours you should be able to do it under $10 of API credit it should be possible on your system and then it'll come up with ideas and start rating them on this fruitfulness surprisingness feasib ability and rigor and you know other kinds of criteria that you may have in research philosophy and finally one thing I've found really really helpful is to actually ask it to take feedback from codex so I've always found because these systems are trained on you know different data sets claude is trained on very different kind of training distribution and codex is trained on different distribution hence they have different kinds of strengths and weaknesses so you can ask cloud code to take feedback from codex and it always helps. before we move further, let's see if our live session has any inputs for us. So I think it has. So so it's sort of done some analysis if as you can see you know it wrote certain ideas based on LLMs and scientific reasoners. It took code back codeex feedback and based on that feedback codeex sort of criticizes so this criticism becomes very important.
21:02 So it says that the research idea A is cooped which is already done D is directly collided E is in a crowded line blah blah blah blah blah. So based on that it just says that here are my three final finalist all fit within reading it I don't know what it'll do but this is the part you need to of course read most I'm just going with clause cross discipline isomorphism which is okay I think take publish >> can you maximize the terminal window.
21:42 Sorry, just difficult to read actually. >> Yeah, I know. I mean, I'm I'm not able to read also. I have time pressure. So, I'm probably just going to go with one and we'll see what happens. because of time constraints. Yeah. So, >> otherwise as well it will be easy to read. >> Yeah. Yeah. So, but I'll go back to presentation next. In the next one, I'll probably try to maximize or zoom in the text size. But as you can see you know taking feedback from codeex becomes really really important. So you should have both codex and cloud code installed and claude can refer to codeex and vice versa.
22:22 what you saw right now and you see in the screenshot also and maybe screenshot is clearer says present top three exploration ideas to the user to select from and give sufficient explanation background knowledge for the claim so that user can pick one in informed manner. So remember my initial starting point for this presentation which was does predicting a distant item work better for serendipity versus predicting the next item. It gave me these three sort of suggestions based on what would be a good research exploration and I picked the first one. If you notice you know it is doing good signs. So for example, it is doing, you know, clear four-way ablation. So it's just not testing what I said needs to be tested, but it's actually trying to make a scientific claim that cluster jump is doing something important and it automatically figured out what are the right sort of metrics to measure, what is the ablation.
23:27 so the framing is where you end up sort of starting out here to select the whole execution loop. So this is where human feedback is needed where it's important that initial framings might not be the most correct ones. Right? So what I did here was I didn't understand I didn't have a lot of background in recommendation systems. So I asked it can you explain the first and second option? I'm not from Rexes and don't understand the I I didn't know what hit rate was. So u it explained me a lot of things. So it's just not like the whole point of auto research is not just to go there blindly. It's to actually build an understanding of the particular question you're interested in so that you're able to guide the system really well. So, and I really love that it suggested doing this four-way comparison that forecloses that alternative explanation because good science is about making claims which are very specific in its mechanism. So, if something else can explain why you found like a particular result, then you need to be very clear that your proposed mechanism is doing that and not something else, right?
24:40 I I sort of did more sort of feedback did like a one or two iterations of chatting with slot code and then after that you know it sort of refined again and this is how the rest of the boiler is. the spikes that you do get organized in this folder called all spikes. So you can see I've done different kind of experiments and all of them get their own folder within all spikes directory and within those directory it keeps a log of all the progress and keeps on sort of logging results insights etc and I also mentioned in the boiler.md that at each critical step it takes codeex feedback and even before writing the first draft it acts as an reviewer at an AI conference.
25:42 So it sort of automatically figures out what are the possible issues before writing the first draft and I think the of course in in like a spike you can't expect main conference level kind of an output but I think within let's say a few hours you can expect a workshop level output and and that's a separate question it worries me the amount of inputs that are going into workshops is just humongous because this kind of a system can do empirical research within a few hours which are which is at a workshop level. but as you can see this is a screenshot where it says code the first draft wasn't good but CEX reviewed it so it sort of made you know three edits. LLMs and especially Claude has tendency to hyperbol stuff. So it would write things like cleanly falsifies and if you've ever submitted a paper at a conference or a workshop, you know that this is a very high bar to meet cleanly falsifies. So CEX really helps with calibrating the language to what the actual results are.
26:53 And finally you know it u in autoiler there is this draft format folder where you can drop in the style files for whatever conference you're interested in. It can be new rips or others. Here we have you know cisc this conference that we are organizing with bitsani their template and I had just dropped that as like a default. So you can see that it started writing the paper here. Let me take a pause and see if our live demo is doing anything. Yeah, it's still running something. U and you can see that you know if something is slow it it will just kill it and try it again. We'll get back. It's still going really well. So, and what the what what what VO.mmd also instructs clot code to do is to log exact prompts and summary of session flow. I think I've realized that it's very important to log in the paper.
27:58 If I think it's also philosophically important that when you're using AI systems to do research, there needs to be a clean log of what was the human input and guidance and what is that you know AI did and this I think appendix in the paper that gets generated it has that log very very clearly so anyone in future can know that what was the human input and what was the AI effort and lastly u AI has a lot of writing ticks. so I ask it not to have the specific AI writing style but it doesn't mean that we want to hide that it's AI written. clearly in the acknowledgements it will be written that the paper was assisted by Claude and I mentioned that the final outcome expected is a folder. So you will see that within all spikes there'll be like a folder for the re research idea that you're working on and within that there'll be a paper folder and you'll find the paper being generated as a PDF.
29:02 And lastly in the compile PDF generally citations etc are an issue. So I ask it to make sure that you know compiled PDF is correct and sometimes it will then literally sort of take screenshots of the PDF and then ensure that the PDF is looking really good. So this is what what ended up being produced. this is a GIF here of the paper and as you can see that my input was probably two three interactions.
29:35 and it is a workshop level paper. as I said it it it's exciting and sad at the same time. We can talk about the implications of it on society later. but the fact that the something that was in my curiosity list for many months and I was able to answer it to a certain extent within few hours is just amazing. But after after the paper was generated I did read the paper and had a lot of feedback. So a lot of questions lot of feedback. I l I created a markdown file in the folder human feedback and then asked it to read it. So I said that I read the paper and gave my feedback in human feedback/v1.mmd please read it and get going. then it sort of went on to iterations. So after three iteration I think total time it took was 6 hours. I got to a point where I was very very satisfied by the paper and the discovery was actually a negative result which is again surprising.
30:47 yeah I'll get to what were the results. Let me just see if Claude is doing something. Yeah I think it's where is it? I'll take a pause and see if there's still any questions. >> Yeah, we have a bunch. right. but I think there are two three buckets of it. right. One is around sort of token use and time bound estimation like in the repo you sort of mentioned or like you're interacting with the LLM to like telling it estimate of how long it will take. You've also told how many tokens to use and all of that jazz.
31:23 from the experience of people in the crowd models are not very good at predicting the time something will take or they're not good at like keeping their token use in check. So how is your experience been with that? >> Yeah, I mean for that I mean for the token use obviously the easy thing is to have u disposable tokens. So if you can just spend $5 10 $20 just create a new token of open router open AI and use that instead of like an unlimited thing.
31:52 So I don't worry about that too much. And in terms of time estimate my my sense has been that the time estimate I give is always an upper cap. So if I say 2 hours it ends up completing in less than 1 hour. I mean these things are generally very aggressive to finish. So I've never seen that they've taken more time than was asked for. >> Thank you. Thanks first. That should be helpful. the second set of questions are like can this be used in your experience from for nonCS research? Can it be used for market research for example or psychology? from the chat and I think Lakshman is also asking hey if I can use it to do literature review like iteratively and generate the most comprehensive survey or review paper.
32:42 >> Yeah I I think to it's totally doable. what you should do is you should modify the research philosophy. I think different sub field subdomains may have different kind of rubric on what counts as a good output. So if it's like a market research, I think what is a good market research is something you'll have to specify there. if it's like a review paper, I think you'll have to specify what counts as a good review paper. And then I think everything else should be automatically following from that.
33:13 >> Yeah, great. Thanks Paris. And lastly, like again in your experience, how have you tested this out with let's say local models or is it also something that you can use with codecs or is it limited to clot code? I have not tested it with local models or codecs but there's I mean it should work it's just a few prompt files nothing fancy so if you change them I think it depends on the model capability but now even open source models like deepse v4 or quen or kimi I think they've come a long way so if you pair them up with open code to some extent they should work and they should definitely work with codeex Yeah, I think that's more or less the questions. We have a bunch more but like nothing super critical we can get to later. Yeah.
34:02 >> Okay. So, so coming to I mean I'll just spend maybe three slides talking about the surprising results from this because the whole point is not to produce a paper but to find results. I think what what was very surprising to me was that my assumption that you know that people sort of consume content which is similar in in time. So if I consume if I watch like a funny movie now the next movie will be funny and I'll drift very very slowly. But at least on the Amazon lens data set that it used you can see that the offset position. So if the clusters the subsequent items do not share clusters with each other. So those cluster sharing likelihood is only 6%.
34:55 So if I watch like an action movie now it's only 6% chance that I will watch an action movie next for example. And of course this depends on how many clusters there are but there was an ablation of number of clusters also but this trend is the same people u people sort of within the time as as next item consumption it's not super correlated with previous item correlation and what this ends up meaning is that if if this is the metric that was I was interested in that I I won't go into too many details I The paper has a lot more details but the whole point was that even baseline if you don't have any auxiliary loss of let me try to also predict what was consumed in the future either it's from the same cluster or different cluster or just any item in the future at all the metrics don't change much and that was you know highly surprising to me and also interesting was that this whole system ends ended up sort of testing if you notice it ended up testing these nine variants and also ended up sort of adding these standard deviations which in my experience you know even let's say undergrads may not know that this is an important part of research to show confidence intervals and it tested multiple kind of loss functions.
36:30 So the the the meta point is that this was a very careful technically correct paper even though the result was negative but the the rigor of it the science of it was just very very well done. Yeah. So I think it's still churning along but you can see that it started to write paper already. I think it must have finished the experiment. So we'll see what it ended up doing by the end of the presentation. But I think it's writing the pres. It's still 16 minutes in. I'll show a few more examples of the kinds of experiments and spikes I did. One is I used the system to discover like a perfect coffee cup. So the whole prompt was that I mean I love coffee but I love hot coffee. It but it ends up cooling much faster than I want. So the whole my curiosity was that could you design a perfect coffee cup so where the drinkability is high but the heat loss is minimum. So I had I had read about differentiable physics somewhere. So my whole prompt was can you use differentiable physics for discovering a coffee cup where drinkability is high and the u heat loss is minimal. So it took some time but ended up you know building this and ended up writing a like a paper on it also. So you can use the system for this thing. The second thing was I was again this is something I had been very curious about for a long time now many many months was that lot of pre-training in deep learning happens across a very big data set and I was curious what if you do pre-training on only just one file so which is literally one document so I I asked the system to do pre-training on just one document like which was Wikipedia article on India and and it ended up sort of trying to first what it did was it created two versions one it used pre-trained GP2 checkpoint and the other one it trained from the scratch and it discovered that when you talk about just pre-training on single document there is like a U kind of phenomena within like a few optimizer steps like say in 50 to 100 steps you see this perplexity drop but then it ends up increasing on held out set which was not on the pre-training data. again something which would have taken me a long long time and it's just super interesting that this was possible within one hour and the last one again when you're talking about you know the kind of experiments you can do another thing I was curious about is how does the trajectory of experience change under psychedelics so there are you know online repositories where people report their qualitative experience under psychedelics like LSD So I asked it to do some research on literally this and it ended up sort of making plots on how does the described experience change with time. So you can see that this ego dissolution increases the veency of experience is high then it goes down and then it comes up again.
39:57 visual geometry it's very high initially then it reduces and so on and so forth again very different kind of application and you can imagine you know using this for other things also and as I said autopilot is just prompts you can modify anything modify research philosophy modify the instructions modify the template modify the underlying model and agent you can you can and should just change anything in the repo And some tips on workflow.
40:30 One is that I highly recommend using a wrapper for your agent because otherwise for each action each command it'll ask you whether you want to do it or not. And of course if you put it if you want to put it on the background like I have put it you need to make sure there is like a either you run it in a docker or you run it in a wrapper like CCO. Rapper just means that it ensures that u none of the destructive actions are taken. So it can still keep on doing but your agent cannot overwrite any other files in the system. make sure your computer doesn't go to sleep. This is something which is trivial but happens so often that your computer goes to sleep and your agent simply stops running. You can use some I use a on Mac where you simply click on it and your computer doesn't go to sleep and set your API keys in environment. So whatever keys are in the environment cloud code can access. I set my open router keys there. So it becomes just super simple. before we get to failure modes I'll see if yeah so I think it worked.
41:43 yeah. So it so you you can see I mean this is the answer it took like 20 minutes well under the budget and cost was literally 14 cents. and u it says pre-registered prediction was inverted and it just says I think we'll have to read the paper. I have no idea what it ends up being doing. But the the whole point was you you can see that it did the whole workflow and our input was just a couple of lines and it took code feedback and let's just see what it ended up doing.
42:29 just a second I'll see what it we put it here. So you can see as I said you know spikes spikes as all spikes and u I think this is ISS iso versus lexical is what it ended up creating. So this is the paper it ended up creating when stripping domain wording helps cross discipline mathematical isomorphism recognition in frontier LLMs u I think we'll have to read this but you can see that it has just a nice seven pager that it ended up being producing. Of course, if if we had a more specific u starting point, it would give better answer. I don't even know whether this better or worse. Honestly, you'll have to read it. but some of the failure modes that I observed is you know it can skip skip details. So if you I strongly strongly recommend you to read the paper it generates and ask it to add more plots and details. for some reason I've seen that these systems do not I mean the constraint is that the paper should be eight pages but it ends up producing six pages, five pages and write short paragraphs. so you should read it and just ask it to add a lot more plots and details and of course all the figures and logs are there in the folder. So you should read the raw things that have produced and not just the paper. And the second which is surprising and I think this does deserve a little bit more discussion broadly that a lot of these autoscientist systems produce negative results more than the positive results. And I think primophasia I mean as just Beijians we should expect it. positive results are harder to come back but you would find that a lot of times the result that they give is negative but I think it's partly because positive results are harder to find but there should perhaps be some way to log negative results also I mean the workshops etc don't accept those negative results that easily but these systems end up producing that you should read carefully because it could be because of missed assumption or detail maybe it ended up making an assumption that doesn't hold true. So if you read very carefully, you can nudge it and that can perhaps change it. Sometimes it can forget earlier instructions. So if you have a very complicated starting point with tens of different things going on, it can forget it. So again, read the paper to ensure it's faithful to your instructions. And framing can be too conservative or bombastic. Either it'll say that nothing of interest is there or it'll say that it'll change the world. I think as a human your role is to calibrate and position findings which is proportionate to the evidence you have. So neither to conservative nor to bombastic but somewhere in between and of course it can't do big conceptual reframes like Einstein would have done or Darin would have done. It would operate within the frames that are already there in a domain. But again I think even most humans can't do that. So I don't think it's a very fair expectation from a research system. So I think we are near the end of it.
45:58 strong recommendation get your first paper published. We are doing this conference for AI scientists. you can take a screenshot of this slide and go to the website submit if you even if you've never written a paper before but you have understanding of your domain I think you can create net new knowledge for humanity by using a system like this within your domain so please go and use the system and submit a paper or conference for AI scientists that we're doing in July I think we still have maybe a week or so left before the deadline for papers is over. So go ahead and do that and I think that's about it. I'm happy to take questions in the remaining time.
46:46 >> Great. Thanks Paris. I mean I have three that we have written down and then perhaps we can just ask folks to unmute if you have the time but I think the first one is like how much verification do you do on your output? Like do you read every line? Do you follow all the math? right. >> Yeah. So I don't read the code but I do follow the math and I think it's sort of like you have to have that smell check where things are making sense. For example, if the result is negative, you can't just assume that hey maybe it is the way it is. I mean all research starts from having like a gut feeling, a hypothesis or a sense of how things should be. So if if things are deviating from that either in two positive direction or too negative direction then I think you need to dig deeper and make sure that the whole thing so as I said you know this is technically I don't think these LLMs and these systems are making errors anymore it's just that sometimes their assumptions are wrong or their framings is not aligning to how a human would have done that and that's the role you should be playing.
48:03 >> That's great. Thanks Par. one more I think more principal philosophy question right like folks in the chat are hey does this contribute to research slop and how do you recommend we get around it if you are supporting this way >> I think we need to define what research slop is I think there there's a genuine concern that human reviewer time can be wasted by just overloading them with tons of autogenerated papers that add no value at all. I think we need to differentiate that.
48:40 I mean is using an AI system an issue or is submitting a lowquality unverified paper an issue? I think these two can be orthogonal. you can use an AI system to accelerate your research while still maintaining maintaining an ethical stance that you will not submit something that's not worth another human's time. And I think it's the same thing on social media also, right? The whole social media is full of slop. You can I think use AI to do amazing things and you can use AI to waste people's time. It's it's really a tool in that sense.
49:19 Yeah, that's great. Thanks for having I think someone can extend the question if you don't mind. sorry I can switch on my video unfortunately. so Paris wonderful session very interesting topic my question is just around that exact point paras meaning see I my overall feeling if you ask me even new rips and all I would say that even this in sizable percentage of papers probably where the code or math is not fully correct and even today or even in last 20 years or so we've been making the mistake of accepting some papers it's only when reimplement someone reimplements and figures out actually doesn't exactly work right and I'm wondering whether there's something we can do about it, right? That's something maybe better auto verification systems.
50:02 >> Yeah, >> I I think this is a great opportunity for actually labs or groups to specialize in auto reproducing papers. >> So these systems can take like a paper and try to reproduce it and automatically contact authors if they don't reproduce. I think this should be used for good. I 100% agree. >> Yeah. Yeah. 100%. I mean last question for the session it's a generic one but how would you rate like your experience of doing research in this way etc.
50:36 >> I mean honestly I've been I I mean for me it's a little bit of existential question for how so for me the question is how would new researchers get trained? So a lot of times let's say from research internships point of view undergrads and even masters or even maybe first year PhDs professors and domain experts would give them tractable problems that that are not so easy but not so hard that they can't do it and that was a way for new researchers to get trained but now I think there is this increasing temptation for domain experts to use systems like these to automate their explorations. sprints which earlier they would give it to an intern. so really for me the question is how in what way new research new human researchers would get trained because honestly what this system is doing is at the level of what an intern would do. So the question really is that why wouldn't someone just use this system? I think we have systems that are already at the workshop level already and I think in one or two years we'll I'm reasonably confident we'll get to a place where these systems will produce research equivalent to main conferences.
51:53 I think I've been thinking how does academic research adjust to this is a question for society. but it's it's like a super important question. how do you justify how should human researchers shift given that a lot of this research will get very easy is is the question that I think I would love for everyone to think about. That's great. Thanks Paris. for folks who are raising their hands like we can probably stick around and take some questions after. Paris might have to bounce but I'd love to wrap up the recording at this point. thanks for joining and this was a great session for those of you not familiar where this is the third session as a part of the conference for AI scientists I'm dropping the links to the repo and the conference in the chat please do check it out thank you again I'm happy to hang around for 10 15 minutes we can stop the recording and just do like a casual discussion and then I'll