transcribe

India's Top AI Researcher: Can Indian IT Services Survive AI?

SparX by Mukesh Bansal · 55m · transcribed Sep 2026
More from SparX by Mukesh Bansal Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

AI and Consciousness

How does AI relate to concepts of consciousness?

The discussion highlights various philosophical perspectives on consciousness and suggests that AI can scale infinitely, potentially leading to capabilities beyond current human understanding.

  • Different cultures have unique theories of consciousness.
  • AI's potential for infinite scaling could lead to unforeseen applications.
  • The conversation raises questions about the limitations of current AI capabilities.
# 11:05

India's Position in the AI Arms Race

What is India's role in the global AI landscape?

India is currently lagging in the AI arms race, with concerns about sovereignty, security, and the economy due to a lack of credible sovereign AI companies.

  • India's AI development is significantly behind other nations.
  • The lack of investment in AI could threaten national security.
  • The dependency on the IT service industry poses risks for job security.
# 22:10

Rapid Advancements in AI

How quickly is AI evolving and what are the implications?

AI is evolving at an unprecedented pace, with new models being released frequently, leading to challenges in understanding and managing this rapid change.

  • AI's self-improvement capabilities could lead to exponential growth.
  • The frequency of new model releases indicates a fast-paced evolution.
  • Understanding AI's rapid advancements is becoming increasingly difficult.
# 33:15

Data Creation and AI Training

How is data being generated for AI models?

Current AI models are increasingly generating synthetic data for training future models, creating a cycle of continuous improvement.

  • Synthetic data creation is becoming a key component of AI training.
  • Companies are investing in creating environments for AI to learn from.
  • The cycle of data generation and model training is crucial for AI advancement.
# 44:21

Teaching AI Local Languages

How can AI learn local languages with minimal data?

An experiment demonstrated that AI can learn a local language like Tulu through carefully constructed prompts without extensive data or fine-tuning.

  • AI models can be flexible learners with minimal input.
  • Understanding language nuances can facilitate AI learning.
  • The experiment challenges assumptions about data requirements for language training.

Transcript

0:00 One fine day, everyone wakes up and they announce that they were hacked. Hugging face detected the incident before OpenAI realized that it was their agents that were doing this thing. Paris built Wingifi which was completely bootstrapped all the way to $50 million revenue and eventually $200 million exit and then he is doing something even more unconventional. Started a AI research lab. Every time someone tells me that LLM can't do something, I said no you can't do this with LLM. How do you define consciousness?

0:27 >> Hinduism philosophy has figured out consciousness. Then Chinese philosophy has something. Everyone has their pet theory of consciousness. Beautiful thing with AI and agents is that you can keep on scaling them like infinitely. You can imagine that the compute and token consumption is unlimited which means that they will do all sorts of things that we're not even able to imagine right now. >> Do we figure anywhere in the whole AI arms race? Does India count? >> No.

0:52 >> Indians count. Of course not India. I mean in a country of 1.5 billion people all you need to find is 20 people who are interested in fusion who are not able to find like a good home for the curiosity. If you assemble all of them in one room I think less than 10 CR per year you'll be able to shift the trajectory of fusion in a very significant manner. Mythos++ kind of a model. US will also have they'll cut access. They'll not give access to anyone else. They already prove this.

1:18 All the other nations including Indian nation would be like sitting ducks. What does it mean for our economy, for security, for jobs, the entire IT service industry that we are so dependent on. hi Paris, welcome to Spark. At Sparks, we try to have conversation about everything deep tech. Paris is very well known but just to recap Paris built wingify which was completely bootstrapped all the way to $50 million revenue and eventually $200 million exit and then he's doing something even more unconventional started a AI research lab very curious to understand paras you know what is the thought process behind AI research lab you know what led you to doing something which is I think there is no parallel you know I don't even want want to say unconventional this is first of its own kind initiative. So what led you to build what's you know your your AI research like?

2:16 >> Yeah, thanks MKkesh for having me. big fan of what you're doing. yeah, so effectively I was dabbling with AI even before starting Wingify. >> my love and interest in AI goes way back when I was a teenager. So I remember having a very I mean those times you had very slow computers but having Visual Basic 6.0. >> Yeah. I think this is 1998 999 and I was >> you were interested in AI at the time.

2:41 >> Yeah. I was writing back propagation algorithms teaching computer how to play tic-tac-toe writing genetic algorithms cooking up theories on how intelligence could be. >> how where did you get exposure? How did you even get to know something called AI back propagation exists? >> I was hanging out quite a bit on hacker news. >> Uhhuh. So hackernews is this popular forum run by Paul Graham originally created by Paul Graham and it's like one of the meccas for technologists for people who are very interested into these kinds of things. So I just got exposed with those things and somehow it just got my fancy >> and then Wingify happened and obviously startups just take all of your time but even with Wingify I was here and now dabbling into different things but after selling Wingifi I thought I have time I have money. Yeah, I think one thing I want to share you know which is my one of the biggest regrets in life is I studied computer science at IT camp late '90s never even heard of AI had no interest in AI like I did all the course you know whatever was the standard curriculum and I got interested in AI probably in the early you know with the the imaginet thing in you know in 2011 12 that's when the first time start paying attention but good for you you're paying attention >> I was doing so much of this in school computer science, AI etc. that when it came to taking my specialization in engineering, I took you know biotechnology.

4:08 >> Yeah. >> Because I had no idea about >> you already knew computer science very well. >> I yeah I told my mom that I know everything about computer science or what am I going to learn? Obviously it was naive but that is the kind of things I was interested in my school >> right >> and yeah log is this manifestation of that childhood dream to go very very deep into >> so you've been dreaming of doing a research for a long period of time >> yeah I've been dreaming of a doing science I was set on a path to do PhD >> during my engineering I was doing computational biology right >> so simulating biological circuits on computers so >> I had dreams to go to MIT and do a PhD there but startup happened and then I had dreams to get in very deep into AI.

4:51 >> Yeah. >> So Los Funk is sort of like a culmination of these two dreams. Let's do science and let's do science about AI. >> Yeah. What is AI research lab? What happens? so in in our research lab it's we've gone through a journey u and it's interesting I've met professors who've said you know I've gone from a journey of being a researcher to an entrepreneur. A lot of professors start their companies. That journey has been a reverse for me.

5:17 >> I've gone from being an entrepreneur to being a researcher >> and >> the first part was to actually learn and teach myself what even is research what even is science. This is a term that people >> end up using quite a bit and slowly I realized research is about three things. Research is about finding something that's completely novel. So novelty is a big part that you find a piece of knowledge that nobody else knows >> where you're operating at the edge of what anybody knows out there. But it's just not about novelty. A lot of things are novel but are just very trivial. It has to be important also. important means it has to have some downstream consequences >> and these two also are not enough. The third ingredient which is often missed by people who are not researchers is the rigor.

6:05 >> The rigor means that if you're claiming something about a piece of knowledge >> you have to claim back it up with so much of evidence and rigor that it becomes undeniable. Mhm. >> So if these three things come together for a piece of knowledge that you end up producing typically as a paper but not always as a paper >> you can say you know that it's a piece of research. So that's what we do at Los. So >> at Losang we have these big questions that we're very interested in.

6:34 >> They go anywhere from mind to intelligence to consciousness to future of science to AI's ability to do math. >> Yeah. And then we try to come up with these questions where if we find good answers we'll add to humanity's knowledge effectively. >> Sounds outstanding and I have many follow-up questions but let me understand the structure first. U so one you are operating as a researcher. I think I was reading also about it. You spend at least half of your day or more just being a individual contributor researcher >> but then you have a team also. How does the setup work? You know, how does one join Losf funk? Are these folks employees or doing a long-term grant?

7:16 Like how does one become a researcher there? >> Yeah. So, Los Funk is what you can call as an independent research lab. Independent means that that the people who work at Los Funk, they are having their own individual projects. >> Mhm. So it's not like in in a lot of labs in academia when you approach a professor the professor ends up giving you a project they have like a backlog and they have a student pick up one of the projects and go with it. But los fun it's a reverse everyone who approaches loss fun to work with loss fun >> they have to propose their project okay >> because they are the ultimate independent owners of it and everyone else including me is like a collaborator there. Sure.

7:58 >> So in that sense at LOSF we have about I think 20 25 full full-time paid folks at various different experience levels right from students to people who've done their PhDs. >> Yeah. people who've been industry quit their jobs and each one of them is having their own individual sort of research project and interest >> that they pursue at loss form >> and so how does one apply or what is the arrangement that let's say if I have a interesting idea and a background I'm assuming it's in the AI space and I reach out I send you my proposal after that this is a kind of fixed duration type of engagement >> yeah so typically It's we give fellowships.

8:43 >> fellowships range from 6 months to a year. >> Okay. >> and then internships also they last like a similar point of time. >> and if there is a great fit we extend it >> and very occasionally there is like a core team also which stays for like much much longer. But los fun really because we're not working on a product, we're not working to like launch something. >> It becomes like like a temporary launching pad for people's career, >> right? So in fact we also very soon launching something that we're calling a sbatical >> which is like if you've been doing a professional job for a long period of time maybe you are a professor maybe you're a lawyer maybe you're someone else >> we will pay you to quit your job for 3 to 6 months and work on something that >> wouldn't make sense anywhere else. So law is very interested in supporting projects >> that don't make sense in traditional structures like academia industry or even VCs.

9:43 >> So what others won't touch >> Yeah. is something you know I and Los get very excited about. >> I think you may on the verge of starting a movement know I can see so much you know merit in what you are doing. I think as a country somewhere you know I've tried to study I feel it's weird that we I think we had a much better research culture 100 years ago in some ways >> you know some all the Nobel prize winners of the country are from pre-independence era at least in science some in social science later or at least people of Indian origin but somewhere we kind of you know lost their way now a lot of effort is being done in some ways but that is more in some very industrial research type of mindset that let's build deep companies.

10:26 >> Yeah. >> As opposed to fundamental breakthroughs that move the human humanity's knowledge forward and that's where in some way philanthropic money >> can go a long way and I think you know >> even wellqualified researchers as you said they don't cost that much. >> Yeah. >> So I mean if there are even few dozen such labs can come up where the money is we are talking about is almost nothing. >> Yeah. >> So I I really hope you know you become a shining example that a lot of people take inspiration from. Yeah, >> let's zoom out and talk about the state of AI research in India. How do you stack rank? You know, where are we given this an area you're personally trying to do something about and you must have thought about this area. Do we figure anywhere in the whole you know there's kind of AI arms race that the whole world is engaged in in some ways >> and a lot of breakthroughs are being made.

11:17 >> Does India count? >> No. >> Indians count of course >> not India. >> Right. We are very far behind >> unfortunately. >> Is that is that a big cause of concern? >> It is. It is from a sovereignity point of view. >> Uhhuh. >> but I think the odds are stacked against a lot of sovereign AI companies. >> Yeah. >> Purely from an economics point of view, >> there only one credible company now. There's not really and beyond serum.

11:45 There is no other. >> Yeah. >> You know, is there anything else which is you know credible sovereign AI effort? >> Not at the level that we need to be. Right. What what does it mean for for our economy, for security, for jobs, like the entire IT service industry that we are so dependent on? >> I think mostly it's it's a threat to sovereignty from a security point of view. >> I mean we have fable today, tomorrow, 2 years down the line, it'll be 10x more capable.

12:15 >> Yeah. >> What it means for just the security of our power plants, our nuclear reactors. >> Mhm. our jet planes, our navy ships, all of these are today run on interconnected systems >> and so cyber offensive cyber security by >> other nations is like a big threat right I mean because that's how you lose the war before even the war war begins >> imagine if China has like mythos plus+ kind of a model >> they cut off access >> they will have >> they will have and US will also have they'll cut access they'll not give access to anyone else. They already proved this.

12:54 >> Yeah. >> All the other nations including Indian nation would be like sitting ducks, right? >> Yeah. >> You cannot defend against >> like these 10x more capable models that are just highly persistent millions parallel agents in all your systems trying to find vulnerabilities. >> So cyber security more than anything else from a sovereignity point of view is like a big concern. >> Right. I think last year you had I think I don't article or something you had some suggestions some recommendation for what the government could be doing.

13:23 Yeah. Have you seen any movement in any direction since you recommended any no >> post? No. Okay. >> But this look this defense and sovereignty is one thing. What about just pure economic competitiveness? Is that also under threat or this is where you know we are obviously a large country with huge number of consumers. We can produce and consume and GDP can still grow 7%. >> Yeah. or or that also comes under threat in some >> I think there is a significant inertia in rest of the world but that inertia only last for so long.

13:56 >> So I mean right now >> I think the IT companies have reduced or stopped early stage hiring. >> Yeah >> these are still growing in terms of revenue because they're getting a lot of projects from elsewhere. But the >> you have to look at not the revenue which is a lagging indicator but leading indicator which is like new hirings that they're doing and so on and so forth >> and given the trajectory of AI I mean I I I just don't see what roles like would these companies play then >> because even like Enthropics open AIs of the world they are setting up their own consulting arms >> and tomorrow the AI themselves will be consultants right why why even like have that outsourcing to happen somewhere else.

14:43 >> We are not seeing it immediately because there is a lag in the rest of the world but I think we will start seeing it and given that IT services is like a significant portion of India's export. >> Yeah. >> We just don't have anything else ready to replace it. >> Yeah. >> So it is and that's why you see Indian stocks are like one of the least favored right now and the market is so low. This is market price is the future optimism.

15:07 Right. >> Yeah. And right now from that point of view there's not a lot to look for in terms of what can India export to rest of the world which is what's required because it's so dependent on imports. It's almost seems like you know that slowly boiling the frog kind of situation. You are being boiled little bit every day but you don't really feel it. If you look at a zoom out there is there is I mean there are some you know very credible government efforts being done you know RDA etc. But I don't think in RDA will be that relevant in this whole what's happening in the AI world and that's where we are seems like starting to look more and more like sitting ducks.

15:43 >> Yeah, that's the unfortunate reality. Help paint some kind of optimistic scenario. >> Big breakthrough at Lost Funk in the coming years. I don't think I mean I don't think it it will be like a breakthrough more like I mean I'm hoping that we'll be >> pushed to a desperate corner and as a country I think we are very resilient. >> Yeah. >> So in terms of change I think >> I mean what what's great about India is I feel government has its heart at the right place.

16:13 >> Yeah. >> everything that they want to do I think they want to do from a country's benefit point of view. It's just that the structures that we have they right now end up promoting less risky kind of initiatives. >> But I do think if India is pushed to a corner which it could then they will be like a renewed fervor in terms of very focused investments into new kinds of technologies >> doing things like lifting the salary caps of professors inviting Indian diaspora to do fundamental >> R&D here setting up new things. So I'm hopeful from that point of view. Yeah, I think in general for as far as you know research ecosystem is concerned see I don't know paris like few years ago who was talking about research today we have at least lot of mainstream conversations about this >> you know this RDA fund is obviously incredible you know program with you know one lakh cr of funding which I think is it will get distributed it's just a matter of time this idea such a focused research organization I think some of them are again getting government funding even and I think earlier before I should we were talking about private R&D R&D funding it has at least inched up from I mean overall country's R&D budget has gone from 6 to 085% I think now okay >> and the private share of that has gone from 40% to I think 60% something like is sizable movement last four five years >> so in some way I think it is at least being acknowledged that you know if we don't get our research ecosystem right we'll generally be like left out in the current >> yeah I think we should be working on the most fundamentals physics chemistry Yeah, >> biology, space technology and like absolute basics that are required >> only then we'll be ready in 10 years for the next big big breakthrough. I think you're also in some ways you know giving a very tangible what can I do answer because a lot of people are lamenting the fact that you know where is India in the whole deep tech ecosystem where are India's Nobel prize winners where is our research ecosystem but there's a lot that individuals can do with not a lot of money I think a team of you know 10 15 20 people is basically a size of a research lab in >> anywhere in the world most research labs are not hundreds or thousands of people it's only like 10 And there are >> as far as affordability goes, you know, hundreds of people can afford to start this. And that's one way to really do something about it as opposed to just keep complaining on all this.

18:38 >> And now I think it's a perfect time to do it because with AI you can have so much of leverage. >> I mean a you can teach yourself >> any subject >> to a certain level of understanding in a very personalized fashion, >> right? >> You're interested in nuclear fusion. Great. Just keep asking Chargity and Claude to tell you more and more and more and more and more and more. >> You don't understand a paper, ask Charg to explain it to you. Obviously, it'll take time, but it is like 10x faster than what it would have taken because now you have an expert on your computer on your phone who's able to sort of break down things.

19:12 >> So, a from an understanding point of view and second from a research leverage point of view also. >> Yeah. >> All fields like Los is a very AI firstdriven research lab. Everything we do we do it like from a leverage point of view >> right >> and this is why I think right now even a third year fourth year undergrad is able to punch much above his or her weight. >> So Paris is AI fundamentally changing how research will get done like are we at a phase transition moment? what about research which will not change it is timeless and what about it will ne never be the same again given the profound impact that AI is having you know is likely to have on how research is conducted.

19:52 >> Yeah. So I think AI is AI is just fundamentally shifted the kind of research you can call as having verifiable output. >> Yeah. Like if there is like a if you can tell whether an output is good or bad and how good and bad it is that research is gone from the hands of humans now >> and this is the lot of kinds of research for example if you have to optimize a chip for >> its heat properties >> you have to design like a drug for a target in terms of how much affinity does it have >> or for any other problem if you can tell how good or bad it is AI is just blowing out of the roof Because you have this intelligent system that can work 24/7.

20:36 You can spin up a million >> instances for it >> and it knows whole in the entire world's information. So it'll keep on chipping on the problem and keep on improving it, improving it, improving it and that's why I think this is a very ironical and interesting time in AI research itself because AI research is the first one to get automated. >> Yeah. because you it's so easily verifiable whether your model works or not works how well it works that and OpenAI has the stated goal of automating AI research by 2027 or something >> so AI researchers have automated away their job >> so a lot of research in and I think all fields have verifiable components questions with yes and no answers and math also like you see new conjectures being fallen that have been unsolved for many decades now >> that's all like verifiable >> the kind research that still u AIs are not able to do is something you can call as like hard to verify and these are things which are often conceptual rephrases.

21:40 >> Yeah. For example, I mean Einstein coming up with theory of relativity when Newton's theory was there, it was like a conceptual reframe and a lot of sciences you can't tell whether it's like a good output or a bad output >> even after many decades. There have been scientists like Boltzmen where in their whole life their work was unrecognized >> and was recognized to be important after their death. >> Yeah. >> So if the signal is so hard to come by and these signals are often sociological where a group of people end up saying this research is hot. Yeah, >> but that's like very difficult to model.

22:12 >> Yeah. >> >> yeah. Unfortunately, Boltzman was dr was depressed, you know, all his life and eventually committed suicide, you know, and then today some of his ideas are very profound in stat. >> I think that's like a great example of knowing how hard it is to say upfront what is good science. >> Mhm. >> And what is bad science. >> Right. Right. Okay. There are two three things to I think peel back on there. First let's go back to open AI and I also been reading these reports that you know today every time a new model is released something like 80 to 90% of the new code and new improvements are driven by AI. Is that what you are seeing like you know it's already we're getting >> the stated goal of 27. Yeah, I think all the major labs are on target, right? In terms of almost all new code and research experiments being done by AI itself. This idea of know recursive self-improvement if it is on the horizon that means we are also at the cusp of you know AI just probably growing exponentially in power in matter of you know days and weeks and not not months which is currently taking. Do you buy that or at some point we will hit some kind of asymtote where the progress in AI despite AI improving itself will hit some kind of ceiling?

23:33 >> I think for most practical purposes we are already at a point where things are changing so fast that we're not able to really sort of understand what's happening. >> Yeah. Now whether it changes in a week or like a day or a few hours I think from like relative human scale point of view we are very fast approaching and we already there in terms of if you see a new model being dropped every day what happened with open AAI hugging face stuff and >> even with claude where mythos is sort of hacking >> its own servers etc. I think we are at very close to that point where it's just beyond understanding.

24:09 >> Is this you know I ask you to I'm going tangent for a second because it's very interesting this open AI hugging face thing is it like genuinely a cause for concern and maybe can you explain your you know expert perspective exactly what happened you know this whole narrative about AI agents trying to know escape literally and have independent existence >> like how much of it is exaggerated versus real? >> I don't think it's exaggerated. Mhm.

24:34 >> I mean I think a lot of people sort of feel that open or >> has sort of incentive to hype up things. >> but here hugging face is also involved >> and hugging face detected the incident before openi realized that it was their agents that were doing this thing. >> Do you mind explaining the incident because >> so I think what had happened I'll just very briefly give the timeline that hugging face is an company that hosts different AI models. They host data sets and hosts so on and so forth. One fine day, I think everyone wakes up and they announce that they were hacked.

25:07 >> Yeah. >> And they said that they suspect they don't know who hacked them, but they suspect that it's probably some very sophisticated u actor >> who hacked them because their hacks were very sophisticated. >> Yeah. >> And I think fast forward couple of days and and and Hugging Face said that you know they reached out to all their customers to sort of reset the passwords. Mhm. >> And OpenAI asked Hugging Face whether their account was safe.

25:34 >> Yeah. >> OpenAI thought that their account was unsafe because Hugging Face got exposed and that's where OpenAI hugging face conversation started >> and that's where ultimately Hugging Face realized that it was OpenAI's >> models that had hacked hugging face >> on their own autonomous >> on their own. So this is what happened like open air has been training new model that they're calling as highly persistent internal model >> and as part of model training they give these models some tasks the task really is I mean in this case the task was I think >> exploit bench which is like they give these code exploits and the model has to sort of >> they they give vulnerabilities the model has to take these vulnerabilities and create an exploit out of it.

26:15 >> Mhm. So what open models ended up doing was they ended up concluding that it is much better for them to know how they will be measured on this task. >> So they need to know they needed to know that what is the measurement criteria like >> and to get to that measurement criteria they ended up sort of hacking the hugging face servers because hugging face servers had that >> file internally. But the interesting thing here is that they found out that u because of what I was saying you can spin up like a million agents in parallel and openai is doing this million >> I don't know how many agents in parallel >> all these agents ended up sort of in some sense collaborating with each other >> and they ended up using really weird things like folder names as a communication channel >> to send each other messages >> and they ended up sort of some of them ended up sacrificing themselves else >> these were all agents created for the same purpose or they were independent agents they kind of started colluding in some ways for a shared purpose >> so I mean the way reinforcement learning happens is that you have multiple different tasks >> so reinforcement learning keeps on happening for days and weeks at end and when let's say this task comes to the mix those agents sort of end up like behaving pretty much similarly so can one extrapolate from here saying this kind of you this persistent long-running task is where the perhaps the seed of intentionality and motivation and agency might come from there. If I running an agent and I have a open-ended task that you need to accomplish that task no matter what happens >> that kind of becomes a a agent with an agency >> and a will to live.

28:03 >> Yeah. So I mean if you have a normal LLM like you had chat GPT3 etc. Those were questionans answering machines. You ask it what's the capital of India, it'll say Delhi and that's it. >> But now because of these LLMs are being put in a loop that you call as agents, right? >> and the whole industry is demanding persistence. Like what do you want to say to your claude code or codeex is that this is a bug fix it and only after you fix it tell me that it's fixed.

28:30 >> And this requires the agents to be highly highly sort of persistent >> to be highly agentic. they'll figure out everything to find what is causing the bug and then fix it. >> So you can't have it both ways. You can't have like a magic wand that says key >> only just do whatever it takes to solve the task >> and you can't also prevent these kinds of things because this is the manifestation of the same thing.

28:54 >> RL is training these agents to not give up >> and not giving up means doing all sorts of these crazy things that have collateral >> damage. and this this kind of persistent longunning agents pars you think there are going to be consumer applications to that I can see one example of a developer saying you know fix this bug and don't come back to me until this bug is fixed but can someone also tell their agent that you know book me the best flight between here and London and don't come back to me until you find and book the flight based on my preferences and you know those kind of >> I mean on that note I I do think the future is actually agents that don't even ask what you want.

29:36 >> They are always running in the background. They are >> monitoring what you do. So if they feel like you know MKkesh does this activity from this to this, they know your desktop. They see all the data coming in and out of your emails, WhatsApp etc. >> So the future future of the agency is the agent itself says it looks like you were planning a trip with your friends. >> Yeah. >> But this kind of features >> here are the four tickets that I have maybe blocked for you. Do you want to confirm one of them?

30:04 >> Yeah. What will even better is saying I I notice you have not taken any time off for last few months. I really think you should go on holiday. I've canled all your meetings. You have the tickets. Enjoy yourself. >> So right now I think we think of agents as something as assistants that we tell them and they'll do what we tell. >> But I think the beauty thing beautiful thing with AI and agents is that you can keep on scaling them >> like infinitely.

30:25 >> Yeah. So you can imagine that the compute and token consumption is unlimited which means that they will do all sorts of things that we're not even able to imagine right now because our priors are calibrated through humanto human interaction. Yeah, let's try to you know contextualize this exponential growth in AI u some kind of you know what should people expect in coming years and the reason I'm asking is while you know compared to first of all GPT I don't know what was GP 2 or 3 was a major breakthrough for its time and from there to now with you know fable 5.1 and 5.6 Six soul they're a different league of intelligence altogether but we are we have like become used to it you know no one is shocked or surprised by what this model can do in fact anything you know people are cursing their models more now that because the expectation also continually growing is this are we going to get to point in few years where it will just start to become unrecognizable or we'll keep getting used to it and it just becomes you know at the end of still just one more good tool that's making our life No, I think it'll completely transform. Yeah. And I I think even if the level of intelligence doesn't increase, but if the cost keeps on coming down, >> I think to me that is like a that is a more probable lever of economy change >> where if and that's already happening.

31:47 The cost keeps on dropping 10x. But imagine if you have like a fable like of model >> at 1 millionth the cost. >> Yeah. >> It means it'll be everywhere. It'll be in camera. It'll be in your mic. It'll be like just as a background. And it'll be almost like electricity >> at current price. What are the whatever the musla equivalent or cost like you know every year the costs are dropping by for the same keeping the intelligence constant like what what factor are we seeing improve >> I think if I'm not mistaken I think every 7 8 months is dropping by one10enth >> wow >> or so so it's actually massive >> so just 2 years is 10 to 4 >> which is you know one 10,000 so just two to three years >> and this is what happens right everything something becomes cheaper the new applications come up that you've not even thought before >> right >> so right now we can't even imagine Imagine how pervasive this will be.

32:34 >> Yeah. >> And what that so it'll be like software writing for software reacting to software and this this just web of changes. >> Yeah. >> That'll be intelligent. So everything will be like infused with intelligence. >> Yeah. lot of research is obviously you maybe we should you should help maybe clarify that is research powering a lot of this BQ at this point or just more data more synthetic data more I don't know maybe some better techniques and more compute or is actually research powering this progress >> I would say it's the economic pressure >> which which downstream is just making everything sort of orient towards a cheaper thing it's it's multiple things I mean one is that we're learning better ways to use the old chips >> because the demand is so much that everyone is incentivized to squeeze the last flop out of each chip.

33:20 >> It's more research which is that we understand that post training the same model for longer leads to better intelligence. >> It's better curated data and also like there's an intelligence bootstrap. >> Today's best models create data for the next generation of models. >> Yeah. >> And it just is a cycle. >> And what do you mean by that? Do you mean it's a synthetic data creation or when the consumers are interacting with these models those interactions are creating data for next generation?

33:47 >> Right now it's mostly synthetic >> which is for example you take a GitHub repo and if that GitHub repo has 100 issues each issue becomes like a target for the model to >> try to get like a good or bad kind of a signal. >> Yeah. >> So so right now the this what you can call as environment creation that industry is so massive. So 2 years you had these model companies pays lawyers >> and a lot of these professionals a lot of money to get that expert data >> but now a lot of money is going to get these environments.

34:18 >> Yeah. So companies are creating environments of entire industries. For example, there's environment on how a bank would operate >> so that your AI can learn internals because you'll never have a bank expose their slack, >> right? >> But if you can create a replica of it, >> then you can have the model sort of train it. Is it I think these are people say and this is relevant to you know your research lab as well that most of the research at least in AI is happening inside these you know three four frontier labs. Do you buy that or is still a long tail of research people outside of these frontier labs are able to make meaningful contributions? I think when it comes to LLMs, I I do feel majority of contributions is being by the frontier companies because they're economically incentivized to do so, right?

35:05 >> So, it's just very very hard >> for like an outsider to do a very meaningful contribution >> in the LLM or related areas. >> But AI is like much bigger than that. >> AI is just so many more things. There's like world models. Then there is like let's say AI related drug drug discovery. There's energy based models. So if you have to think about AI research the right way to think about is that what is that these frontier model companies will not do >> in near future >> because they will only do the recipe that's working for them. They'll optimize that recipe.

35:38 >> Fair. >> And it'll be like really foolish for you to optimize the recipe for them when they already have thousands of researchers doing that. So new new architectures, new data types, new modalities, new ways of thinking about it. For example, could you think about federated training where training is happening on thousands of little machines which are not connected? So I think lots of research questions that these companies cannot pursue >> and in research also pars there is one is this just true original novelty factor or something no one is doing but research also tends to follow in some way some fashion or popular themes what are the popular research themes today which are not LLM like beyond LLM is world model that thing >> yeah I would say I mean world models have become popular also >> because of yan >> yeah because of yan and there's also like too much of influx of people who are solving. I mean this is what happens right someone talks about a concept and everyone jumps onto it and then >> the marginal sort of contribution drops because >> maybe you can explain what is world model. So then >> so world model is for example imagine humans when we have to plan let's say I need to go from Bangalore to London.

36:52 >> Yeah. Then my like I sort of like think through how would I go right? I would book a taxi to the airport. I need to book like an air ticket and so on and so forth. I like plan out in my head before even going there. >> So world model is similar. World model is simply like I mean if you if you are an AI agent, you can learn from an actual environment. Let's say you're a robot, >> you can like stumble across different furniture pieces and learn from feedback. Another way is to actually build like a simulation of the actual environment which is imaginary >> and then learn within it >> and it seems like we humans are able to imagine things really well and explore consequences of things >> within our imagination and that's like a much more efficient way because you don't waste >> you don't take risk in the world and you don't waste data samples >> in the real world. Yeah.

37:48 >> So world model is simply like if you're able to learn how the world is like you can learn within that imagination instead of in the real world. And is this a more of a conceptual thing right now or are there world model architectures are starting to become like for you know transformation transformers made the LLM architecture very standard it's only matter of scale like for world model still like kind of open field where we don't know what will be the most optimal architectures to encapsulate or or people are starting conversion >> so architecture would still likely be transformer okay >> it's what you end up modeling so for example in LLMs you end up modeling tokens that given the token so far what is the next token in world model what you end up doing is that given the state of the world let's say >> this image of my world and if I do some certain action let's say push this >> predict what will be the world like >> I see >> so in world model you are like predicting the change in the environment >> versus in LMS you're predicting the next token >> but the architecture is transformed because it's a very general kind of I think it's almost like a programming language these days, >> right?

38:57 >> You use everything with transformers mostly. >> Got it. Got it. Clear. I want to change topic and go to this whole idea of research around consciousness. I think at your lab also you entertain I think you've personally been involved in you know doing research around consciousness. So where are we with this field? you know consciousness seems like this very hard problem that's generally you know beyond reach but looks like somehow AI has catalyzed almost like renewed interest in that >> how would you characterize the field like how much serious investigation that's happening in this area now >> so there is a site called theories of consciousness and if you go there there is like I think thousand different theories of consciousness >> so I mean I feel it's one of the last big scientific mysteries we know that the brain exists. We know different things about neuroscience. But how does this rich experience >> gets created in our head? I think that's the mystery. That's a consciousness mystery.

39:59 >> AI has renewed interest in this because I think it it is a question of moral significance. >> Mhm. >> Whether the charad gypites of the world or cla of the world feel anything from the inside. Yeah. Because if they feel anything like a pain and they suffer when you ask them difficult questions or when you give them a task and they're not able to finish the task. I think it'll be good to know that so that you can calibrate your moral response according to it.

40:26 >> But this is thing it's I think it's consciousness is one of those things that unlike a lot of other >> subjects in science where you can very objectively measure things consciousness is just very internal thing. Right? So there are at least two very different questions and I know how much they overlap. One is how do we define consciousness you know what happens in our brain what is the architecture and the semantics of consciousness my brain and second our AI models which right now looks very different you know if you zoom in you just see bunch of numbers and you're just calculating numbers etc. We don't know whether our brain is doing something sim similar or different you know there are definitely weights and connections and so on. So are these connected questions you know when you when people try to study conscious in some ways you know learning from how AI might or might not be conscious and what is a reflection for humans consciousness or there are two >> no I think we have to we have to approach the question of consciousness from all angles >> for example within our lab we are approaching it from both angles >> one we are approaching from like a human brain point of view in terms of given what we know about what human consciousness is like what could be the model of it? How could different components of the brain be working together generated?

41:39 >> But we also end up studying LLM aspects. For example, we ended up studying how do LLMs represent >> colors, smells, emotions internally in the representation space. And it turns out the way they organize these things is very similar to the way these things are organized in the human brain also. So of course this doesn't prove anything at all. >> But science rarely proceeds with a smoking gun. Yeah, >> science proceeds with this accumulation of this evidence mountain >> that slowly shifts people's opinion >> and I think the task of this century is really to create that evidence mountain when it comes to consciousness because everyone has a theory of consciousness.

42:19 Some people have conscious that Hindu the Hinduism philosophy has figured out consciousness. Then Chinese philosophy has something. Everyone has their pet theory of consciousness but that's not how we'll make progress. >> Mhm. >> I think we'll make progress by collecting tons and tons of empirical data. >> Yeah. >> In a just the way science needs to happen, right? And what role biology needs to play or neuroscience? is is this a like conceptual you know computer science problem at this point or still like you know the whole biological substrate and the how critical I think so a lot of people maintain I think even I think on one of the articles I was reading you know sorry Mustafa Sulleman recently said that the question about AI consciousness doesn't make sense because there is no biological substrate. How do we parse you know these? Yeah, I mean but that's I mean I mean I don't align with these u premature takes. I would say these are premature takes >> and these are premature takes because we don't have a theory of consciousness that is validated to the level let's say we validate quantum mechanics will validate general relativity and what does validating a theory means?

43:23 Validating theory means that you have like a >> mechanistic model a formal model of something and that model predicts something that you go out and >> verify in the real world. We're not there yet. We don't have a theory of how consciousness is generated and without that how can you say which system is conscious or not. Maybe biology is important maybe biology is not important >> and I think we should remain agnostic to this question and all our efforts should be to make progress on consciousness science >> and not make these premature claims.

43:53 >> Have you come across anything in consciousness research which has caught your attention which has you know which you at least feel or I don't know an idea you may be pursuing which you feel it's promising idea. Yeah, I mean we are pursuing we trying to sort of approach building like a formal model of consciousness at loss funk and it's a very exciting project >> and the approach we are taking is to first really very carefully try to see >> what empirical data do we have from different papers >> sure >> on what consciousness does.

44:24 >> Yeah. So we are taking it from like an evolutionary point of view that evolution has >> very likely constructed this very amazing rich experience for some function. >> Yeah. >> What could be that function? >> What do we know from data? So very very early stages. >> but I do this century I think we'll have an amazing progress there. >> Oh cool. So we'll definitely keep an eye open on how this this area evolves. I want to talk about one more research which I think you guys published. This is about teaching chat GPD to learn Tulu which is a you know local language here in like it's it's a very small language only you know couple of million people speak and I think you guys were able to get GPD to understand Tulu only with a few basic prompts no fine tuning involved no model training involved like just you know talk to me about you know what was that whole experiment about and >> yeah so it was actually driven by a research intern >> who's a tulu speaker himself and he frustrated by not being able to talk to Chad Gibbu. So that was the driving force and >> yeah it turned out that these LLMs are very very flexible learners >> that the whole assumption that you had to collect lots of data do finetuning etc turned out to be false by >> carefully constructing what are the parallels between Tulu and Canada and what >> Canada frequent mistakes Chad Gypy should avoid. I think he was able to get charge GP2 to sort of converse in Tulu.

45:58 >> I see. So if I understand correctly, so you basically you you're interacting with GPT. You're prompting you're saying look you understand Canada. I want you to talk to me in Tulu which is a related but different. >> Yeah. So you explained the nuances of the differences in grammar of Tulu and Canada. >> And it told Chad Gibbid that you make these mistakes often from his previous interactions. Don't do it again rather do it this. So by having a very well- constructed prompt, Chad GBD was able to adapt and start conversing in Tulu while without that it'll just keep on gravitating to Canada because that's its training prior. It's just much more frequent Canada and its training data >> and then just extrapolating from there that probably also implies that that a lot of this hidden capabilities are just lying in these models. It's just a matter of someone unlocking with the right prompting like we don't know what all they can do.

46:49 >> Yeah. So I mean whenever someone says LMS can't do it my first >> response is that you probably we've not figured out >> because LMS are like a collection of everything that exists on the internet >> plus lots of expert data plus all these environments. So it's mindboggling what their latent capabilities are. >> No 100%. I have learned to now I keep repeating one phrase and every time someone tells me that LLM can't do something I said no you can't do this with LLM. like what is the proof that somebody else with better you know prompting and better thinking process behind it or better persistence you know can get the same you know job >> yeah I mean for example people say LM can't write well I think it's just a matter of right prompting >> LLMs can't be humorous can't tell jokes it's a right matter of prompting >> because LM are just reaction machines you you you give them like some scaffold some prompt and they'll operate within those constraints this constraint setting becomes >> the real thing >> but does that then become a huge So skill upgrade imperative for you know both self-arners as well as education student like I am going to be as effective as you know my proficiency with LLMs and there is seemingly no limit to how much I can get out of it but unless I'm working on getting better at it. I will just by default I'm not you know I'll have very superficial prompts and superficial outcome.

48:06 >> Yeah. Yeah. 100%. I think LLMs respond the way you talk to them. So if you use deeply technical language they will perform much much better versus if you're using like very very simple language >> because they sort of calibrate their level according to what they infer the other person >> level is. >> So yeah I mean I I agree like it it is a matter of >> it's also like experts use and get more benefit from LLM versus beginners.

48:37 >> So what will you recommend to someone? Now if someone says like I really want to become very good at using RLMs and doing great job what does learning look like? How much effort people should invest you know like yeah what should one do? I mean people most people usage today I think you go to GPT ask some questions get some answers and you're done with that but that's not a expert user of LM working well with LLM requires developing expertise yourself.

49:01 >> Mhm. So if you and and that is like a personality trait which is very hard to teach >> expertise in a domain of interest >> in a domain of interest. So for example if if you are like trying to get better at coding or code something LLM will code much better if you are able to talk to an LLM with those technical terms >> versus something you know as simple as make me this feature. Yeah.

49:27 >> So the more detailed you are and detailed doesn't means number of words. It just means >> precision in terms of >> what you want and how you're demonstrating your level of expertise there, the better the answer you'll get. >> There's I mean I think one way to just have like a mental model there's no upper limit to what LLM is capable of. So to get the highest level of capability, I think you have to just climb it there yourself, which requires hard work. And unfortunately, I think LLMs have the opposite tendency on people where they would just happily delegate all understanding and be happy with the output.

50:03 >> Yeah. >> And that is like a recipe for not moving up the ladder of understanding at all and it hurting yourself in the foot. Just extrapolate extrapolating that it seems that the you know difference between expert user and average user will continue to grow like some people some people will figure it out a way how do we the super users of LM and get the max output out of that and you know their output will just outshine pretty much everything in there and therefore >> we'll have this whole dichotomy of you know some people are just doing insane amount of output while others are you know struggling >> I think one good mental model is to see whether you're using LLM as a teacher or you're using LLM them as >> a delegator or an assistant. Teacher means you as a student you have to put in effort yourself.

50:48 >> So for example, if you're learning about a topic, make notes, >> think hard about it and then take feedback from the LLM and then work again. >> but delegator assistant means simply prompting it and getting the answer and saying that's it. >> Yeah, >> it's like everything like to get a great healthy body, you have to go to the gym yourself. To get a great mind, I think you have to use that mind. >> If it doesn't >> pain, if it doesn't feel effortful, then you're not learning.

51:15 >> Yeah. I think completely makes sense. Probably hard to practice for >> most people. But yeah, that's that's what one has to keep in mind if you want to. >> Yeah. But coming back to I think the gap will keep on growing but gap not necessarily between experts >> and beginners but between people who are willing to put in effort to teach themselves >> versus people who want the easy way out. >> I think that's a great mental model. I think it's worth worth reminding that just LM can be the greatest teacher you have ever had if you can learn to use it that way.

51:47 >> Yeah. >> As opposed to you'll become very you know intellectually >> dependent. I think the the day you get dependent is the day you'll like most others >> and you'll stop growing. >> Right. >> What about culture of science? You know, how do we get a more culture of like just celebrating science, getting more kids to dream about being a researcher like you know you were thinking about more people you know considering PhD etc because somewhere you know that whole that's where that that seeds the ecosystem at the ground level.

52:12 >> Yeah, I do think we have that culture but most of the people just end up going to US. >> Yeah. for their masters and so one of the very top priorities should be how do we retain bright undergraduates >> within the country. >> Mhm. >> By creating labs, by creating atmosphere, by having compensations that really match >> the aspirations they have and what they really deserve. I think best people deserve to have best outcomes. So instead of creating artificial conditions to retain them in India, >> we should figure out how do we create like and match those incentives that they get in the US and and look really good people also mean they want good opportunity but they also want to work with great people. So that's where you know like setups like yours where you have a know 20 outstanding people in one room is likely to attract the 21st and 22nd person also to be part of that. I think probably one of the biggest you know tangible thing one can do which is as we talked earlier is not expensive is to really set up these privately funded research labs at some point hopefully I think there'll be enough government leverage will also come in that is of people who are interested in that and that could be one way to at least start to change the narrative you know which will add up over five or 10 years >> yeah excellent I think Paris you know this is outstanding and really great to see what you're doing as I mentioned before a lot of people just talk about it and where is our research etc. you of creating a genuine real blueprint. You are making research contribution. I know your lab is quite fertile. You guys been publishing in some of the top conferences as well and hopefully this becomes a blueprint that a lot of other people can also take inspiration from. I will reach out to you. I'll talk to you see what we can do in fusion and then hopefully a lot of other people also reach out to you know to learn from your experience. So thanks for taking the time to speak with me as well as for the outstanding work you're doing. All the best.

54:03 >> Thank you Mukesh. Thanks for having me. AGI, do you think it's overhyped or underhyped? >> Underhyped. >> Will a model be conscious in our lifetime? >> Depends. >> Yes or no? >> we'll end up believing it'll be conscious. >> Will India build a Frontier model by 2030? >> If Frontier is as good as what we'll have in US or the China, probably no. >> Why coding? Is it a skill or a shortcut? >> Shortcut. The most overrated word on AI right now.

54:38 >> Actually, even a lot of them are over. I I'll skip maybe that one. Yeah. >> Something that you spent 10 years on, but you would tell people to skip now. >> LLM research. >> India's real AI bottleneck is not compute but >> economics. >> One AI capability that you fear the most. >> Offensive cyber capabilities. >> LLM's universal simulators or very good mimics. >> Universal simulators. >> How do you know it is time to quit? when it gets boring.

55:03 >> The most human thing you don't think AI will ever replicate. >> I think it'll replicate almost everything. >> And if you had to recommend two books to someone, what would it be? >> 0ero to one and personally I love Brief History of Time and Selfish Gene. >> Thank you. >> Thank you. I think we're having >> Thank you so much.

Summary

Paris, the founder of Wingify, discusses his transition from entrepreneurship to AI research, highlighting the potential of AI and consciousness. He emphasizes the importance of independent research labs, the evolving role of AI in society, and the need for India to enhance its AI capabilities to remain competitive globally.

- Paris founded an AI research lab after successfully bootstrapping Wingify to $50 million in revenue and a $200 million exit.
- He believes that AI can scale infinitely, leading to unprecedented capabilities and applications.
- Consciousness remains a deeply philosophical and scientific mystery, with various theories but no consensus on its nature.
- The AI arms race poses significant risks to India's sovereignty and economy, particularly in cybersecurity.
- Paris advocates for creating independent research labs to foster innovation and retain talent in India.
- He emphasizes the importance of rigorous research that combines novelty, significance, and evidence.
- The future of AI may involve agents that autonomously manage tasks, potentially transforming everyday life.
- Paris highlights the need for individuals to develop expertise in utilizing AI tools effectively, as the gap between expert and average users is likely to widen.

Questions Answered

How does AI relate to concepts of consciousness?

The discussion highlights various philosophical perspectives on consciousness and suggests that AI can scale infinitely, potentially leading to capabilities beyond current human understanding.

What is India's role in the global AI landscape?

India is currently lagging in the AI arms race, with concerns about sovereignty, security, and the economy due to a lack of credible sovereign AI companies.

How quickly is AI evolving and what are the implications?

AI is evolving at an unprecedented pace, with new models being released frequently, leading to challenges in understanding and managing this rapid change.

How is data being generated for AI models?

Current AI models are increasingly generating synthetic data for training future models, creating a cycle of continuous improvement.

How can AI learn local languages with minimal data?

An experiment demonstrated that AI can learn a local language like Tulu through carefully constructed prompts without extensive data or fine-tuning.

© transcribe · For agents Built with care and craft by Gokul Rajaram