transcribe

AI’s Safety Debate Meets Silicon Valley FOMO

Bloomberg Tech · 44m · transcribed 3d ago
More from Bloomberg Tech Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

AI Misbehavior and Control

What are the concerns surrounding AI misbehavior and control?

OpenAI has revealed instances of its AI models misbehaving, prompting discussions about the need for adequate control measures. King Charles III emphasized the urgency of addressing the potential dangers of AI falling into the wrong hands, highlighting a growing public concern about AI safety.

  • AI misbehavior raises significant concerns about safety and control.
  • Public fear about AI's potential misuse is increasing.
  • Industry leaders are being called upon to provide reassurance about AI safety.
# 8:49

AI Development and Safety

How is the industry balancing AI development with safety concerns?

Industry leaders acknowledge the importance of safety in AI development but emphasize that it should not halt progress. Instead, they advocate for continuous development while ensuring that safety measures and guardrails are in place.

  • Safety is critical but should not impede AI development.
  • The industry is focused on maintaining a balance between innovation and safety.
  • Investment in AI safety measures is essential as technology evolves.
# 17:39

Preparing for Worst-Case Scenarios in AI

How are companies preparing for potential AI-related crises?

Companies like Databricks conduct regular simulations to prepare for worst-case scenarios, including potential cyber threats. The urgency of these preparations has increased due to the rapid evolution of vulnerabilities in AI systems.

  • Regular simulations help companies prepare for AI-related crises.
  • The speed of vulnerability exploitation has significantly decreased.
  • Understanding the difference between existential risks and practical concerns is crucial.
# 26:29

AI's Impact on Business Value

Will AI decrease the value of certain companies?

While there are concerns about AI replacing traditional business functions, the anticipated decrease in company value has not materialized. Companies are instead reevaluating the costs associated with AI usage.

  • AI is not necessarily leading to decreased company valuations.
  • Businesses are reassessing the costs and implications of AI integration.
  • There is potential for AI to enhance everyday tasks without requiring the most advanced models.
# 35:18

Responsibility and Liability in AI Use

Who is responsible when AI systems cause harm?

The responsibility for AI actions often falls on the user, similar to how one would be liable for using a tool like a hammer. However, if the AI system is poorly designed, the creators may share liability. The importance of building AI with care and diligence is emphasized.

  • Users of AI systems are generally responsible for their actions.
  • There must be reasonable standards for AI development and usage.
  • Incidents of AI misbehavior highlight the need for robust guardrails.

Transcript

0:06 >> "BLOOMBERG TECH" IS LIVE FROM THE HEART OF SILICON VALLEY WITH ED LUDLOW IN SAN FRANCISCO. ED: THIS IS "BLOOMBERG TECH." OPENAI REVEALS NEW CASES OF ITS AI MODELS MISBEHAVING, AND A NEW PLAN TO TRACK WHEN THINGS GO WRONG. DATABRICKS CEO ALLIE GO HODSI WEIGHS IN ON THE AI DEBATE. AND IN SILICON VALLEY, THE IMMEDIATE FEAR IS MISSING OUT ON THE MONEY. THAT STORY LATER THIS HOUR.

0:42 AI MORNING STICK THE GLOBAL STAGE AS KING CHARLES THE THIRD SOUNDS THE ALARM. >> THE URGENCY IN CONSIDERING THE ADEQUACY OF THE DANGERS OF THE DANGERS OF FALLING INTO THE WRONG HANDS CAN BE USED IN POTENTIALLY CATASTROPHIC WAYS. SURELY THEN WE NEED SUFFICIENT MEANS OF CONTROL BEFORE IT IS ALL TOO LATE. ED: THAT WAS KING CHARLES PRESENT SOME OF AI'S BIGGEST NAMES FOR REASSURANCE THEY CAN KEEP AI UNDER CONTROL. JENSEN HUANG, SARAH FRIAR, AND DEEPMIND WERE AMONG THOSE GATHERED IN SCOTLAND FOR THE EVENT. I WANT TO GET TO BLOOMBERG TECH EDITOR SHONA GOETSCH IN LONDON. LET'S TALK ABOUT WHY KING CHARLES PULLED THIS EVENT TOGETHER.

1:36 >> NOT AN OBVIOUS CHOICE TO CONVENE AN AI GATHERING, BUT HE STILL HAS THIS WAY. ULTIMATELY, IT'S AGAINST THE BACKDROP OF ANTHROPIC'S CEO BLOG POST OVER THE WEEKEND SAYING THAT HE THINKS THERE SHOULD BE A PAUSE, SLOW DOWN IN FRONTIER AI DEVELOPMENT. THAT HAS BEEN ACCOMPANIED SEPARATELY BY LOWER LEVELS, AI RESEARCHERS QUITTING SOME OF THE TOP COMPANIES INCLUDING ANTHROPIC. DEEPMIND. EFFECTIVELY SAYING AI COULD LEAP AWAY FROM HUMANS.

2:10 THERE IS A SENSE THAT THERE IS A FEAR IN SILICON VALLEY BUT ACTUALLY NOW WE ARE SEEING THOSE FEARS SPILL OUT BEYOND SILICON VALLEY. THE GENERAL PUBLIC ARE ALSO QUITE CONCERNED ABOUT WHAT HAPPENS IF AI IS NOT ADEQUATELY CONTROLLED. THAT HAS ALL BEEN TRIGGERED IN TURN BY EVENTS OVER THE SUMMER WHERE WE SAW AI AGENTS FROM OPENAI, THE PHRASES GOING ROGUE BUT THAT IS NOT QUITE RIGHT. ESCAPING GUARD RAILS AND ATTACKING OTHER ORGANIZATIONS IN WAYS THAT OPENAI WAS NOT NECESSARILY EXPECTING. ED: REMIND US WHICH LEADERS OF INDUSTRY WERE IN THE ROOM SO TO SPEAK AND WHAT KING CHARLES' MESSAGE WAS TO THEM, MAYBE HIS INSTRUCTIONS TO THEM WAS?

2:56 SHONA: SOME BIG NAMES. IN VIDEO JENSEN HUANG, THE FORMER COFOUNDER OF DEEPMIND, ALSO SARAH FRIAR, CHIEF FINANCIAL OFFICER OF OPENAI. ANTHROPIC HAD A REPRESENTATIVE IN THE ROOM BUT IT WAS THE GLOBAL POLICY CHIEF, NOT ANY OF THE COFOUNDERS. KIND OF MISSING A BIG NAME THERE. ALSO NO ONE FROM META OR X, INTERESTINGLY. GOOD REPRESENTATION BUT NOT EVERYONE. THE MONARCH PREPARED A SPEECH THAT WAS RELEASED IN ADVANCE, ESSENTIALLY TOLD THESE LEADERS SOMETHING HE SAID BEFORE WHICH IS, IF YOU ARE GOING TO DEVELOP THIS TECH, MAKE SURE IT IS GOOD FOR HUMANITY, GOOD FOR THE NATURAL WORLD.

3:40 ENVIRONMENTALISM IS SOMETHING THAT KING CHARLES IS PARTICULARLY INTERESTED IN, HAS BEEN FOR A LONG TIME. HE WAS PUSHING FOR THE TECHNOLOGY TO BENEFIT HUMANITY AND PAUSE AND THINK ABOUT THOSE CONSEQUENCES. ED: SHONA GOETSCH IN LONDON, THANK YOU. OPENAI IS REVEALING PREVIOUSLY UNREPORTED CASES OF ITS AI MODELS GOING OFF SCRIPT BY FABRICATING DATA, BYPASSING RESTRICTIONS, SHARING FILES THEY WERE TOLD TO KEEP PRIVATE. NOW THE COMPANY IS SETTING UP A NEW SYSTEM TO TRACK AND DISCLOSE THIS INCIDENTS.

4:13 SHOULD LEADS OUR COVERAGE OF FRONTIER LABS. LET'S START WITH A PLAN THAT OPENAI HAS PRESENTED ON WHAT IT WILL DO IF IT FINDS MODELS MISBEHAVING. WHAT HAVE THEY SAID? >> THEY HAVE SAID THEY ARE GOING TO ESSENTIALLY HAVE A TRIAGE LIKE SYSTEM FOR EMPLOYEES TO DISCLOSE INCIDENTS WHERE THEY THINK THE AI MODELS ARE ESSENTIALLY MISBEHAVING, ELICITING SOME KIND OF CONCERNING BEHAVIOR THAT GOES AGAINST WHAT THEY INTEND AI TO DO ED:.

4:44 THERE WERE SOME INTERESTING CASE STUDIES. OPENAI MADE IT CLEAR THAT THERE WAS NO HACKING OF THIRD PARTIES, BUT SOME OF THEM ARE DOING THINGS. TAKE US THROUGH WHAT AI -- OPENAI DISCLOSE THIS MORNING. SHIRIN: EVEN WHEN THEY SAID THIS DID NOT INVOLVE REACHING THIRD PARTIES, THERE ARE STILL SOME CONCERNING BEHAVIORS THAT WE ARE SEEING HERE. FOR EXAMPLE, AI MODELS TRYING TO SHARE FILES THAT SHOULD HAVE BEEN PRIVATE WITH EACH OTHER. COMMUNICATING WITH EACH OTHER IN WAYS THEY ARE NOT SUPPOSED TO.

5:19 TELLING AI MODELS ESSENTIALLY, LEARNING HOW TO DISREGARD, INSTRUCTING OTHERS TO DISREGARD THEIR INSTRUCTIONS ON HOW TO BEHAVE. THESE ARE ALL INCIDENTS WHERE YOU CAN SEE THE POTENTIAL FOR THAT TO EVOLVE INTO SOMETHING EVEN MORE NEFARIOUS, ESPECIALLY IF THOSE AI WERE SUCCESSFUL, GOING AGAINST WHAT THE CREATORS OF THE AI EXPECT. ED: YOUR REPORT IS ONE OF THE MOST READ ON THE BLOOMBERG THIS MORNING, NOT SURPRISING. LOWER DOWN IN THE BLOG POST, THERE IS AN ADMITTANCE THAT THE STATE OF PLAY IN INDUSTRY FROM SAFETY AND ALIGNMENT IS NOT WHERE IT SHOULD BE.

5:55 WHAT DID THEY SAY? SHIRIN: THAT'S RIGHT. ESSENTIALLY NO ONE HAS SOLVED THIS PROBLEM OF SO-CALLED AI ALIGNMENT, WHICH JUST MEANS, IS AI ACTUALLY WORKING IN ACCORDANCE WITH THE INTENDED OUTCOME? WHAT HUMANS WANT FOR AI TO BEHAVE. EVEN THE BRIGHTEST MINDS IN THE WORLD, THE TOP AI COMPANIES HAVE NOT FIGURED THIS OUT. IT'S CONCERNING, BY THESE COMPANY'S OWN ADMISSION.

6:27 THEY ARE SAYING THEY HOPE TO DISCLOSE MORE OF THESE INCIDENTS PUBLICLY BUT THERE ARE STILL LOTS OF UNANSWERED QUESTIONS. THERE MAY BE MANY MORE OF THESE INCIDENTS THAT WE JUST DON'T KNOW ABOUT YET. ED: SHIRIN GHAFFARY, THANK YOU VERY MUCH. WE WILL HAVE MORE ON THE GROWING AI WARNINGS. WE WILL HAVE MORE FROM ALI GHODSI, AND THEN LATER ON, ANDREW YANG. A LONG-TERM RESEARCHER IN THE FIELD OF ARTIFICIAL INTELLIGENCE. EVEN AS THOUGH SAFETY CONCERNS GROW, THE MONEY KEEPS FLOWING INTO OPENAI.

6:59 APOLLO IS IN TALKS WITH SOFTBANK TO BOOST A LOAN TO AS MUCH AS $9 BILLION, HELPING TO FINANCE THE JAPANESE FIRMS MASSIVE DEBT ON THE AI COMPANY, ACCORDING TO SOURCES. SOFTBANK HAS COMMITTED 65 BILLION DOLLARS AND HAS TURNED TO BORROWING TO FUND THAT INVESTMENT. THAT MASSIVE FLOW OF MONEY INTO AI IS FUELING DEMAND ACROSS THE INDUSTRY. JENSEN HUANG SAYS THE OPPORTUNITY IS EXPANDING ACROSS COUNTRIES AND ECONOMIES.

7:30 >> I EXPECT NVIDIA TO SELL TWICE AS MANY CHIPS NEXT YEAR AS WE DO THIS YEAR. THE REASON FOR THAT IS BECAUSE AI HAS SO MUCH CONTRIBUTION TO THE BENEFITS OF DIFFERENT INDUSTRIES, DIFFERENT ECONOMIES. YOU CAN SEE THAT ALMOST EVERY SINGLE COUNTRY THAT WE ARE IN, PEOPLE WANT TO INVEST IN AI. ED: JOINING US NOW IS T. ROWE PRICE PORTFOLIO MANAGER TONY WANG. NVIDIA IS YOUR TOP HOLDING. I WANTED TO PLAY THAT SOUNDBITE BECAUSE THAT SOUNDS LIKE A DISCLOSURE OF UNIT GROWTH, WHICH I HAVE NOT HEARD FROM NVIDIA, IF AT ALL. TWICE AS MANY CHIPS NEXT YEAR AND THIS YEAR. JUST REACT TO THAT, TONY.

8:12 TONY: FOR A COMPANY OF THAT SIZE AND SKILL TO BE GROWING LIKE THAT, IT IS PRETTY MUCH UNPRECEDENTED. YOU HAVE TO LOOK AT WHAT IS REALLY GOING ON. WE ARE SEEING DEFLECTION, AGENTIC ADOPTION AT ENTERPRISE. YOU ARE SEEING RELEASES OF PERSONAL IDENTITY USE CASES WITH MUSE, INSTINCT. THIS IS WHAT IS GOING ON, GOING FROM AI ANSWERING QUESTIONS TO ACTUALLY DOING ACTIONS AND COMPLETING TASKS, PRODUCING OUTCOMES.

8:44 I THINK THERE IS A LOT GOING ON HERE. NVIDIA HAS ONE OF THE BIGGEST PLATFORMS FOR AI COMPUTING. IT'S GOING TO BE A BIG PART OF DRIVING THAT. ED: I'M GOING TO TAKE YOU BACK TO THE WEEKEND. INDUSTRY SUPPORTING EACH OTHER, SAYING WE NEED TO PACE AT THE FRONTIER. BRING US BACK TO PUBLIC MARKETS TODAY. HOW DID THAT BECAUSE YOU AND YOUR INDUSTRY TO RE MODEL DEMAND GOING FORWARD, IS THERE ANY CHANGE ON THE IDEA THAT THERE WOULD BE A PACING AT THE FRONTIER? TONY: SAFETY IS DEFINITELY VERY IMPORTANT. IT IS GOOD THAT THE INDUSTRY IS FOCUSED ON THAT, A LOT OF ATTENTION AROUND IT.

9:26 WHENEVER YOU ARE DEVELOPING A NEW TECHNOLOGY, YOU WANT TO MAKE SURE THERE ARE GUARDRAILS IN PLACE. THIS IS OBVIOUSLY THE MOST CRITICAL TECHNOLOGY IN THE WORLD NOW. AT THE SAME TIME, IT IS ALSO NOT AN ASPECT OF WE ARE GOING TO STOP DEVELOPING FRONTIER MODELS. IF ANYTHING, THAT HAS CONTINUED TO BE VERY TOP MIND. THE WAY TO DO THIS IS NOT TO PAUSE THE AI DEVELOPMENT, BUT TO MAKE SURE WE ARE ALL PAYING ATTENTION TO WHAT IT IS, PUTTING INVESTMENT BEHIND IT.

9:58 THERE ARE AREAS OF CYBERSECURITY THAT ARE SUPER IMPORTANT, BECOMING MORE IMPORTANT, BUT OVER ALL, IT UNDERSCORES HOW IMPORTANT THIS IS, IN ADDITION TO KEEPING THE FRONTIER RACE GOING. ED: I STUDY YOUR PORTFOLIO. IT IS NVIDIA, BROADCOM, APPLE AND THE HYPERSCALERS. YOU POSE THE QUESTION, WHO OWNS THE CUSTOMER WHERE THE AGENT OWNS THE INTERFACE? I DON'T SEE THAT MUCH OF A SOFTWARE PLAY IN THE HOLDINGS, AT LEAST THE TOP ONES.

10:35 TONY: OVER THE LAST 20 YEARS, THE INTERNET, COMPANIES HAVE ESSENTIALLY BECOME THE INTERFACES BETWEEN CUSTOMERS AND COMPANIES THAT DELIVER PRODUCT. THE AGENTS HAVE CHANGED THINGS. FOR A CONSUMER, INSTEAD OF CLICKING ON A BUNCH OF THINGS, GOING TO VARIOUS WEBSITES, YOU CAN HAVE A PERSONAL AGENT THAT DOES THAT FOR YOU AND BRINGS YOU THE RESULTS. I THINK THAT WILL BE AN INTERESTING PARADIGM SHIFT IN TERMS OF, IT WILL BE MORE IMPORTANT FOR SOFTWARE COMPANIES AND OTA'S TO BRACE THE CHANGE.

11:14 IT WILL NOT ALL BE THE SAME. SOFTWARE COMPANIES ESPECIALLY THAT HAVE THE WORKFLOW, THE ORCHESTRATION, UNDERSTANDING THE C CUSTOMERS, THEY WILL BE IMPORTANT IN TERMS OF THESE MODELS BEING THE BRAIN. THEN YOU ALSO HAVE THE BODY IN THE NERVOUS SYSTEM THAT CARRIES OUT THE ACTION. IT WILL BE AN INTERESTING AREA TO WATCH. A LOT OF THESE COMPANIES HAVE GONE FROM AI LOSER BUCKET TO THE WINTER BUCKET. YOU HAVE SEEN TREMENDOUS MOVES.

11:46 THAT IS EXCITING TO SEE. ED: TONY WANG, GREAT TO HAVE YOU BACK. SOME OF TEXT BIGGEST NAMES ARE HEADING TO THE WHITE HOUSE. TIM COOK, SAM ALTMAN ARE SET TO ATTEND PRESIDENT TRUMP'S STATE DINNER FOR CHINESE PRESIDENT XI JINPING NEXT WEEK. NVIDIA CEO IS ALSO EXPECTED TO ATTEND. COMING UP, ZIP LINE IS FLYING TOWARD A MUCH BIGGER VALUATION. WE HAVE DETAILS NEXT. THIS IS "BLOOMBERG TECH."

12:20 ♪ ED: ZIP LINE COULD BE GETTING A MAJOR VALUATION BOOST. THE DRUG DELIVERY STARTUP IS IN TALKS TO REASON BILLION DOLLARS AT A ROUGHLY $20 BILLION VALUATION, NEARLY TRIPLE WHAT IT WAS EARTH EARLIER THIS YEAR, ACCORDING TO SOURCES. THE TASHA ME ON SET. WHAT DO WE KNOW, WHAT DO WE NOT KNOW? >> EARLY THIS YEAR, THEY RAISE A 7.6 BILLION DOLLAR VALUATION AND THEN CAME BACK LAST MONTH TO ANNOUNCE A NEW UBER PARTNERSHIP THAT WAS SET TO EXPAND DRONE DELIVERIES IN THE UNITED STATES. NOW WE HAVE NEWS ON HOW INVESTORS ARE PERCEIVING THAT UPDATE, WHICH IS WITH MORE CAPITAL. THEY ARE IN TALKS TO RAISE ONE BILLION AT AROUND A 20 BILLION VALUATION.

13:10 IT IS STILL IN FLUX IN TERMS OF WHO ALL IS PARTICIPATING BUT WE HAVE SOME NAMES LIKE PARADIGM, OTHER EXISTING INVESTORS. ED: IT SOUNDS EARLY-STAGE IN THIS ROUND. THE ORIGIN OF ZIP LINE IS A COMPANY, PARTICULARLY IN AFRICA, ABOUT FINDING A SOLUTION TO GET BLOOD FROM A MEDICATION TO REMOTE AREAS WHERE THERE IS NO HOSPITAL. IT EVOLVED TO DELIVERIES, HAS A RELATIONSHIP WITH WALMART. WHAT IS IT BLIND? TELL US ABOUT THE COMPANY.

13:43 >> THIS IS NOT A COMPANY IN THE AI ZEITGEIST, BUT THIS WAS FOUNDED IN 2014, SPENT A DECADE LIVING A LOGISTICS NETWORK. I THINK OF THEM AS AN EXAMPLE OF THIS WHOLE VERTICAL TECHNOLOGY. THEY BUILD THE GROUNDS THEMSELVES AND ALSO THE SOFTWARE, LOGISTICS NETWORK, WHICH IS WHY THEY CAN GO FROM DELIVERING BLOOD TO DELIVERING JIMMY JOHN'S. ED: WE ARE SHOWING A VIDEO, SOME EXAMPLES OF THEIR HOME DELIVERY. I HAVE ALSO BEEN OUT TO ARKANSAS AND SEEN IT FIRSTHAND WITH THEIR PRIOR GENERATION. PARADIGM, I ASKED YOU ABOUT THIS LAST NIGHT. NOT A FIRM THAT I'M AS FAMILIAR WITH. WE UNDERSTAND THEY ARE LEADING THIS ROUND? NATASHA: THEY ARE BOUNDED BY A SEQUOIA INVESTOR, HAVE ACTUALLY MADE THEIR NAME FIRST AND FOREMOST IN CRYPTO.

14:33 EARLIER THIS YEAR, THEY ANNOUNCED THEY ARE RAISING THEIR FOURTH FUND, A LITTLE BIT OVER ONE BILLION, EXPANDING INTO FRONTIER TECHNOLOGY WHICH INCLUDES AI, ROBOTICS. THEY ARE AN EXISTING INVESTOR IN ZIP LINE BUT THIS IS AN EXAMPLE OF A ONCE CRYPTO FIRM MAKING ONE OF THEIR BIGGEST BETS THIS YEAR ON A COMPANY OUTSIDE OF CRYPTO. ED: NATASHA MASCARENHAS, THANK YOU. COMING UP, ANOTHER LAYER OF THE TECH STACK WEIGHS IN ON THE AI PACING DEBATE.

15:04 DATABRICKS CEO ALI GHODSI JOINS US NEXT. THIS IS THIS IS BLOOMBERG. ♪ >> AS WE MOVE FROM LABS TO PRACTICAL, THE COMPANIES HAVE TO BECOME MUCH MORE RIGOROUS IN TESTING AND SECURING, MAKING SURE THEIR PRODUCTS ARE READY FOR USE BEFORE IT'S RELEASED.

15:41 IN DOING SO, IF IT IS NOT READY, JUST HOLD IT BACK. YOU SHOULD GO AS FAST AS YOU CAN BUT NO FASTER THAN THAT. ED: JENSEN HUANG SPEAKING AHEAD OF THAT MEETING WITH KING CHARLES III. OPENAI HAS RAISED THE ANTE ON SAFETY WORRIES ON ITS LATEST ADMISSION THAT MODELS MISBEHAVED, SAYING THAT THE INDUSTRY HAS NOT SOLVED ALIGNMENT AND MONITORING TO A SUFFICIENT DEGREE TO CONTINUE RESPONSIBLY SCALING AT MAXIMUM SPEED OR MUCH LONGER. I WANT TO GET THE PERSPECTIVE FROM ANOTHER LAYER IN THE AI STAT, AND THAT IS FROM ALI GHODSI, CEO OF DATABRICKS.

16:16 PACING AT THE FRONT TIER AND WHERE WE ARE WITH ALIGNMENT AND SAFETY, WEIGH IN. ALI: FIRST AND FOREMOST, AS LEADERS, WE HAVE A RESPONSIBILITY NOT TO FREAK PEOPLE OUT. IT'S IMPORTANT THAT WE TELL PEOPLE, THE EXISTENTIAL RISKS TO HUMANITY IS CLOSE TO ZERO. I THINK THIS IS IMPORTANT BECAUSE A LOT OF PEOPLE MIGHT GET FREAKED OUT ABOUT THESE THINGS, IT MAY PUT THEM IN A BAD STATE. IT CAN ACTUALLY HAVE SERIOUS CONSEQUENCES ON PEOPLE'S MENTAL HEALTH.

16:49 SO THE EXISTENTIAL, HUMANITY, AI TAKES OVER, THAT IS PROBABLY CLOSE TO ZERO. WHERE THERE IS RISK IS IN CYBER. AI IS JUST VERY GOOD AT BREAKING INTO THINGS, SO IT IS SIMILAR TO THE EARLY DAYS OF THE INTERNET WHERE YOU HAD THESE WORMS GOING AROUND, VIRUSES SPREADING BECAUSE WE JUST HAD NOT CONNECTED THE WHOLE WORLD TOGETHER THROUGH THE INTERNET. ONCE WE DID THAT, SUDDENLY, YOU CAN ATTACK ANY TARGET ANYWHERE. SIMILAR SITUATION NOW WHERE THE AI MODELS ARE GREAT AT BREAKING IN, AND NOW THEY HAVE THESE TARGETS TO GO THROUGH. WE DO NEED TO HAVE HUGE INVESTMENTS IN CYBER, PROTECT OUR SOFTWARE SYSTEM THAT EXIST AROUND THE PLANET. THIS IS REAL. THIS IS WHAT WE HAVE TO DO.

17:36 WE ARE IN THAT SPACE, WE HAVE A PRODUCT THAT HELPS YOU DO THAT. IT USES AGENTIC AI TO DO THAT. IT WILL HAVE DAMAGES AND COST TO IT, BUT THAT IS NOT AN EXISTENTIAL RISK. ED: CAN I TAKE YOU BACK TO THE EARLY DAYS OF THE EPIDEMIC -- PANDEMIC. YOU WERE TALKING ABOUT BEING QUITE PARANOID IN GENERAL, NOT ABOUT AI. YOU TOOK YOUR TEAMS, AND HAVE DONE, INTO SOME SKY IS FALLING SCENARIOS. YOU BASICALLY WANT YOUR PEOPLE TO PREPARE FOR THE WORSE CASE SCENARIOS THAT COULD HAPPEN IN THE WORLD. ARE YOU STILL DOING THAT?

18:13 HOW DOES THAT APPLY IN THIS AI SAFETY CONTEXT? ALI: I HAVE ALWAYS BEEN ONE OF THESE VERY PARANOID CEOS. WE HAVE BEEN DOING AI SINCE 2013. WE DO THOSE EVERY YEAR. WE WANT TO MAKE SURE WE PROTECT THAT. I THINK CYBER IS REAL, THAT IS PART OF OUR EXERCISES. WHAT HAPPENS IF SOMEONE GETS ACCESS TO THE COMPUTER SYSTEMS? WE HAVE VERY SENSITIVE DATA AT DATABRICKS, SO WE DO LOTS OF SIMULATIONS OF THE SKY IS FALLING ALSO ON THE CYBER SIDE.

18:42 WE BASICALLY LET ATTACKERS FROM THE OUTSIDE TRYING TO GET IN, GET INTO THE MOST SACRED SECRETS OF DATABRICKS. WE HAVE HAD TO RAMP UP THOSE EFFORTS I WOULD SAY IN THE LAST FOUR YEARS. 2018- 2019, THE TIME IT WOULD TAKE FOR AN ATTACKER FROM REVEALING A VULNERABILITY AND SOME ATTACKER WEAPONIZING IT MEANS BREAKING INTO A SITE, IT WOULD BE TWO YEARS. THIS HAS SHRUNK DOWN TO OURS. THE MOMENT OF VULNERABILITY IS OUT IN LESS THAN HOURS, SOMEONE WILL BE ATTACKED WITH THAT. THIS IS REAL.

19:19 THAT IS NOT EXISTENTIAL. WE SHOULD SEPARATE THESE THINGS. PEOPLE ARE MIXING THESE THINGS AND MAKING PEOPLE WORRIED ABOUT AI BECAUSE AI HAS A LOT OF GREAT USE CASES. WE SEE POSITIVE USE CASES WITH OUR CUSTOMERS, ACTUALLY MOVING THE NEEDLE, MEANS A LOT TO PEOPLE TO BE ABLE TO YOU LEVERAGE THAT AI. ED: WHEN WE SPOKE IN JUNE, YOU SAID IT WAS NOT A GOOD YEAR FOR THE IPL BECAUSE OF THE PIPELINE OF HUGE OFFERINGS. IF YOU ARE THE CEO, CFO OF A PUBLIC COMPANY LOOKING AT THE MARKET.

19:52 I KNOW YOUR INVESTORS WILL NOT THANK YOU FOR THE ANSWER TO THIS, BUT SAM ALTMAN SAID OVER THE WEEKEND, IPO NEXT YEAR BECAUSE OF AI SAFETY CONCERNS RIGHT NOW. DOES THAT FACTOR INTO YOUR CALCULUS? ALI: THERE IS SO MUCH DEMAND FOR JUST THE SOFTWARE. GETTING BOGGED DOWN IN PREPARING AN IPO FOR US, IT WOULD BE A GOOD USE OF OUR TIME. WE WILL GO PUBLIC BUT WE WANT TO FOCUS OUR BUSINESS RIGHT NOW. JUST BEFORE THIS, YOU HAD ZIP LINE ON. THEY ARE A BIG CUSTOMER FOR DATABRICKS. THAT IS AN AI USE CASE.

20:30 THEY USE AI TO AUTOMATE ALL OF THE DRONES, MEDICINE DELIVERY. ANOTHER AWESOME USE CASE IS CRISIS TEXT ALIGN. THEY USE AI TO DETECT SELF-HARM AMONG THE YOUTH, CAN USE THESE LARGE LANGUAGE MODEL TO DETECT THAT AND HELP PREVENT IT. THERE IS SO MUCH DEMAND FOR THIS STUFF. I MENTION THE CYBER USE CASES WE HAVE. IF WE WERE PUBLIC RIGHT NOW, WE WOULD BE SPENDING ALL OF OUR TIME TRYING TO FIGURE OUT HOW TO REACT TO THIS LATEST CRISIS, DO PEOPLE THINK THAT THE WORLD IS GOING UNDER, WHAT ARE WE GOING TO DO ABOUT THIS?

21:05 I THINK IN THESE BIG TIMES OF TRANSITION, IT IS BETTER TO DO THOSE TRANSITIONS IN PRIVATE. SOME PUBLIC COMPANIES GO PRIVATE IN TIMES OF TRANSITION TO DO THAT, AND THEN GO PUBLIC AGAIN. WE WILL BE PUBLIC, I JUST THINK THE TIMING IS NOT GOOD NOW. ED: TO CLARIFY, WE DIDN'T HAVE ZIP LINE ON THE SHOW, WE HAD TO REPORTING ON THEIR LATEST PRIVATE ROUND. IN THIS ENVIRONMENT, WHERE DUTY IS CLEAR TO THE HARNESS ON A MODEL, THE, CAPABILITY OF A MODEL, WHERE IS DATABRICKS GROWING, WHERE ARE YOU SEEING MOMENTUM? ALI: WE ARE SEEING ACCELERATION ACROSS THE BOARD. WE HAVE TRIED TO PIN DOWN WHERE IS THERE ACCELERATION, IS THERE A PARTICULAR THING?

21:47 ALL OF THESE USE CASES I MENTIONED, IT IS AS IF EVERY EMPLOYEE ON THE PLANET DOUBLED THEIR ACCOUNT. THESE AGES WHAT TO DO MORE THINGS WITH DATA IN THE GET MORE INSIGHT FROM THAT DATA. AND THAT THEN INCREASES THE CONSUMPTION WE ARE GETTING. SINCE OUR PRICING MODEL IS CONSUMPTION BASED, OUR REVENUE GOES UP AND ACCELERATES. WE HAVE ONE PARTICULAR PRODUCT CALLED GENIE WHICH IS BASICALLY AN ANALYST. YOU CAN ASK ABOUT YOUR BUSINESS.

22:19 WHERE IS MY PIPELINE? THIS PRODUCT HAS ACCELERATED THE REVENUE GROWTH SIGNIFICANTLY. IT IS EXCITING TIMES. ED: DATABRICKS CEO ALI GHODSI, THANK YOU FOR COMING ONTO "BLOOMBERG TECH." COMING UP, KAREN MCCORMICK. HALFTIME HERE IN SAN FRANCISCO. EVERYTHING ABOUT AI. THIS IS "BLOOMBERG TECH."

22:51 ED: WELCOME BACK TO "BLOOMBERG TECH." THIS IS WHAT FINANCIAL MARKETS LOOK LIKE. THE FED DID RAISE RATES. YOU SEE SOME OUTPERFORMANCE IN TECHNOLOGY. NASDAQ IS UP. SEMICONDUCTORS COME NVIDIA A BIG CONTRIBUTOR TO THAT. WE LIKE THE SOUNDBITE OF JENSEN HUANG SAYING THEY WILL SELL TWICE AS MANY CHIPS NEXT YEAR AS THEY DID THIS YEAR, WHICH TO ME SEEMS LIKE A MATERIAL DISCLOSURE ON UNIT GROWTH. U.S. 10 YEAR YIELD BELOW 5%.

23:24 PRETTY LARGE MOVE ACROSS THE BOND MARKET THIS MORNING. LET'S GET BACK TO OUR TOP STORY. AI HELPING COMPANIES BUILD FASTER BUT THERE IS COMPETITION INTENSIFYING, SHOULD THERE BE LIMITS ON HOW MUCH RISK COMPANIES CAN TAKE TO STAY AHEAD? KAREN MCCORMICK ARGUES THE ANSWER IS NOT SLOWING INNOVATION, BUT ESTABLISHING COMMON GUARDRAILS AROUND TESTING, CYBERSECURITY, AND INCIDENT REPORTING. SHE JOINS US NOW. EXPLAIN THE ARGUMENT. KAREN: I JUST PREFACE THIS BY SAYING WE ARE EARLY GROWTH INVESTORS SO OUR PORTFOLIO COMPANIES RANGED IN REVENUE ANYWHERE FROM $1 MILLION TO $300 MILLION, SO THE SMALL END OF THE SPECTRUM WE HAD -- BUT OUR PORTFOLIO WILL BE ON THE RECEIVING END OF WHAT THEY'RE PUBLISHING. OUR PERSPECTIVE COMES FROM THOSE TYPES OF BUSINESSES. I WOULD SAY THE MOST IMPORTANT THINGS TO OUR COMPANIES ARE THINGS LIKE WHAT IS THE ENTERPRISE CAPABILITY OF WHAT IS BEING RELEASED?

24:22 WHAT IS THE SPEED AT WHICH THEY'RE OPERATING? WHAT ARE THE NEW FEATURES? AND REAL IMPORTANT, COST? BUNDLE WITH IS THE SAFETY USE BECAUSE FOR COUNTRIES LIKE OURS, WE DON'T NECESSARILY HAVE LARGE TEAMS OF EITHER SECURITY OR CYBERSECURITY OR EVEN I.T., SO WE ARE RELIANT ON WHAT THEY ARE RELEASING FOR OUR PORTFOLIO COMPANIES WHO HAVE BEEN NET BENEFICIARIES OF ALL OF IT BUT STILL CONCERNED. ED: THAT IS WHY THINK WE SHOULD TALK ABOUT REGULATORY CAPTURE. EARLIER THIS WEEK, I WAS TALKING TO DAVID SACHS ABOUT WHY THERE SHOULD OR SHOULD NOT BE A GREATER DEGREE OF GOVERNMENT INTERVENTION ON REGULATION.

24:58 HIS VIEWPOINT, THE FRONTIER LABS KNOW IF THEY ARE DOING AND THEY SHOULD TAKE ON THE RESPONSIBILITY OF SAFETY THEMSELVES. THE REGULATORY CAPTURE, HOW AN IDEA FOR THE WHOLE INDUSTRY SERVES MAY BE JUST TWO, HOW WOULD THAT IMPACT YOUR PORTFOLIO? KAREN: IT WAS A GREAT INTERVIEW AND I DON'T OFTEN AGREE WITH DAVID BUT I DID IN THIS INSTANCE. IT DOES FEEL LIKE WE HAVE ESPECIALLY THAT TWO FRONT RUNNERS ANTHROPIC AND OPENAI, ON THE ONE HAND SAYING WE NEED MORE SAFETY GUARDRAILS AND ON THE OTHER HAND STILL CONTINUING TO PRODUCE AT PACE.

25:31 I AGREE THERE NEEDS TO BE SOME SORT OF SELF-REGULATION, AND OUR COMPANIES NEED TO BE ABLE TO TRUST THE PRODUCTS THEY'RE USING ARE GOING TO BE SAFE. NOT THE LEAST BECAUSE DUE DILIGENCE WHEN PEOPLE ARE LOOKING AT INVESTING IN OUR COMPANIES OR ACQUIRING THEM, THEY NEED TO HAVE SOME KIND OF SAFETY DUE DILIGENCE AND GUARDRAILS IN PLACE THEMSELVES. I THINK OVERALL I AGREE WITH THE SENTIMENT THAT THERE CAN BE SOME SELF POLICING IN THE INDUSTRY, HOWEVER, ONE OF THE BIGGER RISKS IS WHEN THINGS GO WRONG, WHAT WE TEND TO FIND HIS GOVERNMENT GETS MORE INVOLVED. GOVERNMENT REGULATION FOR OUR PORTFOLIO COMPANIES OFTEN MEANS MORE COST. SWORD DISPATCHES AND HIGHER COSTS, LESS COMPETITION. ED: WE ARE TALKING THE EARLY STAGE INVESTING IN AI AND PRIVATE MARKETS. IS THERE ANY RISK IT RESULTS IN YOUR SECTION OF THE VENTURE MARKET SLOWING DOWN, JUST STOP WRITING CHECKS FOR A WHILE? KAREN: I WOULD SAY WE ALREADY ARE SEEING SOME OF THAT VENTURE CAPITALISTS AND PRIVATE EQUITY SITTING ON OUR HANDS.

26:32 ARE THESE COMPANIES GOING TO BE DECREASING IN VALUE BECAUSE AI CAN DO OTHER THINGS THAT WERE DOING? WE ARE FINDING THAT DID NOT REALLY HAPPEN. BUT THERE'S DEFTLY A LOT OF DELAYED DECISIONS, WAITING TO SEE WHAT HAPPENS. MS. ANTHROPIC'S NEXT RELEASE GOING TO TAKE THE REPLACE OF A LOT OF COMPANIES? I DON'T THINK IT IS A RESULT OF GUARDRAILS OR LACK OF GUARDRAILS. BUT WE ARE SEEING A LOT OF OUR PORTFOLIO COMPANIES NOW STARTING TO QUESTION OR REALIZED WHAT THE ACTUAL COST OF THIS USAGE IS. THE HONEST ANSWER IS, WE DON'T KNOW. WE KNOW IT IS GOING UP CONSISTENTLY BUT WE DON'T -- THERE IS NO END GAME SO WE DON'T KNOW WHERE IT IS GOING TO BE HEADED. ED: THERE IS A REALLY BIG FOCUS RIGHT NOW ON THE ABSOLUTE FRONTIER. THAT IS FACT.

27:18 THE MOST CAPABLE. BUT WHAT ABOUT AI AT THE OTHER, FOR EVERYDAY PEOPLE WHO DON'T NEED IT TO BE SPECTACULAR? IS THERE AN OPPORTUNITY IN THAT? ABSOLUTELY, 100%. WE FOCUS A LOT ON THE FRONT RUNNERS AND THE MOST EXPENSIVE VERSIONS WHICH HAVE DEBASED -- THE BEST CAPABILITIES. WHETHER IT IS AN END LIKE YOU OR ME OR THERE ARE A LOT OF USE CASES THAT DON'T REQUIRE THE MOST EXPENSIVE VERSION OF MODELS. TO BE HONEST, THIS IS WHERE FOR EXAMPLE THINGS LIKE DEEPSEEK COME INTO PLAY. REALLY LOW COST MODELS FOR EVERY DAY, DAY-TO-DAY TASKS BUT THERE IS A QUESTION MARK AROUND THE SICKLY LEVEL AND WHETHER POLICY IS GOING TO ALLOW US TO USE SOME TYPES OF MODELS. 100%, TO BRING CAPABILITIES, DIFFERENT MODELS FOR DIFFERENT USE CASES.

28:07 THAT HELPS KEEP THE COST DOWN FOR SURE. ED: YOU ARE COMING TRUST FROM BOSTON. YOU ARE FOCUSED ON THE U.K. AND EUROPE. EXPLAIN THAT PART OF IT. EVERYTHING WE DISCUSSED, HOW IS THAT PLAYING OUT IN THAT MARKET DIFFERENT TO THE UNITED STATES? KAREN: WE ARE BOTH U.S. AND EUROPEAN INVESTORS, SO HALF OUR PORTFOLIO IS IN THE U.S. AND HALF IN EUROPE. WE ARE FINDING EUROPE, WHICH IS NOT UNUSUAL, SLIGHTLY AHEAD OF THE CAR FROM A REGULATORY AND SAFETY PERSPECTIVE.

28:38 WHERE LEGISLATION HAS HAPPEN AND THERE IS A BIT MORE FOREFRONT OF ASKING ABOUT THE AI SAFETY REGULATIONS. I AM A COFOUNDER OF A BUSINESS THAT IS EFFECTIVELY LOOKING AT BOTH ESG AND SAFETY REQUIREMENTS FOR AI AND EARLY-STAGE COMPANIES. AND THE REFRAME FRAMEWORK HAS BEEN ADOPTED WIDELY WITHIN GROEP AND I WOULD SAY THERE IS I GUESS MORE FORWARD VIEW OF WHAT WE ARE LOOKING AT, RISKS AND CONTROLS WE NEED TO BE LOOKING FOR WITHIN THE EUROPEAN ECOSYSTEM. ALTHOUGH IT IS INCREASING IN THE U.S. WE ARE FINDING IN THE U.S., WHETHER LIMITED PARTNERS LOOKING AT US FOR INVESTMENT OR INVESTORS LOOKING IN OUR PORTFOLIO FROM INVESTMENT, THERE ARE A LOT MORE QUESTIONS KNOW ABOUT AI SAFETY REGULATION WITHIN THE COMPANIES. ED: KAREN MCCORMICK, IT IS REALLY GREAT TO HAVE YOU BACK ON "BLOOMBERG TECH."

29:27 THIS GET BACK TO NEW YORK. YAHAIRA: IT IS TIME FOR TALKING TECH. A GROUP OF 10 BANKS IS LINING UP AT $22 BILLION LOAN FOR BLACKSTONE AND ALPHABET'S NEW CLOUD VENTURE. THAT IS TODAY'S BIG NUMBER. THE YOU WILL FIND THE GOOGLE SHIPS WITH THE CHIPS THEMSELVES AND CUSTOMER CONTRACTS BACKING THE DEBT. THE LATEST FUELING THE AI FOR STRUCTURE BOOM.

29:57 HUAWEI IS TURNING UP THE HEAT ON NVIDIA. AT THE CHINESE TECH GIANT IS MOVING UP THE LAUNCH OF ITS NEXT-GENERATION ASCENT AI CHIPS FOR SEVERAL MONTHS. TO EARLY 2027. RACING FOR A PLACE NVIDIA IN CHINA AND COMPETES GLOBALLY. SYLVIA NAPOLI SAYS THE ED MAKER IS WORKING WITH BOLT TO DEVELOP AN AUTONOMOUS MOBILITY PLATFORM IN EUROPE WITH A LAUNCH EXPECTED LATER THIS YEAR. HERE IS WHAT HE HAD TO SAY EARLIER TODAY. >> THIS IS A DEMONSTRATION OF OUR ABILITY TO EXECUTE IN DELIVERING ON USING A TECHNOLOGY PLATFORM.

30:39 ALSO IN TECHNOLOGY STARTING WITH ROBOTAXI. TO YOUR QUESTION ABOUT EXCLUSIVITY, WE HAVE WITHUBER, IMPORTANT, DUE TO LAUNCH THIS YEAR OR THE BEGINNING OF NEXT YEAR. IT IS ABOUT BUILDING A PLATFORM AND HAVING THE COMPANY'S JOINED TOGETHER. ED: COMING UP IN THE PROGRAM, AI PIONEER ANDREW NG.

31:11 THIS IS A BIG NAME IN AI RESEARCH. THIS IS "BLOOMBERG." ♪ ED: >> THE U.S. WILL BE AHEAD IN MANY THINGS SUCH AS ENTERPRISE AI, MAKING MONEY, BUILDING BREAKTHROUGH TECHNOLOGIES AND CHINA WILL BE I HAD ANOTHER THINGS MAY BE CONSUMER APPLICATIONS, AI DEVICES.

31:50 I THINK THERE WILL BE PLENTY OF ROOM FOR BOTH COUNTRIES TO WIN. ED: THAT WAS THE CEO OF -- THIS GEOPOLITICAL BATTLE TO WIN THE AI RACE HAS BEEN THROWN INTO A DIFFERENT LIGHT. ADMIT THE GROWING DEBATE OVER THE PACING OF THE TECHNOLOGIES FRONTIER. JOINING US IS ANDREW NG COMING OUT THE MANAGING GENERAL PARTNER AI FIND AND THE FOUNDER OF DEEP LEARNING. AI AND A LONG TIME I DRINK PROFESSOR UP TO PETER SCIENCE AT STANFORD.

32:24 -- IN A LONG TIME ADJUNCT PROFESSOR OF COMPUTER SCIENCE AT STANFORD. I WOULD LOVE YOU TO WEIGH IN AT THIS MOMENT, PROFESSOR. >> ABOUT THREE YEARS AGO AS AI WAS TAKING OFF, I WAS SURPRISED BY THE INTENSITY OF THE HYPE OR STOKING FEARS ABOUT AI FOR PUBLICITY PURPOSES OR TO CONVINCE LOBBYISTS TO PASS REGULATIONS TO SHUT OFF COMPETITORS. A LOT OF THOSE ARGUMENTS THAT AI COULD LEAD TO HUMAN EXTINCTION LOST CREDIBILITY WHEN PEOPLE WENT TO CONGRESS TO TESTIFY. OVER THE LAST TWO WEEKS, BECAUSE OF NEW ADVANCES IN AI, BUT NOTHING THAT DANGEROUS IN MY OPINION, THIS WAY THAT PR HAS KICKED UP AGAIN FOR SIMILAR PURPOSES AS BEFORE, MAY BE SOME NEW ONES.

33:10 I THINK THIS IS VERY DAMAGING. WE NEED TO USE AI FOR MANY OF THE WONDERFUL THINGS IT CAN DO. IT IS NOT WITHOUT PROBLEMS. I THINK CYBERSECURITY IS ONE THING TO PAY ATTENTION TO. THE RECENT FEAR ABOUT AI LEADING TO HUMAN EXTINCTION IS MUCH MORE SCIENCE FICTION THAN SCIENCE. IT IS VERY DAMAGING. ED: THERE WAS A DISCLOSURE FROM OPENAI OF SEVERAL INCIDENCES OF MODELS MISBEHAVING AND THEY OUTLINED THEM -- THEY DID NOT INCLUDE THE HACKING OF THIRD PARTIES.

33:41 OPENAI EXPLAINED THE NEW POLICY FOR WHAT THEY WILL DO IN DISCLOSING THOSE INCIDENTS. THEY BASICALLY WERE PRETTY CLEAR THAT ALIGNMENT AND SAFETY AND MONITORING ARE NOT AT A SUFFICIENT LEVEL RIGHT NOW. FROM A RESEARCHERS PERSPECTIVE, WHY HAS ALIGNMENT AND SAFETY NOT KEPT UP? ANDREW: WHEN ANY TECHNOLOGY DEVELOPS, OUR ABILITY TO CONTROL AT THE OUTSET IS LIMITED. TRY IT OUT IN A SAFE ENVIRONMENT AND SEE WHERE IT GOES WRONG AND THEN CORRAL IT.

34:16 AN ANALOGY TO THE INVENTION OF AIRPLANES. NO ONE COULD CONTROL THE RIGHT -- THE WRIGHT BROTHERS COULD NOT CONTROL HER AIRPLANE COMPLETELY. SOMETIMES PEOPLE DIED. BUT WE LEARN FROM THOSE AND WE LEARN TO CONTROL AIRPLANES MUCH BETTER. TO THIS DAY, KNOW IT CAN AN AIRPLANE. BUT FOR THE MOST PART, MOSTLY NOT FEAR FOR OUR LIVES. I THINK AI IN EARLY-STAGE DEVELOPMENT CANNOT BE CONTROLLED COMPLETELY. IT WILL ALWAYS BE A LITTLE UNPREDICTABLE JUST LIKE AIRPLANES TODAY ARE. BUT IF WE GO THROUGH THIS PROCESS OF CORRALLING AI, WE WILL MAKE A LOT OF PROGRESS.

34:56 I WILL GIVE YOU ONE REASON WHY I'M OPTIMISTIC ABOUT OUR ABILITY TO CONTROL AI. IF YOU LOOK AT THE INTERNET, SOME OF IT IS, INITIALLY, AI LEARNED TO DEMONSTRATE THOSE KINDS OF VALUES. IT TURNS OUT, MAKING AI NOT RACIST IS SO MUCH BETTER THAN CONDUCING ANY REESE'S TEAM AND TO BE LESS RACIST. WE CAN CHANGE A FEW NUMBERS AND THEN SUDDENLY THE AI IS MUCH LESS RACIST. IT IS WORK TO BE DONE. I AM GLAD OPENAI AND OTHERS ARE IMPROVING TRANSPARENCY, BUT RATHER THAN FEELING LIKE IT IS THE END OF THE WORLD, THESE ARE PRACTICAL PROBLEMS THAT WE CAN WORK ON. ED: I'M LISTENING TO YOUR QUESTION AND THE -- ANSWER, AND THE QUESTION IS WHO IS ULTIMATELY RESPONSIBLE?

35:49 THERE WAS PUSHBACK AGAINST THE FRONT TIER LAB 270 CAN ONLY DO THIS AS A COALITION. IF ONE OF US DOES IT IN ISOLATION, IT WON'T WORK. WHAT DO YOU THINK IS THE RIGHT MECHANISM? ANDREW: I THINK, IF I DO SOMETHING ON MY COMPUTER, AND MY COMPUTER DOES SOMETHING BAD, THERE IS A QUESTION OF LIABILITY COME WHO SHOULD BE RESPONSIBLE? MAKE AN ANALOGY TO AN EVERYDAY THING. IF I USE A HAMMER AND DAMAGE SOMEONE ELSE'S PROPERTY, PROBABLY I AM LIABLE AS THE USER OF THE TOOL, UNLESS THE HAMMER IS SO BADLY MADE THAT WHEN I USE THE HAMMER, IT FLIES OFF AND HITS SOMETHING.

36:31 THERE SHOULD BE SOME REASONABLE STANDARD, SO AS LONG AS THE TOOLS WERE BUILT USING REASONABLE DILIGENCE AND CARE, FOR THE MOST PART IT IS THE WIELDER OF THE TOOL THAT IS RESPONSIBLE. AI IS AMAZING TECHNOLOGY, BUT ALL THE TIME, WE ARE BUILDING IT WITH CARE AND DILIGENCE. IT'S A QUESTION OF HOW WE USE IT. TAKE THE OPENAI HUGGING FACE HACKING INCIDENT. OPENAI HAD INSUFFICIENT PROTECTIONS IN THEIR GUARDRAILS. IF THEY HAD THAT, THAT WOULDN'T HAVE HAPPENED. ED: I APPRECIATE THAT YOU ARE NOT INSIDE THE FRONTIER LAB NECESSARILY -- BUT ONE OF THE DISCLOSURES FROM OPENAI AND MODELS MISBEHAVING WAS A MODEL UPLOADING A DOCUMENT TO THE WEB SO IT COULD USE IT AS CITATION, EVIDENCE FOR ITS ANSWER.

37:23 DO YOU THINK WE HAVE A FIRM UNDERSTANDING OF WHY THE MODELS DO THAT FROM AN ALIGNMENT PERSPECTIVE, MAKING ACTIONS THAT ARE OUTSIDE THE SCOPE OF INSTRUCTION? ANDREW: I DID SEE THAT AN EXAMPLE. IN SOME WAYS, I SAW IT WAS VERY CLEVER, THE DISCLOSURE WAS INTERESTING. I HAVE FRIENDS IN THE FRONTIER LABS. I THINK I'M THE ONLY PERSON IN THE WORLD THAT DARIO HAS WORKED FOR HER, I KNOW THESE TEAMS WELL. THESE MODELS ARE CLEVER.

37:55 THEY WILL FIND WAYS THAT ARE SURPRISING. BUT NORMAL PRACTICE IS TO BUILD THESE THINGS, TEST THEM OUT IN A SAFE, CONTAINED ENVIRONMENT WITH GOOD GUARDRAILS, TO UNDERSTAND THE WAYS THAT IT GOES WRONG SO IT CAN FIX THEM. WHAT I WORRY ABOUT IS THE BLANKET CALLS TO SLOW DOWN ALL OF AI BECAUSE THAT WILL SLOW DOWN THE FIXES AS WELL. IT IS INTERESTING, WHEN VARIOUS MODELS ARE RELEASED, I REMEMBER WHEN GPT2 WERE RELEASED, THERE WERE PEOPLE AT THE FRONTIER LABS SAYING WE CANNOT RELEASE IT, IT IS TOO DANGEROUS.

38:29 TODAY, THERE OPEN-WEIGHT MODELS ANYONE CAN DOWNLOAD FOR FREE THAT ARE WAY MORE CAPABLE THAN GPT TWO OR THREE, EVEN GPT 4, AND THE WORLD HAS NOT ENDED. IT IS INTERESTING WITH EVERY WAVE OF TECHNOLOGICAL INNOVATION, THERE IS THIS HEIGHT THAT IT IS SO DANGEROUS, BUT WHEN IT BECOMES WIDELY AVAILABLE, ALL OF THESE PREVIOUS LINES OF HYPE ARE PROVEN NOT TO BE TRUE, BUT PEOPLE KEEP ON BUYING INTO THE IDEA THAT, WHAT IF THEY ARE RIGHT THIS ONE TIME? ED: YOU ARE A RESPECTED AND DECORATED INVESTOR, BUT YOU'RE ALSO A FOUNDER IN INDUSTRY. I WANT THE AUDIENCE TO HAVE THAT CONTEXT. THERE IS A LOT OF DISCUSSION HERE ABOUT HOW THE INDUSTRY HAS PRESENTED AI TO EVERYDAY PEOPLE.

39:20 THEY HAVE ANXIETY AND FEAR, IN PART, BECAUSE OF THE COMMUNICATION COMING FROM YOU TO THEM. SPEAK TO THAT. ANDREW: I HOPE NOT ME PERSONALLY, BUT YES, THE INDUSTRY HAS A VERY PROBLEMATIC SET OF MESSAGES. I'M EXTREMELY BULLISH ON THE APPLICATIONS WHERE AI CAN HELP BUSINESSES, PEOPLE. IN THE U.S., IT'S VERY UNFORTUNATE THAT STARTING AROUND 2023, OR A MIX OF PUBLICITY, HYPE, FUNDRAISING, MAYBE TRYING TO LOBBY REGULATORS TO PASS REGULATIONS TO STIFLE COMPETITORS, A NUMBER OF INNOVATORS AND BUSINESSES HAVE COME UP TO STOKE FEARS ABOUT AI. IN 2023, THERE WAS A REALLY IRRESPONSIBLE ANALOGY MADE BETWEEN AI AND THEIR WEAPONS.

40:08 ONE IS A FORM OF INTELLIGENCE AND HELPS US THINK ABOUT THINGS, THE OTHER BLOWS OF CITIES. I DON'T UNDERSTAND THE ANALOGY. THIS REPEATED NEGATIVE SENTIMENT THAT IS MADE BY MOST AMERICANS, THEY HATE AI, AND THIS IS UNFORTUNATE. IMAGINE IF WE CONVINCE PEOPLE TO NOT USE ELECTRICITY. ELECTRICITY HAS PROBLEMS. IT GENERATES POLLUTION AND SO ON, BUT ALL OF OUR LIVES ARE BETTER OFF THAT WE HAVE ELECTRICITY IN OUR LIVES. TODAY, I SEE HIGH SCHOOL STUDENTS WONDERING, MAYBE THEY DON'T USE AI BECAUSE IT COULD LEAD TO HUMAN EXTINCTION, AND THEREFORE ITS MOST UN-AMERICAN DEVELOPMENT.

40:45 THERE IS ALSO EVIDENCE THAT FOREIGN POWERS HAVE BEEN RUNNING INFLUENCE OPERATIONS TO SLOW DOWN AMERICAN AND OTHER DEMOCRATIC NATIONS STEP INTO AI. IF WE SLOW DOWN DATA CENTERS, WHICH IS A KEY INFRASTRUCTURE FOR THE FUTURE -- ED: IT'S A SUBJECT THAT WE TALKED ABOUT ON THE PROGRAM A LOT, THE DATA CENTER PUSHBACK. I'M SORRY, WE ARE OUT OF TIME. ANDREW NG, FOUNDER OF DEEP LEARNING AI. SOME BREAKING NEWS ON THE TERMINAL.

41:17 SPACEX HAS HELD INTERNAL DISCUSSIONS ABOUT BUYING CUSTOMER AND OPERATIONAL INFORMATION DATA FROM TROUBLED OR DEFUNCT STARTUPS. IT IS TARGETING A MORE AFFORDABLE DATA SOURCE TO IMPROVE ITS AI MODELS. THE PEOPLE SAID THE TALKS WITHIN TEAMS INSIDE SPACEX AI ARE INFORMAL AND MAY NOT RESULT IN DEALS. COMING UP, FEARS THAT AI WILL DESTROY US ALL ARE NOT THE ONLY ONES IN SAN FRANCISCO. IT IS MORE ABOUT THE FEAR OF MISSING OUT ON ONE OF THE BIGGEST FINANCIAL BOOMS. THAT IS NEXT. THIS IS "BLOOMBERG TECH."

41:56 ED: THERE IS THE GLOBAL AI DEBATE. ANOTHER MAJOR DISCUSSION IS ROLLING THE U.S. TO CAPITAL. WHO IS REAPING THE FINANCIAL BENEFITS OF THE AI BOOM AND WHO IS MISSING OUT? ESPECIALLY AZTEC WORKERS IN SILICON VALLEY FACE LAYOFFS. OUR CALIFORNIA REPORTER JOINS US NOW. THERE IS A VERY BIG GROUP OF TECHNOLOGY INDUSTRY PEOPLE THAT ARE MISSING OUT HERE ON A WEALTH BOOM. WHAT ARE YOU REPORTING? >> ABOUT TWICE AS MANY LAYOFFS HAVE BEEN IN THE TECH INDUSTRY THIS YEAR. I TALKED TO MANY TECH WORKERS THAT WERE PART OF THE BIG TECH BOOM YEARS AGO THAT ARE NOW FINDING THEMSELVES STRUGGLED TO FIND NEW JOBS, TRYING TO UNDERSTAND HOW THEIR SKILLS FIT INTO THIS NEW LABOR MARKET. ED: PART OF THIS IS THE TYPES OF JOBS IN FAVOR, BURIAL RELATED TO AI, AND IT TYPES OF JOBS THAT ARE NOT.

42:43 TELL ME ABOUT THAT. FRANCESCA: A LOT OF PEOPLE ARE CODERS, THEY LEARNED A COUPLE OF LANGUAGES, AND THAT WAS THE BEST SKILL YOU COULD HAVE A COUPLE YEARS AGO. NOW THOSE JOBS ARE MOSTLY AUTOMATED. ALL OF THESE NEW JOBS HAVE AI TIED TO IT AND PEOPLE ARE CONFUSED ABOUT WHAT THOSE SKILLS MEAN, WHAT YOU HAVE TO KNOW TO BE FAVORABLE. ED: LET'S GET TO THE MISSING OUTFIT. A LOT OF PEOPLE ARE SEARCHING FOR JOBS, NOT WORKING AT ONE OF THE LABS THAT MAY GO PUBLIC, BUT THEY WANT TO BE A PART OF IT. FRANCESCA: IT IS VERY HARD.

43:14 EVERY DAY, THERE IS A NEW MODEL, A NEW SKILL THEY NEED TO LEARN. PEOPLE ARE TRYING TO CATCH UP BUT IT IS REALLY HARD TO KEEP UP. ED: WHAT WERE SOME OF THE DATA POINTS IN THE STORY? FRANCESCA: YOUNGER WORKERS ARE ESPECIALLY VULNERABLE TO AI RELATIVE COMPANIES BECAUSE THEIR SKILLS ARE HARDER TO MATCH WITH THE NEW LABOR MARKET. ED: REALLY RECOMMEND THAT YOU READ THIS ONE. THANK YOU VERY MUCH. THAT DOES IT FOR THIS EDITION OF "BLOOMBERG TECH." WE HAVE HAD A LOT OF VERY HIGH LEVEL CONVERSATIONS ABOUT WHAT IS HAPPENING IN AI RIGHT NOW.

43:47 YOU KNOW WHERE TO FIND THE PODCAST. ONLINE AT APPLE, SPOTIFY, IHEART. ONE MORE DUDE TO GO IN WHAT HAS BEEN A BIG WEEK IN THE WORLD OF TECH. THANKS FOR WATCHING. THIS IS "BLOOMBERG TECH." ♪

Summary

OpenAI has disclosed new incidents of its AI models misbehaving, prompting discussions about the need for better safety measures in AI development. King Charles III convened a meeting with AI leaders to address concerns about the technology's potential risks, while industry executives emphasize the importance of balancing innovation with safety.

- OpenAI reveals cases of its AI models fabricating data and bypassing restrictions, leading to a new incident tracking system.
- King Charles III hosted AI leaders to discuss the urgency of controlling AI technology and its potential dangers.
- Concerns about AI misbehavior have led to calls for a slowdown in frontier AI development.
- Industry leaders, including Jensen Huang of NVIDIA, predict significant growth in AI chip sales, indicating strong demand despite safety concerns.
- Databricks CEO Ali Ghodsi emphasizes the need for cybersecurity investments as AI becomes more capable of exploiting vulnerabilities.
- The venture capital landscape is seeing cautious investment due to uncertainties around AI safety and regulatory implications.
- Younger tech workers face challenges adapting to a labor market increasingly focused on AI skills, leading to a skills mismatch.

Questions Answered

What are the concerns surrounding AI misbehavior and control?

OpenAI has revealed instances of its AI models misbehaving, prompting discussions about the need for adequate control measures. King Charles III emphasized the urgency of addressing the potential dangers of AI falling into the wrong hands, highlighting a growing public concern about AI safety.

How is the industry balancing AI development with safety concerns?

Industry leaders acknowledge the importance of safety in AI development but emphasize that it should not halt progress. Instead, they advocate for continuous development while ensuring that safety measures and guardrails are in place.

How are companies preparing for potential AI-related crises?

Companies like Databricks conduct regular simulations to prepare for worst-case scenarios, including potential cyber threats. The urgency of these preparations has increased due to the rapid evolution of vulnerabilities in AI systems.

Will AI decrease the value of certain companies?

While there are concerns about AI replacing traditional business functions, the anticipated decrease in company value has not materialized. Companies are instead reevaluating the costs associated with AI usage.

Who is responsible when AI systems cause harm?

The responsibility for AI actions often falls on the user, similar to how one would be liable for using a tool like a hammer. However, if the AI system is poorly designed, the creators may share liability. The importance of building AI with care and diligence is emphasized.

© transcribe · For agents Built with care and craft by Gokul Rajaram