Section Insights
AMD's Acquisition of Talus
What is significant about AMD's acquisition of Talus?
AMD has acquired Talus, a company that creates AI chips designed to run a single model for the chip's entire lifespan, which is a unique approach in the AI hardware space.
- Talus's AI chips can only run one model throughout their life.
- This approach contrasts with traditional methods that allow for model updates.
- AMD's acquisition indicates a strategic move towards specialized AI hardware.
Innovative Chip Design
How does Talus's chip design enhance performance?
Talus etches AI model weights directly into the silicon, which eliminates the need for memory storage and allows for extremely high processing speeds.
- The etching of weights into silicon leads to faster processing speeds.
- Talus claims their chips can achieve speeds of around 17,000 tokens per second.
- This method could potentially outperform existing GPUs significantly.
Limitations of Fixed Models
What are the drawbacks of Talus's chip design?
The main limitation is that once a model is etched into the chip, it cannot be updated or modified, requiring new silicon to be manufactured for any changes.
- Models etched into chips cannot be fine-tuned or patched.
- Manufacturing new silicon is costly, with a new mask set costing around $1.5 million.
- This inflexibility poses a risk in a rapidly evolving AI landscape.
Risks of Obsolescence
What are the implications of a model becoming obsolete?
If the model etched into the chip becomes deprecated, the chip's value could plummet, as it cannot run any other model, unlike traditional GPUs.
- A chip with a deprecated model could lose all its value.
- Traditional GPUs can still be used for different models even after depreciation.
- The risk of rapid obsolescence is a significant concern for this technology.
AMD's Strategic Gamble
What is AMD betting on with this acquisition?
AMD is betting that the AI models will remain relevant long enough to justify the investment in fixed silicon designs, despite the fast-paced changes in AI technology.
- AMD's strategy hinges on the longevity of AI models.
- The rapid evolution of AI poses a challenge to the viability of fixed models.
- This acquisition reflects a bold move in the competitive AI hardware market.
Transcript
0:00 AMD just bought a company that makes AI chips that can only ever run one model. And I don't mean one model at a time, I mean one model for the entire life of the chip. The startup is called Talus, and their idea is kind of insane. Instead of storing an AI model's weights in memory and loading them whenever you run it, Talus literally etches those weights into the silicon. The weights aren't data anymore, they're the actual wiring of it all. And that's where the ridiculous speed comes from. They're pitching around 17,000 tokens per second, and they've claimed numbers as high as 48 times an Nvidia B200. But, there's a pretty massive catch. If the model is physically wired into the chip, you just can't update it. You need to manufacture new silicon. A new mask set costs somewhere around 1.5 million dollars. There's no fine-tune, there's no patch. And this next part even grows on all this. The model they welded into that silicon, Groq, deprecated it the exact same month AMD bought the company.
1:03 Now, think about what that means. A normal GPU might depreciate over five or six years, but somebody can still buy it and run a completely different model. A chip with a dead model physically etched into it just can't do that. Its value could basically go straight to zero. So, AMD is making a pretty fascinating bet here, that an AI model can stay useful long enough to actually be worth carving into metal. And with how quickly models are changing right now, that's a pretty big bet. If you enjoy coding tips and tricks like this, be sure to subscribe to the Better Stack channel.
Summary
- Talus's AI chips can only run one model for their entire lifespan, with weights etched directly into silicon.
- The chips boast impressive performance, claiming speeds of 17,000 tokens per second and up to 48 times faster than Nvidia's B200.
- A major drawback is the inability to update the model; new silicon must be manufactured for any changes, costing around $1.5 million.
- The model used, Groq, was deprecated the same month AMD acquired Talus, highlighting potential risks in model longevity.
- Unlike traditional GPUs, which can run various models over time, these chips may lose value quickly if the etched model becomes obsolete.
- AMD's investment hinges on the assumption that a single AI model can remain relevant long enough to justify the permanent etching into silicon.
Questions Answered
What is significant about AMD's acquisition of Talus?
AMD has acquired Talus, a company that creates AI chips designed to run a single model for the chip's entire lifespan, which is a unique approach in the AI hardware space.
How does Talus's chip design enhance performance?
Talus etches AI model weights directly into the silicon, which eliminates the need for memory storage and allows for extremely high processing speeds.
What are the drawbacks of Talus's chip design?
The main limitation is that once a model is etched into the chip, it cannot be updated or modified, requiring new silicon to be manufactured for any changes.
What are the implications of a model becoming obsolete?
If the model etched into the chip becomes deprecated, the chip's value could plummet, as it cannot run any other model, unlike traditional GPUs.
What is AMD betting on with this acquisition?
AMD is betting that the AI models will remain relevant long enough to justify the investment in fixed silicon designs, despite the fast-paced changes in AI technology.