Transcript
0:00 This cute little guy is the Intel Arc Pro B50. This is its bigger brother, the Arc Pro B70. It just came out. Now, the B50 didn't need any extra power cables. It got all its power straight from the PCIe bus. Nice, simple, civilized. The B70 is a little bit less civilized. This one needs power, but it also brings 32 GB of VRAM and comes in under $1,000. And because local AI tends to turn into a hardware addiction, I'm plugging in four of them. That gives me 128 GB of total VRAM, which means I'm going to need a computer that can actually power all four, which is exactly the kind of sentence local AI makes you say out loud. Now, I already did a video on the B50 last year, and it was the most bang for the buck GPU you can get in 2025.
0:46 So, now I want to know if the B70 could do the same thing for local AI in 2026. So, I've got vLLM running Qwen 34B AWQ. Let's do concurrency of one, and let's see. >> [laughter] >> Whoa. Is there a hamster in there? That's ridiculous. Computers don't make little sounds when they print stuff out. Well, now it's actually true. What? So, prompt processing pretty much doubled here from 9,281 tokens per second with one GPU to 18,170 tokens per second, but token generation went from 72 to 52. That's a pretty nasty hit for this model. By the way, I checked, they're all running up to PCIe Gen 5, but even though per GPU bandwidth is high, 608 GB per second, the GPU-to-GPU PCIe communication is limited to 63 GB per second. And that's why on smaller models, we're not going to see an increase. We might even see a decrease, like we're seeing right now.
Summary
- The Intel Arc Pro B70 requires extra power, unlike the B50, which draws power solely from the PCIe bus.
- The B70 features 32 GB of VRAM and is priced under $1,000.
- The presenter plans to use four B70 GPUs, totaling 128 GB of VRAM, for local AI tasks.
- Initial tests show a significant increase in prompt processing speed but a decrease in token generation efficiency.
- The GPUs utilize PCIe Gen 5, providing high bandwidth per GPU, but GPU-to-GPU communication is limited to 63 GB per second.
- Performance gains may not be consistent across different model sizes, with smaller models potentially experiencing reduced performance.