There are lot of people building systems that use old GPUs, retired repurposed datacenter hardware, and other options that are less expensive than mainstream consumer GPU hardware.
This guy got Qwen 3. 27b at 8 bit quant (that's the very slow dense version with a verse high precision quantization) running at 20 tokens per second on a GMKtec Evo T1 mini PC with 2 AMD Mi50-32gb GPUs!
https://www.youtube.com/watch?v=ll-N6IFlZvk
Here's what Gemini had to say about that system:
Building this custom hardware setup will cost approximately $1,000 to $1,250 in total. The bulk of this will be the mini PC, while the used server GPUs are highly affordable. A cost breakdown of the components:
-
GMKtec Evo T1 Mini PCPrice: $820 - $1,000 (depending on retailer deals and configuration) Details: The Evo T1 features an Intel Core Ultra 9 285H processor and a native OCuLink port.Where to buy: You can purchase it directly from the GMKtec Store or check listings on Amazon.
-
AMD Radeon Instinct MI50 (32GB) GPUsPrice: $150 - $250 per card (used/refurbished) Details: These are retired data center server GPUs. While they offer massive VRAM for the price, they were not designed for consumer desktop use. Where to find them: You can typically find these on eBay or secondary hardware markets.
-
OCuLink eGPU Docks & Cables Price: $90 - 200 Details: To run these GPUs via OCuLink, you will need an eGPU PCIe docking station (like the AD-GP1 eGPU dock) that supports external PCIe 4.0. You will also need a separate desktop power supply (PSU) to provide adequate power (around 300W} per card) to the MI50 GPUs. Where to buy: eGPU docks are available on Newegg or directly from the GMKtec Advance Series eGPU Page.
Important Technical Considerations
Dual GPU Support: Most OCuLink eGPU docks only support a single graphics card. To run two Mi50s, you would need to modify your setup using multi-port PCIe switch boards, which can add $150 - $300 in hardware complexity.
Lack of Display Outputs: The Mi50 is designed for headless compute tasks. It has no display output enabled on the stock VBIOS. You will need to flash a custom vBIOS (such as a Radeon VII vBIOS) to get video outputs working.
Cooling: Server GPUs generally lack built-in fans. You will need to build or buy custom fans/shrouds to keep them from overheating