Only because AMD/Intel aren’t pricing competitively. I define “best experience” as the largest LLM/context I can fit on my GPU, and right now that’s essentially dictated by VRAM.
That being said, I get how most wouldn’t want to go through the fuss of setting up Intel/AMD inference.
Intel is not as bad in LLM land as you’d think. Llama.cpp support gets better every day.
Nvidia may be first class, but in this case, it doesn’t matter if the model you want doesn’t fit in VRAM. I’d trade my 3090 for a 48GB Arc card without even blinking, even if the setup is an absolute pain.
The bizarre thing about this is that AMD’s workstation card volume is comically small, and Intel’s is probably nonexistant.
On the high end… Intel literally discontinued their HPC GPUs. The AMD MI300X is doing OK, but clearly suffering from a lack of grassroots software support.
WTF are they afraid of losing?
I’ve thought about it as some of my niche interests have dried up on Reddit and apparently only live on Twitter now.
And maybe Discord. It’s hard to tell because it’s closed, but I see hints of discussion spilling out into GH issues or whatever.
I despise both for different reasons, but where else am OlI gonna go if I actually want to talk with the community?
Not that the mods are aware, but if Reddit disappeared tomorrow, I (sadly) suspect most users would go to Twitter or various Discords.
An important thing to note is Intel does not have an enterprise class GPU anymore, and appear to have abandoned most plans for such a thing.
So while consumer GPU inference is great, and would seed support for their future laptop/desktop IGPs, Intel is not in the same boat as AMD anymore, who’s consumer efforts would seed support for the enterprise MI300X.
I see basically zero chance of Intel supplanting CUDA for this reason, especially if they don’t foster cooperation with anyone else.