just_another_person@lemmy.world to Linux@lemmy.worldEnglish · 2 months agoAMD Announces "Instella" Fully Open-Source 3B Language Modelswww.phoronix.comexternal-linkmessage-square30fedilinkarrow-up1100arrow-down13
arrow-up197arrow-down1external-linkAMD Announces "Instella" Fully Open-Source 3B Language Modelswww.phoronix.comjust_another_person@lemmy.world to Linux@lemmy.worldEnglish · 2 months agomessage-square30fedilink
minus-squarebrokenlcd@feddit.itlinkfedilinkEnglisharrow-up3·2 months agoThe problem is… How do we run it if rocm is still a mess for most of their gpus? Cpu time?
minus-squarejust_another_person@lemmy.worldOPlinkfedilinkEnglisharrow-up5arrow-down1·2 months agoWell it’s not necessarily geared towards consumer devices. As mentioned in the writeup, it’s not trained on consumer gear.
minus-squarevividspecter@lemm.eelinkfedilinkEnglisharrow-up1·edit-22 months agoIs it still a mess? I thought it was reasonably well supported on Linux with GPUs from the past few years.
minus-squareswelter_spark@reddthat.comlinkfedilinkEnglisharrow-up1·8 days agoThere are ROCm versions of llama.cpp, ollama, and kobold.cpp that work well, although they’ll have to add support for this model before they could run it.
The problem is… How do we run it if rocm is still a mess for most of their gpus? Cpu time?
Well it’s not necessarily geared towards consumer devices. As mentioned in the writeup, it’s not trained on consumer gear.
Is it still a mess? I thought it was reasonably well supported on Linux with GPUs from the past few years.
There are ROCm versions of llama.cpp, ollama, and kobold.cpp that work well, although they’ll have to add support for this model before they could run it.