Looking for a recommendation on an LLM that can guide a user on how to manually set up a home server that runs Jellyfin, Arr Stack, etc. Have a couple of friends interested and I have some spare old NUCs I was going to give them, so I was going to download the software for them and then walk them through it.
But it’ll be difficult to align our schedules for the next month or so (school starting and different time zones) so then I thought maybe I could just run an LLM they could use on my main PC for when they have time.
Why not just use one of the existing services? Those cost extra money and none of us want to financially support those companies.
I am interested in this in general too. I have a 3090 which I know isn’t much but have been thinking of trying to get a second 3090 and using an NVLink to increase to 48Gb. Mostly for learning and testing purposes, and then to eventually offer to donate to my wife’s local school district a local model they own (since teachers aren’t supposed to put student info online, that includes private LLM models like ChatGPT, but they keep accidentally doing so anyway).
But one project at a time, and setting up this LLM would help my two friends if they have issues (it’s easy to forget for people like us that to nearly everyone else, setting up Jellyfin properly with remote access as well as things like Docker are not easy, even when they have base level knowledge), but it might help others in the future too that might then get interested in self hosting. A 24/7 guide can make a difference I think in someone actually trying self hosting or not.