cecilkorik

I’ve already told them a few hundred million dollars of government assistance would expedite the investigation significantly but unfortunately it does not seem to be a priority of the current administration. It is a very complex and unprecedented situation, after all. At this rate it may take decades, possibly centuries before it concludes.

Meanwhile I’m keying the AI companies’ cars.

Yes. The shit always flows downwards, it’s just a fundamental law of physics. Not much can be done about that unfortunate reality.

But in my fundamental laws of morality, responsibility for the shit always flows upwards, and those are the people I’m going to hold morally responsible.

I’m glad you understood the metaphor but it did seem to trip you up a bit, so I’ll explain: “Nuke it from orbit” is a reference to the Alien movies. You should watch them, they are good cinema.

Nuking AI datacenters from orbit is something that could be done about it. They wouldn’t like it, but it would definitely do something about it.

Computing capacity that is not being used to make our lives better does not deserve to exist. If we’re not going to destroy it, we should seize it and start to use it to make our lives better.

It really wouldn’t surprise me if this powerful Machiavellian cabal of pedos have existed since long before that. I’m not saying that it was actually planned this way, of course, but maybe we shouldn’t be so quick to rule it out. :P

I want the subreddit to be at least 95% NOT AI, but without completely excluding AI content (which must be tagged) and I don’t want to see everything tagged “[NOT AI]” because that’s genuinely obnoxious.

I understand that this is maybe not realistically achievable given the technical limitations within the Lemmy platform, but those limitations are not going to make such an implementation any less obnoxious, even if it is implemented that way for my benefit.

I would rather trust the mods and downvoters to clean up not-tagged or dishonestly Not-AI-tagged AI content, personally.

Projecting their own insecurities onto other people seems to be a hallmark of this administration. I would argue it’s a guilty conscience perhaps if I thought they had one.

250W and $1000 is plenty for a GPU and I am willing to die on this hill, apparently with an AMD card in my hand.

Yes, let’s intentionally break our glass house by throwing much deserved stones at something else. Once it’s broken, we’ll replace it with a house that isn’t made of glass and doesn’t tolerate the mutilation of children of either gender. This is not about assigning blame or deciding who’s right or wrong, and it’s disingenuous and distracting to try to compare them, maintain your focus on stopping the practices.

Sometimes, yes. When they’re done without consent or emotionally pressured or coerced upon children too young to be capable of informed consent, yes. Consenting adults, fine. Consenting teens or even tweens, fine. Teen who feels socially pressured? No. 5-year-old? No. This is not something that affects your future. This is not something that has some medical or preventive purpose.

Ear piercings may only cause minor pain and have minor infection risks and may not be a serious harm in the great scheme of things, but they set a horrible precedent for consent and bodily autonomy and I find it unconscionable that anyone can defend that unless they are just expressing how normalized it is because of the way it happened to them too. Beauty standards for women are oppressive things that even women tend to unwittingly enforce on each other, and it starts during childhood. It’s not good, it’s not healthy. It’s sick, and it’s malicious, and we need to stop perpetuating it. Let people have the bodies they are born with and accept them as they are, let them decide if or when they want to change it.

I’m not an expert by any means I’m just a dabbler, but my understanding is: In theory, more parameters make richer, wider, and deeper model knowledge possible, and with extensive enough training, those parameters could all be important. That said, there is a lot of megapixel-like inflation and there is no guarantee that any of those parameters are actually useful so in practice, really “advanced” models tend to do a better job of maximizing the usefulness of the limited parameters they do have to run on smaller devices. In general, I tend towards the highest parameter size of a particular model that I can reasonably run. My typical target range is between 8GB up to maybe 20GB, which depending on model might be in the 9b to 30b parameters range, and I might even be erring on the wrong side of this and maybe I’d even be better off with smaller parameter models.

There’s also a lot of models nowadays that use “active” parameters, so the model itself will have X parameters, but then it will determine which of those parameters are most relevant to the task or query at hand, and prune off all but the most relevant ones, so you might have a 30B model, but as soon as you run it, it turns itself into a specialized 4B model. You still need to load the whole model into some kind of RAM typically so it can decide which parameters are relevant, but once it does, it will run much faster. This is another way you can try to run larger models on more limited hardware. Older “dense” models that don’t use this technique with all parameters always active are still typically preferred for some tasks like coding, but YMMV.

Either way, it’s still sort of a crapshoot, there’s a lot of randomness and subjectiveness, and very small parameter models often seem to realistically be able to outperform much bigger models when they are “good”, “well-trained” advanced models, and they will typically be much faster, so if you don’t like the response, it’s much easier to just ask again or retry. I tend to trust the community wisdom when it comes to this, although I also think there’s a lot of cargo-culting and herd-following going on, I don’t know enough to do anything too much different from the herd myself, other than be willing to experiment a little. Latest is not always greatest, but in a field as quickly moving as this it often is. Don’t be afraid to try older models, or less popular models. You’ll often be disappointed, but not always.

Quantization is a form of compression, basically instead of using floating point precision to weigh the “strengths” of the various parameters (default is typically F16 or 16 bits per parameter weight), they get quantized down to smaller groups of bits. Q4 means you’re using 4 bits (essentially ranking each parameter on an integer scale from 0 to 15 instead of a floating point from 0 to 1) and in practice this is usually almost as good. Q8 would be even closer to the original full-size model, but smaller quants like Q2 and Q3 start losing quality. Other quantization-related techniques like i-Matrix (imat) map these values non-linearly and situationally, which is particularly helpful on quantizations Q3 and smaller, which are then called IQ3. The community has adopted Q4 as pretty much the go-to quantization level as the best available compromise between having more parameters being squeezed into less memory without destroying the inherent accuracy of those parameters.

For chat usage (which is strictly a more efficient way to generate code on the LLM’s part, although you have to keep carefully guided and compartmentalized otherwise it typically requires a lot more testing and sometimes back-and-forth iteration on your part) 12GB is plenty to run many decent LLMs, you’ll typically want to use a Q4 quantization to make models with larger parameter fit into smaller memory, sometimes an IQ2 or IQ3 if you really want a particular model.

For agentic usage (where the LLM is trained and optimized to use a harness like this to start requesting tool calls and getting their results and using the results of the tool calls to inform what it’s trying to do) it’s quite a bit more challenging to do on consumer hardware at a tolerable speed. The tools often generate large amounts of output which then take a long time to process, and the models and harnesses are both typically quite a bit stupider about using your limited resources efficiently. If you’re using to commercial “frontier” agentic models like Claude Code you’re going to have a bad time.

That said, it is absolutely possible to do agentic AI on consumer hardware (just the GPU you have, not 6 of them), as long as you’re reasonably patient, using a harness properly tuned for efficiency. Out-of-the-box, many if not most are designed for remote API usage, even the “open source, local” ones realistically rely on free tier APIs and are inherently wasteful in terms of them not really caring how many tokens you burn in these remote datacenters and they’re expecting to just be able to iterate over and over again until they get it right. You don’t have that luxury when you’re getting slow tokens.

Is PewDiePie’s any better or more efficient? I don’t know, I haven’t tried it yet. I prefer more minimal harnesses personally, OpenCode is about the most usable I’ve found personally, although I’m starting to experiment with Pi-mono (called Pi, but that’s unsearchable) which seems very promising, and I know quite a few people who have had good successful agent usage with Hermes Agent.

I’m not going to pretend it’s going to be easy or that you’ll necessarily have very good results. I am pretty lukewarm on AI as a whole, but I am personally deeply invested in making sure I have fully local access to it in as much capacity as is currently technologically possible, as a personal digital sovereignty issue.

As for hardware, I have a 12GB card myself and you don’t really need to fit everything into VRAM these days. I have an AMD X3D CPU which allows me to offload some of the model to system RAM with pretty decent performance, maybe it’s prohibitive on different architectures or configurations I don’t know but it’s worth a try. glm-4.7-flash:Q4_K_M from ollama is the model I’ve had the most consistent success with and with ollama running it with the context window set to 50,000 (context should also be set to be quantized to Q4_K_M), I end up with almost half of it offloaded to system RAM and it still runs quite fast thanks to the flash attention feature. I’ve worked with gemma4 quite a lot too and it’s definitely really fast but it’s also a bit unstable/weird at times, at least the heretic version hf.co/Stabhappy/gemma-4-26B-A4B-it-heretic-GGUF:Q4_K_M I’m running is. Still, if you really do need to fit everything into a smaller set of RAM you might try the gemma4 E4B models which clock in around 9GB when quantized. Qwen3.6 is I guess supposed to be really good too and should fit nicely on your 12GB card, but I haven’t had much opportunity to play with it yet. Qwen3 and 3.5 felt rather disappointing to me for agentic use but YMMV.

You’re not completely going to outsource all software and all code you write to AI using a local model, the way companies are doing with those commercial models. But I consider that an advantage, not a flaw. I find it’s much more useful to have it help, suggest and advise, not to completely replace everything I’m doing. Yes, sometimes it’s slow and sometimes it’s wrong, but so are other people when I ask them sometimes. I’m prepared for it, and you should be too. Don’t get complacent.

Yes. mine is exposed publicly (with fail2ban) on a VPS with a public IP and a public DNS name and it’s fine. Use a minimal configuration that meets your needs, use secure passwords like you would for any public service and keep it up to date, and stay aware of any potential news that might make you aware of any severe and widespread vulnerabilities in the future (there haven’t been any in Nextcloud so far). It is not nearly as terrifying as people make it out to be to share public services on the public internet. Most decent software is secure-by-default. Yes vulnerabilities and attacks can happen but they are the exception not the rule.

In my experience PikaOS devs fix any critical issues before they get to me. I have auto-updates on and haven’t seen a serious problem in the year+ I’ve been using it on multiple machines, YMMV.

Not that I’ve noticed, but isn’t that part of the territory? Like do latest Nvidia graphics drivers not sometimes break on Windows? My experience is that they certainly do. If you want perfect tried and tested stability, you have to sacrifice the ability to run the latest games using the latest features. This is a tradeoff you have to make. Which would be better? I’d run stable on servers, and latest on desktop, which is… what we’re doing here, right?

There is also a KDE default ISO, which is what I use, as well as Niri, Hyprland, and Cosmic. Pick whichever you prefer. Also Debian has a package manager that makes switching desktop environments literally a one-liner command of installing a single metapackage. I don’t know why people think this is such an obstacle.

I use PikaOS which is a Debian-derived gaming distro with newer drivers and gaming packages, run by some of the same folks involved in Nobara and sharing a lot of the common framework used for other major gaming distros. It is mostly indistinguishable from Debian, and I use it basically interchangeably. The main differences include the installer, the default background image, and some post-install helpers to install the latest drivers for various different graphics cards and many types of typical gaming software. Biggest downside is that support and community is through Discord, blech.

Maybe if the model trains could actually bring in your groceries and mow your lawn they’d be comparable. Granted, self-hosted software can’t do those exact things either, but it can do an awful lot of the digital stuff that’s part of our lives now which often takes up just as much time and effort if not more. Model trains are a banger hobby, but homelabbing can easily be more than just a hobby, it’s deeply practical too, and I’d argue it’s actually a necessity for establishing personal digital sovereignty and privacy going forward.

As an analogy, consider it like joining a gang. If you’re truly an independent free spirit, you truly don’t belong in a gang. This may be not the right path for you. Gangs have rules, often even stricter and more harshly punished ones than the actual laws they’re breaking. If that’s surprising, you probably don’t understand how gangs work. In doing the dangerous stuff they do, they also need to protect themselves (and their shit) from a lot of just-as-dangerous people in addition to the lawful people who want them shut down and imprisoned. They are not places of freedom, they are places of ultraviolence.

That said, if you happen to be the sort of person who wants, say, some of the really interesting drugs the gangs are involved in trafficking, well then, you either have to join the gang, or stay on their periphery and keep a few loose connections with them without ruffling their feathers.

Obviously a private tracker is not quite as violent and high stakes as a gang, but some of the same basic principles more or less apply. You want their loot, you gotta follow their rules, or at least make a plausible attempt to. Don’t draw attention to yourself. Get in, get your stuff, and leave. If you don’t want to join the gang, don’t make a scene, don’t make friends, just keep your eyes open, do your business and then go home and enjoy what you found. It doesn’t have to be that complicated, but don’t oversimplify it either.