What tools and projects do you use for your local LLM chats and agents?

I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.

  • YeahToast@aussie.zoneEnglish
    53·
    4 hours ago

    AI effectively raped and pillaged all human information for the benefit of a handful of corporations. It’s use has been shown to reduce the skills of school children and even deskill those with established knowledge. Then there’s the environmental considerations in a time where the world is decades behind appropriate climate change management… Id say it’s pretty understandable that it evokes passionate responses.

    • FauxLiving@lemmy.worldEnglish
      34·
      4 hours ago

      I’m sure OPs RTX 3060 will have a tolerable effect on the school children and the rainforest.

      • Andres@social.ridetrans.it
        31·
        3 hours ago

        @FauxLiving @YeahToast “I’m sure my driving to work will doom the world” same mentality.

        Also, it’s not so much use of a model as the creation of the model. Training is expensive in terms of resources and energy. Using the model and normalizing AI usage leads to more AI usage and more AI normalization, which leads to new models being trained…

        • Riskable@programming.devEnglish
          11·
          36 minutes ago

          Put it into perspective:

          • Everyone doing their laundry uses more water than all data centers combined. Golf courses also use vastly more water.
          • Data centers use about 1-2% of global electricity.

          People driving cars pollutes more as does global shipping, regular old consumerism, etc. The environmental impact of data centers is negligible in the grand scheme of things.

          Now that AI is making open source software the most secure software ever written (the hard way, hehe), not using it to scan your stuff is now considered worst practices and basically idiotic negligence. No company that uses software or anything with firmware can afford to ignore it anymore.

          There’s no way out of it. Accept the new AI reality and push for good regulations. Example: If we made it so that all AI data centers had to be powered via renewable energy on site that would allay most of the energy use/environmental concerns.