What tools and projects do you use for your local LLM chats and agents?

I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.

  • FauxLiving@lemmy.worldEnglish
    34·
    3 hours ago

    I’m sure OPs RTX 3060 will have a tolerable effect on the school children and the rainforest.

    • Andres@social.ridetrans.it
      31·
      3 hours ago

      @FauxLiving @YeahToast “I’m sure my driving to work will doom the world” same mentality.

      Also, it’s not so much use of a model as the creation of the model. Training is expensive in terms of resources and energy. Using the model and normalizing AI usage leads to more AI usage and more AI normalization, which leads to new models being trained…

      • Riskable@programming.devEnglish
        1·
        5 minutes ago

        Put it into perspective:

        • Everyone doing their laundry uses more water than all data centers combined. Golf courses also use vastly more water.
        • Data centers use about 1-2% of global electricity.

        People driving cars pollutes more as does global shipping, regular old consumerism, etc. The environmental impact of data centers is negligible in the grand scheme of things.

        Now that AI is making open source software the most secure software ever written (the hard way, hehe), not using it to scan your stuff is now considered worst practices and basically idiotic negligence. No company that uses software or anything with firmware can afford to ignore it anymore.

        There’s no way out of it. Accept the new AI reality and push for good regulations. Example: If we made it so that all AI data centers had to be powered via renewable energy on site that would allay most of the energy use/environmental concerns.