Recent post re: AI as utility

https://www.tomsguide.com/ai/people-will-buy-intelligence-from-us-on-a-meter-chatgpts-ceo-sam-altman-has-critics-worried-with-his-ai-vision

Myself, I’m a fan of local LLM / self hosted ML… but if you ever needed a clarion call that a hard pivot is coming (soon) for online/ cloud based AI…Altman et al are making some concerning mouth noises (to say nothing of broader concerns with OAI, Anthropic etc).

Right now, I’m sketching out a plan where my Raspberry Pi (always on, 2-3w) uses a magic packet to wake up my modest AI server (Lenovo P330 with Tesla P4) if/when needed (Qwen 3.6-35B-A3B); no point in chugging down 80-100w, 24/7 for no good reason.

If the trend continues the direction it appears to be (increasing costs, environmental impacts etc) then I’d feel a lot better hosting my own as port of first call and replacing simpler tasks with more traditional programs. YMMV.

  • WhyJiffie@sh.itjust.worksEnglish
    2·
    1 day ago

    The only limitation I know is WOL doesn’t work after a power outage, because the switch and RPI doesn’t know where to find the target machine

    maybe, but the pi does not need to know that, only the mac address and the interface. the switch doesn’t need to know either because it’s a broadcast frame, it’s forwarded to all cables. the problem sometimes is that if you configure WOL from linux, the network adapter will probably forget on power cycling that it is supposed to react to magic packets. I think not all hardware is susceptible to that, but even then it could help to configure WOL in the BIOS

    @SuspiciousCarrot78@aussie.zone

    • klangcola@reddthat.comEnglish
      1·
      1 day ago

      Maybe something else going on then, but ive never gotten WOL to work after a blackout when there’s two switches between sender and receiver. After powering up the receiver once, WOL works again

      • homik@slrpnk.netEnglish
        2·
        11 hours ago

        Switches probably need to figure out which way a particular MAC is (unlike a hub, which just express everywhere). That’s the switching part. If they power off, the tables will be empty.

        • klangcola@reddthat.comEnglish
          2·
          9 hours ago

          Yeah that was my assumption. But I hadn’t considered WOL being broadcast, so now I’m not so sure. I would assume it’s broadcast on both IP and Ethernet layer. It’s time to do some wiresharking :)

          • homik@slrpnk.netEnglish
            1·
            3 hours ago

            I don’t think WoL works over IP. In my mind it’s lower (LAN, e.g. ethernet) level. But if it used IP, you’d need to get ARP going before it routes. An “offline” network chip could probably manage that, though.

            I’m curious to know what you find. Wireshark is always fun and fun and enlightening. :)

      • WhyJiffie@sh.itjust.worksEnglish
        2·
        22 hours ago

        that’s probably the BIOS only loading the configuration on the first boot. you could try enabling fast boot or disabling the right energy saving settings in the BIOS and see if that fixes it.