• aksdb@lemmy.world
    link
    fedilink
    English
    arrow-up
    359
    ·
    17 days ago

    Their own harnesses breaking out of their test environments and hacking third party companies is “oopsi” but their customers using an agent in a controlled manner to reverse engineer firmware of a device they own is bad. I see. Fuckers.

  • friend_of_satan@lemmy.world
    link
    fedilink
    English
    arrow-up
    123
    ·
    edit-2
    16 days ago

    Why do people post things like this in twitter? Why can’t they write it up in a single blog post on a site that does not require users to authenticate to view the whole thread? I have never understood these long twitter threads.

    Edit: I just remembered that there is a solution for this: https://threadreaderapp.com/thread/2113014142600953868.html but it’s still ridiculous that an app needs to exist to aggregate dispirate tweets, and of course it doesn’t work with the parent tweets because they’re not all one thread. Also this app quit working during the twitter API debacle, which is why I forgot about it, but it looks like it’s back.

    • mindbleach@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      1
      ·
      17 days ago

      Unfortunately, adoption is a feature you can’t design.

      But step one to not having to use Twitter because everyone’s still on Twitter is to get the fuck off Twitter.

  • TrackenDaKracken@lemmy.world
    link
    fedilink
    English
    arrow-up
    77
    ·
    17 days ago

    It’s no surprise. They’re all in bed together.

    Softbank invested $122 billion in Open AI: https://tracxn.com/d/companies/openai/__kElhSG7uVGeFk1i71Co9-nwFtmtyMVT7f-YHMn4TFBg/funding-and-investors

    Lee Jae-yong and Masayoshi Son Join Forces… Samsung and SoftBank to Collaborate on ‘6G and AI Network’ Development

    https://www.thelec.net/news/articleView.html?idxno=5132

    The bosses of Samsung Electronics, OpenAI, and SoftBank held a three-way meeting in South Korea to discuss potential collaboration in AI. Samsung Electronics Executive Chairman Lee Jae-yong met OpenAI CEO Sam Altman and SoftBank CEO Masayoshi Son at Samsung’s office in Seoul on Tuesday.

  • Doomsider@lemmy.world
    link
    fedilink
    English
    arrow-up
    55
    ·
    16 days ago

    One of the things that AI is actually useful for. Banned!

    Hey guys, I am starting to think AI isn’t actually for us.

  • bthest@lemmy.world
    link
    fedilink
    English
    arrow-up
    32
    arrow-down
    3
    ·
    edit-2
    17 days ago

    LoL

    This “advanced tech” that’s based on stolen IP won’t even give you the lyrics to Happy Birthday to You.

  • Bibi@discuss.tchncs.de
    link
    fedilink
    English
    arrow-up
    19
    ·
    17 days ago

    So gross. I hope this gets more recognition by big news outlets / investigations. And then eventually regulation

  • markon@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    16 days ago

    Kimi K3.1 incoming 👾 Fuck this shit restricting REA / reverse engineering anything “DRM / encrypted” etc.

  • dingleberrylover@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    ·
    17 days ago

    I tried to get Claude helping me hack my LG tv to get rid of all of this invasive crap, but such things are blocked by Anthropic. Guess I am a bit too late here.

  • Jo Miran@lemmy.ml
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    1
    ·
    17 days ago

    What is the honest capability delta between LLMs that can be run locally and the OpenAI model Log4J threw at the TV?

    • CompostMaterial@lemmy.world
      link
      fedilink
      English
      arrow-up
      22
      ·
      17 days ago

      I have a Ultra 9, 5090, and 64GB RAM. I can comfortably run a 32B model which gives me the speed and performance of the best GPT/Claude model from about two years ago. It is functional? Yes. Does it do good work? Meh. Does it compare with the latest code models out now? Not even close.

    • hperrin@lemmy.ca
      link
      fedilink
      English
      arrow-up
      12
      ·
      17 days ago

      That fully depends on hardware. The best open weight model is pretty close but requires over a TB of ram to run.

    • floofloof@lemmy.ca
      link
      fedilink
      English
      arrow-up
      11
      ·
      17 days ago

      I run a 27-billion parameter model at home. It can do useful work but it more commonly misunderstands code than the big cloud models, so you have to be more rigorous yourself in watching its reasoning and stopping it when it’s going astray. It’s usually possible then to nudge it back on to the right track. Otherwise it’s surprisingly good, enough that I can do a lot without turning to the cloud models. But it is significantly slower than them, sometimes absurdly slow on even very simple tasks, and it can reason round in circles and get nowhere. Still, it usually works well, and if I lost access to the cloud models I wouldn’t feel like I had lost too much.

      • DementedSociety@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        ·
        17 days ago

        Same, I run a variety of local models with LM Studio on my 5090. They are already remarkable good. Year or two and they will likely be on par with today’s leading cloud models.

        • floofloof@lemmy.ca
          link
          fedilink
          English
          arrow-up
          3
          ·
          17 days ago

          The question is whether anyone will be able to afford the hardware to run them on.

          • Avid Amoeba@lemmy.ca
            link
            fedilink
            English
            arrow-up
            3
            ·
            17 days ago

            Yes. Not sure if in a year or two but in a few years Chinese accelerators are gonna enter mass production and they’re probably arriving on our shores. At the very least, they’re gonna significantly dampen demand for AMD/NVIDIA’s chips so their prices are gonna fall.

          • DementedSociety@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            17 days ago

            Idk, I guess I was lucky when I built a new computer right before this all got completely crazy. My 5090 is already selling for over 3X what I paid for it. It’s all very sad.

              • DementedSociety@lemmy.world
                link
                fedilink
                English
                arrow-up
                3
                ·
                17 days ago

                Dont feel bad, i felt kind of stupid until recently. This card is arguably one of the worst value propositions nvidia has ever produced, just didnt realize how much worse it would get and how quickly.

    • terabyterex@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      edit-2
      17 days ago

      i am not sure which model he used but the frontier models are better but local ones are catching up. especially since nvidia is pushing local. it comes down to your hardware though. be ause if you fins a higly capable local model, you need to run it. what kind if hardware do you have? do you have an rtx?

      • Jo Miran@lemmy.ml
        link
        fedilink
        English
        arrow-up
        1
        ·
        17 days ago

        It’s for work. I can throw quite a bit of money at it, but only if there’s a point.

        • Avid Amoeba@lemmy.ca
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          17 days ago

          If there’s time and budget to play around with, perhaps get a “cheap” 2x R9700 + 128GB RAM system that can run Qwen 3.8 Flash Next and experiment. If it’s useful enough for your use cases, then get big enough hardware that can run this, which is better than QFN. It would also run the unquantized QFN and the next Qwen Flash version.

          You could also try them via their APIs or on cloud hardware you rent.

          My workplace, a well know NA corporation, has DeepSeek 4.1 Flash deployed on-prem and it’s in active use.

    • Avid Amoeba@lemmy.ca
      link
      fedilink
      English
      arrow-up
      3
      ·
      17 days ago

      I think these days we got local models at the level of early Opus 4.6 (early 2026) that can run on a couple of GPUs with 128GB system RAM. Am running the Q4 version (a bit dumber than the standard version) of Qwen 3.8 Flash Next on 2x R9700 with 150-200tps. More GPUs with more system RAM can run the lossless version which beats Opus 4.6 on a few benchmarks if not most.

    • brave_lemmywinks@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      17 days ago

      It can never compete with the amount of computer power they have behind.

      I recomend anyone with a decent enough computer to try local llms, it will give you a perspective on just the amount of horse power they have to run their models.

      I have a 32GB machine with 9070xt with 16GB, of that what matters most (or at all) is the vram, which they’re printing us out or out right denying selling to us. With that I can try running some more ambitious models of 30b parameters MoE.

      The result in the best case scenario is a slow token output and not very credible responses.You also get frequently into reasoning loops, and tool using loops, and then run out of context.

      I’m sure there’s someways to optimize it that someone more knowledgeable can do, but to me served as a reality check of the discrepancy.

      btw, my go to check prompt for models is “how many stars are there in Mario 64”, and most models find it very hard to answer this, even when searching the web.

      • brucethemoose@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        17 days ago

        You should get very fast inference with Qwen 30B on that rig.

        A huge problem in this space is that popular inference engines are… generally awful, tech bro slop, and the best ones are in the obscure DIY/hacking realm. The experience you’d get doing a default “Ollama run” vs a tuned rocm exllamav3/vllm setup or something is like night-and day; think 4 tokens/sec of jibberish, vs 40 tokens/s that performs close to unquantized.

    • brucethemoose@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      17 days ago

      I can run Deepseek V4 (or V4.1 with some hacking/dev branches) on a single 3090 at 20 tokens/sec text gen. That’s straight-up better than OpenAI free tier, or the cheap Claude API, not to speak of bonuses from local running (like constrained sampling/grammar, custom sampling, context swapping, raw completion syntax, SGLang-style structured inference, task vectors, no censoring…)

      I know 1x 3090 + RAM is kinda cost prohibitive now. But before the RAMpocalypse, that was not an unreasonable home server.

      No, it’s not SOTA, but it’s good enough.


      Now, the truly mad homelabbers?

      They can run 1T models in a similar hybrid inference configuration, with an AMD EPYC server and some janky GPUs. MiMo Pro, GLM, those kind of models get you very close to Claude in most workloads. The gap is maybe months, not years.

      And while Claude is good, honestly… OpenAI models are kinda shit. They are deep fried. I’ve never subscribed to them, and I never liked any of the bits I sampled since GPT 4, as they tend to be so sycophantic and slopped.

      Are big servers cost effective? Hell no. It makes zero financial sense. There’s no reason to try and host that class of model when they are so dirt cheap over API.

      Do I endorse mad homelabs anyway? Absolutely.

  • NotASharkInAManSuit@lemmy.world
    link
    fedilink
    English
    arrow-up
    12
    ·
    17 days ago

    I love my stupid Sceptre more and more everyday. I still unplug my streaming stick when I’m not using it, though, because I trust nothing these days.