• FloridaBoi [he/him]@hexbear.net
    link
    fedilink
    English
    arrow-up
    33
    ·
    edit-2
    3 days ago

    Someone told me it’s like 97% more energy efficient or that it consumes 97% less energy. Is that true

    Edit: this comment has a summary saying that the model has 93% compression ratios so maybe that’s there efficiency number canes from

    • ragebutt@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      27
      ·
      4 days ago

      The exact figures aren’t documented but it’s a pretty massive decline in energy usage (though probably not 97%), enough so that stocks related to power consumption took a pretty notable hit

    • sewer_rat_420 [he/him, any]@hexbear.net
      link
      fedilink
      English
      arrow-up
      13
      ·
      4 days ago

      It consumes less energy now, but also consumed less energy in its creation. This is directly reflected in the cost to the user - the API is 10-30x cheaper per token than openAI

      • sewer_rat_420 [he/him, any]@hexbear.net
        link
        fedilink
        English
        arrow-up
        6
        ·
        4 days ago

        Without a profit motive and the need to commercialize it immediately, I hope deepseek continues to make efficiency gains will continue open source all their models. Right now it is 10x-30x cheaper per token, imagine if another generation could continue to reduce it by another order of magnitude? Project Stargate will just be a fancy bonfire to throw Arctic oil reserves at, while the rest of the world has access to state of the art LLM at a fraction of the price

      • That’s because LLMs aren’t supposed to be search engines. They are pretty good at summarizing documents in certain cases, but don’t have a big enough context window to effectively plow through massive troughs of data.

    • Stolen_Stolen_Valor [any]@hexbear.net
      link
      fedilink
      English
      arrow-up
      8
      arrow-down
      1
      ·
      4 days ago

      The “AI” is effectively just autocomplete powered by the internet. It could by powered by your 2001 flip phone probably. The whole thing is smoke and mirrors, hype, and snake oil bought by people who don’t understand what’s happening or people only concerned with line go up.

      • It could by powered by your 2001 flip phone probably

        LLMs are fundamentally billion-dimensional logistic regressions that require massive context windows and training sets. It is difficult to create a more computationally expensive system than an LLM for that reason. I have a fairly nice new laptop, and it can barely run Deepseek-r1:14b (14 billion parameter model. Not technically the same model as deepseek-r1:671b as it is a fine-tune of qwen-2.5:14b that uses the deepseek chain reasoning. It can run the 7b model fine, however. There isn’t a single piece of consumer-grade hardware capable of running the full 671b model.

    • LaughingLion [any, any]@hexbear.net
      link
      fedilink
      English
      arrow-up
      12
      ·
      3 days ago

      its the priming prompt to tell it to be sarcastic, the prompt is in my comment history i think they might have used a version of it and its very entertaining

  • Does deepseek on the web not show the thinking portion of the response? The thing deepseek does really well is CoT reasoning, and I always enjoy reading it.

    This is what it might look like behind the scenes (deepseek-r1:14b)
    >>> what is the airspeed velocity of an unladen swallow?                                                                                                                                                           
    <think>                                                                                                                                                                                                            
    Okay, let me figure out how to approach this. The user just asked, "what is the airspeed velocity of an unladen swallow?" Hmm, that's a classic question from Monty Python. I remember they mention it in          
    *The Holy Grail*. So, first, I should recognize where this is coming from—it’s definitely a reference to that movie.                                                                                               
                                                                                                                                                                                                                       
    But wait, do I have the exact answer? I know there was a scene where someone tried to calculate it, but I'm not sure about the specifics. Maybe I should look up the actual figure they used in the movie. Oh      
    right, in the film, it's mentioned as 4.5 mph, but that might be more of a joke than an accurate scientific answer.                                                                                                
                                                                                                                                                                                                                       
    Now, thinking about real-world biology, swallows do have different species—like the European Swallow and the American Swallow. Their flight speeds can vary based on whether they're cruising or flapping          
    their wings. I should probably provide some context about how airspeed velocity is measured in birds and mention the factors that affect it.
    
    I also want to make sure I don't just give a one-size-fits-all answer because different sources might have varying numbers, especially for something like an unladen swallow. Maybe I can suggest that the 
    number from the movie is more of a fun fact than a real measurement.
    
    So putting it all together: Acknowledge the reference, mention the figure from the movie, then provide some real-world info about swallows' flight speeds and factors influencing them. That way, the user 
    gets both the humor and a bit of factual knowledge.
    </think>
    
    Ah, I see you're asking about a classic question from *Monty Python and the Holy Grail*! In that film, Sir Bedevere attempts to calculate the airspeed velocity of an unladen swallow. The answer he comes up 
    with is **4.5 miles per hour**.
    
    However, in reality, the airspeed velocity of a European Swallow (a common reference for this question) can vary depending on the species and flight conditions. A rough estimate for cruising speed might be 
    around **1122 kilometers per hour** or **714 miles per hour**, but this isn't an exact science. 😊```
    
    This is, of course, incorrect. The airspeed velocity of a European Swallow is around 25mph, but deepseek isn't trying to be Wolfram Alpha or anything.