• 8 Posts
  • 449 Comments
Joined 1 year ago
cake
Cake day: June 13th, 2023

help-circle









  • You know that there are two unrelated words, and you’ve seen two different spellings—it’s a natural assumption that the latter stems from the former.

    Why so many people would pair them up the same (etymologically unsupported) way, I don’t know… maybe we’re used to correlating words relating to art with French, and assuming that words with “ou” come from French as well (and this case just happens to be an exception).




  • Not using their turn signals if the only other traffic is pedestrians.

    So many times I’ve been crossing an intersection to the opposite corner where I could cross either street first, so I pick the street that won’t block the car crossing the other way. They’re not signalling so I figure they’re going straight, and cross the other way so they won’t have to wait for me—but seemingly every time it turns out the car was really turning after all. So they’re stuck because they couldn’t conceive of pedestrians as traffic they need to communicate with.







  • AbouBenAdhem@lemmy.worldtoAI@lemmy.mlDo I understand LLMs?
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    19 days ago

    There’s a part of our brain called the salience network, that continually models and predicts our environment and directs our conscious attention to things it can’t predict. When we talk to each other, most of the formal content is predictable, and the salience network filters it out; the unpredictable part that’s left is the actual meaningful part.

    LLMs basically recreate the salience network. They continually model and predict the content of a text stream the same way we do—except instead of modeling someone else’s words so they can recognize the unpredictable/meaningful part, they model their own words so they can keep predicting the next ones.

    This raises an obvious issue: when our salience networks process the stream of words coming out of such an LLM, it’s all predictable, so our brains tell us there’s no actual message. When AI developers ran into this, they added a feature called “temperature” that basically injects randomness into the generated text—enough to make it unpredictable, but not obvious nonsense—so our salience networks will get fooled into thinking there’s meaningful content.