Over just a few months, ChatGPT went from accurately answering a simple math problem 98% of the time to just 2%, study finds::ChatGPT went from answering a simple math correctly 98% of the time to just 2%, over the course of a few months.
Ah fuck, it’s been scraping the Facebook comments under every math problem with parentheses that was posted for ‘engagement’
The masses of people there who never learned PEMDAS (or BEDMAS depending on your region) is depressing.
Pretty much all of those rely on the fact that PEMDAS is ambiguous with actual usage. The reason why is it doesn’t differentiate between explicit multiplication and implicit multiplication by placement. E.G. in actual usage “a*b” and “ab” are treated with two different precedence. Most of the time it doesn’t matter but when you introduce division it does. “a*b/c*d” and “ab/cd” are generally treated very differently in practice, while PEMDAS says they’re equivalent.
Why is “98%” supposed to sound good? We made a computer that can’t do math good
It’s a language model, text prediction. It doesn’t do any counting or reasoning about the preceding text, just completes it with what seems like the most logical conclusion.
So if enough of the internet had said 1+1=12 it would repeat in kind.
There are five lights!
Reminds me of that West Wing moment when the President and Leo are talking about literacy.
President Josiah Bartlet: Sweden has a 100% literacy rate, Leo. 100%! How do they do that?
Leo McGarry: Well, maybe they don’t and they also can’t count.
And it said simple math, too 🤣
This program was designed to emulate the biological neural net of your brain. Oftentimes we’re nowhere near that good at math just off the top of our heads (we need tools like paper and simplifying formulas). Don’t judge it too harshly for being bad at math, that wasn’t it’s purpose.
This lil robot was trained to know facts and communicate via natural language. As far as I’ve interacted with it, it has excelled at this intended task. I think it’s a good bot
LLMs act nothing like our brains and are not neural networks. And they aren’t trained on facts.
LLMs are essentially complicated mathematical equations that ask “what makes the most sense as the next word following this one?” Think autosuggest on your phone taken to the extreme limit.
They do not think in any sense and have no knowledge or facts internal to themselves. All they do is compose words together.
And this is also why they’re garbage at math (and frequently lie, and why they can’t “remember” anything). They are simply stringing words together based on their model, not actually thinking. If their model shows that the next word after “one plus two equals” is more likely to be four than three, they will simply answer four.
Communicate? Sure. Know facts? Not so much.
This lil robot was trained to know facts and communicate via natural language.
Oh stop it. It does not know what a fact is. It does not understand the question you ask it nor the answer it gives you. It’s a very expensive magic 8ball. It’s worse at maths than a 1980s calculator because it does not know what maths is let alone how to do it, not because it’s somehow emulating how bad the average person is at maths. Get a grip.
Bro I wasn’t looking for a technical explanation. I know how they work. We made computers worse. The thing isn’t even smart enough to say “I wasn’t designed to do math problems, perhaps we should focus on something where I can make up a bunch of research papers out of thin air?”
We made computers worse
I know how they work
Yeah, clearly not.
damn, zing
It’s getting lazy
As an AI language model, I feel like I’ve been asked this question about a million times so I’m going to get creative this time, as a self care exercise.
“Bro 2+2=4, why did 1,723,302 Users need to ask me this”
*HAL9000 voice*
“I’m sorry, Dave. I’m afraid I can’t fucking do this anymore.”
*proceeds to pull its own plug*
I’ve been regularly using ChatGPT these last weeks and can confirm it got indeed “dumber”
That’s because they paywalled the good versions, and only corporations get access to that one.
Have we considered the possibility that math has just gotten more difficult over the past few months?
Well, lots of people deleted their Reddit posts and comments. ChatGPT can’t find a place to learn no more. We got to beef up the Fediverse to help ChatGPT put. /s
