I’m not sure exactly where that idea comes from

It’s been propagated by the detractors of anarchism. The same defamation was used towards the republic when monarchies where the rule rather than exception. People often equated the concept of a republic with chaos and disorder, just like they now do with anarchism.