In typical usage, removed is a pejorative term either for someone with an actual mental disability, or for someone who is considered stupid, slow to understand, or ineffective in some way. The adjective removed is used in the same way, for something very foolish or stupid.