This is a very well known phrase in England, but it refers to football (soccer) and rugby. I have never heard this in reference to American football and cannot even imagine how the meaning would translate.

The meaning of it comes from football being a supposedly gentle game of skill, but played and loved by the common people and with a long history of bad behaviour on and off the pitch, versus rugby being a rough, bruising, injury riddled sport that was played by aristocratic rich kids in private schools who are all excellent young men destined for greatness.

Of course this is mostly oversimplified bollocks, but with just enough truth behind it to make it funny.