I bought several VPSes across different VPS providers. Although I chose Ubuntu 20.04 on all providers, I found they use different Linux kernels. There might be other differences as well. In order to keep my running environment stable, I want to use the same OS image in all VPSes. But I think it’s impossible, it seems, that every vendor has some modification on internet options at least.
Possibly NixOS? It’s all about having a reproducible OS based on a configuration file. I think with Nix flakes you could do exactly what you want on the OS level.
Though beware that it’s kinda different to all the other Linux systems you might have tried.