Not sure if there’s something else missing in your setup. I’m using WSL2 (Ubuntu), and in my case all I had to do was compile Emacs passing the --with-pgtk flag, then:

sudo apt install gnome-tweaks yaru-theme-gtk yaru-theme-icons

And ran gnome-tweaks in the terminal, and changed the application and icon themes to Yaru-dark.

Yep that’s what finally caused me to switch to using Emacs on WSL2 (instead of the Windows build). Then you can also get rid of that ugly white border from WSLg.