Tailscale has the Funnel feature that doesn’t require an open port (well, it’s a UPnP port), and maintains an encrypted tunnel to your music server for anyone you decide to share it with.
Alternatively you could get as many of your friends to use Tailscale itself to minimize the need for the Funnel feature (so anyone you know get them using TS, for the general sharing let it happen via Funnel).