Oh if you even have your own router then have a firewall (primarily) there, and simply block every incoming forward connection except the ones you actually want (probably forwarded to your server). Similarly even for the router input rules you likely need only ICMP and not much else.