First up, fandom is free advertising; fuck them I’m not promoting their product for them, even if I don’t buy it.
But more than that, it’s sending a message that the behaviour is something we’re willing to condone, that we stand with the abuser rather than their victims.
Imagine telling a sexual assault survivor to just lie back and enjoy the masterful comic stylings of Bill Cosby, or at least to shut up and let you enjoy it, because they’re ruining the funny.
Would that person have reason to consider you a friend or ally after that?
The Harry Potter IP, for instance, is just a giant anti-trans flag now, and the people who wave it around are picking a side. They can’t pretend they’re not; pinning the logo to their chest is explicitly endorsing the author’s views, and spitting in the face of every trans person in their life.