It would be useful if people actually used it to burrow trolls, sealions and irrelevant comments as intended, but as I’ve seen people can’t be trusted with that because as you say: It becomes a “disagree” instead, that targets everything that people disagree with. It gets inane on political topics where useless comments for the right tribe gets immensely upvoted. “Covfefe” Yes, very informative. There could be alternate vote for agreement, funny, or troll mark.
Moral panic? What? It’s about healthy community dialogue and slightly how downvotes impacts the psyche.
If someone tells you why they dislike something you like, you’re not doing anyone a favor by downvoting it.