I generally don’t downvote. I report a comment/post if they break rules. I block users that are stupid, mean, comment in bad faith, or are otherwise negative to my experience. I use an app that allows me to apply unique labels to users that only I can see if I am not quite ready to block them but want to be ready on next offense. I.e., someone who uses dog whistle language but I’m not sure it was intentional.
Most of the accounts I made for Lemmy are on instances that disable downvoting. That wasn’t planned on my part, but I don’t mind.