It’s like using literally to add emphasis to something that you are saying figuratively. It’s not objectively “wrong” to do it, but the practice is adding uncertainty where there didn’t need to be any, and thus slightly diminishes our ability to communicate clearly.