That’s just a fundamental problem with security. You can vault up your home but give your idiot brother in law a key and find the back door wide open, him drunk on the kitchen floor.
Prompts don’t work and aren’t really the right way to go because they are annoying and pretty cryptic as apps often assign a myriad of features to a single permission. Everyone’s just going to hit OK.
It’s a difficult issue to solve because there are so many edge cases. And fundamentally you can’t really control what others do with your number.
Honestly. I wish we started talking about doing away with phone numbers altogether. I feel tech is there. And it’s honestly such a massive fingerprint. I’ve had mine for 20 years ffs.