I think your point would be true in a high population city in a high population nation where most of it is accessible to the average person.
I live in Australia, though.
Fishing requires gear, skill, and knowledge. For much of our fish abundance the average person wouldn’t survive getting to the area or know what to do once they got there. This is why they have untouched abundance.