There are so many styles, patterns, and colors of caps under the very wide term of flat cap. I think the pretentious feeling is silly, and probably geographical. I have baseball hats, beanies, newsboys, whatever. It doesn’t have to be your thing, just a thing you have.