Yes you could break the connector if you’re not careful. However at the end of the day, it’s not like you can’t replace both sides of the connector with standard hardware, and at the very worst you could solder the fan to the pins and cover each wire with heat shrink.