As far as I know the borg backup store should only add new blocks as new files and remove them when you purge the last backup that uses that block. Obviously some of the metadata files are going to change and will be backed up more frequently but the main data should not.