Just don’t engage with anyone who unfairly accuses you of “antisemitism”. Antisemitism is indeed a serious problem, but it doesn’t apply here. There is a huge difference between criticizing a government and criticizing a religion, but some people can’t/won’t accept that.
People who casually throw the word “antisemitism” around are usually too emotionally invested to have a real conversation, and/or they’re just trying to put you on the defensive. Never be afraid to abandon a conversation. You are not required to respond to anything someone says.