It could just be poor communication between you and him. I was the lead in a project over several teams without any onboarding, and several people to answer to without clear lines of hierarchy. I bent over backwards on evening and weekends for 2 years trying to make everyone happy and thus making no one. Had I better communicated that I was struggling, or at least had the life perspective to understand that what was happening wasn’t completely my fault, I could have defended myself better and communicated better to my bosses that their wonderful plan of bringing several teams together with one in-between guy (me) might require some extra thought.
In retrospect I should have quit, but I was new to the country and had no idea what my rights were.