no, it isn’t bidirectional, public = encrypt, private = decrypt, that’s it. You can address a message to multiple recipients though (when using GPG), so often in case of email a message is addressed both to yourself and your recipient, so both you and your recipient have access to message text
the encryption keys, why can't the government just sneak on them?
disclaimer: I’m just asking to get understanding of the theory behind network traffic encryption, I know this doesn’t happen irl most likely....