It's illegal if you copy-paste someone's work verbatim. It's not illegal to, for example, summarize someone's work and write a short version of it.
As long as overfitting doesn't happen and the machine learning model actually learns general patterns, instead of memorizing training data, it should be perfectly capable of generating data that's not copied verbatim from humans. Whom, exactly, a model is plagiarizing if it generates a summarized version of some work you give it, particularly if that work is novel and was created or published after the model was trained?