Learning is, essentially, "algorithmically copy-paste". The vast majority of things you know, you've learned from other people or other people's works. What makes you more than a copy-pasting machine is the ability to extrapolate from that acquired knowledge to create new knowledge.
And currently existing models can often do the same! Sometimes they make pretty stupid mistakes, but they often do, in fact, manage to end up with brand new information derived from old stuff.
I've tortured various LLMs with short stories, questions and riddles, which I've written specifically for the task and which I've asked the models to explain or rewrite. Surprisingly, they often get things either mostly or absolutely right, despite the fact it's novel data they've never seen before. So, there's definitely some actual learning going on. Or, at least, something incredibly close to it, to the point it's nigh impossible to differentiate it from actual learning.