thatsnothowyoudoit, 1 year ago (edited 1 year ago)

You’re conferring a level of agency where none exists.

It appears to “understand.” It appears to be “knowledgeable. “

But LLMs do neither of those things.

Take this note from an OpenAI dev: https://lemmy.ca/pictrs/image/f9e0f536-713c-43f9-a97f-ef2b2dabe0c3.png

It’s that these models have leveraged so much data they’ve been able to map out relationships between words (or images) in way as to be able to generate what seem like new versions of those things.

I grant you that an LLM has more base level knowledge than any one human, but again this is thanks to terrifyingly large dataset and a design that means it can access this data reasonably reliably.

But it is still a prediction model. It just has more context, better design and (most importantly) data to make predictions at a level never before seen.

If you’ve ever had a chance to play with a model at level where you can control some of its basic parameters it offers a glimpse into just how much of a prediction machine it can be.

My favourite game for a while was to give midjourney a wildly vague prompt but crank the chaos up to 100 (literally the chaos flag at the highest level) to see what kind of wild connections exist but are being filtered out during “normal” use.

The same with the GPT-3.5 API in the “early days” - you could return multiple versions of the response and see the sausage being made to a very small degree.

It doesn’t take away from the sense of magic using these tools. It just helps frame what’s going on under the hood.