navigatron, 2 years ago

Go a level deeper, beyond this news about news, and read the moat memo.

The third faction is the open source community.

The memo has an entire timeline section, dedicated to showing the speed at which the open source community absorbed and iterated on the leaked facebook model, LaMMa.

The memo puts a lot of emphasis on how google and co are building new models from scratch, over months, with millions of dollars - and yet open source is building patches, in days, with only a few hundred dollars - and the patches stack, and are easily shareable.

The open source models, through these patches, are getting better faster than google can re-architect and re-train new models from scratch.

The main point of the memo is that google needs to change their strategy, if they want to stay “ahead” (some would argue they’re already behind) of the competition.