Sims,

If low on hw then look into petals or the kobold horde frameworks. Both share models in a p2p fashion afaik.

Petals at least, lets you create private networks, so you could host some of a model on your 24/7 server, some on your laptop CPU and the rest on your laptop GPU - as an example.

Haven’t tried tho, so good luck ;)

  • All
  • Subscribed
  • Moderated
  • Favorites
  • selfhosted@lemmy.world
  • localhost
  • All magazines
  • Loading…
    Loading the web debug toolbar…
    Attempt #