this post was submitted on 09 Jan 2026
5 points (77.8% liked)

Machine Learning

596 readers
10 users here now

A community for posting things related to machine learning

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

founded 2 years ago
MODERATORS
Ategon@programming.dev
Akisamb@programming.dev
ericjmorey@programming.dev
5
Researchers figured out how to run a 120-billion parameter model across four regular desktop PCs (actu.epfl.ch)
submitted 5 days ago by noumenon@lemmy.world to c/machine_learning@programming.dev
4 comments fedilink hide all child comments
top 4 comments
sorted by: hot top controversial new old
[–] slazer2au@lemmy.world 3 points 4 days ago

Regular desktop PC with a €2500 euro graphics card.

[–] FlexibleToast@lemmy.world 2 points 5 days ago

A 120b parameter model is small compared to the models running in datacenters. However, this does seem like the current "Moore's Law" for AI. Finding more and more efficient ways to run larger parameter models.

[–] Mikina@programming.dev 1 points 4 days ago* (last edited 4 days ago)

Do we really need datacenters to host (load balanced) web servers and services?

This could challenge the business model of Big Tech!

[–] Kissaki@programming.dev 1 points 4 days ago

The article isn't very concrete on what the substance of this Anyway System is.

The FAQ answered my question though, and it seems mostly about dynamic management of PCs as execution nodes.

I assume that means I could run the model on one of those PCs as well, despite the article claiming you can use as few as four PCs? Or does this system somehow distribute a single model prompt process into multiple execution pipelines?