Sounds like every instance is doing the preview generation. Maybe it could grab the preview data it needs on the original instance, and pass that around with the post...
this post was submitted on 20 Oct 2026
670 points (98.6% liked)
Fediverse
44499 readers
499 users here now
A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, Mbin, etc).
If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!
Rules
- Posts must be on topic.
- Be respectful of others.
- Cite the sources used for graphs and other statistics.
- Follow the general Lemmy.world rules.
Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)
founded 3 years ago
MODERATORS
Another option could be to have the originating Mastodon server provide the preview image in content-adressable way, and have a standard allowing websites to provide the same: if the hash of the file(s) provided by the first / a Mastodon server to another matches the one from the target website, no need to download from the target website. Only if they differ then you'd actual download the data from the target website.
Each Mastodon server from the federation could quickly decide whether to trust that preview data from the peer it gets the rest of the post from or not.
This is basically taking ideas from IPFS but applying it to just the preview data that ends up being downloaded over and over again. You could also make that part a blob of data stored directly in IPFS. But IPFS was pretty slow to resolve content, when I last tested.
Well you can't really trust the original instance to be truthful with its preview.
There are ways around that. See my other comment replying to the same parent as your comment.
Why not have the users fetch the previews themselves. The user's browser, that is.
For sure there is more users than instances. So the amount of requests would be grater but at least they wouldn't happen at the same time.
But if fediverse ever grow to be as big as other social media, then a single post/link can be viewed by thousands of users at the same time causing the same effect.
Or get the preview on the first user seeing the post in that instance.
Thanks for bringing this up. I think in general we need to work on optimisation of Activitypub and Threadiverse implementations of it.
Either publishing server should serve the metadata - although it would be prone to bad actors, so a refetch in the next 0-180 seconds would be appropriate. Or just add a random 0-60 second delay before fetching the data.
There are two proposed solutions I have seen. I do not claim that this is exhaustive, just only limited to the proposals that I have stumbled across.
FASP - Fediverse Auxiliary Service Provider
The idea that mastodon has is to have a third-party service that does something like generate a link preview once and then share that to fediverse instances that need it. They have a whole spec they wrote up about these kinds of services that can be run to help out fediverse instances with different things like this, search, or discoverability. They call these FASPs - the repo with the spec is here.
I don't yet know of any FASP that is live and running, so it is kind of a paper spec at the moment. I don't do too much on the microblogging side of the fediverse though, so I very well could be wrong.
Federate link metadata and the post at the same time
This comes from an issue on PieFed's repo. It is basically just an idea at this point, but it is a pretty basic one; have the originating instance fetch the link preview information and simply federate it out alongside the post itself.
That second solution is what I thought of as doing as I read this. It doesn’t rely on a third party service being up. And that instance has to do that work anyway. So why not just share the results with others?
load more comments
(12 replies)
I'm a fan of the second one. My thought is, if you trust a server enough to accept messages from it, then you should also trust it enough to let it scrape the page title, article image, etc. for you, too.
load more comments
(5 replies)
load more comments
(9 replies)
I added a rule in my .htaccess for their user agents to direct all requests to a static page on my domain.
So we need people to just mass post links to bad websites, is what you're saying.
i'm feeling less ambivalent about those reddit repost bots for a moment
Is this the same thing as the “Reddit hug of death”?
Similar, except those are caused by users visiting a link, whereas the Fediverse issue is federated servers gathering info before users even see a link.
How many servers are we talking about? 10,000 ?
Is there a reason why the publishing instance is not downloading the preview image, and then submits it with the post to the other instances? This way they would not have to fetch it themselves.
Yeah, thats how it should be: one download, everyone else downloads from that server, not the original source.
Seems abusable though
It is, but that's what defederation is for: If there are instances that are being sketchy, they are supposed to get cut off from the rest of the Fediverse.
Seems abusable though
If one instance doesn't trust another, they shouldn't federate in the first place.
load more comments
(6 replies)
it seems like ideally it shouldn't attempt to preview the link before a user tries to view the content. i guess that would increase latency for users though
Sure would.
I have my blog address (which goes to my home server) in my Mastodon profile, and every time I post on Masto, my server gets a request from every Mastodon site that federates with my instance - i.e. hundreds of requests. It's no problem for my server, because my blog is a static site, but for anything that dynamically generates every single page impression (like worpress or joomla), it could be a problem. Caching settings might help, I suppose.
Truth is, though, I get far more traffic from vulnerability scanners and AI crawlers that don't honour robots.txt (or aren't yet listed).
that don’t honour robots.txt (or aren’t yet listed).
Throw'em in the tar pit.
Limited mental capacity these days, so I don't have the spoons to properly research and implement a solution at the moment. I installed Nepenthes last year, but I've never managed to get round to integrating it into my site. It's low priority at the moment.
load more comments
(4 replies)
Lemmy does this to itself, too - when you create a post it federates out and then every Lemmy instance that receives it makes a GET request to retrieve the post from the original server. Even though it just received the exact same data. Some other fedi software is even worse - Akkoma does a GET of the id of every activity it receives, not just for posts but for Likes too.
It's insane.
Anyway, proper caching at the reverse proxy / Cloudflare level solves it. The same kind of caching setup would do the same for sites serving HTML, like NPR too.
load more comments
(4 replies)
I mean, just how many instances are gonna ping the URL? Several hundred, low thousands at worst?
Unless it happens to be some very large media content or a dynamic page with no caching, I'd think the automated federation hits shouldn't be too bad.
The influx of users that might come after a viral post though, that's rough....
It's just bad design. What happens when/if the fediverse scales up to many more instances?
load more comments
(10 replies)
Yeah but we can't say we want to encourage people to have/build/share independent smaller websites then beat them up when they are linked. That's a huge mismatch of morality. Either we want to encourage people off the big platforms and should make it easier for the little guys or we are blowing magic smoke.
load more comments
(2 replies)
Randomly stagger the times of each instance's request.
+1-300 seconds
load more comments
(2 replies)
view more: next ›