Programming

27651 readers

104 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev

founded 3 years ago

MODERATORS

snowe@programming.dev

Ategon@programming.dev

UlrikHD@programming.dev

bugsmith@programming.dev

Spyro@programming.dev

What's your setup for capturing screenshots of web apps automatically? (lemmy.world)

submitted 2 days ago by indidev@lemmy.world to c/programming@programming.dev

4 comments fedilink hide all child comments

Working on a project where I need to grab screenshots of pages on a schedule — mostly for QA and keeping a visual history of what changed and when.

Right now I'm running Puppeteer in a cron job on a cheap VPS. It works, but honestly it's a pain to maintain. Chromium eats RAM like crazy, sometimes it hangs and the whole thing needs a restart, and the screenshots come out wrong on pages that load content dynamically.

A few things I've been struggling with:

Pages with lazy-loaded images — half the time the screenshot fires before everything renders
Cookie consent banners blocking the actual content
Memory usage goes through the roof when I try to do more than ~50 pages in a batch

I've looked into Playwright as a replacement but from what I can tell the resource usage is about the same. Also tried running headless Chrome in Docker which at least makes cleanup easier, but didn't solve the core problems.

Curious what others are using. Are screenshot APIs worth it for this kind of thing, or is self-hosting still the way to go? Anyone running something similar at scale?

you are viewing a single comment's thread
view the rest of the comments

[–] fuckwit_mcbumcrumble@lemmy.dbzer0.com 1 points 2 days ago

Pages with lazy-loaded images — half the time the screenshot fires before everything renders

Are you using waitUntil? You need to adjust what it's waiting for. Networkidle2 works great for simple pages, but if there's a lot of BS that never stops firing then it may never conder it loaded. I use domcontentloaded plus a hard coded delay.

Cookie consent banners blocking the actual content

Write a quick queryselector and have it click accept if it's there.

Memory usage goes through the roof when I try to do more than ~50 pages in a batch

Not much you can do about that. Chrome just sucks. Any alternative is either going to use just as much ram, or not be representative of the real world. So you're stuck with Chrome wearing different hats. Not using shitty hardware for it is amazing. We have some hosts that are almost 15 year old hardware and they're painfully slow. But put on something modern with an ssd and it's pretty fast. You just have to keep memory under control to keep it relatively cheap. You could probably buy an old laptop and run it as a server for about a years worth of decent VPS costs. As long as it's got an ssd it'll be fast enough for this.