Opensource

Simple agents won't understand git. They can make web requests. If they want to check for reference code/source code, I assume they would request the rendered html.

I assume the frequency comes from many people asking various things, and the agents in the background pulling this data.

I'm not sure whether the scale-to-load ratio is plausible, because I lack the numbers, but it doesn't seem implausible to me that various agents for various prompts and tasks from various providers for many people repeatedly make these requests to a degree that significantly impacts the hoster.