Will check this out. Thanks!

That’s pretty neat. Thanks!

Will check this out. Thanks!

Thank you for the detailed reply.

keeping on top of this is a full time job!

I guess that’s why I’m interested in a tooling based solution. My selfhosting is small-fry junk, but a lot of others like me are hosting entire fedi communities or larger websites.

In that case I’m interested in tools to automate doing that.

I hadn’t heard of that before, thanks for the link.

I haven’t read through the docs yet… But PoW makes me wonder what the work is and if it’s cryptocurrency related.

Edit: Found it: https://altcha.org/docs/proof-of-work/

In the hackernews comments for that geraspora link people discussed websites shutting down due to hosting costs, which may be attributed in part to the overly aggressive crawling. So maybe it’s just a different form of DDOS than we’re used to.

A commenter in the hackernews post has created this: https://marcusb.org/hacks/quixotic.html

I’m interested, but it seems like an easy way for bots to exhaust your own server resources before they give up crawling.

Thank you for the detailed response. It’s disheartening to consider the traffic is coming from ‘real’ browsers/IPs, but that actually makes a lot of sense.

I’m coming at this from the angle of AI bots ingesting a website over and over to obsessively look for new content.

My understanding is there are two reasons to try blocking this: to protect bandwidth from aggressive crawling, or to protect the page contents from AI ingestion. I think the former is doable, and the latter is an unwinnable task. My personal reason is because I’m an AI curmudgeon, I’d rather spend CPU resources blocking bots than serving any content to them.

Thank you for the reply, but at least one commenter claims they’ll impersonate Chrome UAs.

https://github.com/laylavish/uBlockOrigin-HUGE-AI-Blocklist

I didn’t know that aspect. Thanks for sharing.

Ammonia was first manufactured using the Haber process on an industrial scale in 1913 in BASF’s Oppau plant in Germany, reaching 20 tonnes/day in 1914.[12] During World War I, the production of munitions required large amounts of nitrate. The Allied powers had access to large deposits of sodium nitrate in Chile (Chile saltpetre) controlled by British companies. India had large supplies too, but it was also controlled by the British.[13] Moreover, even if German commercial interests had nominal legal control of such resources, the Allies controlled the sea lanes and imposed a highly effective blockade which would have prevented such supplies from reaching Germany. The Haber process proved so essential to the German war effort[5][14] that it is considered virtually certain Germany would have been defeated in a matter of months without it. Synthetic ammonia from the Haber process was used for the production of nitric acid, a precursor to the nitrates used in explosives.

Via Wikipedia

Quarto is probably the most unique one I’ve seen so far. Thanks for the input.

Thanks for the recommendation!

Thanks for the recommendations!

Neat, thanks!

Will give that a look, thanks!

Thanks!