Dryusdan 237a6a98e2 Bump ai.robots.txt to v1.39 (#982)		2025-08-18 06:52:23 -04:00
..
irc-bots	fix(bots/phrik): add IPv6 address for phrik (#494)	2025-05-11 14:04:44 -04:00
_deny-pathological.yaml	feat(default-rules): add weight to Custom-AsyncHttpClient (#914)	2025-07-27 00:41:43 +00:00
aggressive-brazilian-scrapers.yaml	feat(lib): implement request weight (#621)	2025-06-09 15:25:04 -04:00
ai-catchall.yaml	chore(default-config): allowlist common crawl (#753)	2025-07-04 00:10:45 +00:00
ai-robots-txt.yaml	Bump ai.robots.txt to v1.39 (#982)	2025-08-18 06:52:23 -04:00
cloudflare-workers.yaml	feat(lib): implement request weight (#621)	2025-06-09 15:25:04 -04:00
custom-async-http-client.yaml	feat(default-rules): add weight to Custom-AsyncHttpClient (#914)	2025-07-27 00:41:43 +00:00
headless-browsers.yaml	feat: enable loading config fragments (#321)	2025-04-23 07:01:28 -04:00
us-ai-scraper.yaml	feat: enable loading config fragments (#321)	2025-04-23 07:01:28 -04:00