nuke/data/crawlers
Xe Iaso c96c229b68
feat(default-config): block tencent cloud by default (#1216)
* feat(default-config): block tencent cloud by default

This is what happens when you don't have an abuse contact.

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: update spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-10-24 19:43:42 +00:00
..
_allow-good.yaml chore(default-config): allowlist common crawl (#753) 2025-07-04 00:10:45 +00:00
ai-search.yaml Split up AI filtering files (#592) 2025-06-01 20:21:18 +00:00
ai-training.yaml Split up AI filtering files (#592) 2025-06-01 20:21:18 +00:00
alibaba-cloud.yaml fix(default-config): also block alibaba cloud (#1005) 2025-08-20 23:01:49 +00:00
applebot.yaml Add Applebot definition (#589) 2025-05-31 10:18:32 -04:00
bingbot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
commoncrawl.yaml chore(default-config): allowlist common crawl (#753) 2025-07-04 00:10:45 +00:00
duckduckbot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
googlebot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
huawei-cloud.yaml fix(default-config): block Huawei Cloud (#1004) 2025-08-20 22:40:07 +00:00
internet-archive.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
kagibot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
marginalia.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
mojeekbot.yaml Fix: mojeekbot regex (#351) 2025-04-24 14:24:41 +00:00
openai-gptbot.yaml Opt-in policies for OpenAI and MistralAI bots (#590) 2025-05-31 16:48:57 -04:00
openai-searchbot.yaml Opt-in policies for OpenAI and MistralAI bots (#590) 2025-05-31 16:48:57 -04:00
qwantbot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
tencent-cloud.yaml feat(default-config): block tencent cloud by default (#1216) 2025-10-24 19:43:42 +00:00