nuke/data/crawlers
Xe Iaso ee55d857eb
fix(default-config): block Huawei Cloud (#1004)
* fix(default-config): block Huawei Cloud

Closes #978

Huawei Cloud has been egregious about its scraping. All attempts to
contact their abuse team have failed. If you work for Huawei Cloud,
please raise this issue internally and get the scraping to just stop.

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-08-20 22:40:07 +00:00
..
_allow-good.yaml chore(default-config): allowlist common crawl (#753) 2025-07-04 00:10:45 +00:00
ai-search.yaml Split up AI filtering files (#592) 2025-06-01 20:21:18 +00:00
ai-training.yaml Split up AI filtering files (#592) 2025-06-01 20:21:18 +00:00
applebot.yaml Add Applebot definition (#589) 2025-05-31 10:18:32 -04:00
bingbot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
commoncrawl.yaml chore(default-config): allowlist common crawl (#753) 2025-07-04 00:10:45 +00:00
duckduckbot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
googlebot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
huawei-cloud.yaml fix(default-config): block Huawei Cloud (#1004) 2025-08-20 22:40:07 +00:00
internet-archive.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
kagibot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
marginalia.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00
mojeekbot.yaml Fix: mojeekbot regex (#351) 2025-04-24 14:24:41 +00:00
openai-gptbot.yaml Opt-in policies for OpenAI and MistralAI bots (#590) 2025-05-31 16:48:57 -04:00
openai-searchbot.yaml Opt-in policies for OpenAI and MistralAI bots (#590) 2025-05-31 16:48:57 -04:00
qwantbot.yaml feat: enable loading config fragments (#321) 2025-04-23 07:01:28 -04:00