sophie/nuke
1
0
Fork 0
Code Packages 1 Activity Actions

Opt-in policies for OpenAI and MistralAI bots (#590)

Browse source 
* Define OpenAI bot ALLOW policies

Allows OpenAI bots to be allowlisted at the choice of the Anubis administrator. None are enabled by default.

* Define MistralAI bot ALLOW policy

* chore: spelling
This commit is contained in:
Corry Haines 2025-05-31 13:48:57 -07:00 • committed by GitHub
parent ba00cdacd2
commit 0d9ebebff6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 135 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

16
data/crawlers/openai-gptbot.yaml Normal file
View file

@ -0,0 +1,16 @@
# Collects AI training data
# https://platform.openai.com/docs/bots/overview-of-openai-crawlers
- name: openai-gptbot
user_agent_regex: GPTBot/1\.1; \+https\://openai\.com/gptbot
action: ALLOW
# https://openai.com/gptbot.json
remote_addresses: [
"52.230.152.0/24",
"20.171.206.0/24",
"20.171.207.0/24",
"4.227.36.0/25",
"20.125.66.80/28",
"172.182.204.0/24",
"172.182.214.0/24",
"172.182.215.0/24",
]