sophie/nuke
1
0
Fork 0
Code Packages 1 Activity Actions

Bump AI-robots.txt rules to version 1.30 (#509)

Browse source 
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
This commit is contained in:
Dryusdan 2025-05-16 18:40:25 +02:00 • committed by GitHub
parent 91c21fbb4b
commit 961320540b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 4 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

1
web/static/robots.txt
View file

@ -46,6 +46,7 @@ User-agent: PanguBot
User-agent: Perplexity-User
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: QualifiedBot
User-agent: Scrapy
User-agent: SemrushBot-OCOB
User-agent: SemrushBot-SWA