sophie/nuke
1
0
Fork 0
Code Packages 1 Activity Actions

Add Applebot definition (#589)

Browse source 
* Add Applebot definition

Adds Apple's search indexing bot, and allowlists it by default.

Allowlisted by default because it is equivalent to Googlebot/Bingbot. Remove Applebot from `ai-robots-txt.yaml` for the same reasons.

Remove `Applebot-Extended` from `ai-robots-txt.yaml` as it has no effect.

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
This commit is contained in:
Corry Haines 2025-05-31 07:18:32 -07:00 • committed by GitHub
parent fbbab5a035
commit 68a71c6a99
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 28 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/docs/CHANGELOG.md
View file

@ -40,6 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added OpenRC init.d script.
- Added `--version` flag.
- Added `anubis_proxied_requests_total` metric to count proxied requests.
- Add `Applebot` as "good" web crawler
## v1.18.0: Varis zos Galvus