* feat: replace cidranger with bart improving performance by 3-20x
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* perf: replace cidranger with bart for IP range checking
- Replace cidranger.Ranger with bart.Lite in RemoteAddrChecker
- Use netip.ParsePrefix instead of net.ParseCIDR for modern IP handling
- Improve performance: 3-20x faster lookups with zero heap allocations
- Update imports to use github.com/gaissmai/bart and net/netip
- Remove cidranger dependency from go.mod
Benchmark results:
- IPv4 lookups: 4x faster (15.58ns vs 63.25ns, 0 vs 2 allocs)
- IPv6 lookups: 3x faster (26.51ns vs 76.96ns, 0 vs 2 allocs)
- Insertions: 20x faster (976ns vs 19,191ns)
- Large tables: 14x faster (5.2ns vs 74.85ns)
* docs: clarify CHANGELOG to not give false impressions
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* perf: optimize string concatenation in RemoteAddrChecker hash generation
Replace fmt.Fprintln with strings.Join for 7x faster performance:
- Before: 935.1 ns/op, 784 B/op, 22 allocs/op
- After: 133.2 ns/op, 192 B/op, 1 alloc/op
The hash is used for JWT cookie validation and error code generation.
Comma separation provides the same deterministic uniqueness as newlines
but with significantly better performance during policy initialization.
* chore: remove accidentally commited string benchmark
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* style: apply Copilot suggestions
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* fix: reference the right var name
i cannot write a merge commit
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* refactor(ogtags): optimize URL construction and memory allocations
* test(ogtags): add benchmarks and memory usage tests for OGTagCache
* refactor(ogtags): optimize OGTags subsystem to reduce allocations and improve request runtime by up to 66%
* Update docs/docs/CHANGELOG.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com>
* refactor(ogtags): optimize URL string construction to reduce allocations
* Update internal/ogtags/ogtags.go
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com>
* test(ogtags): add fuzz tests for getTarget and extractOGTags functions
* fix(ogtags): update memory calculation logic
Prev it would say that we had allocated 18pb
=== RUN TestMemoryUsage
mem_test.go:107: Memory allocated for 10k getTarget calls: 18014398509481904.00 KB
mem_test.go:135: Memory allocated for 1k extractOGTags calls: 18014398509481978.00
Now it's fixed with
=== RUN TestMemoryUsage
mem_test.go:109: Memory allocated for 10k getTarget calls:
mem_test.go:110: Total: 630.56 KB (0.62 MB)
mem_test.go:111: Per operation: 64.57 bytes
mem_test.go:140: Memory allocated for 1k extractOGTags calls:
mem_test.go:141: Total: 328.17 KB (0.32 MB)
mem_test.go:142: Per operation: 336.05 bytes
* refactor(ogtags): optimize meta tag extraction for improved performance
* Update metadata
check-spelling run (pull_request) for json/ogmem
Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
on-behalf-of: @check-spelling <check-spelling-bot@check-spelling.dev>
* chore: update CHANGELOG for recent optimizations and version bump
* refactor: improve URL construction and meta tag extraction logic
* style: cleanup fuzz tests
---------
Signed-off-by: Jason Cameron <jasoncameron.all@gmail.com>
Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* feat(lib): implement request weight
Replaces #608
This is a big one and will be what makes Anubis a generic web
application firewall. This introduces the WEIGH option, allowing
administrators to have facets of request metadata add or remove
"weight", or the level of suspicion. This really makes Anubis weigh
the soul of requests.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(lib): maintain legacy challenge behavior
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(lib): make weight have dedicated checkers for the hashes
Signed-off-by: Xe Iaso <me@xeiaso.net>
* feat(data): convert some rules over to weight points
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: document request weight
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(CHANGELOG): spelling error
Signed-off-by: Xe Iaso <me@xeiaso.net>
* chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: fix links to challenge information
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs(policies): fix formatting
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(config): make default weight adjustment 5
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
* feat(lib/challenge): HTTP meta refresh challenge method
Closes#95
This challenge method enables users that don't (or won't) support
JavaScript to pass Anubis challenges. It works by using HTML meta
refresh directives to ensure that the client is a browser.
This is OFF by default. In order to enable it, an administrator MUST
choose to make the default challenge method `metarefresh`.
TODO(Xe):
- [ ] Documentation on this challenge method
- [ ] Amend wording around Anubis being a proof of work proxy in the docs
- [ ] Add configuration file syntax for the default challenge method and settings
- [ ] Test with early customers
Signed-off-by: Xe Iaso <me@xeiaso.net>
* chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(lib/challenge/metarefresh): use this value of err
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: add metarefresh challenge info, Web AI Firewall Utility
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
* Split up AI filtering files
Create aggressive/moderate/permissive policies to allow administrators to choose their AI/LLM stance.
Aggressive policy matches existing default in Anubis.
Removes `Google-Extended` flag from `ai-robots-txt.yaml` as it doesn't exist in requests.
Rename `ai-robots-txt.yaml` to `ai-catchall.yaml` as the file is no longer a copy of the source repo/file.
* chore: spelling
* chore: fix embeds
* chore: fix data includes
* chore: fix file name typo
* chore: Ignore READMEs in configs
* chore(lib/policy/config): go tool goimports -w
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* Define OpenAI bot ALLOW policies
Allows OpenAI bots to be allowlisted at the choice of the Anubis administrator. None are enabled by default.
* Define MistralAI bot ALLOW policy
* chore: spelling
* Add Applebot definition
Adds Apple's search indexing bot, and allowlists it by default.
Allowlisted by default because it is equivalent to Googlebot/Bingbot. Remove Applebot from `ai-robots-txt.yaml` for the same reasons.
Remove `Applebot-Extended` from `ai-robots-txt.yaml` as it has no effect.
* chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* feat(lib): ensure that clients store cookies
If a client is misconfigured and does not store cookies, then they can
get into a proof of work death spiral with Anubis. This fixes the
problem by setting a test cookie whenever the user gets hit with a
challenge page. If the test cookie is not there at challenge pass time,
then they are blocked. Administrators will also get a log message
explaining that the user intentionally broke cookie support and that this
behavior is not an Anubis bug.
Additionally, this ensures that clients being shown a challenge support
gzip-compressed responses by showing the challenge page at gzip level 1.
This level is intentionally chosen in order to minimize system impacts.
The ClearCookie function is made more generic to account for cookie
names as an argument. A correlating SetCookie function was also added to
make it easier to set cookies.
* chore(lib): clean up test code
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
* feat(ci): use dynamic repository owner and name in Docker actions
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* feat(ci): support forks
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* feat(ci): support forks
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* feat(ci): add debug output for Docker repository information
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* feat(ci): update Docker image naming convention in workflow
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* feat(ci): set lowercase image name in Docker workflow
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* feat(ci): remove json/gha branch from Docker workflow triggers
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* feat(ci): simplify Docker registry configuration in workflow
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
This means that yeet's version will be managed by `go.mod` and
auto-bumped with dependabot. This removes human error from the equation
and ensures that Anubis is always built with the newest version of yeet.
This also makes it trivial to make your own local packages for testing:
```text
go tool yeet
```
Signed-off-by: Xe Iaso <me@xeiaso.net>
* test(playwright): Add support to run tests in Docker/Podman
* fix command name
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Henri Vasserman <henv@hot.ee>
* up the pw version as it is in package.json
* add convenience npm scripts
* chore: changelog update
Also removed a period from my other item.
* chore: fix spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Henri Vasserman <henv@hot.ee>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
Closes#333
I'm very bad at design so I just picked colors that looked reasonable
enough to me. Hopefully this will be enough to get us to the next stage!
Signed-off-by: Xe Iaso <me@xeiaso.net>
