* chore(deps): update dependencies in go.mod and go.sum
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* refactor: rename variables for clarity in anubis.go and main.go
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* fix(checker): handle error when inserting IP range in ranger
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* fix(tests): simplify boolean checks in header and URL value tests
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* refactor(api): remove unused /test-error endpoint and restrict /make-challenge to development
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* build(deps): update golang-set to v2.8.0 in go.sum
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* Update metadata
check-spelling run (pull_request) for json/stuff
Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
on-behalf-of: @check-spelling <check-spelling-bot@check-spelling.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
* feat(lib/challenge): HTTP meta refresh challenge method
Closes#95
This challenge method enables users that don't (or won't) support
JavaScript to pass Anubis challenges. It works by using HTML meta
refresh directives to ensure that the client is a browser.
This is OFF by default. In order to enable it, an administrator MUST
choose to make the default challenge method `metarefresh`.
TODO(Xe):
- [ ] Documentation on this challenge method
- [ ] Amend wording around Anubis being a proof of work proxy in the docs
- [ ] Add configuration file syntax for the default challenge method and settings
- [ ] Test with early customers
Signed-off-by: Xe Iaso <me@xeiaso.net>
* chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(lib/challenge/metarefresh): use this value of err
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: add metarefresh challenge info, Web AI Firewall Utility
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
* Split up AI filtering files
Create aggressive/moderate/permissive policies to allow administrators to choose their AI/LLM stance.
Aggressive policy matches existing default in Anubis.
Removes `Google-Extended` flag from `ai-robots-txt.yaml` as it doesn't exist in requests.
Rename `ai-robots-txt.yaml` to `ai-catchall.yaml` as the file is no longer a copy of the source repo/file.
* chore: spelling
* chore: fix embeds
* chore: fix data includes
* chore: fix file name typo
* chore: Ignore READMEs in configs
* chore(lib/policy/config): go tool goimports -w
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* Add Applebot definition
Adds Apple's search indexing bot, and allowlists it by default.
Allowlisted by default because it is equivalent to Googlebot/Bingbot. Remove Applebot from `ai-robots-txt.yaml` for the same reasons.
Remove `Applebot-Extended` from `ai-robots-txt.yaml` as it has no effect.
* chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
For some reason, Google Chrome will randomly send a "full"
Accept-Language header, and other times it will send a "partial"
Accept-Language header. This makes the challenge construction
inconsistent.
This commit fixes this issue by only considering up to the first five
characters of the Accept-Language header when making a challenge string.
Signed-off-by: Xe Iaso <me@xeiaso.net>
Closes#565
The page already had the version number embedded into it, but that was
not printed to the page. This prints the version number set at compile
time to the page.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(expression): add validation for empty ExpressionOrList
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* fix(imports): block empty file imports with improved error checking logic
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
* docs(expression): improve validation to error on empty CEL expressions
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
---------
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Closes#531
This changes `anubis_challenges_issued` to be a vector counter that
records the challenge issuance method.
Signed-off-by: Xe Iaso <me@xeiaso.net>
Closes#520
For some reason, Chrome and Firefox are very picky over what they use to
match cookies that need to be deleted. Listen to me for my tale of woe:
The basic problem here is that cookies were an early hack added on the
side of the HTTP spec and they're basically impossible to upgrade or
change because who knows what relies on the exact behavior cookies use.
As a result, cookies don't just match by name, but by every setting that
exists on them. You can also have two cookies with the same name but
different values. This spec is a nightmare lol.
Even more fun: browsers will make up values for cookies if they aren't
set, meaning that getting a challenge token at `/docs` is semantically
different than a challenge token you got from `/`.
This PR fixes this issue by explicitly setting the "make sure cookie
support is working" cookie's path to `/`, meaning that it will always be
sent. Additionally, cookies are expired by setting the expiry time to
one minute in the past.
Hopefully this will fix it. I'm testing this locally and it seems to
work fine.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* feat(lib): ensure that clients store cookies
If a client is misconfigured and does not store cookies, then they can
get into a proof of work death spiral with Anubis. This fixes the
problem by setting a test cookie whenever the user gets hit with a
challenge page. If the test cookie is not there at challenge pass time,
then they are blocked. Administrators will also get a log message
explaining that the user intentionally broke cookie support and that this
behavior is not an Anubis bug.
Additionally, this ensures that clients being shown a challenge support
gzip-compressed responses by showing the challenge page at gzip level 1.
This level is intentionally chosen in order to minimize system impacts.
The ClearCookie function is made more generic to account for cookie
names as an argument. A correlating SetCookie function was also added to
make it easier to set cookies.
* chore(lib): clean up test code
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Previously this made ClearCookie always clear cookies by name even when
CookieDomain was set. This change fixes this and adds tests to make sure
that this doesn't happen again.
Signed-off-by: Xe Iaso <me@xeiaso.net>
Also properly re-brand the cookies so that some of the /x/ heritage is
lost.
This will invalidate existing cookies and probably affects tests.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* test(playwright): Add support to run tests in Docker/Podman
* fix command name
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Henri Vasserman <henv@hot.ee>
* up the pw version as it is in package.json
* add convenience npm scripts
* chore: changelog update
Also removed a period from my other item.
* chore: fix spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Henri Vasserman <henv@hot.ee>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
* Overhaul anubis.freebsd
Some changes here to reflect the discussion in pull request 274 regarding the `anubis_env`, `anubis_env_file` and `anubis_args` variables.
At the risk of improving personal choices in configuration with a minor amount more complexity, this new script now allows for the use of all three of these, together, with no interference between them all
i.e.
- if `anubis_env_file` is set, environment variables will be taken from this file
- if `anubis_env` is set, environment variables will be taken from this string of variables, and override matching variables set in `anubis_env_file`
- if `anubis_args` is set, runtime parameters will be taken from this string and override matching ones in both `anubis_env_file` and `anubis_env`
Thanks to @dlangille for the advice with this.
Signed-off-by: Paul Wilde <31094984+pswilde@users.noreply.github.com>
* Update CHANGELOG.md
Signed-off-by: Paul Wilde <31094984+pswilde@users.noreply.github.com>
* Remove unnecessary comment line
Signed-off-by: Paul Wilde <31094984+pswilde@users.noreply.github.com>
* Correct helper information for anubis_env_file
Signed-off-by: Paul Wilde <31094984+pswilde@users.noreply.github.com>
---------
Signed-off-by: Paul Wilde <31094984+pswilde@users.noreply.github.com>
The big ticket feature in this release is [CEL expression matching support](https://anubis.techaro.lol/docs/admin/configuration/expressions). This allows you to tailor your approach for the individual services you are protecting.
These can be as simple as:
```yaml
- name: allow-api-requests
action: ALLOW
expression:
all:
- '"Accept" in headers'
- 'headers["Accept"] == "application/json"'
- 'path.startsWith("/api/")'
```
Or as complicated as:
```yaml
- name: allow-git-clients
action: ALLOW
expression:
all:
- >-
(
userAgent.startsWith("git/") ||
userAgent.contains("libgit") ||
userAgent.startsWith("go-git") ||
userAgent.startsWith("JGit/") ||
userAgent.startsWith("JGit-")
)
- '"Git-Protocol" in headers'
- headers["Git-Protocol"] == "version=2"
```
The docs have more information, but here's a tl;dr of the variables you have access to in expressions:
| Name | Type | Explanation | Example |
| :-------------- | :-------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------- |
| `headers` | `map[string, string]` | The [headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) of the request being processed. | `{"User-Agent": "Mozilla/5.0 Gecko/20100101 Firefox/137.0"}` |
| `host` | `string` | The [HTTP hostname](https://web.dev/articles/url-parts#host) the request is targeted to. | `anubis.techaro.lol` |
| `method` | `string` | The [HTTP method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods) in the request being processed. | `GET`, `POST`, `DELETE`, etc. |
| `path` | `string` | The [path](https://web.dev/articles/url-parts#pathname) of the request being processed. | `/`, `/api/memes/create` |
| `query` | `map[string, string]` | The [query parameters](https://web.dev/articles/url-parts#query) of the request being processed. | `?foo=bar` -> `{"foo": "bar"}` |
| `remoteAddress` | `string` | The IP address of the client. | `1.1.1.1` |
| `userAgent` | `string` | The [`User-Agent`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent) string in the request being processed. | `Mozilla/5.0 Gecko/20100101 Firefox/137.0` |
This will be made more elaborate in the future. Give me time. This is a [simple, lovable, and complete](https://longform.asmartbear.com/slc/) implementation of this feature so that administrators can get hacking ASAP.
Other changes:
- Use CSS variables to deduplicate styles
- Fixed native packages not containing the stdlib and botPolicies.yaml
- Change import syntax to allow multi-level imports
- Changed the startup logging to use JSON formatting as all the other logs do.
- Added the ability to do [expression matching with CEL](./admin/configuration/expressions.mdx)
- Add a warning for clients that don't store cookies
- Disable Open Graph passthrough by default ([#435](https://github.com/TecharoHQ/anubis/issues/435))
- Clarify the license of the mascot images ([#442](https://github.com/TecharoHQ/anubis/issues/442))
- Started Suppressing 'Context canceled' errors from http in the logs ([#446](https://github.com/TecharoHQ/anubis/issues/446))
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(js): use pure JS SHA256 library, refactor
Closes#458
Additionally, I made a horrifying discovery: Firefox seems to actively
hinder performance if you are using more than one Worker per page. It
does not spread the load out across cores like I expected. Instead it
seems to make that one Worker thrash and have to constantly context
switch, which caused a lot of slowdown.
The benchmarks in #155 continue to be the best contribution ever made to
Anubis. What clued me into there being a problem here was the fact that
the "slow" algorithm was faster than the "fast" algorithm on my laptop.
This made no intuitive sense to me so I dug further.
Either way I think this is a Firefox bug at its core, but for now we
have to work around it by doing the hacky terrible thing that I hate.
I also swapped the SHA256 operations to @aws-crypto/sha256-js on the
advice of a trusted cryptography expert. I don't know what performance
differences this makes, but I'm getting 150-225 kilohashes per second,
which is pretty dang good.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(js): apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(js): use fast algo for fast worker
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat(lib/policy): add support for CEL checkers
This adds the ability for administrators to use Common Expression
Language[0] (CEL) for more advanced check logic than Anubis previously
offered.
These can be as simple as:
```yaml
- name: allow-api-routes
action: ALLOW
expression:
and:
- '!(method == "HEAD" || method == "GET")'
- path.startsWith("/api/")
```
or get as complicated as:
```yaml
- name: allow-git-clients
action: ALLOW
expression:
and:
- userAgent.startsWith("git/") || userAgent.contains("libgit") || userAgent.startsWith("go-git") || userAgent.startsWith("JGit/") || userAgent.startsWith("JGit-")
- >
"Git-Protocol" in headers && headers["Git-Protocol"] == "version=2"
```
Internally these are compiled and evaluated with cel-go[1]. This also
leaves room for extensibility should that be desired in the future. This
will intersect with #338 and eventually intersect with TLS fingerprints
as in #337.
[0]: https://cel.dev/
[1]: https://github.com/google/cel-go
Signed-off-by: Xe Iaso <me@xeiaso.net>
* feat(data/apps): add API route allow rule for non-HEAD/GET
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: document expression syntax
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix: fixes in review
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
* deduplicate css rules by using media query to set variables
* Update xess/xess.css
Co-authored-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Benjamin Armintor <armintor@gmail.com>
---------
Signed-off-by: Benjamin Armintor <armintor@gmail.com>
Co-authored-by: Xe Iaso <me@xeiaso.net>
- Added customization of authorization cookie expiration time with `--cookie-expiration-time` flag or envvar
- Updated the `OG_PASSTHROUGH` to be true by default, thereby allowing OpenGraph tags to be passed through by default
- Added the ability to [customize Anubis' HTTP status codes](./admin/configuration/custom-status-codes.mdx) ([#355](https://github.com/TecharoHQ/anubis/issues/355))
Signed-off-by: Xe Iaso <me@xeiaso.net>
