Implement FCrDNS and other DNS features (#1308)
* Implement FCrDNS and other DNS features * Redesign DNS cache and methods * Fix DNS cache * Rename regexSafe arg * Alter verifyFCrDNS(addr) behaviour * Remove unused dnsCache field from Server struct * Upd expressions docs * Update docs/docs/CHANGELOG.md Signed-off-by: Xe Iaso <me@xeiaso.net> * refactor(dns): simplify FCrDNS logging * docs: clarify verifyFCrDNS behavior Add a note to the documentation for `verifyFCrDNS` to clarify that it returns true when no PTR records are found for the given IP address. * fix(dns): Improve FCrDNS error handling and tests The `VerifyFCrDNS` function previously ignored errors returned from reverse DNS lookups. This could lead to incorrect passes when a DNS failure (other than a simple 'not found') occurred. This change ensures that any error from a reverse lookup will cause the FCrDNS check to fail. The test suite for FCrDNS has been updated to reflect this change. The mock DNS lookups now simulate both 'not found' errors and other generic DNS errors. The test cases have been updated to ensure that the function behaves correctly in both scenarios, resolving a situation where two test cases were effectively duplicates. * docs: Update FCrDNS documentation and spelling Corrected a typo in the `verifyFCrDNS` function documentation. Additionally, updated the spelling exception list to include new terms and remove redundant entries. * chore: update spelling Signed-off-by: Xe Iaso <me@xeiaso.net> --------- Signed-off-by: Xe Iaso <me@xeiaso.net> Co-authored-by: Xe Iaso <me@xeiaso.net>
This commit is contained in:
The Ninth
GitHub
parent
4ead3ed16e
commit
00fa939acf
22 changed files with 1652 additions and 480 deletions
|
|
@ -23,7 +23,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
- Add support to simple Valkey/Redis cluster mode
|
||||
- Open Graph passthrough now reuses the configured target Host/SNI/TLS settings, so metadata fetches succeed when the upstream certificate differs from the public domain. ([1283](https://github.com/TecharoHQ/anubis/pull/1283))
|
||||
- Stabilize the CVE-2025-24369 regression test by always submitting an invalid proof instead of relying on random POW failures.
|
||||
- Add Polish locale ([#1292](https://github.com/TecharoHQ/anubis/pull/1309))
|
||||
|
||||
### Deprecate `report_as` in challenge configuration
|
||||
|
||||
|
|
@ -81,6 +80,31 @@ logging:
|
|||
|
||||
Additionally, information about [how Anubis uses each logging level](./admin/policies.mdx#log-levels) has been added to the documentation.
|
||||
|
||||
### DNS Features
|
||||
|
||||
- CEL expressions for:
|
||||
- FCrDNS checks
|
||||
- Forward DNS queries
|
||||
- Reverse DNS queries
|
||||
- `arpaReverseIP` to transform IPv4/6 addresses into ARPA reverse IP notation.
|
||||
- `regexSafe` to escape regex special characters (useful for including `remoteAddress` or headers in regular expressions).
|
||||
- DNS cache and other optimizations to minimize unnecessary DNS queries.
|
||||
|
||||
The DNS cache TTL can be changed in the bots config like this:
|
||||
```yaml
|
||||
dns_ttl:
|
||||
forward: 600
|
||||
reverse: 600
|
||||
```
|
||||
The default value for both forward and reverse queries is 300 seconds.
|
||||
|
||||
The `verifyFCrDNS` CEL function has two overloads:
|
||||
- `(addr)`
|
||||
Simply verifies that the remote side has PTR records pointing to the target address.
|
||||
- `(addr, ptrPattern)`
|
||||
Verifies that the remote side refers to a specific domain and that this domain points to the target IP.
|
||||
|
||||
|
||||
## v1.23.1: Lyse Hext - Echo 1
|
||||
|
||||
- Fix `SERVE_ROBOTS_TXT` setting after the double slash fix broke it.
|
||||
|
|
|
|||
|
|
@ -233,6 +233,27 @@ This is best applied when doing explicit block rules, eg:
|
|||
|
||||
It seems counter-intuitive to allow known bad clients through sometimes, but this allows you to confuse attackers by making Anubis' behavior random. Adjust the thresholds and numbers as facts and circumstances demand.
|
||||
|
||||
### `regexSafe`
|
||||
|
||||
Available in `bot` expressions.
|
||||
|
||||
```ts
|
||||
function regexSafe(input: string): string;
|
||||
```
|
||||
|
||||
`regexSafe` takes a string and escapes it for safe use inside of a regular expression. This is useful when you are creating regular expressions from headers or variables such as `remoteAddress`.
|
||||
|
||||
| Input | Output |
|
||||
| :------------------------ | :------------------------------ |
|
||||
| `regexSafe("1.2.3.4")` | `1\\.2\\.3\\.4` |
|
||||
| `regexSafe("techaro.lol")` | `techaro\\.lol` |
|
||||
| `regexSafe("star*")` | `star\\*` |
|
||||
| `regexSafe("plus+")` | `plus\\+` |
|
||||
| `regexSafe("{braces}")` | `\\{braces\\}` |
|
||||
| `regexSafe("start^")` | `start\\^` |
|
||||
| `regexSafe("back\\slash")` | `back\\\\slash` |
|
||||
| `regexSafe("dash-dash")` | `dash\\-dash` |
|
||||
|
||||
### `segments`
|
||||
|
||||
Available in `bot` expressions.
|
||||
|
|
@ -266,6 +287,99 @@ This is useful if you want to write rules that allow requests that have no query
|
|||
- size(segments(path)) < 2
|
||||
```
|
||||
|
||||
### DNS Functions
|
||||
|
||||
Anubis can also perform DNS lookups as a part of its expression evaluation. This can be useful for doing things like checking for a valid [Forward-confirmed reverse DNS (FCrDNS)](https://en.wikipedia.org/wiki/Forward-confirmed_reverse_DNS) record.
|
||||
|
||||
#### `arpaReverseIP`
|
||||
|
||||
Available in `bot` expressions.
|
||||
|
||||
```ts
|
||||
function arpaReverseIP(ip: string): string;
|
||||
```
|
||||
|
||||
`arpaReverseIP` takes an IP address and returns its value in [ARPA notation](https://www.ietf.org/rfc/rfc2317.html). This can be useful when matching PTR record patterns.
|
||||
|
||||
| Input | Output |
|
||||
| :----------------------------- | :------------------------------------------------------------------- |
|
||||
| `arpaReverseIP("1.2.3.4")` | `4.3.2.1` |
|
||||
| `arpaReverseIP("2001:db8::1")` | `1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2` |
|
||||
|
||||
#### `lookupHost`
|
||||
|
||||
Available in `bot` expressions.
|
||||
|
||||
```ts
|
||||
function lookupHost(host: string): string[];
|
||||
```
|
||||
|
||||
`lookupHost` performs a DNS lookup for the given hostname and returns a list of IP addresses.
|
||||
|
||||
```yaml
|
||||
- name: cloudflare-ip-in-host-header
|
||||
action: DENY
|
||||
expression: '"104.16.0.0" in lookupHost(headers["Host"])'
|
||||
```
|
||||
|
||||
#### `reverseDNS`
|
||||
|
||||
Available in `bot` expressions.
|
||||
|
||||
```ts
|
||||
function reverseDNS(ip: string): string[];
|
||||
```
|
||||
|
||||
`reverseDNS` takes an IP address and returns the DNS names associated with it. This is useful when you want to check PTR records of an IP address.
|
||||
|
||||
```yaml
|
||||
- name: allow-googlebot
|
||||
action: ALLOW
|
||||
expression: 'reverseDNS(remoteAddress).endsWith(".googlebot.com")'
|
||||
```
|
||||
|
||||
::: warning
|
||||
|
||||
Do not use this for validating the legitimacy of an IP address. It is possible for DNS records to be out of date or otherwise manipulated. Use [`verifyFCrDNS`](#verifyfcrdns) instead for a more reliable result.
|
||||
|
||||
:::
|
||||
|
||||
#### `verifyFCrDNS`
|
||||
|
||||
Available in `bot` expressions.
|
||||
|
||||
```ts
|
||||
function verifyFCrDNS(ip: string): bool;
|
||||
function verifyFCrDNS(ip: string, pattern: string): bool;
|
||||
```
|
||||
|
||||
`verifyFCrDNS` checks if the reverse DNS of an IP address matches its forward DNS. This is a common technique to filter out spam and bot traffic. `verifyFCrDNS` comes in two forms:
|
||||
|
||||
- `verifyFCrDNS(remoteAddress)` will check that the reverse DNS of the remote address resolves back to the remote address. If no PTR records, returns true.
|
||||
- `verifyFCrDNS(remoteAddress, pattern)` will check that the reverse DNS of the remote address is matching with pattern and that name resolves back to the remote address.
|
||||
|
||||
This is best used in rules like this:
|
||||
|
||||
```yaml
|
||||
- name: require-fcrdns-for-post
|
||||
action: DENY
|
||||
expression:
|
||||
all:
|
||||
- method == "POST"
|
||||
- "!verifyFCrDNS(remoteAddress)"
|
||||
```
|
||||
|
||||
Here is an another example that allows requests from telegram:
|
||||
|
||||
```yaml
|
||||
- name: telegrambot
|
||||
action: ALLOW
|
||||
expression:
|
||||
all:
|
||||
- userAgent.matches("TelegramBot")
|
||||
- verifyFCrDNS(remoteAddress, "ptr\\.telegram\\.org$")
|
||||
```
|
||||
|
||||
## Life advice
|
||||
|
||||
Expressions are very powerful. This is a benefit and a burden. If you are not careful with your expression targeting, you will be liable to get yourself into trouble. If you are at all in doubt, throw a `CHALLENGE` over a `DENY`. Legitimate users can easily work around a `CHALLENGE` result with a [proof of work challenge](../../design/why-proof-of-work.mdx). Bots are less likely to be able to do this.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue