sophie/nuke
1
0
Fork 0
Code Packages 1 Activity Actions

feat(og): Foward host header (#370)

Browse source 
* feat(ogtags): enhance target URL handling for OGTagCache, support Unix sockets

Closes: #323 #319
Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* docs: update CHANGELOG.md to include Opengraph passthrough support for Unix sockets

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* docs: update CHANGELOG.md to include Opengraph passthrough support for Unix sockets

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* feat(ogtags): add option to consider host in Open Graph tag cache key

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* feat(ogtags): add option to consider host in OG tag cache key

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* test(ogtags): enhance tests for OGTagCache with host consideration scenarios

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor(ogtags): extract constants for HTTP timeout and max content length

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor(ogtags): restore fetchHTMLDocument method for cache key generation

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor(ogtags): replace maxContentLength field with constant and ensure HTTP scheme is set correctly

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* fix(fetch): add proxy headers

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

---------

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
This commit is contained in:
Jason Cameron 2025-04-29 08:20:04 -04:00 • committed by GitHub
parent 7a20a46b0d
commit 4184b42282
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
14 changed files with 484 additions and 91 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/docs/admin/installation.mdx
View file

@ -63,6 +63,7 @@ Anubis uses these environment variables for configuration:
| `METRICS_BIND_NETWORK` | `tcp` | The address family that the Anubis metrics server listens on. See `BIND_NETWORK` for more information. |
| `OG_EXPIRY_TIME` | `24h` | The expiration time for the Open Graph tag cache. |
| `OG_PASSTHROUGH` | `false` | If set to `true`, Anubis will enable Open Graph tag passthrough. |
| `OG_CACHE_CONSIDER_HOST` | `false` | If set to `true`, Anubis will consider the host in the Open Graph tag cache key. |
| `POLICY_FNAME` | unset | The file containing [bot policy configuration](./policies.mdx). See the bot policy documentation for more details. If unset, the default bot policy configuration is used. |
| `REDIRECT_DOMAINS` | unset | If set, restrict the domains that Anubis can redirect to when passing a challenge.<br/><br/>If this is unset, Anubis may redirect to any domain which could cause security issues in the unlikely case that an attacker passes a challenge for your browser and then tricks you into clicking a link to your domain. |
| `SERVE_ROBOTS_TXT` | `false` | If set `true`, Anubis will serve a default `robots.txt` file that disallows all known AI scrapers by name and then additionally disallows every scraper. This is useful if facts and circumstances make it difficult to change the underlying service to serve such a `robots.txt` file. |