From c12a19db1310ece25e93f02f9d0a709a54b8a607 Mon Sep 17 00:00:00 2001 From: ErmiasG Date: Tue, 10 Nov 2026 16:09:22 +0100 Subject: [PATCH] [HWORKS-3754] Click jacking, cross-site scripting and other attack mitigation https://hopsworks.atlassian.net/browse/HWORKS-3754 Hopsworks sets restrictive security response headers itself, in its application server, on the UI, its REST endpoints, its error pages and the services it proxies, so they apply whatever sits in front of the cluster. Every cookie it issues or passes on carries Secure and SameSite=Lax, and its own cookies are HttpOnly. The UI runs under an enforced Content-Security-Policy, which Helm values tune without a restart. The backend adds the headers filter, the cookie handling and the image changes; the chart adds the policy values, ConfigMap and mounts; the frontend becomes CSP-compatible; the loadtest checks the headers on a cluster; the docs describe the headers, the values and the upgrade. The docs add an admin page on the security headers: the headers per kind of response, the cookie attributes, the Helm values, the report-only switch and how to use it, extra sources for model deployments on another host and for sites that frame the UI, and HSTS with its prerequisite and undo. A 5.2.x to 5.3.x upgrade note points to it. Reviewed-by: OpenAI Codex (codex-plugin-cc 1.0.4) Signed-off-by: Ermias Gebremeskel Co-Authored-By: Claude Opus 5.5 --- .../admin/security_headers.md | 103 ++++++++++++++++++ docs/setup_installation/admin/upgrade.md | 10 ++ mkdocs.yml | 1 + 3 files changed, 114 insertions(+) create mode 100644 docs/setup_installation/admin/security_headers.md diff --git a/docs/setup_installation/admin/security_headers.md b/docs/setup_installation/admin/security_headers.md new file mode 100644 index 0000000000..dcf647f6bb --- /dev/null +++ b/docs/setup_installation/admin/security_headers.md @@ -0,0 +1,103 @@ +--- +description: The security response headers and cookie attributes Hopsworks sets on every response, and the Helm values that tune its Content-Security-Policy and HSTS +--- + +# Security Response Headers { #security-headers } + +## Introduction + +Hopsworks sets security response headers itself, in its application server, on the UI, its REST endpoints, its error pages and the services it proxies, so they apply whatever sits in front of the cluster: an ingress controller, a load balancer or an OpenShift route. +Each kind of response gets its own set: + +| Response | Headers | +| --- | --- | +| The Hopsworks UI | `Content-Security-Policy`, `X-Content-Type-Options: nosniff`, `Strict-Transport-Security`, `Referrer-Policy: same-origin`, `Permissions-Policy`, `Cross-Origin-Opener-Policy: same-origin`, `Cross-Origin-Resource-Policy: same-origin`, `Cache-Control` | +| Hopsworks REST endpoints | `Content-Security-Policy: default-src 'none'; frame-ancestors 'none'`, `X-Content-Type-Options: nosniff`, `Strict-Transport-Security`, `Referrer-Policy: same-origin`, `Cross-Origin-Resource-Policy: same-origin`, `Cache-Control: no-store` unless an endpoint sets its own | +| Hopsworks error pages, such as the page that renews an expired session | The UI's headers, with `Cache-Control: no-store` | +| Services Hopsworks proxies, such as Jupyter, Grafana or Airflow | `X-Content-Type-Options: nosniff`, `Strict-Transport-Security`, `Referrer-Policy: same-origin`; the service's own `Content-Security-Policy` and `Cache-Control`, or the UI's `frame-ancestors` and `private, no-cache` when it sends none; a `304 Not Modified` gets neither fallback, so the browser keeps the headers it cached with the response | + +The UI's Content-Security-Policy allows scripts and connections from the Hopsworks origin only, frames from the Hopsworks origin, `blob:` URLs and `https://www.youtube-nocookie.com`, and lets no other site frame the UI. +Other sites cannot embed the UI's files or the REST responses, and a page of another site that the UI opens in a new tab gets no handle to the UI's window. +Browsers check the UI's entry page with Hopsworks on every load, and keep its versioned assets for a year. +No response names the application server or its version. + +Cookies set by Hopsworks or by a service it proxies carry `SameSite=Lax`, and `Secure` when the browser reached Hopsworks over HTTPS, also when TLS ends at the ingress. +Hopsworks' own cookies are also `HttpOnly`. +Cookies that the UI or a proxied service reads in the browser, such as Jupyter's `_xsrf`, keep the `HttpOnly` setting the service chose. + +## Helm values { #security-headers-values } + +The policy is set under `hopsworks.payara.securityHeaders`, listed in [Hopsworks values][helm-values-hopsworks]: + +```yaml +hopsworks: + payara: + securityHeaders: + cspMode: enforce + extraSources: + connect-src: [] + frame-ancestors: [] + hsts: + maxAge: 31536000 + includeSubDomains: true +``` + +- `cspMode`: `enforce` blocks what the Content-Security-Policy forbids. + `report-only` sends the same policy as `Content-Security-Policy-Report-Only`: the browser reports each violation in its console and blocks nothing. + The page that shows a system health report keeps an enforced policy in both modes, because it isolates the report's scripts from the UI. +- `extraSources`: sources added to a directive of the UI's policy, one list per directive. + The directives are `connect-src`, `frame-src`, `img-src`, `font-src`, `script-src`, `style-src`, `form-action` and `frame-ancestors`. + A source is one CSP source expression, such as `https://serving.example.com:32443`, without spaces, `;` or `,`. +- `hsts.maxAge` and `hsts.includeSubDomains`: the `Strict-Transport-Security` header. + The defaults match what ingress-nginx sends by default. + +The values are rendered into the `-security-headers` ConfigMap. +A `helm upgrade` that only changes these values needs no restart: once Kubernetes updates the mounted file in a pod, usually within one to two minutes, Hopsworks applies it within 30 seconds. + +## Model deployments on another host or port { #security-headers-inference } + +The UI calls model deployments, for example from the agent chat, at the inference endpoint. +When that endpoint is on another host or port than the Hopsworks UI, add it to `connect-src`: + +```yaml +hopsworks: + payara: + securityHeaders: + extraSources: + connect-src: + - https://serving.example.com:32443 +``` + +## Embedding Hopsworks in another site { #security-headers-embedding } + +The UI can only be framed by Hopsworks itself. +To let another site frame it, add that site to `frame-ancestors`: + +```yaml +hopsworks: + payara: + securityHeaders: + extraSources: + frame-ancestors: + - https://portal.example.com +``` + +Browsers check every ancestor of a frame, so the frames the UI opens allow the same sites: the page that shows a system health report, and the services Hopsworks proxies that send no `Content-Security-Policy` of their own. +A proxied service that sends its own policy keeps it. + +## When a page does not work { #security-headers-troubleshooting } + +A browser blocks what the policy forbids and logs a `Content-Security-Policy` error in its developer console. +To find what a page needs: + +1. Set `cspMode: report-only` and run `helm upgrade`. +1. Reload the page until its response carries `Content-Security-Policy-Report-Only` instead of `Content-Security-Policy`, which can take up to two and a half minutes, then read the console: each violation names the directive and the blocked source. +1. Add the sources the page needs to `extraSources`, set `cspMode: enforce` again and run `helm upgrade`. + +## Strict-Transport-Security { #security-headers-hsts } + +With `Strict-Transport-Security`, a browser that has visited Hopsworks only uses HTTPS for its host, and with `includeSubDomains` for every subdomain of it, until `maxAge` runs out. +Every service on that host and those subdomains must then be served over HTTPS, the inference endpoint included, whatever its port. + +To undo it, set `maxAge: 0` and run `helm upgrade`: a browser that receives the header over HTTPS forgets the host. +ingress-nginx adds a `Strict-Transport-Security` header of its own by default, so behind it the controller's `hsts` settings need the same change. diff --git a/docs/setup_installation/admin/upgrade.md b/docs/setup_installation/admin/upgrade.md index ddac87ecf6..900de8a7e5 100644 --- a/docs/setup_installation/admin/upgrade.md +++ b/docs/setup_installation/admin/upgrade.md @@ -486,6 +486,16 @@ Enable with one `helm upgrade`: no predictive pod, Knative pod or `kserve-ingres Data Owners can then turn the LLM Gateway off per project and per deployment, both on by default. Disable with one upgrade with `aiGateway.enabled: false`, as described under [Turn it off][llm-gateway-disable], which also lists what a `helm rollback` leaves behind and how to remove it. +### Security response headers { #upgrade-security-headers } + +Hopsworks now sets [security response headers][security-headers] itself, including a Content-Security-Policy that the UI enforces. +An upgrade needs no action. +The policy allows what the Hopsworks UI loads from the Hopsworks origin only. +If the UI calls model deployments on another host or port, or another site frames the UI, add that origin to `hopsworks.payara.securityHeaders.extraSources`, as described under [Model deployments on another host or port][security-headers-inference] and [Embedding Hopsworks in another site][security-headers-embedding]. +A page the policy breaks shows a `Content-Security-Policy` error in the browser console; set `cspMode: report-only` while [finding what it needs][security-headers-troubleshooting]. + +Pages are no longer served from the browser's offline cache, so a page needs the cluster to be reachable. + ## Changes from 5.2.2 { #upgrade-changes-from-522 } These changes ship in 5.2.2 and 5.3.0. diff --git a/mkdocs.yml b/mkdocs.yml index f0df3ed801..49a72623a5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -319,6 +319,7 @@ nav: - Model Serving Ingress Gateway: setup_installation/admin/model_serving_gateway.md - Kafka Quotas: setup_installation/admin/kafka_quotas.md - OnlineFS Configuration: setup_installation/admin/onlinefs.md + - Security Response Headers: setup_installation/admin/security_headers.md - Uninstalling Hopsworks: setup_installation/admin/uninstall.md - Upgrades: - Upgrading Hopsworks: setup_installation/admin/upgrade.md