From fe71e14aa62e0f0e5ed873d6dc55e338d9324ad4 Mon Sep 17 00:00:00 2001 From: Ryan Dahl Date: Sun, 15 Nov 2026 12:23:02 -0800 Subject: [PATCH] Static assets, in standalone workerd Adds an `assets` service to workerd: a directory served as Cloudflare serves a Worker's static assets. Its router sends each request to the user Worker or to a file, as run_worker_first and the files ask, and its AssetWorker entrypoint backs a Worker's ASSETS binding. Events other than HTTP requests go to the user Worker. The routing, the _headers, _redirects, .assetsignore and run_worker_first parsers and the head of an asset's response are a side-effect-free crate, src/rust/assets-router, ported from workers-sdk's workers-shared at dc47e6d2, so that edgeworker can route with it too. The host in the server crate only stats files, streams them and forwards to the user Worker. Files are looked up as requests arrive, so nothing walks the directory at startup and a change to it is seen at once. An asset's ETag is a hash of its path, its size and its modification time; with the path in it, no two files are taken for the same asset. html_handling, not_found_handling and base_path behave as in production, as do the /_image block and the filter on _next/image responses. assets_navigation_prefers_asset_serving is compiled with the user Worker's compatibility flags, or, without one, with the service's own compatibilityDate and compatibilityFlags. A bad config is an error at startup; invalid _headers, _redirects and .assetsignore rules are warnings. With --experimental, assets in wrangler.json becomes one service, -assets, with or without main. A wd-test checks that the router passes JS RPC, scheduled and tail events to the user Worker and that the AssetWorker entrypoint does not. --- README.md | 7 +- deps/rust/Cargo.lock | 1 + deps/rust/Cargo.toml | 2 + deps/rust/cargo-bazel-lock.json | 7 +- src/rust/AGENTS.md | 2 +- src/rust/assets-router/BUILD.bazel | 12 + src/rust/assets-router/base_path-test.rs | 121 +++ src/rust/assets-router/base_path.rs | 106 +++ src/rust/assets-router/decide-test.rs | 556 +++++++++++++ src/rust/assets-router/decide.rs | 159 ++++ src/rust/assets-router/golden-test.rs | 360 ++++++++ src/rust/assets-router/ignore-test.rs | 83 ++ src/rust/assets-router/ignore.rs | 119 +++ src/rust/assets-router/intent-test.rs | 773 ++++++++++++++++++ src/rust/assets-router/intent.rs | 365 +++++++++ src/rust/assets-router/lib-test.rs | 102 +++ src/rust/assets-router/lib.rs | 182 +++++ src/rust/assets-router/mitigations-test.rs | 367 +++++++++ src/rust/assets-router/mitigations.rs | 67 ++ src/rust/assets-router/parse-test.rs | 510 ++++++++++++ src/rust/assets-router/parse.rs | 537 ++++++++++++ src/rust/assets-router/rules-test.rs | 367 +++++++++ src/rust/assets-router/rules.rs | 185 +++++ src/rust/assets-router/serve-test.rs | 137 ++++ src/rust/assets-router/serve.rs | 130 +++ src/workerd/server/AGENTS.md | 1 + src/workerd/server/cli/wrangler-test.rs | 133 +++ src/workerd/server/cli/wrangler.rs | 272 ++++-- .../server/factory/worker-factory-listen.c++ | 47 ++ src/workerd/server/factory/worker-factory.c++ | 35 +- src/workerd/server/factory/worker-factory.h | 4 + src/workerd/server/server/BUILD.bazel | 1 + src/workerd/server/server/bridge.rs | 51 +- src/workerd/server/server/config.rs | 38 + .../server/server/services/assets/mod.rs | 380 +++++++++ src/workerd/server/server/services/disk.rs | 8 +- src/workerd/server/server/services/mod.rs | 1 + src/workerd/server/tests/assets/BUILD.bazel | 18 + .../server/tests/assets/assets-events-app.js | 37 + .../tests/assets/assets-events-producer.js | 11 + .../server/tests/assets/assets-events-test.js | 43 + .../tests/assets/assets-events-test.wd-test | 38 + .../tests/assets/assets-wrangler-test.mjs | 262 ++++++ src/workerd/server/workerd.capnp | 34 + 44 files changed, 6604 insertions(+), 67 deletions(-) create mode 100644 src/rust/assets-router/BUILD.bazel create mode 100644 src/rust/assets-router/base_path-test.rs create mode 100644 src/rust/assets-router/base_path.rs create mode 100644 src/rust/assets-router/decide-test.rs create mode 100644 src/rust/assets-router/decide.rs create mode 100644 src/rust/assets-router/golden-test.rs create mode 100644 src/rust/assets-router/ignore-test.rs create mode 100644 src/rust/assets-router/ignore.rs create mode 100644 src/rust/assets-router/intent-test.rs create mode 100644 src/rust/assets-router/intent.rs create mode 100644 src/rust/assets-router/lib-test.rs create mode 100644 src/rust/assets-router/lib.rs create mode 100644 src/rust/assets-router/mitigations-test.rs create mode 100644 src/rust/assets-router/mitigations.rs create mode 100644 src/rust/assets-router/parse-test.rs create mode 100644 src/rust/assets-router/parse.rs create mode 100644 src/rust/assets-router/rules-test.rs create mode 100644 src/rust/assets-router/rules.rs create mode 100644 src/rust/assets-router/serve-test.rs create mode 100644 src/rust/assets-router/serve.rs create mode 100644 src/workerd/server/server/services/assets/mod.rs create mode 100644 src/workerd/server/tests/assets/BUILD.bazel create mode 100644 src/workerd/server/tests/assets/assets-events-app.js create mode 100644 src/workerd/server/tests/assets/assets-events-producer.js create mode 100644 src/workerd/server/tests/assets/assets-events-test.js create mode 100644 src/workerd/server/tests/assets/assets-events-test.wd-test create mode 100644 src/workerd/server/tests/assets/assets-wrangler-test.mjs diff --git a/README.md b/README.md index 38380a8541f..450449f95b4 100644 --- a/README.md +++ b/README.md @@ -112,9 +112,10 @@ For simple projects, `workerd serve wrangler.jsonc` also accepts a Wrangler config. `main` is bundled with esbuild, as Wrangler does. The supported fields are `name`, `main`, `compatibility_date`, `compatibility_flags`, `vars`, `durable_objects`, `exports`, `migrations`, `no_bundle`, `$schema` and, with -`--experimental`, `kv_namespaces`, `triggers.crons`, `queues` and `workflows`; -any other field is an error. Each KV namespace is one Durable Object, so its -throughput is that of one object. +`--experimental`, `kv_namespaces`, `triggers.crons`, `queues`, `workflows` and +`assets`; any other field is an error. With `assets`, `main` may be left out to +serve files only, and then fields that need a Worker are an error. Each KV +namespace is one Durable Object, so its throughput is that of one object. A simple "Hello World!" config file might look like: diff --git a/deps/rust/Cargo.lock b/deps/rust/Cargo.lock index 0c092477d45..c7bbfae7651 100644 --- a/deps/rust/Cargo.lock +++ b/deps/rust/Cargo.lock @@ -623,6 +623,7 @@ dependencies = [ "proc-macro2", "quote", "rand 0.9.5", + "regex", "ring", "ruff_python_ast", "ruff_python_parser", diff --git a/deps/rust/Cargo.toml b/deps/rust/Cargo.toml index 37bb6e9f0f0..3ac30a1427b 100644 --- a/deps/rust/Cargo.toml +++ b/deps/rust/Cargo.toml @@ -51,6 +51,8 @@ http-body = "1" httpdate = "1" percent-encoding = "2" url = "2" +# The server's assets service: the `_headers`, `_redirects` and `.assetsignore` rules. +regex = { version = "1", default-features = false, features = ["std", "perf"] } # The server's CryptoKey bindings: hex key material. data-encoding = "2" # `workerd test`: the service and entrypoint filters. diff --git a/deps/rust/cargo-bazel-lock.json b/deps/rust/cargo-bazel-lock.json index 097ec05ffc7..7a488eddb2f 100644 --- a/deps/rust/cargo-bazel-lock.json +++ b/deps/rust/cargo-bazel-lock.json @@ -2,7 +2,7 @@ "binary_crates": [ "capnpc 0.27.0" ], - "checksum": "825692ad90398af9f1f5eceade32827bcdd59a8cd38054a0cbb4027f3b8a0634", + "checksum": "eb9ab79751c275672f5b39535545a7891464446ef52bbffe2e1e34e60eb284e5", "conditions": { "aarch64-apple-darwin": [ "aarch64-apple-darwin" @@ -2935,6 +2935,10 @@ "id": "rand 0.9.5", "target": "rand" }, + { + "id": "regex 1.13.1", + "target": "regex" + }, { "id": "ring 0.17.14", "target": "ring" @@ -15873,6 +15877,7 @@ "proc-macro2 1.0.107", "quote 1.0.47", "rand 0.9.5", + "regex 1.13.1", "ring 0.17.14", "ruff_python_ast 0.0.0", "ruff_python_parser 0.0.0", diff --git a/src/rust/AGENTS.md b/src/rust/AGENTS.md index d5be95463a0..f1623ac887f 100644 --- a/src/rust/AGENTS.md +++ b/src/rust/AGENTS.md @@ -66,7 +66,7 @@ _Snapshot — the set drifts as crates come and go; `bazel query //src/rust/...` - In a crate root that attribute would cover the whole crate, so there the item takes `#[expect(unsafe_code, reason = "...")]` instead: the bridge module (after its `#[cxx::bridge]` line), or the function or `impl`. - What can fail returns a `Result`; a panic asserts what cannot happen. `unreachable!` says exactly that and is allowed everywhere. Any other deliberate panic takes `#[expect(clippy::, reason = "...")]` on its function. - Test code is not held to any of this. A module's `-test.rs` is compiled only into the crate's `_test` target, and test targets get no lint levels from the macros; neither do `testonly` crates (`wd_rust_crate(testonly = True)`: test harnesses, the Rust half of a C++ test). So a test file needs no opt-ins, and the production files are still checked where the library is built. - - `#![forbid(unsafe_code)]` in a crate root (`memory-cache`, `jsg-macros`) is stronger than the default and stays. + - `#![forbid(unsafe_code)]` in a crate root (`memory-cache`, `jsg-macros`, `assets-router`) is stronger than the default and stays. - The crates derived from upstream cxx (`src/rust/cxx/{src,syntax,gen,macro,tests}`) are outside these defaults; the `kj-rs*` crates beside them are inside. - **Tests**: a module's unit tests live beside it in `-test.rs` (`dns.rs` → `dns-test.rs`, `lib.rs` → `lib-test.rs`), never in an inline `mod tests { ... }`, so a diff shows production changes and test changes as separate files. See UNIT TEST FILES below. JSG tests use `jsg_test::Harness::run_in_context()`; `Harness::with_flags(|mut f| f.set_(true))` sets raw compat flags and uses workers' shared `configureJsgIsolate()` helper for isolate-level JSG settings. It starts with all flags false and applies no compatibility date or flag implications; see `jsg/README.md` §Testing with flags. `jsg_test::set_autogates(&["gate-name"])` configures the process's autogates for as long as the returned guard lives. Always run the full `src/rust/...` test suite (`bazel test //src/rust/...`) rather than targeting a single crate — changes in shared crates like `jsg` or `jsg-macros` can break downstream consumers - **FFI pointers**: functions receiving raw pointers must be `unsafe fn` (see `jsg/README.md`) diff --git a/src/rust/assets-router/BUILD.bazel b/src/rust/assets-router/BUILD.bazel new file mode 100644 index 00000000000..e92ab3ae302 --- /dev/null +++ b/src/rust/assets-router/BUILD.bazel @@ -0,0 +1,12 @@ +load("//:build/wd_rust_crate.bzl", "wd_rust_crate") + +wd_rust_crate( + name = "assets-router", + test_deps = ["@crates_vendor//serde_json"], + visibility = ["//visibility:public"], + deps = [ + "@crates_vendor//regex", + "@crates_vendor//serde", + "@crates_vendor//url", + ], +) diff --git a/src/rust/assets-router/base_path-test.rs b/src/rust/assets-router/base_path-test.rs new file mode 100644 index 00000000000..5924449f352 --- /dev/null +++ b/src/rust/assets-router/base_path-test.rs @@ -0,0 +1,121 @@ +use super::*; + +fn normalize(value: &str) -> Result { + BasePath::normalize(value).map(|base| base.as_str().to_owned()) +} + +fn base(value: &str) -> BasePath { + BasePath::normalize(value).unwrap() +} + +// Upstream: `utils/testdata/base-path-normalization.json` (contract version 1), the vectors +// the control plane's normalization is held to as well. +#[test] +fn normalize_matches_the_shared_contract() { + for (name, input, canonical) in [ + ("root", "/", Some("/")), + ("absolute", "/docs", Some("/docs/")), + ("relative", "./docs", Some("/docs/")), + ("duplicate segments", "/a//b", Some("/a/b/")), + ("dot segments", "/a/./b/../c", Some("/a/c/")), + ("above root", "../../docs", Some("/docs/")), + ("unicode", "/café/naïve", Some("/café/naïve/")), + ("empty", "", None), + ("url", "https://example.com/docs", None), + ("protocol relative url", "//example.com/docs", None), + ("backslash", "/a\\b", None), + ("query", "/docs?lang=en", None), + ("fragment", "/docs#intro", None), + ("percent encoding", "/docs%2Fapi", None), + ("control character", "/docs\u{1}api", None), + ] { + assert_eq!(normalize(input).ok().as_deref(), canonical, "{name}"); + } +} + +// Upstream: utils/tests/base-path.test.ts `normalizeBasePath > valid values`. +#[test] +fn normalize_accepts_pathnames() { + assert_eq!(normalize("/subpath").unwrap(), "/subpath/"); + assert_eq!(normalize("/a/b/c").unwrap(), "/a/b/c/"); + assert_eq!(normalize("subpath").unwrap(), "/subpath/"); + assert_eq!(normalize("../app").unwrap(), "/app/"); + assert_eq!(normalize("/subpath/").unwrap(), "/subpath/"); + assert_eq!(normalize("/.").unwrap(), "/"); + + let long = format!("/{}", "😀".repeat(2000)); + assert_eq!(normalize(&long).unwrap(), format!("{long}/")); +} + +// Upstream: utils/tests/base-path.test.ts `normalizeBasePath > invalid values`. +#[test] +fn normalize_rejects_values_that_are_not_pathnames() { + assert_eq!(normalize(""), Err("The value must not be empty.")); + assert_eq!( + normalize("/sub\\path"), + Err("The value must not contain backslashes.") + ); + assert_eq!( + normalize("/sub\u{0}path"), + Err("The value must not contain control characters.") + ); + assert_eq!( + normalize("/sub\u{7f}path"), + Err("The value must not contain control characters.") + ); + assert_eq!( + normalize("/subpath?a=1"), + Err("The value must not contain query strings or fragments.") + ); + assert_eq!( + normalize("/subpath#frag"), + Err("The value must not contain query strings or fragments.") + ); + assert_eq!( + normalize("/sub%2Fpath"), + Err("The value must not contain percent-encoded sequences.") + ); + assert_eq!( + normalize("/sub%20path"), + Err("The value must not contain percent-encoded sequences.") + ); + assert_eq!( + normalize("//a"), + Err("The value must be a pathname, not a URL.") + ); + assert_eq!( + normalize("https://example.com/subpath"), + Err("The value must be a pathname, not a URL.") + ); + assert_eq!( + normalize("data:text/plain,hello"), + Err("The value must be a pathname, not a URL.") + ); +} + +#[test] +fn normalize_only_treats_a_leading_scheme_as_a_url() { + assert_eq!(normalize("/a:b").unwrap(), "/a:b/"); + assert_eq!(normalize("1a:b").unwrap(), "/1a:b/"); +} + +// Upstream: asset-worker/tests/base-path.test.ts `stripBasePath`. +#[test] +fn strip_maps_public_paths_to_asset_paths() { + assert_eq!(base("/").strip("/foo/bar"), Some("/foo/bar")); + assert_eq!(base("/subpath/").strip("/subpath/foo"), Some("/foo")); + assert_eq!(base("/subpath/").strip("/subpath"), Some("/")); + assert_eq!(base("/subpath/").strip("/subpath/"), Some("/")); + assert_eq!(base("/subpath/").strip("/subpath-other/foo"), None); + assert_eq!(base("/subpath/").strip("/other"), None); + assert_eq!(base("/路径/").strip("/路径/foo"), Some("/foo")); +} + +// Upstream: asset-worker/tests/base-path.test.ts `addBasePath`. +#[test] +fn add_maps_asset_paths_to_public_paths() { + assert_eq!(base("/").add("/foo/"), "/foo/"); + assert_eq!(base("/subpath/").add("/"), "/subpath/"); + assert_eq!(base("/subpath/").add("/foo/"), "/subpath/foo/"); + assert_eq!(base("/subpath/").add(""), "/subpath"); +} diff --git a/src/rust/assets-router/base_path.rs b/src/rust/assets-router/base_path.rs new file mode 100644 index 00000000000..c0ed398eefb --- /dev/null +++ b/src/rust/assets-router/base_path.rs @@ -0,0 +1,106 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! The `base_path` under which assets are served. +//! +//! Ported from workers-sdk's workers-shared, `utils/base-path.ts` and +//! `asset-worker/src/utils/base-path.ts`. + +/// The root `/`, or a pathname with a leading and a trailing slash, such as `/subpath/`. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct BasePath(String); + +impl Default for BasePath { + fn default() -> Self { + Self("/".to_owned()) + } +} + +impl BasePath { + /// Mirrors `normalizeBasePath`, and its messages. + pub fn normalize(value: &str) -> Result { + let (scheme, _) = value.split_once(':').unwrap_or_default(); + let mut scheme_chars = scheme.chars(); + let has_scheme = scheme_chars.next().is_some_and(|c| c.is_ascii_alphabetic()) + && scheme_chars.all(|c| c.is_ascii_alphanumeric() || matches!(c, '+' | '.' | '-')); + for (invalid, message) in [ + (value.is_empty(), "The value must not be empty."), + ( + value.starts_with("//") || has_scheme, + "The value must be a pathname, not a URL.", + ), + ( + value.contains('\\'), + "The value must not contain backslashes.", + ), + ( + value.chars().any(|c| c <= '\u{1f}' || c == '\u{7f}'), + "The value must not contain control characters.", + ), + ( + value.contains(['?', '#']), + "The value must not contain query strings or fragments.", + ), + ( + value.contains('%'), + "The value must not contain percent-encoded sequences.", + ), + ] { + if invalid { + return Err(message); + } + } + let mut segments = Vec::new(); + for segment in value.split('/') { + match segment { + "" | "." => {} + ".." => _ = segments.pop(), + segment => segments.push(segment), + } + } + if segments.is_empty() { + return Ok(Self::default()); + } + Ok(Self(format!("/{}/", segments.join("/")))) + } + + pub fn as_str(&self) -> &str { + &self.0 + } + + pub fn is_root(&self) -> bool { + self.0 == "/" + } + + /// The base without its trailing slash, e.g. `/subpath`, unless it is the root. + pub(crate) fn without_trailing_slash(&self) -> Option<&str> { + self.0.strip_suffix('/').filter(|base| !base.is_empty()) + } + + /// The path under the base of a decoded pathname, matching whole segments. Mirrors + /// `stripBasePath`. + pub fn strip<'a>(&self, path: &'a str) -> Option<&'a str> { + match self.without_trailing_slash() { + None => Some(path), + Some(base) if path == base => Some("/"), + Some(base) => path + .strip_prefix(self.as_str()) + .map(|_| &path[base.len()..]), + } + } + + /// The public path of a path under the base, where `""` is the base without its trailing + /// slash. Mirrors `addBasePath`. + pub fn add(&self, path: &str) -> String { + match self.without_trailing_slash() { + None => path.to_owned(), + Some(_) if path == "/" => self.0.clone(), + Some(base) => format!("{base}{path}"), + } + } +} + +#[cfg(test)] +#[path = "base_path-test.rs"] +mod tests; diff --git a/src/rust/assets-router/decide-test.rs b/src/rust/assets-router/decide-test.rs new file mode 100644 index 00000000000..8e75d8dab5d --- /dev/null +++ b/src/rust/assets-router/decide-test.rs @@ -0,0 +1,556 @@ +use url::Url; + +use super::super::Method; +use super::super::intent::HtmlHandling; +use super::super::intent::NotFoundHandling; +use super::super::rules::Redirect; +use super::*; + +fn rules(rules: &[&str]) -> Vec { + globs( + &rules + .iter() + .map(|rule| (*rule).to_owned()) + .collect::>(), + ) +} + +fn config( + has_user_worker: bool, + invoke_ahead: bool, + user_rules: &[&str], + asset_rules: &[&str], +) -> RoutingConfig { + RoutingConfig { + has_user_worker, + invoke_user_worker_ahead_of_assets: invoke_ahead, + user_worker_rules: rules(user_rules), + asset_worker_rules: rules(asset_rules), + intent: IntentConfig { + html_handling: HtmlHandling::AutoTrailingSlash, + not_found_handling: NotFoundHandling::None, + has_static_routing: !user_rules.is_empty() || !asset_rules.is_empty(), + navigation_prefers_asset_serving: false, + }, + base_path: BasePath::default(), + redirects: Redirects::new([], []), + header_rules: RulesMatcher::new([]), + } +} + +fn config_with_base_path( + has_user_worker: bool, + user_rules: &[&str], + asset_rules: &[&str], +) -> RoutingConfig { + RoutingConfig { + base_path: BasePath::normalize("/subpath").unwrap(), + ..config(has_user_worker, false, user_rules, asset_rules) + } +} + +struct TestRequest<'a> { + pathname: &'a str, + method: Method, + sec_fetch_dest: Option<&'a str>, +} + +fn get(pathname: &str) -> TestRequest<'_> { + TestRequest { + pathname, + method: Method::Get, + sec_fetch_dest: None, + } +} + +fn route( + request: &TestRequest<'_>, + config: &RoutingConfig, + assets: &[&str], +) -> Result { + let url = Url::parse(&format!("https://example.com{}", request.pathname)).unwrap(); + config.route( + config.rule_route(url.path()), + &Request { + url: &url, + method: request.method, + sec_fetch_mode: None, + sec_fetch_dest: request.sec_fetch_dest, + }, + &|path: &str| assets.contains(&path).then(|| path.to_owned()), + ) +} + +fn decide_with_assets( + pathname: &str, + config: &RoutingConfig, + assets: &[&str], +) -> Result { + route(&get(pathname), config, assets) +} + +const USER_WORKER: Result = Ok(Route::UserWorker { + filter_image_response: false, + block_image_href: false, +}); + +/// The file at `path`, served. +#[expect( + clippy::unnecessary_wraps, + reason = "compared with the result of routing" +)] +fn asset(path: &str) -> Result { + Ok(Route::Asset(Asset { + path: path.into(), + etag: path.into(), + status: 200, + not_found: false, + })) +} + +/// The assets' 404. +const NOT_FOUND: Result = Ok(Route::Respond(Response { + status: 404, + headers: vec![], + body: "", +})); + +#[expect( + clippy::unnecessary_wraps, + reason = "compared with the result of routing" +)] +fn redirect(status: u16, location: &str) -> Result { + Ok(Route::Respond(Response { + status, + headers: vec![ResponseHeader { + name: "Location".into(), + value: location.into(), + }], + body: "", + })) +} + +#[test] +fn invoke_ahead_without_user_worker_errors() { + assert_eq!( + decide_with_assets("/asset", &config(false, true, &[], &[]), &["/asset"]), + Err("Fetch for user worker without having a user worker binding".into()) + ); +} + +#[test] +fn assets_only_miss_routes_to_asset_worker() { + assert_eq!( + decide_with_assets("/missing", &config(false, false, &[], &[]), &["/asset"]), + NOT_FOUND + ); +} + +#[test] +fn assets_only_hit_routes_to_asset_worker() { + assert_eq!( + decide_with_assets("/asset", &config(false, false, &[], &[]), &["/asset"]), + asset("/asset") + ); +} + +#[test] +fn invoke_ahead_routes_to_user() { + assert_eq!( + decide_with_assets("/asset", &config(true, true, &[], &[]), &["/asset"]), + USER_WORKER + ); +} + +#[test] +fn asset_hit_routes_to_asset() { + assert_eq!( + decide_with_assets("/asset", &config(true, false, &[], &[]), &["/asset"]), + asset("/asset") + ); +} + +#[test] +fn static_routing_user_worker_include_matches() { + assert_eq!( + decide_with_assets( + "/api/x", + &config(true, false, &["/api/*"], &[]), + &["/api/x"] + ), + USER_WORKER + ); +} + +#[test] +fn static_routing_asset_worker_exclude_matches() { + assert_eq!( + decide_with_assets("/public/x", &config(true, true, &[], &["/public/*"]), &[]), + NOT_FOUND + ); +} + +#[test] +fn static_routing_asset_worker_wins_over_user_worker() { + assert_eq!( + decide_with_assets( + "/overlap", + &config(true, false, &["/overlap"], &["/overlap"]), + &[] + ), + NOT_FOUND + ); +} + +#[test] +fn static_routing_missing_user_worker_rules_wc_5014() { + assert_eq!( + decide_with_assets( + "/asset", + &config(true, false, &[], &["/other"]), + &["/asset"] + ), + asset("/asset") + ); +} + +#[test] +fn static_routing_miss_falls_through_to_asset_hit() { + assert_eq!( + decide_with_assets( + "/asset", + &config(true, false, &["/api/*"], &[]), + &["/asset"] + ), + asset("/asset") + ); +} + +#[test] +fn static_routing_miss_falls_through_to_user_worker() { + assert_eq!( + decide_with_assets("/missing", &config(true, false, &["/api/*"], &[]), &[]), + USER_WORKER + ); +} + +#[test] +fn html_canonicalization_redirect_is_native() { + assert_eq!( + decide_with_assets("/foo/", &config(true, false, &[], &[]), &["/foo.html"]), + redirect(307, "/foo") + ); +} + +#[test] +fn unsupported_method_on_an_asset_is_native() { + let request = TestRequest { + method: Method::Other, + ..get("/asset") + }; + assert_eq!( + route(&request, &config(true, false, &[], &[]), &["/asset"]), + Ok(Route::Respond(Response { + status: 405, + headers: vec![], + body: "", + })) + ); + assert_eq!( + route(&request, &config(true, false, &[], &[]), &[]), + USER_WORKER + ); +} + +// The assets answer with `not_found_handling`, which `canFetch` leaves out. +#[test] +fn not_found_handling_answers_only_what_reaches_the_assets() { + let spa = |has_user_worker| RoutingConfig { + intent: IntentConfig { + not_found_handling: NotFoundHandling::SinglePageApplication, + ..IntentConfig::default() + }, + ..config(has_user_worker, false, &[], &[]) + }; + let assets = ["/index.html", "/about.html"]; + assert_eq!( + decide_with_assets("/about", &spa(true), &assets), + asset("/about.html") + ); + assert_eq!( + decide_with_assets("/missing", &spa(true), &assets), + USER_WORKER + ); + assert_eq!( + decide_with_assets("/missing", &spa(false), &assets), + Ok(Route::Asset(Asset { + path: "/index.html".into(), + etag: "/index.html".into(), + status: 200, + not_found: true, + })) + ); +} + +fn config_with_redirects(has_user_worker: bool, invoke_ahead: bool) -> RoutingConfig { + RoutingConfig { + redirects: Redirects::new( + [( + "/api/old".to_owned(), + ( + 1, + Redirect { + status: 301, + to: "/api/new".into(), + }, + ), + )], + [ + ( + "/blog/*".to_owned(), + Redirect { + status: 302, + to: "/posts/:splat".into(), + }, + ), + ( + "/users/*".to_owned(), + Redirect { + status: 200, + to: "/users/id".into(), + }, + ), + ], + ), + header_rules: RulesMatcher::new([( + "/blog/*".to_owned(), + HeaderRule { + set: vec![("X-Blog".into(), ":splat".into())], + unset: vec![], + }, + )]), + ..config(has_user_worker, invoke_ahead, &["/api/*"], &[]) + } +} + +// Upstream applies `_redirects` in the asset worker, so only to a request that is routed to +// the assets; a rule that matches makes `canFetch` true. +#[test] +fn redirect_rules_apply_to_requests_routed_to_assets() { + let config = config_with_redirects(true, false); + assert_eq!( + decide_with_assets("/blog/hello", &config, &[]), + Ok(Route::Respond(Response { + status: 302, + headers: vec![ + ResponseHeader { + name: "Location".into(), + value: "/posts/hello".into(), + }, + ResponseHeader { + name: "X-Blog".into(), + value: "hello".into(), + }, + ], + body: "", + })) + ); + assert_eq!( + decide_with_assets("/blog/hello", &config_with_redirects(false, false), &[]), + decide_with_assets("/blog/hello", &config, &[]) + ); +} + +#[test] +fn redirect_rules_do_not_apply_to_requests_routed_to_the_user_worker() { + assert_eq!( + decide_with_assets("/api/old", &config_with_redirects(true, false), &[]), + USER_WORKER + ); + assert_eq!( + decide_with_assets("/blog/hello", &config_with_redirects(true, true), &[]), + USER_WORKER + ); +} + +#[test] +fn rewrite_rule_claims_the_request_for_the_assets() { + let config = config_with_redirects(true, false); + assert_eq!( + decide_with_assets("/users/12345", &config, &["/users/id.html"]), + asset("/users/id.html") + ); + assert_eq!(decide_with_assets("/users/12345", &config, &[]), NOT_FOUND); +} + +#[test] +fn base_path_asset_is_looked_up_relative_to_the_asset_directory() { + let config = config_with_base_path(true, &[], &[]); + assert_eq!( + decide_with_assets("/subpath/asset", &config, &["/asset"]), + asset("/asset") + ); + assert_eq!( + decide_with_assets("/subpath/asset", &config, &["/subpath/asset"]), + USER_WORKER + ); +} + +#[test] +fn base_path_redirect_is_public() { + assert_eq!( + decide_with_assets( + "/subpath/foo/", + &config_with_base_path(true, &[], &[]), + &["/foo.html"] + ), + redirect(307, "/subpath/foo") + ); +} + +#[test] +fn off_base_path_request_routes_to_user_worker() { + assert_eq!( + decide_with_assets( + "/asset", + &config_with_base_path(true, &[], &[]), + &["/asset"] + ), + USER_WORKER + ); +} + +// Without a user worker the asset worker still receives the request, and answers that it has +// nothing to serve outside the base. +#[test] +fn off_base_path_request_without_user_worker_routes_to_asset_worker() { + assert_eq!( + decide_with_assets( + "/asset", + &config_with_base_path(false, &[], &[]), + &["/asset"] + ), + NOT_FOUND + ); +} + +#[test] +fn static_routing_matches_the_public_path() { + let config = config_with_base_path(true, &["/subpath/api/*"], &["/subpath/public/*"]); + assert_eq!( + decide_with_assets("/subpath/api/x", &config, &["/api/x"]), + USER_WORKER + ); + assert_eq!( + decide_with_assets("/subpath/public/x", &config, &[]), + NOT_FOUND + ); +} + +// Upstream: `/_image` and `_next/image` are only checked for a request that runs the user +// worker. +#[test] +fn image_mitigations_apply_to_the_user_worker_route() { + let config = config(true, false, &[], &[]); + assert_eq!( + decide_with_assets("/_image?href=//evil.com/ssrf", &config, &[]), + Ok(Route::UserWorker { + filter_image_response: false, + block_image_href: true, + }) + ); + assert_eq!( + decide_with_assets("/_image?href=//evil.com/ssrf", &config, &["/_image"]), + asset("/_image") + ); + assert_eq!( + decide_with_assets("/_next/image?url=https://evil.com/image.jpg", &config, &[]), + Ok(Route::UserWorker { + filter_image_response: true, + block_image_href: false, + }) + ); + let from_img_tag = TestRequest { + sec_fetch_dest: Some("image"), + ..get("/_next/image?url=https://evil.com/image.jpg") + }; + assert_eq!(route(&from_img_tag, &config, &[]), USER_WORKER); +} + +fn matches(rules: &[&str], pathname: &str) -> bool { + let owned: Vec = rules.iter().map(|s| (*s).to_owned()).collect(); + globs(&owned).iter().any(|r| r.is_match(pathname)) +} + +// Upstream: rules-engine.test.ts `static routing rules > should return true for a request that +// matches` (L121). +#[test] +fn matches_exact_and_glob_paths() { + assert!(matches(&["/some/path"], "/some/path")); + assert!(matches(&["/some/*"], "/some/path")); + assert!(matches(&["/no/match", "/some/*"], "/some/path")); +} + +// Upstream: rules-engine.test.ts `static routing rules > should return false for a request that +// does not match` (L140). +#[test] +fn returns_false_on_no_match() { + assert!(!matches(&["/some/path"], "/")); + assert!(!matches(&["/some/*"], "/path")); + assert!(!matches(&["/some/path", "/other/path"], "/path")); + assert!(!matches(&[], "/some/path")); +} + +// Upstream: rules-engine.test.ts `static routing rules > should ignore regex characters other +// than a glob` — rule "/" (L166). +#[test] +fn rule_root_matches_only_root() { + assert!(matches(&["/"], "/")); + assert!(!matches(&["/"], "/foo")); +} + +// Upstream: rules-engine.test.ts `... ignore regex characters` — rule "/foo" (L187). +#[test] +fn rule_exact_path_no_prefix_or_subpath() { + assert!(matches(&["/foo"], "/foo")); + assert!(!matches(&["/foo"], "/")); + assert!(!matches(&["/foo"], "/foo/")); + assert!(!matches(&["/foo"], "/foo/bar")); + assert!(!matches(&["/foo"], "/baz")); + assert!(!matches(&["/foo"], "/baz/foo")); + assert!(!matches(&["/foo"], "/foobar")); +} + +// Upstream: rules-engine.test.ts `... ignore regex characters` — rule "/:placeholder" (L213). +#[test] +fn placeholder_treated_literally_not_as_placeholder() { + assert!(!matches(&["/:placeholder"], "/foo")); + assert!(matches(&["/:placeholder"], "/:placeholder")); +} + +// Upstream: rules-engine.test.ts `... ignore regex characters` — rule "/foo*" (L223). +#[test] +fn glob_suffix_matches_prefixes_and_children() { + assert!(matches(&["/foo*"], "/foo")); + assert!(matches(&["/foo*"], "/foo/")); + assert!(matches(&["/foo*"], "/foo/bar")); + assert!(matches(&["/foo*"], "/foobar")); + assert!(!matches(&["/foo*"], "/")); + assert!(!matches(&["/foo*"], "/baz")); + assert!(!matches(&["/foo*"], "/baz/foo")); +} + +// Upstream: rules-engine.test.ts `... ignore regex characters` — further metacharacter block +// (L249). Rules containing regex-special characters must match them literally. +#[test] +fn more_regex_metacharacter_escapes() { + // A dot is literal, not "any char". + assert!(matches(&["/foo.html"], "/foo.html")); + assert!(!matches(&["/foo.html"], "/fooXhtml")); + // Parentheses / plus / question mark are literal. + assert!(matches(&["/a(b)+c?"], "/a(b)+c?")); + assert!(!matches(&["/a(b)+c?"], "/abbc")); + // Glob still works alongside literal metacharacters. + assert!(matches(&["/foo.*"], "/foo.bar")); + assert!(!matches(&["/foo.*"], "/fooXbar")); +} diff --git a/src/rust/assets-router/decide.rs b/src/rust/assets-router/decide.rs new file mode 100644 index 00000000000..f64a3bf0630 --- /dev/null +++ b/src/rust/assets-router/decide.rs @@ -0,0 +1,159 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! Routing a request between the user Worker and its assets. +//! +//! Ported from workers-sdk's workers-shared, `router-worker/src/worker.ts`. + +use regex::Regex; + +use super::Request; +use super::Response; +use super::ResponseHeader; +use super::base_path::BasePath; +use super::header; +use super::intent::Asset; +use super::intent::AssetResponse; +use super::intent::IntentConfig; +use super::mitigations; +use super::rules::HeaderRule; +use super::rules::Redirects; +use super::rules::RulesMatcher; +use super::rules::apply_headers; + +/// Where the static rules alone send a request. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum RuleRoute { + UserWorker, + AssetWorker, + /// The assets handle the request if they have something for it. + Undecided, +} + +#[derive(Debug, Eq, PartialEq)] +pub enum Route { + /// Run the user Worker, whose response, with `filter_image_response`, must pass + /// [`mitigations::should_block_image_response`]. With `block_image_href`, the host refuses + /// the request with [`Response::blocked`] instead, after any checks of its own. + UserWorker { + filter_image_response: bool, + block_image_href: bool, + }, + Respond(Response), + Asset(Asset), +} + +pub struct RoutingConfig { + pub has_user_worker: bool, + pub invoke_user_worker_ahead_of_assets: bool, + pub user_worker_rules: Vec, + pub asset_worker_rules: Vec, + pub intent: IntentConfig, + pub base_path: BasePath, + pub redirects: Redirects, + pub header_rules: RulesMatcher, +} + +/// `run_worker_first` rules, in which `*` is the only special character. Mirrors +/// `generateGlobOnlyRuleRegExp`. +pub fn globs(rules: &[String]) -> Vec { + let glob = |rule: &String| { + rule.split('*') + .map(regex::escape) + .collect::>() + .join(".*") + }; + rules + .iter() + .filter_map(|rule| Regex::new(&format!("^{}$", glob(rule))).ok()) + .collect() +} + +impl RoutingConfig { + pub fn rule_route(&self, pathname: &str) -> RuleRoute { + let matches = |rules: &[Regex]| rules.iter().any(|r| r.is_match(pathname)); + if matches(&self.asset_worker_rules) { + RuleRoute::AssetWorker + } else if matches(&self.user_worker_rules) || self.invoke_user_worker_ahead_of_assets { + RuleRoute::UserWorker + } else { + RuleRoute::Undecided + } + } + + /// Route `request`, which [`Self::rule_route`] sent to `rule_route`. `exists` gives the + /// `ETag` of a file at a path relative to the directory. + pub fn route( + &self, + rule_route: RuleRoute, + request: &Request<'_>, + exists: &impl Fn(&str) -> Option, + ) -> Result { + let response = match rule_route { + RuleRoute::UserWorker => return self.user_worker_route(request), + RuleRoute::Undecided if self.has_user_worker => { + let config = self.intent.for_can_fetch(request.sec_fetch_mode); + let can_fetch = self.asset_response(request, config, exists)?; + if can_fetch == AssetResponse::NoIntent { + return self.user_worker_route(request); + } + if config == self.intent { + can_fetch + } else { + self.asset_response(request, self.intent, exists)? + } + } + _ => self.asset_response(request, self.intent, exists)?, + }; + let respond = |status, headers| { + let headers = self.custom_headers(request, headers); + Ok(Route::Respond(Response::new(status, headers))) + }; + match response { + AssetResponse::Redirect { status, location } => { + respond(status, vec![header("Location", location)]) + } + AssetResponse::MethodNotAllowed => respond(405, vec![]), + AssetResponse::NotFound | AssetResponse::NoIntent => respond(404, vec![]), + AssetResponse::Asset(asset) => Ok(Route::Asset(asset)), + } + } + + fn asset_response( + &self, + request: &Request<'_>, + config: IntentConfig, + exists: &impl Fn(&str) -> Option, + ) -> Result { + let redirect = self.redirects.find(request.url); + super::intent::respond(request, redirect.as_ref(), config, &self.base_path, exists) + .map_err(|e| format!("invalid _redirects target: {e}")) + } + + /// Mirrors `routeToUserWorker`: a missing user Worker is an error even for a request that + /// would be blocked. + fn user_worker_route(&self, request: &Request<'_>) -> Result { + if !self.has_user_worker { + return Err("Fetch for user worker without having a user worker binding".into()); + } + Ok(Route::UserWorker { + filter_image_response: mitigations::filters_next_image_response(request), + block_image_href: mitigations::blocks_image_href(request), + }) + } + + /// `headers`, with the `_headers` rules for `request` applied. + pub fn custom_headers( + &self, + request: &Request<'_>, + headers: Vec, + ) -> Vec { + let host = request.url.host_str().unwrap_or_default(); + apply_headers(&self.header_rules, host, request.url.path(), headers) + } +} + +#[cfg(test)] +#[path = "decide-test.rs"] +mod tests; diff --git a/src/rust/assets-router/golden-test.rs b/src/rust/assets-router/golden-test.rs new file mode 100644 index 00000000000..7881d6cd060 --- /dev/null +++ b/src/rust/assets-router/golden-test.rs @@ -0,0 +1,360 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! Golden parity vectors for the highest-risk pure functions (path encode/decode, the +//! `_redirects`/static-routing rules engine, and the placeholder replacer). +//! +//! Recorded from the output of the *verbatim* upstream workers-sdk +//! functions (`asset-worker/src/handler.ts` decodePath/encodePath and +//! `asset-worker/src/utils/rules-engine.ts`), run over a fixed input set at snapshot main @ +//! 2026-07-14 (commit 2cd84d4). Those functions, and `generateStaticRoutingRuleMatcher`, are +//! unchanged from there to dc47e6d2, the commit this crate is ported from. The generator is not +//! in this tree: to extend the vectors, run the upstream functions over new inputs and record +//! their outputs here. These assert the native Rust ports reproduce the upstream +//! JS output byte-for-byte, catching divergences the hand-picked ported unit tests might miss. + +use crate::decide::globs; +use crate::intent::decode_path; +use crate::intent::encode_path; +use crate::rules::RulesMatcher; +use crate::rules::replacer; + +/// (input, upstream decodePath, upstream encodePath) +const PATH_VECTORS: &[(&str, &str, &str)] = &[ + ("/", "/", "/"), + ("/foo", "/foo", "/foo"), + ("/foo/bar", "/foo/bar", "/foo/bar"), + ("/foo//bar", "/foo/bar", "/foo//bar"), + ("///a///b///", "/a/b/", "///a///b///"), + ("/foo.html", "/foo.html", "/foo.html"), + ("/dir/", "/dir/", "/dir/"), + ("/[boop].html", "/[boop].html", "/%5Bboop%5D.html"), + ("/%5Bboop%5D.html", "/[boop].html", "/%255Bboop%255D.html"), + ("/%255Bboop%255D", "/%5Bboop%5D", "/%25255Bboop%25255D"), + ("/caf\u{e9}", "/caf\u{e9}", "/caf%C3%A9"), + ("/caf%C3%A9", "/caf\u{e9}", "/caf%25C3%25A9"), + ("/a b", "/a b", "/a%20b"), + ("/a%20b", "/a b", "/a%2520b"), + ("/a+b", "/a+b", "/a%2Bb"), + ("/%zz", "/%zz", "/%25zz"), + ("/%", "/%", "/%25"), + ("/%2", "/%2", "/%252"), + ("/100%", "/100%", "/100%25"), + ("/%E0%A4%A", "/%E0%A4%A", "/%25E0%25A4%25A"), + ( + "/hello%20world/%2E%2E", + "/hello world/..", + "/hello%2520world/%252E%252E", + ), + ("/../etc/passwd", "/../etc/passwd", "/../etc/passwd"), + ("/%2e%2e/x", "/../x", "/%252e%252e/x"), + ( + "/emoji/\u{1f600}", + "/emoji/\u{1f600}", + "/emoji/%F0%9F%98%80", + ), + ("/space test/path", "/space test/path", "/space%20test/path"), + ("/a?b#c", "/a?b#c", "/a%3Fb%23c"), + ("/\u{a0}", "/\u{a0}", "/%C2%A0"), + ("/%A0", "/%A0", "/%25A0"), + ( + "/tilde~/dash-/star*", + "/tilde~/dash-/star*", + "/tilde~/dash-/star*", + ), + ("/(paren)", "/(paren)", "/(paren)"), + ("/a=b&c=d", "/a=b&c=d", "/a%3Db%26c%3Dd"), + ("/path%2Fslash", "/path/slash", "/path%252Fslash"), + ( + "/mixed/CaSe/File.HTML", + "/mixed/CaSe/File.HTML", + "/mixed/CaSe/File.HTML", + ), + ("/trailing%2f", "/trailing/", "/trailing%252f"), + ("/%c3%28", "/%c3%28", "/%25c3%2528"), + ( + "/\u{221a}/\u{3c0}", + "/\u{221a}/\u{3c0}", + "/%E2%88%9A/%CF%80", + ), + ("/;semi", "/;semi", "/%3Bsemi"), + ("/@at", "/@at", "/%40at"), +]; + +#[test] +fn golden_path_encode_decode() { + for (input, decoded, encoded) in PATH_VECTORS { + assert_eq!(&decode_path(input), decoded, "decode_path({input:?})"); + assert_eq!(&encode_path(input), encoded, "encode_path({input:?})"); + } +} + +/// (rules, pathname, upstream generateStaticRoutingRuleMatcher result) +const STATIC_ROUTING_VECTORS: &[(&[&str], &str, bool)] = &[ + (&["/"], "/", true), + (&["/"], "/foo", false), + (&["/"], "/foo/", false), + (&["/"], "/foobar", false), + (&["/"], "/foo/bar", false), + (&["/"], "/:placeholder", false), + (&["/"], "/anything", false), + (&["/"], "/api/", false), + (&["/"], "/api/x/y", false), + (&["/"], "/a.b", false), + (&["/"], "/axb", false), + (&["/"], "/x+y", false), + (&["/"], "/xy", false), + (&["/"], "/deep/mid/end", false), + (&["/"], "/deep/a/b/end", false), + (&["/"], "/exact", false), + (&["/"], "/exacts", false), + (&["/foo"], "/", false), + (&["/foo"], "/foo", true), + (&["/foo"], "/foo/", false), + (&["/foo"], "/foobar", false), + (&["/foo"], "/foo/bar", false), + (&["/foo"], "/:placeholder", false), + (&["/foo"], "/anything", false), + (&["/foo"], "/api/", false), + (&["/foo"], "/api/x/y", false), + (&["/foo"], "/a.b", false), + (&["/foo"], "/axb", false), + (&["/foo"], "/x+y", false), + (&["/foo"], "/xy", false), + (&["/foo"], "/deep/mid/end", false), + (&["/foo"], "/deep/a/b/end", false), + (&["/foo"], "/exact", false), + (&["/foo"], "/exacts", false), + (&["/foo*"], "/", false), + (&["/foo*"], "/foo", true), + (&["/foo*"], "/foo/", true), + (&["/foo*"], "/foobar", true), + (&["/foo*"], "/foo/bar", true), + (&["/foo*"], "/:placeholder", false), + (&["/foo*"], "/anything", false), + (&["/foo*"], "/api/", false), + (&["/foo*"], "/api/x/y", false), + (&["/foo*"], "/a.b", false), + (&["/foo*"], "/axb", false), + (&["/foo*"], "/x+y", false), + (&["/foo*"], "/xy", false), + (&["/foo*"], "/deep/mid/end", false), + (&["/foo*"], "/deep/a/b/end", false), + (&["/foo*"], "/exact", false), + (&["/foo*"], "/exacts", false), + (&["/:placeholder"], "/", false), + (&["/:placeholder"], "/foo", false), + (&["/:placeholder"], "/foo/", false), + (&["/:placeholder"], "/foobar", false), + (&["/:placeholder"], "/foo/bar", false), + (&["/:placeholder"], "/:placeholder", true), + (&["/:placeholder"], "/anything", false), + (&["/:placeholder"], "/api/", false), + (&["/:placeholder"], "/api/x/y", false), + (&["/:placeholder"], "/a.b", false), + (&["/:placeholder"], "/axb", false), + (&["/:placeholder"], "/x+y", false), + (&["/:placeholder"], "/xy", false), + (&["/:placeholder"], "/deep/mid/end", false), + (&["/:placeholder"], "/deep/a/b/end", false), + (&["/:placeholder"], "/exact", false), + (&["/:placeholder"], "/exacts", false), + (&["/api/*"], "/", false), + (&["/api/*"], "/foo", false), + (&["/api/*"], "/foo/", false), + (&["/api/*"], "/foobar", false), + (&["/api/*"], "/foo/bar", false), + (&["/api/*"], "/:placeholder", false), + (&["/api/*"], "/anything", false), + (&["/api/*"], "/api/", true), + (&["/api/*"], "/api/x/y", true), + (&["/api/*"], "/a.b", false), + (&["/api/*"], "/axb", false), + (&["/api/*"], "/x+y", false), + (&["/api/*"], "/xy", false), + (&["/api/*"], "/deep/mid/end", false), + (&["/api/*"], "/deep/a/b/end", false), + (&["/api/*"], "/exact", false), + (&["/api/*"], "/exacts", false), + (&["/a.b"], "/", false), + (&["/a.b"], "/foo", false), + (&["/a.b"], "/foo/", false), + (&["/a.b"], "/foobar", false), + (&["/a.b"], "/foo/bar", false), + (&["/a.b"], "/:placeholder", false), + (&["/a.b"], "/anything", false), + (&["/a.b"], "/api/", false), + (&["/a.b"], "/api/x/y", false), + (&["/a.b"], "/a.b", true), + (&["/a.b"], "/axb", false), + (&["/a.b"], "/x+y", false), + (&["/a.b"], "/xy", false), + (&["/a.b"], "/deep/mid/end", false), + (&["/a.b"], "/deep/a/b/end", false), + (&["/a.b"], "/exact", false), + (&["/a.b"], "/exacts", false), + (&["/x+y"], "/", false), + (&["/x+y"], "/foo", false), + (&["/x+y"], "/foo/", false), + (&["/x+y"], "/foobar", false), + (&["/x+y"], "/foo/bar", false), + (&["/x+y"], "/:placeholder", false), + (&["/x+y"], "/anything", false), + (&["/x+y"], "/api/", false), + (&["/x+y"], "/api/x/y", false), + (&["/x+y"], "/a.b", false), + (&["/x+y"], "/axb", false), + (&["/x+y"], "/x+y", true), + (&["/x+y"], "/xy", false), + (&["/x+y"], "/deep/mid/end", false), + (&["/x+y"], "/deep/a/b/end", false), + (&["/x+y"], "/exact", false), + (&["/x+y"], "/exacts", false), + (&["/deep/*/end"], "/", false), + (&["/deep/*/end"], "/foo", false), + (&["/deep/*/end"], "/foo/", false), + (&["/deep/*/end"], "/foobar", false), + (&["/deep/*/end"], "/foo/bar", false), + (&["/deep/*/end"], "/:placeholder", false), + (&["/deep/*/end"], "/anything", false), + (&["/deep/*/end"], "/api/", false), + (&["/deep/*/end"], "/api/x/y", false), + (&["/deep/*/end"], "/a.b", false), + (&["/deep/*/end"], "/axb", false), + (&["/deep/*/end"], "/x+y", false), + (&["/deep/*/end"], "/xy", false), + (&["/deep/*/end"], "/deep/mid/end", true), + (&["/deep/*/end"], "/deep/a/b/end", true), + (&["/deep/*/end"], "/exact", false), + (&["/deep/*/end"], "/exacts", false), + (&["/exact"], "/", false), + (&["/exact"], "/foo", false), + (&["/exact"], "/foo/", false), + (&["/exact"], "/foobar", false), + (&["/exact"], "/foo/bar", false), + (&["/exact"], "/:placeholder", false), + (&["/exact"], "/anything", false), + (&["/exact"], "/api/", false), + (&["/exact"], "/api/x/y", false), + (&["/exact"], "/a.b", false), + (&["/exact"], "/axb", false), + (&["/exact"], "/x+y", false), + (&["/exact"], "/xy", false), + (&["/exact"], "/deep/mid/end", false), + (&["/exact"], "/deep/a/b/end", false), + (&["/exact"], "/exact", true), + (&["/exact"], "/exacts", false), + (&["/foo", "/bar*"], "/barbaz", true), + (&["/foo", "/bar*"], "/nope", false), +]; + +#[test] +fn golden_static_routing() { + for (rules, pathname, expected) in STATIC_ROUTING_VECTORS { + let owned: Vec = rules.iter().map(|r| (*r).to_owned()).collect(); + let matcher = globs(&owned); + assert_eq!( + matcher.iter().any(|r| r.is_match(pathname)), + *expected, + "rules={rules:?} path={pathname:?}" + ); + } +} + +/// (rules as (pattern, target) pairs, host, pathname, upstream matched+replaced targets) +type RulesVector = ( + &'static [(&'static str, &'static str)], + &'static str, + &'static str, + &'static [&'static str], +); +const RULES_VECTORS: &[RulesVector] = &[ + (&[("/old", "/new")], "example.com", "/old", &["/new"]), + ( + &[("/blog/*", "/posts/:splat")], + "example.com", + "/blog/a/b", + &["/posts/a/b"], + ), + ( + &[("/users/:id", "/u/:id")], + "example.com", + "/users/42", + &["/u/42"], + ), + (&[("/:a/:b", "/:b/:a")], "example.com", "/x/y", &["/y/x"]), + ( + &[("https://:sub.example.com/", "/tenant/:sub")], + "acme.example.com", + "/", + &["/tenant/acme"], + ), + ( + &[("/multi/:x", "/:x/:x/:x")], + "example.com", + "/multi/z", + &["/z/z/z"], + ), + (&[("/a", "/1"), ("/b", "/2")], "example.com", "/b", &["/2"]), + (&[("/none", "/x")], "example.com", "/other", &[]), + ( + &[("/sp/*", "/dst/:splat/end")], + "example.com", + "/sp/deep/path", + &["/dst/deep/path/end"], + ), +]; + +#[test] +fn golden_rules_engine() { + for (rules, host, pathname, expected) in RULES_VECTORS { + let matcher = RulesMatcher::new( + rules + .iter() + .map(|(pattern, target)| ((*pattern).to_owned(), (*target).to_owned())), + ); + let got: Vec = matcher + .matches(host, pathname) + .map(|(target, repl)| replacer(target, &repl)) + .collect(); + let expected: Vec = expected.iter().map(|s| (*s).to_owned()).collect(); + assert_eq!( + got, expected, + "rules={rules:?} host={host:?} path={pathname:?}" + ); + } +} + +/// (template, replacements as (name, value) pairs, upstream replacer output) +type ReplacerVector = ( + &'static str, + &'static [(&'static str, &'static str)], + &'static str, +); +const REPLACER_VECTORS: &[ReplacerVector] = &[ + ("/:a/:b", &[("a", "1"), ("b", "2")], "/1/2"), + ( + "/static/:value/main.:value", + &[("value", "js")], + "/static/js/main.js", + ), + ("/no/placeholders", &[("x", "y")], "/no/placeholders"), + ("/:missing", &[], "/:missing"), + (":a:b", &[("a", "X"), ("b", "Y")], "XY"), +]; + +#[test] +fn golden_replacer() { + for (tpl, repl, expected) in REPLACER_VECTORS { + let owned: Vec<(String, String)> = repl + .iter() + .map(|(n, v)| ((*n).to_owned(), (*v).to_owned())) + .collect(); + assert_eq!( + &replacer(tpl, &owned), + expected, + "replacer({tpl:?}, {repl:?})" + ); + } +} diff --git a/src/rust/assets-router/ignore-test.rs b/src/rust/assets-router/ignore-test.rs new file mode 100644 index 00000000000..0e8b1cf07a3 --- /dev/null +++ b/src/rust/assets-router/ignore-test.rs @@ -0,0 +1,83 @@ +use super::*; + +fn ignored(patterns: &str, path: &str) -> bool { + Matcher::new(patterns, &mut |w| panic!("{w}")).ignored(path) +} + +#[test] +fn skips_patterns_that_cannot_compile() { + let mut warnings = Vec::new(); + let matcher = Matcher::new("[z-a]\n*.log", &mut |w| warnings.push(w)); + assert_eq!( + warnings, + ["Skipping the .assetsignore pattern [z-a]: it cannot be compiled."] + ); + assert!(matcher.ignored("/a.log")); + assert!(!matcher.ignored("/z")); +} + +#[test] +fn ignores_the_rule_files_at_the_root() { + for path in ["/_headers", "/_redirects", "/.assetsignore"] { + assert!(ignored("", path), "{path}"); + } + assert!(!ignored("", "/sub/_headers")); + assert!(!ignored("", "/index.html")); +} + +#[test] +fn a_pattern_without_a_slash_matches_at_any_depth() { + assert!(ignored("*.log", "/a.log")); + assert!(ignored("*.log", "/d/b.log")); + assert!(!ignored("*.log", "/a.log.txt")); + assert!(ignored("secret/", "/secret/key.txt")); + assert!(ignored("secret/", "/a/secret/key.txt")); + // A directory pattern does not match a file. + assert!(!ignored("secret/", "/secret")); +} + +#[test] +fn a_pattern_with_a_slash_is_anchored() { + assert!(ignored("/build", "/build/x.js")); + assert!(!ignored("/build", "/d/build/x.js")); + assert!(ignored("d/build", "/d/build")); + assert!(!ignored("d/build", "/e/d/build")); +} + +#[test] +fn globs() { + assert!(ignored("**/foo", "/foo")); + assert!(ignored("**/foo", "/a/b/foo")); + assert!(ignored("a/**/b", "/a/b")); + assert!(ignored("a/**/b", "/a/x/y/b")); + assert!(ignored("a/**", "/a/x/y")); + assert!(ignored("?.txt", "/a.txt")); + assert!(!ignored("?.txt", "/ab.txt")); + assert!(ignored("[ab].txt", "/b.txt")); + assert!(!ignored("[ab].txt", "/c.txt")); + assert!(ignored("[!ab].txt", "/c.txt")); + assert!(!ignored("*.txt", "/d.x/y")); + assert!(ignored(r"\*.txt", "/*.txt")); + assert!(!ignored(r"\*.txt", "/a.txt")); +} + +#[test] +fn negation_does_not_reach_under_an_ignored_directory() { + let patterns = "*.txt\n!keep.txt\ndir/\n!dir/keep"; + assert!(ignored(patterns, "/a.txt")); + assert!(!ignored(patterns, "/keep.txt")); + assert!(ignored(patterns, "/dir/keep")); + // The rule files can be served. + assert!(!ignored("!_headers", "/_headers")); +} + +#[test] +fn comments_whitespace_and_byte_order_mark() { + let patterns = "\u{feff}a.txt\r\n# b.txt\n\nc.txt \nd\\ "; + assert!(ignored(patterns, "/a.txt")); + assert!(!ignored(patterns, "/b.txt")); + assert!(!ignored(patterns, "/# b.txt")); + assert!(ignored(patterns, "/c.txt")); + assert!(ignored(patterns, "/d ")); + assert!(!ignored(patterns, "/d")); +} diff --git a/src/rust/assets-router/ignore.rs b/src/rust/assets-router/ignore.rs new file mode 100644 index 00000000000..948de3bc1a3 --- /dev/null +++ b/src/rust/assets-router/ignore.rs @@ -0,0 +1,119 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! `.assetsignore`: gitignore-style patterns naming files that are not assets. `_headers`, +//! `_redirects` and `.assetsignore` themselves are always ignored. + +use regex::Regex; + +/// Each pattern: whether it negates, whether it names directories only, and its regex. +pub struct Matcher(Vec<(bool, bool, Regex)>); + +impl Matcher { + /// A pattern that cannot be compiled is skipped with a warning to `warn`. + pub fn new(patterns: &str, warn: &mut dyn FnMut(String)) -> Self { + // A byte order mark is not part of the first pattern. + let patterns = patterns.strip_prefix('\u{feff}').unwrap_or(patterns); + let lines = ["/.assetsignore", "/_redirects", "/_headers"] + .into_iter() + .chain(patterns.split('\n')); + Self( + lines + .filter_map(|line| { + let line = trim_end(line.strip_suffix('\r').unwrap_or(line)); + if line.is_empty() || line.starts_with('#') { + return None; + } + let (negate, line) = + line.strip_prefix('!').map_or((false, line), |l| (true, l)); + let (dir_only, line) = + line.strip_suffix('/').map_or((false, line), |l| (true, l)); + let re = glob_regex(line.strip_prefix('/').unwrap_or(line)); + let re = if line.contains('/') { + format!("^{re}$") + } else { + format!("^(?:.*/)?{re}$") + }; + let Ok(re) = Regex::new(&re) else { + warn(format!( + "Skipping the .assetsignore pattern {line}: it cannot be compiled." + )); + return None; + }; + Some((negate, dir_only, re)) + }) + .collect(), + ) + } + + /// Whether `pathname`, relative to the directory and starting with `/`, is ignored. As in + /// git, a file under an ignored directory is ignored, whatever later patterns say. + pub fn ignored(&self, pathname: &str) -> bool { + let segments: Vec<&str> = pathname.split('/').filter(|s| !s.is_empty()).collect(); + (1..=segments.len()).any(|i| { + let (path, dir) = (segments[..i].join("/"), i < segments.len()); + self.0 + .iter() + .fold(false, |ignored, (negate, dir_only, regex)| { + if (!dir_only || dir) && regex.is_match(&path) { + !negate + } else { + ignored + } + }) + }) + } +} + +/// `line` without trailing whitespace, but for one whitespace character a backslash escapes. +fn trim_end(line: &str) -> &str { + let trimmed = line.trim_end(); + let escaped = line[trimmed.len()..] + .chars() + .next() + .filter(|_| trimmed.ends_with('\\')); + &line[..trimmed.len() + escaped.map_or(0, char::len_utf8)] +} + +fn glob_regex(pattern: &str) -> String { + let p: Vec = pattern.chars().collect(); + let escape = |c: char| regex::escape(c.encode_utf8(&mut [0; 4])); + let mut re = String::new(); + let mut i = 0; + while i < p.len() { + let end = p.iter().skip(i + 2).position(|&c| c == ']'); + match (p[i], end.map(|end| i + 2 + end)) { + ('*', _) if p.get(i + 1) == Some(&'*') => { + i += 1; + if p.get(i + 1) == Some(&'/') { + i += 1; + re += "(?:.*/)?"; + } else { + re += ".*"; + } + } + ('*', _) => re += "[^/]*", + ('?', _) => re += "[^/]", + ('[', Some(end)) => { + let body: String = p[i + 1..end].iter().collect(); + let body = body + .strip_prefix('!') + .map_or_else(|| body.clone(), |b| format!("^{b}")); + re += &format!("[{}]", body.replace('\\', "\\\\").replace('[', "\\[")); + i = end; + } + ('\\', _) if i + 1 < p.len() => { + i += 1; + re += &escape(p[i]); + } + (c, _) => re += &escape(c), + } + i += 1; + } + re +} + +#[cfg(test)] +#[path = "ignore-test.rs"] +mod tests; diff --git a/src/rust/assets-router/intent-test.rs b/src/rust/assets-router/intent-test.rs new file mode 100644 index 00000000000..38c52b7900c --- /dev/null +++ b/src/rust/assets-router/intent-test.rs @@ -0,0 +1,773 @@ +use std::collections::HashSet; + +use url::Url; + +use super::*; + +/// Assets at `paths`, each its path for an `ETag`. +fn assets(paths: &[&str]) -> impl Fn(&str) -> Option { + let paths: HashSet = paths.iter().map(|path| (*path).to_owned()).collect(); + move |path| paths.contains(path).then(|| path.to_owned()) +} + +/// An `ASSET`, whichever asset it is. +const ASSET: AssetResponse = AssetResponse::Asset(Asset { + path: String::new(), + etag: String::new(), + status: 200, + not_found: false, +}); + +/// The `_redirects` rule matching a test request. +#[derive(Clone, Copy, Debug)] +enum RedirectAction<'a> { + None, + Redirect, + Rewrite(&'a str), +} + +#[derive(Clone, Copy)] +struct TestRequest<'a> { + pathname: &'a str, + method: Method, + sec_fetch_mode: Option<&'a str>, + redirect: RedirectAction<'a>, +} + +fn request(pathname: &str) -> TestRequest<'_> { + TestRequest { + pathname, + method: Method::Get, + sec_fetch_mode: None, + redirect: RedirectAction::None, + } +} + +fn respond_under( + base_path: &str, + request: TestRequest<'_>, + config: IntentConfig, + exists: &impl Fn(&str) -> Option, +) -> AssetResponse { + let url = Url::parse(&format!("https://example.com{}", request.pathname)).unwrap(); + let redirect = match request.redirect { + RedirectAction::None => None, + RedirectAction::Redirect => Some(Redirect { + status: 301, + to: "/redirected".into(), + }), + RedirectAction::Rewrite(to) => Some(Redirect { + status: 200, + to: to.into(), + }), + }; + let response = respond( + &Request { + url: &url, + method: request.method, + sec_fetch_mode: request.sec_fetch_mode, + sec_fetch_dest: None, + }, + redirect.as_ref(), + config, + &BasePath::normalize(base_path).unwrap(), + exists, + ); + match response.unwrap() { + AssetResponse::Asset(_) => ASSET, + response => response, + } +} + +/// Mirrors upstream `canFetch`. +fn can_fetch( + request: TestRequest<'_>, + config: IntentConfig, + exists: &impl Fn(&str) -> Option, +) -> bool { + let config = config.for_can_fetch(request.sec_fetch_mode); + respond_under("/", request, config, exists) != AssetResponse::NoIntent +} + +// Upstream: `canFetch > should return "true" if for exact and nearby assets with +// html_handling on` (L1083). +#[test] +fn canfetch_auto_trailing_slash_matches_nearby_paths() { + let exists = assets(&["/foo.html"]); + let config = IntentConfig::default(); + for pathname in ["/foo.html", "/foo", "/foo/"] { + assert!(can_fetch(request(pathname), config, &exists), "{pathname}"); + } +} + +// Upstream: `canFetch > should not consider 404s or SPAs` (L1125). +#[test] +fn canfetch_ignores_not_found_fallbacks() { + let exists = assets(&["/404.html", "/index.html", "/foo.html"]); + for not_found_handling in [ + NotFoundHandling::SinglePageApplication, + NotFoundHandling::NotFoundPage, + ] { + let config = IntentConfig { + not_found_handling, + ..IntentConfig::default() + }; + assert!(can_fetch(request("/foo"), config, &exists)); + assert!(!can_fetch(request("/bar"), config, &exists)); + assert!(can_fetch(request("/"), config, &exists)); + assert!(can_fetch(request("/404"), config, &exists)); + } +} + +fn assert_static_routing( + not_found_handling: NotFoundHandling, + sec_fetch_mode: Option<&str>, + navigation_prefers_asset_serving: bool, +) { + let exists = assets(&["/404.html", "/index.html"]); + let config = IntentConfig { + not_found_handling, + has_static_routing: true, + navigation_prefers_asset_serving, + ..IntentConfig::default() + }; + for pathname in ["/foo", "/bar", "/", "/404"] { + let mut request = request(pathname); + request.sec_fetch_mode = sec_fetch_mode; + assert!(can_fetch(request, config, &exists), "{pathname}"); + } +} + +macro_rules! static_routing_test { + ($name:ident, $nfh:expr, $header:expr, $flag:expr) => { + #[test] + fn $name() { + assert_static_routing($nfh, $header, $flag); + } + }; +} + +static_routing_test!( + canfetch_static_routing_forces_true_spa_no_header_flag_on, + NotFoundHandling::SinglePageApplication, + None, + true +); +static_routing_test!( + canfetch_static_routing_forces_true_spa_no_header_flag_off, + NotFoundHandling::SinglePageApplication, + None, + false +); +static_routing_test!( + canfetch_static_routing_forces_true_spa_navigate_flag_on, + NotFoundHandling::SinglePageApplication, + Some("navigate"), + true +); +static_routing_test!( + canfetch_static_routing_forces_true_spa_navigate_flag_off, + NotFoundHandling::SinglePageApplication, + Some("navigate"), + false +); +static_routing_test!( + canfetch_static_routing_forces_true_404_no_header_flag_on, + NotFoundHandling::NotFoundPage, + None, + true +); +static_routing_test!( + canfetch_static_routing_forces_true_404_no_header_flag_off, + NotFoundHandling::NotFoundPage, + None, + false +); +static_routing_test!( + canfetch_static_routing_forces_true_404_navigate_flag_on, + NotFoundHandling::NotFoundPage, + Some("navigate"), + true +); +static_routing_test!( + canfetch_static_routing_forces_true_404_navigate_flag_off, + NotFoundHandling::NotFoundPage, + Some("navigate"), + false +); + +// Upstream: `canFetch > should return true even for a bad method` (L1256). +#[test] +fn canfetch_method_agnostic() { + let exists = assets(&["/foo.html"]); + let mut req = request("/foo"); + req.method = Method::Other; + assert!(can_fetch(req, IntentConfig::default(), &exists)); + req.pathname = "/bar"; + assert!(!can_fetch(req, IntentConfig::default(), &exists)); +} + +// Upstream: `canFetch > should return true for custom redirects without underlying assets` +// (L1286). Redirect matching is represented by RedirectAction rather than depending on the +// independently ported redirects module. +#[test] +fn canfetch_honors_redirects_without_asset() { + let exists = assets(&["/404.html", "/does-exist"]); + assert!(can_fetch( + request("/does-exist"), + IntentConfig::default(), + &exists + )); + assert!(!can_fetch( + request("/no-match"), + IntentConfig::default(), + &exists + )); + + let mut req = request("/redirect"); + req.redirect = RedirectAction::Redirect; + assert!(can_fetch(req, IntentConfig::default(), &exists)); + + req.pathname = "/proxy-valid"; + req.redirect = RedirectAction::Rewrite("/does-exist"); + assert!(can_fetch(req, IntentConfig::default(), &exists)); + + let missing = assets(&[]); + req.pathname = "/proxy-invalid"; + req.redirect = RedirectAction::Rewrite("/no-match"); + assert!(can_fetch(req, IntentConfig::default(), &missing)); + assert!(can_fetch( + req, + IntentConfig { + not_found_handling: NotFoundHandling::NotFoundPage, + ..IntentConfig::default() + }, + &missing, + )); +} + +// Upstream: `canFetch > assets_navigation_prefers_asset_serving > it.for(matrix)` (L1484). +// Upstream crosses seven compatibility-option rows (date × enable/disable flags) with the +// other dimensions. In the inlined runtime, workerd resolves the compatibility date and the +// enable/disable flags into a single bool before the router runs (see compatibility-date.capnp +// `assetsSecFetchModeNavigateHeaderPrefersAssetServing`, which carries `$compatEnableDate` +// and `$compatDisableFlag`), so those seven rows collapse to their resolved enabled values +// F,T,F,T,T,T,F. Each is crossed with not-found rows [default:F, none:F, 404:T, SPA:T], header +// rows [none:F, navigate:T, cors:F], and static-routing rows [false:F, true:T]. +// 7 * 4 * 3 * 2 = exactly 168 rows. +#[test] +fn canfetch_navigation_matrix() { + const NAVIGATION_ENABLED: [bool; 7] = [false, true, false, true, true, true, false]; + const NOT_FOUND: [(NotFoundHandling, bool); 4] = [ + (NotFoundHandling::None, false), + (NotFoundHandling::None, false), + (NotFoundHandling::NotFoundPage, true), + (NotFoundHandling::SinglePageApplication, true), + ]; + const HEADERS: [(Option<&str>, bool); 3] = [ + (None, false), + (Some("navigate"), true), + (Some("cors"), false), + ]; + const STATIC_ROUTING: [(bool, bool); 2] = [(false, false), (true, true)]; + + let exists = assets(&["/404.html", "/index.html", "/foo.html"]); + let mut rows = 0; + for navigation_prefers_asset_serving in NAVIGATION_ENABLED { + for (not_found_handling, has_fallback) in NOT_FOUND { + for (sec_fetch_mode, is_navigation) in HEADERS { + for (has_static_routing, static_keeps_fallback) in STATIC_ROUTING { + rows += 1; + let config = IntentConfig { + not_found_handling, + has_static_routing, + navigation_prefers_asset_serving, + ..IntentConfig::default() + }; + let expected = (static_keeps_fallback && has_fallback) + || (navigation_prefers_asset_serving && has_fallback && is_navigation); + let mut req = request("/bar"); + req.sec_fetch_mode = sec_fetch_mode; + assert_eq!( + can_fetch(req, config, &exists), + expected, + "row {rows}: navigation={navigation_prefers_asset_serving}, nfh={not_found_handling:?}, mode={sec_fetch_mode:?}, static={has_static_routing}" + ); + for pathname in ["/foo", "/", "/404"] { + req.pathname = pathname; + assert!(can_fetch(req, config, &exists), "row {rows}: {pathname}"); + } + } + } + } + } + assert_eq!(rows, 168); +} + +#[test] +fn html_handling_modes_and_not_found_resolution() { + let exists = assets(&["/foo.html", "/dir/index.html", "/404.html"]); + let asset = |path: &str, status| { + Some(Intent::Asset(Asset { + path: path.into(), + etag: path.into(), + status, + not_found: status == 404, + })) + }; + let redirect = |destination: &str| Some(Intent::Redirect(destination.to_owned())); + + let auto = IntentConfig::default(); + assert_eq!(get_intent("/foo", auto, &exists), asset("/foo.html", 200)); + assert_eq!(get_intent("/foo/", auto, &exists), redirect("/foo")); + + let force = IntentConfig { + html_handling: HtmlHandling::ForceTrailingSlash, + ..auto + }; + assert_eq!(get_intent("/foo", force, &exists), redirect("/foo/")); + + let drop = IntentConfig { + html_handling: HtmlHandling::DropTrailingSlash, + ..auto + }; + assert_eq!( + get_intent("/dir", drop, &exists), + asset("/dir/index.html", 200) + ); + + let fallback = IntentConfig { + html_handling: HtmlHandling::None, + not_found_handling: NotFoundHandling::NotFoundPage, + ..auto + }; + assert_eq!( + get_intent("/missing/nested", fallback, &exists), + asset("/404.html", 404) + ); +} + +// Upstream compares only the `ETag`, so identical files under the two names redirect. +#[test] +fn safe_redirect_compares_only_the_etag() { + let same = |path: &str| { + ["/about.html", "/about/index.html"] + .contains(&path) + .then(String::new) + }; + let redirect = |destination: &str| Some(Intent::Redirect(destination.to_owned())); + let force = html_handling(HtmlHandling::ForceTrailingSlash); + assert_eq!(get_intent("/about.html", force, &same), redirect("/about/")); + let drop = html_handling(HtmlHandling::DropTrailingSlash); + assert_eq!( + get_intent("/about/index.html", drop, &same), + redirect("/about") + ); +} + +// Upstream: `handleRequest` answers a matching asset with 405 for anything but GET and HEAD, +// generated redirects included (base-path-handler.test.ts `returns 405 for POST at the base +// root` (L738) and `allows HEAD to use the base-root redirect` (L764)). +#[test] +fn only_get_and_head_reach_an_asset() { + let exists = assets(&["/foo.html", "/index.html"]); + let respond = |base_path, pathname, method| { + respond_under( + base_path, + TestRequest { + method, + ..request(pathname) + }, + IntentConfig::default(), + &exists, + ) + }; + assert_eq!(respond("/", "/foo", Method::Head), ASSET); + assert_eq!( + respond("/", "/foo", Method::Other), + AssetResponse::MethodNotAllowed + ); + assert_eq!( + respond("/", "/missing", Method::Other), + AssetResponse::NoIntent + ); + assert_eq!( + respond("/subpath", "/subpath", Method::Other), + AssetResponse::MethodNotAllowed + ); + assert_eq!( + respond("/subpath", "/subpath", Method::Head), + redirect("/subpath/") + ); +} + +// Upstream: `handleRedirects` keeps 301, 303, 307 and 308 and answers any other status with +// 302. The location inherits the request's query when the rule has none. +#[test] +fn redirect_rule_status_and_location() { + let exists = assets(&[]); + for (status, expected) in [ + (301, 301), + (302, 302), + (303, 303), + (307, 307), + (308, 308), + (300, 302), + (418, 302), + ] { + let url = Url::parse("https://example.com/old?source=test").unwrap(); + let response = respond( + &Request { + url: &url, + method: Method::Other, + sec_fetch_mode: None, + sec_fetch_dest: None, + }, + Some(&Redirect { + status, + to: "/new".into(), + }), + IntentConfig::default(), + &BasePath::default(), + &exists, + ); + assert_eq!( + response, + Ok(AssetResponse::Redirect { + status: expected, + location: "/new?source=test".into(), + }), + "{status}" + ); + } +} + +// A rewrite is looked up, and canonicalized, as its target. +#[test] +fn rewrite_resolves_its_target() { + let exists = assets(&["/users/id.html"]); + let rewrite = |to| { + respond_under( + "/", + TestRequest { + redirect: RedirectAction::Rewrite(to), + ..request("/users/12345?tab=posts") + }, + IntentConfig::default(), + &exists, + ) + }; + assert_eq!(rewrite("/users/id"), ASSET); + assert_eq!(rewrite("/users/id.html"), redirect("/users/id?tab=posts")); + assert_eq!(rewrite("/users/missing"), AssetResponse::NotFound); +} + +fn redirect(location: &str) -> AssetResponse { + AssetResponse::Redirect { + status: 307, + location: location.to_owned(), + } +} + +fn html_handling(html_handling: HtmlHandling) -> IntentConfig { + IntentConfig { + html_handling, + ..IntentConfig::default() + } +} + +// Upstream: base-path-handler.test.ts `serves an in-prefix asset by looking up the stripped +// path` (L69), `returns NoIntent for an off-prefix request` (L173) and `is segment-aware: +// /subpath-other is off-prefix` (L191). +#[test] +fn base_path_is_stripped_before_asset_lookup() { + let config = html_handling(HtmlHandling::None); + let foo = assets(&["/foo"]); + let everything = |path: &str| Some(path.to_owned()); + assert_eq!( + respond_under("/subpath", request("/subpath/foo"), config, &foo), + ASSET + ); + assert_eq!( + respond_under("/subpath", request("/foo"), config, &foo), + AssetResponse::NoIntent + ); + assert_eq!( + respond_under("/subpath", request("/other"), config, &everything), + AssetResponse::NoIntent + ); + assert_eq!( + respond_under( + "/subpath", + request("/subpath-other/foo"), + config, + &everything + ), + AssetResponse::NoIntent + ); +} + +// Upstream: base-path-handler.test.ts `canonicalizes $publicPath once before prefix lookup` +// (L125). +#[test] +fn base_path_lookup_follows_path_canonicalization() { + for (public_path, asset_path, location) in [ + ("/subpath//foo", "/foo", "/subpath/foo"), + ("/subpath%2Ffoo", "/foo", "/subpath/foo"), + ("/subpath/%ZZ", "/%ZZ", "/subpath/%25ZZ"), + ("/subpath/%FF", "/%FF", "/subpath/%25FF"), + ] { + assert_eq!( + respond_under( + "/subpath", + request(public_path), + html_handling(HtmlHandling::None), + &assets(&[asset_path]), + ), + redirect(location), + "{public_path}" + ); + } +} + +// Upstream: base-path-handler.test.ts `re-prefixes a generated HTML redirect location` (L209) +// and `preserves the query string on a generated redirect` (L234). +#[test] +fn base_path_is_added_to_generated_redirects() { + let exists = assets(&["/foo/index.html"]); + assert_eq!( + respond_under( + "/subpath", + request("/subpath/foo"), + IntentConfig::default(), + &exists + ), + redirect("/subpath/foo/") + ); + assert_eq!( + respond_under( + "/subpath", + request("/subpath/foo?a=1"), + IntentConfig::default(), + &exists + ), + redirect("/subpath/foo/?a=1") + ); +} + +// Upstream: base-path-handler.test.ts `resolves an in-prefix SPA fallback to the +// asset-directory index` (L259). +#[test] +fn base_path_not_found_handling_resolves_in_the_asset_directory() { + let config = IntentConfig { + html_handling: HtmlHandling::None, + not_found_handling: NotFoundHandling::SinglePageApplication, + ..IntentConfig::default() + }; + let exists = assets(&["/index.html"]); + assert_eq!( + respond_under("/subpath", request("/subpath/missing"), config, &exists), + ASSET + ); + assert_eq!( + respond_under("/subpath", request("/missing"), config, &exists), + AssetResponse::NoIntent + ); +} + +// Upstream: base-path-handler.test.ts `accepts an authored 200 rewrite with an in-prefix +// public target` (L286), `claims an authored 200 rewrite whose target is off-prefix` (L320), +// `ignores an authored 200 rewrite from an off-prefix source` (L365), `ignores an off-prefix +// authored non-200 redirect` (L399) and `gives an authored redirect precedence at the base +// root` (L640). +#[test] +fn base_path_applies_to_the_source_of_a_redirect_rule() { + let config = html_handling(HtmlHandling::ForceTrailingSlash); + let exists = assets(&["/foo", "/index.html"]); + let authored = AssetResponse::Redirect { + status: 301, + location: "/redirected".into(), + }; + for (pathname, redirect, expected) in [ + ( + "/subpath/example", + RedirectAction::Rewrite("/subpath/foo"), + ASSET, + ), + ( + "/subpath/example", + RedirectAction::Rewrite("/foo"), + AssetResponse::NotFound, + ), + ( + "/other", + RedirectAction::Rewrite("/subpath/foo"), + AssetResponse::NoIntent, + ), + ("/other", RedirectAction::Redirect, AssetResponse::NoIntent), + ( + "/subpath/example", + RedirectAction::Redirect, + authored.clone(), + ), + ("/subpath", RedirectAction::Redirect, authored), + ] { + assert_eq!( + respond_under( + "/subpath", + TestRequest { + redirect, + ..request(pathname) + }, + config, + &exists + ), + expected, + "{pathname} {redirect:?}" + ); + } +} + +// Upstream: base-path-handler.test.ts `redirects the base root to a trailing slash under +// force-trailing-slash` (L540), `redirects the base root under auto-trailing-slash and +// preserves the query` (L566), `redirects the slash base root under drop-trailing-slash` +// (L591) and `does not generate a base-root redirect under html_handling none` (L616). +#[test] +fn base_root_redirects_to_its_canonical_spelling() { + let exists = assets(&["/index.html"]); + for (handling, pathname, expected) in [ + ( + HtmlHandling::ForceTrailingSlash, + "/subpath", + redirect("/subpath/"), + ), + (HtmlHandling::ForceTrailingSlash, "/subpath/", ASSET), + ( + HtmlHandling::AutoTrailingSlash, + "/subpath?q=1", + redirect("/subpath/?q=1"), + ), + (HtmlHandling::AutoTrailingSlash, "/subpath/", ASSET), + ( + HtmlHandling::DropTrailingSlash, + "/subpath/?q=1", + redirect("/subpath?q=1"), + ), + (HtmlHandling::DropTrailingSlash, "/subpath", ASSET), + (HtmlHandling::None, "/subpath", AssetResponse::NoIntent), + (HtmlHandling::None, "/subpath/", AssetResponse::NoIntent), + ] { + assert_eq!( + respond_under( + "/subpath", + request(pathname), + html_handling(handling), + &exists + ), + expected, + "{handling:?} {pathname}" + ); + } +} + +// Upstream: base-path-handler.test.ts `does not redirect a base root without an index asset` +// (L669) and `does not generate a slash redirect for a non-index root asset` (L690). +#[test] +fn base_root_redirect_requires_an_index() { + let config = html_handling(HtmlHandling::ForceTrailingSlash); + assert_eq!( + respond_under("/subpath", request("/subpath"), config, &assets(&[])), + AssetResponse::NoIntent + ); + assert_eq!( + respond_under("/subpath", request("/subpath"), config, &assets(&["/"])), + ASSET + ); +} + +// Upstream: base-path-handler.test.ts `percent-encodes a Unicode base-root redirect` (L711). +#[test] +fn base_root_redirect_is_percent_encoded() { + assert_eq!( + respond_under( + "/路径", + request("/%E8%B7%AF%E5%BE%84"), + html_handling(HtmlHandling::ForceTrailingSlash), + &assets(&["/index.html"]), + ), + redirect("/%E8%B7%AF%E5%BE%84/") + ); +} + +// A root base has a single spelling, so it never takes the base-root redirect. +#[test] +fn root_base_path_has_no_base_root_redirect() { + let exists = assets(&["/index.html"]); + for handling in [ + HtmlHandling::AutoTrailingSlash, + HtmlHandling::ForceTrailingSlash, + HtmlHandling::DropTrailingSlash, + ] { + assert_eq!( + respond_under("/", request("/"), html_handling(handling), &exists), + ASSET, + "{handling:?}" + ); + } +} + +#[test] +fn decode_path_keeps_segments_decode_uri_component_throws_on() { + assert_eq!(decode_path("/hello%20world/%2E%2E"), "/hello world/.."); + for segment in ["%", "%A", "%ZZ", "%+1", "%A0", "%A0%A0"] { + assert_eq!(decode_path(&format!("/{segment}")), format!("/{segment}")); + } +} + +// Upstream: handler.test.ts `cannot fetch assets outside of configured path` (L133). +// +// The handler does not resolve `..`; it decodes, and the host refuses the path. Both the raw `..` +// and the percent-encoded `%2E%2E` forms decode to the same path, so neither escapes the +// directory. +#[test] +fn decode_path_blocks_traversal() { + assert_eq!(decode_path("/blog/../test"), "/blog/../test"); +} + +#[test] +fn decode_path_blocks_percent_encoded_traversal() { + // `%2E%2E` → `..`, identical to the raw form above. + assert_eq!(decode_path("/blog/%2E%2E/test"), "/blog/../test"); +} + +// Upstream: handler.test.ts `returns expected responses for malformed path` (L190). +// +// `%A0` is a valid percent escape but not valid UTF-8, so `decodeURIComponent` throws and the +// segment is kept verbatim. This lets `/%A0%A0` still match an asset `/%A0%A0.html`. +#[test] +fn decode_path_preserves_non_utf8_percent_sequences() { + assert_eq!(decode_path("/%A0"), "/%A0"); + assert_eq!(decode_path("/%A0%A0"), "/%A0%A0"); +} + +#[test] +fn decode_path_collapses_multiple_slashes() { + assert_eq!(decode_path("/foo//bar///baz"), "/foo/bar/baz"); + assert_eq!(decode_path("//"), "/"); +} + +#[test] +fn encode_path_encodes_reserved_but_not_slashes() { + // Brackets are reserved and get encoded; slashes are the segment separator. + assert_eq!(encode_path("/[boop]"), "/%5Bboop%5D"); + // Unreserved chars pass through. + assert_eq!(encode_path("/foo-bar_baz.html"), "/foo-bar_baz.html"); + // Spaces encode to %20 (not +). + assert_eq!(encode_path("/a b"), "/a%20b"); +} + +#[test] +fn encode_path_round_trips_unreserved_marks() { + // encodeURIComponent leaves - _ . ! ~ * ' ( ) untouched. + assert_eq!(encode_path("/-_.!~*'()"), "/-_.!~*'()"); +} diff --git a/src/rust/assets-router/intent.rs b/src/rust/assets-router/intent.rs new file mode 100644 index 00000000000..f9d6d43e024 --- /dev/null +++ b/src/rust/assets-router/intent.rs @@ -0,0 +1,365 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! What a directory of assets answers to a request. +//! +//! Ported from workers-sdk's workers-shared, `asset-worker/src/handler.ts`. + +use serde::Deserialize; + +use super::Method; +use super::Request; +use super::base_path::BasePath; +use super::rules::Redirect; +use super::rules::resolve_location; + +#[derive(Clone, Copy, Debug, Default, Deserialize, Eq, PartialEq)] +#[serde(rename_all = "kebab-case")] +pub enum HtmlHandling { + #[default] + AutoTrailingSlash, + ForceTrailingSlash, + DropTrailingSlash, + None, +} + +#[derive(Clone, Copy, Debug, Default, Deserialize, Eq, PartialEq)] +#[serde(rename_all = "kebab-case")] +pub enum NotFoundHandling { + SinglePageApplication, + #[serde(rename = "404-page")] + NotFoundPage, + #[default] + None, +} + +/// A file that answers a request, and how. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Asset { + /// The file, relative to the directory, starting with `/`. + pub path: String, + pub etag: String, + pub status: u16, + /// Whether `not_found_handling` chose it. + pub not_found: bool, +} + +type Exists<'a> = &'a dyn Fn(&str) -> Option; + +/// The asset at a path, from `exists`, which gives a file's `ETag`. +fn found(exists: &impl Fn(&str) -> Option) -> impl Fn(&str) -> Option { + |path| { + exists(path).map(|etag| Asset { + path: path.to_owned(), + etag, + status: 200, + not_found: false, + }) + } +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum Intent { + Asset(Asset), + /// Redirect to another spelling of the path, relative to the directory. + Redirect(String), +} + +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub struct IntentConfig { + pub html_handling: HtmlHandling, + pub not_found_handling: NotFoundHandling, + pub has_static_routing: bool, + pub navigation_prefers_asset_serving: bool, +} + +impl IntentConfig { + /// The config `canFetch` resolves with: a not-found fallback only counts under static + /// routing, or for a navigation when the compatibility flag is on. + pub(crate) fn for_can_fetch(mut self, sec_fetch_mode: Option<&str>) -> Self { + let navigation = sec_fetch_mode == Some("navigate"); + if !(self.has_static_routing || (self.navigation_prefers_asset_serving && navigation)) { + self.not_found_handling = NotFoundHandling::None; + } + self + } +} + +/// What the assets answer to a request. Mirrors `getResponseOrAssetIntent`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum AssetResponse { + /// Nothing: the user Worker, if any, handles the request. + NoIntent, + Redirect { + status: u16, + location: String, + }, + MethodNotAllowed, + /// A `_redirects` rewrite whose target has no asset. + NotFound, + Asset(Asset), +} + +/// What the assets answer to `request`, which `redirect` matches. +pub(crate) fn respond( + request: &Request<'_>, + redirect: Option<&Redirect>, + config: IntentConfig, + base_path: &BasePath, + exists: &impl Fn(&str) -> Option, +) -> Result { + let exists = &found(exists); + // The base applies to the request's own path first, so a rewrite cannot bring a request from + // outside it into the assets. + let pathname = request.url.path(); + let decoded = decode_path(pathname); + if base_path.strip(&decoded).is_none() { + return Ok(AssetResponse::NoIntent); + } + let rewrite = match redirect { + Some(redirect) if redirect.status == 200 => Some(request.url.join(&redirect.to)?), + Some(redirect) => { + return Ok(AssetResponse::Redirect { + status: match redirect.status { + status @ (301 | 303 | 307 | 308) => status, + _ => 302, + }, + location: resolve_location(request.url, &redirect.to)?, + }); + } + None => None, + }; + // A rewrite claims the request: where its target has no asset the answer is a 404. + let (pathname, decoded, no_intent) = match &rewrite { + Some(target) => ( + target.path(), + decode_path(target.path()), + AssetResponse::NotFound, + ), + None => (pathname, decoded, AssetResponse::NoIntent), + }; + let Some(asset_path) = base_path.strip(&decoded) else { + return Ok(no_intent); + }; + let base_root = rewrite + .is_none() + .then(|| base_root_intent(&decoded, base_path, config, exists)) + .flatten(); + let Some(intent) = base_root.or_else(|| intent(asset_path, config, exists, false)) else { + return Ok(no_intent); + }; + if request.method == Method::Other { + return Ok(AssetResponse::MethodNotAllowed); + } + let (destination, asset) = match intent { + Intent::Redirect(destination) => (encode_path(&base_path.add(&destination)), None), + Intent::Asset(asset) => (encode_path(&decoded), Some(asset)), + }; + match asset.filter(|_| destination == pathname) { + Some(asset) => Ok(AssetResponse::Asset(asset)), + None => Ok(AssetResponse::Redirect { + status: 307, + location: format!("{destination}{}", super::affix("?", request.url.query())), + }), + } +} + +/// The redirect from the base root to its spelling under `html_handling`, `/base` to `/base/` +/// or the reverse. Mirrors `getBaseRootIntent`. +fn base_root_intent( + decoded: &str, + base_path: &BasePath, + config: IntentConfig, + exists: Exists<'_>, +) -> Option { + let destination = match config.html_handling { + HtmlHandling::AutoTrailingSlash | HtmlHandling::ForceTrailingSlash => { + (base_path.without_trailing_slash() == Some(decoded)).then_some("/") + } + HtmlHandling::DropTrailingSlash => { + (!base_path.is_root() && decoded == base_path.as_str()).then_some("") + } + HtmlHandling::None => None, + }?; + exists("/index.html")?; + Some(Intent::Redirect(destination.to_owned())) +} + +pub fn get_intent( + pathname: &str, + config: IntentConfig, + exists: &impl Fn(&str) -> Option, +) -> Option { + intent(pathname, config, &found(exists), false) +} + +/// A step of `getIntent`: serve a file, or redirect to a destination if `safeRedirect` allows. +type Step = (String, Option); + +fn serve(file: impl Into) -> Step { + (file.into(), None) +} + +fn to(file: impl Into, destination: impl Into) -> Step { + (file.into(), Some(destination.into())) +} + +/// The steps `getIntent` tries for `p` under `html_handling`, in order. +fn steps(p: &str, html_handling: HtmlHandling) -> Vec { + use HtmlHandling as H; + let suffixes = ["/index", "/index.html", "/", ".html"]; + let (suffix, x) = suffixes + .into_iter() + .find_map(|suffix| Some((suffix, p.strip_suffix(suffix)?))) + .unwrap_or_default(); + let (xs, xh, xi) = ( + format!("{x}/"), + format!("{x}.html"), + format!("{x}/index.html"), + ); + let (ps, ph, pi) = ( + format!("{p}/"), + format!("{p}.html"), + format!("{p}index.html"), + ); + let mut steps = match (html_handling, suffix) { + (H::None, _) => return vec![serve(p)], + (H::AutoTrailingSlash, "/index") => vec![serve(p), to(&ph, &xs), to(&xh, x)], + (H::AutoTrailingSlash, "/index.html") => vec![to(p, &xs), to(&xh, x)], + (H::AutoTrailingSlash, "/") => vec![serve(&pi), to(&xh, x)], + (H::AutoTrailingSlash, ".html") => vec![to(p, x), to(&xi, &xs)], + (H::ForceTrailingSlash, "/index") => vec![serve(p), to(&ph, &xs), to(&xh, &xs)], + (H::ForceTrailingSlash, "/index.html") => vec![to(p, &xs), to(&xh, &xs)], + (H::ForceTrailingSlash, "/") => vec![serve(&pi), serve(&xh)], + (H::ForceTrailingSlash, ".html") => vec![to(p, &xs), serve(p), to(&xi, &xs)], + (H::DropTrailingSlash, "/index") if p == "/index" => vec![serve(p), to("/index.html", "/")], + (H::DropTrailingSlash, "/index") => vec![serve(p), to(&xh, x), to(&ph, x)], + (H::DropTrailingSlash, "/index.html") if p == "/index.html" => vec![to(p, "/")], + (H::DropTrailingSlash, "/index.html") => vec![to(p, x), serve(p), to(&xh, x)], + (H::DropTrailingSlash, "/") if p == "/" => vec![serve("/index.html")], + (H::DropTrailingSlash, "/" | ".html") => vec![to(&xh, x), to(&xi, x)], + _ => vec![], + }; + let pi_ = format!("{p}/index.html"); + steps.extend(match html_handling { + H::ForceTrailingSlash => [serve(p), to(&ph, &ps), to(&pi_, &ps)], + H::AutoTrailingSlash => [serve(p), serve(&ph), to(&pi_, &ps)], + _ => [serve(p), serve(&ph), serve(&pi_)], + }); + steps +} + +fn intent(pathname: &str, config: IntentConfig, exists: Exists<'_>, skip: bool) -> Option { + steps(pathname, config.html_handling) + .into_iter() + .find_map(|(file, destination)| match destination { + None => exists(&file).map(Intent::Asset), + Some(destination) => safe_redirect(&file, &destination, config, exists, skip), + }) + .or_else(|| not_found(pathname, config, exists)) +} + +/// A redirect to `destination`, if nothing is there and it resolves to an asset with `file`'s +/// `ETag`. Upstream compares only the `ETag`: with content hashes, a copy of `file` elsewhere +/// counts as the same asset. +fn safe_redirect( + file: &str, + destination: &str, + config: IntentConfig, + exists: Exists<'_>, + skip: bool, +) -> Option { + if skip || exists(destination).is_some() { + return None; + } + let Some(Intent::Asset(asset)) = intent(destination, config, exists, true) else { + return None; + }; + exists(file) + .filter(|file| file.etag == asset.etag) + .map(|_| Intent::Redirect(destination.to_owned())) +} + +fn not_found(pathname: &str, config: IntentConfig, exists: Exists<'_>) -> Option { + let found = |path: &str, status| { + let asset = exists(path)?; + Some(Intent::Asset(Asset { + status, + not_found: true, + ..asset + })) + }; + match config.not_found_handling { + NotFoundHandling::SinglePageApplication => found("/index.html", 200), + NotFoundHandling::NotFoundPage => { + let mut dirs = pathname.rmatch_indices('/').map(|(i, _)| &pathname[..i]); + dirs.find_map(|dir| found(&format!("{dir}/404.html"), 404)) + } + NotFoundHandling::None => None, + } +} + +/// Mirrors `decodePath`: `decodeURIComponent` on each segment, keeping one it would throw on, +/// then runs of `/` collapsed. +pub fn decode_path(pathname: &str) -> String { + let decode = |segment: &str| { + let mut bytes = Vec::new(); + let mut rest = segment.as_bytes(); + while let Some((&b, tail)) = rest.split_first() { + if b == b'%' { + let hex = tail + .get(..2) + .filter(|h| h.iter().all(u8::is_ascii_hexdigit))?; + let hex = std::str::from_utf8(hex).ok()?; + bytes.push(u8::from_str_radix(hex, 16).ok()?); + rest = &tail[2..]; + } else { + bytes.push(b); + rest = tail; + } + } + String::from_utf8(bytes).ok() + }; + let segments: Vec<_> = pathname + .split('/') + .map(|s| decode(s).unwrap_or_else(|| s.to_owned())) + .collect(); + collapse_slashes(&segments.join("/")) +} + +/// Mirrors `encodePath`: `encodeURIComponent` on each segment. +pub fn encode_path(pathname: &str) -> String { + let unreserved = |b: u8| b.is_ascii_alphanumeric() || b"-_.!~*'()".contains(&b); + let encode = |segment: &str| -> String { + segment + .bytes() + .map(|b| { + if unreserved(b) { + char::from(b).to_string() + } else { + format!("%{b:02X}") + } + }) + .collect() + }; + pathname + .split('/') + .map(encode) + .collect::>() + .join("/") +} + +pub(crate) fn collapse_slashes(s: &str) -> String { + let mut out = String::with_capacity(s.len()); + for c in s.chars() { + if c != '/' || !out.ends_with('/') { + out.push(c); + } + } + out +} + +#[cfg(test)] +#[path = "intent-test.rs"] +mod tests; diff --git a/src/rust/assets-router/lib-test.rs b/src/rust/assets-router/lib-test.rs new file mode 100644 index 00000000000..f77fa4c3765 --- /dev/null +++ b/src/rust/assets-router/lib-test.rs @@ -0,0 +1,102 @@ +use super::*; + +/// The config of `json`, an `Options`. +pub fn from_json( + json: &str, + has_user_worker: bool, + navigation: bool, + warn: &mut dyn FnMut(String), +) -> Result { + let options = serde_json::from_str(json).map_err(|e| e.to_string())?; + let deployment = Deployment { + has_user_worker, + navigation_prefers_asset_serving: navigation, + }; + Config::new(options, deployment, warn) +} + +fn parse(json: &str) -> Result { + from_json(json, true, false, &mut |w| panic!("{w}")) +} + +#[test] +fn defaults() { + let config = parse("{}").unwrap(); + let routing = &config.routing; + assert!(routing.has_user_worker); + assert!(!routing.invoke_user_worker_ahead_of_assets); + assert_eq!(routing.intent, intent::IntentConfig::default()); + assert!(routing.base_path.is_root()); + assert!(!config.debug); + assert!(config.ignored.ignored("/_headers")); +} + +#[test] +fn every_field_null_is_the_default() { + let config = parse( + r#"{"html_handling": null, "not_found_handling": null, "base_path": null, "run_worker_first": null, + "headers": null, "redirects": null, "ignore": null, "debug": null}"#, + ); + assert_eq!( + config.unwrap().routing.intent, + intent::IntentConfig::default() + ); +} + +#[test] +fn fields() { + let config = parse( + r#"{"html_handling": "drop-trailing-slash", "not_found_handling": "404-page", + "base_path": "/sub", "run_worker_first": ["/api/*", "!/api/x"], + "headers": "/*\n X: y", "redirects": "/a /b", "ignore": "x", "debug": true}"#, + ) + .unwrap(); + let routing = &config.routing; + assert_eq!( + routing.intent.html_handling, + intent::HtmlHandling::DropTrailingSlash + ); + assert_eq!( + routing.intent.not_found_handling, + intent::NotFoundHandling::NotFoundPage + ); + assert!(routing.intent.has_static_routing); + assert_eq!(routing.base_path.as_str(), "/sub/"); + assert_eq!(routing.rule_route("/api/y"), decide::RuleRoute::UserWorker); + assert_eq!(routing.rule_route("/api/x"), decide::RuleRoute::AssetWorker); + assert_eq!(routing.rule_route("/b"), decide::RuleRoute::Undecided); + assert!(config.debug); + assert!(config.ignored.ignored("/x")); + + let all = parse(r#"{"run_worker_first": true}"#).unwrap(); + assert_eq!(all.routing.rule_route("/b"), decide::RuleRoute::UserWorker); + assert!(!all.routing.intent.has_static_routing); +} + +#[test] +fn errors() { + for (json, error) in [ + ( + r#"{"run_worker_first": ["api/*"]}"#, + "Invalid routes in `run_worker_first`:\n'api/*': rules must start with '/' or '!/'", + ), + ( + r#"{"base_path": "https://x"}"#, + "Invalid assets base_path: The value must be a pathname, not a URL.", + ), + ] { + assert_eq!(parse(json).err().unwrap(), error); + } + assert!(parse(r#"{"html_handling": "sometimes"}"#).is_err()); + assert!(parse("[").is_err()); +} + +#[test] +fn rule_file_warnings() { + let mut warnings = Vec::new(); + let json = r#"{"headers": "nope", "redirects": "nope"}"#; + from_json(json, false, false, &mut |w| warnings.push(w)).unwrap(); + assert_eq!(warnings.len(), 2); + assert!(warnings[0].starts_with("Found 1 invalid redirect rule(s):")); + assert!(warnings[1].starts_with("Found 1 invalid header rule(s):")); +} diff --git a/src/rust/assets-router/lib.rs b/src/rust/assets-router/lib.rs new file mode 100644 index 00000000000..605b64d377f --- /dev/null +++ b/src/rust/assets-router/lib.rs @@ -0,0 +1,182 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +#![forbid(unsafe_code)] + +//! Static assets routing, ported from workers-sdk's `packages/workers-shared` at commit +//! dc47e6d2 (2026-11-11); each module names its upstream file. +//! +//! Whether a request goes to the user Worker or to the assets, and what the assets answer. Pure: +//! the host looks files up and sends responses. + +use serde::Deserialize; + +pub mod base_path; +pub mod decide; +pub mod ignore; +pub mod intent; +pub mod mitigations; +pub mod parse; +pub mod rules; +pub mod serve; + +/// The parts of a request that routing looks at. +#[derive(Clone, Copy, Debug)] +pub struct Request<'a> { + pub url: &'a url::Url, + pub method: Method, + pub sec_fetch_mode: Option<&'a str>, + pub sec_fetch_dest: Option<&'a str>, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum Method { + Get, + Head, + Other, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct ResponseHeader { + pub name: String, + pub value: String, +} + +pub fn header(name: impl Into, value: impl Into) -> ResponseHeader { + let (name, value) = (name.into(), value.into()); + ResponseHeader { name, value } +} + +/// `prefix` and `part`, if `part` is there and not empty. +fn affix(prefix: &str, part: Option<&str>) -> String { + part.filter(|p| !p.is_empty()) + .map_or_else(String::new, |p| format!("{prefix}{p}")) +} + +/// A response sent without running a Worker or reading a file. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct Response { + pub status: u16, + pub headers: Vec, + pub body: &'static str, +} + +impl Response { + /// The response that replaces a request or response a mitigation blocks. + pub fn blocked() -> Self { + let headers = vec![header("Content-Type", "text/plain;charset=UTF-8")]; + Self { + status: 403, + headers, + body: "Blocked", + } + } + + pub fn new(status: u16, headers: Vec) -> Self { + Self { + status, + headers, + body: "", + } + } +} + +/// The assets config, every field optional: wrangler's `assets` options, and the text of the +/// directory's `_headers`, `_redirects` and `.assetsignore`. +#[derive(Default, Deserialize)] +#[serde(default)] +pub struct Options { + pub html_handling: Option, + pub not_found_handling: Option, + pub base_path: Option, + pub run_worker_first: Option, + pub headers: Option, + pub redirects: Option, + pub ignore: Option, + /// For Miniflare: a header on a navigation that `not_found_handling` answered. + pub debug: Option, +} + +#[derive(Deserialize)] +#[serde(untagged)] +pub enum RunWorkerFirst { + All(bool), + Rules(Vec), +} + +pub struct Config { + pub routing: decide::RoutingConfig, + pub debug: bool, + pub ignored: ignore::Matcher, +} + +/// What routing needs from the deployment besides its assets config. +#[derive(Clone, Copy, Debug, Default)] +pub struct Deployment { + pub has_user_worker: bool, + /// The compiled `assets_navigation_prefers_asset_serving` flag. + pub navigation_prefers_asset_serving: bool, +} + +impl Config { + /// `warn` gets the warnings about `_headers`, `_redirects` and `.assetsignore`. + pub fn new( + options: Options, + deployment: Deployment, + warn: &mut dyn FnMut(String), + ) -> Result { + let Deployment { + has_user_worker, + navigation_prefers_asset_serving, + } = deployment; + let (rules, ahead) = match options.run_worker_first { + Some(RunWorkerFirst::Rules(rules)) => (Some(parse::static_routing(&rules)?), false), + Some(RunWorkerFirst::All(all)) => (None, all), + None => (None, false), + }; + let html_handling = options.html_handling.unwrap_or_default(); + let base_path = match options.base_path { + Some(path) => base_path::BasePath::normalize(&path) + .map_err(|e| format!("Invalid assets base_path: {e}"))?, + None => base_path::BasePath::default(), + }; + let (static_redirects, dynamic_redirects) = options + .redirects + .map(|text| parse::redirects(&text, html_handling, warn)) + .unwrap_or_default(); + let header_rules = options + .headers + .map(|text| parse::headers(&text, warn)) + .unwrap_or_default(); + let has_static_routing = rules.is_some(); + let (user, asset) = rules.unwrap_or_default(); + Ok(Self { + routing: decide::RoutingConfig { + has_user_worker, + invoke_user_worker_ahead_of_assets: ahead, + user_worker_rules: decide::globs(&user), + asset_worker_rules: decide::globs(&asset), + intent: intent::IntentConfig { + html_handling, + not_found_handling: options.not_found_handling.unwrap_or_default(), + has_static_routing, + navigation_prefers_asset_serving, + }, + base_path, + redirects: rules::Redirects::new(static_redirects, dynamic_redirects), + header_rules: rules::RulesMatcher::new(header_rules), + }, + debug: options.debug.unwrap_or_default(), + ignored: ignore::Matcher::new(options.ignore.as_deref().unwrap_or_default(), warn), + }) + } +} + +#[cfg(test)] +#[path = "lib-test.rs"] +mod tests; + +#[cfg(test)] +#[path = "golden-test.rs"] +mod golden; diff --git a/src/rust/assets-router/mitigations-test.rs b/src/rust/assets-router/mitigations-test.rs new file mode 100644 index 00000000000..a0b56a09c0c --- /dev/null +++ b/src/rust/assets-router/mitigations-test.rs @@ -0,0 +1,367 @@ +use super::*; + +fn url_with_param(pathname: &str, name: &str, value: &str) -> Url { + let mut url = Url::parse(&format!("https://example.com{pathname}")).unwrap(); + url.query_pairs_mut().append_pair(name, value); + url +} + +fn request<'a>(url: &'a Url, sec_fetch_dest: Option<&'a str>) -> Request<'a> { + Request { + url, + method: Method::Get, + sec_fetch_mode: None, + sec_fetch_dest, + } +} + +/// Whether a response to a `_next/image` request is replaced by a 403. +fn next_image_blocked( + pathname: &str, + sec_fetch_dest: Option<&str>, + source: &str, + status: u16, + content_type: &str, +) -> bool { + let url = url_with_param(pathname, "url", source); + filters_next_image_response(&request(&url, sec_fetch_dest)) + && should_block_image_response(status, content_type.as_bytes()) +} + +fn assert_next_image_blocked(pathname: &str) { + for content_type in [ + "", + "text/html", + "image/jpg,text/plain", + "image/jpg,text/html", + "text/plain,text/html", + ] { + assert!( + next_image_blocked( + pathname, + None, + "https://example.com/image.jpg", + 200, + content_type + ), + "{content_type}" + ); + } +} + +fn assert_next_image_allowed( + pathname: &str, + sec_fetch_dest: Option<&str>, + source: &str, + status: u16, + content_type: Option<&str>, +) { + assert!(!next_image_blocked( + pathname, + sec_fetch_dest, + source, + status, + content_type.unwrap_or_default() + )); +} + +#[test] +fn next_image_blocks_html_content_type_at_subpath() { + assert_next_image_blocked("/some/subpath/_next/image"); +} + +#[test] +fn next_image_allows_image_fetch_dest_at_subpath() { + assert_next_image_allowed( + "/some/subpath/_next/image", + Some("image"), + "https://example.com/image.jpg", + 200, + Some("image/jpeg"), + ); +} + +#[test] +fn next_image_allows_image_content_type_at_subpath() { + assert_next_image_allowed( + "/some/subpath/_next/image", + None, + "https://example.com/image.jpg", + 200, + Some("image/jpeg"), + ); +} + +#[test] +fn next_image_allows_text_plain_at_subpath() { + assert_next_image_allowed( + "/some/subpath/_next/image", + None, + "https://example.com/image.jpg", + 200, + Some("text/plain"), + ); +} + +#[test] +fn next_image_allows_text_plain_with_charset_at_subpath() { + assert_next_image_allowed( + "/some/subpath/_next/image", + None, + "https://example.com/image.jpg", + 200, + Some("text/plain;charset=UTF-8"), + ); +} + +#[test] +fn next_image_image_header_trumps_html_body_at_subpath() { + assert_next_image_allowed( + "/some/subpath/_next/image", + Some("image"), + "https://example.com/image.jpg", + 200, + Some("text/html"), + ); +} + +#[test] +fn next_image_allows_local_urls_at_subpath() { + assert_next_image_allowed( + "/some/subpath/_next/image", + None, + "/local/image.jpg", + 200, + Some("text/html"), + ); +} + +#[test] +fn next_image_allows_304_status_at_subpath() { + assert_next_image_allowed( + "/some/subpath/_next/image", + None, + "https://example.com/image.jpg", + 304, + Some("text/html"), + ); +} + +#[test] +fn next_image_blocks_html_content_type_at_root() { + assert_next_image_blocked("/_next/image"); +} + +#[test] +fn next_image_allows_image_fetch_dest_at_root() { + assert_next_image_allowed( + "/_next/image", + Some("image"), + "https://example.com/image.jpg", + 200, + Some("image/jpeg"), + ); +} + +#[test] +fn next_image_allows_image_content_type_at_root() { + assert_next_image_allowed( + "/_next/image", + None, + "https://example.com/image.jpg", + 200, + Some("image/jpeg"), + ); +} + +#[test] +fn next_image_allows_text_plain_at_root() { + assert_next_image_allowed( + "/_next/image", + None, + "https://example.com/image.jpg", + 200, + Some("text/plain"), + ); +} + +#[test] +fn next_image_allows_text_plain_with_charset_at_root() { + assert_next_image_allowed( + "/_next/image", + None, + "https://example.com/image.jpg", + 200, + Some("text/plain;charset=UTF-8"), + ); +} + +#[test] +fn next_image_image_header_trumps_html_body_at_root() { + assert_next_image_allowed( + "/_next/image", + Some("image"), + "https://example.com/image.jpg", + 200, + Some("text/html"), + ); +} + +#[test] +fn next_image_allows_local_urls_at_root() { + assert_next_image_allowed( + "/_next/image", + None, + "/local/image.jpg", + 200, + Some("text/html"), + ); +} + +#[test] +fn next_image_allows_304_status_at_root() { + assert_next_image_allowed( + "/_next/image", + None, + "https://example.com/image.jpg", + 304, + Some("text/html"), + ); +} + +// Upstream only checks the response when `url` is non-empty. +#[test] +fn next_image_ignores_an_empty_url() { + assert_next_image_allowed("/_next/image", None, "", 200, Some("text/html")); +} + +#[test] +fn next_image_checks_any_method_but_get() { + let url = url_with_param("/_next/image", "url", "https://example.com/image.jpg"); + let mut request = request(&url, Some("image")); + assert!(!filters_next_image_response(&request)); + request.method = Method::Head; + assert!(filters_next_image_response(&request)); +} + +#[test] +fn non_utf8_content_type_is_blocked() { + assert!(should_block_image_response(200, b"image/\xff")); +} + +fn image_href_blocked(href: &str, sec_fetch_dest: Option<&str>) -> bool { + let url = url_with_param("/_image", "href", href); + blocks_image_href(&request(&url, sec_fetch_dest)) +} + +#[test] +fn image_blocks_protocol_relative_different_hostname() { + assert!(image_href_blocked("//evil.com/ssrf", None)); +} + +#[test] +fn image_blocks_whatwg_backslash_hostname_bypass() { + assert!(image_href_blocked(r"//evil.com\@example.com/image", None)); +} + +#[test] +fn image_allows_same_hostname_protocol_relative() { + assert!(!image_href_blocked("//example.com/image.jpg", None)); + assert!(!image_href_blocked("//EXAMPLE.com/image.jpg", None)); +} + +#[test] +fn image_allows_different_hostname_with_image_fetch_dest() { + assert!(!image_href_blocked( + "//another.com/image.jpg", + Some("image") + )); +} + +#[test] +fn image_allows_an_href_that_is_not_protocol_relative_or_does_not_parse() { + assert!(!image_href_blocked("/local/image.jpg", None)); + assert!(!image_href_blocked("//", None)); + assert!(!image_href_blocked("//[", None)); +} + +fn assert_redirect(raw_url: &str, normalized_url: &str) { + let url = Url::parse(raw_url).unwrap(); + assert!(is_cdn_cgi_backslash_bypass(raw_url, &url)); + assert_eq!(url.as_str(), normalized_url); +} + +#[test] +fn cdn_cgi_backslash_redirects_image() { + assert_redirect( + r"https://example.com/cdn-cgi\image/q=75/https://evil.com/ssrf", + "https://example.com/cdn-cgi/image/q=75/https://evil.com/ssrf", + ); +} + +#[test] +fn cdn_cgi_backslash_redirects_next_cache() { + assert_redirect( + r"https://example.com/cdn-cgi\_next_cache/some-data", + "https://example.com/cdn-cgi/_next_cache/some-data", + ); +} + +#[test] +fn cdn_cgi_backslash_redirects_arbitrary_subpath() { + assert_redirect( + r"https://example.com/cdn-cgi\something-else/path", + "https://example.com/cdn-cgi/something-else/path", + ); +} + +#[test] +fn cdn_cgi_backslash_redirects_with_query() { + assert_redirect( + r"https://example.com/cdn-cgi\something-else/path?value=/cdn-cgi/param=foo", + "https://example.com/cdn-cgi/something-else/path?value=/cdn-cgi/param=foo", + ); +} + +fn assert_no_redirect(raw_url: &str, pathname: &str) { + let url = Url::parse(raw_url).unwrap(); + assert_eq!(url.path(), pathname); + assert!(!is_cdn_cgi_backslash_bypass(raw_url, &url)); +} + +#[test] +fn cdn_cgi_no_interference_forward_slash() { + assert_no_redirect( + "https://example.com/cdn-cgi/image/q=75/https://other.com/image.jpg", + "/cdn-cgi/image/q=75/https://other.com/image.jpg", + ); +} + +#[test] +fn cdn_cgi_no_interference_escaped_backslash() { + assert_no_redirect( + "https://example.com/cdn-cgi%5Cimage/q=75/https://other.com/image.jpg", + "/cdn-cgi%5Cimage/q=75/https://other.com/image.jpg", + ); +} + +#[test] +fn cdn_cgi_no_interference_escaped_forward_slash() { + assert_no_redirect( + "https://example.com/cdn-cgi%2Fimage/q=75/https://other.com/image.jpg", + "/cdn-cgi%2Fimage/q=75/https://other.com/image.jpg", + ); +} + +#[test] +fn cdn_cgi_no_interference_non_cdn_cgi() { + assert_no_redirect("https://example.com/some-page", "/some-page"); +} + +#[test] +fn cdn_cgi_no_interference_cdn_cgi_in_query() { + assert_no_redirect( + r"https://example.com/some-page?redirect=/cdn-cgi\image", + "/some-page", + ); +} diff --git a/src/rust/assets-router/mitigations.rs b/src/rust/assets-router/mitigations.rs new file mode 100644 index 00000000000..43cc78d7939 --- /dev/null +++ b/src/rust/assets-router/mitigations.rs @@ -0,0 +1,67 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! Security mitigations around assets routing. +//! +//! Ported from workers-sdk's workers-shared, `router-worker/src/worker.ts`. + +use std::borrow::Cow; + +use url::Url; + +use super::Method; +use super::Request; + +/// Whether URL parsing alone made this a `/cdn-cgi/` request, from a raw `/cdn-cgi\`. +/// +/// Such a request slipped past the routing of `/cdn-cgi/` ahead of Workers, so it is redirected +/// to its parsed URL before any routing. +pub fn is_cdn_cgi_backslash_bypass(raw_url: &str, url: &Url) -> bool { + url.path().starts_with("/cdn-cgi/") && raw_url.contains("/cdn-cgi\\") +} + +/// Whether to refuse `/_image?href=//other-host/...`, unless the browser asks for an image. +pub(crate) fn blocks_image_href(request: &Request<'_>) -> bool { + let other_host = |href: Cow<'_, str>| { + href.len() > 2 + && href.starts_with("//") + && Url::parse(&format!("https:{href}")) + .is_ok_and(|href| href.host_str() != request.url.host_str()) + }; + request.url.path() == "/_image" + && query_param(request.url, "href").is_some_and(other_host) + && request.sec_fetch_dest != Some("image") +} + +/// Whether the user Worker's response must pass [`should_block_image_response`]: a request for +/// `*/_next/image?url=` that is not the browser's for an image. +pub(crate) fn filters_next_image_response(request: &Request<'_>) -> bool { + request.url.path().ends_with("/_next/image") + && query_param(request.url, "url").is_some_and(|s| !s.is_empty() && !s.starts_with('/')) + && (request.method != Method::Get || request.sec_fetch_dest != Some("image")) +} + +/// Whether a filtered response is replaced by a 403: anything but a 304 must have one +/// `Content-Type`, an image type or plain text. +pub fn should_block_image_response(status: u16, content_type: &[u8]) -> bool { + let Ok(content_type) = std::str::from_utf8(content_type) else { + return status != 304; + }; + status != 304 + && (content_type.is_empty() + || content_type.contains(',') + || !(content_type.starts_with("image/") + || content_type.split(';').next() == Some("text/plain"))) +} + +/// What `URLSearchParams.get(name)` returns. +fn query_param<'a>(url: &'a Url, name: &str) -> Option> { + url.query_pairs() + .find(|(key, _)| key == name) + .map(|(_, value)| value) +} + +#[cfg(test)] +#[path = "mitigations-test.rs"] +mod tests; diff --git a/src/rust/assets-router/parse-test.rs b/src/rust/assets-router/parse-test.rs new file mode 100644 index 00000000000..f2834b0ea2e --- /dev/null +++ b/src/rust/assets-router/parse-test.rs @@ -0,0 +1,510 @@ +// Upstream: workers-shared `utils/tests/parseHeaders.*.test.ts`, `parseRedirects.*.test.ts` and +// `parseStaticRouting.test.ts`. + +use super::*; + +type Rule = (String, Vec<(String, String)>, Vec); + +fn headers(input: &str) -> (Vec, Vec) { + let (rules, invalid) = parse_headers(input, MAX_HEADER_RULES); + ( + rules + .into_iter() + .map(|(path, r)| (path, r.set, r.unset)) + .collect(), + invalid, + ) +} + +fn rule(path: &str, headers: &[(&str, &str)], unset: &[&str]) -> Rule { + let headers = headers + .iter() + .map(|(n, v)| ((*n).into(), (*v).into())) + .collect(); + ( + path.into(), + headers, + unset.iter().map(|&u| u.into()).collect(), + ) +} + +fn redirects(input: &str) -> (Vec<(String, usize, Redirect)>, Vec) { + let (max_static, max_dynamic) = (MAX_STATIC_REDIRECT_RULES, MAX_DYNAMIC_REDIRECT_RULES); + parse_redirects( + input, + HtmlHandling::AutoTrailingSlash, + max_static, + max_dynamic, + ) +} + +fn redirect(from: &str, to: &str, status: u16, line_number: usize) -> (String, usize, Redirect) { + ( + from.into(), + line_number, + Redirect { + status, + to: to.into(), + }, + ) +} + +fn invalid(line: &str, line_number: usize, message: &str) -> Invalid { + Invalid::at(line, line_number, message) +} + +#[test] +fn headers_rules() { + let input = " + # This is a comment + /a + Name : Value + /b + Set-Cookie: test=cookie; expires=never + Set-Cookie: another=cookie; magic! + !Name: Value + ! Place +"; + assert_eq!( + headers(input), + ( + vec![ + rule("/a", &[("name", "Value")], &[]), + rule( + "/b", + &[ + ( + "set-cookie", + "test=cookie; expires=never, another=cookie; magic!" + ), + ("!name", "Value") + ], + &["Place"] + ), + ], + vec![] + ) + ); +} + +#[test] +fn headers_hosts() { + let input = " + https://example.com + a: a + https://example.com/blog + c: c + https://:subdomain.example.*/path + e:e +"; + assert_eq!( + headers(input).0, + [ + rule("https://example.com/", &[("a", "a")], &[]), + rule("https://example.com/blog", &[("c", "c")], &[]), + rule("https://:subdomain.example.*/path", &[("e", "e")], &[]), + ] + ); +} + +#[test] +fn headers_reject_names_with_non_token_characters() { + let (rules, bad) = headers("/a\n X@Foo: bar\n Foo/Bar: baz\n Ok: yes"); + assert_eq!(rules, [rule("/a", &[("ok", "yes")], &[])]); + let message = "Header name can only include token characters"; + assert_eq!( + bad, + [ + invalid("X@Foo: bar", 2, message), + invalid("Foo/Bar: baz", 3, message) + ] + ); +} + +#[test] +fn headers_invalid_lines() { + let input = " + c + Access-Control-Allow-Origin: * + /b + I'm invalid! + !x-content-type + But: I'm okay! + ! Content-Type: application/json + /c + /d + : + test: + "; + assert_eq!( + headers(input), + ( + vec![rule("/b", &[("but", "I'm okay!")], &[])], + vec![ + invalid( + "c", + 2, + "Expected a path beginning with at least one forward-slash" + ), + invalid( + "Access-Control-Allow-Origin: *", + 3, + "Path should come before header (access-control-allow-origin: *)" + ), + invalid( + "I'm invalid!", + 5, + "Expected a colon-separated header pair (e.g. name: value)" + ), + invalid( + "!x-content-type", + 6, + "Expected a colon-separated header pair (e.g. name: value)" + ), + invalid( + "! Content-Type: application/json", + 8, + "Header name cannot include spaces" + ), + invalid("/c", 10, "No headers specified"), + invalid(":", 11, "No header name specified"), + invalid("test:", 12, "No header value specified"), + Invalid { + line: Some("/d".into()), + line_number: None, + message: "No headers specified".into(), + }, + ] + ) + ); +} + +#[test] +fn headers_invalid_paths() { + let input = " + /some page with spaces + valid: yup + /://so;`me + valid: yup + /nons:/&@%+~{}ense + valid: yup + http://nah.com/blog + invalid: things + https://nah.com:8080 + invalid: also + //yeah.com/blog + valid: things + /blog/*/posts/* + x: y + /blog/*/:splat + x: y +"; + let (rules, invalid_) = headers(input); + let paths: Vec<_> = rules.iter().map(|r| r.0.as_str()).collect(); + assert_eq!( + paths, + [ + "/some%20page%20with%20spaces", + "/://so;%60me", + "/nons:/&@%+~%7B%7Dense", + "//yeah.com/blog" + ] + ); + assert_eq!( + invalid_, + [ + invalid( + "http://nah.com/blog", + 8, + "URLs should either be relative (e.g. begin with a forward-slash), or use HTTPS \ + (e.g. begin with \"https://\")." + ), + invalid( + "https://nah.com:8080", + 10, + "Specifying ports is not supported. Skipping absolute URL https://nah.com:8080." + ), + invalid( + "/blog/*/posts/*", + 14, + "Only one wildcard is allowed per rule. Use a named placeholder (e.g. :project) \ + instead. Skipping /blog/*/posts/*." + ), + invalid( + "/blog/*/:splat", + 16, + "Cannot combine a wildcard * with a :splat placeholder because wildcards are \ + converted to :splat at runtime. Skipping /blog/*/:splat." + ), + ] + ); +} + +#[test] +fn headers_limits() { + let huge = format!("/b\n{}: b", "a".repeat(2001)); + let (rules, invalid_) = headers(&format!("/a\n Name: Value\n{}\n{huge}", "#".repeat(2001))); + assert_eq!(rules, [rule("/a", &[("name", "Value")], &[])]); + assert_eq!( + invalid_[0], + Invalid::new("Ignoring line 5 as it exceeds the maximum allowed length of 2000.".into()) + ); + + let many: Vec<_> = (0..150).map(|i| format!("/a/{i}\nx-index: {i}")).collect(); + let (rules, invalid_) = headers(&format!("\n{}\n# end\n", many.join("\n"))); + assert_eq!(rules.len(), 101); + assert_eq!( + invalid_, + [Invalid::new( + "Maximum number of rules supported is 100. Skipping remaining 100 lines of file." + .into() + )] + ); +} + +#[test] +fn redirects_rules() { + let input = " + # This is a comment + /a /b 301 + /c /d + /e /f?query=string 307 # comment + /g?this=rejected /h 308 + /ext https://some.domain:1234/route?q=string#anchor + /frag /b##blah-1 # + /proxy /b 200 + /cat https://cat.com:12345 302 + /* /foo 200 +"; + assert_eq!( + redirects(input), + ( + vec![ + redirect("/a", "/b", 301, 3), + redirect("/c", "/d", 302, 4), + redirect("/e", "/f?query=string", 307, 5), + redirect("/g", "/h", 308, 6), + redirect( + "/ext", + "https://some.domain:1234/route?q=string#anchor", + 302, + 7 + ), + redirect("/frag", "/b##blah-1", 302, 8), + redirect("/proxy", "/b", 200, 9), + redirect("/cat", "https://cat.com:12345/", 302, 10), + redirect("/*", "/foo", 200, 11), + ], + vec![] + ) + ); +} + +#[test] +fn redirects_invalid_lines() { + let input = " + /c + /d /e 302 !important + /f /g 418 + /a /b + /a /c + https://yeah.com https://nah.com + /x https://example.com/b 200 + /* /index.html 200 + / /index + /some%20page /somewhere%20else + /://so;`me /nons:/&@%+~{}ense + yeah.com https://nah.com + "; + let (rules, invalid_) = redirects(input); + assert_eq!( + rules, + [ + redirect("/a", "/b", 302, 5), + redirect("/some%20page", "/somewhere%20else", 302, 11), + redirect("/://so;%60me", "/nons:/&@%+~%7B%7Dense", 302, 12), + redirect("/yeah.com", "https://nah.com/", 302, 13), + ] + ); + let looping = "Infinite loop detected in this rule and has been ignored. This will cause a \ + redirect to strip `.html` or `/index` and end up triggering this rule again. \ + Please fix or remove this rule to silence this warning."; + assert_eq!( + invalid_, + [ + invalid( + "/c", + 2, + "Expected exactly 2 or 3 whitespace-separated tokens. Got 1." + ), + invalid( + "/d /e 302 !important", + 3, + "Expected exactly 2 or 3 whitespace-separated tokens. Got 4." + ), + invalid( + "/f /g 418", + 4, + "Valid status codes are 200, 301, 302 (default), 303, 307, or 308. Got 418." + ), + invalid("/a /c", 6, "Ignoring duplicate rule for path /a."), + invalid( + "https://yeah.com https://nah.com", + 7, + "Only relative URLs are allowed. Skipping absolute URL https://yeah.com." + ), + invalid( + "/x https://example.com/b 200", + 8, + "Proxy (200) redirects can only point to relative paths. Got \ + https://example.com/b" + ), + invalid("/* /index.html 200", 9, looping), + invalid("/ /index", 10, looping), + ] + ); + // Without HTML handling `/` to `/index` does not loop. + let (rules, _) = parse_redirects("/ /index", HtmlHandling::None, 1, 1); + assert_eq!(rules, [redirect("/", "/index", 302, 1)]); +} + +#[test] +fn redirects_limits() { + let dynamic: Vec<_> = (0..150) + .map(|i| format!("/a/{i}/* /b/{i}/:splat")) + .collect(); + let (rules, invalid_) = redirects(&format!("\n{}\n", dynamic.join("\n"))); + assert_eq!(rules.len(), 100); + assert_eq!( + invalid_, + [Invalid::new( + "Maximum number of dynamic rules supported is 100. Skipping remaining 51 lines of \ + file." + .into() + )] + ); + let lines: Vec<_> = (0..3).map(|i| format!("/a/{i} /b/{i}")).collect(); + let (rules, invalid_) = parse_redirects(&lines.join("\n"), HtmlHandling::None, 2, 1); + assert_eq!(rules.len(), 2); + assert_eq!( + invalid_, + [Invalid::new( + "Maximum number of static rules supported is 2. Skipping line.".into() + )] + ); +} + +#[test] +fn constructed_rules_and_warnings() { + let mut warnings = Vec::new(); + let text = "/a /b 301\n/c/* /d/:splat\n/e /f\nnope\n"; + let (static_rules, dynamic_rules) = + super::redirects(text, HtmlHandling::None, &mut |w| warnings.push(w)); + let to = |r: &Redirect| (r.status, r.to.clone()); + assert_eq!(static_rules.len(), 1); + assert_eq!(static_rules[0].0, "/a"); + // After the first dynamic rule, every rule is dynamic. + assert_eq!( + dynamic_rules + .iter() + .map(|(f, r)| (f.as_str(), to(r))) + .collect::>(), + [ + ("/c/*", (302, "/d/:splat".into())), + ("/e", (302, "/f".into())) + ] + ); + assert_eq!( + warnings, + [ + "Found 1 invalid redirect rule(s):\n Expected exactly 2 or 3 whitespace-separated \ + tokens. Got 1.\n at _redirects:4 | nope" + ] + ); + + let mut warnings = Vec::new(); + let rules = super::headers("/a\n x: 1\n/a\n y: 2\n! z\n", &mut |w| warnings.push(w)); + assert_eq!(rules.len(), 1); + assert_eq!(rules[0].1.set, [("y".to_owned(), "2".to_owned())]); + assert_eq!(rules[0].1.unset, ["z"]); + assert!(warnings.is_empty()); +} + +#[test] +fn static_routing_rules() { + let rules = + |rules: &[&str]| static_routing(&rules.iter().map(|&r| r.into()).collect::>()); + assert_eq!( + rules(&["/api/*", "/oauth/callback", "!/api/assets/*"]), + Ok(( + vec!["/api/*".into(), "/oauth/callback".into()], + vec!["/api/assets/*".into()] + )) + ); + assert_eq!( + rules(&[]), + Err("No `run_worker_first` rules were provided; must provide at least 1 rule.".into()) + ); + assert_eq!( + rules(&["!/assets"]), + Err( + "Only negative `run_worker_first` rules were provided; must provide at least 1 \ + non-negative rule" + .into() + ) + ); + assert_eq!( + rules(&["api/*", "!asset"]), + Err( + "Invalid routes in `run_worker_first`:\n'api/*': rules must start with '/' or \ + '!/'\n'!asset': negative rules must start with '!/'" + .into() + ) + ); + assert_eq!( + rules(&[ + "/api/*", + "/oauth/callback", + "/api/some/route", + "/oauth/callback" + ]), + Err( + "Invalid routes in `run_worker_first`:\n'/api/some/route': rule '/api/*' makes it \ + redundant\n'/oauth/callback': rule is a duplicate; rules must be unique" + .into() + ) + ); + let long = format!("/api/{}", "a".repeat(130)); + assert_eq!( + rules(&[&long]), + Err(format!( + "Invalid routes in `run_worker_first`:\n'{long}': all rules must be less than 100 \ + characters in length" + )) + ); + let many: Vec = (0..99) + .map(|i| format!("/rule/{i}")) + .chain((0..5).map(|_| "/rule/0".into())) + .collect(); + assert_eq!( + static_routing(&many), + Err( + "Too many `run_worker_first` rules were provided; 104 rules provided (99 \ + distinct) exceeds max of 100." + .into() + ) + ); +} + +#[test] +fn skips_rules_that_cannot_compile() { + let mut warnings = Vec::new(); + let rules = super::headers("/:a/:a\n X: 1\n/ok\n X: 2\n", &mut |w| warnings.push(w)); + assert_eq!(rules.len(), 1); + assert_eq!(rules[0].0, "/ok"); + let (_, dynamic) = super::redirects("/:a/:a /b\n/c/* /d\n", HtmlHandling::None, &mut |w| { + warnings.push(w); + }); + assert_eq!(dynamic.len(), 1); + assert_eq!(warnings.len(), 2); +} diff --git a/src/rust/assets-router/parse.rs b/src/rust/assets-router/parse.rs new file mode 100644 index 00000000000..248cd797738 --- /dev/null +++ b/src/rust/assets-router/parse.rs @@ -0,0 +1,537 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! Parsing `_headers`, `_redirects` and `run_worker_first`. +//! +//! Ported from workers-sdk's workers-shared, `utils/configuration/`, with its messages. + +use std::collections::HashSet; + +use url::Url; + +use super::affix; +use super::intent::HtmlHandling; +use super::rules::HeaderRule; +use super::rules::Redirect; +use super::rules::compile; + +const MAX_LINE_LENGTH: usize = 2000; +const MAX_HEADER_RULES: usize = 100; +const MAX_DYNAMIC_REDIRECT_RULES: usize = 100; +const MAX_STATIC_REDIRECT_RULES: usize = 2000; +const MAX_ROUTES_RULES: usize = 100; +const MAX_ROUTES_RULE_LENGTH: usize = 100; + +/// A line that was skipped, and why. +#[derive(Debug, Eq, PartialEq)] +pub struct Invalid { + pub line: Option, + pub line_number: Option, + pub message: String, +} + +impl Invalid { + fn new(message: String) -> Self { + Self { + line: None, + line_number: None, + message, + } + } + + fn at(line: &str, line_number: usize, message: impl Into) -> Self { + Self { + line: Some(line.to_owned()), + line_number: Some(line_number), + message: message.into(), + } + } +} + +/// `path` as a URL normalizes it, with its query and fragment for a `target`. +fn extract_pathname(path: &str, target: bool) -> Option { + let slash = if path.starts_with('/') { "" } else { "/" }; + let url = Url::parse(&format!("relative://{slash}{path}")).ok()?; + let (query, fragment) = ( + url.query().filter(|_| target), + url.fragment().filter(|_| target), + ); + Some(format!( + "{}{}{}", + url.path(), + affix("?", query), + affix("#", fragment) + )) +} + +/// The host and path of an `https://` URL. +fn url_host(token: &str) -> Option<(&str, &str)> { + let rest = token.strip_prefix("https://")?.trim_start_matches('/'); + let (host, path) = rest.split_once('/').unwrap_or((rest, "")); + (!host.is_empty()).then_some((host, path)) +} + +/// Mirrors `validateUrl`. A `target`, a redirect's, may have a port, a query and a fragment. +pub fn validate_url(token: &str, only_relative: bool, target: bool) -> Result { + if let Some((host, path)) = url_host(token) { + if only_relative { + return Err(format!( + "Only relative URLs are allowed. Skipping absolute URL {token}." + )); + } + let port = host.rsplit_once(':').map(|(_, port)| port); + if !target && port.is_some_and(|p| !p.is_empty() && p.bytes().all(|b| b.is_ascii_digit())) { + return Err(format!( + "Specifying ports is not supported. Skipping absolute URL {token}." + )); + } + return extract_pathname(path, target) + .map(|path| format!("https://{host}{path}")) + .ok_or_else(|| format!("Error parsing URL segment {token}. Skipping.")); + } + let token = if token.starts_with('/') || !only_relative { + token.to_owned() + } else { + format!("/{token}") + }; + if token.starts_with('/') { + return extract_pathname(&token, target) + .ok_or_else(|| format!("Error parsing URL segment {token}. Skipping.")); + } + Err(if only_relative { + "URLs should begin with a forward-slash.".into() + } else { + "URLs should either be relative (e.g. begin with a forward-slash), or use HTTPS (e.g. \ + begin with \"https://\")." + .into() + }) +} + +/// Whether `path` has a `:placeholder`. +fn has_placeholder(path: &str) -> bool { + path.match_indices(':') + .any(|(i, _)| path[i + 1..].starts_with(|c: char| c.is_ascii_alphabetic())) +} + +/// Mirrors `parseHeaders`. +pub fn parse_headers(input: &str, max_rules: usize) -> (Vec<(String, HeaderRule)>, Vec) { + let lines: Vec<&str> = input.split('\n').collect(); + let (mut rules, mut invalid) = (Vec::new(), Vec::new()); + let mut rule: Option<(String, HeaderRule, &str)> = None; + let mut skip_until_next_path = false; + for (i, line) in lines.iter().enumerate() { + let (line, n) = (line.trim(), i + 1); + if line.is_empty() || line.starts_with('#') { + continue; + } + if line.len() > MAX_LINE_LENGTH { + invalid.push(Invalid::new(format!( + "Ignoring line {n} as it exceeds the maximum allowed length of {MAX_LINE_LENGTH}." + ))); + continue; + } + let first = line + .split_once(char::is_whitespace) + .map_or(line, |(first, _)| first); + if line.starts_with('/') || first.find("://").is_some_and(|i| i > 0) { + skip_until_next_path = false; + if rules.len() >= max_rules { + invalid.push(Invalid::new(format!( + "Maximum number of rules supported is {max_rules}. Skipping remaining {} \ + lines of file.", + lines.len() - i + ))); + break; + } + if let Some(rule) = rule.take() { + push(&mut rules, &mut invalid, rule, Some(n)); + } + match validate_url(line, false, false) + .and_then(|path| no_multiple_wildcards(&path).map(|()| path)) + { + Ok(path) => rule = Some((path, HeaderRule::default(), line)), + Err(message) => { + invalid.push(Invalid::at(line, n, message)); + skip_until_next_path = true; + } + } + continue; + } + if skip_until_next_path { + continue; + } + let Some((name, value)) = line.split_once(':') else { + match &mut rule { + None => invalid.push(Invalid::at( + line, + n, + "Expected a path beginning with at least one forward-slash", + )), + Some((_, rule, _)) if line.starts_with("! ") => rule.unset.push(line[2..].into()), + Some(_) => invalid.push(Invalid::at( + line, + n, + "Expected a colon-separated header pair (e.g. name: value)", + )), + } + continue; + }; + let (name, value) = (name.trim().to_lowercase(), value.trim()); + let message = if name.contains(' ') { + "Header name cannot include spaces".into() + } else if name.is_empty() { + "No header name specified".into() + } else if !name.bytes().all(is_token_char) { + // Upstream accepts these, and setting the header then fails every response. + "Header name can only include token characters".into() + } else if value.is_empty() { + "No header value specified".into() + } else if let Some((_, rule, _)) = &mut rule { + match rule.set.iter_mut().find(|(n, _)| *n == name) { + Some((_, existing)) => *existing = format!("{existing}, {value}"), + None => rule.set.push((name, value.into())), + } + continue; + } else { + format!("Path should come before header ({name}: {value})") + }; + invalid.push(Invalid::at(line, n, message)); + } + if let Some(rule) = rule { + push(&mut rules, &mut invalid, rule, None); + } + (rules, invalid) +} + +fn push( + rules: &mut Vec<(String, HeaderRule)>, + invalid: &mut Vec, + (path, rule, line): (String, HeaderRule, &str), + line_number: Option, +) { + if rule.set.is_empty() && rule.unset.is_empty() { + let line = Some(line.to_owned()); + invalid.push(Invalid { + line, + line_number, + message: "No headers specified".into(), + }); + } else { + rules.push((path, rule)); + } +} + +fn no_multiple_wildcards(path: &str) -> Result<(), String> { + let wildcards = path.matches('*').count(); + if wildcards > 1 { + return Err(format!( + "Only one wildcard is allowed per rule. Use a named placeholder (e.g. :project) \ + instead. Skipping {path}." + )); + } + let splat = path + .match_indices(":splat") + .any(|(i, _)| !path[i + 6..].starts_with(|c: char| c.is_ascii_alphanumeric() || c == '_')); + if wildcards > 0 && splat { + return Err(format!( + "Cannot combine a wildcard * with a :splat placeholder because wildcards are \ + converted to :splat at runtime. Skipping {path}." + )); + } + Ok(()) +} + +/// Mirrors `parseRedirects`. +pub fn parse_redirects( + input: &str, + html_handling: HtmlHandling, + max_static_rules: usize, + max_dynamic_rules: usize, +) -> (Vec<(String, usize, Redirect)>, Vec) { + let lines: Vec<&str> = input.split('\n').collect(); + let (mut rules, mut seen, mut invalid) = (Vec::new(), HashSet::new(), Vec::new()); + let (mut static_rules, mut dynamic_rules) = (0, 0); + let mut can_create_static_rule = true; + for (i, line) in lines.iter().enumerate() { + let (line, n) = (line.trim(), i + 1); + let fail = |message| Invalid::at(line, n, message); + if line.is_empty() || line.starts_with('#') { + continue; + } + if line.len() > MAX_LINE_LENGTH { + invalid.push(Invalid::new(format!( + "Ignoring line {n} as it exceeds the maximum allowed length of {MAX_LINE_LENGTH}." + ))); + continue; + } + // An inline comment starts with whitespace and `#`. + let comment = line + .match_indices('#') + .find(|(i, _)| line[..*i].ends_with(char::is_whitespace)); + let tokens: Vec<&str> = line[..comment.map_or(line.len(), |(i, _)| i)] + .split_whitespace() + .collect(); + if !(2..=3).contains(&tokens.len()) { + invalid.push(fail(format!( + "Expected exactly 2 or 3 whitespace-separated tokens. Got {}.", + tokens.len() + ))); + continue; + } + let status = tokens.get(2).copied().unwrap_or("302"); + let from = match validate_url(tokens[0], true, false) { + Ok(from) => from, + Err(message) => { + invalid.push(fail(message)); + continue; + } + }; + if can_create_static_rule && !from.contains('*') && !has_placeholder(&from) { + static_rules += 1; + if static_rules > max_static_rules { + invalid.push(Invalid::new(format!( + "Maximum number of static rules supported is {max_static_rules}. Skipping \ + line." + ))); + continue; + } + } else { + dynamic_rules += 1; + can_create_static_rule = false; + if dynamic_rules > max_dynamic_rules { + invalid.push(Invalid::new(format!( + "Maximum number of dynamic rules supported is {max_dynamic_rules}. Skipping \ + remaining {} lines of file.", + lines.len() - i + ))); + break; + } + } + let to = match validate_url(tokens[1], false, true) { + Ok(to) => to, + Err(message) => { + invalid.push(fail(message)); + continue; + } + }; + let Some(status) = status + .parse() + .ok() + .filter(|s| [200, 301, 302, 303, 307, 308].contains(s)) + else { + invalid.push(fail(format!( + "Valid status codes are 200, 301, 302 (default), 303, 307, or 308. Got {status}." + ))); + continue; + }; + let to_index = to.ends_with("/index") || to.ends_with("/index.html"); + let absolute = url_host(&to).is_some(); + if !absolute + && to_index + && (from.ends_with("/*") + || (from.ends_with('/') && html_handling != HtmlHandling::None)) + { + invalid.push(fail( + "Infinite loop detected in this rule and has been ignored. This will cause a \ + redirect to strip `.html` or `/index` and end up triggering this rule again. \ + Please fix or remove this rule to silence this warning." + .into(), + )); + continue; + } + if !seen.insert(from.clone()) { + invalid.push(fail(format!("Ignoring duplicate rule for path {from}."))); + continue; + } + if status == 200 && absolute { + invalid.push(fail(format!( + "Proxy (200) redirects can only point to relative paths. Got {to}" + ))); + continue; + } + rules.push((from, n, Redirect { status, to })); + } + (rules, invalid) +} + +/// Mirrors `parseStaticRouting`: the user Worker's rules and the assets'. +pub fn static_routing(input: &[String]) -> Result<(Vec, Vec), String> { + if input.is_empty() { + return Err( + "No `run_worker_first` rules were provided; must provide at least 1 rule.".into(), + ); + } + if input.len() > MAX_ROUTES_RULES { + let distinct = input.iter().collect::>().len(); + let distinct = if distinct < input.len() { + format!("({distinct} distinct) ") + } else { + String::new() + }; + return Err(format!( + "Too many `run_worker_first` rules were provided; {} rules provided {distinct}exceeds \ + max of {MAX_ROUTES_RULES}.", + input.len() + )); + } + let (mut raw_asset, mut asset, mut user, mut errors) = (vec![], vec![], vec![], vec![]); + for rule in input { + if let Some(path) = rule.strip_prefix('!').filter(|path| path.starts_with('/')) { + asset.push(path.to_owned()); + raw_asset.push(rule.clone()); + } else if rule.starts_with('/') { + user.push(rule.clone()); + } else if rule.starts_with('!') { + errors.push(format!("'{rule}': negative rules must start with '!/'")); + } else { + errors.push(format!("'{rule}': rules must start with '/' or '!/'")); + } + } + if !asset.is_empty() && user.is_empty() { + return Err( + "Only negative `run_worker_first` rules were provided; must provide at least \ + 1 non-negative rule" + .into(), + ); + } + errors.extend(validate_static_routing_rules(&user)); + errors.extend(validate_static_routing_rules(&raw_asset)); + if !errors.is_empty() { + return Err(format!( + "Invalid routes in `run_worker_first`:\n{}", + errors.join("\n") + )); + } + Ok((user, asset)) +} + +fn validate_static_routing_rules(rules: &[String]) -> Vec { + let (mut invalid, mut seen) = (Vec::new(), HashSet::new()); + for rule in rules { + if rule.len() > MAX_ROUTES_RULE_LENGTH { + invalid.push(format!( + "'{rule}': all rules must be less than {MAX_ROUTES_RULE_LENGTH} characters in \ + length" + )); + } + if !seen.insert(rule) { + invalid.push(format!( + "'{rule}': rule is a duplicate; rules must be unique" + )); + } + if let Some(prefix) = rule.strip_suffix('*') { + for other in rules { + if other != rule && other.starts_with(prefix) { + invalid.push(format!("'{other}': rule '{rule}' makes it redundant")); + } + } + } + } + invalid +} + +/// Mirrors `warnInvalid`. +fn warn_invalid(warn: &mut dyn FnMut(String), kind: &str, file: &str, invalid: &[Invalid]) { + if invalid.is_empty() { + return; + } + let list = invalid.iter().map( + |Invalid { + line, + line_number, + message, + }| match line { + Some(line) => { + let n = line_number.map(|n| format!(":{n}")).unwrap_or_default(); + format!(" {message}\n at {file}{n} | {line}") + } + None => format!(" {message}"), + }, + ); + warn(format!( + "Found {} invalid {kind} rule(s):\n{}", + invalid.len(), + list.collect::>().join("\n") + )); +} + +type Redirects = (Vec<(String, (usize, Redirect))>, Vec<(String, Redirect)>); + +/// The static and dynamic rules of `_redirects`. Mirrors `constructRedirects`. +pub fn redirects( + text: &str, + html_handling: HtmlHandling, + warn: &mut dyn FnMut(String), +) -> Redirects { + let (rules, invalid) = parse_redirects( + text, + html_handling, + MAX_STATIC_REDIRECT_RULES, + MAX_DYNAMIC_REDIRECT_RULES, + ); + warn_invalid(warn, "redirect", "_redirects", &invalid); + let (mut static_rules, mut dynamic_rules) = (Vec::new(), Vec::new()); + for (from, line_number, redirect) in rules { + if dynamic_rules.is_empty() && !from.contains('*') && !has_placeholder(&from) { + static_rules.push((from, (line_number, redirect))); + } else { + dynamic_rules.push((from, redirect)); + } + } + dynamic_rules.retain(|(from, _)| compiles(from, "_redirects", warn)); + (static_rules, dynamic_rules) +} + +/// Whether `rule` compiles to a matcher; a rule that does not, such as one repeating a +/// placeholder, is skipped with a warning. +fn compiles(rule: &str, file: &str, warn: &mut dyn FnMut(String)) -> bool { + let ok = compile(rule).is_some(); + if !ok { + warn(format!( + "Skipping the {file} rule {rule}: it cannot be compiled." + )); + } + ok +} + +/// An HTTP token character (RFC 9110 `tchar`). +const fn is_token_char(b: u8) -> bool { + b.is_ascii_alphanumeric() + || matches!( + b, + b'!' | b'#' + | b'$' + | b'%' + | b'&' + | b'\'' + | b'*' + | b'+' + | b'-' + | b'.' + | b'^' + | b'_' + | b'`' + | b'|' + | b'~' + ) +} + +/// The rules of `_headers`. Mirrors `constructHeaders`. +pub fn headers(text: &str, warn: &mut dyn FnMut(String)) -> Vec<(String, HeaderRule)> { + let (rules, invalid) = parse_headers(text, MAX_HEADER_RULES); + warn_invalid(warn, "header", "_headers", &invalid); + let mut out: Vec<(String, HeaderRule)> = Vec::new(); + for (path, rule) in rules { + match out.iter_mut().find(|(p, _)| *p == path) { + Some((_, existing)) => *existing = rule, + None => out.push((path, rule)), + } + } + out.retain(|(path, _)| compiles(path, "_headers", warn)); + out +} + +#[cfg(test)] +#[path = "parse-test.rs"] +mod tests; diff --git a/src/rust/assets-router/rules-test.rs b/src/rust/assets-router/rules-test.rs new file mode 100644 index 00000000000..9396146837a --- /dev/null +++ b/src/rust/assets-router/rules-test.rs @@ -0,0 +1,367 @@ +use super::super::header; +use super::*; + +fn values(matcher: &RulesMatcher, hostname: &str, pathname: &str) -> Vec { + matcher + .matches(hostname, pathname) + .map(|(value, _)| *value) + .collect() +} + +fn rendered(rules: &RulesMatcher<&str>, hostname: &str, pathname: &str) -> Vec { + let found = rules.matches(hostname, pathname); + found + .map(|(value, captures)| replacer(value, &captures)) + .collect() +} + +fn url(pathname: &str) -> Url { + Url::parse(&format!("https://example.com{pathname}")).unwrap() +} + +/// The target of the rule that applies to a request for `pathname`. +fn found(redirects: &Redirects, pathname: &str) -> String { + redirects.find(&url(pathname)).unwrap().to +} + +#[test] +fn rules_matcher_matches_simple_pathnames() { + let matcher = RulesMatcher::new([("/test".into(), 1), ("/some%20page".into(), 2)]); + assert_eq!(values(&matcher, "example.com", "/test"), [1]); + assert_eq!(values(&matcher, "example.com", "/some%20page"), [2]); +} + +#[test] +fn rules_matcher_matches_cross_host_requests() { + let matcher = RulesMatcher::new([ + ("/test".into(), 1), + ("/anotherpage".into(), 2), + ("https://custom.domain/test".into(), 3), + ("https://another.domain/test".into(), 4), + ("//fake.host/actually-a-path".into(), 5), + ]); + assert_eq!(values(&matcher, "custom.domain", "/test"), [1, 3]); + assert_eq!( + values(&matcher, "example.com", "//fake.host/actually-a-path"), + [5] + ); + assert_eq!(values(&matcher, "custom.domain", "/test"), [1, 3]); +} + +#[test] +fn rules_matcher_escapes_regex_metacharacters() { + let matcher = RulesMatcher::new([("/$~.%20/!+-/[bo|%7Bo%7D]...()".into(), 1)]); + assert_eq!( + values(&matcher, "example.com", "/$~.%20/!+-/[bo|%7Bo%7D]...()"), + [1] + ); + assert_eq!(values(&matcher, "example.com", "/anything-else"), [0u8; 0]); +} + +#[test] +fn rules_matcher_supports_splats_and_placeholders() { + let matcher = RulesMatcher::new([ + ("/foo/test/*".into(), "1/:splat"), + ("/foo/*".into(), "2/:splat"), + ("/blog/:code/:name".into(), "3/:name/:code"), + ("/blog/*".into(), "4/:splat"), + ( + "https://:subdomain.pages.:tld/*".into(), + "5/:subdomain/:tld/:splat", + ), + ( + "https://next.:subdomain.pages.:tld/*".into(), + "6/:subdomain/", + ), + ]); + let rendered = |host, path| rendered(&matcher, host, path); + assert_eq!( + rendered("example.com", "/foo/test/yes"), + ["1/yes", "2/test/yes"] + ); + assert_eq!(rendered("example.com", "/foo/test/"), ["1/", "2/test/"]); + assert_eq!(rendered("example.com", "/foo/test"), ["2/test"]); + assert_eq!(rendered("example.com", "/foo/"), ["2/"]); + assert!(rendered("example.com", "/foo").is_empty()); + assert_eq!( + rendered("example.com", "/blog/123/tricycle"), + ["3/tricycle/123", "4/123/tricycle"] + ); + assert_eq!(rendered("my.pages.dev", "/magic"), ["5/my/dev/magic"]); + assert_eq!(rendered("next.my.pages.dev", "/magic"), ["6/my/"]); +} + +#[test] +fn replacer_replaces_splats() { + assert_eq!( + replacer("/blog/:splat", &[("splat".into(), "look/a/value".into())]), + "/blog/look/a/value" + ); +} + +#[test] +fn replacer_replaces_placeholders() { + assert_eq!( + replacer( + "/:code/:name.jpg", + &[ + ("name".into(), "tricycle".into()), + ("code".into(), "123".into()) + ] + ), + "/123/tricycle.jpg" + ); +} + +#[test] +fn replacer_replaces_splats_and_placeholders() { + assert_eq!( + replacer( + "/:code/:splat", + &[ + ("splat".into(), "tricycle/images".into()), + ("code".into(), "123".into()) + ] + ), + "/123/tricycle/images" + ); +} + +#[test] +fn replacer_replaces_all_instances_globally() { + let input = "Link: ; rel=preload; as=script, ; rel=preload; as=script"; + assert_eq!( + replacer(input, &[("value".into(), "js".into())]), + "Link: ; rel=preload; as=script, ; rel=preload; as=script" + ); +} + +#[test] +fn replacer_preserves_capture_order_for_prefix_names() { + let matcher = RulesMatcher::new([("/:foobar/:foo".into(), "/:foobar/:foo")]); + assert_eq!( + rendered(&matcher, "example.com", "/long/short"), + ["/long/short"] + ); +} + +#[test] +fn redirect_location_matches_url_resolution_and_query_inheritance() { + let request = url::Url::parse("https://example.com/a/old?orig=1").unwrap(); + assert_eq!(resolve_location(&request, "new").unwrap(), "/a/new?orig=1"); + assert_eq!(resolve_location(&request, "/new").unwrap(), "/new?orig=1"); + assert_eq!(resolve_location(&request, "/new?x=2").unwrap(), "/new?x=2"); + assert_eq!(resolve_location(&request, "/new?").unwrap(), "/new?orig=1"); + assert_eq!( + resolve_location(&request, "/new#frag").unwrap(), + "/new?orig=1#frag" + ); + assert_eq!(resolve_location(&request, "/new#").unwrap(), "/new?orig=1"); + assert_eq!( + resolve_location(&request, "https://other.test/new").unwrap(), + "https://other.test/new?orig=1" + ); +} + +#[test] +fn redirects_static_and_dynamic_scenarios() { + let static_rules = [ + ( + "/compete".into(), + ( + 4, + Redirect { + status: 302, + to: "/hostless".into(), + }, + ), + ), + ( + "https://example.com/compete".into(), + ( + 5, + Redirect { + status: 302, + to: "/withhost".into(), + }, + ), + ), + ( + "https://example.com/backwards".into(), + ( + 6, + Redirect { + status: 302, + to: "/withhost".into(), + }, + ), + ), + ( + "/backwards".into(), + ( + 7, + Redirect { + status: 302, + to: "/hostless".into(), + }, + ), + ), + ]; + let redirects = Redirects::new(static_rules, []); + assert_eq!(found(&redirects, "/compete"), "/hostless"); + assert_eq!(found(&redirects, "/backwards"), "/withhost"); + + let redirects = Redirects::new( + [], + [ + ( + "/dynamic/:seg".into(), + Redirect { + status: 302, + to: "/:seg/new-dynamic/?with#params".into(), + }, + ), + ( + "/splat/*".into(), + Redirect { + status: 302, + to: "/:splat/new-splat".into(), + }, + ), + ( + "/splat/foo/*".into(), + Redirect { + status: 302, + to: "/will-never-fire".into(), + }, + ), + ( + "/partialPlaceholder:placeholder".into(), + Redirect { + status: 302, + to: "/new-partialPlaceholder:placeholder".into(), + }, + ), + ], + ); + assert_eq!( + found(&redirects, "/dynamic/foo"), + "/foo/new-dynamic/?with#params" + ); + assert_eq!(found(&redirects, "/splat/foo/bar"), "/foo/bar/new-splat"); + assert_eq!( + found(&redirects, "/partialPlaceholderfoo"), + "/new-partialPlaceholderfoo" + ); + assert_eq!(redirects.find(&url("/partialPlaceholderfoo/")), None); +} + +#[test] +fn static_rule_takes_precedence_and_matches_a_host_with_a_port() { + let rule = |to: &str, line: usize| { + ( + line, + Redirect { + status: 301, + to: to.into(), + }, + ) + }; + let redirects = Redirects::new( + [ + ("/old".into(), rule("/static", 1)), + ("https://example.com:8443/ported".into(), rule("/port", 2)), + ], + [( + "/*".into(), + Redirect { + status: 302, + to: "/dynamic".into(), + }, + )], + ); + assert_eq!(found(&redirects, "/old"), "/static"); + assert_eq!(found(&redirects, "/other"), "/dynamic"); + assert_eq!(found(&redirects, "/ported"), "/dynamic"); + assert_eq!( + redirects + .find(&Url::parse("https://example.com:8443/ported").unwrap()) + .unwrap() + .to, + "/port" + ); +} + +#[test] +fn open_redirect_defense_normalizes_double_slash() { + let redirects = Redirects::new( + [], + [( + "/foo/*".into(), + Redirect { + status: 302, + to: "/:splat".into(), + }, + )], + ); + assert_eq!(found(&redirects, "/foo//google.com"), "/google.com"); +} + +#[test] +fn applies_matching_rules_in_order() { + let matcher = RulesMatcher::new([ + ( + "/blog/:slug".into(), + HeaderRule { + set: vec![("Location".into(), "/first/:slug".into())], + unset: vec!["X-Remove".into()], + }, + ), + ( + "/blog/*".into(), + HeaderRule { + set: vec![ + ("location".into(), "/second/:splat".into()), + ("X-Added".into(), ":splat".into()), + ], + unset: vec![], + }, + ), + ]); + + assert_eq!( + apply_headers( + &matcher, + "example.com", + "/blog/post", + vec![header("Location", "/original"), header("X-Remove", "yes")], + ), + [ + header("Location", "/first/post"), + header("location", "/second/post"), + header("X-Added", "post"), + ] + ); +} + +// Upstream `handleRequest > _headers` unsets a header whose case differs from the response +// (`unset: ["ETAg"]` against an `ETag` header). Unset must be ASCII-case-insensitive. +#[test] +fn unset_is_case_insensitive() { + let matcher = RulesMatcher::new([( + "/art/*".into(), + HeaderRule { + set: vec![], + unset: vec!["ETAg".into()], + }, + )]); + assert_eq!( + apply_headers( + &matcher, + "example.com", + "/art/x", + vec![header("ETag", "\"abc\""), header("X-Keep", "1")], + ), + [header("X-Keep", "1")] + ); +} diff --git a/src/rust/assets-router/rules.rs b/src/rust/assets-router/rules.rs new file mode 100644 index 00000000000..0545cec0fe7 --- /dev/null +++ b/src/rust/assets-router/rules.rs @@ -0,0 +1,185 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! `_redirects` and `_headers` rule matching. +//! +//! Ported from workers-sdk's workers-shared, `asset-worker/src/utils/rules-engine.ts`. Relative +//! rules match the pathname; `https://` rules match hostname and pathname, ignoring the port. + +use std::collections::HashMap; +use std::collections::HashSet; + +use regex::Regex; +use url::Url; + +use super::ResponseHeader; +use super::affix; +use super::header; +use super::intent::collapse_slashes; + +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct Redirect { + pub status: u16, + pub to: String, +} + +#[derive(Default)] +pub struct HeaderRule { + pub set: Vec<(String, String)>, + pub unset: Vec, +} + +/// Rules in order, each a regex over the pathname, or over `https://host/path` for a rule with +/// a host, and its value. +pub struct RulesMatcher(Vec<(bool, Regex, T)>); + +impl RulesMatcher { + pub fn new(rules: impl IntoIterator) -> Self { + Self( + rules + .into_iter() + .filter_map(|(rule, value)| { + Some((rule.starts_with("https://"), compile(&rule)?, value)) + }) + .collect(), + ) + } + + /// Every matching rule's value, with its placeholders' values. + pub fn matches<'a>( + &'a self, + hostname: &str, + pathname: &'a str, + ) -> impl Iterator)> { + let absolute = format!("https://{hostname}{pathname}"); + self.0.iter().filter_map(move |(cross_host, regex, value)| { + let captures = regex.captures(if *cross_host { &absolute } else { pathname })?; + let names = regex.capture_names().flatten(); + let values = names.filter_map(|n| Some((n.into(), captures.name(n)?.as_str().into()))); + Some((value, values.collect())) + }) + } +} + +/// Mirrors `generateRuleRegExp`: `*` is a splat, `:name` a placeholder, which in the host does +/// not match a `.`. +pub(crate) fn compile(rule: &str) -> Option { + let tokens = Regex::new(r"\*|:[A-Za-z][A-Za-z0-9_]*").ok()?; + let host_end = rule + .strip_prefix("https://") + .and_then(|rest| rest.find('/')) + .map_or(0, |i| i + "https://".len()); + let (mut pattern, mut last) = (String::from("^"), 0); + for token in tokens.find_iter(rule) { + pattern += ®ex::escape(&rule[last..token.start()]); + pattern += &match token.as_str() { + "*" => "(?P.*)".to_owned(), + name if token.start() < host_end => format!("(?P<{}>[^/.]+)", &name[1..]), + name => format!("(?P<{}>[^/]+)", &name[1..]), + }; + last = token.end(); + } + Regex::new(&format!("{pattern}{}$", regex::escape(&rule[last..]))).ok() +} + +pub fn replacer(value: &str, replacements: &[(String, String)]) -> String { + replacements + .iter() + .fold(value.to_owned(), |value, (name, replacement)| { + value.replace(&format!(":{name}"), replacement) + }) +} + +/// `headers`, with the `_headers` rules that match applied. +pub fn apply_headers( + rules: &RulesMatcher, + hostname: &str, + pathname: &str, + mut headers: Vec, +) -> Vec { + let mut set = HashSet::new(); + for (rule, replacements) in rules.matches(hostname, pathname) { + for name in &rule.unset { + headers.retain(|h| !h.name.eq_ignore_ascii_case(name)); + } + for (name, value) in &rule.set { + if set.insert(name.to_ascii_lowercase()) { + headers.retain(|h| !h.name.eq_ignore_ascii_case(name)); + } + headers.push(header(name, replacer(value, &replacements))); + } + } + headers +} + +pub fn resolve_location(request: &Url, target: &str) -> Result { + let mut to = request.join(target)?; + if to.query().is_none_or(str::is_empty) { + to.set_query(request.query()); + } + if to.origin() != request.origin() { + return Ok(to.into()); + } + Ok(format!( + "{}{}{}", + to.path(), + affix("?", to.query()), + affix("#", to.fragment()) + )) +} + +/// A deployment's `_redirects` rules: static ones by path, with their line numbers, and dynamic +/// ones. +pub struct Redirects { + static_rules: HashMap, + dynamic_rules: RulesMatcher, +} + +impl Redirects { + pub fn new( + static_rules: impl IntoIterator, + dynamic_rules: impl IntoIterator, + ) -> Self { + Self { + static_rules: static_rules.into_iter().collect(), + dynamic_rules: RulesMatcher::new(dynamic_rules), + } + } + + /// The rule for a request for `url`: a static rule, the first declared of one with the host + /// and one without, else the first dynamic rule. + pub fn find(&self, url: &Url) -> Option { + let (hostname, pathname) = (url.host_str().unwrap_or_default(), url.path()); + let port = url.port().map(|port| port.to_string()); + let with_host = format!( + "https://{hostname}{}{pathname}", + affix(":", port.as_deref()) + ); + let static_rule = [with_host.as_str(), pathname] + .into_iter() + .filter_map(|key| self.static_rules.get(key)) + .min_by_key(|(line, _)| *line); + if let Some((_, redirect)) = static_rule { + return Some(redirect.clone()); + } + let (redirect, replacements) = self.dynamic_rules.matches(hostname, pathname).next()?; + let to = replacer(&redirect.to, &replacements).trim().to_owned(); + let scheme = to.split_once("://").map(|(scheme, _)| scheme); + let has_scheme = scheme.is_some_and(|s| { + !s.is_empty() && s.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'_') + }); + Some(Redirect { + status: redirect.status, + to: if has_scheme { + to + } else { + collapse_slashes(&to) + }, + }) + } +} + +#[cfg(test)] +#[path = "rules-test.rs"] +mod tests; diff --git a/src/rust/assets-router/serve-test.rs b/src/rust/assets-router/serve-test.rs new file mode 100644 index 00000000000..70a57638dfb --- /dev/null +++ b/src/rust/assets-router/serve-test.rs @@ -0,0 +1,137 @@ +// Upstream: asset-worker/tests/handler.test.ts, the headers and status of a served asset. + +use url::Url; + +use super::*; + +fn config(json: &str) -> Config { + crate::tests::from_json(json, false, false, &mut |w| panic!("{w}")).unwrap() +} + +fn asset(path: &str, status: u16, not_found: bool) -> Asset { + Asset { + path: path.into(), + etag: "e".into(), + status, + not_found, + } +} + +fn serve(config: &Config, method: Method, conditions: Conditions<'_>, asset: &Asset) -> Served { + let url = Url::parse(&format!("https://example.com{}", asset.path)).unwrap(); + let request = Request { + url: &url, + method, + sec_fetch_mode: None, + sec_fetch_dest: None, + }; + config.serve(&request, conditions, asset, "1-2") +} + +fn header(name: &str, value: &str) -> ResponseHeader { + ResponseHeader { + name: name.into(), + value: value.into(), + } +} + +#[test] +fn content_types() { + assert_eq!( + content_type("/a/b.HTML").as_deref(), + Some("text/html; charset=utf-8") + ); + assert_eq!(content_type("/x.wasm").as_deref(), Some("application/wasm")); + assert_eq!(content_type("/x.unknown"), None); + assert_eq!(content_type("/html"), None); +} + +#[test] +fn served_headers() { + let config = config("{}"); + let get = Conditions::default(); + assert_eq!( + serve(&config, Method::Get, get, &asset("/a.css", 200, false)), + Served { + status: 200, + headers: vec![ + header("ETag", "\"1-2\""), + header("Content-Type", "text/css; charset=utf-8"), + header("Cache-Control", "public, max-age=0, must-revalidate"), + ], + body: true, + } + ); + let private = Conditions { + private: true, + ..get + }; + let served = serve( + &config, + Method::Head, + private, + &asset("/404.html", 404, true), + ); + assert_eq!(served.status, 404); + assert!(!served.body); + assert_eq!(served.headers.len(), 2); +} + +#[test] +fn not_modified() { + let config = config("{}"); + for (if_none_match, status) in [ + ("\"1-2\"", 304), + ("W/\"1-2\"", 304), + ("\"1-3\"", 200), + ("1-2", 200), + ] { + let conditions = Conditions { + if_none_match: Some(if_none_match), + private: false, + }; + let served = serve(&config, Method::Get, conditions, &asset("/a", 200, false)); + assert_eq!(served.status, status, "{if_none_match}"); + assert_eq!(served.body, status == 200); + } +} + +#[test] +fn debug_header_for_a_navigation_not_found_answered() { + let header = "X-Mf-Additional-Response-Log"; + let has = |json, navigation, not_found| { + let config = + crate::tests::from_json(json, false, navigation, &mut |w| panic!("{w}")).unwrap(); + let served = serve( + &config, + Method::Get, + Conditions::default(), + &asset("/index.html", 200, not_found), + ); + served.headers.iter().any(|h| h.name == header) + }; + let debug = r#"{"debug": true}"#; + assert!(has(debug, true, true)); + assert!(!has(debug, true, false)); + assert!(!has(debug, false, true)); + assert!(!has("{}", true, true)); +} + +#[test] +fn custom_headers_apply() { + let config = config(r#"{"headers": "/*\n X-Site: :splat\n ! Cache-Control\n ETag: x"}"#); + let served = serve( + &config, + Method::Get, + Conditions::default(), + &asset("/a.txt", 200, false), + ); + assert_eq!( + served.headers, + [ + header("Content-Type", "text/plain; charset=utf-8"), + header("x-site", "a.txt"), + header("etag", "x"), + ] + ); +} diff --git a/src/rust/assets-router/serve.rs b/src/rust/assets-router/serve.rs new file mode 100644 index 00000000000..65b5e2175ce --- /dev/null +++ b/src/rust/assets-router/serve.rs @@ -0,0 +1,130 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! The head of the response that serves an asset, whatever store its body comes from. +//! +//! Ported from workers-sdk's workers-shared, `asset-worker/src/utils/headers.ts`. + +use super::Config; +use super::Method; +use super::Request; +use super::ResponseHeader; +use super::header; +use super::intent::Asset; + +#[derive(Debug, Eq, PartialEq)] +pub struct Served { + pub status: u16, + pub headers: Vec, + /// Whether the asset's body follows. + pub body: bool, +} + +/// The request headers that serving looks at. +#[derive(Clone, Copy, Debug, Default)] +pub struct Conditions<'a> { + pub if_none_match: Option<&'a str>, + /// Whether the request has an `Authorization` or a `Range` header. + pub private: bool, +} + +/// The `Content-Type` of `path`, by its extension, with a charset for text. +pub fn content_type(path: &str) -> Option { + let extension = path + .rsplit_once('.') + .map_or("", |(_, e)| e) + .to_ascii_lowercase(); + let content_type = match extension.as_str() { + "html" | "htm" => "text/html", + "css" => "text/css", + "js" | "mjs" | "cjs" => "text/javascript", + "json" | "map" => "application/json", + "jsonld" => "application/ld+json", + "webmanifest" => "application/manifest+json", + "txt" => "text/plain", + "md" => "text/markdown", + "csv" => "text/csv", + "ics" => "text/calendar", + "xml" => "application/xml", + "rss" => "application/rss+xml", + "atom" => "application/atom+xml", + "svg" => "image/svg+xml", + "png" => "image/png", + "apng" => "image/apng", + "jpg" | "jpeg" => "image/jpeg", + "gif" => "image/gif", + "webp" => "image/webp", + "avif" => "image/avif", + "bmp" => "image/bmp", + "ico" => "image/vnd.microsoft.icon", + "woff" => "font/woff", + "woff2" => "font/woff2", + "ttf" => "font/ttf", + "otf" => "font/otf", + "eot" => "application/vnd.ms-fontobject", + "wasm" => "application/wasm", + "pdf" => "application/pdf", + "zip" => "application/zip", + "gz" => "application/gzip", + "tar" => "application/x-tar", + "mp3" => "audio/mpeg", + "m4a" => "audio/mp4", + "wav" => "audio/wav", + "ogg" | "oga" => "audio/ogg", + "flac" => "audio/flac", + "mp4" => "video/mp4", + "webm" => "video/webm", + "ogv" => "video/ogg", + "mov" => "video/quicktime", + _ => return None, + }; + Some(if content_type.starts_with("text/") { + format!("{content_type}; charset=utf-8") + } else { + content_type.to_owned() + }) +} + +impl Config { + /// The response to `request` that serves `asset`, whose `ETag` is now `etag`. + pub fn serve( + &self, + request: &Request<'_>, + conditions: Conditions<'_>, + asset: &Asset, + etag: &str, + ) -> Served { + let etag = format!("\"{etag}\""); + let mut headers = vec![header("ETag", etag.clone())]; + headers.extend(content_type(&asset.path).map(|t| header("Content-Type", t))); + if !conditions.private { + headers.push(header( + "Cache-Control", + "public, max-age=0, must-revalidate", + )); + } + if self.debug && asset.not_found && self.routing.intent.navigation_prefers_asset_serving { + let value = "`Sec-Fetch-Mode: navigate` header present - using `not_found_handling` \ + behavior"; + headers.push(header("X-Mf-Additional-Response-Log", value)); + } + let headers = self.routing.custom_headers(request, headers); + let if_none_match = conditions.if_none_match.unwrap_or_default(); + let (status, body) = + if if_none_match == etag || if_none_match.strip_prefix("W/") == Some(&etag) { + (304, false) + } else { + (asset.status, request.method != Method::Head) + }; + Served { + status, + headers, + body, + } + } +} + +#[cfg(test)] +#[path = "serve-test.rs"] +mod tests; diff --git a/src/workerd/server/AGENTS.md b/src/workerd/server/AGENTS.md index 9ad54c7db3c..f3afc01ee55 100644 --- a/src/workerd/server/AGENTS.md +++ b/src/workerd/server/AGENTS.md @@ -81,6 +81,7 @@ Bazel labels: `//src/workerd/server:workerd`, `//src/workerd/server:workerd-cli` | `services/network.rs` | `TlsOptions` to rustls and the `allow` / `deny` peer filter | | `services/disk.rs` | The `disk` service (synchronous file I/O in 64 KiB steps) | | `services/bucket.rs` | The `bucket` service (`ClusterBucket`): a directory of the cluster's bucket over HTTP; KV (`kv/kv.js`) keeps large values there in a cluster; tests in `tests/kv/` and `tests/cluster/` | +| `services/assets/` | The `assets` service: a Worker's static assets, routed between the user Worker and the files. `router/` is pure (routing, `_headers` / `_redirects` / `.assetsignore` / `run_worker_first` parsing, response heads), ported from workers-sdk's workers-shared; `mod.rs` stats and streams files and forwards to the user Worker; wrangler's `assets` becomes `-assets`; tests in `tests/assets/` | | `services/rewriter.rs` | A socket's or external server's `HttpOptions` applied to traffic: URL style, cf blob header, injected headers, the capnp-over-CONNECT host. Headers stay `kj::HttpHeaders`: the rewriter names the edits (`HeaderEdit`) and the factory applies them by table id | | `listen/mod.rs` | Sockets bound through kj-rs-io (`TokioAddress`; a `loopback:` name is a queue of the server's registry) before the services start; HTTP/HTTPS through kj-hyper, TCP as `connect()` events, the debug port. One future per listener (accept loop plus its connections); an accept-loop failure is fatal, a connection failure ends that connection. A failed HTTP call is logged (`Uncaught exception: ` and the error as KJ prints it, `log_uncaught`); kj-hyper answers it with a bare 500 | | `listen/udp.rs` | UDP sockets: each peer's datagrams as one flow (`UdpFlow`), dispatched as a `connect()` event | diff --git a/src/workerd/server/cli/wrangler-test.rs b/src/workerd/server/cli/wrangler-test.rs index 9692196c993..f3815b266a9 100644 --- a/src/workerd/server/cli/wrangler-test.rs +++ b/src/workerd/server/cli/wrangler-test.rs @@ -845,3 +845,136 @@ fn failed_bundle_watches_sources_and_missing_main() { [dir.join("index.ts"), dir.join("lib.ts")] ); } + +#[test] +fn assets_require_experimental() { + let dir = project(&[ + ( + "wrangler.jsonc", + r#"{ "compatibility_date": "2026-01-01", "assets": { "directory": "public" } }"#, + ), + ("public/index.html", ""), + ]); + let process = Process::new(None, None, Vec::new()); + let path = dir.join("wrangler.jsonc"); + let error = load(path.to_str().unwrap(), &process, false).err().unwrap(); + assert!(error.contains("assets"), "{error}"); + assert!(error.contains("--experimental"), "{error}"); + let error = load_deployment(path.to_str().unwrap(), Some("app"), &process) + .err() + .unwrap(); + assert!(error.contains("assets"), "{error}"); +} + +#[test] +fn assets_route_through_the_assets_service() { + let wrangler = r#"{ "name": "app", "main": "index.js", "compatibility_date": "2026-01-01", + "compatibility_flags": ["nodejs_compat"], "no_bundle": true, + "assets": { "directory": "public", "binding": "ASSETS", + "not_found_handling": "404-page", "run_worker_first": ["/api/*"] } }"#; + let config = load_project(&[ + ("wrangler.jsonc", wrangler), + ("index.js", ""), + ("public/_headers", "/*\n X: y\n"), + ]) + .unwrap(); + let message = read(config); + let conf = message.get_root::().unwrap(); + let socket = conf.get_sockets().unwrap().get(0); + assert_eq!( + socket.get_service().unwrap().get_name().unwrap(), + "app-assets" + ); + let services = conf.get_services().unwrap(); + let names: Vec<_> = services + .iter() + .map(|s| s.get_name().unwrap().to_str().unwrap().to_owned()) + .collect(); + assert_eq!(names, ["app", "app-assets"]); + + let service::Which::Worker(w) = services.get(0).which().unwrap() else { + panic!("not a worker"); + }; + let b = w.unwrap().get_bindings().unwrap().get(0); + assert_eq!(b.get_name().unwrap(), "ASSETS"); + let worker::binding::Which::Service(d) = b.which().unwrap() else { + panic!("not a service binding"); + }; + let d = d.unwrap(); + assert_eq!(d.get_name().unwrap(), "app-assets"); + assert_eq!(d.get_entrypoint().unwrap(), "AssetWorker"); + + let service::Which::Assets(assets) = services.get(1).which().unwrap() else { + panic!("not assets"); + }; + let assets = assets.unwrap(); + assert!( + assets + .get_path() + .unwrap() + .to_str() + .unwrap() + .ends_with("public") + ); + assert_eq!(assets.get_user_worker().unwrap().get_name().unwrap(), "app"); + assert!(!assets.has_compatibility_date()); + let config: Value = + serde_json::from_str(assets.get_config().unwrap().to_str().unwrap()).unwrap(); + assert_eq!( + config, + serde_json::json!({ + "html_handling": null, + "not_found_handling": "404-page", + "base_path": null, + "run_worker_first": ["/api/*"], + "headers": "/*\n X: y\n", + "redirects": null, + "ignore": null, + }) + ); +} + +#[test] +fn assets_without_main_serve_only_files() { + let config = load_project(&[( + "wrangler.jsonc", + r#"{ "compatibility_date": "2026-01-01", "assets": { "directory": "." } }"#, + )]) + .unwrap(); + let message = read(config); + let conf = message.get_root::().unwrap(); + let services = conf.get_services().unwrap(); + assert_eq!(services.len(), 1); + assert_eq!(services.get(0).get_name().unwrap(), "main-assets"); + let service::Which::Assets(assets) = services.get(0).which().unwrap() else { + panic!("not assets"); + }; + let assets = assets.unwrap(); + assert!(!assets.has_user_worker()); + assert_eq!(assets.get_compatibility_date().unwrap(), "2026-01-01"); + + for (wrangler, expected) in [ + ( + r#"{ "compatibility_date": "2026-01-01" }"#, + "Missing entry-point", + ), + ( + r#"{ "compatibility_date": "2026-01-01", + "assets": { "directory": ".", "html_handling": "sometimes" } }"#, + "unknown variant `sometimes`", + ), + ( + r#"{ "compatibility_date": "2026-01-01", "vars": { "A": 1 }, + "assets": { "directory": "." } }"#, + "vars needs a Worker, but there is no main", + ), + ( + r#"{ "compatibility_date": "2026-01-01", + "assets": { "directory": ".", "binding": "ASSETS" } }"#, + "assets.binding needs a Worker, but there is no main", + ), + ] { + let error = load_project(&[("wrangler.jsonc", wrangler)]).err().unwrap(); + assert!(error.contains(expected), "{error}"); + } +} diff --git a/src/workerd/server/cli/wrangler.rs b/src/workerd/server/cli/wrangler.rs index ee07e7df80b..b49ebbd5e1d 100644 --- a/src/workerd/server/cli/wrangler.rs +++ b/src/workerd/server/cli/wrangler.rs @@ -10,8 +10,8 @@ //! `main` must be an ES module and the modules it imports are found by scanning its directory with //! wrangler's default module rules. Supported fields: `name`, `main`, `compatibility_date`, //! `compatibility_flags`, `vars`, `durable_objects`, `exports`, `migrations`, `kv_namespaces`, -//! `triggers` (`crons`), `queues`, `workflows`, `no_bundle` and `$schema`. Any other field is an -//! error rather than silently ignored. +//! `triggers` (`crons`), `queues`, `workflows`, `assets`, `no_bundle` and `$schema`. Any other +//! field is an error rather than silently ignored. //! //! Durable Objects are stored on disk under `.wrangler/state/workerd/` next to the config, one //! directory per class, named `-` by its storage key (see [`do_classes`]). @@ -28,15 +28,18 @@ //! `/workerd/workflows.js`), the engine of this Worker's `workflowsEngine`; each binding is the //! Workflow API over it, and its instances are in `.wrangler/state/workerd/workflows/`. //! `limits.steps` and `default_retention` apply; `concurrency` and `jurisdiction` are ignored; -//! `schedules`, and a `script_name` other than this Worker, are errors. The -//! server listens on `localhost:8787`, as `wrangler dev` does; `--socket-addr http=` -//! overrides it. +//! `schedules`, and a `script_name` other than this Worker, are errors. `assets`, also +//! `--experimental`, are the files in `directory`, served by the assets service `-assets`, +//! which routes each request to this Worker or to a file as Cloudflare does; `binding` is its +//! `AssetWorker` entrypoint. Without `main` it serves only +//! the files. The server listens on `localhost:8787`, as `wrangler dev` does, to `-assets` +//! if there are assets, else to the worker; `--socket-addr http=` overrides the address. //! //! For `workerd deploy`, [`load_deployment`] translates the same fields into a cluster deployment //! (`cluster::compose`), which holds only services: the worker, named after the deployment, with //! no socket, and its Durable Objects stored on the disk service [`DEPLOYMENT_STORAGE`], which //! each node's own config defines. A deployment cannot have `kv_namespaces`, `triggers.crons`, -//! `queues` or `workflows` yet. +//! `queues`, `workflows` or `assets` yet. use std::collections::BTreeMap; use std::fs; @@ -45,6 +48,7 @@ use std::path::PathBuf; use cluster::compose::workflow_namespace_key; use serde::Deserialize; +use serde::Serialize; use serde_json::Value; use workerd_capnp::config; use workerd_capnp::service; @@ -68,7 +72,7 @@ struct Wrangler { #[serde(rename = "$schema")] _schema: Option, name: Option, - main: String, + main: Option, compatibility_date: String, #[serde(default)] compatibility_flags: Vec, @@ -89,6 +93,43 @@ struct Wrangler { workflows: Vec, #[serde(default)] no_bundle: bool, + assets: Option, +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct Assets { + directory: String, + binding: Option, + html_handling: Option, + not_found_handling: Option, + base_path: Option, + run_worker_first: Option, +} + +#[derive(Deserialize, Serialize)] +#[serde(rename_all = "kebab-case")] +enum HtmlHandling { + AutoTrailingSlash, + ForceTrailingSlash, + DropTrailingSlash, + None, +} + +#[derive(Deserialize, Serialize)] +#[serde(rename_all = "kebab-case")] +enum NotFoundHandling { + SinglePageApplication, + #[serde(rename = "404-page")] + Page404, + None, +} + +#[derive(Deserialize, Serialize)] +#[serde(untagged)] +enum RunWorkerFirst { + All(bool), + Rules(Vec), } #[derive(Deserialize)] @@ -340,11 +381,18 @@ fn translate(path: &str, process: &Process, target: &Target) -> Result<(String, }; let classes = do_classes(&wrangler)?; - let sources = if wrangler.no_bundle { - unbundled(root, &wrangler.main, &watch)? - } else { - bundle(&esbuild(root), root, &wrangler.main, &watch)? + let sources = match &wrangler.main { + None if wrangler.assets.is_some() => { + assets_only(&wrangler)?; + vec![] + } + None => { + return Err("Missing entry-point to Worker script or to assets directory".to_owned()); + } + Some(main) if wrangler.no_bundle => unbundled(root, main, &watch)?, + Some(main) => bundle(&esbuild(root), root, main, &watch)?, }; + let workers = u32::from(wrangler.main.is_some()); let serving = matches!(target, Target::Serve { .. }); let storage = root.join(".wrangler/state/workerd"); @@ -355,50 +403,38 @@ fn translate(path: &str, process: &Process, target: &Target) -> Result<(String, let mut conf = message.init_root::(); let do_storage = u32::from(serving && !classes.is_empty()); let mut services = conf.reborrow().init_services( - 1 + do_storage + workers + + do_storage + 3 * u32::from(!wrangler.kv_namespaces.is_empty()) + 2 * u32::from(!wrangler.triggers.crons.is_empty()) + 2 * u32::from(!wrangler.queues.is_empty()) - + 2 * u32::from(!wrangler.workflows.is_empty()), + + 2 * u32::from(!wrangler.workflows.is_empty()) + + u32::from(wrangler.assets.is_some()), ); - bundled_services(&mut services, 1 + do_storage, &name, &storage, &wrangler)?; - - let mut service = services.reborrow().get(0); - service.set_name(&name); - let mut worker = service.init_worker(); - worker.set_compatibility_date(&wrangler.compatibility_date); - worker - .set_compatibility_flags(&wrangler.compatibility_flags[..]) - .map_err(|e| e.to_string())?; - - set_modules(worker.reborrow(), &sources)?; - set_bindings(worker.reborrow(), &wrangler, &name); - if !wrangler.workflows.is_empty() { - set_workflows_engine(worker.reborrow(), &name, &wrangler.workflows); - } - - if !classes.is_empty() { - let mut ns = worker - .reborrow() - .init_durable_object_namespaces(classes.len() as u32); - for (i, (class, key, sql)) in classes.iter().enumerate() { - let mut n = ns.reborrow().get(i as u32); - n.set_class_name(class); - n.set_unique_key(format!("{name}-{key}")); - n.set_enable_sql(*sql); - } - if serving { - // Named after the worker, so it cannot collide with it. - let storage_service = format!("{name}-do-storage"); - worker - .init_durable_object_storage() - .set_local_disk(&storage_service); - disk_service(services.get(1), &storage_service, &storage); - } else { - worker - .init_durable_object_storage() - .set_local_disk(DEPLOYMENT_STORAGE); + let at = workers + do_storage; + bundled_services(&mut services, at, &name, root, &storage, &wrangler, &watch)?; + + if workers == 1 { + let mut service = services.reborrow().get(0); + service.set_name(&name); + let mut worker = service.init_worker(); + worker.set_compatibility_date(&wrangler.compatibility_date); + worker + .set_compatibility_flags(&wrangler.compatibility_flags[..]) + .map_err(|e| e.to_string())?; + set_modules(worker.reborrow(), &sources)?; + set_bindings(worker.reborrow(), &wrangler, &name); + if !wrangler.workflows.is_empty() { + set_workflows_engine(worker.reborrow(), &name, &wrangler.workflows); } + set_durable_objects(worker, &classes, &name, serving); + } + if do_storage == 1 { + disk_service( + services.get(workers), + &format!("{name}-do-storage"), + &storage, + ); } if serving { @@ -406,7 +442,11 @@ fn translate(path: &str, process: &Process, target: &Target) -> Result<(String, socket.set_name("http"); socket.set_address(DEFAULT_ADDRESS); socket.reborrow().init_http(); - socket.init_service().set_name(&name); + let target = match &wrangler.assets { + Some(_) => format!("{name}-assets"), + None => name.clone(), + }; + socket.init_service().set_name(target); } let config = Config::from_words(&words_from_bytes( @@ -416,6 +456,59 @@ fn translate(path: &str, process: &Process, target: &Target) -> Result<(String, Ok((name, config)) } +/// Assets without `main` have no Worker for Worker-only fields to configure. +fn assets_only(wrangler: &Wrangler) -> Result<(), String> { + let worker_only = [ + ("vars", !wrangler.vars.is_empty()), + ("durable_objects", wrangler.durable_objects.is_some()), + ("exports", !wrangler.exports.is_empty()), + ("migrations", !wrangler.migrations.is_empty()), + ("kv_namespaces", !wrangler.kv_namespaces.is_empty()), + ("triggers.crons", !wrangler.triggers.crons.is_empty()), + ("queues", !wrangler.queues.is_empty()), + ("workflows", !wrangler.workflows.is_empty()), + ( + "assets.binding", + wrangler + .assets + .as_ref() + .is_some_and(|a| a.binding.is_some()), + ), + ]; + match worker_only.iter().find(|(_, used)| *used) { + Some((field, _)) => Err(format!("{field} needs a Worker, but there is no main")), + None => Ok(()), + } +} + +/// The Durable Object namespaces of `classes`, stored on `-do-storage` when `serving`, else +/// on [`DEPLOYMENT_STORAGE`]. +fn set_durable_objects( + mut worker: worker::Builder<'_>, + classes: &[(String, String, bool)], + name: &str, + serving: bool, +) { + if !classes.is_empty() { + let mut ns = worker + .reborrow() + .init_durable_object_namespaces(classes.len() as u32); + for (i, (class, key, sql)) in classes.iter().enumerate() { + let mut n = ns.reborrow().get(i as u32); + n.set_class_name(class); + n.set_unique_key(format!("{name}-{key}")); + n.set_enable_sql(*sql); + } + // Named after the worker, so it cannot collide with it. + let storage = if serving { + format!("{name}-do-storage") + } else { + DEPLOYMENT_STORAGE.to_owned() + }; + worker.init_durable_object_storage().set_local_disk(storage); + } +} + fn set_modules(worker: worker::Builder<'_>, sources: &[Source]) -> Result<(), String> { let mut ms = worker.init_modules(sources.len() as u32); for (i, (module_name, kind, bytes)) in sources.iter().enumerate() { @@ -433,14 +526,15 @@ fn set_modules(worker: worker::Builder<'_>, sources: &[Source]) -> Result<(), St Ok(()) } -/// `kv_namespaces`, `triggers.crons`, `queues` and `workflows` need `--experimental`, and a -/// deployment cannot have them yet. +/// `kv_namespaces`, `triggers.crons`, `queues`, `workflows` and `assets` need `--experimental`, +/// and a deployment cannot have them yet. fn check_experimental(wrangler: &Wrangler, target: &Target) -> Result<(), String> { for (used, field) in [ (!wrangler.kv_namespaces.is_empty(), "kv_namespaces"), (!wrangler.triggers.crons.is_empty(), "triggers.crons"), (!wrangler.queues.is_empty(), "queues"), (!wrangler.workflows.is_empty(), "workflows"), + (wrangler.assets.is_some(), "assets"), ] { match target { _ if !used => {} @@ -459,15 +553,17 @@ fn check_experimental(wrangler: &Wrangler, target: &Target) -> Result<(), String Ok(()) } -/// The KV, cron, Queues and Workflows Workers `wrangler` asks for, with their disks under +/// The KV, cron, Queues, Workflows and assets services `wrangler` asks for, with their disks under /// `storage`, from `services[at]`. Each is named after the worker, `-`, so it cannot /// collide with it. fn bundled_services( services: &mut capnp::struct_list::Builder<'_, service::Owned>, mut at: u32, name: &str, + root: &Path, storage: &Path, wrangler: &Wrangler, + watch: Watch, ) -> Result<(), String> { if !wrangler.kv_namespaces.is_empty() { kv_services(services, at, &format!("{name}-kv"), &storage.join("kv"))?; @@ -486,6 +582,63 @@ fn bundled_services( if !wrangler.workflows.is_empty() { let dir = storage.join("workflows"); workflows_services(services, at, name, &dir, wrangler)?; + at += 2; + } + if let Some(assets) = &wrangler.assets { + assets_services(services, at, name, root, assets, wrangler, watch)?; + } + Ok(()) +} + +/// The assets service `-assets` at `services[at]`, which the socket serves: the files +/// of `assets.directory`, and `target` if there is a `main`. Its config holds the text of the +/// directory's `_headers`, `_redirects` and `.assetsignore`, which are watched. +fn assets_services( + services: &mut capnp::struct_list::Builder<'_, service::Owned>, + at: u32, + target: &str, + root: &Path, + assets: &Assets, + wrangler: &Wrangler, + watch: Watch, +) -> Result<(), String> { + let dir = root.join(&assets.directory); + if !dir.is_dir() { + return Err(format!( + "assets.directory: {}: not a directory", + dir.display() + )); + } + let text = |file| { + let path = dir.join(file); + watch(&path)?; + match fs::read_to_string(&path) { + Ok(text) => Ok(Some(text)), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(None), + Err(e) => Err(format!("{}: {e}", path.display())), + } + }; + let config = serde_json::json!({ + "html_handling": assets.html_handling, + "not_found_handling": assets.not_found_handling, + "base_path": assets.base_path, + "run_worker_first": assets.run_worker_first, + "headers": text("_headers")?, + "redirects": text("_redirects")?, + "ignore": text(".assetsignore")?, + }); + let mut service = services.reborrow().get(at); + service.set_name(format!("{target}-assets")); + let mut service = service.init_assets(); + service.set_path(dir.to_string_lossy()); + service.set_config(config.to_string()); + if wrangler.main.is_some() { + service.init_user_worker().set_name(target); + } else { + service.set_compatibility_date(&wrangler.compatibility_date); + service + .set_compatibility_flags(&wrangler.compatibility_flags[..]) + .map_err(|e| e.to_string())?; } Ok(()) } @@ -617,7 +770,9 @@ fn set_bindings(worker: worker::Builder<'_>, wrangler: &Wrangler, name: &str) { .as_ref() .map_or(&[][..], |d| &d.bindings); let n = wrangler.vars.len() + do_bindings.len() + kv.len(); - let mut bs = worker.init_bindings((n + producers.len() + workflows.len()) as u32); + let assets = wrangler.assets.as_ref().and_then(|a| a.binding.as_ref()); + let m = n + producers.len() + workflows.len(); + let mut bs = worker.init_bindings((m + usize::from(assets.is_some())) as u32); for (i, (k, v)) in wrangler.vars.iter().enumerate() { let mut b = bs.reborrow().get(i as u32); b.set_name(k); @@ -660,6 +815,13 @@ fn set_bindings(worker: worker::Builder<'_>, wrangler: &Wrangler, name: &str) { fetcher.set_name("fetcher"); workflow_binding(fetcher.init_service(), name, &w.name); } + if let Some(binding) = assets { + let mut b = bs.get(m as u32); + b.set_name(binding); + let mut designator = b.init_service(); + designator.set_name(format!("{name}-assets")); + designator.set_entrypoint("AssetWorker"); + } } /// The Workflow API of the Workflow `workflow` of the worker `name`: the entrypoint diff --git a/src/workerd/server/factory/worker-factory-listen.c++ b/src/workerd/server/factory/worker-factory-listen.c++ index 02e082cd381..bc79a15a92a 100644 --- a/src/workerd/server/factory/worker-factory-listen.c++ +++ b/src/workerd/server/factory/worker-factory-listen.c++ @@ -71,6 +71,45 @@ class RewritingResponse final: public kj::HttpService::Response { } }; +// The user Worker's answer to a request for a remote image, sent only if +// `should_block_image_response` lets it through, else replaced by a 403 whose body `finish()` +// writes. +class ImageFilterResponse final: public kj::HttpService::Response { + public: + explicit ImageFilterResponse(kj::HttpService::Response& inner): inner(inner) {} + + kj::Own send(uint statusCode, + kj::StringPtr statusText, + const kj::HttpHeaders& headers, + kj::Maybe expectedBodySize = kj::none) override { + auto type = headers.get(kj::HttpHeaderId::CONTENT_TYPE).orDefault(""_kj); + if (!should_block_image_response( + static_cast(statusCode), type.asBytes().as())) { + return inner.send(statusCode, statusText, headers, expectedBodySize); + } + auto blocked = headers.cloneShallow(); + blocked.clear(); + blocked.setPtr(kj::HttpHeaderId::CONTENT_TYPE, "text/plain;charset=UTF-8"); + body = inner.send(403, "Forbidden", blocked, BLOCKED.size()); + return newNullOutputStream(); + } + + kj::Own acceptWebSocket(const kj::HttpHeaders& headers) override { + return inner.acceptWebSocket(headers); + } + + kj::Promise finish() { + KJ_IF_SOME(out, body) { + co_await out->write(BLOCKED.asBytes()); + } + } + + private: + static constexpr kj::StringPtr BLOCKED = "Blocked"_kj; + kj::HttpService::Response& inner; + kj::Maybe> body; +}; + // The connect() answer of a raw TCP connection: there is no HTTP response to write. class NullConnectResponse final: public kj::HttpService::ConnectResponse { public: @@ -127,6 +166,14 @@ kj::Own new_rewriting_response(kj::HttpService::Respo return kj::heap(inner, resolveEdits(table, edits)); } +kj::Own new_image_filter_response(kj::HttpService::Response& inner) { + return kj::heap(inner); +} + +kj::Promise image_filter_finish(kj::HttpService::Response& response) { + return kj::downcast(response).finish(); +} + kj::Own new_null_connect_response() { return kj::heap(); } diff --git a/src/workerd/server/factory/worker-factory.c++ b/src/workerd/server/factory/worker-factory.c++ index adac4c8e87a..ee6f4a9980f 100644 --- a/src/workerd/server/factory/worker-factory.c++ +++ b/src/workerd/server/factory/worker-factory.c++ @@ -1457,9 +1457,15 @@ MainModuleIsPython isPythonMainModule(config::Worker::Reader conf) { return conf.getModules()[0].isPythonModule() ? MainModuleIsPython::YES : MainModuleIsPython::NO; } -// Compiles a config worker's compatibility flags into `arena`, as the worker's errors. +MainModuleIsPython isPythonMainModule(config::AssetsService::Reader) { + return MainModuleIsPython::NO; +} + +// Compiles the compatibility flags of a config worker, or of an assets service with no user +// worker, into `arena`, as the worker's errors. +template CompatibilityFlags::Reader compileFlags(const WorkerFactory::Options& options, - config::Worker::Reader conf, + Conf conf, capnp::MallocMessageBuilder& arena, ErrorReporter& errorReporter) { auto flags = arena.initRoot(); @@ -1924,6 +1930,31 @@ kj::Promise<::rust::Vec> factory_bench_startup(const WorkerFactory& fa .as(); } +bool factory_assets_navigation_prefers_asset_serving( + const WorkerFactory& constFactory, uint32_t configService) { + auto& factory = constFactory.getImpl(); + auto services = factory.config.getServices(); + auto assets = services[configService].getAssets(); + capnp::MallocMessageBuilder arena; + ErrorReporter errorReporter; + if (assets.hasUserWorker()) { + // The user worker reports errors in its flags itself. + auto name = assets.getUserWorker().getName(); + for (auto service: services) { + if (service.getName() == name && service.isWorker()) { + return compileFlags(*factory.options, service.getWorker(), arena, errorReporter) + .getAssetsSecFetchModeNavigateHeaderPrefersAssetServing(); + } + } + return false; + } + auto flags = compileFlags(*factory.options, assets, arena, errorReporter); + if (!errorReporter.errors.empty()) { + kj::throwFatalException(KJ_EXCEPTION(FAILED, kj::strArray(errorReporter.errors, "; "))); + } + return flags.getAssetsSecFetchModeNavigateHeaderPrefersAssetServing(); +} + WorkerInfo worker_info(const CompiledWorker& worker) { return worker.getImpl().info; } diff --git a/src/workerd/server/factory/worker-factory.h b/src/workerd/server/factory/worker-factory.h index 1c24d96d831..7086ea10905 100644 --- a/src/workerd/server/factory/worker-factory.h +++ b/src/workerd/server/factory/worker-factory.h @@ -216,6 +216,8 @@ class CompiledWorker final { mutable kj::Own impl; }; +bool factory_assets_navigation_prefers_asset_serving( + const WorkerFactory& factory, uint32_t configService); WorkerInfo worker_info(const CompiledWorker& worker); void worker_set_ctx_exports(const CompiledWorker& worker, ::rust::Slice globals); void worker_unlink(const CompiledWorker& worker); @@ -454,6 +456,8 @@ kj::Own edit_headers(const kj::HttpHeaderTable& table, kj::Own new_rewriting_response(kj::HttpService::Response& inner, const kj::HttpHeaderTable& table, ::rust::Slice edits); +kj::Own new_image_filter_response(kj::HttpService::Response& inner); +kj::Promise image_filter_finish(kj::HttpService::Response& response); kj::Own new_null_connect_response(); kj::Own new_jsgify_websocket_errors(); kj::Own new_udp_connect_event( diff --git a/src/workerd/server/server/BUILD.bazel b/src/workerd/server/server/BUILD.bazel index ec937c17432..dc618ab5762 100644 --- a/src/workerd/server/server/BUILD.bazel +++ b/src/workerd/server/server/BUILD.bazel @@ -29,6 +29,7 @@ wd_rust_crate( ], visibility = ["//visibility:public"], deps = [ + "//src/rust/assets-router", "//src/rust/cxx/kj-hyper", "//src/rust/cxx/kj-rs-io", "//src/rust/cxx/kj-rs-tokio", diff --git a/src/workerd/server/server/bridge.rs b/src/workerd/server/server/bridge.rs index 5b64b1fa872..79b9f533372 100644 --- a/src/workerd/server/server/bridge.rs +++ b/src/workerd/server/server/bridge.rs @@ -12,9 +12,11 @@ #![allow( unsafe_code, - reason = "holds the cxx bridge, which expands to unsafe FFI glue, and `edit_headers` and `rewriting_response`, the safe wrappers over the bridge's `unsafe fn`s of those names" + reason = "holds the cxx bridge, which expands to unsafe FFI glue, and `edit_headers`, `rewriting_response` and `image_filter_response`, the safe wrappers over the bridge's `unsafe fn`s of those names" )] +pub use assets_router::mitigations::should_block_image_response; + pub use crate::actor::MovedHook; pub use crate::channels::AbortReason; pub use crate::channels::ActorClassChannel; @@ -380,6 +382,14 @@ pub mod ffi { params: &[u64], ) -> Result>; + /// Whether the config's assets service `services[config_service]` prefers serving an + /// asset to a navigation (`assetsSecFetchModeNavigateHeaderPrefersAssetServing`), under + /// its user worker's compatibility flags, or, with none, its own. + fn factory_assets_navigation_prefers_asset_serving( + factory: &WorkerFactory, + config_service: u32, + ) -> Result; + /// A compiled worker: the isolate, script and `Worker` plus what was learned compiling /// them. type CompiledWorker; @@ -715,6 +725,15 @@ pub mod ffi { table: &HttpHeaderTable, edits: &[HeaderEdit], ) -> Result>; + /// A response that sends the user Worker's answer to a request for a remote image only + /// if `should_block_image_response` lets it through, else a 403 whose body + /// `image_filter_finish` writes. Borrows `inner` for its life. + unsafe fn new_image_filter_response<'a>( + inner: Pin<&'a mut HttpServiceResponse>, + ) -> KjOwn; + async unsafe fn image_filter_finish<'a>( + response: Pin<&'a mut HttpServiceResponse>, + ) -> Result<()>; /// The `ConnectResponse` of a raw TCP socket's `connect()`: accepting does nothing, /// rejecting discards the body. fn new_null_connect_response() -> KjOwn; @@ -1042,6 +1061,9 @@ pub mod ffi { /// One UDP flow (every datagram to and from one peer, until idle), as the /// `workerd::DatagramChannel` of a `UdpConnectCustomEvent`. type UdpFlow; + /// Whether the user Worker's answer to a request for a remote image is blocked. + fn should_block_image_response(status: u16, content_type: &[u8]) -> bool; + /// The next datagram, or `ended`. One call at a time. async unsafe fn receive<'a>(self: &'a UdpFlow) -> Result; async unsafe fn send<'a>(self: &'a UdpFlow, datagram: &'a [u8]) -> Result<()>; @@ -1112,6 +1134,33 @@ pub fn edit_headers<'a>( }) } +/// See `ffi::new_image_filter_response`. +pub struct ImageFilterResponse<'a>(RewritingResponse<'a>); + +impl ImageFilterResponse<'_> { + pub fn as_mut(&mut self) -> std::pin::Pin<&mut ffi::HttpServiceResponse> { + self.0.as_mut() + } + + /// Writes the body of the 403, if the response was blocked. + pub async fn finish(&mut self) -> crate::Result<()> { + // SAFETY: the response outlives the future, which borrows it. + Ok(unsafe { ffi::image_filter_finish(self.as_mut()) }.await?) + } +} + +/// See `ffi::new_image_filter_response`. +pub fn image_filter_response( + inner: std::pin::Pin<&mut ffi::HttpServiceResponse>, +) -> ImageFilterResponse<'_> { + // SAFETY: the returned wrapper carries `inner`'s lifetime, so it cannot outlive it. + let response = unsafe { ffi::new_image_filter_response(inner) }; + ImageFilterResponse(RewritingResponse { + response, + _inner: std::marker::PhantomData, + }) +} + /// See `ffi::exception_throw`. impl From<&ffi::Exception> for crate::Error { fn from(exception: &ffi::Exception) -> Self { diff --git a/src/workerd/server/server/config.rs b/src/workerd/server/server/config.rs index eb27593debd..1d2cda8f1bf 100644 --- a/src/workerd/server/server/config.rs +++ b/src/workerd/server/server/config.rs @@ -58,6 +58,7 @@ use crate::cluster::ClusterNode; use crate::listen::loopback::Loopback; use crate::loader::WorkerLoaderNamespace; use crate::services::DiskDirectoryService; +use crate::services::assets::Assets; use crate::worker::LinkedChannels; use crate::worker::WorkerService; @@ -104,6 +105,14 @@ impl Factory { ffi::factory_experimental(&self.inner) } + /// The navigation flag of the assets service `services[service]`. + pub fn assets_navigation_prefers_asset_serving(&self, service: u32) -> Result { + Ok(ffi::factory_assets_navigation_prefers_asset_serving( + &self.inner, + service, + )?) + } + /// Whether requests get tail workers: not under `workerd bench`, unless given `--trace`. #[must_use] pub fn tail_workers(&self) -> bool { @@ -417,6 +426,7 @@ pub enum Service { /// the rest of the config can be checked. Leaf(Rc), Disk(Rc), + Assets(Rc), } impl Service { @@ -427,6 +437,7 @@ impl Service { Self::Worker(worker) => worker.default_entrypoint(), Self::Leaf(channel) => Rc::clone(channel), Self::Disk(disk) => Rc::clone(disk) as Rc, + Self::Assets(assets) => assets.channel(false), } } @@ -1260,6 +1271,11 @@ pub fn lookup_service( }; let props = configured_props(props_json.as_deref()); + if let Service::Assets(assets) = service + && entrypoint.as_deref().is_none_or(|e| e == "AssetWorker") + { + return assets.channel(entrypoint.is_some()); + } let Service::Worker(worker) = service else { if let Some(entrypoint) = entrypoint { report.reference_error( @@ -1499,6 +1515,11 @@ impl Server { for (name, worker) in server.workers() { tolerate_deployment(&server.report, name, server.link_worker(name, worker))?; } + for (name, service) in &server.services.0 { + if let Service::Assets(assets) = service { + assets.link(|d| lookup_service(&server.services.0, &server.report, Some(name), d)); + } + } // A deployment's service with errors fails every request. for (name, worker) in server.workers() { @@ -1796,6 +1817,23 @@ async fn make_service( .map(Service::Disk), )) } + Ok(service::Which::Assets(assets)) => { + if !factory.experimental() { + report.error_in( + name, + "Assets services are an experimental feature which may change or go away in \ + the future. You must run workerd with `--experimental` to use this feature.", + ); + return Ok(Service::Leaf(Rc::new(InvalidConfigChannel))); + } + let assets = assets.map_err(capnp_error)?; + Ok(invalid_on_error( + name, + report, + crate::services::assets::make_assets_service(name, index, assets, factory, report) + .map(Service::Assets), + )) + } // Made once the cluster is joined (`make_bucket_services`). Ok(service::Which::Bucket(_)) => Ok(Service::Leaf(Rc::new(InvalidConfigChannel))), Err(capnp::NotInSchema(_)) => { diff --git a/src/workerd/server/server/services/assets/mod.rs b/src/workerd/server/server/services/assets/mod.rs new file mode 100644 index 00000000000..e63f851377b --- /dev/null +++ b/src/workerd/server/server/services/assets/mod.rs @@ -0,0 +1,380 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +//! The `assets` service: a directory of static assets, served as Cloudflare serves a Worker's +//! `assets`. +//! +//! `assets_router` decides everything; this module does the I/O: it stats the files a request looks +//! at, streams the one it serves, and forwards to the user Worker. Nothing is cached, so a +//! change to the directory is seen at once. + +use std::cell::OnceCell; +use std::fs::Metadata; +use std::hash::DefaultHasher; +use std::hash::Hash; +use std::hash::Hasher; +use std::path::PathBuf; +use std::pin::Pin; +use std::rc::Rc; +use std::time::SystemTime; + +use assets_router::Config; +use assets_router::Deployment; +use assets_router::Request; +use assets_router::Response; +use assets_router::ResponseHeader; +use assets_router::decide::Route; +use assets_router::decide::RuleRoute; +use assets_router::mitigations::is_cdn_cgi_backslash_bypass; +use assets_router::serve::Conditions; +use kj::http::ConnectResponse; +use kj::http::ConnectSettings; +use kj::http::HeadersRef; +use kj::http::Method; +use kj::http::Service; +use kj::http::ServiceResponse; +use kj::io::AsyncInputStream; +use kj::io::AsyncIoStream; +use kj::io::AsyncOutputStream; +use kj_hyper::io_kj_error; +use kj_rs::KjOwn; +use worker::AlarmResult; +use worker::CustomEvent; +use worker::CustomEventResult; +use worker::CxxWorkerInterface; +use worker::Interface; +use worker::ScheduledResult; +use workerd_capnp::assets_service; + +use crate::Result; +use crate::bindings::Designator; +use crate::bridge::image_filter_response; +use crate::channels::Channel; +use crate::channels::PendingToken; +use crate::channels::RequestMetadata; +use crate::channels::TokenUsage; +use crate::channels::WorkerInterface; +use crate::config::Factory; +use crate::config::Reporter; +use crate::config::capnp_error; +use crate::config::text; +use crate::services::disk::copy_file; +use crate::services::disk::header_value; +use crate::services::disk::is_entry_name; +use crate::services::disk::kj_headers; +use crate::services::header_table; +use crate::services::not_transferable; +use crate::services::unsupported; + +/// An `assets` service. +pub struct Assets { + factory: Rc, + root: PathBuf, + config: Config, + user_worker: Option, + user: OnceCell>, +} + +impl Assets { + /// The file at `path`, relative to the directory, if it is an asset. + fn stat(&self, path: &str) -> Option<(PathBuf, Metadata)> { + let relative = path.strip_prefix('/')?; + if !relative.split('/').all(is_entry_name) || self.config.ignored.ignored(path) { + return None; + } + let path = self.root.join(relative); + let metadata = std::fs::metadata(&path).ok().filter(Metadata::is_file)?; + Some((path, metadata)) + } + + /// Sends a response's head, with a `Content-Length` of `length`. + fn send<'a>( + &self, + response: ServiceResponse<'a>, + status: u16, + headers: &[ResponseHeader], + length: u64, + ) -> Result> { + let mut map = http::HeaderMap::new(); + for ResponseHeader { name, value } in headers { + let name = http::HeaderName::from_bytes(name.as_bytes()) + .map_err(|_| kj::failed!("invalid header name: {name}"))?; + map.append(name, header_value(value)?); + } + map.insert(http::header::CONTENT_LENGTH, length.into()); + let headers = kj_headers(header_table(&self.factory), &map)?; + let text = http::StatusCode::from_u16(status) + .ok() + .and_then(|s| s.canonical_reason()); + let headers = HeadersRef::from(&*headers); + response.send( + status.into(), + text.unwrap_or_default(), + headers, + Some(length), + ) + } +} + +/// A hash of `path`, the size, and the modification time to the nanosecond. With the path in +/// it, no two files share an `ETag`, which the router takes as the same asset. +fn etag(path: &str, metadata: &Metadata) -> String { + let nanos = metadata + .modified() + .ok() + .and_then(|modified| modified.duration_since(SystemTime::UNIX_EPOCH).ok()) + .map_or(0, |d| d.as_nanos()); + let mut hasher = DefaultHasher::new(); + path.hash(&mut hasher); + format!("{:x}-{:x}-{nanos:x}", hasher.finish(), metadata.len()) +} + +impl Assets { + /// The router, or with `serve_only` the channel that serves files only. + pub fn channel(self: &Rc, serve_only: bool) -> Rc { + let assets = Rc::clone(self); + Rc::new(AssetsChannel { assets, serve_only }) + } + + /// Resolves the user Worker, once every service exists. + pub fn link(&self, lookup: impl FnOnce(&Designator) -> Rc) { + if let Some(designator) = &self.user_worker { + self.user.get_or_init(|| lookup(designator)); + } + } +} + +pub fn make_assets_service( + name: &str, + index: u32, + conf: assets_service::Reader<'_>, + factory: &Rc, + report: &Reporter, +) -> Result> { + let path = text(conf.get_path())?; + let root = std::env::current_dir() + .map_err(|e| io_kj_error(&e))? + .join(&path); + if !root.is_dir() { + return Err(kj::failed!("Directory named \"{name}\" not found: {path}")); + } + let mut errors = Vec::new(); + let user_worker = if conf.has_user_worker() { + let reader = conf.get_user_worker().map_err(capnp_error)?; + let context = format!("Assets service \"{name}\"'s userWorker"); + Some(Designator::from_reader(reader, context, &mut errors).map_err(capnp_error)?) + } else { + None + }; + for error in errors { + report.service_error(name, error); + } + if user_worker.is_some() && (conf.has_compatibility_date() || conf.has_compatibility_flags()) { + return Err(kj::failed!( + "Assets service \"{name}\" takes its user Worker's compatibility date and flags" + )); + } + let deployment = Deployment { + has_user_worker: user_worker.is_some(), + navigation_prefers_asset_serving: factory.assets_navigation_prefers_asset_serving(index)?, + }; + let json = Some(text(conf.get_config())?).filter(|json| !json.is_empty()); + let options = serde_json::from_str(json.as_deref().unwrap_or("{}")) + .map_err(|e| kj::failed!("Assets service \"{name}\"'s config: {e}"))?; + let warn = &mut |warning| report.service_warning(name, warning); + let config = Config::new(options, deployment, warn).map_err(|e| kj::failed!("{e}"))?; + Ok(Rc::new(Assets { + factory: Rc::clone(factory), + root, + config, + user_worker, + user: OnceCell::new(), + })) +} + +struct AssetsChannel { + assets: Rc, + serve_only: bool, +} + +impl Channel for AssetsChannel { + fn start_request(&self, metadata: KjOwn) -> Result> { + let user = self.assets.user.get().filter(|_| !self.serve_only); + Ok(AssetsRequest { + assets: Rc::clone(&self.assets), + user: user.map(|user| (Rc::clone(user), metadata)), + started: None, + serve_only: self.serve_only, + } + .into_kj()) + } + + fn token(&self, _usage: TokenUsage) -> Result> { + Err(not_transferable("Assets services")) + } +} + +struct AssetsRequest { + assets: Rc, + /// The user Worker, and the metadata to start the request on it with, + /// until `started` on it. + user: Option<(Rc, KjOwn)>, + started: Option, + serve_only: bool, +} + +impl AssetsRequest { + fn user(&mut self) -> Result<&mut CxxWorkerInterface> { + if let Some((user, metadata)) = self.user.take() { + self.started = Some(CxxWorkerInterface::new(user.start_request(metadata)?)); + } + self.started + .as_mut() + .ok_or_else(|| unsupported("Assets services")) + } +} + +#[async_trait::async_trait(?Send)] +impl Service for AssetsRequest { + async fn request<'a>( + &'a mut self, + method: Method, + url: &'a [u8], + headers: HeadersRef<'a>, + request_body: Pin<&'a mut AsyncInputStream>, + response: ServiceResponse<'a>, + ) -> Result<()> { + let assets = Rc::clone(&self.assets); + let raw = std::str::from_utf8(url).map_err(|_| kj::failed!("request URL is not UTF-8"))?; + let parsed = url::Url::parse(raw).map_err(|e| kj::failed!("{e}: {raw}"))?; + let header = |name| { + headers + .get_by_name(name) + .and_then(|v| std::str::from_utf8(v).ok()) + }; + let request = Request { + url: &parsed, + method: match method { + Method::GET => assets_router::Method::Get, + Method::HEAD => assets_router::Method::Head, + _ => assets_router::Method::Other, + }, + sec_fetch_mode: header("sec-fetch-mode"), + sec_fetch_dest: header("sec-fetch-dest"), + }; + let routing = &assets.config.routing; + let answer = if !self.serve_only && is_cdn_cgi_backslash_bypass(raw, &parsed) { + Response::new( + 307, + vec![assets_router::header("Location", parsed.as_str())], + ) + } else { + let route = if self.serve_only { + RuleRoute::AssetWorker + } else { + routing.rule_route(parsed.path()) + }; + let exists = |path: &str| assets.stat(path).map(|(_, metadata)| etag(path, &metadata)); + match routing + .route(route, &request, &exists) + .map_err(|e| kj::failed!("{e}"))? + { + Route::UserWorker { + block_image_href: true, + .. + } => Response::blocked(), + Route::UserWorker { + filter_image_response, + .. + } => { + let user = self.user()?; + if !filter_image_response { + return user + .request(method, url, headers, request_body, response) + .await; + } + let mut filter = image_filter_response(response.into_ffi()); + let response = ServiceResponse::from(filter.as_mut()); + user.request(method, url, headers, request_body, response) + .await?; + return filter.finish().await; + } + Route::Respond(answer) => answer, + // The file went away since routing found it. + Route::Asset(asset) => match assets.stat(&asset.path) { + None => Response::new(404, routing.custom_headers(&request, vec![])), + Some((path, metadata)) => { + let conditions = Conditions { + if_none_match: header("if-none-match"), + private: header("authorization").is_some() || header("range").is_some(), + }; + let served = assets.config.serve( + &request, + conditions, + &asset, + &etag(&asset.path, &metadata), + ); + // A HEAD gets the length a GET would; a 304 has no body. + let length = if served.status == 304 { + 0 + } else { + metadata.len() + }; + let mut out = + assets.send(response, served.status, &served.headers, length)?; + if served.body { + copy_file(&path, 0, length, &mut out).await?; + } + return Ok(()); + } + }, + } + }; + let length = answer.body.len() as u64; + let mut out = assets.send(response, answer.status, &answer.headers, length)?; + out.write(answer.body.as_bytes()).await + } + + async fn connect<'a>( + &'a mut self, + host: &'a [u8], + headers: HeadersRef<'a>, + connection: Pin<&'a mut AsyncIoStream>, + response: ConnectResponse<'a>, + settings: ConnectSettings<'a>, + ) -> Result<()> { + self.user()? + .connect(host, headers, connection, response, settings) + .await + } +} + +#[async_trait::async_trait(?Send)] +impl Interface for AssetsRequest { + async fn prewarm(&mut self, url: &str) -> Result<()> { + match self.user() { + Ok(user) => user.prewarm(url).await, + Err(_) => Ok(()), + } + } + + async fn run_scheduled(&mut self, time: &SystemTime, cron: &str) -> Result { + self.user()?.run_scheduled(time, cron).await + } + + async fn run_alarm(&mut self, time: &SystemTime, retry_count: u32) -> Result { + self.user()?.run_alarm(time, retry_count).await + } + + async fn test(&mut self) -> Result { + self.user()?.test().await + } + + async fn custom_event(&mut self, event: KjOwn) -> Result { + match self.user() { + Ok(user) => user.custom_event(event).await, + Err(_) => worker::not_supported(event).await, + } + } +} diff --git a/src/workerd/server/server/services/disk.rs b/src/workerd/server/server/services/disk.rs index 6639e53507e..b9173888e12 100644 --- a/src/workerd/server/server/services/disk.rs +++ b/src/workerd/server/server/services/disk.rs @@ -61,7 +61,7 @@ use crate::services::unsupported; /// Whether `segment` names one entry of a directory: a single normal path component, so nothing /// a separator would split, no `.`, `..`, root or drive, and (on Windows) no stream of a file. -fn is_entry_name(segment: &str) -> bool { +pub(crate) fn is_entry_name(segment: &str) -> bool { let mut components = Path::new(segment).components(); matches!(components.next(), Some(Component::Normal(name)) if name == segment) && components.next().is_none() @@ -239,14 +239,14 @@ pub fn make_disk_directory_service( // ======================================================================================= // Requests -fn kj_headers<'t>( +pub(crate) fn kj_headers<'t>( table: &'t HeaderTable, map: &http::HeaderMap, ) -> Result> { kj_hyper::HeaderBlock::new(map, &http::Extensions::new()).to_kj(table) } -fn header_value(value: impl AsRef) -> Result { +pub(crate) fn header_value(value: impl AsRef) -> Result { http::HeaderValue::from_str(value.as_ref()) .map_err(|_| kj::failed!("invalid header value: {}", value.as_ref())) } @@ -256,7 +256,7 @@ fn len_u64(len: usize) -> u64 { } /// Copies `length` bytes of the file from `start` into `out`. -async fn copy_file( +pub(crate) async fn copy_file( path: &Path, start: u64, length: u64, diff --git a/src/workerd/server/server/services/mod.rs b/src/workerd/server/server/services/mod.rs index 22ef8422391..f7665b4a2c3 100644 --- a/src/workerd/server/server/services/mod.rs +++ b/src/workerd/server/server/services/mod.rs @@ -10,6 +10,7 @@ //! whole life, so connections are reused across events, and lends it to a `Client` per event, //! which borrows the factory's header table. +pub mod assets; pub mod bucket; pub mod disk; pub mod network; diff --git a/src/workerd/server/tests/assets/BUILD.bazel b/src/workerd/server/tests/assets/BUILD.bazel new file mode 100644 index 00000000000..b0e301f435d --- /dev/null +++ b/src/workerd/server/tests/assets/BUILD.bazel @@ -0,0 +1,18 @@ +load("@aspect_rules_js//js:defs.bzl", "js_test") +load("//:build/wd_test.bzl", "wd_test") + +# `workerd serve` of a wrangler.jsonc's assets. +js_test( + name = "assets-wrangler-test", + data = ["//src/workerd/server:workerd"], + entry_point = "assets-wrangler-test.mjs", + env = {"WORKERD_BINARY": "$(rootpath //src/workerd/server:workerd)"}, + tags = ["js-test"], +) + +# The events an assets service passes to its user Worker. +wd_test( + src = "assets-events-test.wd-test", + args = ["--experimental"], + data = glob(["assets-events-*.js"]), +) diff --git a/src/workerd/server/tests/assets/assets-events-app.js b/src/workerd/server/tests/assets/assets-events-app.js new file mode 100644 index 00000000000..5cfcb5e9bcd --- /dev/null +++ b/src/workerd/server/tests/assets/assets-events-app.js @@ -0,0 +1,37 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +// The user Worker behind an assets service. Each event it gets settles a promise the test awaits +// over RPC. +import { WorkerEntrypoint } from 'cloudflare:workers'; + +let onScheduled, onTail; +const scheduled = new Promise((resolve) => (onScheduled = resolve)); +const tailed = new Promise((resolve) => (onTail = resolve)); + +export default class extends WorkerEntrypoint { + fetch() { + return new Response('from the user Worker'); + } + + scheduled(controller) { + onScheduled(controller.cron); + } + + tail(events) { + onTail(events.flatMap((event) => event.logs.map((log) => log.message[0]))); + } + + hello(name) { + return `hello, ${name}`; + } + + scheduledCron() { + return scheduled; + } + + tailedLogs() { + return tailed; + } +} diff --git a/src/workerd/server/tests/assets/assets-events-producer.js b/src/workerd/server/tests/assets/assets-events-producer.js new file mode 100644 index 00000000000..bce87dc0f12 --- /dev/null +++ b/src/workerd/server/tests/assets/assets-events-producer.js @@ -0,0 +1,11 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +// A Worker whose tail is the assets service. +export default { + fetch() { + console.log('produced'); + return new Response('ok'); + }, +}; diff --git a/src/workerd/server/tests/assets/assets-events-test.js b/src/workerd/server/tests/assets/assets-events-test.js new file mode 100644 index 00000000000..593d8c65b54 --- /dev/null +++ b/src/workerd/server/tests/assets/assets-events-test.js @@ -0,0 +1,43 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 + +import * as assert from 'node:assert'; + +export const rpc = { + async test(ctrl, env) { + assert.strictEqual(await env.ROUTER.hello('router'), 'hello, router'); + }, +}; + +export const scheduled = { + async test(ctrl, env) { + const result = await env.ROUTER.scheduled({ cron: '* * * * 1' }); + assert.strictEqual(result.outcome, 'ok'); + assert.strictEqual(await env.ROUTER.scheduledCron(), '* * * * 1'); + }, +}; + +export const tail = { + async test(ctrl, env) { + const response = await env.PRODUCER.fetch('http://producer/'); + assert.strictEqual(await response.text(), 'ok'); + assert.deepStrictEqual(await env.ROUTER.tailedLogs(), ['produced']); + }, +}; + +export const fetchRoutes = { + async test(ctrl, env) { + const missing = await env.ROUTER.fetch('http://app/missing'); + assert.strictEqual(await missing.text(), 'from the user Worker'); + }, +}; + +export const serveOnlyDoesNotForward = { + async test(ctrl, env) { + const missing = await env.ASSETS.fetch('http://app/missing'); + assert.strictEqual(missing.status, 404); + await assert.rejects(env.ASSETS.hello('assets')); + await assert.rejects(env.ASSETS.scheduled({ cron: '* * * * 1' })); + }, +}; diff --git a/src/workerd/server/tests/assets/assets-events-test.wd-test b/src/workerd/server/tests/assets/assets-events-test.wd-test new file mode 100644 index 00000000000..b5b08322e58 --- /dev/null +++ b/src/workerd/server/tests/assets/assets-events-test.wd-test @@ -0,0 +1,38 @@ +using Workerd = import "/workerd/workerd.capnp"; + +# An assets service with a user Worker: the router passes JS RPC, scheduled and tail events to the +# user Worker; the `AssetWorker` entrypoint serves files only. +const unitTests :Workerd.Config = ( + services = [ + ( name = "test", + worker = ( + modules = [(name = "worker", esModule = embed "assets-events-test.js")], + compatibilityFlags = ["nodejs_compat", "rpc", "service_binding_extra_handlers"], + bindings = [ + (name = "ROUTER", service = "app-assets"), + (name = "ASSETS", service = (name = "app-assets", entrypoint = "AssetWorker")), + (name = "PRODUCER", service = "producer"), + ], + ), + ), + ( name = "app-assets", + assets = ( + path = ".", + userWorker = (name = "app"), + ), + ), + ( name = "app", + worker = ( + modules = [(name = "worker", esModule = embed "assets-events-app.js")], + compatibilityFlags = ["nodejs_compat", "rpc"], + ), + ), + ( name = "producer", + worker = ( + modules = [(name = "worker", esModule = embed "assets-events-producer.js")], + compatibilityFlags = ["nodejs_compat"], + tails = ["app-assets"], + ), + ), + ], +); diff --git a/src/workerd/server/tests/assets/assets-wrangler-test.mjs b/src/workerd/server/tests/assets/assets-wrangler-test.mjs new file mode 100644 index 00000000000..4aa86d7b402 --- /dev/null +++ b/src/workerd/server/tests/assets/assets-wrangler-test.mjs @@ -0,0 +1,262 @@ +// Copyright (c) 2026 Cloudflare, Inc. +// Licensed under the Apache 2.0 license found in the LICENSE file or at: +// https://opensource.org/licenses/Apache-2.0 +// +// `workerd serve` of a wrangler.jsonc with assets: the router sends each +// request to a file or to the Worker, `_redirects`, `_headers` and +// `.assetsignore` apply, `env.ASSETS` serves files, a project without +// `main` serves only files, and without --experimental the server refuses +// to start and says why. + +import { spawn } from 'node:child_process'; +import { request } from 'node:http'; +import { mkdir, mkdtemp, stat, utimes, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { env } from 'node:process'; +import { test } from 'node:test'; +import assert from 'node:assert'; + +const WORKER = `export default { + async fetch(request, env) { + const url = new URL(request.url); + if (url.pathname === '/via-binding') { + return env.ASSETS.fetch(new URL('/about', url)); + } + return new Response('worker ' + url.pathname); + }, +}; +`; + +const FILES = { + 'index.html': '

home

', + 'about.html': '

about

', + 'blog/index.html': '

post

', + 'style.css': 'body {}', + '404.html': '

missing

', + '.well-known/security.txt': 'contact', + 'secret/key.txt': 'hidden', + _redirects: '/old /about 301\n/proxy /about 200\n', + _headers: '/*\n X-Site: yes\n', + '.assetsignore': 'secret/\n', +}; + +const CONTROL_FD = 3; + +// Runs workerd on `dir`'s wrangler.jsonc until it listens, or until it +// exits. Answers {port, stop} or {code, stderr}. +function serve(dir, args) { + const child = spawn( + env.WORKERD_BINARY, + [ + 'serve', + join(dir, 'wrangler.jsonc'), + `--control-fd=${CONTROL_FD}`, + '--socket-addr=http=127.0.0.1:0', + ...args, + ], + { stdio: ['ignore', 'inherit', 'pipe', 'pipe'] } + ); + let stderr = ''; + child.stderr.on('data', (d) => { + stderr += d; + process.stderr.write(d); + }); + // `close`, unlike `exit`, waits for stderr to be drained. + const exited = new Promise((resolve) => child.once('close', resolve)); + return new Promise((resolve, reject) => { + let buffer = ''; + child.stdio[CONTROL_FD].on('data', (d) => { + buffer += d; + for (const line of buffer.split('\n').slice(0, -1)) { + const m = JSON.parse(line); + if (m.event === 'listen') { + const stop = () => (child.kill(), exited); + resolve({ port: m.port, stop }); + } + } + buffer = buffer.slice(buffer.lastIndexOf('\n') + 1); + }); + child.once('error', reject); + exited.then((code) => resolve({ code, stderr })); + }); +} + +async function project(wrangler, worker) { + const dir = await mkdtemp(join(env.TEST_TMPDIR ?? tmpdir(), 'assets-')); + await writeFile(join(dir, 'wrangler.jsonc'), JSON.stringify(wrangler)); + if (worker) await writeFile(join(dir, 'index.js'), worker); + for (const [path, text] of Object.entries(FILES)) { + await mkdir(dirname(join(dir, 'public', path)), { recursive: true }); + await writeFile(join(dir, 'public', path), text); + } + return dir; +} + +const get = (port, path, init) => + fetch(`http://127.0.0.1:${port}${path}`, { redirect: 'manual', ...init }); + +test('assets with a Worker', async (t) => { + const dir = await project( + { + name: 'site', + main: 'index.js', + compatibility_date: '2026-01-01', + no_bundle: true, + assets: { + directory: 'public', + binding: 'ASSETS', + not_found_handling: '404-page', + }, + }, + WORKER + ); + const refused = await serve(dir, []); + assert.notStrictEqual(refused.code, 0); + assert.match(refused.stderr, /assets.*--experimental/s); + + const server = await serve(dir, ['--experimental']); + t.after(server.stop); + const expect = async (path, status, body, init) => { + const r = await get(server.port, path, init); + assert.strictEqual(r.status, status, path); + if (typeof body === 'string') + assert.strictEqual(await r.text(), body, path); + return r; + }; + const home = await expect('/', 200, FILES['index.html']); + assert.strictEqual( + home.headers.get('content-type'), + 'text/html; charset=utf-8' + ); + assert.strictEqual(home.headers.get('x-site'), 'yes'); + assert.strictEqual( + home.headers.get('cache-control'), + 'public, max-age=0, must-revalidate' + ); + await expect('/about', 200, FILES['about.html']); + const html = await expect('/about.html', 307); + assert.strictEqual(html.headers.get('location'), '/about'); + await expect('/blog/', 200, FILES['blog/index.html']); + assert.strictEqual( + (await expect('/blog', 307)).headers.get('location'), + '/blog/' + ); + const css = await expect('/style.css', 200, FILES['style.css']); + assert.strictEqual( + css.headers.get('content-type'), + 'text/css; charset=utf-8' + ); + const etag = css.headers.get('etag'); + await expect('/style.css', 304, '', { headers: { 'If-None-Match': etag } }); + const head = await expect('/style.css', 200, '', { method: 'HEAD' }); + assert.strictEqual( + head.headers.get('content-length'), + String(FILES['style.css'].length) + ); + await expect('/.well-known/security.txt', 200, 'contact'); + + // No file: the Worker, also for what is ignored; but not for a + // navigation, which gets the 404 page. + await expect('/missing', 200, 'worker /missing'); + // Not with fetch, which sets its own Sec-Fetch-Mode. + const navigation = await new Promise((resolve, reject) => { + const headers = { 'Sec-Fetch-Mode': 'navigate' }; + const url = `http://127.0.0.1:${server.port}/missing`; + request(url, { headers }, (r) => { + let body = ''; + r.on('data', (d) => (body += d)); + r.on('end', () => resolve({ status: r.statusCode, body })); + }) + .on('error', reject) + .end(); + }); + assert.deepStrictEqual(navigation, { status: 404, body: FILES['404.html'] }); + await expect('/secret/key.txt', 200, 'worker /secret/key.txt'); + // An image optimizer fetching another host is refused before the Worker. + await expect('/_image?href=//evil.com/x', 403, 'Blocked'); + for (const file of ['_headers', '_redirects', '.assetsignore']) { + await expect(`/${file}`, 200, `worker /${file}`); + } + + assert.strictEqual( + (await expect('/old', 301)).headers.get('location'), + '/about' + ); + await expect('/proxy', 200, FILES['about.html']); + await expect('/via-binding', 200, FILES['about.html']); + await expect('/about', 405, '', { method: 'POST' }); + + // A file appears at once. + await writeFile(join(dir, 'public', 'new.txt'), 'new'); + await expect('/new.txt', 200, 'new'); + // As does an edit of the same size within the same second. + const path = join(dir, 'public', 'style.css'); + const { mtimeMs } = await stat(path); + await writeFile(path, 'body{} '); + const second = Math.floor(mtimeMs / 1000) * 1000; + const mtime = second + (mtimeMs - second === 500 ? 250 : 500); + await utimes(path, new Date(), new Date(mtime)); + const edited = await expect('/style.css', 200, 'body{} ', { + headers: { 'If-None-Match': etag }, + }); + assert.notStrictEqual(edited.headers.get('etag'), etag); +}); + +test('run_worker_first', async (t) => { + const wrangler = (run_worker_first) => ({ + main: 'index.js', + compatibility_date: '2026-01-01', + no_bundle: true, + assets: { directory: 'public', run_worker_first }, + }); + for (const [first, css, about] of [ + [['/*.css'], 'worker /style.css', FILES['about.html']], + [['/*', '!/about'], 'worker /style.css', FILES['about.html']], + [true, 'worker /style.css', 'worker /about'], + ]) { + const server = await serve(await project(wrangler(first), WORKER), [ + '--experimental', + ]); + t.after(server.stop); + const text = async (path) => (await get(server.port, path)).text(); + assert.strictEqual(await text('/style.css'), css); + assert.strictEqual(await text('/about'), about); + } +}); + +test('assets without a Worker', async () => { + const dir = await project({ + compatibility_date: '2026-01-01', + assets: { + directory: 'public', + not_found_handling: 'single-page-application', + }, + }); + const server = await serve(dir, ['--experimental']); + const r = await get(server.port, '/no/such/page'); + assert.strictEqual(r.status, 200); + assert.strictEqual(await r.text(), FILES['index.html']); + await server.stop(); + + const bad = await project( + { + main: 'index.js', + compatibility_date: '2026-01-01', + no_bundle: true, + assets: { directory: 'public', run_worker_first: ['api/*'] }, + }, + WORKER + ); + const failed = await serve(bad, ['--experimental']); + assert.notStrictEqual(failed.code, 0); + assert.match(failed.stderr, /rules must start with '\/'/); + + const missing = await project({ + compatibility_date: '2026-01-01', + assets: { directory: 'nowhere' }, + }); + const refused = await serve(missing, ['--experimental']); + assert.notStrictEqual(refused.code, 0); + assert.match(refused.stderr, /assets\.directory: .*nowhere: not a directory/); +}); diff --git a/src/workerd/server/workerd.capnp b/src/workerd/server/workerd.capnp index 28440d1e03d..a8e656ec83a 100644 --- a/src/workerd/server/workerd.capnp +++ b/src/workerd/server/workerd.capnp @@ -449,6 +449,9 @@ struct Service { bucket @6 :ClusterBucket; # A directory in the cluster's bucket, served over HTTP. Requires `Config.cluster`. + + assets @7 :AssetsService; + # A directory of static assets, served as Cloudflare serves a Worker's `assets`. } # TODO(someday): Allow defining a list of middlewares to stack on top of the service. This would @@ -468,6 +471,37 @@ struct ClusterBucket { # The directory below `services/`: one or more `/`-separated names, none `.` or `..`. } +struct AssetsService { + # Static assets, as Cloudflare serves a Worker's `assets` (wrangler's `assets` config): each + # request goes to the user Worker or to a file of the directory, as `run_worker_first` and the + # files ask. Files are looked up as each request arrives, so a change to the directory is seen + # at once. A directory is never an asset, nor is a file `.assetsignore` ignores, which by + # default are `_headers`, `_redirects` and `.assetsignore` themselves. Experimental: requires + # `--experimental`. + # + # A designator with no entrypoint names the router; one with the entrypoint `AssetWorker` + # serves files only, and is what a Worker's assets binding names. + + path @0 :Text; + # The directory. Relative paths are interpreted relative to the current directory, as a + # `disk` service's are. + + config @1 :Text; + # JSON, every field optional: `html_handling`, `not_found_handling`, `base_path` and + # `run_worker_first`, as in wrangler's `assets`; `headers`, `redirects` and `ignore`, the text + # of the directory's `_headers`, `_redirects` and `.assetsignore`; `debug`, for Miniflare, which + # logs and removes the header it adds to a navigation that `not_found_handling` answered. + + userWorker @2 :ServiceDesignator; + # The user Worker, if any. Events other than HTTP requests go to it, and its compatibility flags + # select assets behavior, as they do on Cloudflare. + + compatibilityDate @3 :Text; + compatibilityFlags @4 :List(Text); + # Without a user Worker, the compatibility date and flags that select assets behavior, compiled + # as a Worker's are. +} + struct ServiceDesignator { # A reference to a service from elsewhere in the config file, e.g. from a service binding in a # Worker.