diff --git a/.gitignore b/.gitignore index c8abe9a..453c60b 100644 --- a/.gitignore +++ b/.gitignore @@ -8,6 +8,8 @@ build*/ # Data /data/ +/cache/ +/results/ /input/ /output/ diff --git a/README.md b/README.md index 096c210..76fc55b 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,8 @@ and it then falls back to its default. [engine/configs/geolife.json](engine/conf | `where_at` | Probes per trajectory and max time gap (s) between points to probe | | `where_at_scoring` | Tolerance range (m) for the primary average-hit score, and the hit radius (m) for the within-radius diagnostic | | `frechet` | Fréchet variant for kNN, join and clustering: `Discrete` (kept vertices only, the default) or `Continuous` (polyline distance, unaffected by adding a vertex on an edge) | -| `clustering` | Route clustering metric: `enabled`, neighbour threshold `eps_m` (m) and `min_size` per cluster. Reuses the self join when `join.mode` is `Self` with the same eps | +| `join` | `mode`, threshold `eps_m` (m), `query_count` and `self_join`. `HeldOut` (the default) joins `query_count` original trajectories from the query split (`0` = all) against the simplified database. `Self` joins the database with itself, both sides simplified. `self_join: true` also scores the self join next to a held-out join, as a diagnostic outside the combined score. `score_eps_m` lists more thresholds: the combined score's join term then averages the normalised join at `eps_m` and at each of them | +| `clustering` | Route clustering metric: `enabled`, neighbour threshold `eps_m` (m) and `min_size` per cluster. Reuses the self join when one is scored with the same eps | | `allocation_alphas` | Allocation exponents to test: budgets follow size^α (`1` = size-proportional, `0` = equal). Exports for α ≠ 1 get an `_a<α>` suffix | ## Tests diff --git a/TEACHER_NEXT_STEPS.md b/TEACHER_NEXT_STEPS.md new file mode 100644 index 0000000..7358841 --- /dev/null +++ b/TEACHER_NEXT_STEPS.md @@ -0,0 +1,80 @@ +# Teacher: status and next steps + +Updated 2026-11-17 on branch `teacher`. The teacher is described in +[engine/src/teacher/README.md](engine/src/teacher/README.md); this file says what is settled and +what is open. + +## The teacher + +`BUDGET_LOOP`: budgets searched on value curves, the best candidate recipe per trajectory, then the +selection loop, which re-picks each trajectory's recipe against its neighbours' current compressed +state until nothing switches. kNN is scored as a ranking among the compressed database +(`teacher.ranking_knn`, the default). + +| Arm | Role | +|---|---| +| `EQUAL_CHOICE`, `EQUAL_CHOICE_RDP_PARTNERS`, `EQUAL_LOOP` | the same steps at equal budgets | +| `BUDGET_RDP` | searched budgets with RDP's points | +| `BUDGET_CHOICE` | searched budgets and recipe choice, no loop | +| `BUDGET_LOOP_REALLOCATE` | budgets re-searched after the loop, so they react to neighbours too | +| `BUDGET_LOOP_NO_UPDATE` | control: one look at the start, no reacting | +| `BUDGET_LOOP_SHUFFLED` | control: join partners and kNN neighbours relabelled | +| `RECIPE_` (baselines in `teacher_eval`) | one recipe for every trajectory | + +## Settled (Geolife, 0.25 % unless stated) + +- **Budgets per trajectory:** `BUDGET_RDP` 0.553 J4 against `RDP_a0` 0.456, every seed. +- **Recipe per trajectory:** `EQUAL_CHOICE` beats the best single recipe (`RECIPE_GREEDY_RDP2`) by + 0.014 to 0.035 J4 at 0.25 % to 1 %, every seed. +- **Relations, database self join:** trained on the self join, the loop lifts the 150 m self join + well above the start; the shuffled control falls far below it. Trained on several thresholds + (`teacher.extra_join_eps_m`) the gain holds at thresholds not trained on, at small budgets. +- **Relations, kNN on held-out queries:** the ranking loop adds 0.01 to 0.05 held-out kNN over the + same loop with the per-trajectory score, every seed and budget tested; shuffled neighbours turn + it into a loss. +- **Overall:** `BUDGET_LOOP` J4 0.600 against `RDP_a0` 0.458 at 0.25 %, 0.876 against 0.795 at 1 % + (seed 42). + +## Open + +1. **Held-out kNN and join at larger budgets.** Above 0.25 % the teacher trails `RDP_a0` on + held-out kNN, and at 1 % on the held-out join. Its kNN losses carry over to new queries while + most of its gains do not. +2. **Value curves still use the per-trajectory kNN check.** A ranking version against a fixed + state that only rescored each query's true neighbours made no difference; one that also rescores + intruders is far too slow as is. +3. **Term weights** (`teacher.term_weights`, `--term-weights`) trade range and where-at for kNN. + Seeds 17, 42 and 43 at 0.25 %: with kNN weighted 1×, 2× or 4× `BUDGET_LOOP` beats `RDP_a0` on + all four held-out terms on every seed; at 4× held-out kNN is 0.719 against 0.669, for 0.019 + J4. At 0.5 % (seeds 17 and 43) 4× brings kNN level with `RDP_a0` (0.827 against 0.828) for + 0.006 J4. At 1 % (seed 42) it barely moves kNN: there the gap comes from the candidates. +4. **Held-out join thresholds.** `teacher_eval` now scores both joins at every `--eval-eps` + threshold. Trained at 150 m only, the held-out join lead at 0.25 % (seeds 42 and 17) peaks at + 150 m (+0.10 over the best baseline) and falls to −0.01 at 100 m. Trained on 100, 125, 150, 200 + and 300 m (`teacher.extra_join_eps_m`) it is +0.07 to +0.10 at every threshold from 100 to + 300 m, including 175 and 250 m, which neither trained on; J4 −0.006. `engine/configs/geolife.json` now trains on these thresholds, and + `teacher_eval` scores both joins at 100 to 300 m by default. At 1 % it narrows the gap + to the baselines from −0.16 to −0.08 at 100 m and from −0.03 to −0.01 at 300 m, J4 +0.002. +5. **Budget moves between trajectories scored exactly** with `KnnState` and `JoinState`, beyond + what `BUDGET_LOOP_REALLOCATE` does. +6. **For a paper:** published baselines, a second dataset, the 2 % budget, more seeds. + +## Evaluation + +With `engine/configs/geolife.json`, J4's join term is the held-out join averaged over 100, 150, 200 +and 300 m (`join.score_eps_m`), each normalised against its own floor, so a teacher cannot win it +by fitting 150 m. `results.csv` keeps the 150 m value alone as `norm_join_main`, and +`thresholds.csv` has both joins at every `--eval-eps` threshold, including 175 and 250 m, which +nothing trains on. Numbers above from before 2026-11-17 use the 150 m join alone. + +## Run + +```bash +cmake --build build-release && ctest --test-dir build-release +./build-release/engine/teacher_eval --config engine/configs/geolife.json \ + --seeds 42 --shares 0.0025 --out results/ +python3 engine/apps/teacher_stats.py results/ +``` + +Use a new `--out` directory each time: the CSVs are appended to. Runs may rewrite the cache files +in `cache/`, which are tracked although `.gitignore` lists them. diff --git a/cache/teacher_cache_seed17.bin b/cache/teacher_cache_seed17.bin new file mode 100644 index 0000000..c33ee57 Binary files /dev/null and b/cache/teacher_cache_seed17.bin differ diff --git a/cache/teacher_cache_seed42.bin b/cache/teacher_cache_seed42.bin new file mode 100644 index 0000000..ee4683a Binary files /dev/null and b/cache/teacher_cache_seed42.bin differ diff --git a/engine/CMakeLists.txt b/engine/CMakeLists.txt index b0f0f3f..a7b869c 100644 --- a/engine/CMakeLists.txt +++ b/engine/CMakeLists.txt @@ -17,6 +17,8 @@ add_library(engine src/io/workload_export.cpp src/pipeline/config.cpp src/pipeline/pipeline.cpp + src/pipeline/scorer.cpp + src/pipeline/test_workloads.cpp src/selection/rdp.cpp src/selection/sed.cpp src/selection/uniform.cpp @@ -40,6 +42,11 @@ add_library(engine src/evaluator/combined.cpp src/query/route_clustering.cpp src/evaluator/route_clustering.cpp + src/teacher/workloads.cpp + src/teacher/objective.cpp + src/teacher/curves.cpp + src/teacher/greedy.cpp + src/teacher/teacher.cpp ) target_include_directories(engine PUBLIC include) @@ -52,6 +59,12 @@ add_executable(run target_link_libraries(run PRIVATE engine) +add_executable(teacher_eval + apps/teacher_eval.cpp +) + +target_link_libraries(teacher_eval PRIVATE engine) + add_executable(engine_tests tests/parquet_loader_test.cpp tests/projection_test.cpp @@ -74,6 +87,8 @@ add_executable(engine_tests tests/combined_test.cpp tests/route_clustering_test.cpp tests/continuous_frechet_test.cpp + tests/teacher/curves_test.cpp + tests/teacher/teacher_test.cpp ) target_link_libraries(engine_tests PRIVATE engine GTest::gtest_main Arrow::arrow_shared Parquet::parquet_shared) gtest_discover_tests(engine_tests) diff --git a/engine/apps/teacher_eval.cpp b/engine/apps/teacher_eval.cpp new file mode 100644 index 0000000..448b0e1 --- /dev/null +++ b/engine/apps/teacher_eval.cpp @@ -0,0 +1,540 @@ +// Runs the teacher and baselines on several seeds and scores every arm on the held-out test +// workloads, built only after every arm is done. Settings come from the --config JSON; flags +// override parts of it. The database self join is the teacher's training data, so it is reported +// but kept out of J4 when join.mode is HeldOut. Writes CSVs to --out; summarise with +// teacher_stats.py. +#include "backend/engine/allocation/budget_allocator.hpp" +#include "backend/engine/core/parallel.hpp" +#include "backend/engine/core/simplification.hpp" +#include "backend/engine/core/timer.hpp" +#include "backend/engine/evaluator/combined.hpp" +#include "backend/engine/evaluator/evaluator.hpp" +#include "backend/engine/evaluator/join.hpp" +#include "backend/engine/io/parquet_loader.hpp" +#include "backend/engine/pipeline/config.hpp" +#include "backend/engine/pipeline/test_workloads.hpp" +#include "backend/engine/query/frechet_helper.hpp" +#include "backend/engine/query/join.hpp" +#include "backend/engine/query/knn.hpp" +#include "backend/engine/query/route_clustering.hpp" +#include "backend/engine/selection/selector.hpp" +#include "backend/engine/split/split.hpp" +#include "backend/engine/teacher/teacher.hpp" +#include "backend/engine/teacher/workloads.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +using core::KeptSets; +constexpr double kNan = std::numeric_limits::quiet_NaN(); +constexpr std::array kEvalEps{100.0, 125.0, 150.0, 175.0, 200.0, 250.0, 300.0}; +constexpr double kRdpAlpha = 0.25; // RDP_a025 +constexpr std::array kPartnerLambdas{0.5, 1.0}; +// Unset options keep the config's value; arms default to all teacher arms +struct Args { + std::filesystem::path config = "engine/configs/geolife.json"; + std::filesystem::path out = "results/teacher"; + std::optional data; + std::optional cache; + std::optional> seeds; // config: seed + std::optional> shares; // config: budget_shares + std::vector arms{teacher::kArms.begin(), teacher::kArms.end()}; + bool baselines = true; + std::optional all_join_pairs; + std::optional teacher_join; // config: teacher.join, else join.mode + std::optional> train_eps; // config: teacher.extra_join_eps_m + std::optional held_out_term; // config: teacher.held_out_term + std::optional ranking_knn; // config: teacher.ranking_knn + std::optional> term_weights; // config: teacher.term_weights + std::vector eval_eps{kEvalEps.begin(), kEvalEps.end()}; + bool knn_check = false; +}; + +template std::vector parse_list(const std::string &text) { + std::vector items; + std::stringstream stream(text); + for (std::string item; std::getline(stream, item, ',');) { + if constexpr (std::is_same_v) + items.push_back(item); + else + items.push_back(static_cast(std::stod(item))); + } + return items; +} + +Args parse_args(int argc, char **argv) { + Args args; + for (int k = 1; k < argc; ++k) { + const std::string key = argv[k]; + auto value = [&] { + if (k + 1 >= argc) + throw std::invalid_argument("missing value for " + key); + return std::string(argv[++k]); + }; + if (key == "--config") + args.config = value(); + else if (key == "--data") + args.data = value(); + else if (key == "--out") + args.out = value(); + else if (key == "--cache") + args.cache = value(); + else if (key == "--seeds") + args.seeds = parse_list(value()); + else if (key == "--shares") + args.shares = parse_list(value()); + else if (key == "--arms") + args.arms = parse_list(value()); + else if (key == "--train-eps") + args.train_eps = parse_list(value()); + else if (key == "--eval-eps") + args.eval_eps = parse_list(value()); + else if (key == "--term-weights") { + args.term_weights = parse_list(value()); + if (args.term_weights->size() != teacher::kTermCount) + throw std::invalid_argument("--term-weights takes kNN,range,where-at,join"); + } else if (key == "--independent-knn") + args.ranking_knn = false; + else if (key == "--held-out-term") + args.held_out_term = true; + else if (key == "--knn-check") + args.knn_check = true; + else if (key == "--teacher-join") { + const auto mode = value(); + if (mode != "HeldOut" && mode != "Self") + throw std::invalid_argument("--teacher-join must be HeldOut or Self"); + args.teacher_join = mode == "Self" ? query::JoinMode::Self : query::JoinMode::HeldOut; + } else if (key == "--no-baselines") + args.baselines = false; + else if (key == "--seen-pairs-only") + args.all_join_pairs = false; + else + throw std::invalid_argument("unknown argument " + key + + " (--config --data --out --cache --seeds --shares --arms " + "--teacher-join --train-eps --held-out-term " + "--independent-knn --term-weights --eval-eps --knn-check " + "--no-baselines --seen-pairs-only)"); + } + teacher::check_arms(args.arms); + return args; +} + +pipeline::Config effective_config(const Args &args) { + auto config = pipeline::load_config(args.config); + if (args.data) + config.data_path = *args.data; + if (args.cache) + config.teacher.cache_dir = *args.cache; + if (args.shares) + config.budget_shares = *args.shares; + if (args.all_join_pairs) + config.teacher.all_join_pairs = *args.all_join_pairs; + if (args.teacher_join) + config.teacher.join = *args.teacher_join; + if (args.train_eps) + config.teacher.extra_join_eps_m = *args.train_eps; + if (args.held_out_term) + config.teacher.held_out_term = *args.held_out_term; + if (args.ranking_knn) + config.teacher.ranking_knn = *args.ranking_knn; + if (args.term_weights) + std::ranges::copy(*args.term_weights, config.teacher.term_weights.begin()); + config.self_join = true; + return config; +} + +const char *mode_name(query::JoinMode mode) { + return mode == query::JoinMode::Self ? "Self" : "HeldOut"; +} + +// Interior points shared in proportion to w, capped at each trajectory's interior, the rest by +// largest remainder. Budgets include the endpoints +std::vector weighted_budgets(const core::TrajectoryDb &db, + const std::vector &weights, + std::size_t interior_total) { + const std::size_t n = db.size(); + std::vector cap(n), interior(n, 0); + for (std::size_t i = 0; i < n; ++i) + cap[i] = db.trajectory(i).size() >= 2 ? db.trajectory(i).size() - 2 : 0; + std::vector open(n, 1); + std::size_t left = interior_total; + while (left > 0) { + double weight_sum = 0; + for (std::size_t i = 0; i < n; ++i) + if (open[i]) + weight_sum += weights[i]; + if (weight_sum <= 0) + break; + std::vector frac(n, 0); + std::size_t given = 0; + bool capped = false; + for (std::size_t i = 0; i < n; ++i) { + if (!open[i]) + continue; + const double want = static_cast(left) * weights[i] / weight_sum; + auto add = static_cast(want); + if (interior[i] + add >= cap[i]) { + add = cap[i] - interior[i]; + open[i] = 0; + capped = true; + } else { + frac[i] = want - static_cast(add); + } + interior[i] += add; + given += add; + } + left -= given; + if (capped) + continue; + std::vector idx; + for (std::size_t i = 0; i < n; ++i) + if (open[i]) + idx.push_back(i); + std::ranges::sort(idx, [&](auto a, auto b) { + return frac[a] > frac[b] || (frac[a] == frac[b] && a < b); + }); + for (std::size_t k = 0; k < idx.size() && left > 0; ++k) { + ++interior[idx[k]]; + --left; + if (interior[idx[k]] == cap[idx[k]]) + open[idx[k]] = 0; + } + if (given == 0 && idx.empty()) + break; + } + for (std::size_t i = 0; i < n; ++i) + interior[i] += std::min(2, db.trajectory(i).size()); + return interior; +} + +KeptSets select(const core::TrajectoryDb &db, selection::AlgorithmType algorithm, + const std::vector &budgets) { + KeptSets kept(db.size()); + core::parallel_for(db.size(), [&](std::size_t i) { + kept[i] = selection::simplify_trajectory(db.trajectory(i), algorithm, budgets[i]); + }); + return kept; +} + +struct Arm { + std::string name; + double share; + KeptSets kept; +}; + +// PARTNER: RDP with weight 1 + lambda for trajectories that have a join partner +std::vector baseline_arms(const core::TrajectoryDb &db, + const std::vector &pairs, bool all_join_pairs, + std::span shares) { + using selection::AlgorithmType; + std::vector partnered(db.size(), 0); + for (const auto &pair : pairs) + if (all_join_pairs || pair.seen) + partnered[pair.i] = partnered[pair.j] = 1; + std::vector arms; + for (const double share : shares) { + const auto budget = allocation::total_budget(db, share); + auto geometric = [&](double alpha) { + return allocation::create_trajectory_budgets( + db, allocation::AllocatorType::SizeProportional, budget, alpha); + }; + const auto equal = geometric(0.0); + arms.push_back({"RDP_a0", share, select(db, AlgorithmType::RDP, equal)}); + arms.push_back({"RDP_a025", share, select(db, AlgorithmType::RDP, geometric(kRdpAlpha))}); + arms.push_back({"SED_a0", share, select(db, AlgorithmType::SED, equal)}); + arms.push_back({"UNI_a0", share, select(db, AlgorithmType::UNIFORM, equal)}); + for (const double lambda : kPartnerLambdas) { + std::vector weights(db.size()); + for (std::size_t i = 0; i < db.size(); ++i) + weights[i] = 1.0 + (partnered[i] ? lambda : 0.0); + arms.push_back({std::format("PARTNER_l{:g}", lambda), share, + select(db, AlgorithmType::RDP, + weighted_budgets(db, weights, budget - 2 * db.size()))}); + } + } + return arms; +} + +void validate(const Arm &arm, const core::TrajectoryDb &db) { + if (arm.kept.size() != db.size()) + throw std::runtime_error(arm.name + ": wrong number of trajectories"); + std::size_t total = 0; + for (std::size_t i = 0; i < db.size(); ++i) { + const auto &kept = arm.kept[i]; + const auto n = db.trajectory(i).size(); + if (kept.size() < 2 || kept.front() != 0 || kept.back() != n - 1 || + !std::ranges::is_sorted(kept) || std::ranges::adjacent_find(kept) != kept.end()) + throw std::runtime_error( + std::format("{}: invalid kept set for trajectory {}", arm.name, i)); + total += kept.size(); + } + const auto budget = allocation::total_budget(db, arm.share); + if (total != budget) + throw std::runtime_error(std::format("{}: {} points, budget {}", arm.name, total, budget)); +} + +struct PairRecall { + double seen = kNan, unseen = kNan; +}; + +// Mean per-trajectory recall on the seen and unseen halves of the teacher's join pairs +PairRecall pair_recall(const core::TrajectoryDb &compressed, + const std::vector &pairs, const query::JoinConfig &join) { + std::vector matched(pairs.size()); + core::parallel_for(pairs.size(), [&](std::size_t k) { + matched[k] = + query::frechet_within(compressed.trajectory(pairs[k].i), + compressed.trajectory(pairs[k].j), join.eps_m, join.frechet); + }); + PairRecall result; + for (const bool seen : {true, false}) { + std::vector total(compressed.size(), 0), hits(compressed.size(), 0); + for (std::size_t k = 0; k < pairs.size(); ++k) { + if (pairs[k].seen != seen) + continue; + for (const auto t : {pairs[k].i, pairs[k].j}) { + ++total[t]; + hits[t] += matched[k]; + } + } + double sum = 0; + std::size_t count = 0; + for (std::size_t t = 0; t < total.size(); ++t) + if (total[t] != 0) { + sum += static_cast(hits[t]) / static_cast(total[t]); + ++count; + } + (seen ? result.seen : result.unseen) = count ? sum / static_cast(count) : kNan; + } + return result; +} + +// Whether the teacher trained on the join at eps_m in mode: one of its thresholds in its own join +// mode, or its held-out term at join.eps_m +bool trained_on(const pipeline::Config &config, const teacher::ObjectiveConfig &objective, + double eps_m, query::JoinMode mode) { + const auto &extra = config.teacher.extra_join_eps_m; + const bool teacher_eps = + eps_m == config.join.eps_m || std::ranges::find(extra, eps_m) != extra.end(); + const bool held_out_term = + objective.held_out_term && mode == query::JoinMode::HeldOut && eps_m == config.join.eps_m; + return (teacher_eps && mode == objective.join_mode) || held_out_term; +} + +std::ofstream open_csv(const std::filesystem::path &file, const std::string &header) { + const bool fresh = !std::filesystem::exists(file); + std::ofstream out(file, std::ios::app); + if (fresh) + out << header << '\n'; + return out; +} + +} // namespace + +int main(int argc, char **argv) try { + const auto args = parse_args(argc, argv); + const auto base_config = effective_config(args); + std::filesystem::create_directories(args.out); + const auto all = io::load_trajectories(base_config.data_path); + std::cerr << std::format("loaded {} trajectories, {} points\n", all.size(), all.num_points()); + + auto results = open_csv( + args.out / "results.csv", + "seed,share,arm,budget,knn_recall,range_f1,where_at_avg_hit,join_f1,clustering," + "where_at_within_100,join_micro_f1,knn_tau,norm_knn,norm_range,norm_where_at,norm_join," + "norm_clustering,J4,J_T,join_recall_seen,join_recall_unseen,predicted_clusters," + "eval_s,join_mode,self_join_f1,norm_self_join,teacher_join_mode,norm_join_main," + "norm_clustering_main"); + auto floors = + open_csv(args.out / "floors.csv", + "seed,knn_recall,range_f1,where_at_avg_hit,join_f1,clustering,self_join_f1"); + auto thresholds = + open_csv(args.out / "thresholds.csv", + "seed,share,arm,join,eps_m,join_f1,norm_join,recall,teacher_threshold"); + auto terms = open_csv(args.out / "terms.csv", + "seed,share,arm,teacher_knn,teacher_range,teacher_where_at,teacher_join," + "norm_knn,norm_range,norm_where_at,norm_join,norm_self_join"); + auto budgets = open_csv(args.out / "budgets.csv", "seed,share,arm,points,budget"); + auto timings = open_csv(args.out / "timings.csv", "seed,step,seconds"); + std::ofstream knn_check; + if (args.knn_check) + knn_check = open_csv(args.out / "knn_check.csv", + "seed,share,arm,proxy_recall,db_recall,test_recall,norm_proxy," + "norm_db,norm_test"); + + for (const auto seed : args.seeds.value_or(std::vector{base_config.seed})) { + auto config = base_config; + config.seed = seed; + const auto parts = split::split(all, config.split, seed); + const auto db = core::subset(all, parts.database); + std::cerr << std::format("seed {}: database {} trajectories, {} points\n", seed, db.size(), + db.num_points()); + + // Database only + auto start = std::chrono::steady_clock::now(); + auto workloads = teacher::load_or_build_workloads( + db, pipeline::teacher_workload_config(config), seed, config.teacher.cache_dir); + const auto pairs = workloads.positives; + const teacher::Teacher model(db, std::move(workloads), + pipeline::teacher_objective_config(config), seed); + timings << std::format("{},teacher_setup,{:.2f}\n", seed, core::seconds_since(start)); + + std::vector arms; + for (const double share : config.budget_shares) { + start = std::chrono::steady_clock::now(); + for (auto &arm : model.run(allocation::total_budget(db, share), args.arms)) + arms.push_back({arm.name, share, core::to_kept(arm.simplification)}); + timings << std::format("{},teacher_{:g},{:.2f}\n", seed, share, + core::seconds_since(start)); + } + if (args.baselines) { + for (auto &arm : + baseline_arms(db, pairs, config.teacher.all_join_pairs, config.budget_shares)) + arms.push_back(std::move(arm)); + for (const double share : config.budget_shares) + for (auto &arm : model.single_recipes(allocation::total_budget(db, share))) + arms.push_back({arm.name, share, core::to_kept(arm.simplification)}); + } + for (const auto &arm : arms) + validate(arm, db); + + // Held-out data from here on + start = std::chrono::steady_clock::now(); + const auto pool = core::subset(all, parts.queries); + const pipeline::TestWorkloads test_workloads(db, pool, config); + // Both joins at other thresholds: does the join gain hold beyond the ones trained on? + std::vector other_joins; + for (const double eps : args.eval_eps) { + other_joins.push_back(query::generate_self_join(db, eps, config.join.frechet)); + other_joins.push_back( + query::generate_join_queries(db, pool, + {.query_count = config.join.query_count, + .eps_m = eps, + .mode = query::JoinMode::HeldOut, + .frechet = config.join.frechet}, + seed)); + } + timings << std::format("{},test_workloads,{:.2f}\n", seed, core::seconds_since(start)); + + const auto test = test_workloads.view(); + const auto &scoring = config.where_at_scoring; + const auto floor_db = core::materialize(db, core::endpoints_only(db)); + const auto floor = evaluator::Evaluator(db, floor_db, test, scoring).evaluate().scores; + floors << std::format("{},{},{},{},{},{},{}\n", seed, floor.knn.recall_at_k, floor.range_f1, + floor.where_at.average_hit, floor.join.f1, floor.clustering.score, + floor.self_join.f1); + std::vector other_floor; + for (const auto &workload : other_joins) + other_floor.push_back(evaluator::evaluate_join(floor_db, workload).f1); + // Route clustering at each threshold, built on the self join there + std::vector other_clusters; + std::vector cluster_floor; + if (config.clustering.enabled) + for (const auto &workload : other_joins) + if (workload.mode == query::JoinMode::Self) { + auto clustering = config.clustering; + clustering.eps_m = workload.eps_m; + other_clusters.push_back( + query::generate_route_clustering(db, clustering, &workload)); + cluster_floor.push_back( + evaluator::evaluate_route_clustering(floor_db, other_clusters.back()) + .score); + } + + const auto &objective = model.objective(); + const double db_knn_floor = args.knn_check ? objective.knn_recall(floor_db) : kNan; + for (const auto &arm : arms) { + start = std::chrono::steady_clock::now(); + const auto compressed = core::materialize(db, arm.kept); + const auto scores = + evaluator::Evaluator(db, compressed, test, scoring).evaluate().scores; + const auto normalized = evaluator::normalize_scores(scores, floor); + const auto recall = pair_recall(compressed, pairs, config.join); + results << std::format( + "{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{:.2f}," + "{},{},{},{},{},{}\n", + seed, arm.share, arm.name, allocation::total_budget(db, arm.share), + scores.knn.recall_at_k, scores.range_f1, scores.where_at.average_hit, + scores.join.f1, scores.clustering.score, scores.where_at.within_radius, + scores.join.micro_f1, scores.knn.kendalls_tau_topk, normalized.knn, + normalized.range, normalized.where_at, normalized.join, normalized.clustering, + normalized.combined, objective.total(arm.kept), recall.seen, recall.unseen, + scores.clustering.predicted_clusters, core::seconds_since(start), + mode_name(config.join.mode), scores.self_join.f1, normalized.self_join, + mode_name(objective.config().join_mode), normalized.join_f1, + normalized.clustering_score); + + for (std::size_t k = 0; k < other_joins.size(); ++k) { + const auto &workload = other_joins[k]; + const auto join_scores = evaluator::evaluate_join(compressed, workload); + const bool teacher_threshold = + trained_on(config, objective.config(), workload.eps_m, workload.mode); + thresholds << std::format("{},{},{},{},{},{},{},{},{}\n", seed, arm.share, arm.name, + mode_name(workload.mode), workload.eps_m, join_scores.f1, + (join_scores.f1 - other_floor[k]) / + (1.0 - other_floor[k]), + join_scores.recall, teacher_threshold ? 1 : 0); + } + for (std::size_t k = 0; k < other_clusters.size(); ++k) { + const auto &workload = other_clusters[k]; + const double score = + evaluator::evaluate_route_clustering(compressed, workload).score; + // Clustering is built on the self join at its threshold + const bool teacher_threshold = + trained_on(config, objective.config(), workload.eps_m, query::JoinMode::Self); + thresholds << std::format("{},{},{},Clustering,{},{},{},,{}\n", seed, arm.share, + arm.name, workload.eps_m, score, + (score - cluster_floor[k]) / (1.0 - cluster_floor[k]), + teacher_threshold ? 1 : 0); + } + // The teacher's own estimate of each metric, next to the test's + const auto teacher_metrics = objective.metrics(arm.kept); + const auto &teacher_floors = objective.floors(); + auto norm = [&](teacher::Objective::Term t) { + return (teacher_metrics[t] - teacher_floors[t]) / (1.0 - teacher_floors[t]); + }; + terms << std::format("{},{},{},{},{},{},{},{},{},{},{},{}\n", seed, arm.share, arm.name, + norm(teacher::Objective::Knn), norm(teacher::Objective::Range), + norm(teacher::Objective::WhereAt), norm(teacher::Objective::Join), + normalized.knn, normalized.range, normalized.where_at, + normalized.join, normalized.self_join); + if (args.knn_check) { + const double db_knn = objective.knn_recall(compressed); + knn_check << std::format("{},{},{},{},{},{},{},{},{}\n", seed, arm.share, arm.name, + teacher_metrics[teacher::Objective::Knn], db_knn, + scores.knn.recall_at_k, norm(teacher::Objective::Knn), + (db_knn - db_knn_floor) / (1.0 - db_knn_floor), + normalized.knn); + knn_check.flush(); + } + if (teacher::is_arm(arm.name)) + for (std::size_t i = 0; i < arm.kept.size(); ++i) + budgets << std::format("{},{},{},{},{}\n", seed, arm.share, arm.name, + db.trajectory(i).size(), arm.kept[i].size()); + for (auto *out : {&results, &thresholds, &terms, &budgets}) + out->flush(); + std::cerr << std::format("seed {} {} {:g}: J4 {:.4f}, join {:.4f}, self join {:.4f}\n", + seed, arm.name, arm.share, normalized.combined, + normalized.join, normalized.self_join); + } + timings.flush(); + floors.flush(); + } + return 0; +} catch (const std::exception &e) { + std::cerr << "error: " << e.what() << '\n'; + return 1; +} diff --git a/engine/apps/teacher_stats.py b/engine/apps/teacher_stats.py new file mode 100644 index 0000000..2be3679 --- /dev/null +++ b/engine/apps/teacher_stats.py @@ -0,0 +1,132 @@ +"""Summarises a teacher_eval run: python3 teacher_stats.py [summary.md] + +Per share: means and SDs over seeds, paired J4 differences with t intervals, the gate against +the best cheap baseline, the coordination checks, and the self join at 100, 150 and 300 m. + +norm_join is J4's join term (held-out when join.mode is HeldOut). norm_self_join is the database +self join, which the teacher trains on: an in-sample diagnostic that is not part of J4. +""" +import csv +import math +import statistics +import sys +from pathlib import Path +from collections import defaultdict + +T95 = {1: 12.706, 2: 4.303, 3: 3.182, 4: 2.776, 5: 2.571, 6: 2.447, 7: 2.365, 8: 2.306, 9: 2.262, 10: 2.228} +METRICS = ["J4", "norm_knn", "norm_range", "norm_where_at", "norm_join", "norm_clustering", "norm_self_join"] +CHEAP = ["RDP_a0", "RDP_a025", "PARTNER_l0.5", "PARTNER_l1"] + + +def num(v): + try: + x = float(v) + return x if math.isfinite(x) else None + except ValueError: + return None + + +def mean_sd(xs): + if not xs: + return "–" + m = statistics.fmean(xs) + return f"{m:.4f}" if len(xs) < 2 else f"{m:.4f} ± {statistics.stdev(xs):.4f}" + + +def paired(a, b): + seeds = sorted(set(a) & set(b)) + d = [a[s] - b[s] for s in seeds] + if len(d) < 2: + return (statistics.fmean(d), None, None, len(d)) if d else None + m, sd = statistics.fmean(d), statistics.stdev(d) + h = T95.get(len(d) - 1, 1.96) * sd / math.sqrt(len(d)) + return m, m - h, m + h, len(d) + + +def fmt(p): + if p is None: + return "–" + m, lo, hi, n = p + return f"{m:+.4f}" if lo is None else f"{m:+.4f} [{lo:+.4f}, {hi:+.4f}] (n={n})" + + +def main(): + run = Path(sys.argv[1]) + rows = list(csv.DictReader(open(run / "results.csv"))) + data = defaultdict(lambda: defaultdict(lambda: defaultdict(dict))) + for r in rows: + # Runs from before clustering became a diagnostic have no J4 column: derive it + if num(r.get("J4", "")) is None: + parts = [num(r.get(c, "")) for c in ["norm_knn", "norm_range", "norm_where_at", "norm_join"]] + parts = [x for x in parts if x is not None] + r["J4"] = str(statistics.fmean(parts)) if parts else "" + for m in METRICS + ["J_T", "join_recall_seen", "join_recall_unseen"]: + v = num(r.get(m, "")) + if v is not None: + data[r["share"]][r["arm"]][m][int(r["seed"])] = v + other = defaultdict(lambda: defaultdict(lambda: defaultdict(dict))) # share, arm, eps, seed + if (run / "thresholds.csv").exists(): + for r in csv.DictReader(open(run / "thresholds.csv")): + if r.get("join", "Self") != "Self": + continue + v = num(r["norm_join"]) + if v is not None: + other[r["share"]][r["arm"]][float(r["eps_m"])][int(r["seed"])] = v + out = [] + for share in sorted(data, key=float): + arms = data[share] + out.append(f"## Share {float(share) * 100:g}%\n") + out.append("| Arm | " + " | ".join(METRICS) + " | seen / unseen join recall |") + out.append("|---|" + "---:|" * (len(METRICS) + 1)) + for arm in sorted(arms): + cells = [mean_sd(list(arms[arm][m].values())) for m in METRICS] + seen = list(arms[arm]["join_recall_seen"].values()) + unseen = list(arms[arm]["join_recall_unseen"].values()) + su = f"{statistics.fmean(seen):.4f} / {statistics.fmean(unseen):.4f}" if seen and unseen else "–" + out.append(f"| {arm} | " + " | ".join(cells) + f" | {su} |") + out.append("") + cheap = [a for a in CHEAP if a in arms and arms[a]["J4"]] + best = max(cheap, key=lambda a: statistics.fmean(arms[a]["J4"].values())) if cheap else None + out.append(f"Best cheap baseline by mean J4: **{best}**\n") + out.append("| Paired J4 difference | mean [95% CI] |") + out.append("|---|---|") + for arm in sorted(arms): + if arm != "RDP_a0" and "RDP_a0" in arms: + out.append(f"| {arm} − RDP_a0 | {fmt(paired(arms[arm]['J4'], arms['RDP_a0']['J4']))} |") + if best and "BUDGET_CHOICE" in arms: + gate = paired(arms["BUDGET_CHOICE"]["J4"], arms[best]["J4"]) + out.append(f"| **BUDGET_CHOICE − {best} (gate: ≥ .03, CI above 0)** | {fmt(gate)} |") + if all(a in arms for a in ["RDP_a0", "EQUAL_CHOICE_RDP_PARTNERS", "BUDGET_RDP", "BUDGET_CHOICE"]): + j = {a: arms[a]["J4"] for a in ["RDP_a0", "EQUAL_CHOICE_RDP_PARTNERS", "BUDGET_RDP", "BUDGET_CHOICE"]} + seeds = set.intersection(*(set(v) for v in j.values())) + inter = {s: j["BUDGET_CHOICE"][s] - j["EQUAL_CHOICE_RDP_PARTNERS"][s] - j["BUDGET_RDP"][s] + j["RDP_a0"][s] for s in seeds} + out.append(f"| Interaction (BUDGET_CHOICE − EQUAL_CHOICE_RDP_PARTNERS − BUDGET_RDP + RDP_a0) | {fmt(paired(inter, {s: 0.0 for s in seeds}))} |") + for a, b in [("BUDGET_LOOP", "BUDGET_CHOICE"), ("BUDGET_LOOP_REALLOCATE", "BUDGET_LOOP"), + ("EQUAL_LOOP", "EQUAL_CHOICE_RDP_PARTNERS"), ("BUDGET_LOOP", "BUDGET_LOOP_SHUFFLED")]: + if a in arms and b in arms: + out.append(f"| {a} − {b} | {fmt(paired(arms[a]['J4'], arms[b]['J4']))} |") + out.append("") + if share in other: + out.append("Normalised self-join F1 at the teacher's 150 m and at thresholds it does not optimise:\n") + out.append("| Arm | 100 m | 150 m | 300 m |") + out.append("|---|---:|---:|---:|") + for arm in sorted(arms): + # Runs from before the held-out join have the self join in norm_join + at_150 = arms[arm]["norm_self_join"] or arms[arm]["norm_join"] + cells = [mean_sd(list(other[share][arm].get(100.0, {}).values())), + mean_sd(list(at_150.values())), + mean_sd(list(other[share][arm].get(300.0, {}).values()))] + out.append(f"| {arm} | " + " | ".join(cells) + " |") + out.append("") + pairs = [(arms[a]["J_T"][s], arms[a]["J4"][s]) for a in arms for s in arms[a]["J_T"] if s in arms[a]["J4"]] + if len(pairs) >= 3: + xs, ys = zip(*pairs) + out.append(f"Surrogate check: Pearson r(J_T, J4) = {statistics.correlation(xs, ys):.3f} over {len(pairs)} arm-seeds\n") + text = "\n".join(out) + if len(sys.argv) > 2: + open(sys.argv[2], "w").write("# Teacher summary\n\n" + text + "\n") + print(text) + + +if __name__ == "__main__": + main() diff --git a/engine/configs/geolife.json b/engine/configs/geolife.json index 55a96ba..697ad46 100644 --- a/engine/configs/geolife.json +++ b/engine/configs/geolife.json @@ -32,13 +32,26 @@ "max_tolerance_m": 500.0 }, "join": { - "mode": "Self", - "eps_m": 150.0 + "mode": "HeldOut", + "query_count": 0, + "eps_m": 150.0, + "score_eps_m": [100.0, 200.0, 300.0], + "self_join": true }, "clustering": { "enabled": true, "eps_m": 150.0, "min_size": 5 }, - "frechet": "Continuous" + "frechet": "Continuous", + "teacher": { + "enabled": false, + "cache_dir": "cache", + "arms": ["BUDGET_CHOICE", "BUDGET_LOOP"], + "range_query_count": 64000, + "extra_join_eps_m": [100.0, 125.0, 200.0, 300.0], + "held_out_term": false, + "ranking_knn": true, + "term_weights": [1.0, 1.0, 1.0, 1.0] + } } diff --git a/engine/include/backend/engine/allocation/budget_allocator.hpp b/engine/include/backend/engine/allocation/budget_allocator.hpp index 04d745d..7c82485 100644 --- a/engine/include/backend/engine/allocation/budget_allocator.hpp +++ b/engine/include/backend/engine/allocation/budget_allocator.hpp @@ -10,4 +10,7 @@ std::vector create_trajectory_budgets(const core::TrajectoryDb &tra AllocatorType allocator, std::size_t total_budget, double alpha = 1.0); +// Every trajectory's endpoints plus share of the remaining points, rounded +std::size_t total_budget(const core::TrajectoryDb &trajectories, double share); + } // namespace allocation \ No newline at end of file diff --git a/engine/include/backend/engine/core/simplification.hpp b/engine/include/backend/engine/core/simplification.hpp index 357db44..dbe188a 100644 --- a/engine/include/backend/engine/core/simplification.hpp +++ b/engine/include/backend/engine/core/simplification.hpp @@ -43,9 +43,19 @@ class Simplification { std::vector m_kept; }; +// Kept point indices per trajectory, sorted, endpoints included +using KeptSets = std::vector>; + +Simplification to_simplification(const KeptSets &kept); +KeptSets to_kept(const Simplification &simplification); + +// First and last point of each trajectory, fewer for trajectories shorter than 2 points +KeptSets endpoints_only(const TrajectoryDb &trajectories); + // Builds a database holding only the kept points, with the same trajectories in the same order. // Copies projected coordinates and rebuilds bounding boxes from the retained points. // Kept indices must be in range and strictly increasing within each trajectory. TrajectoryDb materialize(const TrajectoryDb &trajectories, const Simplification &simplification); +TrajectoryDb materialize(const TrajectoryDb &trajectories, const KeptSets &kept); } // namespace core \ No newline at end of file diff --git a/engine/include/backend/engine/core/timer.hpp b/engine/include/backend/engine/core/timer.hpp new file mode 100644 index 0000000..269e5a4 --- /dev/null +++ b/engine/include/backend/engine/core/timer.hpp @@ -0,0 +1,12 @@ +#pragma once + +#include + +namespace core { + +// Wall-clock seconds since start +inline double seconds_since(std::chrono::steady_clock::time_point start) { + return std::chrono::duration(std::chrono::steady_clock::now() - start).count(); +} + +} // namespace core diff --git a/engine/include/backend/engine/evaluator/combined.hpp b/engine/include/backend/engine/evaluator/combined.hpp index 105556f..3094715 100644 --- a/engine/include/backend/engine/evaluator/combined.hpp +++ b/engine/include/backend/engine/evaluator/combined.hpp @@ -6,12 +6,12 @@ namespace evaluator { -// 0 = the floor (endpoints only), 1 = perfect, without clipping. Combined averages -// one primary score per type: recall@k, range F1, where-at average-hit, join F1 and route -// clustering. Missing primaries are skipped, never replaced with a diagnostic score. -// Average-hit is the hit rate averaged over a tolerance drawn uniformly from -// [min_tolerance_m, max_tolerance_m]. The within-radius hit rate at hit_radius_m is a -// diagnostic and does not contribute to the combined score. +// 0 = the floor (endpoints only), 1 = perfect, unclipped. Combined (J4) averages recall@k, +// range F1, where-at average-hit and join F1, skipping missing ones. Within-radius, clustering +// (it repeats the join's Fréchet test) and the self join (seen by anything fitted to the +// database) are diagnostics only. +// join and clustering average over their threshold grids, each against its own floor; +// join_f1 and clustering_score keep the main threshold alone. struct NormalizedScores { static constexpr double missing = std::numeric_limits::quiet_NaN(); @@ -29,6 +29,7 @@ struct NormalizedScores { double where_at = missing; double join = missing; double clustering = missing; + double self_join = missing; double combined = missing; }; diff --git a/engine/include/backend/engine/evaluator/evaluator.hpp b/engine/include/backend/engine/evaluator/evaluator.hpp index 73e4c3a..6d28998 100644 --- a/engine/include/backend/engine/evaluator/evaluator.hpp +++ b/engine/include/backend/engine/evaluator/evaluator.hpp @@ -24,6 +24,11 @@ struct Workloads { std::span range = {}; const query::JoinWorkload *join = nullptr; // a pointer, as it also carries eps and the queries const query::RouteClusteringWorkload *clustering = nullptr; + // Diagnostic; may point at join itself + const query::JoinWorkload *self_join = nullptr; + // Extra thresholds, averaged into the reported scores + std::span join_grid = {}; + std::span clustering_grid = {}; }; // Kendall's tau is in [-1, 1] and the others in [0, 1], or NaN if it could not be computed @@ -34,6 +39,9 @@ struct Scoretable { JoinScores join; double range_f1 = std::numeric_limits::quiet_NaN(); RouteClusteringScores clustering; + JoinScores self_join; + std::vector join_grid; // in Workloads::join_grid order + std::vector clustering_grid; // in Workloads::clustering_grid order }; struct EvaluationResult { @@ -57,6 +65,7 @@ class Evaluator { double get_range_f1() const; WhereAtScores get_where_at_scores() const; JoinScores get_join_scores() const; + JoinScores get_self_join_scores() const; RouteClusteringScores get_clustering_scores() const; private: diff --git a/engine/include/backend/engine/pipeline/config.hpp b/engine/include/backend/engine/pipeline/config.hpp index 74f47db..9599397 100644 --- a/engine/include/backend/engine/pipeline/config.hpp +++ b/engine/include/backend/engine/pipeline/config.hpp @@ -8,6 +8,9 @@ #include "backend/engine/query/where_at_query.hpp" #include "backend/engine/selection/selector.hpp" #include "backend/engine/split/split.hpp" +#include "backend/engine/teacher/objective.hpp" +#include "backend/engine/teacher/teacher.hpp" +#include "backend/engine/teacher/workloads.hpp" #include #include @@ -30,10 +33,18 @@ struct Config { query::WhereAtConfig where_at; evaluator::WhereAtScoringConfig where_at_scoring; query::JoinConfig join; + bool self_join = false; // diagnostic, outside J4 + // Extra thresholds averaged into the join and clustering scores + std::vector join_score_eps_m; query::RouteClusteringConfig clustering; + teacher::TeacherConfig teacher; }; // Reads a JSON config. Only data_path is required, a missing key keeps its default. Config load_config(const std::filesystem::path &path); +teacher::WorkloadConfig teacher_workload_config(const Config &config); + +teacher::ObjectiveConfig teacher_objective_config(const Config &config); + } // namespace pipeline diff --git a/engine/include/backend/engine/pipeline/scorer.hpp b/engine/include/backend/engine/pipeline/scorer.hpp new file mode 100644 index 0000000..27dad50 --- /dev/null +++ b/engine/include/backend/engine/pipeline/scorer.hpp @@ -0,0 +1,32 @@ +#pragma once + +#include "backend/engine/core/simplification.hpp" +#include "backend/engine/core/trajectory_db.hpp" +#include "backend/engine/evaluator/evaluator.hpp" +#include "backend/engine/pipeline/config.hpp" +#include "backend/engine/pipeline/test_workloads.hpp" + +#include + +namespace pipeline { + +// Scores simplifications against the endpoints-only floor, prints and optionally exports them +class Scorer { + public: + // workloads must outlive the scorer + Scorer(const core::TrajectoryDb &database, const TestWorkloads &workloads, + const Config &config); + + void score(std::string_view label, std::string_view name, + const core::Simplification &simplification) const; + + private: + const core::TrajectoryDb &m_database; + evaluator::Workloads m_workloads; + const Config &m_config; + evaluator::Scoretable m_floor; +}; + +void print_size(std::string_view name, const core::TrajectoryDb &trajectories); + +} // namespace pipeline diff --git a/engine/include/backend/engine/pipeline/test_workloads.hpp b/engine/include/backend/engine/pipeline/test_workloads.hpp new file mode 100644 index 0000000..a092efa --- /dev/null +++ b/engine/include/backend/engine/pipeline/test_workloads.hpp @@ -0,0 +1,35 @@ +#pragma once + +#include "backend/engine/core/trajectory_db.hpp" +#include "backend/engine/evaluator/evaluator.hpp" +#include "backend/engine/pipeline/config.hpp" +#include "backend/engine/query/join.hpp" +#include "backend/engine/query/knn.hpp" +#include "backend/engine/query/range_query.hpp" +#include "backend/engine/query/route_clustering.hpp" +#include "backend/engine/query/where_at_query.hpp" + +#include + +namespace pipeline { + +// The held-out workloads, generated in declaration order. Not copyable: view() points into it +struct TestWorkloads { + query::KnnWorkload knn; + query::WhereAtWorkload where_at; + query::RangeWorkload range; + query::JoinWorkload join; + std::vector join_grid; // the join at config.join_score_eps_m + std::optional self_join; + std::optional clustering; + std::vector clustering_grid; // at config.join_score_eps_m + + TestWorkloads(const core::TrajectoryDb &database, const core::TrajectoryDb &queries, + const Config &config); + TestWorkloads(const TestWorkloads &) = delete; + TestWorkloads &operator=(const TestWorkloads &) = delete; + + evaluator::Workloads view() const; +}; + +} // namespace pipeline diff --git a/engine/include/backend/engine/query/frechet_helper.hpp b/engine/include/backend/engine/query/frechet_helper.hpp index c05fa0b..a6c4ac3 100644 --- a/engine/include/backend/engine/query/frechet_helper.hpp +++ b/engine/include/backend/engine/query/frechet_helper.hpp @@ -49,4 +49,8 @@ double continuous_frechet_distance(const core::TrajectoryView &a, const core::Tr bool frechet_within(const core::TrajectoryView &a, const core::TrajectoryView &b, double max_dist, FrechetVariant variant); +// The Fréchet distance of the given variant +double frechet_distance(const core::TrajectoryView &a, const core::TrajectoryView &b, + FrechetVariant variant); + } // namespace query diff --git a/engine/include/backend/engine/query/join.hpp b/engine/include/backend/engine/query/join.hpp index aec8dd8..baefe64 100644 --- a/engine/include/backend/engine/query/join.hpp +++ b/engine/include/backend/engine/query/join.hpp @@ -5,6 +5,8 @@ #include #include +#include +#include #include namespace query { @@ -15,7 +17,7 @@ struct JoinConfig { std::size_t query_count = 10; // 0 = use every trajectory in the query pool double eps_m = 500.0; // join threshold in metres, must be positive JoinMode mode = JoinMode::HeldOut; // Self ignores query_count and the query pool - FrechetVariant frechet = FrechetVariant::Discrete; + FrechetVariant frechet = FrechetVariant::Continuous; }; struct JoinQuery { @@ -30,7 +32,7 @@ struct JoinWorkload { std::vector entries; double eps_m = 0.0; JoinMode mode = JoinMode::HeldOut; - FrechetVariant frechet = FrechetVariant::Discrete; + FrechetVariant frechet = FrechetVariant::Continuous; // Self entries contain only indices j > i; both operands are evaluated compressed. }; @@ -43,10 +45,15 @@ JoinWorkload generate_join_queries(const core::TrajectoryDb &original, // All unordered database pairs, endpoint-pruned, with reference truth on originals. // No held-out queries or sampling. Nonempty trajectories must retain their endpoints/IDs/order. JoinWorkload generate_self_join(const core::TrajectoryDb &original, double eps_m = 500.0, - FrechetVariant variant = FrechetVariant::Discrete); + FrechetVariant variant = FrechetVariant::Continuous); + +// Throws std::invalid_argument unless every threshold in eps_m is finite, positive, distinct and +// differs from main_eps_m. name says which setting in the message +void check_extra_thresholds(std::span eps_m, double main_eps_m, + std::string_view name); std::vector join(const core::TrajectoryDb &trajectories, const core::TrajectoryView &query, double eps, - FrechetVariant variant = FrechetVariant::Discrete); + FrechetVariant variant = FrechetVariant::Continuous); } // namespace query diff --git a/engine/include/backend/engine/query/knn.hpp b/engine/include/backend/engine/query/knn.hpp index ba820d0..9a13661 100644 --- a/engine/include/backend/engine/query/knn.hpp +++ b/engine/include/backend/engine/query/knn.hpp @@ -18,7 +18,7 @@ struct KnnConfig { std::size_t k = 5; std::size_t query_count = 10; // 0 = use every trajectory in the query pool double p = 0.5; // top-k tau penalty for a pair one list holds neither of, in [0, 1] - FrechetVariant frechet = FrechetVariant::Discrete; + FrechetVariant frechet = FrechetVariant::Continuous; }; struct KnnWorkload { @@ -26,7 +26,7 @@ struct KnnWorkload { std::vector entries; std::size_t k; double p = 0.5; - FrechetVariant frechet = FrechetVariant::Discrete; + FrechetVariant frechet = FrechetVariant::Continuous; }; // Samples query_count trajectories from query_pool (the held-out split) and stores their k @@ -40,6 +40,6 @@ KnnWorkload generate_knn_queries(const core::TrajectoryDb &original, // as equal. std::vector knn(const core::TrajectoryDb &trajectories, const core::TrajectoryView &query, std::size_t k, - FrechetVariant variant = FrechetVariant::Discrete); + FrechetVariant variant = FrechetVariant::Continuous); } // namespace query diff --git a/engine/include/backend/engine/query/route_clustering.hpp b/engine/include/backend/engine/query/route_clustering.hpp index f1af806..dacf4fb 100644 --- a/engine/include/backend/engine/query/route_clustering.hpp +++ b/engine/include/backend/engine/query/route_clustering.hpp @@ -15,7 +15,7 @@ struct RouteClusteringConfig { bool enabled = false; double eps_m = 500.0; // neighbour threshold, same test as the self join std::size_t min_size = 5; // a cluster needs at least this many trajectories, medoid included - FrechetVariant frechet = FrechetVariant::Discrete; + FrechetVariant frechet = FrechetVariant::Continuous; }; using TrajectoryPair = std::pair; @@ -48,7 +48,7 @@ struct RouteClusteringWorkload { std::vector originals; double eps_m = 0.0; std::size_t min_size = 0; - FrechetVariant frechet = FrechetVariant::Discrete; + FrechetVariant frechet = FrechetVariant::Continuous; }; // Clusters the original database. A self join workload over the same trajectories, eps and diff --git a/engine/include/backend/engine/selection/rdp.hpp b/engine/include/backend/engine/selection/rdp.hpp index 3caaac2..ee0de39 100644 --- a/engine/include/backend/engine/selection/rdp.hpp +++ b/engine/include/backend/engine/selection/rdp.hpp @@ -11,4 +11,8 @@ namespace selection::rdp { std::vector simplify(const core::TrajectoryView &trajectory, std::size_t points_to_keep); +// Endpoints first, then points in importance order. Sorting the first b entries +// reproduces simplify(trajectory, b), for 2 <= b <= min(limit, size). +std::vector sequence(const core::TrajectoryView &trajectory, std::size_t limit); + } // namespace selection::rdp diff --git a/engine/include/backend/engine/selection/sed.hpp b/engine/include/backend/engine/selection/sed.hpp index 52f25d4..38c919a 100644 --- a/engine/include/backend/engine/selection/sed.hpp +++ b/engine/include/backend/engine/selection/sed.hpp @@ -16,4 +16,8 @@ double synchronous_distance(const core::TrajectoryView &trajectory, std::size_t std::vector simplify(const core::TrajectoryView &trajectory, std::size_t points_to_keep); +// Endpoints first, then points in importance order. Sorting the first b entries +// reproduces simplify(trajectory, b), for 2 <= b <= min(limit, size). +std::vector sequence(const core::TrajectoryView &trajectory, std::size_t limit); + } // namespace selection::sed diff --git a/engine/include/backend/engine/teacher/curves.hpp b/engine/include/backend/engine/teacher/curves.hpp new file mode 100644 index 0000000..87d52ec --- /dev/null +++ b/engine/include/backend/engine/teacher/curves.hpp @@ -0,0 +1,24 @@ +#pragma once + +#include +#include +#include + +namespace teacher { + +// One trajectory's value at b = 2 + k points. A step function, not necessarily concave or +// increasing +struct Curve { + std::vector value; + std::vector method; // the candidate that reached value[k] + + std::size_t max_budget() const { return value.size() + 1; } + double at(std::size_t budget) const { return value[budget - 2]; } +}; + +// Maximises sum curves[i].at(b_i) with sum b_i = total_budget, greedily on the best gain per point +// over the next 1 to 3 points, which steps over flat stretches. Throws if the budget cannot fit +std::vector allocate(const std::vector &curves, std::size_t total_budget, + double *value = nullptr); + +} // namespace teacher diff --git a/engine/include/backend/engine/teacher/greedy.hpp b/engine/include/backend/engine/teacher/greedy.hpp new file mode 100644 index 0000000..929c8e0 --- /dev/null +++ b/engine/include/backend/engine/teacher/greedy.hpp @@ -0,0 +1,37 @@ +#pragma once + +#include "backend/engine/teacher/objective.hpp" + +#include +#include +#include +#include + +namespace teacher { + +// BKH's workload-driven selector: from the endpoints and a seed, repeatedly keep the point that +// raises i's range and where-at terms most, then take the rest from fill, skipping kept points +class Greedy { + public: + Greedy(const Objective &objective, std::size_t i); + + // Exactly min(budget, n) points, sorted, endpoints and seed included + std::vector run(std::size_t budget, std::span seed, + std::span fill) const; + + // In the order kept: its first b points are run(b) + std::vector sequence(std::size_t limit, std::span seed, + std::span fill) const; + + private: + core::TrajectoryView m_trajectory; + std::vector> m_box_points; // points inside each of i's boxes + std::vector m_box_weight; // scaled J_T weight of each box + std::vector m_offsets; // boxes of each point, CSR + std::vector m_point_boxes; + std::vector m_probe_t, m_probe_x, m_probe_y; // i's where-at probes, by time + double m_probe_weight = 0; // scaled J_T weight of each probe + double m_min_tolerance = 0, m_max_tolerance = 0; +}; + +} // namespace teacher diff --git a/engine/include/backend/engine/teacher/objective.hpp b/engine/include/backend/engine/teacher/objective.hpp new file mode 100644 index 0000000..41b7065 --- /dev/null +++ b/engine/include/backend/engine/teacher/objective.hpp @@ -0,0 +1,257 @@ +#pragma once + +#include "backend/engine/core/simplification.hpp" +#include "backend/engine/core/trajectory_db.hpp" +#include "backend/engine/query/join.hpp" +#include "backend/engine/teacher/workloads.hpp" + +#include +#include +#include +#include +#include +#include + +namespace teacher { + +// Slack when pruning by an upper bound, so rounding cannot drop a candidate that would tie +inline constexpr double kBoundSlack = 1e-15; + +using core::KeptSets; + +// A kept set's coordinates, owned. Views are made on demand, so copies and moves stay safe +class PreparedTrajectory { + public: + PreparedTrajectory(const core::TrajectoryView &original, std::span kept); + // Invalidates earlier views + void assign(const core::TrajectoryView &original, std::span kept); + core::TrajectoryView view() const { return {m_id, m_t, {}, {}, m_x, m_y, m_box}; } + + private: + std::int32_t m_id; + std::vector m_t; + std::vector m_x, m_y; + core::BoundingBox m_box; +}; + +class PreparedTrajectories { + public: + PreparedTrajectories(const core::TrajectoryDb &db, const KeptSets &kept); + std::vector views() const; + + private: + std::vector m_trajectories; +}; + +inline constexpr std::size_t kTermCount = 4; // kNN, range, where-at, join +using TermWeights = std::array; + +struct ObjectiveConfig { + double min_tolerance_m = 25.0; // where-at ramp, as in the evaluator + double max_tolerance_m = 500.0; + bool all_join_pairs = true; // false: only the seen half + // Self: both sides compressed. HeldOut: each trajectory also queries its partners as original + query::JoinMode join_mode = query::JoinMode::Self; + bool held_out_term = false; // with Self: add the held-out form at join_eps_m as a term + // kNN ranked on the compressed database; false: per-trajectory check. Curves always use the + // latter + bool ranking_knn = false; + TermWeights term_weights{1.0, 1.0, 1.0, 1.0}; +}; + +// Throws std::invalid_argument unless finite, not negative and not all zero. Returns their sum +double check_term_weights(const TermWeights &weights); + +struct JoinLink { + std::uint32_t other; + std::uint32_t pair; // index into the objective's pair list + bool positive; + std::uint8_t term = 0; // join term the pair belongs to +}; + +// J_T: weighted mean of normalised kNN, range, where-at and join on the teacher's workloads, +// (metric - floor) / (1 - floor) with floor = endpoints only, like J4. +// range recall: some kept point still inside each box i is in +// where-at average-hit ramp over i's probes +// join macro F1 over candidate pairs, pooled over all join terms (thresholds, held-out form) +// Range, where-at and the per-trajectory kNN add up over trajectories; join and ranking kNN do +// not, and are tracked under one-at-a-time changes by JoinState and KnnState +class Objective { + public: + enum Term : std::size_t { Knn, Range, WhereAt, Join, TermCount }; + using Terms = std::array; + + // Keeps references to db and workloads + Objective(const core::TrajectoryDb &db, const TeacherWorkloads &workloads, + const ObjectiveConfig &config = {}); + + // i's scaled separable terms, 0 with endpoints only + double separable(std::size_t i, std::span kept) const; + double separable(std::size_t i, const core::TrajectoryView &candidate) const; + + // Approximate score of i alone: separable plus i's join F1 against fixed partners (empty = + // original; ignored when held out). Exact above beat, otherwise some value below beat + double independent_value(std::size_t i, const core::TrajectoryView &candidate, + std::span partners, + double beat = -std::numeric_limits::infinity()) const; + // independent_value with every kNN check a hit + double independent_bound(std::size_t i, const core::TrajectoryView &candidate, + std::span partners) const; + + // A bound followed by scoring without recomputing. Borrows the candidate's coordinates + struct IndependentScore { + double bound = 0.0; + + private: + friend class Objective; + std::size_t trajectory = 0; + core::TrajectoryView candidate{}; + Terms terms{}; + double join = 0.0; + }; + IndependentScore prepare_independent(std::size_t i, const core::TrajectoryView &candidate, + std::span partners) const; + double independent_value(const IndependentScore &score, + double beat = -std::numeric_limits::infinity()) const; + + // Exact J_T + double total(const KeptSets &kept) const; + + // Unscaled metrics + Terms metrics(const KeptSets &kept) const; + const Terms &floors() const { return m_floor; } + // Recall@k of the database kNN queries ranked on compressed, as in metrics with ranking_knn + double knn_recall(const core::TrajectoryDb &compressed) const; + double scale(Term m) const { return m_scale[m]; } + + std::span links(std::size_t i) const { return m_links[i]; } + std::size_t pair_count() const { return m_pair_i.size(); } + std::size_t positive_count() const { return m_positive_count; } + std::size_t slot(std::size_t t, std::size_t term) const { return t + term * m_db.size(); } + std::size_t slots() const { return m_truth.size(); } + std::size_t truth(std::size_t slot) const { return m_truth[slot]; } + bool held_out(std::size_t term) const { return m_term_held_out[term]; } + + struct RangeMembership { + std::uint32_t box; + double weight; // 1 / (|reference| * scored boxes) + }; + const core::TrajectoryDb &db() const { return m_db; } + const TeacherWorkloads &workloads() const { return m_workloads; } + const ObjectiveConfig &config() const { return m_config; } + std::span boxes(std::size_t i) const { return m_range[i]; } + std::span probes(std::size_t i) const { return m_probes[i]; } // where-at + double where_at_weight() const { return m_where_at_weight; } + + bool match(const core::TrajectoryView &a, const core::TrajectoryView &b, + std::size_t term = 0) const; + + private: + struct KnnMembership { + std::uint32_t query; + double cutoff; + }; + + Terms raw_separable(std::size_t i, std::span kept) const; + Terms raw_separable(std::size_t i, const core::TrajectoryView &candidate, + bool knn = true) const; + double independent_sum(const IndependentScore &score, double knn) const; + double join_metric(std::span state) const; + + const core::TrajectoryDb &m_db; + const TeacherWorkloads &m_workloads; + ObjectiveConfig m_config; + + std::vector> m_knn; + double m_knn_weight = 0; // 1 / (k * valid queries) + std::vector> m_range; + std::vector> m_probes; // where-at query indices per trajectory + double m_where_at_weight = 0; // 1 / trajectories with probes + + std::vector m_pair_i, m_pair_j; // pairs used, positives first + std::vector m_pair_term; + std::vector m_eps; // threshold of each join term + std::vector m_term_held_out; + std::size_t m_positive_count = 0; + std::vector> m_links; // per trajectory + std::vector m_truth; // positives per slot + std::size_t m_with_truth = 0; // slots with any positive + + Terms m_floor{}; + Terms m_scale{}; // weight / (weight sum * (1 - floor)) + std::vector m_endpoint_separable; // scaled separable terms, endpoints only +}; + +// The exact join metric under one-at-a-time changes; changing i only rescores i's pairs +class JoinState { + public: + JoinState(const Objective &objective, std::span state); + + struct Proposal { + std::size_t trajectory; + double gain = 0.0; + std::vector matched; // outcomes in links(trajectory) order + }; + + // Valid until the next apply() + Proposal propose(std::size_t i, const core::TrajectoryView &candidate, + std::span state) const; + void apply(const Proposal &proposal); + + double metric() const; // macro F1 over the pairs used + + private: + bool counted(std::size_t t, std::size_t found) const; + double f1(std::size_t t, std::size_t hits, std::size_t found) const; + std::size_t count(std::size_t t, const JoinLink &link) const { + return m_objective.slot(t, link.term); + } + // Where the outcome of i's link is stored. Held out: the direction with i compressed + std::size_t slot(std::size_t i, const JoinLink &link) const { + return 2 * std::size_t{link.pair} + (m_objective.held_out(link.term) && i > link.other); + } + // Held out: only the query's counts change + std::size_t sides(const JoinLink &link) const { + return m_objective.held_out(link.term) ? 1 : 2; + } + + const Objective &m_objective; + std::vector m_matched; // per pair and direction + std::vector m_hits, m_found; + double m_sum = 0; // sum of F1 over counted trajectories + std::size_t m_counted = 0; +}; + +// The ranking kNN metric under one-at-a-time changes. Each query watches its true top k and its +// nearest others in state and watch_states; unwatched trajectories are assumed to stay outside +class KnnState { + public: + KnnState(const Objective &objective, const KeptSets &state, + std::span watch_states, std::size_t watch); + + bool watched(std::size_t i) const { return !m_watchers[i].empty(); } + // From every query watching i, in the order propose and apply expect + std::vector distances(std::size_t i, const core::TrajectoryView &candidate) const; + // Change in J_T + double propose(std::size_t i, std::span distances) const; + void apply(std::size_t i, std::span distances); + + double metric() const; // recall@k over the valid queries + + private: + struct Watcher { + std::uint32_t query, position; // position in the query's list + }; + // True neighbours in e's top k, with the distance at position replaced + std::size_t hits(std::size_t e, std::size_t position, double distance) const; + + const Objective &m_objective; + std::size_t m_valid = 0; + std::vector> m_list; // per query, its true top k first + std::vector> m_distance; // per query, in list order + std::vector m_hits; // per query + std::size_t m_hit_sum = 0; + std::vector> m_watchers; // per trajectory +}; + +} // namespace teacher diff --git a/engine/include/backend/engine/teacher/teacher.hpp b/engine/include/backend/engine/teacher/teacher.hpp new file mode 100644 index 0000000..e96ca03 --- /dev/null +++ b/engine/include/backend/engine/teacher/teacher.hpp @@ -0,0 +1,89 @@ +#pragma once + +#include "backend/engine/core/simplification.hpp" +#include "backend/engine/core/trajectory_db.hpp" +#include "backend/engine/teacher/objective.hpp" +#include "backend/engine/teacher/workloads.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace teacher { + +// "Loop" is best response against the neighbours' current state until nothing switches. +// EQUAL_CHOICE equal budgets, best candidate each, partners original +// EQUAL_CHOICE_RDP_PARTNERS the same, partners RDP +// EQUAL_LOOP the loop from EQUAL_CHOICE_RDP_PARTNERS +// BUDGET_RDP budgets from value curves, RDP +// BUDGET_CHOICE budgets from value curves, best candidate each +// BUDGET_LOOP the loop from BUDGET_CHOICE: the teacher +// BUDGET_LOOP_REALLOCATE then curves, budgets and loop again while J_T rises. With only +// held-out join terms the curves ignore partners, so this only reruns +// the loop in a new visit order +// BUDGET_LOOP_SHUFFLED control: loop on relabelled partners and neighbours +// BUDGET_LOOP_NO_UPDATE control: one pass against the starting state +inline constexpr std::array kArms{"EQUAL_CHOICE", + "EQUAL_CHOICE_RDP_PARTNERS", + "EQUAL_LOOP", + "BUDGET_RDP", + "BUDGET_CHOICE", + "BUDGET_LOOP", + "BUDGET_LOOP_REALLOCATE", + "BUDGET_LOOP_SHUFFLED", + "BUDGET_LOOP_NO_UPDATE"}; + +bool is_arm(std::string_view name); +// Throws std::invalid_argument on an unknown or repeated arm +void check_arms(std::span arms); + +struct TeacherConfig { + bool enabled = false; + std::filesystem::path cache_dir = "cache"; + std::vector arms{"BUDGET_CHOICE", "BUDGET_LOOP"}; + std::size_t range_query_count = kRangeQueryCount; + bool all_join_pairs = true; // false: only the seen half of the join pairs + // Unset: join.mode + std::optional join; + std::vector extra_join_eps_m; + bool held_out_term = false; + bool ranking_knn = true; + TermWeights term_weights{1.0, 1.0, 1.0, 1.0}; +}; + +struct Arm { + std::string name; + core::Simplification simplification; + double objective = 0.0; // J_T +}; + +// Sees only the database it is given and workloads built from it +class Teacher { + public: + Teacher(const core::TrajectoryDb &db, TeacherWorkloads workloads, const ObjectiveConfig &config, + std::uint64_t seed); + Teacher(const Teacher &) = delete; + Teacher &operator=(const Teacher &) = delete; + + // The requested arms, in order + std::vector run(std::size_t total_budget, std::span arms) const; + + // RECIPE_: one recipe for every trajectory at equal budgets + std::vector single_recipes(std::size_t total_budget) const; + + const Objective &objective() const { return m_objective; } + + private: + const core::TrajectoryDb &m_db; + TeacherWorkloads m_workloads; // before m_objective, which refers to it + Objective m_objective; + std::uint64_t m_seed; +}; + +} // namespace teacher diff --git a/engine/include/backend/engine/teacher/workloads.hpp b/engine/include/backend/engine/teacher/workloads.hpp new file mode 100644 index 0000000..dce0223 --- /dev/null +++ b/engine/include/backend/engine/teacher/workloads.hpp @@ -0,0 +1,71 @@ +#pragma once + +#include "backend/engine/core/trajectory_db.hpp" +#include "backend/engine/query/frechet_helper.hpp" +#include "backend/engine/query/range_query.hpp" +#include "backend/engine/query/where_at_query.hpp" + +#include +#include +#include +#include + +namespace teacher { + +// Offsets added to the run seed, one per random draw, so no two draws share a stream +inline constexpr std::uint64_t kRangeSeedOffset = 7001; +inline constexpr std::uint64_t kWhereAtSeedOffset = 7002; +inline constexpr std::uint64_t kPairSplitSeedOffset = 7003; +inline constexpr std::uint64_t kOrderSeedOffset = 7004; +inline constexpr std::uint64_t kNegativeSeedOffset = 7005; +inline constexpr std::uint64_t kShuffleSeedOffset = 7006; + +inline constexpr std::size_t kRangeQueryCount = 64000; + +// Training workloads from the database alone, with their own seeds, never the held-out pool +struct WorkloadConfig { + std::size_t k = 10; + double join_eps_m = 150.0; + std::vector extra_join_eps_m; + query::FrechetVariant frechet = query::FrechetVariant::Continuous; + query::RangeConfig range{.query_count = kRangeQueryCount}; + query::WhereAtConfig where_at; +}; + +// A database trajectory used as a kNN query against the rest, itself excluded +struct KnnEntry { + bool valid = false; // false with fewer than k + 1 neighbours + std::vector top; // the k nearest, closest first + double cutoff = 0.0; // Fréchet distance to the (k + 1)th neighbour +}; + +struct JoinPair { + std::uint32_t i, j; // i < j + bool seen; // in the half used with all_join_pairs = false +}; + +// Self-join pairs at a threshold other than join_eps_m, all of them used +struct ExtraJoin { + double eps_m = 0.0; + std::vector positives, negatives; +}; + +struct TeacherWorkloads { + std::size_t k = 0; + double join_eps_m = 0.0; + query::FrechetVariant frechet = query::FrechetVariant::Continuous; + std::vector knn; // one per database trajectory + std::vector positives; // self-join pairs, half seen + std::vector negatives; // endpoint candidates that are not pairs, half seen + std::vector extra_joins; + query::RangeWorkload range; + query::WhereAtWorkload where_at; +}; + +// kNN and join pairs come from cache_dir/teacher_cache_seed.bin when it matches the database +// and settings, otherwise they are built (slow) and saved. The rest is regenerated every time +TeacherWorkloads load_or_build_workloads(const core::TrajectoryDb &db, const WorkloadConfig &config, + std::uint64_t seed, + const std::filesystem::path &cache_dir); + +} // namespace teacher diff --git a/engine/src/allocation/budget_allocator.cpp b/engine/src/allocation/budget_allocator.cpp index 3ee5c5c..449df36 100644 --- a/engine/src/allocation/budget_allocator.cpp +++ b/engine/src/allocation/budget_allocator.cpp @@ -3,6 +3,7 @@ #include "backend/engine/allocation/budget_allocator.hpp" #include "backend/engine/allocation/size_proportional.hpp" #include "backend/engine/core/trajectory_db.hpp" +#include #include #include @@ -18,4 +19,10 @@ std::vector create_trajectory_budgets(const core::TrajectoryDb &tra } throw std::invalid_argument("create_trajectory_budgets: unknown allocator"); } + +std::size_t total_budget(const core::TrajectoryDb &trajectories, double share) { + const std::size_t endpoints = 2 * trajectories.size(); + const double extra = share * static_cast(trajectories.num_points() - endpoints); + return endpoints + static_cast(std::llround(extra)); +} } // namespace allocation diff --git a/engine/src/core/simplification.cpp b/engine/src/core/simplification.cpp index 1ad7033..8278c99 100644 --- a/engine/src/core/simplification.cpp +++ b/engine/src/core/simplification.cpp @@ -7,6 +7,32 @@ namespace core { +Simplification to_simplification(const KeptSets &kept) { + Simplification simplification; + for (const auto &k : kept) + simplification.add_trajectory(k); + return simplification; +} + +KeptSets to_kept(const Simplification &simplification) { + KeptSets kept(simplification.size()); + for (std::size_t i = 0; i < simplification.size(); ++i) + kept[i].assign(simplification.trajectory(i).begin(), simplification.trajectory(i).end()); + return kept; +} + +KeptSets endpoints_only(const TrajectoryDb &trajectories) { + KeptSets kept(trajectories.size()); + for (std::size_t i = 0; i < trajectories.size(); ++i) { + const std::size_t n = trajectories.trajectory(i).size(); + if (n > 0) + kept[i].push_back(0); + if (n > 1) + kept[i].push_back(n - 1); + } + return kept; +} + TrajectoryDb materialize(const TrajectoryDb &trajectories, const Simplification &simplification) { if (simplification.size() != trajectories.size()) throw std::invalid_argument("materialize: simplification and database sizes differ"); @@ -53,4 +79,8 @@ TrajectoryDb materialize(const TrajectoryDb &trajectories, const Simplification return result; } +TrajectoryDb materialize(const TrajectoryDb &trajectories, const KeptSets &kept) { + return materialize(trajectories, to_simplification(kept)); +} + } // namespace core diff --git a/engine/src/evaluator/combined.cpp b/engine/src/evaluator/combined.cpp index 63d6fa2..e2a7316 100644 --- a/engine/src/evaluator/combined.cpp +++ b/engine/src/evaluator/combined.cpp @@ -1,15 +1,20 @@ #include "backend/engine/evaluator/combined.hpp" +#include #include #include -#include #include +#include +#include +#include +#include +#include namespace evaluator { namespace { -double mean_ignoring_nan(std::initializer_list values) { +double mean_ignoring_nan(std::span values) { double sum = 0.0; std::size_t count = 0; for (const double value : values) { @@ -21,6 +26,20 @@ double mean_ignoring_nan(std::initializer_list values) { return count == 0 ? std::numeric_limits::quiet_NaN() : sum / static_cast(count); } +// The main normalised score averaged with each grid entry, normalised against its own floor, +// skipping NaNs. raw(entry) reads an entry's unnormalised score +template +double grid_mean(double main, const std::vector &scores, const std::vector &floors, + Raw raw, std::string_view grid) { + if (scores.size() != floors.size()) + throw std::invalid_argument("normalize_scores: scores and floor hold different " + + std::string(grid) + " grids"); + std::vector values{main}; + for (std::size_t k = 0; k < scores.size(); ++k) + values.push_back(normalize(raw(scores[k]), raw(floors[k]))); + return mean_ignoring_nan(values); +} + } // namespace double normalize(double score, double floor) { @@ -46,11 +65,17 @@ NormalizedScores normalize_scores(const Scoretable &scores, const Scoretable &fl result.knn = result.knn_recall_at_k; result.range = result.range_f1; result.where_at = result.where_at_average_hit; - result.join = result.join_f1; - result.clustering = result.clustering_score; + result.join = grid_mean( + result.join_f1, scores.join_grid, floor.join_grid, + [](const JoinScores &join) { return join.f1; }, "join"); + result.clustering = grid_mean( + result.clustering_score, scores.clustering_grid, floor.clustering_grid, + [](const RouteClusteringScores &clustering) { return clustering.score; }, "clustering"); + result.self_join = normalize(scores.self_join.f1, floor.self_join.f1); - result.combined = mean_ignoring_nan( - {result.knn, result.range, result.where_at, result.join, result.clustering}); + // Clustering and the self join stay out (see combined.hpp) + result.combined = + mean_ignoring_nan(std::array{result.knn, result.range, result.where_at, result.join}); return result; } diff --git a/engine/src/evaluator/evaluator.cpp b/engine/src/evaluator/evaluator.cpp index 94f93bc..b79504c 100644 --- a/engine/src/evaluator/evaluator.cpp +++ b/engine/src/evaluator/evaluator.cpp @@ -72,12 +72,25 @@ EvaluationResult Evaluator::evaluate() const { } } + std::vector join_grid; + for (const auto &w : m_workloads.join_grid) + join_grid.push_back(evaluate_join(m_simplified, w)); + std::vector clustering_grid; + for (const auto &w : m_workloads.clustering_grid) + clustering_grid.push_back(evaluate_route_clustering(m_simplified, w)); + return {.scores = {.knn = knn.scores, .range_recall = range.recall, .where_at = get_where_at_scores(), .join = join.scores, .range_f1 = range.f1, - .clustering = clustering}, + .clustering = clustering, + // The same workload is not evaluated twice + .self_join = m_workloads.self_join == m_workloads.join + ? join.scores + : get_self_join_scores(), + .join_grid = std::move(join_grid), + .clustering_grid = std::move(clustering_grid)}, .knn = std::move(knn.answers), .range = std::move(range.answers), .join = std::move(join.pairs), @@ -104,6 +117,11 @@ JoinScores Evaluator::get_join_scores() const { return m_workloads.join ? evaluate_join(m_simplified, *m_workloads.join) : JoinScores{}; } +JoinScores Evaluator::get_self_join_scores() const { + return m_workloads.self_join ? evaluate_join(m_simplified, *m_workloads.self_join) + : JoinScores{}; +} + RouteClusteringScores Evaluator::get_clustering_scores() const { return m_workloads.clustering ? evaluate_route_clustering(m_simplified, *m_workloads.clustering) : RouteClusteringScores{}; diff --git a/engine/src/pipeline/config.cpp b/engine/src/pipeline/config.cpp index 1bd1a2b..0fbe693 100644 --- a/engine/src/pipeline/config.cpp +++ b/engine/src/pipeline/config.cpp @@ -90,11 +90,15 @@ void parse_where_at_scoring_config(Config &config, const json &scoring) { } void parse_join_config(Config &config, const json &join) { - validate_keys(join, {"query_count", "eps_m", "mode"}, "join"); + validate_keys(join, {"query_count", "eps_m", "mode", "self_join", "score_eps_m"}, "join"); + config.self_join = join.value("self_join", config.self_join); if (join.contains("mode")) config.join.mode = parse_enum(join["mode"]); config.join.query_count = join.value("query_count", config.join.query_count); config.join.eps_m = join.value("eps_m", config.join.eps_m); + config.join_score_eps_m = join.value("score_eps_m", config.join_score_eps_m); + query::check_extra_thresholds(config.join_score_eps_m, config.join.eps_m, + "config: join.score_eps_m"); } void parse_clustering_config(Config &config, const json &clustering) { @@ -110,11 +114,40 @@ void parse_clustering_config(Config &config, const json &clustering) { } } +void parse_teacher_config(Config &config, const json §ion) { + validate_keys(section, + {"enabled", "cache_dir", "arms", "range_query_count", "all_join_pairs", "join", + "extra_join_eps_m", "held_out_term", "ranking_knn", "term_weights"}, + "teacher"); + auto &t = config.teacher; + t.enabled = section.value("enabled", t.enabled); + t.cache_dir = section.value("cache_dir", t.cache_dir.string()); + t.arms = section.value("arms", t.arms); + t.range_query_count = section.value("range_query_count", t.range_query_count); + t.all_join_pairs = section.value("all_join_pairs", t.all_join_pairs); + if (section.contains("join")) + t.join = parse_enum(section["join"]); + t.extra_join_eps_m = section.value("extra_join_eps_m", t.extra_join_eps_m); + t.held_out_term = section.value("held_out_term", t.held_out_term); + t.ranking_knn = section.value("ranking_knn", t.ranking_knn); + t.term_weights = section.value("term_weights", t.term_weights); + teacher::check_term_weights(t.term_weights); + query::check_extra_thresholds(t.extra_join_eps_m, config.join.eps_m, + "config: teacher.extra_join_eps_m"); + if (t.cache_dir.empty()) + throw std::invalid_argument("config: teacher.cache_dir must not be empty"); + if (t.enabled && t.arms.empty()) + throw std::invalid_argument("config: teacher.arms must not be empty when enabled"); + teacher::check_arms(t.arms); + if (t.range_query_count == 0) + throw std::invalid_argument("config: teacher.range_query_count must be positive"); +} + Config parse_config(const json &data) { validate_keys(data, {"data_path", "export_results", "export_dir", "seed", "budget_shares", "allocation_alphas", "algorithms", "split", "knn", "range", "where_at", - "where_at_scoring", "join", "clustering", "frechet"}, + "where_at_scoring", "join", "clustering", "frechet", "teacher"}, "root"); Config config; @@ -149,6 +182,7 @@ Config parse_config(const json &data) { parse_where_at_scoring_config(config, data.value("where_at_scoring", json::object())); parse_join_config(config, data.value("join", json::object())); parse_clustering_config(config, data.value("clustering", json::object())); + parse_teacher_config(config, data.value("teacher", json::object())); // One distance for every Fréchet-based query, so both sides of each comparison agree if (data.contains("frechet")) { const auto variant = parse_enum(data["frechet"]); @@ -175,4 +209,26 @@ Config load_config(const std::filesystem::path &path) { } } +teacher::WorkloadConfig teacher_workload_config(const Config &config) { + auto range = config.range; + range.query_count = config.teacher.range_query_count; + return {.k = config.knn.k, + .join_eps_m = config.join.eps_m, + .extra_join_eps_m = config.teacher.extra_join_eps_m, + .frechet = config.join.frechet, + .range = range, + .where_at = config.where_at}; +} + +teacher::ObjectiveConfig teacher_objective_config(const Config &config) { + return {.min_tolerance_m = config.where_at_scoring.min_tolerance_m, + .max_tolerance_m = config.where_at_scoring.max_tolerance_m, + .all_join_pairs = config.teacher.all_join_pairs, + // Trains on the kind of join it is scored on, unless told otherwise + .join_mode = config.teacher.join.value_or(config.join.mode), + .held_out_term = config.teacher.held_out_term, + .ranking_knn = config.teacher.ranking_knn, + .term_weights = config.teacher.term_weights}; +} + } // namespace pipeline diff --git a/engine/src/pipeline/pipeline.cpp b/engine/src/pipeline/pipeline.cpp index cf560f3..eb8491a 100644 --- a/engine/src/pipeline/pipeline.cpp +++ b/engine/src/pipeline/pipeline.cpp @@ -1,18 +1,14 @@ #include "backend/engine/pipeline/pipeline.hpp" #include "backend/engine/allocation/budget_allocator.hpp" -#include "backend/engine/core/simplification.hpp" #include "backend/engine/core/trajectory_db.hpp" -#include "backend/engine/evaluator/combined.hpp" -#include "backend/engine/evaluator/evaluator.hpp" #include "backend/engine/io/parquet_loader.hpp" #include "backend/engine/io/workload_export.hpp" -#include "backend/engine/query/join.hpp" -#include "backend/engine/query/knn.hpp" -#include "backend/engine/query/range_query.hpp" -#include "backend/engine/query/route_clustering.hpp" -#include "backend/engine/query/where_at_query.hpp" +#include "backend/engine/pipeline/scorer.hpp" +#include "backend/engine/pipeline/test_workloads.hpp" #include "backend/engine/selection/selector.hpp" #include "backend/engine/split/split.hpp" +#include "backend/engine/teacher/teacher.hpp" +#include "backend/engine/teacher/workloads.hpp" #include @@ -20,9 +16,8 @@ #include #include #include +#include #include -#include -#include namespace pipeline { @@ -40,53 +35,24 @@ Databases load_split(const Config &config) { .queries = core::subset(all, parts.queries)}; } -void print_size(std::string_view name, const core::TrajectoryDb &trajectories) { - std::cout << name << ": " << trajectories.size() << " trajectories, " - << trajectories.num_points() << " points\n"; +// alpha = 1 keeps the old name, which the Travis frontend expects +std::string baseline_name(selection::AlgorithmType algorithm, double share, double alpha) { + std::string name = std::format("{}_{:g}pct", magic_enum::enum_name(algorithm), share * 100); + if (alpha != 1.0) + name += std::format("_a{:g}", alpha); + return name; } -std::size_t total_budget(const core::TrajectoryDb &trajectories, double share) { - const std::size_t endpoints = 2 * trajectories.size(); - const double extra = share * static_cast(trajectories.num_points() - endpoints); - return endpoints + static_cast(std::llround(extra)); -} - -void print_scores(const evaluator::Scoretable &scores) { - std::cout << " knn top-k tau: " << scores.knn.kendalls_tau_topk - << ", recall@k: " << scores.knn.recall_at_k << " (" << scores.knn.query_count - << " queries)\n"; - std::cout << " where-at: average hit=" << scores.where_at.average_hit - << ", within radius=" << scores.where_at.within_radius << '\n'; - std::cout << " range F1: " << scores.range_f1 << ", recall: " << scores.range_recall << '\n'; - std::cout << " join F1: " << scores.join.f1 << ", precision: " << scores.join.precision - << ", recall: " << scores.join.recall << ", micro F1: " << scores.join.micro_f1 - << '\n'; - if (!std::isnan(scores.clustering.score) || scores.clustering.reference_clusters != 0) - std::cout << " route clustering: " << scores.clustering.score << " (" - << scores.clustering.predicted_clusters << " clusters, " - << scores.clustering.predicted_covered << " trajectories)\n"; -} - -void print_normalized(const evaluator::NormalizedScores &scores) { - std::cout << " normalized: knn=" << scores.knn << ", range=" << scores.range - << ", where-at=" << scores.where_at << ", join=" << scores.join; - if (!std::isnan(scores.clustering)) - std::cout << ", clustering=" << scores.clustering; - std::cout << ", combined=" << scores.combined << '\n'; -} - -core::Simplification endpoints_only(const core::TrajectoryDb &trajectories) { - core::Simplification simplification; - for (std::size_t i = 0; i < trajectories.size(); ++i) { - const std::size_t n = trajectories.trajectory(i).size(); - std::vector kept; - if (n > 0) - kept.push_back(0); - if (n > 1) - kept.push_back(n - 1); - simplification.add_trajectory(kept); - } - return simplification; +// Sees the database only. Null unless enabled +std::unique_ptr make_teacher(const core::TrajectoryDb &database, + const Config &config) { + if (!config.teacher.enabled) + return nullptr; + return std::make_unique( + database, + teacher::load_or_build_workloads(database, teacher_workload_config(config), config.seed, + config.teacher.cache_dir), + teacher_objective_config(config), config.seed); } } // namespace @@ -96,86 +62,36 @@ void run(const Config &config) { print_size("database", database); print_size("query pool", queries); - const auto knn_workload = - query::generate_knn_queries(database, queries, config.knn, config.seed); - std::cout << "knn: " << knn_workload.entries.size() << " queries, k=" << knn_workload.k << '\n'; - const auto where_at = query::generate_where_at_queries(database, config.where_at, config.seed); - std::cout << "where-at: " << where_at.queries.size() << " probes, " - << where_at.skipped_trajectories << " trips without eligible time\n"; - const auto range = query::generate_range_queries(database, config.range, config.seed); - std::cout << "range: " << range.queries.size() << " queries\n"; - const auto join = query::generate_join_queries(database, queries, config.join, config.seed); - std::size_t join_pairs = 0; - for (const auto &entry : join.entries) - join_pairs += entry.reference.size(); - std::cout << "join " << magic_enum::enum_name(join.mode) << ": " << join.entries.size() - << (join.mode == query::JoinMode::Self ? " database trajectories, eps=" - : " queries, eps=") - << join.eps_m << " m, " << join_pairs << " matching pairs\n"; - - query::RouteClusteringWorkload clustering; - if (config.clustering.enabled) { - clustering = query::generate_route_clustering(database, config.clustering, &join); - std::size_t covered = 0; - for (const auto &members : clustering.reference.members) - covered += members.size(); - std::cout << "route clustering: " << clustering.reference.members.size() - << " clusters covering " << covered << " trajectories, eps=" << clustering.eps_m - << " m, min size " << clustering.min_size << '\n'; - } - + const TestWorkloads workloads(database, queries, config); if (config.export_results) { - io::export_workloads(config.export_dir, database, range.queries, knn_workload, join, - config.clustering.enabled ? &clustering : nullptr); + io::export_workloads(config.export_dir, database, workloads.range.queries, workloads.knn, + workloads.join, + workloads.clustering ? &*workloads.clustering : nullptr); std::cout << "workloads written to " << config.export_dir.string() << '\n'; } - const evaluator::Workloads workloads{.knn = &knn_workload, - .where_at = where_at.queries, - .range = range.queries, - .join = &join, - .clustering = - config.clustering.enabled ? &clustering : nullptr}; - - const auto floor_db = core::materialize(database, endpoints_only(database)); - std::cout << "floor, endpoints only: "; - print_size("compressed", floor_db); - const auto floor = evaluator::Evaluator(database, floor_db, workloads, config.where_at_scoring) - .evaluate() - .scores; - print_scores(floor); + const Scorer scorer(database, workloads, config); + const auto teacher = make_teacher(database, config); for (const double share : config.budget_shares) { + const std::size_t budget = allocation::total_budget(database, share); + for (const double alpha : config.allocation_alphas) { const auto budgets = allocation::create_trajectory_budgets( - database, allocation::AllocatorType::SizeProportional, - total_budget(database, share), alpha); - - for (const auto algorithm : config.algorithms) { - const auto simplification = - selection::simplify_database(database, algorithm, budgets); - const auto compressed = core::materialize(database, simplification); - std::cout << share * 100 << "% " << magic_enum::enum_name(algorithm) - << " alpha=" << alpha << ": "; - print_size("compressed", compressed); - - const evaluator::Evaluator evaluator(database, compressed, workloads, - config.where_at_scoring); - const auto evaluation = evaluator.evaluate(); - print_scores(evaluation.scores); - print_normalized(evaluator::normalize_scores(evaluation.scores, floor)); - - if (config.export_results) { - std::string name = - std::format("{}_{:g}pct", magic_enum::enum_name(algorithm), share * 100); - // alpha = 1 keeps the old name, which the Travis frontend expects - if (alpha != 1.0) - name += std::format("_a{:g}", alpha); - io::export_trajectories(config.export_dir, name, compressed); - io::export_results(config.export_dir, name, evaluation, - config.clustering.enabled); - } - } + database, allocation::AllocatorType::SizeProportional, budget, alpha); + for (const auto algorithm : config.algorithms) + scorer.score(std::format("{}% {} alpha={}", share * 100, + magic_enum::enum_name(algorithm), alpha), + baseline_name(algorithm, share, alpha), + selection::simplify_database(database, algorithm, budgets)); + } + + if (teacher) { + std::cout << "teacher at " << share * 100 << "%\n"; + for (const auto &arm : teacher->run(budget, config.teacher.arms)) + scorer.score( + std::format("{}% {} (J_T {:.4f})", share * 100, arm.name, arm.objective), + std::format("{}_{:g}pct", arm.name, share * 100), arm.simplification); } } } diff --git a/engine/src/pipeline/scorer.cpp b/engine/src/pipeline/scorer.cpp new file mode 100644 index 0000000..2ad2b83 --- /dev/null +++ b/engine/src/pipeline/scorer.cpp @@ -0,0 +1,77 @@ +#include "backend/engine/pipeline/scorer.hpp" +#include "backend/engine/evaluator/combined.hpp" +#include "backend/engine/io/workload_export.hpp" + +#include +#include + +namespace pipeline { + +namespace { + +void print_scores(const evaluator::Scoretable &scores) { + std::cout << " knn top-k tau: " << scores.knn.kendalls_tau_topk + << ", recall@k: " << scores.knn.recall_at_k << " (" << scores.knn.query_count + << " queries)\n"; + std::cout << " where-at: average hit=" << scores.where_at.average_hit + << ", within radius=" << scores.where_at.within_radius << '\n'; + std::cout << " range F1: " << scores.range_f1 << ", recall: " << scores.range_recall << '\n'; + std::cout << " join F1: " << scores.join.f1 << ", precision: " << scores.join.precision + << ", recall: " << scores.join.recall << ", micro F1: " << scores.join.micro_f1 + << '\n'; + if (scores.join.mode != query::JoinMode::Self && scores.self_join.query_count != 0) + std::cout << " self join F1: " << scores.self_join.f1 + << ", precision: " << scores.self_join.precision + << ", recall: " << scores.self_join.recall << '\n'; + if (!std::isnan(scores.clustering.score) || scores.clustering.reference_clusters != 0) + std::cout << " route clustering: " << scores.clustering.score << " (" + << scores.clustering.predicted_clusters << " clusters, " + << scores.clustering.predicted_covered << " trajectories)\n"; +} + +void print_normalized(const evaluator::NormalizedScores &scores) { + std::cout << " normalized: knn=" << scores.knn << ", range=" << scores.range + << ", where-at=" << scores.where_at << ", join=" << scores.join; + if (!std::isnan(scores.clustering)) + std::cout << ", clustering=" << scores.clustering; + if (!std::isnan(scores.self_join)) + std::cout << ", self join=" << scores.self_join; + std::cout << ", combined J4=" << scores.combined << '\n'; +} + +} // namespace + +void print_size(std::string_view name, const core::TrajectoryDb &trajectories) { + std::cout << name << ": " << trajectories.size() << " trajectories, " + << trajectories.num_points() << " points\n"; +} + +Scorer::Scorer(const core::TrajectoryDb &database, const TestWorkloads &workloads, + const Config &config) + : m_database(database), m_workloads(workloads.view()), m_config(config) { + const auto floor_db = core::materialize(database, core::endpoints_only(database)); + std::cout << "floor, endpoints only: "; + print_size("compressed", floor_db); + m_floor = evaluator::Evaluator(database, floor_db, m_workloads, config.where_at_scoring) + .evaluate() + .scores; + print_scores(m_floor); +} + +void Scorer::score(std::string_view label, std::string_view name, + const core::Simplification &simplification) const { + const auto compressed = core::materialize(m_database, simplification); + std::cout << label << ": "; + print_size("compressed", compressed); + const auto evaluation = + evaluator::Evaluator(m_database, compressed, m_workloads, m_config.where_at_scoring) + .evaluate(); + print_scores(evaluation.scores); + print_normalized(evaluator::normalize_scores(evaluation.scores, m_floor)); + if (m_config.export_results) { + io::export_trajectories(m_config.export_dir, name, compressed); + io::export_results(m_config.export_dir, name, evaluation, m_config.clustering.enabled); + } +} + +} // namespace pipeline diff --git a/engine/src/pipeline/test_workloads.cpp b/engine/src/pipeline/test_workloads.cpp new file mode 100644 index 0000000..45427f6 --- /dev/null +++ b/engine/src/pipeline/test_workloads.cpp @@ -0,0 +1,129 @@ +#include "backend/engine/pipeline/test_workloads.hpp" + +#include + +#include +#include + +namespace pipeline { + +namespace { + +query::KnnWorkload make_knn(const core::TrajectoryDb &database, const core::TrajectoryDb &queries, + const Config &config) { + auto knn = query::generate_knn_queries(database, queries, config.knn, config.seed); + std::cout << "knn: " << knn.entries.size() << " queries, k=" << knn.k << '\n'; + return knn; +} + +query::WhereAtWorkload make_where_at(const core::TrajectoryDb &database, const Config &config) { + auto where_at = query::generate_where_at_queries(database, config.where_at, config.seed); + std::cout << "where-at: " << where_at.queries.size() << " probes, " + << where_at.skipped_trajectories << " trips without eligible time\n"; + return where_at; +} + +query::RangeWorkload make_range(const core::TrajectoryDb &database, const Config &config) { + auto range = query::generate_range_queries(database, config.range, config.seed); + std::cout << "range: " << range.queries.size() << " queries\n"; + return range; +} + +void print_join(const query::JoinWorkload &join) { + std::size_t pairs = 0; + for (const auto &entry : join.entries) + pairs += entry.reference.size(); + std::cout << "join " << magic_enum::enum_name(join.mode) << ": " << join.entries.size() + << (join.mode == query::JoinMode::Self ? " database trajectories, eps=" + : " queries, eps=") + << join.eps_m << " m, " << pairs << " matching pairs\n"; +} + +query::JoinWorkload make_join(const core::TrajectoryDb &database, const core::TrajectoryDb &queries, + const Config &config) { + auto join = query::generate_join_queries(database, queries, config.join, config.seed); + print_join(join); + return join; +} + +std::vector make_join_grid(const core::TrajectoryDb &database, + const core::TrajectoryDb &queries, + const Config &config) { + std::vector grid; + for (const double eps : config.join_score_eps_m) { + auto join_config = config.join; + join_config.eps_m = eps; + grid.push_back(query::generate_join_queries(database, queries, join_config, config.seed)); + print_join(grid.back()); + } + return grid; +} + +// Empty when the join already is the self join +std::optional make_self_join(const core::TrajectoryDb &database, + const Config &config) { + if (!config.self_join || config.join.mode == query::JoinMode::Self) + return std::nullopt; + auto join = query::generate_self_join(database, config.join.eps_m, config.join.frechet); + print_join(join); + return join; +} + +std::optional make_clustering(const core::TrajectoryDb &database, + const query::JoinWorkload &join, + const Config &config) { + if (!config.clustering.enabled) + return std::nullopt; + auto clustering = query::generate_route_clustering(database, config.clustering, &join); + std::size_t covered = 0; + for (const auto &members : clustering.reference.members) + covered += members.size(); + std::cout << "route clustering: " << clustering.reference.members.size() + << " clusters covering " << covered << " trajectories, eps=" << clustering.eps_m + << " m, min size " << clustering.min_size << '\n'; + return clustering; +} + +std::vector make_clustering_grid(const core::TrajectoryDb &database, + const Config &config) { + std::vector grid; + if (!config.clustering.enabled) + return grid; + for (const double eps : config.join_score_eps_m) { + auto clustering = config.clustering; + clustering.eps_m = eps; + grid.push_back(query::generate_route_clustering(database, clustering)); + std::cout << "route clustering: " << grid.back().reference.members.size() + << " clusters, eps=" << eps << " m\n"; + } + return grid; +} + +} // namespace + +TestWorkloads::TestWorkloads(const core::TrajectoryDb &database, const core::TrajectoryDb &queries, + const Config &config) + : knn(make_knn(database, queries, config)), where_at(make_where_at(database, config)), + range(make_range(database, config)), join(make_join(database, queries, config)), + join_grid(make_join_grid(database, queries, config)), + self_join(make_self_join(database, config)), + clustering(make_clustering(database, self_join ? *self_join : join, config)), + clustering_grid(make_clustering_grid(database, config)) {} + +evaluator::Workloads TestWorkloads::view() const { + const query::JoinWorkload *self = nullptr; + if (self_join) + self = &*self_join; + else if (join.mode == query::JoinMode::Self) + self = &join; + return {.knn = &knn, + .where_at = where_at.queries, + .range = range.queries, + .join = &join, + .clustering = clustering ? &*clustering : nullptr, + .self_join = self, + .join_grid = join_grid, + .clustering_grid = clustering_grid}; +} + +} // namespace pipeline diff --git a/engine/src/query/continuous_frechet.cpp b/engine/src/query/continuous_frechet.cpp index 7d40645..f7fd501 100644 --- a/engine/src/query/continuous_frechet.cpp +++ b/engine/src/query/continuous_frechet.cpp @@ -201,4 +201,12 @@ bool frechet_within(const core::TrajectoryView &a, const core::TrajectoryView &b : frechet_within(a, b, max_dist); } +double frechet_distance(const core::TrajectoryView &a, const core::TrajectoryView &b, + FrechetVariant variant) { + const double discrete = frechet_distance(a, b); + // Discrete Fréchet bounds the continuous distance from above + return variant == FrechetVariant::Continuous ? continuous_frechet_distance(a, b, 0.0, discrete) + : discrete; +} + } // namespace query diff --git a/engine/src/query/join.cpp b/engine/src/query/join.cpp index fc648b1..7bdbe1e 100644 --- a/engine/src/query/join.cpp +++ b/engine/src/query/join.cpp @@ -9,6 +9,7 @@ #include #include #include +#include #include namespace query { @@ -106,4 +107,14 @@ JoinWorkload generate_self_join(const core::TrajectoryDb &original, double eps_m return result; } +void check_extra_thresholds(std::span eps_m, double main_eps_m, + std::string_view name) { + for (auto eps = eps_m.begin(); eps != eps_m.end(); ++eps) + if (!std::isfinite(*eps) || *eps <= 0 || *eps == main_eps_m || + std::find(eps_m.begin(), eps, *eps) != eps) + throw std::invalid_argument(std::string(name) + + " must be positive, distinct and differ from the main " + "join threshold"); +} + } // namespace query diff --git a/engine/src/selection/rdp.cpp b/engine/src/selection/rdp.cpp index 1543612..5ef0407 100644 --- a/engine/src/selection/rdp.cpp +++ b/engine/src/selection/rdp.cpp @@ -51,40 +51,37 @@ void push_split(std::priority_queue &heap, std::span x, } // namespace -std::vector simplify(const core::TrajectoryView &trajectory, - std::size_t points_to_keep) { +std::vector sequence(const core::TrajectoryView &trajectory, std::size_t limit) { const std::size_t n = trajectory.size(); - const auto &x = trajectory.x; - const auto &y = trajectory.y; - - if (points_to_keep >= n) { - std::vector all_points_kept(n); - for (std::size_t i = 0; i < n; ++i) - all_points_kept[i] = i; - return all_points_kept; - } - std::vector kept(n, false); - kept.front() = kept.back() = true; - std::size_t num_kept = 2; - + if (n == 0) + return {}; + if (n == 1) + return {0}; + limit = std::clamp(limit, std::size_t{2}, n); + std::vector indices{0, n - 1}; + indices.reserve(limit); std::priority_queue heap; - - push_split(heap, x, y, 0, n - 1); - while (num_kept < points_to_keep && !heap.empty()) { + push_split(heap, trajectory.x, trajectory.y, 0, n - 1); + while (indices.size() < limit && !heap.empty()) { const Split split = heap.top(); heap.pop(); - kept[split.index] = true; - ++num_kept; - push_split(heap, x, y, split.first, split.index); - push_split(heap, x, y, split.index, split.last); + indices.push_back(split.index); + push_split(heap, trajectory.x, trajectory.y, split.first, split.index); + push_split(heap, trajectory.x, trajectory.y, split.index, split.last); } + return indices; +} - std::vector indices; - indices.reserve(num_kept); - for (std::size_t i = 0; i < n; ++i) { - if (kept[i]) - indices.push_back(i); +std::vector simplify(const core::TrajectoryView &trajectory, + std::size_t points_to_keep) { + if (points_to_keep >= trajectory.size()) { + std::vector indices(trajectory.size()); + for (std::size_t i = 0; i < indices.size(); ++i) + indices[i] = i; + return indices; } + auto indices = sequence(trajectory, points_to_keep); + std::ranges::sort(indices); return indices; } diff --git a/engine/src/selection/sed.cpp b/engine/src/selection/sed.cpp index aaa5781..8136537 100644 --- a/engine/src/selection/sed.cpp +++ b/engine/src/selection/sed.cpp @@ -73,8 +73,11 @@ double synchronous_distance(const core::TrajectoryView &trajectory, std::size_t return std::sqrt(squared_synchronous_distance(trajectory, first, last, point)); } -std::vector simplify(const core::TrajectoryView &trajectory, - std::size_t points_to_keep) { +namespace { + +std::vector simplify_impl(const core::TrajectoryView &trajectory, + std::size_t points_to_keep, + std::vector *removed) { const std::size_t n = trajectory.size(); if (points_to_keep >= n || n <= 2) { @@ -108,6 +111,8 @@ std::vector simplify(const core::TrajectoryView &trajectory, points[prev].next = next; points[next].prev = prev; kept_points--; + if (removed) + removed->push_back(top.index); for (const std::size_t curr : {prev, next}) { if (curr == 0 || curr == n - 1) @@ -128,4 +133,25 @@ std::vector simplify(const core::TrajectoryView &trajectory, return simplified_trajectory; } +} // namespace + +std::vector simplify(const core::TrajectoryView &trajectory, + std::size_t points_to_keep) { + return simplify_impl(trajectory, points_to_keep, nullptr); +} + +std::vector sequence(const core::TrajectoryView &trajectory, std::size_t limit) { + const auto n = trajectory.size(); + if (n <= 2) + return simplify(trajectory, n); + limit = std::clamp(limit, std::size_t{2}, n); + std::vector removed; + removed.reserve(n - 2); + auto indices = simplify_impl(trajectory, 2, &removed); + indices.reserve(limit); + // Reverse removal order: the last removed point is the first to restore above budget two. + indices.insert(indices.end(), removed.rbegin(), removed.rbegin() + (limit - 2)); + return indices; +} + } // namespace selection::sed diff --git a/engine/src/teacher/README.md b/engine/src/teacher/README.md new file mode 100644 index 0000000..28b7b09 --- /dev/null +++ b/engine/src/teacher/README.md @@ -0,0 +1,720 @@ +# The teacher + +The teacher is a slow, query-aware simplifier. It searches how many points each trajectory gets +and which candidate keeps them: RDP, SED, UNIFORM or one of three greedy variants. Its output is +meant to be a target that a learned model can later imitate. + +It sees **only the database** (the 85% split) and workloads built from it with their own seeds. It +never sees the held-out query pool that the pipeline scores on. The fixed-budget arms use equal +shares (α = 0); the allocation arms redistribute the same total point budget. + +## How it fits together + +``` +pipeline::run (pipeline.cpp) + ├─ load_or_build_workloads(database, ...) workloads.cpp: kNN, pairs, boxes, probes; cache + ├─ Teacher(database, workloads, ...) teacher.cpp: builds the Objective once + │ └─ Objective(db, workloads) objective.cpp: inverts workloads, computes floors + └─ per budget share: + Teacher::run(budget, arms) + ├─ make_candidates six candidate recipes at equal budgets + ├─ EQUAL_CHOICE: independent_choice best candidate, partners original + ├─ EQUAL_CHOICE_RDP_PARTNERS / EQUAL_LOOP equal budgets: partners guessed as RDP / the loop + ├─ build_curves → allocate partners fixed at equal-budget RDP + │ ├─ BUDGET_RDP searched budgets, RDP's points + │ ├─ BUDGET_CHOICE → BUDGET_LOOP best candidates, then the loop: the teacher + │ ├─ BUDGET_LOOP_REALLOCATE rebuild curves, re-search budgets, loop again + │ ├─ BUDGET_LOOP_NO_UPDATE control: one look at the start, no reacting + │ └─ BUDGET_LOOP_SHUFFLED control: partners and neighbours relabelled + → each arm evaluated and exported like RDP and SED +``` + +Files: + +| File | Contents | +|---|---| +| `workloads.hpp/.cpp` | The teacher's training workloads, and the cache that saves the slow part | +| `objective.hpp/.cpp` | J_T, the score the teacher maximises, and `JoinState` and `KnnState` for fast exact join and kNN updates | +| `curves.hpp/.cpp` | Value curves and the budget allocator | +| `greedy.hpp/.cpp` | Query-aware point selection, with optional classic-method seeds | +| `teacher.hpp/.cpp` | The `Teacher` class and the selection steps | + +## Words used throughout + +- **Kept set:** the sorted point indices one trajectory keeps, endpoints included. `KeptSets` is one + kept set per trajectory, i.e. a whole simplified database. +- **Budget:** the number of points a trajectory keeps, endpoints included. A budget of 2 means + endpoints only. +- **Floor:** a metric's value when every trajectory keeps endpoints only. +- **Normalised metric:** `(metric − floor) / (1 − floor)`. 0 means no better than endpoints only; + 1 means as good as the original. +- **J4:** the pipeline's score on the held-out test workloads: the equal mean of four normalised + metrics (kNN, range, where-at, join). Route clustering is reported as a diagnostic only, because + it is built from the same Fréchet neighbour test as the join and would count it twice. + With `join.mode` `HeldOut` (the Geolife config) the join term compares original trajectories + from the query pool with the simplified database, so all four terms are held out. The database + self join, whose pairs the teacher trains on, is then reported separately (`norm_self_join`, + the seen / unseen recall and the 100 / 300 m thresholds) and is not part of J4. Results quoted + below predate this and have the self join inside the score. +- **J5:** the earlier headline, J4 plus clustering. Results quoted below from earlier runs use it; + `teacher_eval` no longer writes it. +- **J_T:** the teacher's own score on its training workloads. It has the same shape as J4. kNN + and range use training proxies. +- **J_T's kNN term:** with `teacher.ranking_knn` (the default) it is recall@k of the teacher's + database queries answered on the compressed database, so competitors are compressed too, like + the test. The loop tracks it with `KnnState`. Value curves still use the per-trajectory check + against original competitors, which `teacher_eval --independent-knn` restores everywhere as the + ablation. The per-trajectory check misleads the teacher: it credited the teacher with beating + RDP on kNN where held-out queries showed it did not. +- **J_T's join term:** follows `join.mode`, so the teacher trains on the kind of join it is + scored on. `Self`: both sides of a pair compressed, the self join. `HeldOut`: every database + trajectory also acts as an original query against its compressed partners, so each pair is + tested once in each direction and a trajectory's own choice only affects its partners' F1. + This is the database-only stand-in for the held-out join, as the kNN proxy is for held-out + kNN; the teacher still never sees the query pool. `teacher.join` (or `teacher_eval + --teacher-join Self|HeldOut`) overrides it, e.g. to train on the self join for the self-join + experiment. Held out, `EQUAL_CHOICE_RDP_PARTNERS` and `EQUAL_CHOICE` are the same arm, and best response is + coupled only through each query's F1. Everything below describing the join term as "both + sides compressed" is the `Self` mode. +- **Join terms:** the join term can pool several terms. `teacher.extra_join_eps_m` (or + `teacher_eval --train-eps`) adds self-join thresholds next to `join.eps_m`, so the join gain is + not fitted to one threshold. `teacher.held_out_term` (or `--held-out-term`) adds the held-out + form of the join at `join.eps_m` next to a `Self` join term, so one teacher keeps both the self + join and the join from outside queries. Each trajectory's F1 is counted once per term and the + metric is the mean over all of them, so a term with more matching trajectories weighs more. +- **Join pairs:** the teacher uses every 150 m pair, because whoever compresses the database knows + all of it. The pairs are also split in half (seen / unseen); with `all_join_pairs = false` the + teacher only gets the seen half, which shows how much of its join gain is fitting those + specific pairs. Whether the join gain generalises is tested at other thresholds (100 m, 300 m). +- **Arm:** one named output of the teacher, e.g. `EQUAL_CHOICE_RDP_PARTNERS`. + +--- + +## How selection works, in plain terms + +### What is being chosen +- At a given budget, the teacher chooses **one candidate per trajectory**. The six recipes are + RDP, SED, UNIFORM, unseeded greedy, and greedy seeded with half of the interior budget from + SED or from RDP. A variant seeded with a quarter from SED was removed: at small budgets it + kept the same points as unseeded greedy. Seeded candidates combine classic and greedy + point selection. Different recipes can produce identical kept sets. +- The fixed-budget arms use equal shares (α = 0). Allocation arms search per-trajectory budgets + using value curves, while keeping the total fixed. +- For example, the three classic candidates for a trajectory with budget 4 might be: + + ``` + RDP with 4 points → e.g. points {0, 37, 112, 250} + SED with 4 points → e.g. points {0, 61, 140, 250} + UNIFORM with 4 points → e.g. points {0, 83, 166, 250} + ``` + + Candidates at the chosen budgets are built by `make_candidates` and reused during best response. + +### Why a trajectory's choice depends on other trajectories +kNN, range and where-at only involve trajectory i itself, because everyone else is compared in +original form. The join is different: a pair (i, j) is scored with **both** sides simplified. Two +trajectories can gain or lose a join match when either kept set changes. So the best method for i +depends on what its join partners chose. + +### Two stages +The fixed-budget arms illustrate two stages, which differ in how the join partners are seen: + +``` +Stage 1, EQUAL_CHOICE_RDP_PARTNERS: one round, every trajectory on its own, partners ASSUMED to be RDP +Stage 2, EQUAL_LOOP: starts from stage 1, partners are their ACTUAL current choices + pass 1, pass 2, ... until a pass makes no switches +``` + +| | Stage 1 (EQUAL_CHOICE_RDP_PARTNERS) | Stage 2 (BR) | +|---|---|---| +| Points per trajectory | fixed budget | fixed budget | +| Methods tried for i | all 7 recipes | other distinct kept sets | +| Terms scored | all 4 (kNN, range, where-at, join) | all 4, as the change from i's current method | +| i's join partners seen as | RDP (a guess) | their actual current methods | +| Join change counted for | i only | i and every partner whose pair flips | +| Order | all trajectories at once, independently | one at a time; each switch is visible to the next | +| Runs | once | until no switches, at most 10 passes | + +Stage 1 is not needed for correctness: BR could start from all-RDP and still climb. It gives BR a +good starting point, and it is a result of its own (the independent, one-pass teacher), which +shows how much stage 2 adds. + +### Stage 1: EQUAL_CHOICE_RDP_PARTNERS (`independent_choice`, or `curve_methods`) +For each trajectory i, independently, score its candidates at its equal budget with +`independent_value`, with join partners fixed at equal-budget RDP. Choose the highest score; +ties go to the earlier candidate. If allocation arms also need curves, read the same choice from +the curve at that budget instead. + +The independent scores use a fixed partner reference. Their best choices need not maximise the +actual J_T once both sides of every join pair use their chosen candidates. + +### Allocation arms +`build_curves` scores budgets from 2 up to a per-trajectory cap. `allocate` fills the total budget +with greedy additions of up to three points at a time. `BUDGET_RDP` uses RDP-only curves; `BUDGET_CHOICE` uses the best of all candidates. +`BUDGET_LOOP` applies best response at those allocated budgets. `BUDGET_LOOP_SHUFFLED` is a control that +searches with relabelled partners and kNN neighbours, then scores the result against the real +workloads. `BUDGET_LOOP_NO_UPDATE` is the second control: one pass from the same start in which +every trajectory picks its best candidate against the others' starting state and never sees their +switches. The gap between it and `BUDGET_LOOP` is what reacting adds to looking once. + +`Teacher::single_recipes` gives the matching baselines: each candidate recipe for every trajectory +at equal budgets (`RECIPE_RDP`, …, `RECIPE_GREEDY_RDP2`). `teacher_eval` writes them with the +other baselines. `RECIPE_RDP`, `RECIPE_SED` and `RECIPE_UNIFORM` equal `RDP_a0`, `SED_a0` and +`UNI_a0`; the greedy recipes are the query-aware single recipes that choosing per trajectory must +beat. + +`BUDGET_LOOP_REALLOCATE` starts from `BUDGET_LOOP`. Each round rebuilds curves against the current partner +state, reallocates and runs best response. A round is accepted only if the exact J_T improves by +more than 1e-12; at most three rounds are tried. +With only held-out join terms (the outside-query teacher) the curves do not depend on how partners are compressed, so rebuilding them gives the same curves and budgets, and each round only reruns the loop from the same start in a new visit order. Re-budgeting only reacts to neighbours when the teacher has a self join term. + +### Stage 2: BR (`best_response`) +**In one sentence:** go through the trajectories one at a time, and let each switch to whichever +of its other methods makes the *real* J_T highest, given what everyone else currently uses; +repeat until nobody wants to switch. + +1. **Who is visited:** only trajectories with at least one join pair. For the others, the + score doesn't depend on anyone else, so their stage-1 choice is already their best. +2. **Starting state:** stage 1's method for every trajectory, plus a `JoinState` holding every + pair's outcome (match or not) with both sides on their stage-1 methods. +3. **One pass:** visit those trajectories in random order. For trajectory i and each of its other + distinct candidate kept sets c: + + ``` + gain(c) = [separable(i, c) − separable(i, current)] change in i's kNN, range, where-at + + join.propose(i, c, state).gain exact change in the join F1 + ``` + + `join.propose` rescores **all** of i's pairs against the partners' current kept points, and + updates the F1 of i and of every partner whose pair flips. + + **It switches when the best gain is positive** (above 1e-12). The two parts can pull in + opposite directions: a switch that costs i a little where-at can still win if it fixes a join + pair. After a switch, `join.apply` stores the new pair outcomes, and later visits see i's new + method. +4. **Repeat passes** until a pass has no switches (at most 10). + +**Why it repeats.** A switch changes every pair with that trajectory in it, so its partners may +now prefer a different method too. If a partner comes later in the same pass, it is checked +against the new choice right away; otherwise the next pass picks it up. One switch can ripple +through connected trajectories. + +### Example +A reduced example using only the three classic candidates: trajectories 1–4; (1, 4) is a true +join pair, and 1 is also paired with 2. + +``` +stage 1 (EQUAL_CHOICE_RDP_PARTNERS): 1=RDP 2=RDP 3=SED 4=SED + 1 chose RDP assuming its partner 4 would be RDP, but 4 actually chose SED, + so the pair (1, 4) doesn't match. + +stage 2, pass 1 (random order): + visit 3: try RDP, UNIFORM → no gain 1=RDP 2=RDP 3=SED 4=SED + visit 1: try SED, UNIFORM, against 4 as SED: + SED makes (1, 4) match; it also rescores (1, 2); + total gain > 0 → switch 1=SED 2=RDP 3=SED 4=SED + visit 4: try RDP, UNIFORM → would break (1, 4): stays + visit 2: scored against 1 as SED (its new method): + SED makes (1, 2) match again → switch 1=SED 2=SED 3=SED 4=SED +pass 1 had switches → pass 2 +pass 2: nobody gains → stop +``` + +1's switch only became worthwhile once the teacher used 4's actual method, and 2's switch only +became worthwhile after 1 switched. Neither could be found by scoring each trajectory on its own. + +### Why it always stops, and where +Every switch raises J_T by a strictly positive amount, and there are finitely many choices, so it +must stop. Requiring a strict gain also means no trajectory flips back and forth on a tie. In +practice there is also a limit of 10 passes. + +If a pass has no switches, **no single trajectory can raise J_T by switching on its own** within +the current candidate sets and improvement tolerance. Reaching the pass limit does not guarantee +this. Such a local optimum need not be global: switching two trajectories together might still +help. The random visiting order can change which local optimum it lands in. In +optimisation terms this is coordinate ascent; in game-theory terms, best-response dynamics. + +### Why it is fast +`JoinState` keeps every pair's outcome and every trajectory's hit and found counts. Testing one +switch only rescores that trajectory's own pairs (usually a handful) instead of all +pairs, and the result is exact: the tests check it against a full recompute. + +### What it shows +In an earlier three-candidate run, seed 42 at 0.25%: EQUAL_CHOICE_RDP_PARTNERS 0.506 → EQUAL_LOOP 0.544 J5, with the same +methods and budgets. The gain is in join (0.522 → 0.621) and clustering. The only difference is using the +partners' actual choices. Part of the join gain is in-sample, though: recall on unseen pairs falls +below seen pairs (0.78 against 0.89). + +--- + +## workloads.hpp / workloads.cpp + +### Why + +The teacher needs its own queries to optimise against. If it optimised against the test queries, +its score would only show that it can fit its own exam. These workloads come from the database +alone, with different seeds from the test. + +The kNN part is slow: every database trajectory is a query against all the others, with Fréchet +distances on full-resolution trajectories, which takes 25–45 minutes on Geolife. So it is cached +on disk. + +### Structs + +| Struct | Fields | Meaning | +|---|---|---| +| `WorkloadConfig` | `k`, `join_eps_m`, `extra_join_eps_m`, `frechet`, `range`, `where_at` | Settings, taken from the pipeline config (`knn.k`, `join.eps_m`, `teacher.extra_join_eps_m`, `frechet`, `range` with the teacher's own query count, `where_at`) | +| `KnnEntry` | `valid`, `top`, `cutoff` | Database trajectory e as a kNN query: its k nearest other trajectories, and `cutoff`, the Fréchet distance to the (k+1)th. `valid = false` if it has fewer than k+1 neighbours. | +| `JoinPair` | `i < j`, `seen` | A pair of database trajectories | +| `ExtraJoin` | `eps_m`, `positives`, `negatives` | The self-join pairs at one extra threshold, all of them used | +| `TeacherWorkloads` | `knn`, `positives`, `negatives`, `extra_joins`, `range`, `where_at`, plus the settings | Everything the objective needs | + +**Positives** are true self-join pairs: the originals are within `join_eps_m` (Fréchet). +**Negatives** are candidate pairs whose endpoints are within `join_eps_m`, but whose originals are +not. They matter because the join metric is F1: simplification can make negatives falsely match. + +### `load_or_build_workloads(db, config, seed, cache_dir)` — public + +- **Input:** the database, the settings, the seed, and the directory to look in for a cache. +- **Computation:** + 1. Fingerprints the database (`fingerprint`) and looks for `cache_dir/teacher_cache_seed.bin`. + 2. Calls `load`. If the file matches the current workload version, fingerprint, seed, `k`, + `join_eps_m` and `frechet`, it uses the cached kNN entries and pairs. Older cache versions + are rebuilt once because their metadata-only fingerprints cannot verify the point contents. + 3. Otherwise, it builds everything with `build_knn` and `build_pairs`, then `save`s it. + 4. Always regenerates the range boxes and where-at probes. They are fast, and use their own seeds + (`seed + 7001` and `seed + 7002`). + 5. Always regenerates the extra joins, one self join per extra threshold. They are not cached, so + changing the thresholds does not invalidate the kNN cache. +- **Output:** a `TeacherWorkloads`. It prints whether the cache was hit, and the counts. + +### `fingerprint(db)` — internal + +- **Why:** a cache is only valid for the exact same database. A different split or data file must + not reuse it. +- **Input:** the database. +- **Computation:** an FNV-1a hash over the trajectory count, the point count, each trajectory's ID + and length, and every point's timestamp, latitude, longitude and projected x/y coordinates. + Including the projected coordinates also invalidates the cache when the projection changes. +- **Output:** a 64-bit number, stored in the cache header. + +### `build_knn(db, config)` — internal + +- **Why:** this gives the kNN term its training queries. Using database trajectories as queries + means every trajectory appears in about k queries' answers, so every trajectory gets some kNN + signal. With only held-out queries, many trajectories would never be in anyone's top k, and the + teacher would treat them as free to wreck. +- **Input:** the database and the settings. +- **Computation:** in parallel, for each trajectory e: + 1. find its k+2 nearest neighbours (`query::knn`) and remove e itself; + 2. keep the first k as `top`; + 3. compute `cutoff`, the distance to the (k+1)th neighbour, with `distance()`. For continuous + Fréchet, the discrete distance is used as the upper bound of the search. +- **Output:** one `KnnEntry` per trajectory. + +### `build_pairs(db, config, seed, positives, negatives)` — internal + +- **Why:** this gives the join term its training pairs: what should match (positives) and what + shouldn't (negatives). +- **Input:** the database, the settings, the seed, and output vectors. +- **Computation:** runs the engine's `query::generate_self_join`. For every candidate pair of each + entry, a pair in `reference` is a positive and any other pair is a negative. Each list then goes + through `split_seen`. +- **Output:** the filled vectors. + +### `split_seen(pairs, seed)` — internal + +- **Input:** a pair list and a seed. +- **Computation:** a seeded shuffle; the first half is marked `seen`; then sorted by (i, j). + Positives use `seed + 7003`, negatives `seed + 7005`. +- **Output:** the same pairs, half of them seen. The split is deterministic for a given seed. + +### `save` / `load` / `write_pairs` / `read_pairs` / `put` / `get` — internal + +- **Why:** the cache file format. +- **Version 4** (read and written): a header (magic number, seed, full-content fingerprint, k, + eps, variant), then the kNN entries, positives and negatives. The version covers both file + layout and cached workload generation and must be bumped when either changes. +- **Versions 2 and 3** are rejected because they used metadata-only fingerprints. +- `save` writes to a `.tmp` file and renames it, so a crash never leaves a half-written cache. +- `load` returns false on any mismatch, which triggers a rebuild. + +--- + +## objective.hpp / objective.cpp + +### Why + +This is the score the teacher maximises. It has the same shape as J4, so that improving it +improves J4. It is computed only on the teacher's training workloads. Over the arms tested so far, +J_T correlated with the earlier J5 at r ≈ 0.95. + +The four terms, each normalised `(metric − floor) / (1 − floor)` and then averaged: + +| Term | Per trajectory i, simplified | Mirrors | +|---|---|---| +| kNN | for each query e that has i in its top k: Fréchet(original e, simplified i) ≤ cutoff(e) | recall@k | +| Range | for each box i is in: does a kept point still lie inside the box? | range F1, as recall (there are no false positives) | +| Where-at | average-hit ramp over i's probes: `clamp((max − err) / (max − min), 0, 1)` | where-at average-hit | +| Join | macro F1 over the pairs used, both sides simplified | join F1, computed the same way | + +The first three are **separable**: they add up over trajectories, because each check only involves +trajectory i (the others stay original). Join is **not separable**: whether a pair matches depends +on both sides' simplifications. + +Why range has no false positives: kept points are a subset of the original points, so a simplified +trajectory can't have a point in a box the original had none in. + +### `PreparedTrajectory` and `PreparedTrajectories` — public class + +- **Why:** the checks need trajectory i with only its kept points, in the shape the engine's + Fréchet and interpolation code expect. Building a full simplified database for every candidate + would be far too slow. +- **Input:** an original `TrajectoryView` and a kept set. +- **Computation:** copies `t`, `x` and `y` at the kept indices into its own vectors and points a + `TrajectoryView` at them. `lat` and `lon` stay empty, because the Fréchet and interpolation code + only read `t`, `x` and `y`. +- **Output:** `view()`, valid while the object lives and is unmodified. `assign` replaces the + retained points and rebuilds bounds while reusing coordinate storage between budgets. + `PreparedTrajectories` owns a whole set and provides borrowed views. Fixed partners are prepared + once for curve scoring; best response prepares each unique candidate once and shares duplicate + views. Trajectories without join links only need their selected candidate prepared. + +### `inside(box, view, p)` and `f1_of(hits, found, expected)` — internal + +- `inside`: is point p of the view inside the box's space and time window? +- `f1_of`: per-trajectory join F1, `2·hits / (found + expected)`. This is exactly the formula in + the engine's join evaluator. + +### `Objective(db, workloads, config)` — constructor + +- **Why:** everything that doesn't depend on a particular simplification is computed once here, + so each later score is cheap. +- **Input:** the database, the workloads, and the where-at tolerances (`ObjectiveConfig`, from the + pipeline's `where_at_scoring`). +- **Computation:** + 1. **Inverts kNN.** The workloads say "query e has top k {i, …}". The constructor turns this into + `m_knn[i]` = every (e, cutoff) where i is in e's top k, so scoring trajectory i only touches + its own memberships. `m_knn_weight = 1 / (k · valid queries)`, so the term sums to recall@k. + 2. **Inverts range.** `m_range[i]` = every box i is in, each with weight + `1 / (|box reference| · boxes with any reference)`. Summed, this is recall averaged per box. + 3. **Groups where-at probes** per trajectory in `m_probes[i]`, with + `m_where_at_weight = 1 / trajectories with probes` (averaged per trajectory, like the + evaluator). + 4. **Builds the join links.** Positives first, then negatives (all, or only the seen half), stored in + `m_pair_i`/`m_pair_j`. Each pair is linked from both sides in `m_links`. `m_truth[t]` counts + t's positives. + 5. **Floors:** `metrics(endpoints)` gives each metric's floor. If a floor is already 1, it + throws, because that metric can't be normalised. + 6. **Scales:** `m_scale[m] = 1 / (4 · (1 − floor_m))`. This applies the normalisation and the + equal average in one factor. + 7. **`m_v0[i]`:** trajectory i's scaled separable terms with endpoints only. Subtracting it makes + `separable` exactly 0 at the floor. +- **Output:** a ready `Objective`. It keeps references to `db` and `workloads`, so both must + outlive it (`Teacher` guarantees this). + +### `raw_separable(i, kept)` — private + +- **Why:** the actual kNN, range and where-at checks for one trajectory. +- **Input:** trajectory index and kept set. +- **Computation:** + - **kNN:** for each membership (e, cutoff), test `frechet_within(original e, simplified i, + cutoff)` and add `m_knn_weight` if it holds. Competitors stay original. + - **Range:** for each box, add the box's weight if any kept point is `inside`. + - **Where-at:** for each probe, interpolate the simplified position at the probe's time + (`core::position_at`), take the distance to the true position, apply the ramp, and average over + i's probes times `m_where_at_weight`. +- **Output:** `Terms`, with i's raw contribution to each metric (join slot 0). + +### `separable(i, kept)` — public + +- **Input:** trajectory index and kept set. +- **Computation:** `Σ m_scale[m] · raw_separable[m] − m_v0[i]` over kNN, range and where-at. +- **Output:** i's part of J_T from the three separable terms, 0 at endpoints only. Summed over all + trajectories, this equals those terms' share of J_T exactly. + +### `independent_value(i, candidate, partners)` — public + +- **Why:** the value curves need one number per (trajectory, candidate), computed without knowing + what the other trajectories will choose. +- **Input:** trajectory index, a prepared candidate view, and fixed partner views. An empty + partner span means original partners (the `EQUAL_CHOICE` arm); otherwise the views contain RDP + at equal budgets for `EQUAL_CHOICE_RDP_PARTNERS`. +- **Computation:** `separable(i, candidate)` plus i's own join F1 against those fixed partners: count + `found` and `hits` over i's links using `match`, then add + `m_scale[Join] · F1_i / (trajectories with truth)`. +- **Output:** an approximate per-trajectory score. It's approximate because the real join depends + on partners' actual choices and counts each pair on both sides. That is exactly what + `best_response` corrects. +- `prepare_independent` computes the range, where-at and join terms once, together with + the upper bound used to order candidates. `independent_value(score, beat)` reuses those terms + while checking kNN, so a candidate is not rescored just because its bound was promising. The + prepared score borrows its candidate's coordinates and must be used before they change. + +### `match(a, b)` — public + +- **Input:** two prepared trajectory views (or an original view). +- **Computation:** `query::frechet_within` on the simplified views, with the workload's threshold + and Fréchet variant. +- **Output:** whether the pair would be reported by the join. + +### `join_metric(state)` — private + +- **Input:** a whole simplified database. +- **Computation:** + 1. In parallel, `match` every pair used. + 2. For each matched pair, increment `found` on both sides, and `hits` too if it's a positive. + 3. Average `f1_of(hits, found, truth)` over trajectories with any truth or any match. This is the + same counting rule as the engine's evaluator. +- **Output:** macro join F1 on the pairs used. + +### `metrics(kept)` — public + +- **Input:** a whole simplified database. +- **Computation:** prepares the trajectories once, sums `raw_separable` over their views (in + parallel), and reuses those views in `join_metric`. +- **Output:** the four raw metric values. Used for the floors, and in tests. + +### `total(kept)` — public + +- **Input:** a whole simplified database. +- **Computation:** `Σ_m m_scale[m] · (metric_m − floor_m)`. +- **Output:** the exact J_T. It is 0 with endpoints only, and higher is better. The pipeline prints + it next to each arm. + +### `JoinState` — public class + +#### Why + +Best response tries thousands of single-trajectory switches. Recomputing `join_metric` for every +try would rescore about 90,000 pairs each time. `JoinState` keeps every pair's outcome and every +trajectory's counts, so a switch of trajectory i only rescores i's own pairs (usually a handful). +The result is exact, not approximate: the tests check it against a full recompute. + +#### `JoinState(objective, state)` — constructor + +- **Input:** the objective and a starting simplified database. +- **Computation:** matches every pair used once (from its smaller side, in parallel). It fills + `m_matched`, `m_hits[t]` and `m_found[t]`, then sums F1 over counted trajectories into `m_sum` + and `m_counted`. +- **Output:** a state whose `metric()` equals `join_metric(state)`. + +#### `propose(i, candidate, state)` + +- **Why:** "how much would J_T's join term change if i switched to `kept`?", without changing + anything. +- **Input:** trajectory index, prepared candidate view, and current partner views. +- **Computation:** + 1. For each of i's links, compare the pair's current outcome with its outcome under the + candidate. + 2. For every pair that flips, record the changes to hits and found on both trajectories. + 3. For each affected trajectory, remove its old F1 from the sum and add its new one, adjusting + the counted trajectories when one starts or stops being counted. +- **Output:** a `Proposal` with the trajectory index, pair outcomes, and + `gain = m_scale[Join] · (new macro F1 − current macro F1)`. Added to the cached `separable` + difference, this is the exact change in J_T. Proposals remain valid until the next `apply`. + +#### `apply(proposal)` + +- **Input:** the winning proposal evaluated against the current state. +- **Computation:** writes its saved outcomes into `m_matched`, the counts and the sum, without + repeating Fréchet tests. The caller also switches the current view to the winning candidate. +- **Output:** nothing; the state now matches the new solution. + +#### `metric()`, `counted(t, found)`, `f1(t, hits, found)` + +Helpers: the current macro F1; whether trajectory t counts (it has truth or matches); and t's F1. + +--- + +## curves.hpp / curves.cpp + +### `Curve` + +- **Why:** a value curve records what one trajectory is worth at each budget. The allocator uses + these values to redistribute points; selection reads the method at the chosen budget. +- **Fields:** `value[k]` and `method[k]` for budget `b = 2 + k`: the best candidate's + `independent_value` and which candidate reached it. +- **Helpers:** `max_budget()` is the largest budget on the curve, and `at(b)` is the value at b. +- **Note:** Fréchet thresholds flip on and off, so a curve is a step function and need not rise at + every step. + +`allocate(curves, total_budget, value)` starts with endpoints and fills the remaining budget with +greedy steps of up to three points, taking the best gain per point each time. A donor/receiver +exchange pass used to follow; across the runs of November 2026 it never changed the curve sum by +more than 0.0001, so it was removed. It preserves the exact total budget and respects every curve's maximum budget. It is a +heuristic search of the curve sum, which is distinct from the exact J_T with both partners +simplified. + +--- + +## teacher.hpp / teacher.cpp + +### Constants and types + +| Name | Meaning | +|---|---| +| `CandidateIndex`, `kCandidates` | Named indices and labels for the six candidate recipes. The index is the "method". | +| `kArms` | the nine valid arm names, listed in `teacher.hpp` | +| `TeacherConfig` | `enabled`, `cache_dir`, `arms`, `range_query_count`, `all_join_pairs`, `join` (optional, default: the pipeline's `join.mode`), `extra_join_eps_m`, `held_out_term`, `ranking_knn` (default true), `term_weights` (kNN, range, where-at, join; equal by default) | +| `Arm` | one result: `name`, `simplification` (for the pipeline), `objective` (its J_T) | +| `Candidates` | `candidates[c][i]` = trajectory i simplified with method c | +| `Methods` | one method index per trajectory, i.e. a selection | +| `kMaxResponsePasses` | at most 10 best-response passes | +| `kKnnWatchFactor` | each kNN query watches its 3k nearest trajectories under each candidate state | +| `kMaxAllocationRounds` | at most 3 iterative reallocation rounds | +| `kMinImprovement` | strict gain threshold, 1e-12 | + +### `Teacher(db, workloads, config, seed)` — constructor + +- **Input:** the database, the workloads (moved in), the where-at tolerances, and the seed. +- **Computation:** stores the workloads, then builds the `Objective` on them. `m_workloads` is + declared before `m_objective`, so it is constructed first. That ordering matters because the + objective keeps a reference to it. Prints the pair counts and floors. +- **Output:** a ready teacher. It can't be copied, since that would break the reference. + +### `Teacher::run(total_budget, arms)` — public, the main entry + +- **Input:** the total point budget for this share, and the arm names to produce. +- **Computation:** + 1. Rejects unknown arm names. + 2. `equal_budgets` = equal budgets per trajectory (the engine's size^α allocator with α = 0). + Curves are capped at about four times the mean interior budget, at least six interior + points, never below the equal budget and never above trajectory length. + 3. `make_candidates` at those budgets, building only RDP if no fixed-budget arm needs alternatives. + 4. **`EQUAL_CHOICE`** (if requested): `independent_choice` with `partners = nullptr`. + 5. **`EQUAL_CHOICE_RDP_PARTNERS` and `EQUAL_LOOP`** (if either is requested): + - score candidates directly against equal-budget RDP partners, or reuse curves if allocation + arms also need them, to obtain `EQUAL_CHOICE_RDP_PARTNERS`; + - `best_response` starting from `EQUAL_CHOICE_RDP_PARTNERS`, which gives `EQUAL_LOOP`. + 6. Allocation arms build curves against equal-budget RDP, allocate the total budget, and select + candidates. Response arms refine that selection; iterative rounds rebuild against the current + partners and keep only improvements in exact J_T. + 7. `keep` assembles each arm's kept sets, computes its exact J_T, and prints how many + trajectories each method won. `timed` prints step durations. +- **Output:** one `Arm` per requested name, in the requested order. Each keeps exactly + `total_budget` points, because every candidate keeps exactly its trajectory's budget. + +### `make_candidates(objective, budgets, caps, selected)` — internal + +- **Why:** each candidate is simplified once, then reused by every step. +- **Input:** objective, per-trajectory budgets and caps, and optionally selected method indices. +- **Computation:** in parallel, `CandidateMaker` builds all six recipes, or only the selected + recipe for each trajectory. Classic sequences and greedy sequences with the same seed are reused. +- **Output:** `Candidates`. + +### `assemble(candidates, method)` — internal + +- **Input:** candidates and a selection. +- **Computation:** for each trajectory, picks the kept set of its chosen method. +- **Output:** a whole simplified database (`KeptSets`). + +### `independent_choice(db, objective, candidates, partners)` — internal + +- **Why:** the simplest teacher. Each trajectory picks its best method on its own, against a fixed + partner state. +- **Input:** candidates, and the partner state for `independent_value`. +- **Computation:** in parallel, for each trajectory, the method with the highest + `independent_value`. Ties go to the earlier method, so RDP wins when nothing differs. +- **Output:** `Methods`. With `partners = nullptr` this gives `EQUAL_CHOICE`. Original partners + hide the effect of simplifying both sides of a join pair. + +### `build_curves(db, objective, partners, caps, rdp_only)` — internal + +- **Why:** the value curves: for each trajectory and each budget, the best method and its value. +- **Input:** the partner reference state (initially RDP at equal budgets, or the current state in + iterative rounds), curve caps, and whether only RDP is needed. +- **Computation:** in parallel per trajectory, build the RDP addition sequence and reversed SED + removal sequence once, then use sorted prefixes for each budget. Candidate coordinate buffers + are reused between budgets. For b = 2 … max: + - simplify with every candidate (at b = 2 all keep the endpoints, so only one is scored); + - score each with `independent_value` against the fixed partner reference; + - record the best value and method. +- **Output:** RDP-only and best-candidate curves per trajectory. +- Runs requesting only `EQUAL_CHOICE_RDP_PARTNERS` and/or `EQUAL_LOOP` score their fixed budgets directly and skip full + curve construction. +- **Why partners as RDP:** a fixed, plausible guess at how partners will look. It still scores each + trajectory on its own, but it rewards matching how partners will likely be simplified. This took + J5 from 0.446 (`EQUAL_CHOICE`) to 0.506 (`EQUAL_CHOICE_RDP_PARTNERS`) in the earlier three-candidate experiment. + +### `curve_methods(curves, budgets)` — internal + +- **Input:** curves and per-trajectory budgets. +- **Computation:** `method[b − 2]` for each trajectory. Throws if a budget is off its curve. +- **Output:** `Methods` at the supplied budgets. + +### `best_response(db, objective, candidates, method, seed)` — internal + +- **Why:** this step uses the partners' **actual** choices instead of a guess. A trajectory's best + method depends on what its join partners chose, so the teacher revisits trajectories until no + single change helps. +- **Input:** candidates, a starting selection (`EQUAL_CHOICE_RDP_PARTNERS` or `BUDGET_CHOICE`), and the seed. +- **Computation:** + 1. `order` = the trajectories with any join link or, with the ranking kNN term, a watching + query. Only they can gain, since the other terms don't depend on anyone else. + 2. Prepares candidate views, caches their separable scores, and builds the starting `JoinState`. + 3. For up to 10 passes, in a seeded random order: + - for each trajectory i, and each other method c: gain = `separable(c) − separable(current)` + + `join.propose(c).gain` + `knn.propose(c)`. That's the exact change in J_T, up to + trajectories outside a query's watch list entering its top k; + - switches to the best c only if its gain beats 1e-12 (strictly positive, so ties can't + cycle), and calls `join.apply`. + 4. Stops after a pass with no switches. +- **Output:** the improved `Methods`. Stops after a pass with no switches or the 10-pass limit. + +### `to_simplification`, `wins`, `seconds_since`, `is_arm` — helpers + +- `to_simplification`: `KeptSets` → `core::Simplification`, the format the pipeline evaluates and + exports. +- `wins`: per-candidate counts for the log. +- `seconds_since`: wall time for the log. +- `is_arm`: whether a name is in `kArms`. It is also used by the config parser. + +--- + +## In the pipeline + +- **`pipeline/config.cpp`, `parse_teacher_config`:** reads the `teacher` section. It rejects + unknown keys, an empty `cache_dir`, an empty `arms` list when enabled, unknown or duplicate arm + names, and `range_query_count` = 0. +- **`pipeline.cpp`, in `run`:** + - If `teacher.enabled`, it builds the workloads from the `database` split, with settings taken + from the engine config. Then it constructs `teacher_model`. + - Per budget share, after the regular algorithms, `teacher_model->run` produces the arms. Each + goes through the same `evaluate` lambda: materialise, score on the held-out workloads, print, + and export as `_pct`. + +## Earlier three-candidate results (Geolife, 0.25%, seed 42) + +These predate J4 and use J5, which includes clustering. Arm names are the old ones: `GATEORIG_a0` is now `EQUAL_CHOICE`, `GATE_a0` is `EQUAL_CHOICE_RDP_PARTNERS` and `BR_a0` is `EQUAL_LOOP`. + +| Arm | J_T | J5 | +|---|---:|---:| +| RDP_a0 (baseline) | — | 0.492 | +| GATEORIG_a0 | 0.458 | 0.446 | +| GATE_a0 | 0.503 | 0.506 | +| BR_a0 | 0.548 | 0.544 | + +Historical run time: about 7 s for `GATEORIG_a0`, 16 s for the curves and 66 s for best response. +These timings predate the seven-candidate teacher, allocation search and cache changes. + +## Tests + +`engine/tests/teacher/teacher_test.cpp` uses a small synthetic database: + +- `CacheIsWrittenThenReadBack`: a cache round trip gives identical workloads. +- `CacheForOtherSettingsIsRebuilt`: a cache built with a different `k` isn't reused. +- `ObjectiveIsZeroAtTheFloorAndTracksJoinExactly`: J_T = 0 at endpoints only. After random + switches, `JoinState`'s tracked F1 equals a full recompute, and `delta + separable` predicts the + real change in J_T. +- `SeveralJoinTermsAreTrackedExactly`: the same exactness with two extra thresholds, with and + without the held-out term. +- `RankingKnnIsTrackedExactly`: when every query watches every trajectory, `KnnState`'s tracked + recall equals a full recompute after random switches, and the kNN, join and separable changes + together predict the real change in J_T. +- `RankingKnnArmsKeepTheBudget`: the arms keep exactly the budget with the ranking kNN term. +- `SingleRecipesKeepTheBudget`: the six single-recipe baselines each keep exactly the budget. +- `ArmsKeepTheBudgetAndBestResponseOnlyImproves`: every arm keeps exactly the budget, endpoints + kept, indices sorted; and `EQUAL_LOOP`'s J_T ≥ `EQUAL_CHOICE_RDP_PARTNERS`'s. + +`engine/tests/config_test.cpp` has the three teacher config tests. diff --git a/engine/src/teacher/curves.cpp b/engine/src/teacher/curves.cpp new file mode 100644 index 0000000..6402bd2 --- /dev/null +++ b/engine/src/teacher/curves.cpp @@ -0,0 +1,84 @@ +#include "backend/engine/teacher/curves.hpp" + +#include +#include +#include + +namespace teacher { + +namespace { + +constexpr std::size_t kMaxStep = 3; + +struct Offer { + double ratio; // gain per point + std::size_t i, k, version; + bool operator<(const Offer &other) const { + return ratio < other.ratio || (ratio == other.ratio && i > other.i); + } +}; + +// The best step of 1 to kMaxStep points for trajectory i, at most left points. k = 0 if none +Offer best_offer(const Curve &curve, std::size_t i, std::size_t budget, std::size_t left, + std::size_t version) { + Offer best{-std::numeric_limits::infinity(), i, 0, version}; + for (std::size_t k = 1; k <= kMaxStep && k <= left && budget + k <= curve.max_budget(); ++k) { + const double ratio = (curve.at(budget + k) - curve.at(budget)) / static_cast(k); + if (ratio > best.ratio) { + best.ratio = ratio; + best.k = k; + } + } + return best; +} + +// Queues trajectory i's best offer from its current budget, if it has one +void push_offer(std::priority_queue &queue, const Curve &curve, std::size_t i, + std::size_t budget, std::size_t left, std::size_t version) { + if (const auto offer = best_offer(curve, i, budget, left, version); offer.k != 0) + queue.push(offer); +} + +} // namespace + +std::vector allocate(const std::vector &curves, std::size_t total_budget, + double *value) { + const std::size_t n = curves.size(); + if (total_budget < 2 * n) + throw std::invalid_argument("allocate: budget below two endpoints per trajectory"); + for (const auto &curve : curves) + if (curve.value.empty()) + throw std::invalid_argument("allocate: empty curve"); + + std::vector budgets(n, 2); + std::size_t left = total_budget - 2 * n; + + std::vector version(n, 0); + std::priority_queue queue; + for (std::size_t i = 0; i < n; ++i) + push_offer(queue, curves[i], i, budgets[i], left, 0); + while (left > 0) { + if (queue.empty()) + throw std::runtime_error("allocate: curves too short to hold the budget"); + const auto offer = queue.top(); + queue.pop(); + if (offer.version != version[offer.i]) + continue; + // If the budget left shrank since this offer was made, offer again within it + if (offer.k <= left) { + budgets[offer.i] += offer.k; + left -= offer.k; + ++version[offer.i]; + } + push_offer(queue, curves[offer.i], offer.i, budgets[offer.i], left, version[offer.i]); + } + + if (value) { + *value = 0; + for (std::size_t i = 0; i < n; ++i) + *value += curves[i].at(budgets[i]); + } + return budgets; +} + +} // namespace teacher diff --git a/engine/src/teacher/greedy.cpp b/engine/src/teacher/greedy.cpp new file mode 100644 index 0000000..73e6749 --- /dev/null +++ b/engine/src/teacher/greedy.cpp @@ -0,0 +1,184 @@ +#include "backend/engine/teacher/greedy.hpp" + +#include +#include +#include + +namespace teacher { + +namespace { + +// Smallest gain worth keeping a point for; below it greedy stops and fill takes over +constexpr double kMinGain = 1e-15; + +} // namespace + +Greedy::Greedy(const Objective &objective, std::size_t i) + : m_trajectory(objective.db().trajectory(i)), + m_min_tolerance(objective.config().min_tolerance_m), + m_max_tolerance(objective.config().max_tolerance_m) { + const auto &trajectory = m_trajectory; + const std::size_t n = trajectory.size(); + const auto &workloads = objective.workloads(); + + const double range_scale = objective.scale(Objective::Range); + std::vector count(n, 0); + for (const auto &[b, weight] : objective.boxes(i)) { + const auto &box = workloads.range.queries[b].box; + std::vector inside; + // t is sorted, so only the box's time window is scanned + for (auto k = static_cast(std::ranges::lower_bound(trajectory.t, box.t_min) - + trajectory.t.begin()); + k < n && trajectory.t[k] <= box.t_max; ++k) + if (trajectory.x[k] >= box.x_min && trajectory.x[k] <= box.x_max && + trajectory.y[k] >= box.y_min && trajectory.y[k] <= box.y_max) { + inside.push_back(static_cast(k)); + ++count[k]; + } + m_box_points.push_back(std::move(inside)); + m_box_weight.push_back(range_scale * weight); + } + m_offsets.assign(n + 1, 0); + for (std::size_t k = 0; k < n; ++k) + m_offsets[k + 1] = m_offsets[k] + count[k]; + m_point_boxes.resize(m_offsets[n]); + std::vector at(m_offsets.begin(), m_offsets.end() - 1); + for (std::size_t b = 0; b < m_box_points.size(); ++b) + for (const auto k : m_box_points[b]) + m_point_boxes[at[k]++] = static_cast(b); + + const auto probes = objective.probes(i); + std::vector order(probes.begin(), probes.end()); + std::ranges::sort(order, [&](auto a, auto b) { + return workloads.where_at.queries[a].t < workloads.where_at.queries[b].t; + }); + for (const auto k : order) { + const auto &probe = workloads.where_at.queries[k]; + m_probe_t.push_back(probe.t); + m_probe_x.push_back(probe.reference.x); + m_probe_y.push_back(probe.reference.y); + } + if (!order.empty()) + m_probe_weight = objective.scale(Objective::WhereAt) * objective.where_at_weight() / + static_cast(order.size()); +} + +std::vector Greedy::run(std::size_t budget, std::span seed, + std::span fill) const { + auto kept = sequence(budget, seed, fill); + std::ranges::sort(kept); + return kept; +} + +std::vector Greedy::sequence(std::size_t limit, std::span seed, + std::span fill) const { + const auto &trajectory = m_trajectory; + const std::size_t n = trajectory.size(); + const std::size_t budget = std::min(limit, n); + + auto ramp = [&](double error) { + return std::clamp((m_max_tolerance - error) / (m_max_tolerance - m_min_tolerance), 0.0, + 1.0); + }; + // Where-at score of probe p on the segment from kept point a to kept point c + auto score_on = [&](std::size_t p, std::size_t from, std::size_t to) { + const double t_from = static_cast(trajectory.t[from]), + t_to = static_cast(trajectory.t[to]); + const double fraction = + t_to > t_from ? std::clamp((m_probe_t[p] - t_from) / (t_to - t_from), 0.0, 1.0) : 0.0; + const double x = trajectory.x[from] + fraction * (trajectory.x[to] - trajectory.x[from]); + const double y = trajectory.y[from] + fraction * (trajectory.y[to] - trajectory.y[from]); + return ramp(std::hypot(x - m_probe_x[p], y - m_probe_y[p])); + }; + // Probes from t_a up to t_c, the last segment including its end + auto probes_between = [&](std::size_t from, std::size_t to) { + const auto lo = static_cast( + std::ranges::lower_bound(m_probe_t, static_cast(trajectory.t[from])) - + m_probe_t.begin()); + const auto hi = + to == n - 1 + ? m_probe_t.size() + : static_cast( + std::ranges::lower_bound(m_probe_t, static_cast(trajectory.t[to])) - + m_probe_t.begin()); + return std::pair{lo, hi}; + }; + + std::vector kept(n, 0), covered(m_box_points.size(), 0); + std::vector range_gain(n, 0.0), where_at_gain(n, 0.0); + for (std::size_t box = 0; box < m_box_points.size(); ++box) + for (const auto k : m_box_points[box]) + range_gain[k] += m_box_weight[box]; + std::set order; + std::vector sequence; + std::size_t count = 0; + auto keep = [&](std::size_t j) { + if (kept[j]) + return; + kept[j] = 1; + order.insert(j); + sequence.push_back(j); + ++count; + for (auto e = m_offsets[j]; e < m_offsets[j + 1]; ++e) { + const auto box = m_point_boxes[e]; + if (covered[box]) + continue; + covered[box] = 1; + for (const auto k : m_box_points[box]) + range_gain[k] -= m_box_weight[box]; + } + }; + // Where-at gain of every point strictly between kept neighbours a and c + std::vector current_scores(m_probe_t.size()); + auto refresh = [&](std::size_t from, std::size_t to) { + const auto [lo, hi] = probes_between(from, to); + for (std::size_t p = lo; p < hi; ++p) + current_scores[p] = score_on(p, from, to); + for (std::size_t j = from + 1; j < to; ++j) { + double gain = 0; + for (std::size_t p = lo; p < hi; ++p) { + const double now = current_scores[p]; + const double then = m_probe_t[p] < static_cast(trajectory.t[j]) + ? score_on(p, from, j) + : score_on(p, j, to); + gain += then - now; + } + where_at_gain[j] = gain * m_probe_weight; + } + }; + + keep(0); + keep(n - 1); + for (const auto j : seed) + if (count < budget) + keep(j); + if (!m_probe_t.empty() && count < budget) + for (auto it = order.begin(); std::next(it) != order.end(); ++it) + refresh(*it, *std::next(it)); + + while (count < budget) { + double best = kMinGain; + std::size_t pick = n; + for (std::size_t j = 1; j + 1 < n; ++j) + if (!kept[j] && range_gain[j] + where_at_gain[j] > best) { + best = range_gain[j] + where_at_gain[j]; + pick = j; + } + if (pick == n) + break; + keep(pick); + if (!m_probe_t.empty() && count < budget) { + const auto it = order.find(pick); + refresh(*std::prev(it), pick); + refresh(pick, *std::next(it)); + } + } + for (std::size_t z = 0; z < fill.size() && count < budget; ++z) + keep(fill[z]); + for (std::size_t j = 1; j + 1 < n && count < budget; ++j) // only if fill ran short + keep(j); + + return sequence; +} + +} // namespace teacher diff --git a/engine/src/teacher/objective.cpp b/engine/src/teacher/objective.cpp new file mode 100644 index 0000000..f93de0f --- /dev/null +++ b/engine/src/teacher/objective.cpp @@ -0,0 +1,633 @@ +#include "backend/engine/teacher/objective.hpp" + +#include "backend/engine/core/interpolation.hpp" +#include "backend/engine/core/parallel.hpp" +#include "backend/engine/core/simplification.hpp" +#include "backend/engine/query/frechet_helper.hpp" +#include "backend/engine/query/knn.hpp" + +#include +#include +#include +#include +#include +#include +#include + +namespace teacher { + +PreparedTrajectory::PreparedTrajectory(const core::TrajectoryView &original, + std::span kept) + : m_id(original.trajectory_id) { + assign(original, kept); +} + +void PreparedTrajectory::assign(const core::TrajectoryView &original, + std::span kept) { + m_id = original.trajectory_id; + m_t.resize(kept.size()); + m_x.resize(kept.size()); + m_y.resize(kept.size()); + constexpr double inf = std::numeric_limits::infinity(); + m_box = {inf, -inf, inf, -inf}; + for (std::size_t k = 0; k < kept.size(); ++k) { + m_t[k] = original.t[kept[k]]; + m_x[k] = original.x[kept[k]]; + m_y[k] = original.y[kept[k]]; + m_box.x_min = std::min(m_box.x_min, m_x[k]); + m_box.x_max = std::max(m_box.x_max, m_x[k]); + m_box.y_min = std::min(m_box.y_min, m_y[k]); + m_box.y_max = std::max(m_box.y_max, m_y[k]); + } +} + +PreparedTrajectories::PreparedTrajectories(const core::TrajectoryDb &db, const KeptSets &kept) { + if (kept.size() != db.size()) + throw std::invalid_argument("prepared trajectories: one kept set per trajectory needed"); + m_trajectories.reserve(kept.size()); + for (std::size_t i = 0; i < kept.size(); ++i) + m_trajectories.emplace_back(db.trajectory(i), kept[i]); +} + +std::vector PreparedTrajectories::views() const { + std::vector views; + views.reserve(m_trajectories.size()); + for (const auto &trajectory : m_trajectories) + views.push_back(trajectory.view()); + return views; +} + +namespace { + +bool inside(const query::RangeBox &box, const core::TrajectoryView &trajectory, std::size_t p) { + return trajectory.t[p] >= box.t_min && trajectory.t[p] <= box.t_max && + trajectory.x[p] >= box.x_min && trajectory.x[p] <= box.x_max && + trajectory.y[p] >= box.y_min && trajectory.y[p] <= box.y_max; +} + +constexpr std::size_t kMaxJoinTerms = 8; + +double f1_of(std::size_t hits, std::size_t found, std::size_t expected) { + return 2.0 * static_cast(hits) / static_cast(found + expected); +} + +} // namespace + +double check_term_weights(const TermWeights &weights) { + double sum = 0; + for (const double weight : weights) { + if (!std::isfinite(weight) || weight < 0) + throw std::invalid_argument("term weights must be finite and not negative"); + sum += weight; + } + if (sum <= 0) + throw std::invalid_argument("at least one term weight must be positive"); + return sum; +} + +Objective::Objective(const core::TrajectoryDb &db, const TeacherWorkloads &workloads, + const ObjectiveConfig &config) + : m_db(db), m_workloads(workloads), m_config(config) { + const std::size_t n = db.size(); + if (workloads.knn.size() != n) + throw std::invalid_argument("objective: kNN workload does not match the database"); + if (!(config.min_tolerance_m < config.max_tolerance_m)) + throw std::invalid_argument("objective: min_tolerance_m must be below max_tolerance_m"); + const double weight_sum = check_term_weights(config.term_weights); + // Extra thresholds use every pair, which would train on the pairs held back as unseen + if (!config.all_join_pairs && !workloads.extra_joins.empty()) + throw std::invalid_argument( + "objective: seen-pairs-only training cannot use extra join thresholds"); + + // kNN: invert each query's top k + m_knn.resize(n); + std::size_t valid = 0; + for (std::size_t e = 0; e < n; ++e) { + const auto &query = workloads.knn[e]; + if (!query.valid) + continue; + ++valid; + for (const auto i : query.top) + m_knn[i].push_back({static_cast(e), query.cutoff}); + } + m_knn_weight = + valid ? 1.0 / (static_cast(workloads.k) * static_cast(valid)) : 0.0; + + // Range: invert each box's reference + m_range.resize(n); + std::size_t scored = 0; + for (const auto &query : workloads.range.queries) + scored += !query.reference.empty(); + for (std::size_t b = 0; b < workloads.range.queries.size(); ++b) { + const auto &ref = workloads.range.queries[b].reference; + for (const auto i : ref) + m_range[i].push_back( + {static_cast(b), + 1.0 / (static_cast(ref.size()) * static_cast(scored))}); + } + + // Where-at: probes per trajectory + m_probes.resize(n); + for (std::size_t k = 0; k < workloads.where_at.queries.size(); ++k) + m_probes[workloads.where_at.queries[k].trajectory_index].push_back( + static_cast(k)); + const auto with_probes = + std::ranges::count_if(m_probes, [](auto &probes) { return !probes.empty(); }); + m_where_at_weight = with_probes ? 1.0 / static_cast(with_probes) : 0.0; + + // Join terms: join_eps_m, the extra thresholds, then the held-out form of the first. Pairs are + // stored positives first, then negatives + struct TermPairs { + const std::vector *positives, *negatives; + }; + const bool held_out = config.join_mode == query::JoinMode::HeldOut; + std::vector terms{{&workloads.positives, &workloads.negatives}}; + m_eps = {workloads.join_eps_m}; + m_term_held_out = {held_out}; + for (const auto &extra : workloads.extra_joins) { + terms.push_back({&extra.positives, &extra.negatives}); + m_eps.push_back(extra.eps_m); + m_term_held_out.push_back(held_out); + } + if (config.held_out_term && !held_out) { + terms.push_back(terms.front()); + m_eps.push_back(workloads.join_eps_m); + m_term_held_out.push_back(true); + } + if (terms.size() > kMaxJoinTerms) + throw std::invalid_argument("objective: too many join terms"); + m_links.resize(n); + m_truth.assign(n * terms.size(), 0); + auto add = [&](const JoinPair &join_pair, bool positive, std::size_t term) { + if (!config.all_join_pairs && !join_pair.seen) + return; + const auto pair = static_cast(m_pair_i.size()); + const auto k = static_cast(term); + m_pair_i.push_back(join_pair.i); + m_pair_j.push_back(join_pair.j); + m_pair_term.push_back(k); + m_links[join_pair.i].push_back({join_pair.j, pair, positive, k}); + m_links[join_pair.j].push_back({join_pair.i, pair, positive, k}); + if (positive) { + ++m_truth[slot(join_pair.i, term)]; + ++m_truth[slot(join_pair.j, term)]; + } + }; + for (std::size_t k = 0; k < terms.size(); ++k) + for (const auto &join_pair : *terms[k].positives) + add(join_pair, true, k); + m_positive_count = m_pair_i.size(); + for (std::size_t k = 0; k < terms.size(); ++k) + for (const auto &join_pair : *terms[k].negatives) + add(join_pair, false, k); + m_with_truth = static_cast( + std::ranges::count_if(m_truth, [](auto count) { return count > 0; })); + + // Floors: endpoints only, partners included, like the J4 floor + const auto endpoints = core::endpoints_only(db); + m_floor = metrics(endpoints); + for (std::size_t m = 0; m < TermCount; ++m) { + if (m_floor[m] >= 1.0) + throw std::runtime_error("objective: a metric is already perfect with endpoints only"); + m_scale[m] = config.term_weights[m] / (weight_sum * (1.0 - m_floor[m])); + } + m_endpoint_separable.assign(n, 0.0); + core::parallel_for(n, [&](std::size_t i) { + const auto endpoint_terms = raw_separable( + i, PreparedTrajectory(db.trajectory(i), endpoints[i]).view(), !config.ranking_knn); + for (const auto m : {Knn, Range, WhereAt}) + m_endpoint_separable[i] += m_scale[m] * endpoint_terms[m]; + }); +} + +bool Objective::match(const core::TrajectoryView &a, const core::TrajectoryView &b, + std::size_t term) const { + return query::frechet_within(a, b, m_eps[term], m_workloads.frechet); +} + +Objective::Terms Objective::raw_separable(std::size_t i, std::span kept) const { + return raw_separable(i, PreparedTrajectory(m_db.trajectory(i), kept).view()); +} + +Objective::Terms Objective::raw_separable(std::size_t i, const core::TrajectoryView &candidate, + bool knn) const { + Terms terms{}; + + if (knn) { + for (const auto &membership : m_knn[i]) + if (query::frechet_within(m_db.trajectory(membership.query), candidate, + membership.cutoff, m_workloads.frechet)) + terms[Knn] += m_knn_weight; + } + + for (const auto &[b, weight] : m_range[i]) { + const auto &box = m_workloads.range.queries[b].box; + for (std::size_t k = 0; k < candidate.size(); ++k) + if (inside(box, candidate, k)) { + terms[Range] += weight; + break; + } + } + + if (!m_probes[i].empty()) { + double sum = 0; + for (const auto k : m_probes[i]) { + const auto &probe = m_workloads.where_at.queries[k]; + const double error = + core::distance(core::position_at(candidate, probe.t), probe.reference); + sum += std::clamp((m_config.max_tolerance_m - error) / + (m_config.max_tolerance_m - m_config.min_tolerance_m), + 0.0, 1.0); + } + terms[WhereAt] = m_where_at_weight * sum / static_cast(m_probes[i].size()); + } + return terms; +} + +double Objective::separable(std::size_t i, std::span kept) const { + return separable(i, PreparedTrajectory(m_db.trajectory(i), kept).view()); +} + +double Objective::separable(std::size_t i, const core::TrajectoryView &candidate) const { + const auto t = raw_separable(i, candidate, !m_config.ranking_knn); + return m_scale[Knn] * t[Knn] + m_scale[Range] * t[Range] + m_scale[WhereAt] * t[WhereAt] - + m_endpoint_separable[i]; +} + +double Objective::independent_value(std::size_t i, const core::TrajectoryView &candidate, + std::span partners, + double beat) const { + return independent_value(prepare_independent(i, candidate, partners), beat); +} + +Objective::IndependentScore +Objective::prepare_independent(std::size_t i, const core::TrajectoryView &candidate, + std::span partners) const { + IndependentScore score; + score.trajectory = i; + score.candidate = candidate; + score.terms = raw_separable(i, candidate, false); + std::array hits{}, found{}; + for (const auto &link : m_links[i]) { + // Held out: the partner is the original query and i the database side + const bool matched = + m_term_held_out[link.term] + ? match(m_db.trajectory(link.other), candidate, link.term) + : match(candidate, + partners.empty() ? m_db.trajectory(link.other) : partners[link.other], + link.term); + found[link.term] += matched; + hits[link.term] += matched && link.positive; + } + double f1_sum = 0; + for (std::size_t k = 0; k < m_eps.size(); ++k) + if (found[k] + m_truth[slot(i, k)] != 0) + f1_sum += f1_of(hits[k], found[k], m_truth[slot(i, k)]); + score.join = + m_with_truth != 0 ? m_scale[Join] * f1_sum / static_cast(m_with_truth) : 0.0; + score.bound = independent_sum(score, score.terms[Knn]) + + m_scale[Knn] * m_knn_weight * static_cast(m_knn[i].size()); + return score; +} + +double Objective::independent_sum(const IndependentScore &score, double knn) const { + // Same sums in the same order as separable, so the value is identical to the last bit + return m_scale[Knn] * knn + m_scale[Range] * score.terms[Range] + + m_scale[WhereAt] * score.terms[WhereAt] - m_endpoint_separable[score.trajectory] + + score.join; +} + +double Objective::independent_value(const IndependentScore &score, double beat) const { + const auto i = score.trajectory; + double knn = score.terms[Knn]; + auto value = [&] { return independent_sum(score, knn); }; + // Stop once even all remaining hits cannot beat beat + const double per_hit = m_scale[Knn] * m_knn_weight; + std::size_t left = m_knn[i].size(); + for (const auto &m : m_knn[i]) { + if (value() + per_hit * static_cast(left) < beat - kBoundSlack) + return value(); + --left; + if (query::frechet_within(m_db.trajectory(m.query), score.candidate, m.cutoff, + m_workloads.frechet)) + knn += m_knn_weight; + } + return value(); +} + +double Objective::independent_bound(std::size_t i, const core::TrajectoryView &candidate, + std::span partners) const { + return prepare_independent(i, candidate, partners).bound; +} + +double Objective::join_metric(std::span state) const { + // matched[p]: the pair's outcome, counted for i. reverse[p]: counted for j, the same outcome + // unless held out, where each side is in turn the original query + std::vector matched(m_pair_i.size()), reverse(m_pair_i.size()); + core::parallel_for(m_pair_i.size(), [&](std::size_t p) { + const auto i = m_pair_i[p], j = m_pair_j[p]; + const auto k = m_pair_term[p]; + if (m_term_held_out[k]) { + matched[p] = match(m_db.trajectory(i), state[j], k); + reverse[p] = match(m_db.trajectory(j), state[i], k); + } else { + matched[p] = reverse[p] = match(state[i], state[j], k); + } + }); + std::vector hits(m_truth.size(), 0), found(m_truth.size(), 0); + for (std::size_t p = 0; p < m_pair_i.size(); ++p) { + const bool positive = p < m_positive_count; + const auto slot_i = slot(m_pair_i[p], m_pair_term[p]); + const auto slot_j = slot(m_pair_j[p], m_pair_term[p]); + if (matched[p]) { + ++found[slot_i]; + hits[slot_i] += positive; + } + if (reverse[p]) { + ++found[slot_j]; + hits[slot_j] += positive; + } + } + double sum = 0; + std::size_t counted = 0; + for (std::size_t t = 0; t < m_truth.size(); ++t) + if (found[t] + m_truth[t] != 0) { + sum += f1_of(hits[t], found[t], m_truth[t]); + ++counted; + } + return counted ? sum / static_cast(counted) : 0.0; +} + +Objective::Terms Objective::metrics(const KeptSets &kept) const { + if (kept.size() != m_db.size()) + throw std::invalid_argument("objective: one kept set per trajectory needed"); + const PreparedTrajectories prepared(m_db, kept); + const auto views = prepared.views(); + std::vector per(kept.size()); + core::parallel_for(kept.size(), [&](std::size_t i) { + per[i] = raw_separable(i, views[i], !m_config.ranking_knn); + }); + Terms total{}; + for (const auto &terms : per) + for (const auto m : {Knn, Range, WhereAt}) + total[m] += terms[m]; + total[Join] = join_metric(views); + if (m_config.ranking_knn) + total[Knn] = knn_recall(core::materialize(m_db, kept)); + return total; +} + +double Objective::knn_recall(const core::TrajectoryDb &compressed) const { + const std::size_t k = m_workloads.k; + std::vector hits(m_db.size(), 0); + core::parallel_for(m_db.size(), [&](std::size_t e) { + const auto &entry = m_workloads.knn[e]; + if (!entry.valid) + return; + auto found = query::knn(compressed, m_db.trajectory(e), std::min(k + 1, m_db.size()), + m_workloads.frechet); + std::erase(found, e); + found.resize(std::min(found.size(), k)); + for (const auto i : entry.top) + hits[e] += std::ranges::find(found, std::size_t{i}) != found.end(); + }); + return m_knn_weight * static_cast(std::accumulate(hits.begin(), hits.end(), 0ul)); +} + +double Objective::total(const KeptSets &kept) const { + const auto values = metrics(kept); + double sum = 0; + for (std::size_t k = 0; k < TermCount; ++k) + sum += m_scale[k] * (values[k] - m_floor[k]); + return sum; +} + +JoinState::JoinState(const Objective &objective, std::span state) + : m_objective(objective) { + const std::size_t n = state.size(); + m_matched.assign(2 * objective.pair_count(), 0); + m_hits.assign(objective.slots(), 0); + m_found.assign(objective.slots(), 0); + // Each pair once, from its smaller side. Held out: each direction from its compressed side + std::vector>> outcome(n); + core::parallel_for(n, [&](std::size_t i) { + for (const auto &link : objective.links(i)) { + if (objective.held_out(link.term)) + outcome[i].push_back( + {slot(i, link), + objective.match(objective.db().trajectory(link.other), state[i], link.term)}); + else if (i < link.other) + outcome[i].push_back( + {slot(i, link), objective.match(state[i], state[link.other], link.term)}); + } + }); + for (std::size_t i = 0; i < n; ++i) + for (const auto &[at, matched] : outcome[i]) + m_matched[at] = matched; + for (std::size_t i = 0; i < n; ++i) + for (const auto &link : objective.links(i)) + if (m_matched[slot(i, link)]) { + // Held out, the match is found by the query, not by the compressed side + const std::size_t t = count(objective.held_out(link.term) ? link.other : i, link); + ++m_found[t]; + m_hits[t] += link.positive; + } + for (std::size_t t = 0; t < objective.slots(); ++t) + if (counted(t, m_found[t])) { + m_sum += f1(t, m_hits[t], m_found[t]); + ++m_counted; + } +} + +bool JoinState::counted(std::size_t t, std::size_t found) const { + return found + m_objective.truth(t) != 0; +} + +double JoinState::f1(std::size_t t, std::size_t hits, std::size_t found) const { + return f1_of(hits, found, m_objective.truth(t)); +} + +double JoinState::metric() const { + return m_counted ? m_sum / static_cast(m_counted) : 0.0; +} + +JoinState::Proposal JoinState::propose(std::size_t i, const core::TrajectoryView &candidate, + std::span state) const { + Proposal proposal{i, 0.0, {}}; + proposal.matched.reserve(m_objective.links(i).size()); + // New outcomes of i's pairs, and the hits/found changes they cause on each side + std::unordered_map> change; // t -> (hits, found) + for (const auto &link : m_objective.links(i)) { + const bool now = m_matched[slot(i, link)]; + const bool then = + m_objective.held_out(link.term) + ? m_objective.match(m_objective.db().trajectory(link.other), candidate, link.term) + : m_objective.match(candidate, state[link.other], link.term); + proposal.matched.push_back(then); + if (now == then) + continue; + const long delta = then ? 1 : -1; + const std::array touched{count(link.other, link), count(i, link)}; + for (const auto t : std::span(touched).first(sides(link))) { + auto &[hits, found] = change[t]; + found += delta; + hits += link.positive ? delta : 0; + } + } + if (change.empty()) + return proposal; + double sum = m_sum; + long n_counted = static_cast(m_counted); + for (const auto &[t, counts] : change) { + const auto old_found = m_found[t], old_hits = m_hits[t]; + const auto new_found = + static_cast(static_cast(old_found) + counts.second); + const auto new_hits = static_cast(static_cast(old_hits) + counts.first); + if (counted(t, old_found)) { + sum -= f1(t, old_hits, old_found); + --n_counted; + } + if (counted(t, new_found)) { + sum += f1(t, new_hits, new_found); + ++n_counted; + } + } + const double after = n_counted ? sum / static_cast(n_counted) : 0.0; + proposal.gain = m_objective.scale(Objective::Join) * (after - metric()); + return proposal; +} + +void JoinState::apply(const Proposal &proposal) { + const auto i = proposal.trajectory; + const auto links = m_objective.links(i); + if (proposal.matched.size() != links.size()) + throw std::invalid_argument("join proposal: outcome count does not match links"); + for (std::size_t k = 0; k < links.size(); ++k) { + const auto &link = links[k]; + const bool now = m_matched[slot(i, link)]; + const bool then = proposal.matched[k]; + if (now == then) + continue; + m_matched[slot(i, link)] = then; + const long delta = then ? 1 : -1; + const std::array touched{count(link.other, link), count(i, link)}; + for (const auto t : std::span(touched).first(sides(link))) { + if (counted(t, m_found[t])) { + m_sum -= f1(t, m_hits[t], m_found[t]); + --m_counted; + } + m_found[t] = static_cast(static_cast(m_found[t]) + delta); + if (link.positive) + m_hits[t] = static_cast(static_cast(m_hits[t]) + delta); + if (counted(t, m_found[t])) { + m_sum += f1(t, m_hits[t], m_found[t]); + ++m_counted; + } + } + } +} + +KnnState::KnnState(const Objective &objective, const KeptSets &state, + std::span watch_states, std::size_t watch) + : m_objective(objective) { + const auto &db = objective.db(); + const auto &workloads = objective.workloads(); + const std::size_t n = db.size(); + m_list.resize(n); + m_distance.resize(n); + m_hits.assign(n, 0); + m_watchers.resize(n); + std::vector compressed; + compressed.push_back(core::materialize(db, state)); + for (const auto &watch_state : watch_states) + compressed.push_back(core::materialize(db, watch_state)); + core::parallel_for(n, [&](std::size_t e) { + const auto &entry = workloads.knn[e]; + if (!entry.valid) + return; + auto &list = m_list[e]; + list.assign(entry.top.begin(), entry.top.end()); + for (const auto &compressed_db : compressed) + for (const auto j : query::knn(compressed_db, db.trajectory(e), std::min(watch + 1, n), + workloads.frechet)) + if (j != e && std::ranges::find(list, j) == list.end()) + list.push_back(static_cast(j)); + }); + for (std::size_t e = 0; e < n; ++e) { + m_valid += workloads.knn[e].valid; + m_distance[e].assign(m_list[e].size(), 0.0); + for (std::size_t p = 0; p < m_list[e].size(); ++p) + m_watchers[m_list[e][p]].push_back( + {static_cast(e), static_cast(p)}); + } + const PreparedTrajectories prepared(db, state); + const auto views = prepared.views(); + std::vector> own(n); + core::parallel_for(n, [&](std::size_t i) { own[i] = distances(i, views[i]); }); + for (std::size_t i = 0; i < n; ++i) + for (std::size_t watch_state = 0; watch_state < m_watchers[i].size(); ++watch_state) + m_distance[m_watchers[i][watch_state].query][m_watchers[i][watch_state].position] = + own[i][watch_state]; + core::parallel_for(n, [&](std::size_t e) { + if (!m_list[e].empty()) + m_hits[e] = hits(e, m_list[e].size(), 0.0); + }); + m_hit_sum = std::accumulate(m_hits.begin(), m_hits.end(), 0ul); +} + +std::vector KnnState::distances(std::size_t i, + const core::TrajectoryView &candidate) const { + const auto &db = m_objective.db(); + std::vector result; + result.reserve(m_watchers[i].size()); + for (const auto &watcher : m_watchers[i]) + result.push_back(query::frechet_distance(db.trajectory(watcher.query), candidate, + m_objective.workloads().frechet)); + return result; +} + +std::size_t KnnState::hits(std::size_t e, std::size_t position, double distance) const { + const auto &list = m_list[e]; + const std::size_t k = std::min(m_objective.workloads().k, list.size()); + const std::size_t members = m_objective.workloads().knn[e].top.size(); + std::vector> order(list.size()); + for (std::size_t p = 0; p < list.size(); ++p) + order[p] = {p == position ? distance : m_distance[e][p], static_cast(p)}; + // Ties go to the lower database index, like query::knn + std::ranges::partial_sort( + order, order.begin() + static_cast(k), [&](const auto &a, const auto &b) { + return a.first < b.first || (a.first == b.first && list[a.second] < list[b.second]); + }); + std::size_t count = 0; + for (std::size_t p = 0; p < k; ++p) + count += order[p].second < members; + return count; +} + +double KnnState::propose(std::size_t i, std::span distances) const { + long change = 0; + for (std::size_t s = 0; s < m_watchers[i].size(); ++s) { + const auto &watcher = m_watchers[i][s]; + change += static_cast(hits(watcher.query, watcher.position, distances[s])) - + static_cast(m_hits[watcher.query]); + } + return m_objective.scale(Objective::Knn) * static_cast(change) / + static_cast(m_objective.workloads().k * m_valid); +} + +void KnnState::apply(std::size_t i, std::span distances) { + for (std::size_t s = 0; s < m_watchers[i].size(); ++s) { + const auto &watcher = m_watchers[i][s]; + m_distance[watcher.query][watcher.position] = distances[s]; + m_hit_sum -= m_hits[watcher.query]; + m_hits[watcher.query] = hits(watcher.query, m_list[watcher.query].size(), 0.0); + m_hit_sum += m_hits[watcher.query]; + } +} + +double KnnState::metric() const { + return static_cast(m_hit_sum) / + static_cast(m_objective.workloads().k * m_valid); +} + +} // namespace teacher diff --git a/engine/src/teacher/teacher.cpp b/engine/src/teacher/teacher.cpp new file mode 100644 index 0000000..569c1cd --- /dev/null +++ b/engine/src/teacher/teacher.cpp @@ -0,0 +1,690 @@ +#include "backend/engine/teacher/teacher.hpp" + +#include "backend/engine/allocation/budget_allocator.hpp" +#include "backend/engine/core/parallel.hpp" +#include "backend/engine/core/timer.hpp" +#include "backend/engine/selection/rdp.hpp" +#include "backend/engine/selection/sed.hpp" +#include "backend/engine/selection/selector.hpp" +#include "backend/engine/teacher/curves.hpp" +#include "backend/engine/teacher/greedy.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace teacher { + +namespace { + +using selection::AlgorithmType; +enum CandidateIndex : std::size_t { + kRdp, + kSed, + kUniform, + kGreedy, + kGreedySedHalf, + kGreedyRdpHalf, + kCandidateCount, +}; + +// Classic selectors, then greedy seeded with nothing, or SED's or RDP's best half +constexpr std::array kCandidates{ + "RDP", "SED", "UNIFORM", "GREEDY", "GREEDY_SED2", "GREEDY_RDP2"}; +constexpr std::size_t kMaxResponsePasses = 10; +constexpr std::size_t kKnnWatchFactor = 3; +constexpr std::size_t kMaxAllocationRounds = 3; +constexpr double kMinImprovement = 1e-12; + +// candidates[c][i]: trajectory i simplified by kCandidates[c] at budgets[i] +using Candidates = std::array; +using Methods = std::vector; // per trajectory + +std::vector interior(const std::vector &kept) { + return kept.size() <= 2 ? std::vector{} + : std::vector(kept.begin() + 1, kept.end() - 1); +} + +// The first candidate up to c that keeps the same points as c, kept(k) giving candidate k's set +template std::size_t first_equal(std::size_t c, Kept &&kept) { + std::size_t first = 0; + while (first < c && kept(first) != kept(c)) + ++first; + return first; +} + +// One trajectory's candidates up to cap. A candidate depends only on (i, c, b, cap), so curves +// and final kept sets agree +class CandidateMaker { + public: + CandidateMaker(const Objective &objective, std::size_t i, std::size_t cap, + bool reuse_sequences = false) + : m_original(objective.db().trajectory(i)), m_cap(std::min(cap, m_original.size())), + m_objective(objective), m_index(i), m_reuse_sequences(reuse_sequences) {} + + std::vector make(std::size_t candidate, std::size_t budget) { + budget = std::min(budget, m_original.size()); + if (budget > m_cap) + throw std::runtime_error("teacher: candidate budget above its cap"); + if (budget <= 2 || budget == m_original.size()) + return selection::simplify_trajectory(m_original, AlgorithmType::UNIFORM, budget); + const std::size_t interior_budget = budget - 2; + switch (candidate) { + case kRdp: + return simplify(AlgorithmType::RDP, budget); + case kSed: + return simplify(AlgorithmType::SED, budget); + case kUniform: + return simplify(AlgorithmType::UNIFORM, budget); + case kGreedy: + return prefix(greedy_sequence(0, AlgorithmType::SED), budget); + case kGreedySedHalf: + return prefix(greedy_sequence(interior_budget / 2, AlgorithmType::SED), budget); + case kGreedyRdpHalf: + return prefix(greedy_sequence(interior_budget / 2, AlgorithmType::RDP), budget); + default: + throw std::logic_error("teacher: unknown candidate"); + } + } + + private: + std::vector simplify(AlgorithmType type, std::size_t b) { + if (!m_reuse_sequences || type == AlgorithmType::UNIFORM) + return selection::simplify_trajectory(m_original, type, b); + auto &cached = type == AlgorithmType::RDP ? m_rdp : m_sed; + if (!cached) + cached = type == AlgorithmType::RDP ? selection::rdp::sequence(m_original, m_cap) + : selection::sed::sequence(m_original, m_cap); + return prefix(*cached, b); + } + static std::vector prefix(const std::vector &sequence, + std::size_t b) { + std::vector kept(sequence.begin(), + sequence.begin() + static_cast(b)); + std::ranges::sort(kept); + return kept; + } + const std::vector &greedy_sequence(std::size_t seed_size, + AlgorithmType seed_algorithm) { + if (!m_greedy) { + m_greedy.emplace(m_objective, m_index); + // SED's and RDP's points in their own order of importance, then UNIFORM's; greedy + // skips points it already keeps + m_fill = selection::sed::sequence(m_original, m_cap); + const auto rdp_order = selection::rdp::sequence(m_original, m_cap); + m_fill.insert(m_fill.end(), rdp_order.begin(), rdp_order.end()); + const auto uniform = + selection::simplify_trajectory(m_original, AlgorithmType::UNIFORM, m_cap); + m_fill.insert(m_fill.end(), uniform.begin(), uniform.end()); + } + const auto seed_type = seed_size ? seed_algorithm : AlgorithmType::SED; + auto &cached = m_greedy_sequences[{seed_type, seed_size}]; + if (cached.empty()) { + const auto seed = seed_size ? interior(simplify(seed_algorithm, 2 + seed_size)) + : std::vector{}; + cached = m_greedy->sequence(m_cap, seed, m_fill); + } + return cached; + } + + core::TrajectoryView m_original; + std::size_t m_cap; + const Objective &m_objective; + std::size_t m_index; + bool m_reuse_sequences; + std::optional m_greedy; + std::optional> m_rdp, m_sed; + std::vector m_fill; + std::map, std::vector> m_greedy_sequences; +}; + +// All candidates at each trajectory's budget, or only the selected one +Candidates make_candidates(const Objective &objective, const std::vector &budgets, + const std::vector &caps, + const Methods *selected = nullptr) { + Candidates candidates; + for (auto &kept : candidates) + kept.resize(budgets.size()); + core::parallel_for(budgets.size(), [&](std::size_t i) { + // Greedy needs the classic sequences for its seed and fill + CandidateMaker maker(objective, i, caps[i], !selected || (*selected)[i] >= kGreedy); + if (selected) { + const auto c = (*selected)[i]; + candidates[c][i] = maker.make(c, budgets[i]); + } else { + for (std::size_t c = 0; c < kCandidates.size(); ++c) + candidates[c][i] = maker.make(c, budgets[i]); + } + }); + return candidates; +} + +KeptSets assemble(const Candidates &candidates, const Methods &method) { + KeptSets kept(method.size()); + for (std::size_t i = 0; i < method.size(); ++i) + kept[i] = candidates[method[i]][i]; + return kept; +} + +struct Pick { + std::size_t method; + double value, rdp; +}; + +// Best candidate by independent_value, ties to the lower index. RDP is scored exactly, the rest +// best bound first so losers skip most kNN checks +Pick pick(const Objective &objective, std::size_t i, + std::span> options, + std::span partners) { + const double rdp = objective.independent_value(i, options[0].second, partners); + Pick best{options[0].first, rdp, rdp}; + std::vector scores; + scores.reserve(options.size() - 1); + std::vector> order; // bound, position in options + order.reserve(options.size() - 1); + for (std::size_t k = 1; k < options.size(); ++k) { + scores.push_back(objective.prepare_independent(i, options[k].second, partners)); + order.push_back({scores.back().bound, k}); + } + std::ranges::sort(order, std::greater{}); + for (const auto &[bound, k] : order) { + if (bound < best.value - kBoundSlack) + break; + const auto c = options[k].first; + const double value = objective.independent_value(scores[k - 1], best.value); + if (value > best.value || (value == best.value && c < best.method)) { + best.value = value; + best.method = c; + } + } + return best; +} + +Methods independent_choice(const core::TrajectoryDb &db, const Objective &objective, + const Candidates &candidates, const KeptSets *partners) { + Methods method(db.size()); + std::optional prepared; + std::vector partner_views; + if (partners) { + prepared.emplace(db, *partners); + partner_views = prepared->views(); + } + core::parallel_for(db.size(), [&](std::size_t i) { + std::vector owned; + std::vector> options; + owned.reserve(kCandidates.size()); + for (std::size_t c = 0; c < kCandidates.size(); ++c) { + owned.emplace_back(db.trajectory(i), candidates[c][i]); + options.push_back({c, owned.back().view()}); + } + method[i] = pick(objective, i, options, partner_views).method; + }); + return method; +} + +// Value curves up to caps[i], best candidate and RDP alone, partners fixed +struct Curves { + std::vector best, rdp; +}; + +Curves build_curves(const core::TrajectoryDb &db, const Objective &objective, + const KeptSets &partners, const std::vector &caps, bool rdp_only) { + Curves curves{std::vector(db.size()), std::vector(db.size())}; + const PreparedTrajectories prepared(db, partners); + const auto partner_views = prepared.views(); + core::parallel_for(db.size(), [&](std::size_t i) { + CandidateMaker maker(objective, i, caps[i], true); + const std::size_t max_b = std::min(caps[i], db.trajectory(i).size()); + auto &best = curves.best[i]; + auto &rdp = curves.rdp[i]; + best.value.reserve(max_b - 1); + best.method.reserve(max_b - 1); + rdp.value.reserve(max_b - 1); + rdp.method.reserve(max_b - 1); + std::array, kCandidates.size()> kept; + std::vector owned; + std::vector> options; + owned.reserve(kCandidates.size()); + options.reserve(kCandidates.size()); + for (std::size_t b = 2; b <= max_b; ++b) { + options.clear(); + for (std::size_t c = 0; c < (rdp_only ? 1 : kCandidates.size()); ++c) { + kept[c] = maker.make(c, b); + if (first_equal(c, [&](std::size_t k) -> const auto & { return kept[k]; }) != c) + continue; + const auto slot = options.size(); + if (slot == owned.size()) + owned.emplace_back(db.trajectory(i), kept[c]); + else + owned[slot].assign(db.trajectory(i), kept[c]); + options.push_back({c, owned[slot].view()}); + } + const auto chosen = pick(objective, i, options, partner_views); + rdp.value.push_back(chosen.rdp); + rdp.method.push_back(kRdp); + best.value.push_back(chosen.value); + best.method.push_back(static_cast(chosen.method)); + } + }); + return curves; +} + +Methods curve_methods(const std::vector &curves, const std::vector &budgets) { + Methods method(curves.size()); + for (std::size_t i = 0; i < curves.size(); ++i) { + if (budgets[i] < 2 || budgets[i] > curves[i].max_budget()) + throw std::runtime_error("teacher: budget outside its value curve"); + method[i] = curves[i].method[budgets[i] - 2]; + } + return method; +} + +// The loop: coordinate ascent on the tracked J_T, one trajectory at a time in random order. +// Set up once; run() copies the trackers, so the loop and its controls share the setup +class Search { + public: + Search(const core::TrajectoryDb &db, const Objective &objective, const Candidates &candidates, + const Methods &method) + : m_start(method) { + if (objective.config().ranking_knn) + m_knn.emplace(objective, assemble(candidates, method), candidates, + kKnnWatchFactor * objective.workloads().k); + const auto active = [&](std::size_t i) { + return !objective.links(i).empty() || (m_knn && m_knn->watched(i)); + }; + for (std::size_t i = 0; i < db.size(); ++i) + if (active(i)) + m_order.push_back(i); + m_prepared.resize(db.size()); + for (auto &views : m_views) + views.resize(db.size()); + m_separable.resize(db.size()); + m_representative.resize(db.size()); + m_knn_distances.resize(db.size()); + core::parallel_for(db.size(), [&](std::size_t i) { + auto &owned = m_prepared[i]; + const auto original = db.trajectory(i); + if (!active(i)) { + owned.emplace_back(original, candidates[method[i]][i]); + m_views[method[i]][i] = owned.back().view(); + return; + } + // Reserve first: views borrow from these + owned.reserve(kCandidates.size()); + for (std::size_t c = 0; c < kCandidates.size(); ++c) { + const auto first = + first_equal(c, [&](std::size_t k) -> const auto & { return candidates[k][i]; }); + m_representative[i][c] = first; + if (first == c) { + owned.emplace_back(original, candidates[c][i]); + m_views[c][i] = owned.back().view(); + m_separable[i][c] = objective.separable(i, m_views[c][i]); + if (m_knn) + m_knn_distances[i][c] = m_knn->distances(i, m_views[c][i]); + } else { + m_views[c][i] = m_views[first][i]; + m_separable[i][c] = m_separable[i][first]; + } + } + }); + m_state.resize(db.size()); + for (std::size_t i = 0; i < db.size(); ++i) + m_state[i] = m_views[method[i]][i]; + m_join.emplace(objective, m_state); + } + Search(const Search &) = delete; + Search &operator=(const Search &) = delete; + + // update = false: one pass against the starting state + Methods run(std::uint64_t seed, bool update = true) const { + auto method = m_start; + auto state = m_state; + auto join = *m_join; + auto knn = m_knn; + auto order = m_order; + std::mt19937_64 rng(seed + kOrderSeedOffset); + + for (std::size_t pass = 1; pass <= kMaxResponsePasses; ++pass) { + std::ranges::shuffle(order, rng); + std::size_t switches = 0; + for (const auto i : order) { + const double here = m_separable[i][method[i]]; + std::size_t best = method[i]; + JoinState::Proposal best_proposal; + double best_gain = kMinImprovement; // strict gains only, so ties cannot cycle + for (std::size_t c = 0; c < kCandidates.size(); ++c) { + // Each distinct kept set is tried once, and never the one i already has + const bool duplicate = m_representative[i][c] != c; + const bool current = m_representative[i][c] == m_representative[i][method[i]]; + if (duplicate || current) + continue; + auto proposal = join.propose(i, m_views[c][i], state); + const double knn_gain = knn ? knn->propose(i, m_knn_distances[i][c]) : 0.0; + const double gain = m_separable[i][c] - here + proposal.gain + knn_gain; + if (gain > best_gain) { + best = c; + best_gain = gain; + best_proposal = std::move(proposal); + } + } + if (best == method[i]) + continue; + method[i] = best; + ++switches; + if (!update) + continue; + state[i] = m_views[best][i]; + join.apply(best_proposal); + if (knn) + knn->apply(i, m_knn_distances[i][best]); + } + std::cout << std::format("teacher: pass {}: {} switches, join F1 {:.4f}{}\n", pass, + switches, join.metric(), + knn ? std::format(", knn {:.4f}", knn->metric()) : ""); + if (switches == 0 || !update) + break; + } + return method; + } + + private: + Methods m_start; + std::vector m_order; + std::vector> m_prepared; + std::array, kCandidates.size()> m_views; + std::vector> m_separable; + std::vector> m_representative; + std::vector, kCandidates.size()>> m_knn_distances; + std::optional m_knn; + std::vector m_state; + std::optional m_join; +}; + +Methods best_response(const core::TrajectoryDb &db, const Objective &objective, + const Candidates &candidates, Methods method, std::uint64_t seed, + bool update = true) { + return Search(db, objective, candidates, method).run(seed, update); +} + +// The workloads with trajectories relabelled among those sharing 1 km start and end cells, in +// every join pair and kNN neighbour list: wrong but plausible partners +TeacherWorkloads shuffled_partners(const core::TrajectoryDb &db, const TeacherWorkloads &workloads, + std::uint64_t seed) { + constexpr double kCell = 1000.0; + std::map, std::vector> buckets; + for (std::size_t i = 0; i < db.size(); ++i) { + const auto trajectory = db.trajectory(i); + auto cell = [&](double coordinate) { + return static_cast(std::floor(coordinate / kCell)); + }; + buckets[{cell(trajectory.x.front()), cell(trajectory.y.front()), cell(trajectory.x.back()), + cell(trajectory.y.back())}] + .push_back(static_cast(i)); + } + std::vector label(db.size()); + std::mt19937_64 rng(seed + kShuffleSeedOffset); + for (auto &[key, members] : buckets) { + auto shuffled = members; + std::ranges::shuffle(shuffled, rng); + for (std::size_t k = 0; k < members.size(); ++k) + label[members[k]] = shuffled[k]; + } + + auto result = workloads; + std::set> real; + for (const auto &pair : workloads.positives) + real.insert({pair.i, pair.j}); + std::size_t kept_real = 0; + std::vector *> lists{&result.positives, &result.negatives}; + for (auto &extra : result.extra_joins) { + lists.push_back(&extra.positives); + lists.push_back(&extra.negatives); + } + for (auto *pairs : lists) + for (auto &pair : *pairs) { + const auto first = label[pair.i], second = label[pair.j]; + pair.i = std::min(first, second); + pair.j = std::max(first, second); + kept_real += pairs == &result.positives && real.contains({pair.i, pair.j}); + } + std::size_t kept_neighbours = 0, neighbours = 0; + for (std::size_t e = 0; e < result.knn.size(); ++e) { + std::vector relabelled; + for (const auto j : result.knn[e].top) { + const auto to = label[j] == e ? j : label[j]; + if (std::ranges::find(relabelled, to) == relabelled.end()) + relabelled.push_back(to); + kept_neighbours += to == j; + ++neighbours; + } + result.knn[e].top = std::move(relabelled); + } + std::cout << std::format( + "teacher: shuffled in {} buckets, {} of {} positive pairs and {} of {} " + "kNN neighbours still real\n", + buckets.size(), kept_real, result.positives.size(), kept_neighbours, neighbours); + return result; +} + +std::string wins(const Methods &method) { + std::array count{}; + for (const auto candidate : method) + ++count[candidate]; + std::string out; + for (std::size_t c = 0; c < kCandidates.size(); ++c) + out += std::format("{}{} {}", c ? ", " : "", kCandidates[c], count[c]); + return out; +} + +std::vector collect(const std::map> &outputs, + std::span arms) { + std::vector result; + result.reserve(arms.size()); + for (const auto &name : arms) + result.push_back(outputs.at(name)); + return result; +} + +} // namespace + +bool is_arm(std::string_view name) { return std::ranges::find(kArms, name) != kArms.end(); } + +void check_arms(std::span arms) { + for (auto a = arms.begin(); a != arms.end(); ++a) { + if (!is_arm(*a)) + throw std::invalid_argument("unknown teacher arm '" + *a + "'"); + if (std::find(arms.begin(), a, *a) != a) + throw std::invalid_argument("teacher arm '" + *a + "' listed twice"); + } +} + +Teacher::Teacher(const core::TrajectoryDb &db, TeacherWorkloads workloads, + const ObjectiveConfig &config, std::uint64_t seed) + : m_db(db), m_workloads(std::move(workloads)), m_objective(db, m_workloads, config), + m_seed(seed) { + const auto &floors = m_objective.floors(); + std::cout << std::format("teacher: {} join pairs ({} positive), floors knn {:.4f}, range " + "{:.4f}, where-at {:.4f}, join {:.4f}\n", + m_objective.pair_count(), m_objective.positive_count(), + floors[Objective::Knn], floors[Objective::Range], + floors[Objective::WhereAt], floors[Objective::Join]); +} + +std::vector Teacher::run(std::size_t total_budget, std::span arms) const { + check_arms(arms); + const auto requested = [&](std::string_view name) { + return std::ranges::find(arms, name) != arms.end(); + }; + const bool want_choice = requested("EQUAL_CHOICE"); + const bool want_guess = requested("EQUAL_CHOICE_RDP_PARTNERS"); + const bool want_equal_loop = requested("EQUAL_LOOP"); + const bool want_budget_rdp = requested("BUDGET_RDP"); + const bool want_budget_choice = requested("BUDGET_CHOICE"); + const bool want_loop = requested("BUDGET_LOOP"); + const bool want_reallocate = requested("BUDGET_LOOP_REALLOCATE"); + const bool want_shuffled = requested("BUDGET_LOOP_SHUFFLED"); + const bool want_no_update = requested("BUDGET_LOOP_NO_UPDATE"); + const bool needs_loop_candidates = + want_loop || want_reallocate || want_shuffled || want_no_update; + const bool needs_budget_candidates = want_budget_choice || needs_loop_candidates; + const bool needs_curves = want_budget_rdp || needs_budget_candidates; + + const auto &db = m_db; + const auto &objective = m_objective; + const auto equal_budgets = allocation::create_trajectory_budgets( + db, allocation::AllocatorType::SizeProportional, total_budget, 0.0); + // About 4x the mean interior budget, at least 6, never below the equal share + const double mean_interior = + static_cast(total_budget - 2 * db.size()) / static_cast(db.size()); + const auto cap = 2 + static_cast(std::ceil(std::max(6.0, 4.0 * mean_interior))); + std::vector caps(db.size()); + for (std::size_t i = 0; i < db.size(); ++i) + caps[i] = std::min(std::max(cap, equal_budgets[i]), db.trajectory(i).size()); + + std::map> outputs; + auto keep = [&](std::string_view name, const Candidates &from, const Methods &method) { + auto kept = assemble(from, method); + const double j_t = objective.total(kept); + std::cout << std::format("teacher {}: {}, J_T {:.4f}\n", name, wins(method), j_t); + outputs[std::string(name)] = {std::string(name), core::to_simplification(kept), j_t}; + }; + auto start = std::chrono::steady_clock::now(); + auto timed = [&](std::string_view step) { + std::cout << std::format("teacher: {} {:.1f} s\n", step, core::seconds_since(start)); + start = std::chrono::steady_clock::now(); + }; + const Methods rdp_methods(db.size(), kRdp); + const bool equal_alternatives = want_choice || want_guess || want_equal_loop; + const auto equal_candidates = make_candidates(objective, equal_budgets, caps, + equal_alternatives ? nullptr : &rdp_methods); + timed("candidates"); + + if (want_choice) { + keep("EQUAL_CHOICE", equal_candidates, + independent_choice(db, objective, equal_candidates, nullptr)); + timed("EQUAL_CHOICE"); + } + if (!(want_guess || want_equal_loop || needs_curves)) + return collect(outputs, arms); + + // Partners are fixed at RDP with equal budgets, so trajectories score independently + Curves curves; + if (needs_curves) { + const bool rdp_only = !(want_guess || want_equal_loop || needs_budget_candidates); + curves = build_curves(db, objective, equal_candidates[kRdp], caps, rdp_only); + timed(std::format("value curves up to {} points", cap)); + } + if (want_guess || want_equal_loop) { + const auto guess = needs_curves ? curve_methods(curves.best, equal_budgets) + : independent_choice(db, objective, equal_candidates, + &equal_candidates[kRdp]); + if (want_guess) + keep("EQUAL_CHOICE_RDP_PARTNERS", equal_candidates, guess); + if (want_equal_loop) { + keep("EQUAL_LOOP", equal_candidates, + best_response(db, objective, equal_candidates, guess, m_seed)); + timed("EQUAL_LOOP"); + } + } + auto report = [&](std::string_view what, double value, + const std::vector &budgets) { + std::size_t at_cap = 0; + for (std::size_t i = 0; i < budgets.size(); ++i) + at_cap += budgets[i] == caps[i] && caps[i] < db.trajectory(i).size(); + std::cout << std::format("teacher: {}, curve sum {:.4f}, {} trajectories at the cap\n", + what, value, at_cap); + }; + if (want_budget_rdp) { + double value = 0; + const auto budgets = allocate(curves.rdp, total_budget, &value); + report("RDP allocation", value, budgets); + keep("BUDGET_RDP", make_candidates(objective, budgets, caps, &rdp_methods), rdp_methods); + } + if (needs_budget_candidates) { + double value = 0; + const auto budgets = allocate(curves.best, total_budget, &value); + report("allocation", value, budgets); + const auto choice = curve_methods(curves.best, budgets); + const auto candidates = + make_candidates(objective, budgets, caps, needs_loop_candidates ? nullptr : &choice); + if (want_budget_choice) + keep("BUDGET_CHOICE", candidates, choice); + // The loop and its no-update control start from the same state and share the setup + std::optional search; + if (want_loop || want_reallocate || want_no_update) + search.emplace(db, objective, candidates, choice); + if (want_loop || want_reallocate) { + const auto looped = search->run(m_seed); + timed("BUDGET_LOOP"); + if (want_loop) + keep("BUDGET_LOOP", candidates, looped); + if (want_reallocate) { + // Each round is kept only if J_T rises + auto current = assemble(candidates, looped); + auto current_methods = looped; + double current_value = objective.total(current); + for (std::size_t round = 1; round <= kMaxAllocationRounds; ++round) { + const auto round_curves = build_curves(db, objective, current, caps, false); + double round_value = 0; + const auto round_budgets = + allocate(round_curves.best, total_budget, &round_value); + const auto next_candidates = make_candidates(objective, round_budgets, caps); + auto next_methods = best_response( + db, objective, next_candidates, + curve_methods(round_curves.best, round_budgets), m_seed + round); + auto next = assemble(next_candidates, next_methods); + const double next_value = objective.total(next); + std::cout << std::format( + "teacher: reallocation round {}, J_T {:.4f} -> {:.4f}\n", round, + current_value, next_value); + if (next_value <= current_value + kMinImprovement) + break; + current_value = next_value; + current = std::move(next); + current_methods = std::move(next_methods); + } + std::cout << std::format("teacher BUDGET_LOOP_REALLOCATE: {}, J_T {:.4f}\n", + wins(current_methods), current_value); + outputs["BUDGET_LOOP_REALLOCATE"] = { + "BUDGET_LOOP_REALLOCATE", core::to_simplification(current), current_value}; + timed("BUDGET_LOOP_REALLOCATE"); + } + } + if (want_no_update) { + keep("BUDGET_LOOP_NO_UPDATE", candidates, search->run(m_seed, false)); + timed("BUDGET_LOOP_NO_UPDATE"); + } + if (want_shuffled) { + // Searched on fake partners, scored on the real ones + const auto shuffled = shuffled_partners(db, m_workloads, m_seed); + const Objective fake(db, shuffled, objective.config()); + keep("BUDGET_LOOP_SHUFFLED", candidates, + best_response(db, fake, candidates, choice, m_seed)); + timed("BUDGET_LOOP_SHUFFLED"); + } + } + return collect(outputs, arms); +} + +std::vector Teacher::single_recipes(std::size_t total_budget) const { + const auto budgets = allocation::create_trajectory_budgets( + m_db, allocation::AllocatorType::SizeProportional, total_budget, 0.0); + const auto candidates = make_candidates(m_objective, budgets, budgets); + std::vector arms; + for (std::size_t c = 0; c < kCandidates.size(); ++c) { + const auto name = std::format("RECIPE_{}", kCandidates[c]); + arms.push_back( + {name, core::to_simplification(candidates[c]), m_objective.total(candidates[c])}); + std::cout << std::format("teacher {}: J_T {:.4f}\n", name, arms.back().objective); + } + return arms; +} + +} // namespace teacher diff --git a/engine/src/teacher/workloads.cpp b/engine/src/teacher/workloads.cpp new file mode 100644 index 0000000..c5603cf --- /dev/null +++ b/engine/src/teacher/workloads.cpp @@ -0,0 +1,245 @@ +#include "backend/engine/teacher/workloads.hpp" + +#include "backend/engine/core/parallel.hpp" +#include "backend/engine/core/timer.hpp" +#include "backend/engine/query/join.hpp" +#include "backend/engine/query/knn.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace teacher { + +namespace { + +// Bump when the layout or workload generation changes. V4 fingerprints every point +constexpr std::uint64_t kMagicV4 = 0x5431'4341'4348'4534ull; +constexpr std::uint64_t kFnvOffsetBasis = 14695981039346656037ull; +constexpr std::uint64_t kFnvPrime = 1099511628211ull; + +// FNV-1a over IDs, sizes and every point, projected coordinates included +std::uint64_t fingerprint(const core::TrajectoryDb &db) { + std::uint64_t hash = kFnvOffsetBasis; + auto mix = [&](std::uint64_t value) { + for (unsigned int byte = 0; byte < 8; ++byte) { + hash ^= value & 0xff; + hash *= kFnvPrime; + value >>= 8; + } + }; + mix(db.size()); + mix(db.num_points()); + for (std::size_t i = 0; i < db.size(); ++i) { + const auto trajectory = db.trajectory(i); + mix(static_cast(trajectory.trajectory_id)); + mix(trajectory.size()); + for (std::size_t k = 0; k < trajectory.size(); ++k) { + mix(static_cast(trajectory.t[k])); + mix(std::bit_cast(trajectory.lat[k])); + mix(std::bit_cast(trajectory.lon[k])); + mix(std::bit_cast(trajectory.x[k])); + mix(std::bit_cast(trajectory.y[k])); + } + } + return hash; +} + +template void put(std::ofstream &out, const T &value) { + out.write(reinterpret_cast(&value), sizeof(T)); +} +template T get(std::ifstream &in) { + T value{}; + in.read(reinterpret_cast(&value), sizeof(T)); + return value; +} + +std::vector build_knn(const core::TrajectoryDb &db, const WorkloadConfig &config) { + std::vector result(db.size()); + core::parallel_for(db.size(), [&](std::size_t e) { + const auto query = db.trajectory(e); + auto neighbours = query::knn(db, query, config.k + 2, config.frechet); + std::erase(neighbours, e); + auto &entry = result[e]; + if (neighbours.size() < config.k + 1) + return; + entry.cutoff = + query::frechet_distance(query, db.trajectory(neighbours[config.k]), config.frechet); + entry.top.assign(neighbours.begin(), + neighbours.begin() + static_cast(config.k)); + entry.valid = std::isfinite(entry.cutoff); + if (!entry.valid) + entry.top.clear(); + }); + return result; +} + +void split_seen(std::vector &pairs, std::uint64_t seed) { + std::mt19937_64 rng(seed); + std::shuffle(pairs.begin(), pairs.end(), rng); + for (std::size_t k = 0; k < pairs.size(); ++k) + pairs[k].seen = k < pairs.size() / 2; + std::ranges::sort(pairs, [](const JoinPair &a, const JoinPair &b) { + return a.i < b.i || (a.i == b.i && a.j < b.j); + }); +} + +// Positives and negatives from one self-join +void collect_pairs(const core::TrajectoryDb &db, double eps_m, query::FrechetVariant frechet, + bool seen, std::vector &positives, std::vector &negatives) { + const auto join = query::generate_self_join(db, eps_m, frechet); + positives.clear(); + negatives.clear(); + for (const auto &entry : join.entries) + for (const auto j : entry.candidates) { + const JoinPair pair{static_cast(entry.query_index), + static_cast(j), seen}; + if (std::ranges::binary_search(entry.reference, j)) + positives.push_back(pair); + else + negatives.push_back(pair); + } +} + +// The pairs at join_eps_m, each list split into seen and unseen halves +void build_pairs(const core::TrajectoryDb &db, const WorkloadConfig &config, std::uint64_t seed, + std::vector &positives, std::vector &negatives) { + collect_pairs(db, config.join_eps_m, config.frechet, false, positives, negatives); + split_seen(positives, seed + kPairSplitSeedOffset); + split_seen(negatives, seed + kNegativeSeedOffset); +} + +void write_pairs(std::ofstream &out, const std::vector &pairs) { + put(out, pairs.size()); + for (const auto &pair : pairs) { + put(out, pair.i); + put(out, pair.j); + put(out, pair.seen); + } +} + +void read_pairs(std::ifstream &in, std::vector &pairs) { + pairs.resize(get(in)); + for (auto &pair : pairs) { + pair.i = get(in); + pair.j = get(in); + pair.seen = get(in) != 0; + } +} + +void save(const std::filesystem::path &file, std::uint64_t seed, std::uint64_t db_fingerprint, + const TeacherWorkloads &workloads) { + if (file.has_parent_path()) + std::filesystem::create_directories(file.parent_path()); + const auto tmp = file.string() + ".tmp"; + { + std::ofstream out(tmp, std::ios::binary); + put(out, kMagicV4); + put(out, seed); + put(out, db_fingerprint); + put(out, workloads.k); + put(out, workloads.join_eps_m); + put(out, static_cast(workloads.frechet)); + put(out, workloads.knn.size()); + for (const auto &entry : workloads.knn) { + put(out, entry.valid); + put(out, entry.cutoff); + for (std::size_t k = 0; k < workloads.k; ++k) + put(out, k < entry.top.size() ? entry.top[k] : 0); + } + write_pairs(out, workloads.positives); + write_pairs(out, workloads.negatives); + if (!out) + throw std::runtime_error("teacher cache: write failed " + tmp); + } + std::filesystem::rename(tmp, file); +} + +bool load(const std::filesystem::path &file, std::uint64_t seed, std::uint64_t db_fingerprint, + const WorkloadConfig &config, TeacherWorkloads &workloads) { + std::ifstream in(file, std::ios::binary); + if (!in) + return false; + if (get(in) != kMagicV4 || get(in) != seed || + get(in) != db_fingerprint) + return false; + const auto k = get(in); + const auto eps = get(in); + const auto variant = static_cast(get(in)); + if (k != config.k || eps != config.join_eps_m || variant != config.frechet) + return false; + workloads.knn.resize(get(in)); + for (auto &entry : workloads.knn) { + entry.valid = get(in) != 0; + entry.cutoff = get(in); + entry.top.resize(k); + for (auto &neighbour : entry.top) + neighbour = get(in); + if (!entry.valid) + entry.top.clear(); + } + read_pairs(in, workloads.positives); + read_pairs(in, workloads.negatives); + return static_cast(in); +} + +} // namespace + +TeacherWorkloads load_or_build_workloads(const core::TrajectoryDb &db, const WorkloadConfig &config, + std::uint64_t seed, + const std::filesystem::path &cache_dir) { + if (config.k == 0) + throw std::invalid_argument("teacher workloads: k must be positive"); + query::check_extra_thresholds(config.extra_join_eps_m, config.join_eps_m, + "teacher workloads: extra_join_eps_m"); + TeacherWorkloads workloads; + workloads.k = config.k; + workloads.join_eps_m = config.join_eps_m; + workloads.frechet = config.frechet; + const auto db_fingerprint = fingerprint(db); + const auto file = cache_dir / std::format("teacher_cache_seed{}.bin", seed); + + if (load(file, seed, db_fingerprint, config, workloads) && workloads.knn.size() == db.size()) { + std::cout << "teacher: cache loaded from " << file.string() << '\n'; + } else { + std::cout << "teacher: no matching cache at " << file.string() + << ", building it (database-as-query kNN takes a while)\n"; + auto start = std::chrono::steady_clock::now(); + workloads.knn = build_knn(db, config); + std::cout << std::format("teacher: kNN {:.1f} s\n", core::seconds_since(start)); + start = std::chrono::steady_clock::now(); + build_pairs(db, config, seed, workloads.positives, workloads.negatives); + std::cout << std::format("teacher: self-join pairs {:.1f} s\n", core::seconds_since(start)); + save(file, seed, db_fingerprint, workloads); + std::cout << "teacher: cache saved to " << file.string() << '\n'; + } + + workloads.range = query::generate_range_queries(db, config.range, seed + kRangeSeedOffset); + workloads.where_at = + query::generate_where_at_queries(db, config.where_at, seed + kWhereAtSeedOffset); + for (const double eps : config.extra_join_eps_m) { + ExtraJoin extra{.eps_m = eps, .positives = {}, .negatives = {}}; + collect_pairs(db, eps, config.frechet, true, extra.positives, extra.negatives); + std::cout << std::format("teacher: extra join at {} m, {} pairs, {} negatives\n", eps, + extra.positives.size(), extra.negatives.size()); + workloads.extra_joins.push_back(std::move(extra)); + } + + std::size_t invalid = 0; + for (const auto &entry : workloads.knn) + invalid += !entry.valid; + std::cout << std::format("teacher: {} kNN queries ({} invalid), {} pairs, {} negatives, {} " + "boxes, {} probes\n", + workloads.knn.size(), invalid, workloads.positives.size(), + workloads.negatives.size(), workloads.range.queries.size(), + workloads.where_at.queries.size()); + return workloads; +} + +} // namespace teacher diff --git a/engine/tests/combined_test.cpp b/engine/tests/combined_test.cpp index e522191..9aa877a 100644 --- a/engine/tests/combined_test.cpp +++ b/engine/tests/combined_test.cpp @@ -4,6 +4,7 @@ #include #include +#include namespace { @@ -48,6 +49,40 @@ TEST(NormalizeScores, FloorScoresZeroAndPerfectScoresOne) { EXPECT_DOUBLE_EQ(evaluator::normalize_scores(all_scores(1.0), floor).combined, 1.0); } +TEST(NormalizeScores, AveragesTheJoinOverItsThresholds) { + auto floor = all_scores(0.0); + auto scores = all_scores(0.0); + floor.join.f1 = 0.5; + scores.join.f1 = 1.0; + floor.join_grid.resize(2); + scores.join_grid.resize(2); + floor.join_grid[0].f1 = 0.0; + scores.join_grid[0].f1 = 0.5; + floor.join_grid[1].f1 = 0.2; + scores.join_grid[1].f1 = 0.2; + const auto normalized = evaluator::normalize_scores(scores, floor); + EXPECT_DOUBLE_EQ(normalized.join_f1, 1.0); // the main threshold alone + EXPECT_DOUBLE_EQ(normalized.join, (1.0 + 0.5 + 0.0) / 3.0); + EXPECT_DOUBLE_EQ(normalized.combined, normalized.join / 4.0); + floor.join_grid.pop_back(); + EXPECT_THROW(evaluator::normalize_scores(scores, floor), std::invalid_argument); +} + +TEST(NormalizeScores, AveragesClusteringOverItsThresholdsOutsideTheCombinedScore) { + auto floor = all_scores(0.0); + auto scores = all_scores(0.0); + floor.clustering.score = 0.0; + scores.clustering.score = 1.0; + floor.clustering_grid.resize(1); + scores.clustering_grid.resize(1); + floor.clustering_grid[0].score = 0.5; + scores.clustering_grid[0].score = 0.5; + const auto normalized = evaluator::normalize_scores(scores, floor); + EXPECT_DOUBLE_EQ(normalized.clustering_score, 1.0); + EXPECT_DOUBLE_EQ(normalized.clustering, 0.5); + EXPECT_DOUBLE_EQ(normalized.combined, 0.0); // clustering stays out of J4 +} + TEST(NormalizeScores, WeighsQueryTypesEquallyNotScores) { const auto floor = all_scores(0.0); auto scores = all_scores(0.0); @@ -100,6 +135,19 @@ TEST(NormalizeScores, DiagnosticsNeverChangePrimaryScores) { EXPECT_TRUE(std::isnan(evaluator::normalize_scores(scores, floor).combined)); } +TEST(NormalizeScores, SelfJoinIsNormalisedButStaysOutOfCombined) { + auto floor = all_scores(0.0); + auto scores = all_scores(0.5); + floor.self_join.f1 = 0.2; + scores.self_join.f1 = 0.6; + const auto normalized = evaluator::normalize_scores(scores, floor); + EXPECT_DOUBLE_EQ(normalized.self_join, 0.5); + EXPECT_DOUBLE_EQ(normalized.join, 0.5); + scores.self_join.f1 = 1.0; + EXPECT_DOUBLE_EQ(evaluator::normalize_scores(scores, floor).combined, normalized.combined); + EXPECT_TRUE(std::isnan(evaluator::normalize_scores(all_scores(0.5), floor).self_join)); +} + TEST(NormalizeScores, WhereAtUsesAverageHitAndItsOwnEndpointFloor) { auto floor = all_scores(0.0); auto scores = all_scores(0.0); diff --git a/engine/tests/config_test.cpp b/engine/tests/config_test.cpp index 9c43373..433273d 100644 --- a/engine/tests/config_test.cpp +++ b/engine/tests/config_test.cpp @@ -2,6 +2,8 @@ #include +#include + #include #include #include @@ -165,6 +167,11 @@ TEST(Config, SelfJoinModeIsExplicitAndDefaultsToHeldOut) { "data_path": "data.parquet", "join": {"mode": "Self", "eps_m": 500} })")); EXPECT_EQ(config.join.mode, query::JoinMode::Self); + EXPECT_FALSE(config.self_join); + EXPECT_TRUE(pipeline::load_config(write_config("self_join_diagnostic.json", R"({ + "data_path": "data.parquet", "join": {"mode": "HeldOut", "self_join": true} + })")) + .self_join); EXPECT_THROW(pipeline::load_config(write_config("bad_join_mode.json", R"({ "data_path": "data.parquet", "join": {"mode": "typo"} })")), @@ -210,13 +217,15 @@ TEST(Config, RejectsInvalidClustering) { TEST(Config, FrechetVariantAppliesToEveryFrechetQuery) { const auto config = pipeline::load_config( - write_config("frechet.json", R"({"data_path": "d", "frechet": "Continuous"})")); - EXPECT_EQ(config.knn.frechet, query::FrechetVariant::Continuous); - EXPECT_EQ(config.join.frechet, query::FrechetVariant::Continuous); - EXPECT_EQ(config.clustering.frechet, query::FrechetVariant::Continuous); + write_config("frechet.json", R"({"data_path": "d", "frechet": "Discrete"})")); + EXPECT_EQ(config.knn.frechet, query::FrechetVariant::Discrete); + EXPECT_EQ(config.join.frechet, query::FrechetVariant::Discrete); + EXPECT_EQ(config.clustering.frechet, query::FrechetVariant::Discrete); const auto defaults = pipeline::load_config(write_config("frechet_default.json", R"({"data_path": "d"})")); - EXPECT_EQ(defaults.join.frechet, query::FrechetVariant::Discrete); + EXPECT_EQ(defaults.knn.frechet, query::FrechetVariant::Continuous); + EXPECT_EQ(defaults.join.frechet, query::FrechetVariant::Continuous); + EXPECT_EQ(defaults.clustering.frechet, query::FrechetVariant::Continuous); EXPECT_THROW(pipeline::load_config( write_config("frechet_bad.json", R"({"data_path": "d", "frechet": "Weak"})")), std::invalid_argument); @@ -231,3 +240,74 @@ TEST(Config, RejectsInvalidWhereAtScoring) { std::string(R"({"data_path": "d", "where_at_scoring": )") + scoring + "}")), std::invalid_argument); } + +TEST(Config, TeacherIsOffByDefault) { + const auto config = + pipeline::load_config(write_config("teacher_default.json", R"({"data_path": "d"})")); + EXPECT_FALSE(config.teacher.enabled); + EXPECT_EQ(config.teacher.cache_dir, "cache"); + EXPECT_EQ(config.teacher.arms, (std::vector{"BUDGET_CHOICE", "BUDGET_LOOP"})); + EXPECT_TRUE(config.teacher.all_join_pairs); + EXPECT_TRUE(config.teacher.extra_join_eps_m.empty()); + EXPECT_FALSE(pipeline::teacher_objective_config(config).held_out_term); + EXPECT_TRUE(pipeline::teacher_objective_config(config).ranking_knn); + // The teacher's join term follows the join it is scored on + EXPECT_FALSE(config.teacher.join); + EXPECT_EQ(pipeline::teacher_objective_config(config).join_mode, query::JoinMode::HeldOut); + const auto self = pipeline::load_config( + write_config("teacher_self.json", R"({"data_path": "d", "join": {"mode": "Self"}})")); + EXPECT_EQ(pipeline::teacher_objective_config(self).join_mode, query::JoinMode::Self); +} + +TEST(Config, ReadsTheTeacherSection) { + const auto config = pipeline::load_config(write_config("teacher.json", R"({ + "data_path": "d", + "teacher": {"enabled": true, "cache_dir": "my_cache", "arms": ["EQUAL_CHOICE", "EQUAL_LOOP"], + "range_query_count": 1000, "all_join_pairs": false, + "join": "Self", "extra_join_eps_m": [100, 300], "held_out_term": true, + "ranking_knn": false, "term_weights": [2, 1, 1, 0]} + })")); + EXPECT_EQ(pipeline::teacher_objective_config(config).join_mode, query::JoinMode::Self); + EXPECT_TRUE(pipeline::teacher_objective_config(config).held_out_term); + EXPECT_FALSE(pipeline::teacher_objective_config(config).ranking_knn); + EXPECT_EQ(pipeline::teacher_objective_config(config).term_weights, + (std::array{2.0, 1.0, 1.0, 0.0})); + EXPECT_EQ(pipeline::teacher_workload_config(config).extra_join_eps_m, + (std::vector{100.0, 300.0})); + EXPECT_TRUE(config.teacher.enabled); + EXPECT_EQ(config.teacher.cache_dir, "my_cache"); + EXPECT_EQ(config.teacher.arms, (std::vector{"EQUAL_CHOICE", "EQUAL_LOOP"})); + EXPECT_EQ(config.teacher.range_query_count, 1000u); + EXPECT_FALSE(config.teacher.all_join_pairs); +} + +TEST(Config, RejectsBadTeacherSections) { + for (const auto §ion : + {R"({"enabled": true, "arms": ["DP"]})", R"({"enabled": true, "arms": []})", + R"({"arms": ["EQUAL_CHOICE_RDP_PARTNERS", "EQUAL_CHOICE_RDP_PARTNERS"]})", + R"({"cache_dir": ""})", R"({"range_query_count": 0})", R"({"shape_weight": 0.25})", + R"({"term_weights": [0, 0, 0, 0]})", R"({"term_weights": [-1, 1, 1, 1]})", + R"({"seed": 7})", R"({"join": "Both"})", R"({"extra_join_eps_m": [0]})", + R"({"extra_join_eps_m": [-100]})"}) { + SCOPED_TRACE(section); + const auto contents = std::string(R"({"data_path": "d", "teacher": )") + section + "}"; + EXPECT_THROW(pipeline::load_config(write_config("bad_teacher.json", contents)), + std::invalid_argument); + } +} + +TEST(Config, ReadsTheJoinScoreThresholds) { + const auto config = pipeline::load_config(write_config( + "join_grid.json", + R"({"data_path": "d", "join": {"eps_m": 150, "score_eps_m": [100, 200, 300]}})")); + EXPECT_EQ(config.join_score_eps_m, (std::vector{100.0, 200.0, 300.0})); + EXPECT_TRUE(pipeline::load_config(write_config("join_plain.json", R"({"data_path": "d"})")) + .join_score_eps_m.empty()); + for (const auto &grid : {"[150]", "[0]", "[100, 100]"}) { + SCOPED_TRACE(grid); + const auto contents = std::string(R"({"data_path": "d", "join": {"eps_m": 150, )") + + R"("score_eps_m": )" + grid + "}}"; + EXPECT_THROW(pipeline::load_config(write_config("bad_join_grid.json", contents)), + std::invalid_argument); + } +} diff --git a/engine/tests/evaluator_test.cpp b/engine/tests/evaluator_test.cpp index 5779ac4..58fbca1 100644 --- a/engine/tests/evaluator_test.cpp +++ b/engine/tests/evaluator_test.cpp @@ -152,6 +152,24 @@ TEST(Evaluator, ExposesRangeF1AndSelfJoinTogether) { EXPECT_TRUE(std::isnan(eval.get_join_scores().f1)); // one trajectory: no unordered pairs } +TEST(Evaluator, ScoresTheSelfJoinNextToAHeldOutJoin) { + const core::TrajectoryDb db({0, 2, 4}, {10, 20}, {0, 10, 0, 10}, std::vector(4, 39.95), + {116.35, 116.36, 116.35, 116.36}); + const auto held_out = query::generate_join_queries(db, db, {.query_count = 0}, 1); + const auto self = query::generate_self_join(db); + const auto scores = + evaluator::Evaluator(db, db, {.join = &held_out, .self_join = &self}).evaluate().scores; + EXPECT_EQ(scores.join.mode, query::JoinMode::HeldOut); + EXPECT_EQ(scores.join.query_count, 2u); + EXPECT_DOUBLE_EQ(scores.join.f1, 1); + EXPECT_EQ(scores.self_join.mode, query::JoinMode::Self); + EXPECT_EQ(scores.self_join.reference_pairs, 1u); + EXPECT_DOUBLE_EQ(scores.self_join.f1, 1); + // Without a self join the diagnostic stays undefined + EXPECT_TRUE(std::isnan( + evaluator::Evaluator(db, db, {.join = &held_out}).evaluate().scores.self_join.f1)); +} + TEST(Evaluator, SharedJoinMatchesIndependentClusteringAndFallsBackForDifferentWorkloads) { // Same endpoints, but the middle trajectory makes a detour which simplification removes. const core::TrajectoryDb db( diff --git a/engine/tests/join_test.cpp b/engine/tests/join_test.cpp index f09881d..3632b0b 100644 --- a/engine/tests/join_test.cpp +++ b/engine/tests/join_test.cpp @@ -110,7 +110,8 @@ TEST(Join, MatchesBruteForce) { if (frechet(queries.trajectory(q), db.trajectory(i)) <= eps) expected.push_back(i); } - EXPECT_EQ(query::join(db, queries.trajectory(q), eps), expected) + EXPECT_EQ(query::join(db, queries.trajectory(q), eps, query::FrechetVariant::Discrete), + expected) << "query " << q << ", eps " << eps; } } @@ -306,7 +307,7 @@ TEST(SelfJoin, MatchesIndependentFullGridOnAllOriginalAndCompressedPairs) { } const auto compressed = core::materialize(db, s); for (double eps : {100., 500., 1500.}) { - const auto workload = query::generate_self_join(db, eps); + const auto workload = query::generate_self_join(db, eps, query::FrechetVariant::Discrete); std::size_t truth = 0, predicted = 0, hits = 0; for (std::size_t i = 0; i < db.size(); ++i) { std::vector expected; diff --git a/engine/tests/knn_test.cpp b/engine/tests/knn_test.cpp index 9f9a856..b19b34c 100644 --- a/engine/tests/knn_test.cpp +++ b/engine/tests/knn_test.cpp @@ -71,7 +71,7 @@ double frechet(const core::TrajectoryView &a, const core::TrajectoryView &b) { return dp.back().back(); } -// Full Fréchet distance to every trajectory, no pruning +// Full discrete Fréchet distance to every trajectory, no pruning std::vector brute_force_knn(const core::TrajectoryDb &db, const core::TrajectoryView &query, std::size_t k) { std::vector> distances; @@ -109,7 +109,8 @@ TEST(Knn, KLargerThanDatabaseReturnsEverything) { std::mt19937_64 rng(3); const auto db = make_db(random_tracks(rng, 4)); const auto query = db.trajectory(1); - EXPECT_EQ(query::knn(db, query, 10), brute_force_knn(db, query, 10)); + EXPECT_EQ(query::knn(db, query, 10, query::FrechetVariant::Discrete), + brute_force_knn(db, query, 10)); } TEST(Knn, MatchesBruteForceOnRandomTrajectories) { @@ -119,7 +120,8 @@ TEST(Knn, MatchesBruteForceOnRandomTrajectories) { for (const std::size_t k : {1u, 3u, 5u, 10u}) { for (std::size_t q = 0; q < queries.size(); ++q) { const auto query = queries.trajectory(q); - EXPECT_EQ(query::knn(db, query, k), brute_force_knn(db, query, k)) + EXPECT_EQ(query::knn(db, query, k, query::FrechetVariant::Discrete), + brute_force_knn(db, query, k)) << "query " << q << ", k " << k; } } @@ -141,7 +143,7 @@ TEST(Knn, MatchesBruteForceForOverlappingLongTrajectories) { const auto queries = make_db({tracks.begin() + 20, tracks.end()}); for (const std::size_t k : {1u, 5u, 20u, 30u}) { for (std::size_t q = 0; q < queries.size(); ++q) { - EXPECT_EQ(query::knn(db, queries.trajectory(q), k), + EXPECT_EQ(query::knn(db, queries.trajectory(q), k, query::FrechetVariant::Discrete), brute_force_knn(db, queries.trajectory(q), k)) << "query " << q << ", k " << k; } @@ -154,7 +156,7 @@ TEST(Knn, MatchesBruteForceAcrossSeedCountBoundary) { const auto queries = make_db(random_tracks(rng, 3)); for (const std::size_t k : {1u, 63u, 64u, 65u, 90u, 100u}) { for (std::size_t q = 0; q < queries.size(); ++q) { - EXPECT_EQ(query::knn(db, queries.trajectory(q), k), + EXPECT_EQ(query::knn(db, queries.trajectory(q), k, query::FrechetVariant::Discrete), brute_force_knn(db, queries.trajectory(q), k)) << "query " << q << ", k " << k; } @@ -166,7 +168,8 @@ TEST(Knn, AcceptsQueryViewWithoutCachedBoundingBox) { const auto db = make_db(random_tracks(rng, 30)); auto query = db.trajectory(4); query.bbox = {}; - EXPECT_EQ(query::knn(db, query, 5), brute_force_knn(db, query, 5)); + EXPECT_EQ(query::knn(db, query, 5, query::FrechetVariant::Discrete), + brute_force_knn(db, query, 5)); } TEST(Knn, BreaksEqualDistanceTiesByDatabaseIndex) { diff --git a/engine/tests/route_clustering_test.cpp b/engine/tests/route_clustering_test.cpp index 0f998cd..e3e38d1 100644 --- a/engine/tests/route_clustering_test.cpp +++ b/engine/tests/route_clustering_test.cpp @@ -239,7 +239,7 @@ TEST(RouteClustering, RejectsMismatchedDatabasesAndInvalidInput) { std::invalid_argument); } -TEST(RouteClustering, CombinedIncludesClustering) { +TEST(RouteClustering, ClusteringIsNormalizedButNotCombined) { evaluator::Scoretable floor{}; floor.knn.recall_at_k = 0.0; floor.range_f1 = 0.0; @@ -251,7 +251,7 @@ TEST(RouteClustering, CombinedIncludesClustering) { scores.clustering.score = 0.75; const auto normalized = evaluator::normalize_scores(scores, floor); EXPECT_DOUBLE_EQ(normalized.clustering, 0.5); - EXPECT_DOUBLE_EQ(normalized.combined, 1.5 / 5.0); + EXPECT_DOUBLE_EQ(normalized.combined, 0.25); scores.clustering.score = std::numeric_limits::quiet_NaN(); EXPECT_DOUBLE_EQ(evaluator::normalize_scores(scores, floor).combined, 0.25); } diff --git a/engine/tests/selectors/rdp_test.cpp b/engine/tests/selectors/rdp_test.cpp index b89c87b..fccd347 100644 --- a/engine/tests/selectors/rdp_test.cpp +++ b/engine/tests/selectors/rdp_test.cpp @@ -1,5 +1,6 @@ #include +#include #include #include @@ -46,4 +47,20 @@ TEST(Rdp, AddsLargestErrorFirst) { TEST(Rdp, HandlesRoundTrip) { const Trajectory traj({0, 10, 10, 0, 0}, {0, 0, 10, 10, 0}); EXPECT_EQ(selection::rdp::simplify(traj.view(), 3), (Indices{0, 2, 4})); -} \ No newline at end of file +} +TEST(Rdp, SequencePrefixesMatchEachBudgetIncludingTies) { + for (const auto &ys : {std::vector{0, 5, 0, 5, 0, 5, 0}, std::vector(7, 0.0)}) { + const Trajectory traj({0, 1, 2, 3, 4, 5, 6}, ys); + for (std::size_t cap = 2; cap <= traj.x.size(); ++cap) { + const auto sequence = selection::rdp::sequence(traj.view(), cap); + ASSERT_EQ(sequence.size(), cap); + for (std::size_t b = 2; b <= cap; ++b) { + Indices kept(sequence.begin(), sequence.begin() + b); + std::ranges::sort(kept); + EXPECT_EQ(kept, selection::rdp::simplify(traj.view(), b)); + } + } + } + EXPECT_TRUE(selection::rdp::sequence(Trajectory({}, {}).view(), 5).empty()); + EXPECT_EQ(selection::rdp::sequence(Trajectory({0}, {0}).view(), 5), (Indices{0})); +} diff --git a/engine/tests/selectors/sed_test.cpp b/engine/tests/selectors/sed_test.cpp index 5f0d369..153cf90 100644 --- a/engine/tests/selectors/sed_test.cpp +++ b/engine/tests/selectors/sed_test.cpp @@ -107,3 +107,26 @@ TEST(Sed, ReturnsBudgetAndStaysNearBruteForce) { } } } + +TEST(Sed, ReverseRemovalPrefixesMatchEachBudgetIncludingTies) { + std::mt19937_64 rng(42); + for (int trial = 0; trial < 20; ++trial) { + std::vector x(30), y(30); + for (std::size_t i = 0; i < x.size(); ++i) { + x[i] = static_cast(i); + y[i] = trial == 0 ? 0.0 : static_cast(rng() % 10); + } + const Trajectory traj(x, y); + for (const std::size_t cap : {2u, 5u, 30u}) { + const auto sequence = selection::sed::sequence(traj.view(), cap); + ASSERT_EQ(sequence.size(), cap); + for (std::size_t b = 2; b <= cap; ++b) { + Indices kept(sequence.begin(), sequence.begin() + b); + std::ranges::sort(kept); + EXPECT_EQ(kept, selection::sed::simplify(traj.view(), b)); + } + } + } + EXPECT_TRUE(selection::sed::sequence(Trajectory({}, {}).view(), 5).empty()); + EXPECT_EQ(selection::sed::sequence(Trajectory({0}, {0}).view(), 5), (Indices{0})); +} diff --git a/engine/tests/teacher/curves_test.cpp b/engine/tests/teacher/curves_test.cpp new file mode 100644 index 0000000..ae3be8f --- /dev/null +++ b/engine/tests/teacher/curves_test.cpp @@ -0,0 +1,110 @@ +#include + +#include "backend/engine/teacher/curves.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +teacher::Curve curve(std::vector values) { + teacher::Curve c; + c.method.assign(values.size(), 0); + c.value = std::move(values); + return c; +} + +double value_of(const std::vector &curves, const std::vector &b) { + double sum = 0; + for (std::size_t i = 0; i < curves.size(); ++i) + sum += curves[i].at(b[i]); + return sum; +} + +// Best total over every way to split the budget, for small cases +double brute_force(const std::vector &curves, std::size_t budget) { + double best = -INFINITY; + std::vector b(curves.size(), 2); + auto recurse = [&](auto &&self, std::size_t i, std::size_t left) -> void { + if (i + 1 == curves.size()) { + if (left >= 2 && left <= curves[i].max_budget()) { + b[i] = left; + best = std::max(best, value_of(curves, b)); + } + return; + } + for (std::size_t x = 2; x <= curves[i].max_budget() && x <= left; ++x) { + b[i] = x; + self(self, i + 1, left - x); + } + }; + recurse(recurse, 0, budget); + return best; +} + +} // namespace + +TEST(TeacherAllocate, StepsOverFlatStretches) { + // A gains only at its second point, B a little at once: marginal greedy picks B, lookahead A + const std::vector curves{curve({0.0, 0.0, 1.0}), curve({0.0, 0.3, 0.35})}; + const auto b = teacher::allocate(curves, 6); + EXPECT_EQ(b, (std::vector{4, 2})); +} + +TEST(TeacherAllocate, KeepsTheExactBudgetAndBounds) { + std::mt19937_64 rng(3); + std::uniform_real_distribution step(0.0, 1.0); + std::uniform_int_distribution length(1, 8); + std::vector curves; + for (int i = 0; i < 50; ++i) { + std::vector v{0.0}; + for (std::size_t k = 1, n = length(rng); k < n; ++k) + v.push_back(v.back() + step(rng) - 0.3); // not monotone + curves.push_back(curve(v)); + } + std::size_t capacity = 0; + for (const auto &c : curves) + capacity += c.max_budget(); + for (const std::size_t budget : {std::size_t{100}, std::size_t{180}, capacity}) { + const auto b = teacher::allocate(curves, budget); + EXPECT_EQ(std::accumulate(b.begin(), b.end(), std::size_t{0}), budget); + for (std::size_t i = 0; i < curves.size(); ++i) { + EXPECT_GE(b[i], 2u); + EXPECT_LE(b[i], curves[i].max_budget()); + } + } +} + +TEST(TeacherAllocate, IsOptimalOnConcaveCurves) { + std::mt19937_64 rng(7); + std::uniform_real_distribution gain(0.0, 1.0); + for (int trial = 0; trial < 20; ++trial) { + std::vector curves; + for (int i = 0; i < 4; ++i) { + std::vector gains(5); + for (auto &g : gains) + g = gain(rng); + std::ranges::sort(gains, std::greater<>()); + std::vector v{0.0}; + for (const double g : gains) + v.push_back(v.back() + g); + curves.push_back(curve(v)); + } + const std::size_t budget = 8 + static_cast(trial % 12); + const auto b = teacher::allocate(curves, budget); + EXPECT_NEAR(value_of(curves, b), brute_force(curves, budget), 1e-9); + } +} + +TEST(TeacherAllocate, RejectsBudgetsThatDoNotFit) { + const std::vector curves{curve({0.0, 1.0}), curve({0.0})}; + EXPECT_THROW(teacher::allocate(curves, 3), std::invalid_argument); // below 2 per trajectory + EXPECT_THROW(teacher::allocate(curves, 6), std::runtime_error); // more than 3 + 2 fits + EXPECT_EQ(teacher::allocate(curves, 5), (std::vector{3, 2})); +} diff --git a/engine/tests/teacher/teacher_test.cpp b/engine/tests/teacher/teacher_test.cpp new file mode 100644 index 0000000..5025141 --- /dev/null +++ b/engine/tests/teacher/teacher_test.cpp @@ -0,0 +1,696 @@ +#include + +#include "backend/engine/core/projection.hpp" +#include "backend/engine/core/trajectory_db.hpp" +#include "backend/engine/teacher/greedy.hpp" +#include "backend/engine/teacher/objective.hpp" +#include "backend/engine/teacher/teacher.hpp" +#include "backend/engine/teacher/workloads.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +// Nearby random walks in local metres, one point per second, so that some pairs join +core::TrajectoryDb random_db(std::uint64_t seed, std::size_t count, std::size_t length) { + std::mt19937_64 rng(seed); + std::uniform_real_distribution start(-200.0, 200.0); + std::normal_distribution step(40.0, 30.0); + std::vector offsets{0}; + std::vector ids; + std::vector t; + std::vector lat, lon; + for (std::size_t i = 0; i < count; ++i) { + ids.push_back(static_cast(i)); + double x = start(rng), y = start(rng); + for (std::size_t k = 0; k < length; ++k) { + const auto ll = core::to_lat_lon(x, y); + t.push_back(static_cast(k)); + lat.push_back(ll.lat); + lon.push_back(ll.lon); + x += step(rng); + y += 0.5 * step(rng); + } + offsets.push_back(t.size()); + } + return {std::move(offsets), std::move(ids), std::move(t), std::move(lat), std::move(lon)}; +} + +teacher::WorkloadConfig small_config() { + teacher::WorkloadConfig config; + config.k = 3; + config.join_eps_m = 150.0; + config.range = {.query_count = 60, + .box_size_m = 100.0, + .window_seconds = 604800, + .min_results = 1, + .full_accept_at = 1, + .attempts_per_query = 1000}; + config.where_at = {.probes_per_trajectory = 8, .max_gap_seconds = 60.0}; + return config; +} + +class TeacherTest : public ::testing::Test { + protected: + void SetUp() override { + m_dir = std::filesystem::temp_directory_path() / + ("teacher_test_" + std::to_string(std::random_device{}())); + std::filesystem::create_directories(m_dir); + } + void TearDown() override { std::filesystem::remove_all(m_dir); } + std::filesystem::path m_dir; +}; + +teacher::KeptSets endpoints(const core::TrajectoryDb &db) { + teacher::KeptSets kept(db.size()); + for (std::size_t i = 0; i < db.size(); ++i) + kept[i] = {0, db.trajectory(i).size() - 1}; + return kept; +} + +teacher::KeptSets every_point(const core::TrajectoryDb &db) { + teacher::KeptSets kept(db.size()); + for (std::size_t i = 0; i < db.size(); ++i) { + kept[i].resize(db.trajectory(i).size()); + for (std::size_t k = 0; k < kept[i].size(); ++k) + kept[i][k] = k; + } + return kept; +} + +std::string read_cache(const std::filesystem::path &file) { + std::ifstream in(file, std::ios::binary); + return {std::istreambuf_iterator(in), std::istreambuf_iterator()}; +} + +void expect_same_cached_workloads(const teacher::TeacherWorkloads &a, + const teacher::TeacherWorkloads &b) { + ASSERT_EQ(a.knn.size(), b.knn.size()); + for (std::size_t i = 0; i < a.knn.size(); ++i) { + EXPECT_EQ(a.knn[i].valid, b.knn[i].valid); + EXPECT_EQ(a.knn[i].top, b.knn[i].top); + EXPECT_DOUBLE_EQ(a.knn[i].cutoff, b.knn[i].cutoff); + } + auto same_pairs = [](const auto &x, const auto &y) { + return std::ranges::equal(x, y, [](const auto &p, const auto &q) { + return p.i == q.i && p.j == q.j && p.seen == q.seen; + }); + }; + EXPECT_TRUE(same_pairs(a.positives, b.positives)); + EXPECT_TRUE(same_pairs(a.negatives, b.negatives)); +} + +} // namespace + +TEST_F(TeacherTest, CacheIsWrittenThenReadBack) { + const auto db = random_db(1, 24, 20); + const auto built = teacher::load_or_build_workloads(db, small_config(), 5, m_dir); + const auto file = m_dir / "teacher_cache_seed5.bin"; + ASSERT_TRUE(std::filesystem::exists(file)); + // A hit must reuse the file, not recompute + const auto marker = std::filesystem::file_time_type::clock::now() - std::chrono::hours(24); + std::filesystem::last_write_time(file, marker); + const auto before = std::filesystem::last_write_time(file); + const auto loaded = teacher::load_or_build_workloads(db, small_config(), 5, m_dir); + + EXPECT_EQ(std::filesystem::last_write_time(file), before); + expect_same_cached_workloads(built, loaded); + EXPECT_FALSE(built.positives.empty()); +} + +TEST_F(TeacherTest, CacheForChangedInteriorPointsIsRebuilt) { + const auto source = random_db(1, 24, 20); + auto make_db = [&](std::string_view change) { + std::vector offsets{0}; + std::vector ids; + std::vector t; + std::vector lat, lon; + for (std::size_t i = 0; i < source.size(); ++i) { + const auto v = source.trajectory(i); + ids.push_back(v.trajectory_id); + for (std::size_t k = 0; k < v.size(); ++k) { + t.push_back(2 * v.t[k]); + lat.push_back(v.lat[k]); + lon.push_back(v.lon[k]); + if (i == 0 && k == v.size() / 2) { + if (change == "latitude") + lat.back() += 0.01; + if (change == "longitude") + lon.back() += 0.01; + if (change == "timestamp") + ++t.back(); + } + } + offsets.push_back(t.size()); + } + return core::TrajectoryDb(std::move(offsets), std::move(ids), std::move(t), std::move(lat), + std::move(lon)); + }; + const auto original = make_db(""); + for (const auto change : {"latitude", "longitude", "timestamp"}) { + SCOPED_TRACE(change); + const auto dir = m_dir / change; + teacher::load_or_build_workloads(original, small_config(), 5, dir); + const auto file = dir / "teacher_cache_seed5.bin"; + const auto before = read_cache(file); + ASSERT_FALSE(before.empty()); + + // IDs, lengths, endpoints and endpoint timestamps all stay unchanged. + const auto changed = make_db(change); + const auto rebuilt = teacher::load_or_build_workloads(changed, small_config(), 5, dir); + EXPECT_NE(read_cache(file), before); + const auto fresh = + teacher::load_or_build_workloads(changed, small_config(), 5, dir / "fresh"); + expect_same_cached_workloads(rebuilt, fresh); + } +} + +TEST_F(TeacherTest, CacheWithLegacyFingerprintIsRebuilt) { + const auto db = random_db(1, 24, 20); + const auto built = teacher::load_or_build_workloads(db, small_config(), 5, m_dir); + const auto file = m_dir / "teacher_cache_seed5.bin"; + const auto current = read_cache(file); + ASSERT_FALSE(current.empty()); + for (const std::uint64_t legacy : {0x5431'4341'4348'4532ull, 0x5431'4341'4348'4533ull}) { + SCOPED_TRACE(legacy); + { + std::fstream out(file, std::ios::binary | std::ios::in | std::ios::out); + out.write(reinterpret_cast(&legacy), sizeof(legacy)); + ASSERT_TRUE(out); + } + // An obsolete version is rejected even when everything else matches + ASSERT_NE(read_cache(file), current); + const auto rebuilt = teacher::load_or_build_workloads(db, small_config(), 5, m_dir); + EXPECT_EQ(read_cache(file), current); + expect_same_cached_workloads(built, rebuilt); + } +} + +TEST_F(TeacherTest, CacheForOtherSettingsIsRebuilt) { + const auto db = random_db(1, 24, 20); + teacher::load_or_build_workloads(db, small_config(), 5, m_dir); + auto other = small_config(); + other.k = 2; + const auto rebuilt = teacher::load_or_build_workloads(db, other, 5, m_dir); + EXPECT_EQ(rebuilt.k, 2u); + for (const auto &q : rebuilt.knn) { + if (q.valid) { + EXPECT_EQ(q.top.size(), 2u); + } + } +} + +TEST_F(TeacherTest, ObjectiveIsZeroAtTheFloorAndTracksJoinExactly) { + const auto db = random_db(2, 24, 20); + const auto workloads = teacher::load_or_build_workloads(db, small_config(), 9, m_dir); + for (const auto mode : {query::JoinMode::Self, query::JoinMode::HeldOut}) { + SCOPED_TRACE(mode == query::JoinMode::Self ? "Self" : "HeldOut"); + const teacher::Objective objective(db, workloads, {.join_mode = mode}); + + const auto floor = endpoints(db); + EXPECT_NEAR(objective.total(floor), 0.0, 1e-12); + EXPECT_GT(objective.total(every_point(db)), 0.0); + + // Switch random trajectories between endpoints and every point; the tracked join metric + // must equal a full recompute after each switch + auto state = floor; + const teacher::PreparedTrajectories prepared_floor(db, state); + auto views = prepared_floor.views(); + teacher::JoinState join(objective, views); + const auto full = every_point(db); + const teacher::PreparedTrajectories prepared_full(db, full); + const auto full_views = prepared_full.views(); + const auto floor_views = prepared_floor.views(); + std::mt19937_64 rng(4); + for (int step = 0; step < 40; ++step) { + const std::size_t i = rng() % db.size(); + const auto &next = state[i].size() == 2 ? full[i] : floor[i]; + const double before = objective.total(state); + const auto next_view = state[i].size() == 2 ? full_views[i] : floor_views[i]; + const double old_metric = join.metric(); + const auto proposal = join.propose(i, next_view, views); + // Proposing must not change the state or this proposal + const auto stay = join.propose(i, views[i], views); + EXPECT_DOUBLE_EQ(stay.gain, 0.0); + EXPECT_DOUBLE_EQ(join.metric(), old_metric); + const double predicted = proposal.gain; + const double separable = + objective.separable(i, next) - objective.separable(i, state[i]); + state[i] = next; + views[i] = next_view; + join.apply(proposal); + EXPECT_NEAR(join.metric(), objective.metrics(state)[teacher::Objective::Join], 1e-12); + EXPECT_NEAR(objective.total(state) - before, predicted + separable, 1e-9); + } + } +} + +TEST_F(TeacherTest, SeveralJoinTermsAreTrackedExactly) { + const auto db = random_db(2, 24, 20); + auto config = small_config(); + config.extra_join_eps_m = {config.join_eps_m * 0.5, config.join_eps_m * 2.0}; + const auto workloads = teacher::load_or_build_workloads(db, config, 9, m_dir); + ASSERT_EQ(workloads.extra_joins.size(), 2u); + EXPECT_LE(workloads.extra_joins[0].positives.size(), workloads.positives.size()); + EXPECT_GE(workloads.extra_joins[1].positives.size(), workloads.positives.size()); + + const teacher::Objective single(db, workloads, {.join_mode = query::JoinMode::Self}); + for (const bool held_out_term : {false, true}) { + SCOPED_TRACE(held_out_term ? "with held-out term" : "self terms"); + const teacher::Objective objective( + db, workloads, {.join_mode = query::JoinMode::Self, .held_out_term = held_out_term}); + EXPECT_EQ(objective.slots(), db.size() * (held_out_term ? 4u : 3u)); + EXPECT_NEAR(objective.total(endpoints(db)), 0.0, 1e-12); + EXPECT_DOUBLE_EQ(objective.metrics(every_point(db))[teacher::Objective::Join], 1.0); + + auto state = endpoints(db); + const auto floor = state, full = every_point(db); + const teacher::PreparedTrajectories prepared_floor(db, floor), prepared_full(db, full); + const auto floor_views = prepared_floor.views(), full_views = prepared_full.views(); + auto views = floor_views; + teacher::JoinState join(objective, views); + std::mt19937_64 rng(4); + for (int step = 0; step < 40; ++step) { + const std::size_t i = rng() % db.size(); + const bool grow = state[i].size() == 2; + const auto &next = grow ? full[i] : floor[i]; + const auto next_view = grow ? full_views[i] : floor_views[i]; + const double before = objective.total(state); + const auto proposal = join.propose(i, next_view, views); + const double separable = + objective.separable(i, next) - objective.separable(i, state[i]); + state[i] = next; + views[i] = next_view; + join.apply(proposal); + EXPECT_NEAR(join.metric(), objective.metrics(state)[teacher::Objective::Join], 1e-12); + EXPECT_NEAR(objective.total(state) - before, proposal.gain + separable, 1e-9); + } + } + // Without extra terms the workloads' extra joins are the only difference + teacher::TeacherWorkloads plain = workloads; + plain.extra_joins.clear(); + const teacher::Objective base(db, plain, {.join_mode = query::JoinMode::Self}); + EXPECT_EQ(base.slots(), db.size()); + EXPECT_GT(single.pair_count(), base.pair_count()); +} + +TEST_F(TeacherTest, RankingKnnIsTrackedExactly) { + const auto db = random_db(2, 24, 20); + auto config = small_config(); + // Discrete distances order exactly, without the continuous ranking's 1 mm ties + config.frechet = query::FrechetVariant::Discrete; + const auto workloads = teacher::load_or_build_workloads(db, config, 9, m_dir); + const teacher::Objective objective(db, workloads, + {.join_mode = query::JoinMode::Self, .ranking_knn = true}); + const auto floor = endpoints(db), full = every_point(db); + EXPECT_NEAR(objective.total(floor), 0.0, 1e-12); + EXPECT_DOUBLE_EQ(objective.metrics(full)[teacher::Objective::Knn], 1.0); + + // Every query watches every trajectory, so the tracked value must equal a full recompute + auto state = floor; + const std::vector watch{full}; + teacher::KnnState knn(objective, state, watch, db.size()); + EXPECT_NEAR(knn.metric(), objective.metrics(state)[teacher::Objective::Knn], 1e-12); + const teacher::PreparedTrajectories prepared_floor(db, floor), prepared_full(db, full); + const auto floor_views = prepared_floor.views(), full_views = prepared_full.views(); + auto views = floor_views; + teacher::JoinState join(objective, views); + std::mt19937_64 rng(5); + for (int step = 0; step < 40; ++step) { + const std::size_t i = rng() % db.size(); + const bool grow = state[i].size() == 2; + const auto &next = grow ? full[i] : floor[i]; + const auto next_view = grow ? full_views[i] : floor_views[i]; + const double before = objective.total(state); + const auto distances = knn.distances(i, next_view); + const double knn_gain = knn.propose(i, distances); + EXPECT_DOUBLE_EQ(knn.propose(i, knn.distances(i, views[i])), 0.0); + const auto proposal = join.propose(i, next_view, views); + const double separable = objective.separable(i, next) - objective.separable(i, state[i]); + state[i] = next; + views[i] = next_view; + join.apply(proposal); + knn.apply(i, distances); + EXPECT_NEAR(knn.metric(), objective.metrics(state)[teacher::Objective::Knn], 1e-12); + EXPECT_NEAR(objective.total(state) - before, knn_gain + proposal.gain + separable, 1e-9); + } +} + +TEST_F(TeacherTest, RankingKnnArmsKeepTheBudget) { + const auto db = random_db(3, 30, 25); + auto workloads = teacher::load_or_build_workloads(db, small_config(), 11, m_dir); + const teacher::Teacher model(db, std::move(workloads), + {.join_mode = query::JoinMode::HeldOut, .ranking_knn = true}, 11); + const std::size_t budget = 2 * db.size() + 40; + const std::vector arms{"EQUAL_CHOICE", "EQUAL_LOOP", "BUDGET_RDP", "BUDGET_LOOP"}; + for (const auto &arm : model.run(budget, arms)) { + SCOPED_TRACE(arm.name); + EXPECT_EQ(arm.simplification.num_points(), budget); + for (std::size_t i = 0; i < db.size(); ++i) { + const auto kept = arm.simplification.trajectory(i); + EXPECT_EQ(kept.front(), 0u); + EXPECT_EQ(kept.back(), db.trajectory(i).size() - 1); + } + } +} + +TEST_F(TeacherTest, TermWeightsScaleTheTerms) { + const auto db = random_db(2, 24, 20); + const auto workloads = teacher::load_or_build_workloads(db, small_config(), 9, m_dir); + const teacher::Objective equal(db, workloads); + const teacher::Objective weighted(db, workloads, {.term_weights = {2.0, 1.0, 1.0, 0.0}}); + using T = teacher::Objective; + // Weights are relative: kNN gets 2/4 instead of 1/4, join drops out + EXPECT_NEAR(weighted.scale(T::Knn), 2.0 * equal.scale(T::Knn), 1e-12); + EXPECT_NEAR(weighted.scale(T::Range), equal.scale(T::Range), 1e-12); + EXPECT_DOUBLE_EQ(weighted.scale(T::Join), 0.0); + // Every point kept is perfect on every term, so J_T is the weight share of the terms used + EXPECT_NEAR(weighted.total(every_point(db)), (2.0 + 1.0 + 1.0) / 4.0, 1e-9); + EXPECT_THROW(teacher::Objective(db, workloads, {.term_weights = {0.0, 0.0, 0.0, 0.0}}), + std::invalid_argument); + EXPECT_THROW(teacher::Objective(db, workloads, {.term_weights = {-1.0, 1.0, 1.0, 1.0}}), + std::invalid_argument); +} + +TEST_F(TeacherTest, SeenPairsOnlyRejectsExtraThresholds) { + const auto db = random_db(2, 24, 20); + auto config = small_config(); + config.extra_join_eps_m = {config.join_eps_m * 2.0}; + const auto workloads = teacher::load_or_build_workloads(db, config, 9, m_dir); + EXPECT_THROW(teacher::Objective(db, workloads, {.all_join_pairs = false}), + std::invalid_argument); + EXPECT_NO_THROW(teacher::Objective(db, workloads, {.all_join_pairs = true})); +} + +TEST_F(TeacherTest, ExtraJoinThresholdsMustBeDistinctAndPositive) { + const auto db = random_db(2, 24, 20); + for (const auto &extra : + {std::vector{0.0}, std::vector{-5.0}, + std::vector{small_config().join_eps_m}, std::vector{40.0, 40.0}}) { + auto config = small_config(); + config.extra_join_eps_m = extra; + EXPECT_THROW(teacher::load_or_build_workloads(db, config, 9, m_dir), std::invalid_argument); + } +} + +TEST_F(TeacherTest, HeldOutJoinScoresOriginalQueriesAgainstCompressedPartners) { + const auto db = random_db(2, 24, 20); + const auto workloads = teacher::load_or_build_workloads(db, small_config(), 9, m_dir); + const teacher::Objective objective(db, workloads, {.join_mode = query::JoinMode::HeldOut}); + // Every point kept is the original database: each query finds exactly its positives + EXPECT_DOUBLE_EQ(objective.metrics(every_point(db))[teacher::Objective::Join], 1.0); + + std::mt19937_64 rng(21); + auto kept = endpoints(db); + for (std::size_t i = 0; i < db.size(); ++i) { + for (std::size_t k = 1; k + 1 < db.trajectory(i).size(); ++k) + if (rng() % 4 == 0) + kept[i].push_back(k); + std::ranges::sort(kept[i]); + } + const teacher::PreparedTrajectories prepared(db, kept); + const auto views = prepared.views(); + double sum = 0; + std::size_t counted = 0; + for (std::size_t e = 0; e < db.size(); ++e) { + std::size_t hits = 0, found = 0; + for (const auto &link : objective.links(e)) { + const bool matched = objective.match(db.trajectory(e), views[link.other]); + found += matched; + hits += matched && link.positive; + } + if (found + objective.truth(e) == 0) + continue; + sum += 2.0 * static_cast(hits) / static_cast(found + objective.truth(e)); + ++counted; + } + ASSERT_GT(counted, 0u); + EXPECT_NEAR(objective.metrics(kept)[teacher::Objective::Join], + sum / static_cast(counted), 1e-12); +} + +TEST_F(TeacherTest, PreparedIndependentScoresMatchFullScoringAndSurvivePruning) { + const auto db = random_db(2, 24, 20); + const auto workloads = teacher::load_or_build_workloads(db, small_config(), 9, m_dir); + std::mt19937_64 rng(74); + for (const bool held_out : {false, true}) { + { + const teacher::Objective objective( + db, workloads, + {.join_mode = held_out ? query::JoinMode::HeldOut : query::JoinMode::Self}); + std::size_t with_truth = 0; + for (std::size_t i = 0; i < db.size(); ++i) + with_truth += objective.truth(i) != 0; + for (int trial = 0; trial < 4; ++trial) { + auto kept = endpoints(db); + if (trial == 1) + kept = every_point(db); + else if (trial > 1) + for (std::size_t i = 0; i < db.size(); ++i) { + for (std::size_t k = 1; k + 1 < db.trajectory(i).size(); ++k) + if (rng() % 4 == 0) + kept[i].push_back(k); + std::ranges::sort(kept[i]); + } + const teacher::PreparedTrajectories prepared(db, kept); + const auto views = prepared.views(); + for (const bool original_partners : {false, true}) { + const std::span partners = + original_partners ? std::span{} : views; + for (std::size_t i = 0; i < db.size(); ++i) { + std::size_t hits = 0, found = 0; + for (const auto &link : objective.links(i)) { + const auto other = + original_partners ? db.trajectory(link.other) : views[link.other]; + // Held out, i is the database side of an original partner's query + const bool matched = + held_out ? objective.match(db.trajectory(link.other), views[i]) + : objective.match(views[i], other); + found += matched; + hits += matched && link.positive; + } + const double f1 = found + objective.truth(i) + ? 2.0 * static_cast(hits) / + static_cast(found + objective.truth(i)) + : 0.0; + const double join = with_truth ? objective.scale(teacher::Objective::Join) * + f1 / static_cast(with_truth) + : 0.0; + const double expected = objective.separable(i, views[i]) + join; + const auto score = objective.prepare_independent(i, views[i], partners); + EXPECT_GE(score.bound + 1e-12, expected); + EXPECT_DOUBLE_EQ(objective.independent_value(score, expected - 1e-6), + expected); + EXPECT_LE(objective.independent_value( + score, std::numeric_limits::infinity()), + expected + 1e-12); + EXPECT_DOUBLE_EQ(objective.independent_value(score), expected); + EXPECT_DOUBLE_EQ(objective.independent_value(i, views[i], partners), + expected); + } + } + } + } + } +} + +TEST_F(TeacherTest, ArmsKeepTheBudgetAndBestResponseOnlyImproves) { + const auto db = random_db(3, 30, 25); + for (const auto mode : {query::JoinMode::Self, query::JoinMode::HeldOut}) { + SCOPED_TRACE(mode == query::JoinMode::Self ? "Self" : "HeldOut"); + auto workloads = teacher::load_or_build_workloads(db, small_config(), 11, m_dir); + const teacher::Teacher model(db, std::move(workloads), {.join_mode = mode}, 11); + const std::vector arms(teacher::kArms.begin(), teacher::kArms.end()); + const std::size_t budget = 2 * db.size() + 40; + const auto result = model.run(budget, arms); + + ASSERT_EQ(result.size(), arms.size()); + double equal_choice_jt = 0, rdp_partners_jt = 0, equal_loop_jt = 0, budget_choice_jt = 0, + budget_loop_jt = 0, reallocate_jt = 0; + for (const auto &arm : result) { + SCOPED_TRACE(arm.name); + ASSERT_EQ(arm.simplification.size(), db.size()); + EXPECT_EQ(arm.simplification.num_points(), budget); + for (std::size_t i = 0; i < db.size(); ++i) { + const auto kept = arm.simplification.trajectory(i); + ASSERT_GE(kept.size(), 2u); + EXPECT_EQ(kept.front(), 0u); + EXPECT_EQ(kept.back(), db.trajectory(i).size() - 1); + EXPECT_TRUE(std::ranges::is_sorted(kept)); + EXPECT_EQ(std::ranges::adjacent_find(kept), kept.end()); + } + if (arm.name == "EQUAL_CHOICE") + equal_choice_jt = arm.objective; + if (arm.name == "EQUAL_CHOICE_RDP_PARTNERS") + rdp_partners_jt = arm.objective; + if (arm.name == "EQUAL_LOOP") + equal_loop_jt = arm.objective; + if (arm.name == "BUDGET_CHOICE") + budget_choice_jt = arm.objective; + if (arm.name == "BUDGET_LOOP") + budget_loop_jt = arm.objective; + if (arm.name == "BUDGET_LOOP_REALLOCATE") + reallocate_jt = arm.objective; + } + EXPECT_GE(equal_loop_jt, rdp_partners_jt - 1e-12); + EXPECT_GE(budget_loop_jt, budget_choice_jt - 1e-12); + EXPECT_GE(reallocate_jt, budget_loop_jt - 1e-12); + // Held out, the partners are queries and stay original, so the two choices are the same + if (mode == query::JoinMode::HeldOut) { + EXPECT_DOUBLE_EQ(rdp_partners_jt, equal_choice_jt); + } + } +} + +TEST_F(TeacherTest, GreedyKeepsTheBudgetAndTheSeed) { + const auto db = random_db(4, 20, 30); + const auto workloads = teacher::load_or_build_workloads(db, small_config(), 13, m_dir); + const teacher::Objective objective(db, workloads); + for (std::size_t i = 0; i < db.size(); ++i) { + const teacher::Greedy greedy(objective, i); + const std::size_t n = db.trajectory(i).size(); + const std::vector seed{n / 2}; + std::vector fill(n); + for (std::size_t k = 0; k < n; ++k) + fill[k] = n - 1 - k; + for (const std::size_t b : {2u, 3u, 6u, 12u, 40u}) { + SCOPED_TRACE(std::to_string(i) + " at " + std::to_string(b)); + const auto kept = greedy.run( + b, b > 2 ? std::span(seed) : std::span(), + fill); + ASSERT_EQ(kept.size(), std::min(b, n)); + EXPECT_EQ(kept.front(), 0u); + EXPECT_EQ(kept.back(), n - 1); + EXPECT_TRUE(std::ranges::is_sorted(kept)); + EXPECT_EQ(std::ranges::adjacent_find(kept), kept.end()); + if (b > 2 && b < n) { + EXPECT_TRUE(std::ranges::binary_search(kept, n / 2)); + } + } + } +} + +TEST_F(TeacherTest, ExtremeBudgetsKeepEndpointsOrEveryPointForEveryArm) { + const auto db = random_db(3, 30, 25); + auto workloads = teacher::load_or_build_workloads(db, small_config(), 11, m_dir); + const teacher::Teacher model(db, std::move(workloads), {}, 11); + const std::vector arms(teacher::kArms.begin(), teacher::kArms.end()); + for (const std::size_t budget : {2 * db.size(), db.num_points()}) { + SCOPED_TRACE(budget); + const auto result = model.run(budget, arms); + ASSERT_EQ(result.size(), arms.size()); + for (const auto &arm : result) { + SCOPED_TRACE(arm.name); + EXPECT_EQ(arm.simplification.num_points(), budget); + for (std::size_t i = 0; i < db.size(); ++i) { + const auto n = db.trajectory(i).size(); + std::vector expected{0}; + if (budget == db.num_points()) + for (std::size_t k = 1; k + 1 < n; ++k) + expected.push_back(k); + expected.push_back(n - 1); + EXPECT_TRUE(std::ranges::equal(arm.simplification.trajectory(i), expected)); + } + } + } +} + +TEST_F(TeacherTest, SingleRecipesKeepTheBudget) { + const auto db = random_db(3, 30, 25); + auto workloads = teacher::load_or_build_workloads(db, small_config(), 11, m_dir); + const teacher::Teacher model(db, std::move(workloads), {}, 11); + const std::size_t budget = 2 * db.size() + 40; + const auto recipes = model.single_recipes(budget); + ASSERT_EQ(recipes.size(), 6u); + EXPECT_EQ(recipes.front().name, "RECIPE_RDP"); + for (const auto &arm : recipes) { + SCOPED_TRACE(arm.name); + EXPECT_EQ(arm.simplification.num_points(), budget); + } +} + +TEST(Teacher, RejectsUnknownArms) { + EXPECT_TRUE(teacher::is_arm("EQUAL_CHOICE_RDP_PARTNERS")); + EXPECT_FALSE(teacher::is_arm("gate_a0")); + EXPECT_TRUE(teacher::is_arm("BUDGET_CHOICE")); + EXPECT_FALSE(teacher::is_arm("TEACHER_A0")); +} + +TEST_F(TeacherTest, IndividualArmsMatchCombinedRun) { + const auto db = random_db(3, 30, 25); + for (const auto [variant, ranking] : {std::pair{query::FrechetVariant::Discrete, false}, + std::pair{query::FrechetVariant::Continuous, false}, + std::pair{query::FrechetVariant::Continuous, true}}) { + SCOPED_TRACE(ranking ? "ranking kNN" : "per-trajectory kNN"); + auto config = small_config(); + config.frechet = variant; + auto workloads = teacher::load_or_build_workloads(db, config, 11, m_dir); + const teacher::Teacher model(db, std::move(workloads), {.ranking_knn = ranking}, 11); + const std::vector arms(teacher::kArms.begin(), teacher::kArms.end()); + const std::size_t budget = 2 * db.size() + 40; + const auto combined = model.run(budget, arms); + for (std::size_t a = 0; a < arms.size(); ++a) { + SCOPED_TRACE(arms[a]); + const std::vector requested{arms[a]}; + const auto alone = model.run(budget, requested); + ASSERT_EQ(alone.size(), 1u); + EXPECT_DOUBLE_EQ(alone[0].objective, combined[a].objective); + for (std::size_t i = 0; i < db.size(); ++i) + EXPECT_TRUE(std::ranges::equal(alone[0].simplification.trajectory(i), + combined[a].simplification.trajectory(i))); + } + } +} + +TEST(PreparedTrajectory, OwnsCoordinatesAndRebuildsBoundsAfterCopyMoveAndAssign) { + const auto db = random_db(4, 2, 30); + const auto original = db.trajectory(0); + const std::vector kept{3, 8, 12}; + const teacher::PreparedTrajectory prepared(original, kept); + auto copied = prepared; + auto moved = std::move(copied); + const auto view = moved.view(); + EXPECT_NE(view.x.data(), prepared.view().x.data()); + EXPECT_TRUE(view.lat.empty()); + EXPECT_TRUE(view.lon.empty()); + for (std::size_t k = 0; k < kept.size(); ++k) { + EXPECT_EQ(view.t[k], original.t[kept[k]]); + EXPECT_EQ(view.x[k], original.x[kept[k]]); + EXPECT_EQ(view.y[k], original.y[kept[k]]); + } + EXPECT_EQ(view.bbox.x_min, std::ranges::min(view.x)); + EXPECT_EQ(view.bbox.x_max, std::ranges::max(view.x)); + EXPECT_EQ(view.bbox.y_min, std::ranges::min(view.y)); + EXPECT_EQ(view.bbox.y_max, std::ranges::max(view.y)); + + const auto replacement = db.trajectory(1); + for (const auto &indices : + {std::vector{1, 29}, std::vector{0, 4, 8, 16, 29}}) { + moved.assign(replacement, indices); + const auto updated = moved.view(); + EXPECT_EQ(updated.trajectory_id, replacement.trajectory_id); + ASSERT_EQ(updated.size(), indices.size()); + for (std::size_t k = 0; k < indices.size(); ++k) { + EXPECT_EQ(updated.t[k], replacement.t[indices[k]]); + EXPECT_EQ(updated.x[k], replacement.x[indices[k]]); + EXPECT_EQ(updated.y[k], replacement.y[indices[k]]); + } + EXPECT_EQ(updated.bbox.x_min, std::ranges::min(updated.x)); + EXPECT_EQ(updated.bbox.x_max, std::ranges::max(updated.x)); + EXPECT_EQ(updated.bbox.y_min, std::ranges::min(updated.y)); + EXPECT_EQ(updated.bbox.y_max, std::ranges::max(updated.y)); + } +}