Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,9 @@ and it then falls back to its default. [engine/configs/geolife.json](engine/conf
| --- | --- |
| `data_path` | Parquet file produced by `convert` |
| `seed` | Seed for all random sampling |
| `budget_shares` | Point budgets to test, as a share of all points (`0.01` = 1%) |
| `budget_shares` | Share of interior points to retain after reserving every endpoint: `B = 2T + round(share × (P − 2T))`. Report `B/P` for total retention |
| `split` | `type` (`Random` or `Time`) and `query_percentage` held out for queries |
| `algorithms` | Simplification algorithms to run: `RDP`, `SED`, `UNIFORM` |
| `algorithms` | Simplification algorithms to run: `RDP`, `SED`, `UNIFORM`, `RDP_GLOBAL`. The global variant allocates the total budget by PED and runs once per share, independently of `allocation_alphas` |
| `knn` | `k` and `query_count` (`0` = every trajectory in the query split) |
| `range` | Box size (m), time window (s) and sampling limits for range queries |
| `where_at` | Probes per trajectory and max time gap (s) between points to probe |
Expand All @@ -85,6 +85,13 @@ and it then falls back to its default. [engine/configs/geolife.json](engine/conf
| `clustering` | Route clustering metric: `enabled`, neighbour threshold `eps_m` (m) and `min_size` per cluster. Reuses the self join when one is scored with the same eps |
| `allocation_alphas` | Allocation exponents to test: budgets follow size^α (`1` = size-proportional, `0` = equal). Exports for α ≠ 1 get an `_a<α>` suffix |

`RDP_GLOBAL` is database-wide top-down PED: reserve endpoints, then repeatedly split the
segment with the largest perpendicular error anywhere in the database. It uses no query
workloads and receives exactly the same total point budget as the teacher.

The teacher's range term uses macro F1 like J4. The loop tracks box coverage counts exactly;
independent choices and value curves use F1 marginal gains against fixed partners.

## Tests

```sh
Expand Down
6 changes: 6 additions & 0 deletions TEACHER_NEXT_STEPS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Teacher: status and next steps

Implementation update, 2026-11-20: the range objective and endpoint floor now use macro F1 like
J4, with exact incremental updates in the loop and F1 marginals for candidate and budget scoring.
`RDP_GLOBAL`, a database-wide top-down PED baseline, runs alongside the other baselines in
`teacher_eval` and is enabled in `geolife.json`. It is included in the best-cheap-baseline comparison.
The numerical findings below predate this change and need to be checked with fresh runs.

Updated 2026-11-17 on branch `teacher`. The teacher is described in
[engine/src/teacher/README.md](engine/src/teacher/README.md); this file says what is settled and
what is open.
Expand Down
66 changes: 13 additions & 53 deletions engine/apps/teacher_eval.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
#include "backend/engine/query/join.hpp"
#include "backend/engine/query/knn.hpp"
#include "backend/engine/query/route_clustering.hpp"
#include "backend/engine/selection/rdp.hpp"
#include "backend/engine/selection/selector.hpp"
#include "backend/engine/split/split.hpp"
#include "backend/engine/teacher/teacher.hpp"
Expand All @@ -40,7 +41,8 @@ namespace {

using core::KeptSets;
constexpr double kNan = std::numeric_limits<double>::quiet_NaN();
constexpr std::array kEvalEps{100.0, 125.0, 150.0, 175.0, 200.0, 250.0, 300.0};
// Between the training thresholds, so the teacher never saw them
constexpr std::array kEvalEps{125.0, 175.0, 250.0};
constexpr double kRdpAlpha = 0.25; // RDP_a025
constexpr std::array kPartnerLambdas{0.5, 1.0};
// Unset options keep the config's value; arms default to all teacher arms
Expand All @@ -53,7 +55,6 @@ struct Args {
std::optional<std::vector<double>> shares; // config: budget_shares
std::vector<std::string> arms{teacher::kArms.begin(), teacher::kArms.end()};
bool baselines = true;
std::optional<bool> all_join_pairs;
std::optional<query::JoinMode> teacher_join; // config: teacher.join, else join.mode
std::optional<std::vector<double>> train_eps; // config: teacher.extra_join_eps_m
std::optional<bool> held_out_term; // config: teacher.held_out_term
Expand Down Expand Up @@ -125,15 +126,13 @@ Args parse_args(int argc, char **argv) {
args.teacher_join = mode == "Self" ? query::JoinMode::Self : query::JoinMode::HeldOut;
} else if (key == "--no-baselines")
args.baselines = false;
else if (key == "--seen-pairs-only")
args.all_join_pairs = false;
else
throw std::invalid_argument("unknown argument " + key +
" (--config --data --out --cache --seeds --shares --arms "
"--teacher-join --train-eps --held-out-term "
"--independent-knn --knn-query --term-weights --eval-eps "
"--knn-check "
"--no-baselines --seen-pairs-only)");
"--no-baselines)");
}
teacher::check_arms(args.arms);
return args;
Expand All @@ -147,8 +146,6 @@ pipeline::Config effective_config(const Args &args) {
config.teacher.cache_dir = *args.cache;
if (args.shares)
config.budget_shares = *args.shares;
if (args.all_join_pairs)
config.teacher.all_join_pairs = *args.all_join_pairs;
if (args.teacher_join)
config.teacher.join = *args.teacher_join;
if (args.train_eps)
Expand Down Expand Up @@ -246,13 +243,12 @@ struct Arm {

// PARTNER: RDP with weight 1 + lambda for trajectories that have a join partner
std::vector<Arm> baseline_arms(const core::TrajectoryDb &db,
const std::vector<teacher::JoinPair> &pairs, bool all_join_pairs,
const std::vector<teacher::JoinPair> &pairs,
std::span<const double> shares) {
using selection::AlgorithmType;
std::vector<char> partnered(db.size(), 0);
for (const auto &pair : pairs)
if (all_join_pairs || pair.seen)
partnered[pair.i] = partnered[pair.j] = 1;
partnered[pair.i] = partnered[pair.j] = 1;
std::vector<Arm> arms;
for (const double share : shares) {
const auto budget = allocation::total_budget(db, share);
Expand All @@ -261,6 +257,8 @@ std::vector<Arm> baseline_arms(const core::TrajectoryDb &db,
db, allocation::AllocatorType::SizeProportional, budget, alpha);
};
const auto equal = geometric(0.0);
arms.push_back(
{"RDP_GLOBAL", share, core::to_kept(selection::rdp::simplify_database(db, budget))});
arms.push_back({"RDP_a0", share, select(db, AlgorithmType::RDP, equal)});
arms.push_back({"RDP_a025", share, select(db, AlgorithmType::RDP, geometric(kRdpAlpha))});
arms.push_back({"SED_a0", share, select(db, AlgorithmType::SED, equal)});
Expand Down Expand Up @@ -295,47 +293,11 @@ void validate(const Arm &arm, const core::TrajectoryDb &db) {
throw std::runtime_error(std::format("{}: {} points, budget {}", arm.name, total, budget));
}

struct PairRecall {
double seen = kNan, unseen = kNan;
};

// Mean per-trajectory recall on the seen and unseen halves of the teacher's join pairs
PairRecall pair_recall(const core::TrajectoryDb &compressed,
const std::vector<teacher::JoinPair> &pairs, const query::JoinConfig &join) {
std::vector<char> matched(pairs.size());
core::parallel_for(pairs.size(), [&](std::size_t k) {
matched[k] =
query::frechet_within(compressed.trajectory(pairs[k].i),
compressed.trajectory(pairs[k].j), join.eps_m, join.frechet);
});
PairRecall result;
for (const bool seen : {true, false}) {
std::vector<std::size_t> total(compressed.size(), 0), hits(compressed.size(), 0);
for (std::size_t k = 0; k < pairs.size(); ++k) {
if (pairs[k].seen != seen)
continue;
for (const auto t : {pairs[k].i, pairs[k].j}) {
++total[t];
hits[t] += matched[k];
}
}
double sum = 0;
std::size_t count = 0;
for (std::size_t t = 0; t < total.size(); ++t)
if (total[t] != 0) {
sum += static_cast<double>(hits[t]) / static_cast<double>(total[t]);
++count;
}
(seen ? result.seen : result.unseen) = count ? sum / static_cast<double>(count) : kNan;
}
return result;
}

// Whether the teacher trained on the join at eps_m in mode: one of its thresholds in its own join
// mode, or its held-out term at join.eps_m
bool trained_on(const pipeline::Config &config, const teacher::ObjectiveConfig &objective,
double eps_m, query::JoinMode mode) {
const auto &extra = config.teacher.extra_join_eps_m;
const auto extra = pipeline::teacher_extra_join_eps_m(config);
const bool teacher_eps =
eps_m == config.join.eps_m || std::ranges::find(extra, eps_m) != extra.end();
const bool held_out_term =
Expand Down Expand Up @@ -364,7 +326,7 @@ int main(int argc, char **argv) try {
args.out / "results.csv",
"seed,share,arm,budget,knn_recall,range_f1,where_at_avg_hit,join_f1,clustering,"
"where_at_within_100,join_micro_f1,knn_tau,norm_knn,norm_range,norm_where_at,norm_join,"
"norm_clustering,J4,J_T,join_recall_seen,join_recall_unseen,predicted_clusters,"
"norm_clustering,J4,J_T,predicted_clusters,"
"eval_s,join_mode,self_join_f1,norm_self_join,teacher_join_mode,norm_join_main,"
"norm_clustering_main,self_knn_recall,norm_self_knn,norm_self_join_mean,J4_self,"
"teacher_knn_query");
Expand Down Expand Up @@ -412,8 +374,7 @@ int main(int argc, char **argv) try {
core::seconds_since(start));
}
if (args.baselines) {
for (auto &arm :
baseline_arms(db, pairs, config.teacher.all_join_pairs, config.budget_shares))
for (auto &arm : baseline_arms(db, pairs, config.budget_shares))
arms.push_back(std::move(arm));
for (const double share : config.budget_shares)
for (auto &arm : model.single_recipes(allocation::total_budget(db, share)))
Expand Down Expand Up @@ -476,16 +437,15 @@ int main(int argc, char **argv) try {
const auto scores =
evaluator::Evaluator(db, compressed, test, scoring).evaluate().scores;
const auto normalized = evaluator::normalize_scores(scores, floor);
const auto recall = pair_recall(compressed, pairs, config.join);
results << std::format(
"{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{:.2f},"
"{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{:.2f},"
"{},{},{},{},{},{},{},{},{},{},{}\n",
seed, arm.share, arm.name, allocation::total_budget(db, arm.share),
scores.knn.recall_at_k, scores.range_f1, scores.where_at.average_hit,
scores.join.f1, scores.clustering.score, scores.where_at.within_radius,
scores.join.micro_f1, scores.knn.kendalls_tau_topk, normalized.knn,
normalized.range, normalized.where_at, normalized.join, normalized.clustering,
normalized.combined, objective.total(arm.kept), recall.seen, recall.unseen,
normalized.combined, objective.total(arm.kept),
scores.clustering.predicted_clusters, core::seconds_since(start),
mode_name(config.join.mode), scores.self_join.f1, normalized.self_join,
mode_name(objective.config().join_mode), normalized.join_f1,
Expand Down
13 changes: 5 additions & 8 deletions engine/apps/teacher_stats.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
("loop", "BUDGET_CHOICE", "BUDGET_LOOP"),
("looking once", "BUDGET_CHOICE", "BUDGET_LOOP_NO_UPDATE"),
("reacting", "BUDGET_LOOP_NO_UPDATE", "BUDGET_LOOP")]
CHEAP = ["RDP_a0", "RDP_a025", "PARTNER_l0.5", "PARTNER_l1"]
CHEAP = ["RDP_a0", "RDP_a025", "RDP_GLOBAL", "SED_a0", "UNI_a0", "PARTNER_l0.5", "PARTNER_l1"]


def num(v):
Expand Down Expand Up @@ -71,7 +71,7 @@ def main():
parts = [num(r.get(c, "")) for c in ["norm_knn", "norm_range", "norm_where_at", "norm_join"]]
parts = [x for x in parts if x is not None]
r["J4"] = str(statistics.fmean(parts)) if parts else ""
for m in METRICS + ["J_T", "join_recall_seen", "join_recall_unseen"]:
for m in METRICS + ["J_T"]:
v = num(r.get(m, ""))
if v is not None:
data[r["share"]][r["arm"]][m][int(r["seed"])] = v
Expand All @@ -87,14 +87,11 @@ def main():
for share in sorted(data, key=float):
arms = data[share]
out.append(f"## Share {float(share) * 100:g}%\n")
out.append("| Arm | " + " | ".join(METRICS) + " | seen / unseen join recall |")
out.append("|---|" + "---:|" * (len(METRICS) + 1))
out.append("| Arm | " + " | ".join(METRICS) + " |")
out.append("|---|" + "---:|" * len(METRICS))
for arm in sorted(arms):
cells = [mean_sd(list(arms[arm][m].values())) for m in METRICS]
seen = list(arms[arm]["join_recall_seen"].values())
unseen = list(arms[arm]["join_recall_unseen"].values())
su = f"{statistics.fmean(seen):.4f} / {statistics.fmean(unseen):.4f}" if seen and unseen else "–"
out.append(f"| {arm} | " + " | ".join(cells) + f" | {su} |")
out.append(f"| {arm} | " + " | ".join(cells) + " |")
out.append("")
cheap = [a for a in CHEAP if a in arms and arms[a]["J4"]]
best = max(cheap, key=lambda a: statistics.fmean(arms[a]["J4"].values())) if cheap else None
Expand Down
5 changes: 2 additions & 3 deletions engine/configs/geolife.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"type": "Random",
"query_percentage": 0.15
},
"algorithms": ["RDP", "SED", "UNIFORM"],
"algorithms": ["RDP", "SED", "UNIFORM", "RDP_GLOBAL"],
"knn": {
"k": 10,
"query_count": 0
Expand Down Expand Up @@ -48,8 +48,7 @@
"enabled": false,
"cache_dir": "cache",
"arms": ["BUDGET_CHOICE", "BUDGET_LOOP"],
"range_query_count": 64000,
"extra_join_eps_m": [100.0, 125.0, 200.0, 300.0],
"range_query_count": 128000,
"held_out_term": false,
"ranking_knn": true,
"term_weights": [1.0, 1.0, 1.0, 1.0]
Expand Down
2 changes: 2 additions & 0 deletions engine/include/backend/engine/pipeline/config.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ struct Config {
// Reads a JSON config. Only data_path is required, a missing key keeps its default.
Config load_config(const std::filesystem::path &path);

// teacher.extra_join_eps_m, by default join.score_eps_m
std::vector<double> teacher_extra_join_eps_m(const Config &config);
teacher::WorkloadConfig teacher_workload_config(const Config &config);

teacher::ObjectiveConfig teacher_objective_config(const Config &config);
Expand Down
6 changes: 6 additions & 0 deletions engine/include/backend/engine/selection/rdp.hpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#pragma once

#include "backend/engine/core/simplification.hpp"
#include "backend/engine/core/trajectory_db.hpp"

#include <cstddef>
Expand All @@ -15,4 +16,9 @@ std::vector<std::size_t> simplify(const core::TrajectoryView &trajectory,
// reproduces simplify(trajectory, b), for 2 <= b <= min(limit, size).
std::vector<std::size_t> sequence(const core::TrajectoryView &trajectory, std::size_t limit);

// Database-wide top-down PED (RDP_GLOBAL / TD(W, PED)). Reserve every endpoint, then split
// the segment with the largest perpendicular error anywhere in the database until total_budget.
// Rejects budgets below the endpoint count or above the original point count.
core::Simplification simplify_database(const core::TrajectoryDb &db, std::size_t total_budget);

} // namespace selection::rdp
2 changes: 1 addition & 1 deletion engine/include/backend/engine/selection/selector.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

namespace selection {

enum class AlgorithmType { RDP, SED, UNIFORM };
enum class AlgorithmType { RDP, SED, UNIFORM, RDP_GLOBAL };

std::vector<std::size_t> simplify_trajectory(const core::TrajectoryView &, AlgorithmType,
std::size_t);
Expand Down
5 changes: 3 additions & 2 deletions engine/include/backend/engine/teacher/curves.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ struct Curve {
double at(std::size_t budget) const { return value[budget - 2]; }
};

// Maximises sum curves[i].at(b_i) with sum b_i = total_budget, greedily on the best gain per point
// over the next 1 to 3 points, which steps over flat stretches. Throws if the budget cannot fit
// Maximises sum curves[i].at(b_i) with sum b_i = total_budget: greedy on the slopes of each curve's
// upper concave envelope, which steps over flat stretches of any length. Optimal when the budget
// ends on envelope vertices. Throws if the budget cannot fit
std::vector<std::size_t> allocate(const std::vector<Curve> &curves, std::size_t total_budget,
double *value = nullptr);

Expand Down
3 changes: 2 additions & 1 deletion engine/include/backend/engine/teacher/greedy.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@
namespace teacher {

// BKH's workload-driven selector: from the endpoints and a seed, repeatedly keep the point that
// raises i's range and where-at terms most, then take the rest from fill, skipping kept points
// raises range F1 (other trajectories original) and i's where-at term most, then take the rest
// from fill, skipping kept points
class Greedy {
public:
Greedy(const Objective &objective, std::size_t i);
Expand Down
Loading
Loading