Repository navigation
feat(core): validazione dati e metriche di qualità - #37
Conversation
- validateDataset: coordinate (finite, lat ±90, lng ±180), timestamp, intervalli invertiti e punti dei path, con conteggio per motivo - sanitizeDataset: nuovo dataset senza gli elementi non validi - computeQualityMetrics: distribuzione dell'accuratezza, buchi nella timeline (o nei punti) e copertura dei path delle activity - CLI: --quality e --sanitize - test, incluso il test di regressione per #22 sulla copertura dei path Closes #17 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CecUKcJiSVnehcZEiXLgtk
There was a problem hiding this comment.
🟡 Changes recommended
sanitizeDataset removes points/entries but keeps the original dateRange, deviating from the filters' convention of recomputing it and leaving a potentially stale range observable via computeSummary.
2 open findings
What changed in this PR
This PR implements issue #17 by adding a new data-quality module to packages/core. It introduces validateDataset (flags out-of-range/non-finite coordinates, missing/invalid timestamps, reversed intervals, and invalid activity path points), sanitizeDataset (returns a new dataset with invalid items removed, mirroring the filters' immutability), and computeQualityMetrics (accuracy distribution, overlap-aware timeline gaps, and activity path coverage — the metric intended to surface loader bugs like #22). The functions are wired into the public API and the CLI via new --quality and --sanitize flags, and are covered by new unit and CLI tests.
Changes:
- New
quality.tsmodule with validation, sanitization, and quality-metrics functions, re-exported fromindex.ts. - CLI gains
--quality(prints validation + metrics) and--sanitize(drops invalid items before filtering/export), with a dedicatedprintQualityformatter. - 18 new tests (quality + CLI), including a #22 regression asserting 100% activity path coverage on the real-format fixture.
| File | Description |
|---|---|
| packages/core/src/quality.ts | New module: validateDataset, sanitizeDataset, computeQualityMetrics and supporting types/helpers. |
| packages/core/src/cli.ts | Adds --quality and --sanitize flags, printQuality output, and sanitize step in the load pipeline. |
| packages/core/src/index.ts | Re-exports the new quality API and types. |
| packages/core/tests/quality.test.ts | Unit tests for validation, sanitization, and metrics incl. #22 regression. |
| packages/core/tests/cli.test.ts | CLI tests for --quality and --sanitize. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const path = (e.path ?? []).filter(isValidPathPoint); | ||
| return path.length === e.path?.length ? e : { ...e, path }; | ||
| }); | ||
| return { ...dataset, points, timeline }; |
| const before = ds.points.length + ds.timeline.length; | ||
| ds = sanitizeDataset(ds); | ||
| const removed = before - ds.points.length - ds.timeline.length; | ||
| spinner.succeed(`Sanitized — ${removed} invalid items removed`); |
…io CLI Dalla review di Copilot sulla PR #37. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CecUKcJiSVnehcZEiXLgtk


Closes #17
Cosa aggiunge
Nuovo modulo
packages/core/src/quality.ts, esportato daindex.ts:validateDataset(ds, { maxIssues }): controlla punti, voci della timeline e punti dei path. Segnala coordinate non finite o fuori range (lat ±90, lng ±180), timestamp mancanti o non validi e intervalli con fine prima dell'inizio. Restituisce il conteggio per motivo e i primi N problemi con indice.sanitizeDataset(ds): restituisce un nuovo dataset senza punti e voci non validi; dai path toglie solo i punti non validi. Non modifica l'input, come i filtri.computeQualityMetrics(ds, { gapThresholdMinutes, topGaps }):CLI:
--qualitystampa validazione e metriche,--sanitizetoglie gli elementi non validi prima di filtri ed export.Test
quality.test.ts, tra cui il test di regressione per Loader timeline-semantic: i timelinePath vengono scartati e i path delle attività sono vuoti #22: copertura dei path al 100% sulla fixture con la struttura reale--qualitye--sanitizeNote
validateDataset.🤖 Generated with Claude Code
https://claude.ai/code/session_01CecUKcJiSVnehcZEiXLgtk