[jacoblambert]

Summary

Completes the 0.x → 1.0 conversion from #307: vehicle_state, traffic-light links, source LiDARs, capture time and placeholder masks. Every local 0.x dataset loads (16 datasets: 10 j6gen2, 4 annotation datasets, 2 traffic-light autolabel datasets).

vehicle_state

• source is vehicle_interface: a fact, not a placeholder (tier4_perception_dataset reads /vehicle/status/*).
• Shift: 0.x FORWARD and HIGH become DRIVE (the 0.x converter writes DRIVE, but the 0.x devkit enum and the sample use the other two).
• additional_info.speed is dropped: it is norm(longitudinal, lateral), unsigned, so it can't fill the 1.0 velocities. Velocities and heading_rate are null.

Traffic-light links → instance.map_element

• The 0.x traffic_light table (feat(schema): add new traffic light schema for object detection #288) links a lamp instance to the lanelet2 way of that lamp (type=traffic_light, what the REF014 check verifies). It has no states, so it becomes the instance's map_element (id = primitive_id as int, type: way), and the 1.0 traffic_light table (states per regulatory element) stays empty.
• A second 0.x table holds the same link: traffic_light_instance_map (traffic_light_linestring_id), from the unmerged branch feat/traffic-light-instance-map, which real autolabel datasets already use. It is read too and wins when both list a lamp.
• traffic_light rows without primitive_id are skipped with a warning, since no 0.x devkit reads them either.
• Map element ids refer to the dataset's own map/ folder. Changing the map means regenerating the dataset or making a new version of it.

Source LiDARs

The LiDARs merged into a concatenated channel have sensor rows but no frames of their own in 0.x. They are now filled from the concatenated channel's .info files, instead of the default 5-field .bin layout with 0 frames:

• format and fields: the concatenated channel's (their points sit in its files, with the same layout);
• frame_count: the number of concatenated frames that list points from them.

On real data: every source in all 171 frames of one dataset (7 fields); 554/554 frames in another, except one LiDAR missing from 2 frames. The .info files are read once per channel (0.1 s for 554 frames). Points are still split the 0.x .bin way until the .bin → Parquet migration.

Capture time

log.data_captured is "" in every local 0.x dataset, but 1.0 requires an ISO 8601 time. An empty one is filled with the log's first sample time in UTC (on one dataset, this matches the capture time in its README).

2D boxes and masks

• Boxes are (xmin, ymin, xmax, ymax) on pixel edges, xmax = x + w, as the 0.x converter writes them from Deepen labels. Checked on real data: the largest xmax equals the image width (2880.0). Boxes computed from masks use the same convention.
• 0.x object_ann rows with only a mask have no bbox; 1.0 requires one, so it is computed from the mask. The test checks the box against the decoded mask pixels.
• Empty masks become mask: null. One real dataset has the same empty placeholder mask on all 395 box labels, written by hand with size (width, height) instead of pycocotools' (height, width).

Category index

0.x categories without index get their position in category.json, which is how the 0.x T4Devkit reads them (fix_category_table): all unset → positions, a mix of set and unset → error. Of the local datasets, only the semantic segmentation one sets index itself (to the positions); all others leave it unset. 1.0 datasets keep null when there is no index.

Known issues and unknowns

Not fixed here:

1. Autolabels without the flag stay manual. Two real autolabel datasets don't set automatic_annotation, so both 0.x and 1.0 read them as manual. The converter does not guess provenance; datasets must set the flag.
2. Traffic-light states are not converted. The state is in free-form category names (red, red_right, crosswalk_red, red_straight_left, red_circle, green_arrow, unknown, ...) and attributes (light_status.on/off, traffic_light_color.*), with no fixed vocabulary to map to 1.0 color / shape / status. This needs an agreed mapping. A lamp shared by several regulatory elements is normal (one signal head serves several lanes): a state read from it gives one row per regulatory element.
3. map_id / version_id stay unknown: 0.x map.json has no ids.
4. Two 0.x lamp-to-way tables, one from an unmerged branch. If that branch changes before merging, this conversion changes too.
5. Lamp orientation: no local dataset has the 0.x orientation field, so its conversion is untested on real data.
6. Mask size order: a non-empty mask stored as (width, height) would be misread. None seen.
7. 0.x box convention is mixed: boxes the 0.x converter derives from Deepen segmentation (painting2d.py) end at the last pixel, not the edge. Existing 0.x boxes are not re-derived.
8. 0.x SurfaceAnn.bbox bug: it swaps x and y (np.where returns rows, cols) and ends at the last pixel. The conversion doesn't use it; worth a separate 0.x fix.
9. Not covered by any local dataset: mask-only object_ann rows, surface_ann rows, non-empty vehicle_state (every real vehicle_state.json is empty).

Tests

tests/schema/v1/test_convert_v0.py, on the 0.x sample dataset with small edits:

• vehicle_state rows load with shift DRIVE and source vehicle_interface.
• An empty data_captured becomes the first sample time.
• A lamp row in traffic_light (primitive_id) or traffic_light_instance_map (traffic_light_linestring_id) sets the instance's map_element; the 1.0 traffic_light table stays empty.
• A source LiDAR takes the concatenated channel's 7 fields and counts only frames with its points.
• An empty mask becomes null and the box is kept.
• The surface_ann box matches its decoded mask pixels.
• Categories without index get their list position; a partly set index is rejected.

[ktro2828]

LGTM