MaskExpression validates none of its struct items, and nothing checks them against the initial schema when a read relation is built. The two meet only when the record type is derived:
|
@Override |
|
protected final Type.Struct deriveRecordType() { |
|
Type.Struct base = getInitialSchema().struct(); |
|
return getProjection() |
|
.map(projection -> MaskExpressionTypeProjector.project(projection, base)) |
|
.orElse(base); |
|
} |
An out-of-range index builds, and fails only at MaskExpressionTypeProjector.projectStruct's fields.get:
|
private static Type.Struct projectStruct( |
|
MaskExpression.StructSelect structSelect, Type.Struct structType) { |
|
List<Type> fields = structType.fields(); |
|
List<MaskExpression.StructItem> items = structSelect.getStructItems(); |
|
|
|
return TypeCreator.of(structType.nullable()) |
|
.struct(items.stream().map(item -> projectItem(item, fields.get(item.getField())))); |
|
} |
Over t(a i64, b string, c fp64), a mask selecting [0, 5] gives java.lang.IndexOutOfBoundsException: Index 5 out of bounds for length 3, and [-1] gives the same for -1, naming neither the relation nor the mask. field_references.md makes this an invalid plan ("Selecting an ordinal outside the applicable field range results in an invalid plan"), so failing is right; the bare exception is not.
An empty mask builds as well and derives Struct{fields=[]}, a read with no columns. ProtoRelConverter produces one from either proto shape, projection {} with select unset and projection { select {} }, because optionalMaskExpression tests only hasProjection() and ProtoMaskExpressionConverter.fromProto reads getSelect() unconditionally:
|
protected Optional<MaskExpression> optionalMaskExpression(ReadRel rel) { |
|
return Optional.ofNullable( |
|
rel.hasProjection() ? ProtoMaskExpressionConverter.fromProto(rel.getProjection()) : null); |
|
} |
|
public static MaskExpression fromProto(Expression.MaskExpression proto) { |
|
return MaskExpression.builder() |
|
.select(fromProto(proto.getSelect())) |
|
.maintainSingularStruct(proto.getMaintainSingularStruct()) |
|
.build(); |
|
} |
The spec does not say what a present mask selecting nothing means: logical_relations.md says only that the projection "defaults to all of schema" when it is absent.
AbstractReadRel is the one place that holds both the mask and the schema, so a @Value.Check there can reject an out-of-range index as an IllegalArgumentException naming the index and the field count, and it covers the proto read path too, since that builds through the same builders. Whether an empty mask should be refused or read as all of the schema is a spec question rather than one for the check to settle.
So is a repeated index: a mask selecting [1, 1] builds and derives two string columns, where algebra.proto describes a mask as eliminating elements. #1212 raises the same question for the emit mapping, and this issue is that one's twin for the projection mask. Isthmus reaches the bare exception directly once #1280 applies read projections.
Measured on main at 50f8268.
MaskExpressionvalidates none of its struct items, and nothing checks them against the initial schema when a read relation is built. The two meet only when the record type is derived:substrait-java/core/src/main/java/io/substrait/relation/AbstractReadRel.java
Lines 46 to 52 in 50f8268
An out-of-range index builds, and fails only at
MaskExpressionTypeProjector.projectStruct'sfields.get:substrait-java/core/src/main/java/io/substrait/expression/MaskExpressionTypeProjector.java
Lines 26 to 33 in 50f8268
Over
t(a i64, b string, c fp64), a mask selecting[0, 5]givesjava.lang.IndexOutOfBoundsException: Index 5 out of bounds for length 3, and[-1]gives the same for-1, naming neither the relation nor the mask.field_references.mdmakes this an invalid plan ("Selecting an ordinal outside the applicable field range results in an invalid plan"), so failing is right; the bare exception is not.An empty mask builds as well and derives
Struct{fields=[]}, a read with no columns.ProtoRelConverterproduces one from either proto shape,projection {}withselectunset andprojection { select {} }, becauseoptionalMaskExpressiontests onlyhasProjection()andProtoMaskExpressionConverter.fromProtoreadsgetSelect()unconditionally:substrait-java/core/src/main/java/io/substrait/relation/ProtoRelConverter.java
Lines 1643 to 1646 in 50f8268
substrait-java/core/src/main/java/io/substrait/expression/proto/ProtoMaskExpressionConverter.java
Lines 26 to 31 in 50f8268
The spec does not say what a present mask selecting nothing means:
logical_relations.mdsays only that the projection "defaults to all of schema" when it is absent.AbstractReadRelis the one place that holds both the mask and the schema, so a@Value.Checkthere can reject an out-of-range index as anIllegalArgumentExceptionnaming the index and the field count, and it covers the proto read path too, since that builds through the same builders. Whether an empty mask should be refused or read as all of the schema is a spec question rather than one for the check to settle.So is a repeated index: a mask selecting
[1, 1]builds and derives twostringcolumns, wherealgebra.protodescribes a mask as eliminating elements. #1212 raises the same question for the emit mapping, and this issue is that one's twin for the projection mask. Isthmus reaches the bare exception directly once #1280 applies read projections.Measured on
mainat 50f8268.