Split out of #1139, which scoped the integer-parameter shapes (landed as #1141) and deferred this one: "list<any2> is different in kind — its element has to be evaluated recursively rather than a scalar parameter resolved — and ... looks like [its] own issue".
ReturnTypeEvaluator has no visit(ParameterizedType.ListType), so a declared list return reaches the throwing base as Cannot evaluate return-type expression: .... On the pinned catalog the affected variants are the seven ParameterizedReturnTypeTest currently pins as not-yet-derived:
filter:list_func
quantile:req_req_i64_any
regexp_match_substring_all:vchar_vchar_i64_i64
regexp_string_split:vchar_vchar
sort:list
string_split:vchar_vchar
transform:list_func
These are three different problems, not one
a. List<varchar<L1>> — an integer element parameter. string_split, regexp_string_split and regexp_match_substring_all all declare return: "List<varchar<L1>>", where L1 is bound from the varchar<L1> input like any other integer parameter. These need only the list wrapper unwrapped and rewrapped; the element evaluation is exactly what #1141 already implements. This is the cheap subset and can land on its own.
b. list<any1> / list<any2> — a type element parameter, and bind does not recurse. filter declares (list<any1>, func<any1 -> boolean?>) -> list<any1>; sort and transform are the same shape. ParameterBindings.bind is a flat cascade over the scalar parameterized classes with no branch for ListType, Map, Struct or Func, so any1 nested inside a list argument is never bound from anything. Adding visit(ParameterizedType.ListType) alone therefore does not make these derivable — it swaps Cannot evaluate return-type expression for Unbound type parameter 'any1', which reads like a caller mistake rather than a missing binder. The binding side has to learn to recurse into container declarations first.
c. quantile cannot be fixed here. Its LIST?<any> uses a plain any, which by spec binds independently per occurrence and so carries no identity to bind. That is a spec defect, tracked as substrait-io/substrait#1150 with fix PR substrait-io/substrait#1193 open (both positions to any1); it resolves for us on the packaging bump that picks it up, not in this repo.
So (a) and (b) are the work here, and (b) is really "make bind recurse into containers" with the list return as its first consumer.
Note for whoever takes this
TypeExpressionEvaluator's class Javadoc currently attributes all six non-quantile variants to a parameter that is "a type to evaluate rather than an integer to substitute". That is right for filter/sort/transform and wrong for the three List<varchar<L1>> ones; worth correcting alongside, so the paragraph and this issue agree.
@alexandrefimov flagged intent to split this out in #1139 — comment here if you want to take it.
Split out of #1139, which scoped the integer-parameter shapes (landed as #1141) and deferred this one: "
list<any2>is different in kind — its element has to be evaluated recursively rather than a scalar parameter resolved — and ... looks like [its] own issue".ReturnTypeEvaluatorhas novisit(ParameterizedType.ListType), so a declared list return reaches the throwing base asCannot evaluate return-type expression: .... On the pinned catalog the affected variants are the sevenParameterizedReturnTypeTestcurrently pins as not-yet-derived:These are three different problems, not one
a.
List<varchar<L1>>— an integer element parameter.string_split,regexp_string_splitandregexp_match_substring_allall declarereturn: "List<varchar<L1>>", whereL1is bound from thevarchar<L1>input like any other integer parameter. These need only the list wrapper unwrapped and rewrapped; the element evaluation is exactly what #1141 already implements. This is the cheap subset and can land on its own.b.
list<any1>/list<any2>— a type element parameter, andbinddoes not recurse.filterdeclares(list<any1>, func<any1 -> boolean?>) -> list<any1>;sortandtransformare the same shape.ParameterBindings.bindis a flat cascade over the scalar parameterized classes with no branch forListType,Map,StructorFunc, soany1nested inside a list argument is never bound from anything. Addingvisit(ParameterizedType.ListType)alone therefore does not make these derivable — it swapsCannot evaluate return-type expressionforUnbound type parameter 'any1', which reads like a caller mistake rather than a missing binder. The binding side has to learn to recurse into container declarations first.c.
quantilecannot be fixed here. ItsLIST?<any>uses a plainany, which by spec binds independently per occurrence and so carries no identity to bind. That is a spec defect, tracked as substrait-io/substrait#1150 with fix PR substrait-io/substrait#1193 open (both positions toany1); it resolves for us on the packaging bump that picks it up, not in this repo.So (a) and (b) are the work here, and (b) is really "make
bindrecurse into containers" with the list return as its first consumer.Note for whoever takes this
TypeExpressionEvaluator's class Javadoc currently attributes all six non-quantilevariants to a parameter that is "a type to evaluate rather than an integer to substitute". That is right forfilter/sort/transformand wrong for the threeList<varchar<L1>>ones; worth correcting alongside, so the paragraph and this issue agree.@alexandrefimov flagged intent to split this out in #1139 — comment here if you want to take it.