feat(platform)!: compare typed arrays in refersTo where entries (PV14)

book/src/contract-keywords/typed-arrays.md

@@ -124,7 +124,7 @@ Where a typed array cannot be used:
 
 - in an index of an index-only type, or as an index-only type's terminal or entry payload: an index-only type's entries are its rows, one per document (10231, or `InvalidIndexPropertyTypeError` 10206 in a terminal);
 - inside an object, when an index names it: an indexed typed array is a top-level property (10231);
-- on either side of a `where` entry in a reference;
+- in a `where` entry of a reference, other than as a list: two typed arrays of one element kind may be compared as lists (the same elements in the same order), and a referenced typed array may be asked to `contains` a value or `containsAll` of another typed array's elements; a float element is refused, and a typed array is equal to no single value (see [Typed arrays](refers-to.md#typed-arrays) in References);
 - as an operand of a [propertyConstraints](property-constraints.md) rule, other than in a `present` or `absent` test;
 - with [`encryptedFor`](encrypted-for.md), which only a byte array takes.
 

book/src/fees/overview.md

@@ -549,7 +549,10 @@ the fees of documents whose type declares a [`ttl`](../data-model/document-ttl.m
 and the budget of a [`chooseFirst`](../contract-keywords/refers-to-lookup.md#choosing-the-first-commitment-choosefirst)
 competitor scan (`choose_first_credits_per_candidate`, 100,000 credits a
 candidate, times `SystemLimits::max_choose_first_candidates`), which check tx
-reserves on top of a reveal's estimated fee. Platform states store only
+reserves on top of a reveal's estimated fee, and the processing fee of a
+[`where` entry comparing typed arrays](../contract-keywords/refers-to.md#typed-arrays)
+(`where_list_credits_per_element`, 100 credits for each list element it reads,
+billed before it compares them). Platform states store only
 `fee_version_number`, so neither group touches a stored format.
 
 Fee versions are stored in the `FEE_VERSIONS` array and looked up by number. The

packages/js-evo-sdk/README.md

@@ -244,12 +244,18 @@ for (const ref of contract.documentTypeReferences('note')) {
   // the writer's own `$ownerId`: a write gate such as
   // `{ '$ownerId': '$ownerId' }` lets only the referenced document's current
   // owner create or replace the referring document.
+  // Between two typed arrays, e.g. `where: { tags: 'tags' }`, the equality
+  // compares lists: the same elements in the same order, a list left out
+  // being the empty one.
   // A value may instead be a condition object with one key, as the schema
-  // writes it: `{ const: v }`, `{ notEqual: b }`, `{ in: [v1, v2] }`, or an
+  // writes it: `{ const: v }`, `{ notEqual: b }`, `{ in: [v1, v2] }`, an
   // ordering such as `{ lessThanOrEqual: 'amount' }` or
   // `{ greaterThan: '$updatedAt' }` (a bound is a referring property,
   // `'$ownerId'`, a time or height, or `{ const: v }`; an integer past
-  // Number.MAX_SAFE_INTEGER is a bigint). Tell the two apart by type:
+  // Number.MAX_SAFE_INTEGER is a bigint), or, of a typed array,
+  // `{ contains: 'tag' }` (a referring property or `'$ownerId'` the list must
+  // hold) or `{ containsAll: 'tags' }` (a referring typed array whose every
+  // element the list must hold). Tell the two apart by type:
   for (const [referenced, value] of Object.entries(ref.where ?? {})) {
     if (typeof value === 'string') {
       // an equality with the referring property `value`

packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json

@@ -514,7 +514,7 @@
       "minItems": 2
     },
     "referenceWhereCondition": {
-      "description": "A refersTo where entry that is not an equality with a referring value: an object with one key. const, a literal the referenced side must equal (an integer or true or false for such a property, a string for a string property, a base58 identifier for an identifier, $ownerId, $creatorId or $id, hex for a byte array); a referenced side the document leaves out equals none. notEqual, what it must differ from: a property of the referring document type, \"$ownerId\" (the writer, checked on every replace) or a { const }; the negation of an equality, so two absent sides are equal. in, two to thirty distinct literals of one kind, one of which it must equal; an absent side is in none. greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer property the referenced side must compare with this way: bounded by an integer property of the referring document type, an integer { const }, or a time or height the referring type lists in required ($createdAt as written by the create; $updatedAt and $transferredAt, and their heights, change without a write to the referring document, so an entry bounded by one is checked on every replace); it holds only when both sides have a value. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). Available from protocol version 14",
+      "description": "A refersTo where entry that is not an equality with a referring value: an object with one key. const, a literal the referenced side must equal (an integer or true or false for such a property, a string for a string property, a base58 identifier for an identifier, $ownerId, $creatorId or $id, hex for a byte array); a referenced side the document leaves out equals none. notEqual, what it must differ from: a property of the referring document type, \"$ownerId\" (the writer, checked on every replace) or a { const }; the negation of an equality, so two absent sides are equal. in, two to thirty distinct literals of one kind, one of which it must equal; an absent side is in none. greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer property the referenced side must compare with this way: bounded by an integer property of the referring document type, an integer { const }, or a time or height the referring type lists in required ($createdAt as written by the create; $updatedAt and $transferredAt, and their heights, change without a write to the referring document, so an entry bounded by one is checked on every replace); it holds only when both sides have a value. contains, of a typed array, a value it must hold an element equal to: a property of the referring document type of the element kind, or \"$ownerId\" (the writer, among identifier elements, checked on every replace); a list the document leaves out holds nothing, and a value the write leaves out is held by no list. containsAll, of a typed array, a typed array of the referring document type of the same element kind whose every element it must hold, in any order; every list holds all of an empty one, and a list left out is empty. Elements compare by value, never floats. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). Available from protocol version 14",
       "type": "object",
       "minProperties": 1,
       "maxProperties": 1,
@@ -573,6 +573,18 @@
         },
         "lessThanOrEqual": {
           "$ref": "#/$defs/referenceWhereOrderingBound"
+        },
+        "contains": {
+          "type": "string",
+          "minLength": 1,
+          "maxLength": 256,
+          "pattern": "^(\\$ownerId|[a-zA-Z0-9_]{1,64}(\\.[a-zA-Z0-9_]{1,64})*)$"
+        },
+        "containsAll": {
+          "type": "string",
+          "minLength": 1,
+          "maxLength": 256,
+          "pattern": "^[a-zA-Z0-9_]{1,64}(\\.[a-zA-Z0-9_]{1,64})*$"
         }
       },
       "additionalProperties": false
@@ -933,7 +945,7 @@
               }
             },
             "where": {
-              "description": "permanentDocument, moderatedDocument and deletableDocument references only: what the referenced document must hold once found, checked by consensus when the referring document is written. Each entry maps a property of the referenced document type, or one of its $ownerId, $creatorId and $id system identifiers, to the referring document's value it must equal: a schema property of the declaring document type, or \"$ownerId\", the writer, which turns the entry into a write gate: only an identity whose id equals the referenced side may create or replace the document. A referring value may be compared once. $creatorId needs a referenced document type that records creator ids (transferable or tradeable types of a format-1 contract), and $ownerId, $creatorId or $id an identifier on the referring side. Both sides must exist and share one value kind, validated at contract registration. $ownerId follows the referenced document through transfers while $creatorId and $id never change; each is checked when the referring document is written, not when the referenced document later moves. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). An entry may instead hold an object with one key (see referenceWhereCondition): const, a literal the referenced side must equal; notEqual, a referring property, \"$ownerId\" or a const it must differ from; in, two or more literals it must be one of; or greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer bound it must compare with. The one-comparison-per-referring-value rule holds for the equalities only. Beside a findBy function the declaration is judged when the document is created only, so its entries gate the create alone, and every referring value other than $ownerId must be transient or fixed once written (not listed under immutable with a condition). where never finds the document: \".\" and functions belong to findBy. Available from protocol version 14.",
+              "description": "permanentDocument, moderatedDocument and deletableDocument references only: what the referenced document must hold once found, checked by consensus when the referring document is written. Each entry maps a property of the referenced document type, or one of its $ownerId, $creatorId and $id system identifiers, to the referring document's value it must equal: a schema property of the declaring document type, or \"$ownerId\", the writer, which turns the entry into a write gate: only an identity whose id equals the referenced side may create or replace the document. A referring value may be compared once. $creatorId needs a referenced document type that records creator ids (transferable or tradeable types of a format-1 contract), and $ownerId, $creatorId or $id an identifier on the referring side. Both sides must exist and share one value kind, validated at contract registration; two typed arrays of one element kind (not floats, both stored) are equal as lists, the same elements in the same order compared by value, a list the document leaves out the empty one, and a typed array is equal to nothing else. $ownerId follows the referenced document through transfers while $creatorId and $id never change; each is checked when the referring document is written, not when the referenced document later moves. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). An entry may instead hold an object with one key (see referenceWhereCondition): const, a literal the referenced side must equal; notEqual, a referring property, \"$ownerId\" or a const it must differ from; in, two or more literals it must be one of; greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer bound it must compare with; or, of a typed array, contains, a referring value or \"$ownerId\" it must hold, or containsAll, a referring typed array whose every element it must hold. The one-comparison-per-referring-value rule holds for the equalities only. Beside a findBy function the declaration is judged when the document is created only, so its entries gate the create alone, and every referring value other than $ownerId must be transient or fixed once written (not listed under immutable with a condition). where never finds the document: \".\" and functions belong to findBy. Available from protocol version 14.",
               "type": "object",
               "minProperties": 1,
               "maxProperties": 10,

packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs

@@ -7,7 +7,7 @@ use crate::data_contract::document_type::class_methods::consensus_or_protocol_da
 use crate::data_contract::document_type::class_methods::parse_typed_array::parse_typed_array;
 use crate::data_contract::document_type::methods::DocumentTypeBasicMethods;
 use crate::data_contract::document_type::property::reference_where::{
-    WHERE_CONST, WHERE_IN, WHERE_NOT_EQUAL,
+    WHERE_CONST, WHERE_CONTAINS, WHERE_CONTAINS_ALL, WHERE_IN, WHERE_NOT_EQUAL,
 };
 use crate::data_contract::document_type::property_constraints::{
     either_version_path, parse_delete_constraints, parse_property_constraints, referenced_path,
@@ -35,8 +35,8 @@ use crate::data_contract::document_type::{
     IdentityKeyReferenceRequirements, KeyIdReference, KeyReferenceIdentityProperty,
     ListElementReference, LookupChooseFirst, LookupHashKey, LookupKeySource, PropertyReference,
     ReferenceCombinator, ReferenceOperands, SystemFunction, WhereBoundRead, WhereCondition,
-    WhereLiteral, WhereLiteralKind, WhereNotEqualBound, WhereOrdering, WhereOrderingBound,
-    COMBINABLE_REFERENCE_TARGET_TYPES,
+    WhereContainsBound, WhereLiteral, WhereLiteralKind, WhereNotEqualBound, WhereOrdering,
+    WhereOrderingBound, COMBINABLE_REFERENCE_TARGET_TYPES,
 };
 use crate::data_contract::errors::DataContractError;
 use crate::data_contract::{TokenConfiguration, TokenContractPosition};
@@ -1291,7 +1291,13 @@ fn parse_reference_target(
         // `moderatorAbilities.deleteKeepsFields`, are left out where the document
         // held none, and the type may be another contract's): a gate on any other
         // property could refuse every later replace, for good on a property that
-        // can not be repointed or cleared, so it may compare only those two
+        // can not be repointed or cleared, so an equality or a `notEqual` with the
+        // writer may compare only those two. A `contains` of the writer is the one
+        // gate admitted here on another property, a typed array, which no system
+        // identifier is: a list the document held none of is read as empty, as
+        // against the document in state, so the gate refuses after a removal what
+        // it refused before; registration, with the referenced type in hand, keeps
+        // the list to one every removal record holds
         "moderatedDocument" => {
             let (contract_id, document_type_name) =
                 parse_document_reference_type(refers_to_map, reference_type)?;
@@ -1482,12 +1488,15 @@ type ParsedWhere = (BTreeMap<String, String>, BTreeMap<String, WhereCondition>);
 /// other entry is an object with one key, a [`WhereCondition`]: `const`, a
 /// literal it must equal; `notEqual`, a referring property, `"$ownerId"` or a
 /// `const` it must differ from; `in`, two or more distinct literals of one
-/// kind it must be one of; or an ordering (`greaterThan`, `greaterThanOrEqual`,
+/// kind it must be one of; an ordering (`greaterThan`, `greaterThanOrEqual`,
 /// `lessThan`, `lessThanOrEqual`) with a referring property, an integer
 /// `const` or a time or height of the referring document as its bound, never
-/// of an identifier. `where` never finds the document and reads neither `"."`
-/// nor a function, which belong to `findBy`. What the names resolve to, and
-/// whether a literal fits the property it is compared with, is checked at
+/// of an identifier; or, of a typed array, `contains`, a referring property or
+/// `"$ownerId"` it must hold, or `containsAll`, a referring typed array whose
+/// every element it must hold. `where` never finds the document and reads
+/// neither `"."` nor a function, which belong to `findBy`. What the names
+/// resolve to, whether a literal fits the property it is compared with, and
+/// whether an equality or a condition compares lists, is checked at
 /// registration.
 fn parse_where(
     refers_to_map: &BTreeMap<String, &Value>,
@@ -1532,8 +1541,8 @@ fn parse_where(
         let referring_property = value.as_text().ok_or_else(|| {
             DataContractError::InvalidContractStructure(
                 "refersTo where values must name a property of the referring document type or \
-                 \"$ownerId\", the writer, or be an object holding const, notEqual, in or an \
-                 ordering"
+                 \"$ownerId\", the writer, or be an object holding const, notEqual, in, an \
+                 ordering, contains or containsAll"
                     .to_string(),
             )
         })?;
@@ -1585,9 +1594,11 @@ fn parse_where_referring_property<'a>(
 }
 
 /// The [`WhereCondition`] a `where` entry keyed by `referenced_property`
-/// declares with `value`, an object with one key: `const`, `notEqual`, `in` or
-/// an ordering. Any other key is refused: a function computes a key that
-/// finds the document, which belongs to `findBy`.
+/// declares with `value`, an object with one key: `const`, `notEqual`, `in`,
+/// an ordering, `contains` or `containsAll`. Any other key is refused: a
+/// function computes a key that finds the document, which belongs to `findBy`.
+/// `contains` and `containsAll` read a list, which `$ownerId`, `$creatorId`
+/// and `$id` are not.
 fn parse_where_condition(
     referenced_property: &str,
     value: &Value,
@@ -1613,7 +1624,7 @@ fn parse_where_condition(
     let [(Value::Text(key), body)] = entries.as_slice() else {
         return Err(invalid(
             "must be an object with one key: const, notEqual, in, greaterThan, \
-             greaterThanOrEqual, lessThan or lessThanOrEqual"
+             greaterThanOrEqual, lessThan, lessThanOrEqual, contains or containsAll"
                 .to_string(),
         ));
     };
@@ -1674,15 +1685,55 @@ fn parse_where_condition(
             }
             Ok(WhereCondition::In(literals))
         }
+        list_key @ (WHERE_CONTAINS | WHERE_CONTAINS_ALL) => {
+            if identifier_side {
+                return Err(invalid(format!(
+                    "reads {list_key} of an identifier, which is one value: {list_key} reads a \
+                     typed array"
+                )));
+            }
+            let Value::Text(path) = body else {
+                return Err(invalid(format!(
+                    "{list_key} must be a property path of the referring document type{}",
+                    if list_key == WHERE_CONTAINS {
+                        ", or \"$ownerId\""
+                    } else {
+                        ""
+                    }
+                )));
+            };
+            if list_key == WHERE_CONTAINS {
+                let bound = if path == OWNER_ID {
+                    WhereContainsBound::Writer
+                } else {
+                    WhereContainsBound::Property(
+                        parse_where_referring_property(referenced_property, path)?.to_string(),
+                    )
+                };
+                return Ok(WhereCondition::Contains(bound));
+            }
+            // The writer is one identifier, not a list of them
+            if path == OWNER_ID {
+                return Err(invalid(
+                    "containsAll reads $ownerId, the writer, which is one identifier: \
+                     containsAll reads a typed array of the referring document type, and \
+                     contains the writer"
+                        .to_string(),
+                ));
+            }
+            Ok(WhereCondition::ContainsAll(
+                parse_where_referring_property(referenced_property, path)?.to_string(),
+            ))
+        }
         ordering_key => {
             let Some(ordering) = WhereOrdering::ALL
                 .into_iter()
                 .find(|ordering| ordering.wire_name() == ordering_key)
             else {
                 return Err(invalid(format!(
                     "names \"{ordering_key}\", which is not const, notEqual, in, greaterThan, \
-                     greaterThanOrEqual, lessThan or lessThanOrEqual: a function computes a key \
-                     that finds the document, which belongs in findBy"
+                     greaterThanOrEqual, lessThan, lessThanOrEqual, contains or containsAll: a \
+                     function computes a key that finds the document, which belongs in findBy"
                 )));
             };
             if identifier_side {