diff --git a/book/src/contract-keywords/refers-to.md b/book/src/contract-keywords/refers-to.md index 4e0f851fd6f..138f0b484d7 100644 --- a/book/src/contract-keywords/refers-to.md +++ b/book/src/contract-keywords/refers-to.md @@ -100,7 +100,7 @@ The same for a type whose documents disappear only when the contract's moderator A reply to a `post` its author can not delete, which the moderators can take down. The post must exist when the reply is written. Once a moderator removes it, the reply keeps pointing at it, and a replace of the reply is checked as if the reference were permanent: an edit of the reply's text passes, and the reference now resolves to the removal record. What a replace can not do is point a reply at a removed post, or write a new reply to one: a value the write sets must name a document in state. -A removed document has no values left but those its record keeps: its id, its owner, and the fields its type lists under [`moderatorAbilities.deleteKeepsFields`](deletion.md#moderatorabilitiesdeletekeepsfields). A `where` entry the replace checks again (its referring property changed, or it is a writer gate valued `"$ownerId"`) is checked against the record when it compares the referenced `$ownerId`, `$id` or a property the type keeps (a kept path, or one inside a kept object), with the same absence rule as against a document in state, and refused with `ReferencedDocumentRemovedError` (40145) when it compares any other property; the replace can repoint the reference at a document in state, or leave that property unchanged until the post is restored. A writer gate (a `where` entry valued `"$ownerId"`, or a `notEqual` bounded by it) is checked on every replace, so on a `moderatedDocument` reference it may compare only the referenced `$ownerId` or `$id` (`InvalidContractStructure`, 10231, otherwise), the only values sure to be in every record: a gate on anything else could refuse every replace once the post is removed. An ordering bounded by `$updatedAt` or `$transferredAt` (or one of their heights) is checked on every replace too, and compares only a property the type keeps (40126 otherwise). A [rule reading the post](property-constraints.md#values-of-a-referenced-document) reads only its `$ownerId`, its `$id` and the properties its type keeps, for the same reason. The value is kept as long as it is the one the stored document held, whatever else the replace changed in the same object or list. The value is always the document's id: `findBy`, `inList` and operands of an [expression](refers-to-expressions.md) are refused, since a removal frees the unique index keys a `findBy` finds by. +A removed document has no values left but those its record keeps: its id, its owner, and the fields its type lists under [`moderatorAbilities.deleteKeepsFields`](deletion.md#moderatorabilitiesdeletekeepsfields). A `where` entry the replace checks again (its referring property changed, or it is a writer gate valued `"$ownerId"`) is checked against the record when it compares the referenced `$ownerId`, `$id` or a property the type keeps (a kept path, or one inside a kept object), with the same absence rule as against a document in state, and refused with `ReferencedDocumentRemovedError` (40145) when it compares any other property; the replace can repoint the reference at a document in state, or leave that property unchanged until the post is restored. A writer gate (a `where` entry valued `"$ownerId"`, or a `notEqual` bounded by it) is checked on every replace, so on a `moderatedDocument` reference it may compare only the referenced `$ownerId` or `$id` (`InvalidContractStructure`, 10231, otherwise), the only values sure to be in every record: a gate on anything else could refuse every replace once the post is removed. An ordering bounded by `$updatedAt` or `$transferredAt` (or one of their heights), and a `contains` of `"$ownerId"`, are checked on every replace too, and compare only a property the type keeps (40126 otherwise). A [rule reading the post](property-constraints.md#values-of-a-referenced-document) reads only its `$ownerId`, its `$id` and the properties its type keeps, for the same reason. The value is kept as long as it is the one the stored document held, whatever else the replace changed in the same object or list. The value is always the document's id: `findBy`, `inList` and operands of an [expression](refers-to-expressions.md) are refused, since a removal frees the unique index keys a `findBy` finds by. A chained or composite query that joins through a moderated reference proves each removed document's record beside the documents it joins, and reports the removal with it. @@ -159,7 +159,7 @@ The contract holding `documentType`, as a base58 string or an array of 32 bytes. ### `where` -States what the referenced document must hold once found. It takes 1 to 10 entries, on `permanentDocument`, `moderatedDocument` and `deletableDocument` references, each keyed by a property of the referenced document. An entry valued with a referring value, `{ "": "" }`, must hold as an equality when the referring document is written; an entry valued with an object holds a literal, an inequality, a list or an ordering instead (see [Conditions](#conditions)). `where` never finds the document: the value does, as its id, or [`findBy`](refers-to-lookup.md) does. `where` is checked against the document found. +States what the referenced document must hold once found. It takes 1 to 10 entries, on `permanentDocument`, `moderatedDocument` and `deletableDocument` references, each keyed by a property of the referenced document. An entry valued with a referring value, `{ "": "" }`, must hold as an equality when the referring document is written; an entry valued with an object holds a literal, an inequality, a list or an ordering instead (see [Conditions](#conditions)). Two typed arrays compare as lists (see [Typed arrays](#typed-arrays)). `where` never finds the document: the value does, as its id, or [`findBy`](refers-to-lookup.md) does. `where` is checked against the document found. - **The key** is the referenced side: a property of the referenced document type, a dotted path for a nested one, or one of the referenced document's own identifiers: `$ownerId` (its current owner, which follows it through transfers), `$creatorId` (its creator, which never changes, on types that record it, see [System Properties](system-properties.md)) or `$id` (its id). A system name needs an identifier on the referring side. - **The value** is the referring side: a property of the declaring document type, a dotted path for a nested one, or `"$ownerId"`, the writer. An entry whose value is `"$ownerId"` is a write gate: only an identity whose id equals the referenced side may create or replace the document. A referring value may appear once among the equalities. @@ -194,7 +194,7 @@ WHERE $id = :submittedCharterId -- the value, an id reference AND targetContractId = :targetContractId -- where ``` -At registration both sides must exist and hold the same type of value (for integers, the same stored integer type). Neither side may be an object or a typed array, the referring side may not be the reference property itself, and the referenced side may not be `transient` (no stored document carries it; the referring side may be). An entry through which a [`preallocated`](index-only.md#preallocated) index is keyed needs a referenced property whose every value fits an index key, at most 255 bytes: a string of at most 63 characters or with `maxBytes` at most 255, or a byte array of at most 255 bytes. A condition's referenced side must hold a single value a literal is compared with (an integer, a boolean, a string, an identifier or a byte array, not a float); every literal must read as its kind and be a value the property can hold (one of its `enum` values, of a length or size it takes, inside its `minimum` and `maximum`); an `in` must list values distinct once read (`"cafe"` and `"CAFE"` are one byte array); a `notEqual` compares two values of one kind (the writer an identifier), never the reference property itself, whose value found the document; and an ordering compares an integer with an integer bound, a time or height the declaring type records, an integer bound leaving some value of the property on its side. An entry breaking one of these is refused with `ReferencedDocumentPropertyAgreementInvalidError` (40126), whose referring property names the condition as written. +At registration both sides must exist and hold the same type of value (for integers, the same stored integer type). Neither side may be an object, a typed array pairs only with a typed array (see [Typed arrays](#typed-arrays)), the referring side may not be the reference property itself, and the referenced side may not be `transient` (no stored document carries it; the referring side may be). An entry through which a [`preallocated`](index-only.md#preallocated) index is keyed needs a referenced property whose every value fits an index key, at most 255 bytes: a string of at most 63 characters or with `maxBytes` at most 255, or a byte array of at most 255 bytes. A condition's referenced side must hold a single value a literal is compared with (an integer, a boolean, a string, an identifier or a byte array, not a float); every literal must read as its kind and be a value the property can hold (one of its `enum` values, of a length or size it takes, inside its `minimum` and `maximum`); an `in` must list values distinct once read (`"cafe"` and `"CAFE"` are one byte array); a `notEqual` compares two values of one kind (the writer an identifier), never the reference property itself, whose value found the document; and an ordering compares an integer with an integer bound, a time or height the declaring type records, an integer bound leaving some value of the property on its side. An entry breaking one of these is refused with `ReferencedDocumentPropertyAgreementInvalidError` (40126), whose referring property names the condition as written. #### Conditions @@ -234,6 +234,39 @@ A condition reads absence as a `where` entry does, not as a [`propertyConstraint Through a `moderatedDocument` reference a [`propertyConstraints`](property-constraints.md#values-of-a-referenced-document) rule reads, every replace checks the whole `where`, against the removal record once a moderator removes the document, so registration keeps every entry of such a reference, equality or condition, to `$id`, `$ownerId` and the fields the type lists under `deleteKeepsFields`. +#### Typed arrays + +A typed array on the referenced side is compared as a list, in one of three ways: + +| Entry | Holds when the referenced typed array | +|---|---| +| `"": ""` | has the same elements as the referring typed array, in the same order | +| `{ "contains": b }` | holds an element equal to `b`: a property of the declaring type holding one value of the element kind, or `"$ownerId"` (the writer) for identifier elements | +| `{ "containsAll": b }` | holds every element of `b`, a typed array of the declaring type of the same element kind, in any order | + +- **A list left out is the empty one**, as an indexed typed array without elements is absent. Two lists left out, or one left out and one empty, are equal; every list holds all of an empty `containsAll`; a list left out holds nothing, and a `contains` value the document leaves out is held by no list, as a value left out is `in` no list. +- **Elements compare by value**, exactly as two single values do (see **Values, not keys** above): an identifier equals its 32 bytes however a transition carries them, and an integer is the same whatever width it is carried at. +- **Billed by the element.** Unlike a comparison of single values, an entry comparing typed arrays is billed before it compares them, `where_list_credits_per_element` of the fee schedule (100 credits at protocol version 14) for each element it reads: both lists for an equality and a `containsAll`, the referenced list for a `contains`, a list left out none. A `containsAll` looks each element up in a set of the referenced ones, so its work grows with the lists' length, not its square. +- **At registration** (40126): the referenced side is a stored typed array whose elements are integers, booleans, strings, identifiers or byte arrays, never floats; the referring typed array of an equality or a `containsAll` is stored (not `transient`) and its elements share the referenced elements' value kind; a `contains` value is of the element kind; neither is the reference property itself. `const`, `in`, `notEqual` and the orderings compare single values and refuse a typed array. A pair of typed arrays never keys a [`preallocated`](index-only.md#preallocated) or `summableOffCountIndex` index. +- A `contains` of `"$ownerId"` is a writer gate: only an identity the list holds may create or replace the document, and every replace checks it again. Through a `moderatedDocument` reference its list must be one the type keeps under `deleteKeepsFields`, since once a moderator removes the document the check reads the removal record. + +```json +"postId": { + "type": "identifier", + "refersTo": { + "type": "moderatedDocument", + "documentType": "post", + "where": { + "tags": "tags", + "$ownerId": "postAuthor" + } + }, + "position": 0 +} +``` + +A like copies its post's `tags` (a typed array of at most 4 distinct strings) and its author, so a like can be found by tag without reading the post. `"members": { "contains": "$ownerId" }` lets only a member the post lists write, and `"tags": { "containsAll": "tags" }` lets a document name some of its post's tags in any order. `containsAny`, which the [document query](typed-arrays.md) has, is not a `where` entry. + The keyword `propertyAgreement`, whose pairs were keyed the other way (`{ "": "" }`), was replaced by `where` before protocol version 14 shipped. A declaration still carrying it is refused on every parse, with a message saying to key `where` by the referenced document's property. `"propertyAgreement": { "recipientId": "$ownerId" }` is now `"where": { "$ownerId": "recipientId" }`, and a symmetric pair such as `"$ownerId": "$ownerId"` reads the same. ### `contractRequirements` @@ -308,7 +341,7 @@ A replace checks a reference again only when its outcome could have changed: | `identity`, `token`, `contract` | the value changed | | `contract` with an `owner` requirement, on a type whose documents can be transferred or traded | every replace | | `permanentDocument`, `moderatedDocument` | the value changed, or a referring property a `where` entry reads changed (an equality's referring side, a condition's bound) | -| any document reference with a `where` entry whose value is `"$ownerId"`, a `notEqual` bounded by `"$ownerId"`, or an ordering bounded by `$updatedAt`, `$transferredAt` or one of their heights | every replace | +| any document reference with a `where` entry whose value is `"$ownerId"`, a `notEqual` or `contains` bounded by `"$ownerId"`, or an ordering bounded by `$updatedAt`, `$transferredAt` or one of their heights | every replace | | a document reference a `propertyConstraints` rule reads through ([`referenced`](property-constraints.md#values-of-a-referenced-document)) | every replace: every replace judges every rule, and the rule reads the document as it is at the replace | | `permanentDocument` with `findBy`, `inList` included | also when a property `findBy` reads changed | | `deletableDocument`, by id or with `findBy` | every replace | @@ -364,7 +397,7 @@ A contract's declarations are checked when it is registered, and again for the w | `ReferencedIdentityKeyNotFoundError` | 40123 | Write: the identity has no key with that id, or the identity does not exist. | | `ReferencedIdentityKeyDisabledError` | 40124 | Write: the key is disabled. | | `ReferencedKeyIdPropertyInvalidError` | 40125 | Registration: `keyIdProperty` or `identityProperty` names a property that does not fit, `$creatorId` on a type that records no creator ids, or a stored key id paired with a transient identity. Write: a key id without its identity, or an identity without its key id. | -| `ReferencedDocumentPropertyAgreementInvalidError` | 40126 | Registration: a `where` entry names a missing, transient, object or typed array property, or two properties of different types, or `$creatorId` on a type that does not record it, or keys a preallocated index by a property that can hold more than 255 bytes; or a [condition](#conditions) holds a literal of another kind than its property, compares two values of different kinds, orders something that is no integer, or is bounded by a time or height the declaring type does not record. | +| `ReferencedDocumentPropertyAgreementInvalidError` | 40126 | Registration: a `where` entry names a missing, transient or object property, pairs a typed array with something other than a typed array of the same element kind, compares typed arrays of floats, or two properties of different types, or `$creatorId` on a type that does not record it, or keys a preallocated index by a property that can hold more than 255 bytes; or a [condition](#conditions) holds a literal of another kind than its property, compares two values of different kinds, orders something that is no integer, or is bounded by a time or height the declaring type does not record. | | `ReferencedDocumentPropertyMismatchError` | 40127 | Write: the document found does not meet a `where` entry; for a condition, the referring property is the condition as written. | | `ReferencedDocumentTypeNotDeletableError` | 40131 | Registration: a `deletableDocument` reference names a type whose documents can never disappear. | | `ReferencedContractRequirementNotMetError` | 40135 | Write: the referenced contract exists but does not meet a `contractRequirements` entry. | diff --git a/book/src/contract-keywords/typed-arrays.md b/book/src/contract-keywords/typed-arrays.md index c1e2371c5dc..b3370dce787 100644 --- a/book/src/contract-keywords/typed-arrays.md +++ b/book/src/contract-keywords/typed-arrays.md @@ -124,7 +124,7 @@ Where a typed array cannot be used: - in an index of an index-only type, or as an index-only type's terminal or entry payload: an index-only type's entries are its rows, one per document (10231, or `InvalidIndexPropertyTypeError` 10206 in a terminal); - inside an object, when an index names it: an indexed typed array is a top-level property (10231); -- on either side of a `where` entry in a reference; +- in a `where` entry of a reference, other than as a list: two typed arrays of one element kind may be compared as lists (the same elements in the same order), and a referenced typed array may be asked to `contains` a value or `containsAll` of another typed array's elements; a float element is refused, and a typed array is equal to no single value (see [Typed arrays](refers-to.md#typed-arrays) in References); - as an operand of a [propertyConstraints](property-constraints.md) rule, other than in a `present` or `absent` test; - with [`encryptedFor`](encrypted-for.md), which only a byte array takes. diff --git a/book/src/fees/overview.md b/book/src/fees/overview.md index 19bece18584..8cacc4ffb25 100644 --- a/book/src/fees/overview.md +++ b/book/src/fees/overview.md @@ -549,7 +549,10 @@ the fees of documents whose type declares a [`ttl`](../data-model/document-ttl.m and the budget of a [`chooseFirst`](../contract-keywords/refers-to-lookup.md#choosing-the-first-commitment-choosefirst) competitor scan (`choose_first_credits_per_candidate`, 100,000 credits a candidate, times `SystemLimits::max_choose_first_candidates`), which check tx -reserves on top of a reveal's estimated fee. Platform states store only +reserves on top of a reveal's estimated fee, and the processing fee of a +[`where` entry comparing typed arrays](../contract-keywords/refers-to.md#typed-arrays) +(`where_list_credits_per_element`, 100 credits for each list element it reads, +billed before it compares them). Platform states store only `fee_version_number`, so neither group touches a stored format. Fee versions are stored in the `FEE_VERSIONS` array and looked up by number. The diff --git a/packages/js-evo-sdk/README.md b/packages/js-evo-sdk/README.md index c4be0efab3d..4396a9435d0 100644 --- a/packages/js-evo-sdk/README.md +++ b/packages/js-evo-sdk/README.md @@ -244,12 +244,18 @@ for (const ref of contract.documentTypeReferences('note')) { // the writer's own `$ownerId`: a write gate such as // `{ '$ownerId': '$ownerId' }` lets only the referenced document's current // owner create or replace the referring document. + // Between two typed arrays, e.g. `where: { tags: 'tags' }`, the equality + // compares lists: the same elements in the same order, a list left out + // being the empty one. // A value may instead be a condition object with one key, as the schema - // writes it: `{ const: v }`, `{ notEqual: b }`, `{ in: [v1, v2] }`, or an + // writes it: `{ const: v }`, `{ notEqual: b }`, `{ in: [v1, v2] }`, an // ordering such as `{ lessThanOrEqual: 'amount' }` or // `{ greaterThan: '$updatedAt' }` (a bound is a referring property, // `'$ownerId'`, a time or height, or `{ const: v }`; an integer past - // Number.MAX_SAFE_INTEGER is a bigint). Tell the two apart by type: + // Number.MAX_SAFE_INTEGER is a bigint), or, of a typed array, + // `{ contains: 'tag' }` (a referring property or `'$ownerId'` the list must + // hold) or `{ containsAll: 'tags' }` (a referring typed array whose every + // element the list must hold). Tell the two apart by type: for (const [referenced, value] of Object.entries(ref.where ?? {})) { if (typeof value === 'string') { // an equality with the referring property `value` diff --git a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json index 3efc17e99b5..a36be35a067 100644 --- a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json +++ b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json @@ -514,7 +514,7 @@ "minItems": 2 }, "referenceWhereCondition": { - "description": "A refersTo where entry that is not an equality with a referring value: an object with one key. const, a literal the referenced side must equal (an integer or true or false for such a property, a string for a string property, a base58 identifier for an identifier, $ownerId, $creatorId or $id, hex for a byte array); a referenced side the document leaves out equals none. notEqual, what it must differ from: a property of the referring document type, \"$ownerId\" (the writer, checked on every replace) or a { const }; the negation of an equality, so two absent sides are equal. in, two to thirty distinct literals of one kind, one of which it must equal; an absent side is in none. greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer property the referenced side must compare with this way: bounded by an integer property of the referring document type, an integer { const }, or a time or height the referring type lists in required ($createdAt as written by the create; $updatedAt and $transferredAt, and their heights, change without a write to the referring document, so an entry bounded by one is checked on every replace); it holds only when both sides have a value. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). Available from protocol version 14", + "description": "A refersTo where entry that is not an equality with a referring value: an object with one key. const, a literal the referenced side must equal (an integer or true or false for such a property, a string for a string property, a base58 identifier for an identifier, $ownerId, $creatorId or $id, hex for a byte array); a referenced side the document leaves out equals none. notEqual, what it must differ from: a property of the referring document type, \"$ownerId\" (the writer, checked on every replace) or a { const }; the negation of an equality, so two absent sides are equal. in, two to thirty distinct literals of one kind, one of which it must equal; an absent side is in none. greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer property the referenced side must compare with this way: bounded by an integer property of the referring document type, an integer { const }, or a time or height the referring type lists in required ($createdAt as written by the create; $updatedAt and $transferredAt, and their heights, change without a write to the referring document, so an entry bounded by one is checked on every replace); it holds only when both sides have a value. contains, of a typed array, a value it must hold an element equal to: a property of the referring document type of the element kind, or \"$ownerId\" (the writer, among identifier elements, checked on every replace); a list the document leaves out holds nothing, and a value the write leaves out is held by no list. containsAll, of a typed array, a typed array of the referring document type of the same element kind whose every element it must hold, in any order; every list holds all of an empty one, and a list left out is empty. Elements compare by value, never floats. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). Available from protocol version 14", "type": "object", "minProperties": 1, "maxProperties": 1, @@ -573,6 +573,18 @@ }, "lessThanOrEqual": { "$ref": "#/$defs/referenceWhereOrderingBound" + }, + "contains": { + "type": "string", + "minLength": 1, + "maxLength": 256, + "pattern": "^(\\$ownerId|[a-zA-Z0-9_]{1,64}(\\.[a-zA-Z0-9_]{1,64})*)$" + }, + "containsAll": { + "type": "string", + "minLength": 1, + "maxLength": 256, + "pattern": "^[a-zA-Z0-9_]{1,64}(\\.[a-zA-Z0-9_]{1,64})*$" } }, "additionalProperties": false @@ -933,7 +945,7 @@ } }, "where": { - "description": "permanentDocument, moderatedDocument and deletableDocument references only: what the referenced document must hold once found, checked by consensus when the referring document is written. Each entry maps a property of the referenced document type, or one of its $ownerId, $creatorId and $id system identifiers, to the referring document's value it must equal: a schema property of the declaring document type, or \"$ownerId\", the writer, which turns the entry into a write gate: only an identity whose id equals the referenced side may create or replace the document. A referring value may be compared once. $creatorId needs a referenced document type that records creator ids (transferable or tradeable types of a format-1 contract), and $ownerId, $creatorId or $id an identifier on the referring side. Both sides must exist and share one value kind, validated at contract registration. $ownerId follows the referenced document through transfers while $creatorId and $id never change; each is checked when the referring document is written, not when the referenced document later moves. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). An entry may instead hold an object with one key (see referenceWhereCondition): const, a literal the referenced side must equal; notEqual, a referring property, \"$ownerId\" or a const it must differ from; in, two or more literals it must be one of; or greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer bound it must compare with. The one-comparison-per-referring-value rule holds for the equalities only. Beside a findBy function the declaration is judged when the document is created only, so its entries gate the create alone, and every referring value other than $ownerId must be transient or fixed once written (not listed under immutable with a condition). where never finds the document: \".\" and functions belong to findBy. Available from protocol version 14.", + "description": "permanentDocument, moderatedDocument and deletableDocument references only: what the referenced document must hold once found, checked by consensus when the referring document is written. Each entry maps a property of the referenced document type, or one of its $ownerId, $creatorId and $id system identifiers, to the referring document's value it must equal: a schema property of the declaring document type, or \"$ownerId\", the writer, which turns the entry into a write gate: only an identity whose id equals the referenced side may create or replace the document. A referring value may be compared once. $creatorId needs a referenced document type that records creator ids (transferable or tradeable types of a format-1 contract), and $ownerId, $creatorId or $id an identifier on the referring side. Both sides must exist and share one value kind, validated at contract registration; two typed arrays of one element kind (not floats, both stored) are equal as lists, the same elements in the same order compared by value, a list the document leaves out the empty one, and a typed array is equal to nothing else. $ownerId follows the referenced document through transfers while $creatorId and $id never change; each is checked when the referring document is written, not when the referenced document later moves. A document found that fails an entry refuses the write with ReferencedDocumentPropertyMismatchError (40127). An entry may instead hold an object with one key (see referenceWhereCondition): const, a literal the referenced side must equal; notEqual, a referring property, \"$ownerId\" or a const it must differ from; in, two or more literals it must be one of; greaterThan, greaterThanOrEqual, lessThan or lessThanOrEqual, an integer bound it must compare with; or, of a typed array, contains, a referring value or \"$ownerId\" it must hold, or containsAll, a referring typed array whose every element it must hold. The one-comparison-per-referring-value rule holds for the equalities only. Beside a findBy function the declaration is judged when the document is created only, so its entries gate the create alone, and every referring value other than $ownerId must be transient or fixed once written (not listed under immutable with a condition). where never finds the document: \".\" and functions belong to findBy. Available from protocol version 14.", "type": "object", "minProperties": 1, "maxProperties": 10, diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs index 1c02c54ce13..24dee140bb2 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs @@ -7,7 +7,7 @@ use crate::data_contract::document_type::class_methods::consensus_or_protocol_da use crate::data_contract::document_type::class_methods::parse_typed_array::parse_typed_array; use crate::data_contract::document_type::methods::DocumentTypeBasicMethods; use crate::data_contract::document_type::property::reference_where::{ - WHERE_CONST, WHERE_IN, WHERE_NOT_EQUAL, + WHERE_CONST, WHERE_CONTAINS, WHERE_CONTAINS_ALL, WHERE_IN, WHERE_NOT_EQUAL, }; use crate::data_contract::document_type::property_constraints::{ either_version_path, parse_delete_constraints, parse_property_constraints, referenced_path, @@ -35,8 +35,8 @@ use crate::data_contract::document_type::{ IdentityKeyReferenceRequirements, KeyIdReference, KeyReferenceIdentityProperty, ListElementReference, LookupChooseFirst, LookupHashKey, LookupKeySource, PropertyReference, ReferenceCombinator, ReferenceOperands, SystemFunction, WhereBoundRead, WhereCondition, - WhereLiteral, WhereLiteralKind, WhereNotEqualBound, WhereOrdering, WhereOrderingBound, - COMBINABLE_REFERENCE_TARGET_TYPES, + WhereContainsBound, WhereLiteral, WhereLiteralKind, WhereNotEqualBound, WhereOrdering, + WhereOrderingBound, COMBINABLE_REFERENCE_TARGET_TYPES, }; use crate::data_contract::errors::DataContractError; use crate::data_contract::{TokenConfiguration, TokenContractPosition}; @@ -1291,7 +1291,13 @@ fn parse_reference_target( // `moderatorAbilities.deleteKeepsFields`, are left out where the document // held none, and the type may be another contract's): a gate on any other // property could refuse every later replace, for good on a property that - // can not be repointed or cleared, so it may compare only those two + // can not be repointed or cleared, so an equality or a `notEqual` with the + // writer may compare only those two. A `contains` of the writer is the one + // gate admitted here on another property, a typed array, which no system + // identifier is: a list the document held none of is read as empty, as + // against the document in state, so the gate refuses after a removal what + // it refused before; registration, with the referenced type in hand, keeps + // the list to one every removal record holds "moderatedDocument" => { let (contract_id, document_type_name) = parse_document_reference_type(refers_to_map, reference_type)?; @@ -1482,12 +1488,15 @@ type ParsedWhere = (BTreeMap, BTreeMap); /// other entry is an object with one key, a [`WhereCondition`]: `const`, a /// literal it must equal; `notEqual`, a referring property, `"$ownerId"` or a /// `const` it must differ from; `in`, two or more distinct literals of one -/// kind it must be one of; or an ordering (`greaterThan`, `greaterThanOrEqual`, +/// kind it must be one of; an ordering (`greaterThan`, `greaterThanOrEqual`, /// `lessThan`, `lessThanOrEqual`) with a referring property, an integer /// `const` or a time or height of the referring document as its bound, never -/// of an identifier. `where` never finds the document and reads neither `"."` -/// nor a function, which belong to `findBy`. What the names resolve to, and -/// whether a literal fits the property it is compared with, is checked at +/// of an identifier; or, of a typed array, `contains`, a referring property or +/// `"$ownerId"` it must hold, or `containsAll`, a referring typed array whose +/// every element it must hold. `where` never finds the document and reads +/// neither `"."` nor a function, which belong to `findBy`. What the names +/// resolve to, whether a literal fits the property it is compared with, and +/// whether an equality or a condition compares lists, is checked at /// registration. fn parse_where( refers_to_map: &BTreeMap, @@ -1532,8 +1541,8 @@ fn parse_where( let referring_property = value.as_text().ok_or_else(|| { DataContractError::InvalidContractStructure( "refersTo where values must name a property of the referring document type or \ - \"$ownerId\", the writer, or be an object holding const, notEqual, in or an \ - ordering" + \"$ownerId\", the writer, or be an object holding const, notEqual, in, an \ + ordering, contains or containsAll" .to_string(), ) })?; @@ -1585,9 +1594,11 @@ fn parse_where_referring_property<'a>( } /// The [`WhereCondition`] a `where` entry keyed by `referenced_property` -/// declares with `value`, an object with one key: `const`, `notEqual`, `in` or -/// an ordering. Any other key is refused: a function computes a key that -/// finds the document, which belongs to `findBy`. +/// declares with `value`, an object with one key: `const`, `notEqual`, `in`, +/// an ordering, `contains` or `containsAll`. Any other key is refused: a +/// function computes a key that finds the document, which belongs to `findBy`. +/// `contains` and `containsAll` read a list, which `$ownerId`, `$creatorId` +/// and `$id` are not. fn parse_where_condition( referenced_property: &str, value: &Value, @@ -1613,7 +1624,7 @@ fn parse_where_condition( let [(Value::Text(key), body)] = entries.as_slice() else { return Err(invalid( "must be an object with one key: const, notEqual, in, greaterThan, \ - greaterThanOrEqual, lessThan or lessThanOrEqual" + greaterThanOrEqual, lessThan, lessThanOrEqual, contains or containsAll" .to_string(), )); }; @@ -1674,6 +1685,46 @@ fn parse_where_condition( } Ok(WhereCondition::In(literals)) } + list_key @ (WHERE_CONTAINS | WHERE_CONTAINS_ALL) => { + if identifier_side { + return Err(invalid(format!( + "reads {list_key} of an identifier, which is one value: {list_key} reads a \ + typed array" + ))); + } + let Value::Text(path) = body else { + return Err(invalid(format!( + "{list_key} must be a property path of the referring document type{}", + if list_key == WHERE_CONTAINS { + ", or \"$ownerId\"" + } else { + "" + } + ))); + }; + if list_key == WHERE_CONTAINS { + let bound = if path == OWNER_ID { + WhereContainsBound::Writer + } else { + WhereContainsBound::Property( + parse_where_referring_property(referenced_property, path)?.to_string(), + ) + }; + return Ok(WhereCondition::Contains(bound)); + } + // The writer is one identifier, not a list of them + if path == OWNER_ID { + return Err(invalid( + "containsAll reads $ownerId, the writer, which is one identifier: \ + containsAll reads a typed array of the referring document type, and \ + contains the writer" + .to_string(), + )); + } + Ok(WhereCondition::ContainsAll( + parse_where_referring_property(referenced_property, path)?.to_string(), + )) + } ordering_key => { let Some(ordering) = WhereOrdering::ALL .into_iter() @@ -1681,8 +1732,8 @@ fn parse_where_condition( else { return Err(invalid(format!( "names \"{ordering_key}\", which is not const, notEqual, in, greaterThan, \ - greaterThanOrEqual, lessThan or lessThanOrEqual: a function computes a key \ - that finds the document, which belongs in findBy" + greaterThanOrEqual, lessThan, lessThanOrEqual, contains or containsAll: a \ + function computes a key that finds the document, which belongs in findBy" ))); }; if identifier_side { diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs index 9f048bcca1a..4e26f25b222 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs @@ -2047,6 +2047,8 @@ mod typed_array_reference_tests; mod typed_array_test_helpers; #[cfg(all(test, feature = "validation", feature = "random-documents"))] mod typed_array_tests; +#[cfg(all(test, feature = "validation"))] +mod typed_array_where_tests; #[cfg(test)] mod tests { diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_where_tests.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_where_tests.rs new file mode 100644 index 00000000000..b64b8724e4c --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/typed_array_where_tests.rs @@ -0,0 +1,247 @@ +//! `where` entries over typed arrays (protocol version 14): an equality +//! between two typed arrays, kept with the other equalities, and the +//! `contains` and `containsAll` conditions. The parse reads their shape; what +//! the names resolve to, and whether the two sides compare as lists, is +//! checked at registration. + +use super::reference_test_helpers::{ + assert_refused_by_the_parser, contract, contract_on, identifier, CONTRACT_ID, +}; +use super::typed_array_test_helpers::expect_json_schema_error; +use crate::data_contract::accessors::v0::DataContractV0Getters; +use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; +use crate::data_contract::document_type::{ + DocumentPropertyType, DocumentReferenceDeclaration, WhereCondition, WhereContainsBound, +}; +use crate::data_contract::DataContract; +use platform_value::string_encoding::Encoding; +use platform_value::Identifier; +use platform_version::version::PlatformVersion; +use serde_json::json; +use std::collections::BTreeMap; + +/// A post's tags, as Yappr declares them: up to 4 distinct strings, never +/// changed once written. +fn tags(position: u32) -> serde_json::Value { + json!({ + "type": "array", + "items": { "type": "string", "minLength": 1, "maxLength": 63 }, + "maxItems": 4, + "uniqueItems": true, + "position": position + }) +} + +/// A contract with a `post` holding `tags` and `members`, and a `like` of a +/// post whose reference declares `where`. +fn contract_with_where(where_value: serde_json::Value) -> serde_json::Value { + let mut post_id = identifier(0); + post_id["refersTo"] = json!({ + "type": "permanentDocument", + "documentType": "post", + "where": where_value + }); + json!({ + "$formatVersion": "1", + "id": Identifier::from(CONTRACT_ID).to_string(Encoding::Base58), + "ownerId": Identifier::from([8; 32]).to_string(Encoding::Base58), + "version": 1, + "documentSchemas": { + "post": { + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "tags": tags(0), + "members": { + "type": "array", + "items": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier" + }, + "maxItems": 4, + "position": 1 + } + }, + "additionalProperties": false + }, + "like": { + "type": "object", + "documentsMutable": false, + "properties": { + "postId": post_id, + "tags": tags(1), + "tag": { "type": "string", "minLength": 1, "maxLength": 63, "position": 2 } + }, + "required": ["postId"], + "additionalProperties": false + } + } + }) +} + +/// The `where` of the like's reference, as registration parses it. +fn parsed_where( + where_value: serde_json::Value, +) -> (BTreeMap, BTreeMap) { + where_of(&contract(contract_with_where(where_value)).expect("the contract parses")) +} + +/// The `where` of the like's reference in `parsed`. +fn where_of(parsed: &DataContract) -> (BTreeMap, BTreeMap) { + let like = parsed + .document_type_for_name("like") + .expect("the like type"); + let Some(DocumentPropertyType::IdentifierWithReference(target)) = like + .flattened_properties() + .get("postId") + .map(|property| &property.property_type) + else { + panic!("postId carries a reference"); + }; + let Some(DocumentReferenceDeclaration { + property_agreement, + where_conditions, + .. + }) = target.as_document_reference() + else { + panic!("postId refers to a post by id"); + }; + (property_agreement.clone(), where_conditions.clone()) +} + +#[test] +fn should_parse_an_equality_between_typed_arrays_as_an_equality() { + let (property_agreement, where_conditions) = parsed_where(json!({ "tags": "tags" })); + assert_eq!( + property_agreement, + BTreeMap::from([("tags".to_string(), "tags".to_string())]) + ); + assert!(where_conditions.is_empty()); +} + +#[test] +fn should_parse_contains_and_contains_all() { + let (property_agreement, where_conditions) = parsed_where(json!({ + "members": { "contains": "$ownerId" }, + "tags": { "containsAll": "tags" } + })); + assert!(property_agreement.is_empty()); + assert_eq!( + where_conditions, + BTreeMap::from([ + ( + "members".to_string(), + WhereCondition::Contains(WhereContainsBound::Writer) + ), + ( + "tags".to_string(), + WhereCondition::ContainsAll("tags".to_string()) + ), + ]) + ); + + let (_, where_conditions) = parsed_where(json!({ "tags": { "contains": "tag" } })); + assert_eq!( + where_conditions.get("tags"), + Some(&WhereCondition::Contains(WhereContainsBound::Property( + "tag".to_string() + ))) + ); +} + +/// A `contains` and a `containsAll` read a list, which the referenced +/// document's system identifiers are not; the writer is one identifier a list +/// may hold, never a list itself; and the bound is a property path, never a +/// literal, `"."` or another system property. The meta-schema refuses most of +/// these too; the parser refuses each on its own. +#[test] +fn should_refuse_a_malformed_contains_or_contains_all() { + for (where_value, fragment) in [ + ( + json!({ "$ownerId": { "contains": "tag" } }), + "reads contains of an identifier", + ), + ( + json!({ "$id": { "containsAll": "tags" } }), + "reads containsAll of an identifier", + ), + ( + json!({ "tags": { "containsAll": "$ownerId" } }), + "containsAll reads $ownerId", + ), + ( + json!({ "tags": { "contains": { "const": "news" } } }), + "contains must be a property path of the referring document type, or", + ), + ( + json!({ "tags": { "containsAll": ["tags"] } }), + "containsAll must be a property path of the referring document type", + ), + (json!({ "tags": { "contains": "." } }), "reads \".\""), + ( + json!({ "tags": { "containsAll": "$createdAt" } }), + "its $ownerId", + ), + ( + json!({ "tags": { "containsAny": "tags" } }), + "contains or containsAll", + ), + ] { + assert_refused_by_the_parser(contract_with_where(where_value), fragment); + } +} + +/// The meta-schema admits both keys with a property path, and `contains` with +/// `"$ownerId"`, before the parser sees them. +#[test] +fn should_admit_the_list_keys_in_the_meta_schema() { + for where_value in [ + json!({ "tags": { "contains": "tag" } }), + json!({ "members": { "contains": "$ownerId" } }), + json!({ "tags": { "containsAll": "tags" } }), + ] { + let parsed = contract(contract_with_where(where_value.clone())); + assert!(parsed.is_ok(), "{where_value}: {parsed:?}"); + } + let error = expect_json_schema_error(contract(contract_with_where( + json!({ "tags": { "contains": 5 } }), + ))); + // The entry matches none of an equality's path, the writer and a condition + assert_eq!( + error.instance_path(), + "/properties/postId/refersTo/where/tags", + "the meta-schema refuses the entry: {error:?}" + ); +} + +/// A stored contract read back parses the same entries, without the +/// meta-schema. +#[test] +fn should_read_back_a_stored_list_where() { + let stored = contract_on( + contract_with_where(json!({ + "tags": "tags", + "members": { "contains": "$ownerId" } + })), + false, + PlatformVersion::latest(), + ) + .expect("the stored contract reads"); + assert_eq!(stored.id(), Identifier::from(CONTRACT_ID)); + let (property_agreement, where_conditions) = where_of(&stored); + assert_eq!( + property_agreement, + BTreeMap::from([("tags".to_string(), "tags".to_string())]) + ); + assert_eq!( + where_conditions, + BTreeMap::from([( + "members".to_string(), + WhereCondition::Contains(WhereContainsBound::Writer) + )]) + ); +} diff --git a/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs b/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs index a39611f3227..e0af44931ee 100644 --- a/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs +++ b/packages/rs-dpp/src/data_contract/document_type/index/preallocation.rs @@ -36,8 +36,8 @@ use crate::data_contract::document_type::accessors::{ DocumentTypeV0Getters, DocumentTypeV2Getters, }; use crate::data_contract::document_type::property::{ - is_path_listed, DocumentProperty, DocumentPropertyType, DocumentReferenceDeclaration, - DocumentReferenceKind, + is_path_listed, typed_array_property, DocumentProperty, DocumentPropertyType, + DocumentReferenceDeclaration, DocumentReferenceKind, }; use crate::data_contract::document_type::{DocumentTypeRef, Index}; use crate::document::property_names::{ID, OWNER_ID}; @@ -249,6 +249,14 @@ impl Index { // it: that agreement is a write gate, and deriving an // owner-prefixed path from it is left for later. None + } else if typed_array_property(flattened_properties, &index_property.name) + .is_some() + { + // A list is no tree key: a `where` equality between two + // typed arrays binds no index property. Inert while an + // indexOnly type, the only one declaring `preallocated`, + // indexes no typed array + None } else { reference.property_agreement.get(&index_property.name).map( |referenced| { @@ -377,6 +385,52 @@ mod tests { } } + fn tags_property() -> DocumentProperty { + use crate::data_contract::document_type::array::TypedArrayProperty; + use crate::data_contract::document_type::property::StringPropertySizes; + DocumentProperty { + property_type: DocumentPropertyType::TypedArray(TypedArrayProperty { + item_type: Box::new(DocumentPropertyType::String(StringPropertySizes { + min_length: Some(1), + max_length: Some(63), + max_bytes: None, + })), + item_constraints: Default::default(), + min_items: None, + max_items: 4, + unique_items: true, + }), + ..string_property() + } + } + + /// A `where` equality between two typed arrays compares lists, and a list + /// is no tree key: no preallocated path and no `summableOffCountIndex` + /// group is fixed through it. + #[test] + fn should_bind_no_index_property_through_a_pair_of_typed_arrays() { + let own_contract_id = Identifier::from([1u8; 32]); + let mut properties = IndexMap::new(); + properties.insert("tags".to_string(), tags_property()); + properties.insert( + "postId".to_string(), + identifier_reference_property("post", None, &[("tags", "tags")]), + ); + + let index = index_on(&["tags", "postId"]); + assert!(index + .preallocation_bindings(&properties, own_contract_id) + .is_empty()); + + let source = index_on(&["postId"]); + let mut counts = index_on(&["postId", "tags"]); + counts.summable_off_count_index = Some(source.name.clone()); + assert_eq!( + counts.count_index_derivations(&source, &properties, own_contract_id), + Err("tags") + ); + } + #[test] fn binds_agreement_prefix_and_referring_property() { let own_contract_id = Identifier::from([1u8; 32]); diff --git a/packages/rs-dpp/src/data_contract/document_type/index/summable_off_count_index.rs b/packages/rs-dpp/src/data_contract/document_type/index/summable_off_count_index.rs index f9a04b7f3af..36272de8430 100644 --- a/packages/rs-dpp/src/data_contract/document_type/index/summable_off_count_index.rs +++ b/packages/rs-dpp/src/data_contract/document_type/index/summable_off_count_index.rs @@ -18,7 +18,9 @@ //! referenced document is the one the source property's value names. use super::preallocation::same_contract_record_reference; -use crate::data_contract::document_type::property::{DocumentProperty, DocumentReferenceKind}; +use crate::data_contract::document_type::property::{ + typed_array_property, DocumentProperty, DocumentReferenceKind, +}; use crate::data_contract::document_type::Index; use indexmap::IndexMap; use platform_value::Identifier; @@ -70,7 +72,13 @@ impl Index { { continue; } - if property.starts_with('$') { + // A list fixes no group: a `where` equality between two typed + // arrays compares lists, which no tree key holds. Inert while an + // indexOnly type, the only one declaring a summableOffCountIndex + // index, indexes no typed array + if property.starts_with('$') + || typed_array_property(flattened_properties, property).is_some() + { return Err(property); } let bound_before = derivations.len(); diff --git a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs index b12bfbc1dd4..820c66d51dc 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs @@ -65,8 +65,9 @@ pub use reference_lookup::{ ChooseFirstOrder, DocumentReferenceLookup, LookupChooseFirst, LookupKeySource, }; pub use reference_where::{ - WhereBoundRead, WhereCondition, WhereLiteral, WhereLiteralKind, WhereNotEqualBound, - WhereOrdering, WhereOrderingBound, + where_lists_elements_read, where_lists_equal, WhereBoundRead, WhereCondition, + WhereContainsBound, WhereLiteral, WhereLiteralKind, WhereNotEqualBound, WhereOrdering, + WhereOrderingBound, }; #[cfg(test)] @@ -823,7 +824,9 @@ pub enum DocumentPropertyReferenceTarget { /// are validated at contract registration: both sides must exist /// and share one value kind (an identifier on the referring side /// for the system names), and `$creatorId` needs a referenced - /// document type that records creator ids at all. + /// document type that records creator ids at all. Two typed arrays + /// of one element kind may be paired, and are then equal as lists + /// ([`where_lists_equal`]); a typed array pairs with nothing else. #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] property_agreement: BTreeMap, /// The other `where` entries, by referenced property: a literal the diff --git a/packages/rs-dpp/src/data_contract/document_type/property/reference_where.rs b/packages/rs-dpp/src/data_contract/document_type/property/reference_where.rs index a24f6618d9f..7064793b5d9 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/reference_where.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/reference_where.rs @@ -10,7 +10,9 @@ //! "status": { "in": ["open", "extended"] }, //! "endsAt": { "greaterThan": "$createdAt" }, //! "reservePrice": { "lessThanOrEqual": "amount" }, -//! "$creatorId": { "notEqual": "$ownerId" } +//! "$creatorId": { "notEqual": "$ownerId" }, +//! "voters": { "contains": "$ownerId" }, +//! "tags": { "containsAll": "tags" } //! } //! ``` //! @@ -23,6 +25,12 @@ //! referenced property it is compared with ([`WhereLiteralKind`]), which the //! check reads beside the referenced document, since the type may be another //! contract's. What the names resolve to is checked at registration. +//! +//! A typed array is compared as a list: an equality between two typed arrays +//! holds for the same elements in the same order ([`where_lists_equal`]), +//! `contains` for a list holding a value of the referring document or the +//! writer, and `containsAll` for a list holding every element of a typed array +//! of the referring document. A list the document leaves out is the empty one. use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; use crate::data_contract::document_type::property::{ @@ -37,6 +45,8 @@ use bincode::{Decode, DecodeUntrusted, Encode}; use platform_value::string_encoding::Encoding; use platform_value::{Identifier, Value}; use serde::Serialize; +use std::borrow::Cow; +use std::collections::BTreeSet; use std::fmt; /// The key of a `where` entry's literal: `{ "const": ... }`. @@ -45,6 +55,12 @@ pub const WHERE_CONST: &str = "const"; pub const WHERE_NOT_EQUAL: &str = "notEqual"; /// The key of a `where` entry listing the literals the referenced side may be. pub const WHERE_IN: &str = "in"; +/// The key of a `where` entry naming a value the referenced typed array must +/// hold. +pub const WHERE_CONTAINS: &str = "contains"; +/// The key of a `where` entry naming a typed array whose every element the +/// referenced typed array must hold. +pub const WHERE_CONTAINS_ALL: &str = "containsAll"; /// A literal of a `where` entry, `{ "const": ... }` or a value an `in` lists, /// as the declaration writes it. A string is a string, a base58 identifier or a @@ -258,8 +274,32 @@ impl fmt::Display for WhereOrderingBound { } } -/// A value of the referring write a `notEqual` or an ordering compares the -/// referenced side with ([`WhereCondition::bound_read`]). +/// What the referenced typed array of a `contains` must hold an element equal +/// to. +// Embedded in `DocumentPropertyReferenceTarget`, which consensus errors carry. +// @append_only +#[derive(Debug, PartialEq, Eq, Clone, Serialize, Encode, Decode, DecodeUntrusted)] +#[serde(rename_all = "camelCase")] +pub enum WhereContainsBound { + /// A property of the referring document holding one value of the element + /// kind, by its dotted path. + Property(String), + /// `"$ownerId"`: the writer, among identifier elements. + Writer, +} + +impl fmt::Display for WhereContainsBound { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + WhereContainsBound::Property(path) => write!(f, "\"{path}\""), + WhereContainsBound::Writer => write!(f, "\"{OWNER_ID}\""), + } + } +} + +/// A value of the referring write a `notEqual`, an ordering, a `contains` or a +/// `containsAll` compares the referenced side with +/// ([`WhereCondition::bound_read`]). #[derive(Debug, PartialEq, Eq, Clone, Copy)] pub enum WhereBoundRead<'a> { /// The referring document's property at this dotted path. @@ -362,6 +402,17 @@ pub enum WhereCondition { ordering: WhereOrdering, bound: WhereOrderingBound, }, + /// `{ "contains": bound }`: a typed array holding an element equal to the + /// bound, a property of the referring document of the element kind or the + /// writer, as [`where_lists_equal`] compares two elements. A list the + /// document leaves out holds nothing, and a bound the write leaves out is + /// held by no list. + Contains(WhereContainsBound), + /// `{ "containsAll": path }`: a typed array holding every element of the + /// typed array of the referring document at `path`, each as + /// [`where_lists_equal`] compares two elements, in any order. Every list + /// holds all of an empty one, and a list the write leaves out is empty. + ContainsAll(String), } impl WhereCondition { @@ -372,9 +423,20 @@ impl WhereCondition { WhereCondition::NotEqual(_) => WHERE_NOT_EQUAL, WhereCondition::In(_) => WHERE_IN, WhereCondition::Ordering { ordering, .. } => ordering.wire_name(), + WhereCondition::Contains(_) => WHERE_CONTAINS, + WhereCondition::ContainsAll(_) => WHERE_CONTAINS_ALL, } } + /// Whether the condition compares the referenced side as a list, a typed + /// array (`contains` and `containsAll`), rather than as one value. + pub fn reads_a_list(&self) -> bool { + matches!( + self, + WhereCondition::Contains(_) | WhereCondition::ContainsAll(_) + ) + } + /// The value of the referring write the condition compares with, `None` /// for one that carries its own: a literal or a list of them. pub fn bound_read(&self) -> Option> { @@ -383,8 +445,11 @@ impl WhereCondition { | WhereCondition::Ordering { bound: WhereOrderingBound::Property(path), .. - } => Some(WhereBoundRead::Property(path)), - WhereCondition::NotEqual(WhereNotEqualBound::Writer) => Some(WhereBoundRead::Writer), + } + | WhereCondition::Contains(WhereContainsBound::Property(path)) + | WhereCondition::ContainsAll(path) => Some(WhereBoundRead::Property(path)), + WhereCondition::NotEqual(WhereNotEqualBound::Writer) + | WhereCondition::Contains(WhereContainsBound::Writer) => Some(WhereBoundRead::Writer), WhereCondition::Ordering { bound: WhereOrderingBound::System(property), .. @@ -417,13 +482,28 @@ impl WhereCondition { .is_some_and(WhereBoundRead::changes_without_a_write) } + /// How many list elements [`Self::holds`] reads for `referenced` and + /// `bound`: the referenced list's for a `contains`, and the bound list's + /// too for a `containsAll`; none for a condition comparing one value. What + /// document reference validation bills before checking the condition. + pub fn list_elements_read(&self, referenced: Option<&Value>, bound: Option<&Value>) -> usize { + match self { + WhereCondition::Contains(_) => where_list_len(referenced), + WhereCondition::ContainsAll(_) => { + where_list_len(referenced).saturating_add(where_list_len(bound)) + } + _ => 0, + } + } + /// Whether the condition holds for `referenced`, the referenced side's /// value (`None` when the document leaves it out) of `kind` (`None` for a /// property no literal reads as, which registration refuses a literal - /// for), and `bound`, the value [`Self::bound_read`] names (`None` when the - /// write leaves it out, and for a literal or a list, which carry their - /// own): a time or height reads as an integer, the writer as an - /// identifier. + /// for, a typed array among them), and `bound`, the value + /// [`Self::bound_read`] names (`None` when the write leaves it out, and for + /// a literal or a list, which carry their own): a time or height reads as + /// an integer, the writer as an identifier. A `contains` or a `containsAll` + /// reads `referenced` as a list and needs no `kind`. pub fn holds( &self, referenced: Option<&Value>, @@ -471,15 +551,167 @@ impl WhereCondition { (Some(referenced), Some(bound)) if ordering.holds(referenced, bound) ) } + // An absent bound is no element of any list + WhereCondition::Contains(_) => match (where_list_elements(referenced), bound) { + (Some(elements), Some(bound)) => elements + .iter() + .any(|element| element.same_scalar_data(bound)), + _ => false, + }, + WhereCondition::ContainsAll(_) => { + match (where_list_elements(referenced), where_list_elements(bound)) { + (Some(held), Some(wanted)) => list_holds_all(held, wanted), + _ => false, + } + } } } } +/// Whether `held` has, for every element of `wanted`, an element equal to it +/// as `Value::same_scalar_data` compares two single values, each wanted one +/// looked up in a set of the held ones, so two lists of up to +/// `max_typed_array_items` elements each are compared in time near their +/// length, not its square, whatever they hold. Every value but a float is +/// looked up by the order [`ListElement`] gives it. A float equals a number, +/// a float or an integer, exactly when the two read as the same `f64` bits +/// (`Value::as_float`), so a wanted float is looked up among the bits of every +/// held number, and a wanted integer, beside its order, among those of every +/// held float. A map or a list of anything but bytes equals nothing. +fn list_holds_all(held: &[Value], wanted: &[Value]) -> bool { + let keyed: BTreeSet = held.iter().filter_map(ListElement::of).collect(); + let number_bits: BTreeSet = held + .iter() + .filter_map(|element| element.as_float().map(f64::to_bits)) + .collect(); + let float_bits: BTreeSet = held + .iter() + .filter_map(|element| match element { + Value::Float(float) => Some(float.to_bits()), + _ => None, + }) + .collect(); + wanted.iter().all(|element| match element { + Value::Float(float) => number_bits.contains(&float.to_bits()), + element => ListElement::of(element).is_some_and(|key| { + keyed.contains(&key) + || element + .as_float() + .is_some_and(|number| float_bits.contains(&number.to_bits())) + }), + }) +} + +/// The elements of a typed array a `where` entry over lists reads: a list the +/// document leaves out, or holds as null, is the empty one, as an indexed typed +/// array without elements is absent. `None` for a value that is no list, which +/// no stored typed array is and which then holds nothing and equals no list. +pub fn where_list_elements(value: Option<&Value>) -> Option<&[Value]> { + match value { + None | Some(Value::Null) => Some(&[]), + Some(Value::Array(elements)) => Some(elements), + Some(_) => None, + } +} + +/// How many elements a `where` entry reads of the typed array `value`: its +/// length, none for a list left out or a value that is no list. +fn where_list_len(value: Option<&Value>) -> usize { + where_list_elements(value).map_or(0, <[Value]>::len) +} + +/// How many elements [`where_lists_equal`] reads of `a` and `b`, both lists': +/// what document reference validation bills before comparing them. +pub fn where_lists_elements_read(a: Option<&Value>, b: Option<&Value>) -> usize { + where_list_len(a).saturating_add(where_list_len(b)) +} + +/// Whether two typed array values are equal as a `where` equality between two +/// typed arrays compares them: the same elements in the same order, a list the +/// document leaves out the empty one. Two elements are compared as a scalar +/// equality compares two single values (`Value::same_scalar_data`): an integer +/// whatever width it is held at, the bytes of an identifier or a byte array +/// whichever variant holds them (an identifier and its 32 bytes are equal), a +/// string as a string, a boolean as a boolean. +pub fn where_lists_equal(a: Option<&Value>, b: Option<&Value>) -> bool { + match (where_list_elements(a), where_list_elements(b)) { + (Some(a), Some(b)) => { + a.len() == b.len() && a.iter().zip(b).all(|(a, b)| a.same_scalar_data(b)) + } + _ => false, + } +} + +/// A single value other than a float as `Value::same_scalar_data` compares it: +/// two such values are equal exactly when they read as the same `ListElement`, +/// which orders them, so a `containsAll` looks each element up in a set. A +/// float compares with an integer as the float the integer rounds to, which is +/// no order, so it reads as none and is compared one by one. +#[derive(Debug, PartialEq, Eq, PartialOrd, Ord)] +enum ListElement<'a> { + Null, + Boolean(bool), + Integer(i128), + /// An unsigned integer past `i128::MAX`. + LargeUnsigned(u128), + Text(&'a str), + Bytes(Cow<'a, [u8]>), + EnumU8(&'a [u8]), + EnumString(&'a [String]), +} + +impl<'a> ListElement<'a> { + /// The element `value` reads as, `None` for a float and for a value + /// `same_scalar_data` finds equal to nothing: a map, or a list of anything + /// but bytes. + fn of(value: &'a Value) -> Option { + Some(match value { + Value::Null => ListElement::Null, + Value::Bool(flag) => ListElement::Boolean(*flag), + Value::Text(text) => ListElement::Text(text), + Value::U128(integer) => match i128::try_from(*integer) { + Ok(integer) => ListElement::Integer(integer), + Err(_) => ListElement::LargeUnsigned(*integer), + }, + Value::I128(integer) => ListElement::Integer(*integer), + Value::U64(integer) => ListElement::Integer(i128::from(*integer)), + Value::I64(integer) => ListElement::Integer(i128::from(*integer)), + Value::U32(integer) => ListElement::Integer(i128::from(*integer)), + Value::I32(integer) => ListElement::Integer(i128::from(*integer)), + Value::U16(integer) => ListElement::Integer(i128::from(*integer)), + Value::I16(integer) => ListElement::Integer(i128::from(*integer)), + Value::U8(integer) => ListElement::Integer(i128::from(*integer)), + Value::I8(integer) => ListElement::Integer(i128::from(*integer)), + Value::EnumU8(values) => ListElement::EnumU8(values), + Value::EnumString(values) => ListElement::EnumString(values), + Value::Bytes(bytes) => ListElement::Bytes(Cow::Borrowed(bytes)), + Value::Bytes20(bytes) => ListElement::Bytes(Cow::Borrowed(bytes)), + Value::Bytes32(bytes) => ListElement::Bytes(Cow::Borrowed(bytes)), + Value::Bytes36(bytes) => ListElement::Bytes(Cow::Borrowed(bytes)), + Value::Identifier(bytes) => ListElement::Bytes(Cow::Borrowed(bytes)), + // A transition may carry an identifier or a byte array as the bytes + // it lists + Value::Array(items) => ListElement::Bytes(Cow::Owned( + items + .iter() + .map(|item| match item { + Value::U8(byte) => Some(*byte), + _ => None, + }) + .collect::>>()?, + )), + _ => return None, + }) + } +} + impl fmt::Display for WhereCondition { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { WhereCondition::Constant(literal) => write!(f, "{{ \"const\": {literal} }}"), WhereCondition::NotEqual(bound) => write!(f, "{{ \"notEqual\": {bound} }}"), + WhereCondition::Contains(bound) => write!(f, "{{ \"contains\": {bound} }}"), + WhereCondition::ContainsAll(path) => write!(f, "{{ \"containsAll\": \"{path}\" }}"), WhereCondition::In(literals) => { write!(f, "{{ \"in\": [")?; for (index, literal) in literals.iter().enumerate() { @@ -690,6 +922,250 @@ mod tests { WhereCondition::Constant(WhereLiteral::Text("say \"hi\" \\o/".to_string())).to_string(), r#"{ "const": "say \"hi\" \\o/" }"# ); + assert_eq!( + WhereCondition::Contains(WhereContainsBound::Writer).to_string(), + "{ \"contains\": \"$ownerId\" }" + ); + assert_eq!( + WhereCondition::Contains(WhereContainsBound::Property("tag".to_string())).to_string(), + "{ \"contains\": \"tag\" }" + ); + assert_eq!( + WhereCondition::ContainsAll("tags".to_string()).to_string(), + "{ \"containsAll\": \"tags\" }" + ); + } + + fn texts(texts: &[&str]) -> Value { + Value::Array( + texts + .iter() + .map(|text| Value::Text(text.to_string())) + .collect(), + ) + } + + #[test] + fn should_equal_a_list_with_the_same_elements_in_the_same_order() { + let ab = texts(&["a", "b"]); + assert!(where_lists_equal(Some(&ab), Some(&texts(&["a", "b"])))); + assert!(!where_lists_equal(Some(&ab), Some(&texts(&["b", "a"])))); + assert!(!where_lists_equal(Some(&ab), Some(&texts(&["a"])))); + assert!(!where_lists_equal( + Some(&ab), + Some(&texts(&["a", "b", "b"])) + )); + // A list left out, or null, is the empty one + assert!(where_lists_equal(None, None)); + assert!(where_lists_equal(None, Some(&texts(&[])))); + assert!(where_lists_equal(Some(&Value::Null), Some(&texts(&[])))); + assert!(!where_lists_equal(None, Some(&ab))); + // A value that is no list equals no list + assert!(!where_lists_equal( + Some(&Value::Text("a".to_string())), + None + )); + } + + #[test] + fn should_compare_list_elements_by_value_not_by_encoding() { + let id = [7; 32]; + let held = Value::Array(vec![Value::Identifier(id), Value::Identifier([8; 32])]); + // An identifier equals its 32 bytes, however a transition carries them + let carried = Value::Array(vec![ + Value::Bytes(id.to_vec()), + Value::Array(vec![Value::U8(8); 32]), + ]); + assert!(where_lists_equal(Some(&held), Some(&carried))); + // An integer at any width, and a whole number carried as a float + let held = Value::Array(vec![Value::U16(3), Value::I64(-4)]); + let carried = Value::Array(vec![Value::U64(3), Value::Float(-4.0)]); + assert!(where_lists_equal(Some(&held), Some(&carried))); + assert!(!where_lists_equal( + Some(&held), + Some(&Value::Array(vec![Value::U64(3), Value::Float(-4.5)])) + )); + // A string never equals bytes, and an element of no list kind equals + // nothing, not even itself + assert!(!where_lists_equal( + Some(&texts(&["a"])), + Some(&Value::Array(vec![Value::Bytes(b"a".to_vec())])) + )); + let maps = Value::Array(vec![Value::Map(vec![])]); + assert!(!where_lists_equal(Some(&maps), Some(&maps))); + // An unsigned integer past i128::MAX equals only itself + let large = Value::Array(vec![Value::U128(u128::MAX)]); + assert!(where_lists_equal(Some(&large), Some(&large))); + assert!(!where_lists_equal( + Some(&large), + Some(&Value::Array(vec![Value::I128(-1)])) + )); + } + + /// The three list forms compare two elements exactly as a scalar equality + /// compares two single values, floats included: `-0.0` is not `0`, and an + /// integer past 2^53 equals the float it rounds to. + #[test] + fn should_compare_list_elements_as_a_scalar_equality_does() { + let sample = [ + Value::Null, + Value::Bool(true), + Value::Bool(false), + Value::U8(0), + Value::I8(0), + Value::I8(-4), + Value::U16(3), + Value::I16(3), + Value::U32(3), + Value::I32(-4), + Value::I64(0), + Value::U64(3), + Value::U128(3), + Value::I128(-4), + Value::U128(u128::MAX), + Value::I128(i128::MAX), + Value::Float(0.0), + Value::Float(-0.0), + Value::Float(3.0), + Value::Float(3.5), + Value::Float(f64::NAN), + Value::Float(9007199254740992.0), + Value::U64(9007199254740993), + Value::Text("a".to_string()), + Value::Text(String::new()), + Value::Bytes(vec![7; 32]), + Value::Bytes(vec![7; 20]), + Value::Bytes(vec![7; 36]), + Value::Bytes(vec![]), + Value::Bytes20([7; 20]), + Value::Bytes32([7; 32]), + Value::Bytes36([7; 36]), + Value::Identifier([7; 32]), + Value::Array(vec![Value::U8(7); 20]), + Value::Array(vec![Value::U8(7); 32]), + Value::Array(vec![Value::U8(7); 36]), + Value::Array(vec![]), + Value::Array(vec![Value::Text("a".to_string())]), + Value::Map(vec![]), + Value::EnumU8(vec![1]), + Value::EnumU8(vec![7; 32]), + Value::EnumString(vec!["a".to_string()]), + Value::EnumString(vec![]), + ]; + let all_tags = WhereCondition::ContainsAll("tags".to_string()); + let tag = WhereCondition::Contains(WhereContainsBound::Property("tag".to_string())); + for held in &sample { + for wanted in &sample { + let scalar = held.same_scalar_data(wanted); + let held_list = Value::Array(vec![held.clone()]); + let wanted_list = Value::Array(vec![wanted.clone()]); + let pair = format!("{held:?} against {wanted:?}"); + assert_eq!( + where_lists_equal(Some(&held_list), Some(&wanted_list)), + scalar, + "equality of {pair}" + ); + assert_eq!( + tag.holds(Some(&held_list), None, Some(wanted)), + scalar, + "contains of {pair}" + ); + assert_eq!( + all_tags.holds(Some(&held_list), None, Some(&wanted_list)), + scalar, + "containsAll of {pair}" + ); + } + } + // Every value held at once, floats among them, each looked up in the set + let everything = Value::Array(sample.to_vec()); + for wanted in &sample { + assert_eq!( + all_tags.holds( + Some(&everything), + None, + Some(&Value::Array(vec![wanted.clone()])) + ), + sample.iter().any(|held| held.same_scalar_data(wanted)), + "containsAll of {wanted:?} among the whole sample" + ); + } + } + + /// What document reference validation bills before comparing: every + /// element of both lists for an equality and a `containsAll`, the held + /// list's for a `contains`, a list left out none, and nothing for an entry + /// comparing single values. + #[test] + fn should_count_the_list_elements_an_entry_reads() { + let three = texts(&["a", "b", "c"]); + let two = texts(&["a", "b"]); + assert_eq!(where_lists_elements_read(Some(&three), Some(&two)), 5); + assert_eq!(where_lists_elements_read(None, Some(&two)), 2); + let all_tags = WhereCondition::ContainsAll("tags".to_string()); + assert_eq!(all_tags.list_elements_read(Some(&three), Some(&two)), 5); + assert_eq!(all_tags.list_elements_read(Some(&three), None), 3); + let tag = WhereCondition::Contains(WhereContainsBound::Property("tag".to_string())); + assert_eq!( + tag.list_elements_read(Some(&three), Some(&Value::Text("a".to_string()))), + 3 + ); + let constant = WhereCondition::Constant(WhereLiteral::Text("a".to_string())); + assert_eq!(constant.list_elements_read(Some(&three), None), 0); + } + + #[test] + fn should_hold_contains_only_for_a_held_element() { + let writer = WhereCondition::Contains(WhereContainsBound::Writer); + let members = Value::Array(vec![Value::Identifier([1; 32]), Value::Identifier([2; 32])]); + assert!(writer.holds(Some(&members), None, Some(&Value::Identifier([2; 32])))); + assert!(writer.holds(Some(&members), None, Some(&Value::Bytes(vec![1; 32])))); + assert!(!writer.holds(Some(&members), None, Some(&Value::Identifier([3; 32])))); + // An absent list holds nothing, and an absent value is held by no list + assert!(!writer.holds(None, None, Some(&Value::Identifier([1; 32])))); + let tag = WhereCondition::Contains(WhereContainsBound::Property("tag".to_string())); + assert!(!tag.holds(Some(&texts(&["a"])), None, None)); + let b = Value::Text("b".to_string()); + assert!(tag.holds(Some(&texts(&["a", "b"])), None, Some(&b))); + assert!(!tag.holds(None, None, Some(&b))); + } + + #[test] + fn should_hold_contains_all_for_every_element_in_any_order() { + let all_tags = WhereCondition::ContainsAll("tags".to_string()); + let held = texts(&["a", "b", "c"]); + assert!(all_tags.holds(Some(&held), None, Some(&texts(&["c", "a"])))); + assert!(all_tags.holds(Some(&held), None, Some(&texts(&["a", "a"])))); + assert!(!all_tags.holds(Some(&held), None, Some(&texts(&["a", "d"])))); + // Every list holds all of an empty one, a list left out among them + assert!(all_tags.holds(Some(&held), None, Some(&texts(&[])))); + assert!(all_tags.holds(Some(&held), None, None)); + assert!(all_tags.holds(None, None, None)); + assert!(!all_tags.holds(None, None, Some(&texts(&["a"])))); + // By value, as the equality compares + let held = Value::Array(vec![Value::Identifier([1; 32]), Value::Identifier([2; 32])]); + let wanted = Value::Array(vec![Value::Array(vec![Value::U8(2); 32])]); + assert!(all_tags.holds(Some(&held), None, Some(&wanted))); + } + + #[test] + fn should_recheck_a_contains_of_the_writer_on_every_replace() { + assert!( + WhereCondition::Contains(WhereContainsBound::Writer).is_rechecked_on_every_replace() + ); + assert!( + !WhereCondition::Contains(WhereContainsBound::Property("tag".to_string())) + .is_rechecked_on_every_replace() + ); + assert!(!WhereCondition::ContainsAll("tags".to_string()).is_rechecked_on_every_replace()); + assert_eq!( + WhereCondition::ContainsAll("tags".to_string()).referring_property(), + Some("tags") + ); + assert_eq!( + WhereCondition::Contains(WhereContainsBound::Writer).referring_property(), + None + ); } #[test] diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs index c4e963090d2..a6eee31a0fd 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs @@ -24,7 +24,10 @@ use dpp::data_contract::config::moderation::{kept_value_at, ContractDocumentRemo use dpp::data_contract::document_type::property_constraints::{ referenced_document_value, referenced_document_values, referenced_fields, DocumentSystemValues, }; -use dpp::data_contract::document_type::{WhereBoundRead, WhereCondition, WhereLiteralKind}; +use dpp::data_contract::document_type::{ + typed_array_property, where_lists_elements_read, where_lists_equal, WhereBoundRead, + WhereCondition, WhereLiteralKind, +}; use dpp::data_contract::DataContract; use dpp::document::property_names::{ID, OWNER_ID}; use dpp::document::document_methods::DocumentMethodsV0; @@ -44,6 +47,7 @@ use dpp::identifier::Identifier; use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; use dpp::identity::KeyID; use dpp::platform_value::btreemap_extensions::BTreeValueMapPathHelper; +use dpp::platform_value::Error as ValueError; use dpp::platform_value::Value; use dpp::validation::SimpleConsensusValidationResult; use dpp::version::PlatformVersion; @@ -1449,6 +1453,8 @@ fn validate_reference_target_v0( referring_system, path, rule_reads, + execution_context, + platform_version, ); } } @@ -1528,6 +1534,33 @@ fn validate_reference_target_v0( else { return Ok(mismatch()); }; + // Two typed arrays, which registration pairs with nothing + // else, are equal as lists: the same elements in the same + // order, each compared by value, a list either document + // leaves out the empty one. In place in generation 0: a + // pair of lists registers from protocol version 14 only + let lists = typed_array_property( + referenced_document_type.flattened_properties(), + referenced_property, + ) + .is_some(); + if lists { + bill_where_list_elements( + where_lists_elements_read( + referring_value.as_deref(), + referenced_value.as_deref(), + ), + execution_context, + platform_version, + )?; + if !where_lists_equal( + referring_value.as_deref(), + referenced_value.as_deref(), + ) { + return Ok(mismatch()); + } + continue; + } let (referring_value, referenced_value) = match (referring_value, referenced_value) { (Some(referring_value), Some(referenced_value)) => { @@ -1562,8 +1595,16 @@ fn validate_reference_target_v0( referenced_document_type, referenced_property, ); - let bound = - where_bound_value(condition, document_data, owner_id, referring_system); + let Ok(bound) = + where_bound_value(condition, document_data, owner_id, referring_system) + else { + return Ok(condition_mismatch(path, condition, referenced_property)); + }; + bill_where_list_elements( + condition.list_elements_read(referenced_value.as_deref(), bound.as_deref()), + execution_context, + platform_version, + )?; if !condition.holds(referenced_value.as_deref(), kind, bound.as_deref()) { return Ok(condition_mismatch(path, condition, referenced_property)); } @@ -2138,28 +2179,66 @@ fn condition_mismatch( ) } +/// Bills `elements`, the list elements a `where` entry comparing typed arrays +/// reads (`where_lists_elements_read`, `WhereCondition::list_elements_read`), +/// before it compares them: `where_list_credits_per_element` each, of the +/// fee schedule's document reference group. Nothing for an entry comparing +/// single values, which reads none. Every list holds at most +/// `max_typed_array_items` elements, so the product never nears overflow; an +/// overflow is still an error rather than a wrapped bill. +fn bill_where_list_elements( + elements: usize, + execution_context: &mut StateTransitionExecutionContext, + platform_version: &PlatformVersion, +) -> Result<(), Error> { + if elements == 0 { + return Ok(()); + } + let credits = u64::try_from(elements) + .ok() + .and_then(|elements| { + elements.checked_mul( + platform_version + .fee_version + .document_reference + .where_list_credits_per_element, + ) + }) + .ok_or(Error::Execution(ExecutionError::Overflow( + "where list comparison fee overflow", + )))?; + execution_context.add_operation(ValidationOperation::PrecalculatedOperation( + FeeResult::new_from_processing_fee(credits), + )); + Ok(()) +} + /// The value of the bound `condition` compares the referenced side with, read /// from the referring document: a property of `document_data` (`None` when it -/// leaves it out or a path on the way is no object), the writer `owner_id`, or -/// a time or height of `referring_system` as an integer. `None` for a literal -/// or a list of them, which the condition carries. +/// leaves it out), the writer `owner_id`, or a time or height of +/// `referring_system` as an integer. `None` for a literal or a list of them, +/// which the condition carries. An error when a path on the way to the property +/// is no object, which the caller refuses as a mismatch, as an equality's +/// unreadable side is, never as absence: a `containsAll` would read an absent +/// list as empty and hold. fn where_bound_value<'a>( condition: &WhereCondition, document_data: &'a BTreeMap, owner_id: Identifier, referring_system: &DocumentSystemValues, -) -> Option> { - match condition.bound_read()? { - WhereBoundRead::Property(path) => document_data - .get_optional_at_path(path) - .ok() - .flatten() - .map(Cow::Borrowed), +) -> Result>, ValueError> { + let Some(bound_read) = condition.bound_read() else { + return Ok(None); + }; + Ok(match bound_read { + WhereBoundRead::Property(path) => { + document_data.get_optional_at_path(path)?.map(Cow::Borrowed) + } WhereBoundRead::Writer => Some(Cow::Owned(Value::Identifier(owner_id.to_buffer()))), WhereBoundRead::System(property) => referring_system .value(property) .map(|value| Cow::Owned(Value::I128(value))), - } + }) } /// What a removal record holds of the document a moderator removed, read field @@ -2230,7 +2309,9 @@ impl RemovalRecord<'_> { /// it is, unless a rule reads the document through the reference: then every /// entry is checked, as the replace checks every entry against a document in /// state, and registration keeps each to what every record holds. No read: the -/// record was read and billed by the caller. +/// record was read and billed by the caller. The list elements an entry +/// comparing typed arrays reads are billed before it compares them +/// ([`bill_where_list_elements`]), as against a document in state. /// /// In place in generation 0: reached from protocol version 14 only, the only /// version whose parser produces a `moderatedDocument` reference. @@ -2247,6 +2328,8 @@ fn validate_pairs_against_removal( referring_system: &DocumentSystemValues, path: &str, rule_reads: Option>, + execution_context: &mut StateTransitionExecutionContext, + platform_version: &PlatformVersion, ) -> Result { let mut record = RemovalRecord { removal, @@ -2288,12 +2371,28 @@ fn validate_pairs_against_removal( Err(_) => None, } }; - let holds = match (referring_value, referenced_value) { - (Some(referring_value), Some(referenced_value)) => { - referring_value.same_scalar_data(&referenced_value) + // Two typed arrays are equal as lists, a list the record holds none of + // the empty one, as against a document in state + let lists = typed_array_property( + referenced_document_type.flattened_properties(), + referenced_property, + ) + .is_some(); + let holds = if lists { + bill_where_list_elements( + where_lists_elements_read(referring_value.as_deref(), referenced_value.as_ref()), + execution_context, + platform_version, + )?; + where_lists_equal(referring_value.as_deref(), referenced_value.as_ref()) + } else { + match (referring_value, referenced_value) { + (Some(referring_value), Some(referenced_value)) => { + referring_value.same_scalar_data(&referenced_value) + } + (None, None) => true, + (Some(_), None) | (None, Some(_)) => false, } - (None, None) => true, - (Some(_), None) | (None, Some(_)) => false, }; if !holds { return Ok(SimpleConsensusValidationResult::new_with_error( @@ -2324,7 +2423,15 @@ fn validate_pairs_against_removal( return Ok(removed(referenced_property)); }; let kind = WhereLiteralKind::of_referenced(referenced_document_type, referenced_property); - let bound = where_bound_value(condition, document_data, owner_id, referring_system); + let Ok(bound) = where_bound_value(condition, document_data, owner_id, referring_system) + else { + return Ok(condition_mismatch(path, condition, referenced_property)); + }; + bill_where_list_elements( + condition.list_elements_read(referenced_value.as_ref(), bound.as_deref()), + execution_context, + platform_version, + )?; if !condition.holds(referenced_value.as_ref(), kind, bound.as_deref()) { return Ok(condition_mismatch(path, condition, referenced_property)); } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs index 77fb35d9e81..29b531d815a 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs @@ -39,6 +39,7 @@ mod system_agreement; mod token_cost_policy; mod transfer; mod typed_array_references; +mod typed_array_where; use super::*; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/reference_test_setup.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/reference_test_setup.rs index 2beafbf00bf..ab6c1652b46 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/reference_test_setup.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/reference_test_setup.rs @@ -116,6 +116,36 @@ pub(super) async fn create_document>( rng: &mut StdRng, platform_version: &PlatformVersion, ) -> (Document, StateTransitionsProcessingResult) { + let (document, create) = create_transition( + contract, + type_name, + properties, + owner, + key, + nonce, + signer, + rng, + platform_version, + ) + .await; + let result = process_and_commit(platform, platform_state, &create, platform_version); + (document, result) +} + +/// The create transition of a document of `type_name` with exactly +/// `properties` set, and the document it creates, not yet processed. +#[allow(clippy::too_many_arguments)] +pub(super) async fn create_transition>( + contract: &DataContract, + type_name: &str, + properties: &[(&str, Value)], + owner: Identifier, + key: &IdentityPublicKey, + nonce: u64, + signer: &S, + rng: &mut StdRng, + platform_version: &PlatformVersion, +) -> (Document, StateTransition) { let document_type = contract .document_type_for_name(type_name) .expect("doctype exists"); @@ -153,8 +183,7 @@ pub(super) async fn create_document>( ) .await .expect("expected the create transition"); - let result = process_and_commit(platform, platform_state, &create, platform_version); - (document, result) + (document, create) } /// Replaces `document` in the block `block_info`. diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/typed_array_where.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/typed_array_where.rs new file mode 100644 index 00000000000..909182f9e23 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/typed_array_where.rs @@ -0,0 +1,558 @@ +//! `where` entries over typed arrays through the full ABCI pipeline (protocol +//! version 14). The fixture's `post` holds `tags`, `members` and `scores`; a +//! `like` echoes its post's `tags` (an equality between two typed arrays), +//! a `copy` its `members` and `scores`, a `tagged` names one of its tags +//! (`contains`), a `memberNote` is written only by one of its members +//! (`contains` of the writer), and a `subset` names some of its tags +//! (`containsAll`). Lists are compared element by element, by value, and a +//! list a document leaves out is the empty one. + +use super::*; + +mod typed_array_where_tests { + use super::super::reference_test_setup::{ + assert_successful, create_transition, process_and_commit, ReferenceTestSetup as Setup, + }; + use super::*; + use crate::platform_types::state_transitions_processing_result::StateTransitionsProcessingResult; + use dpp::identifier::Identifier; + use dpp::state_transition::batch_transition::accessors::DocumentsBatchTransitionAccessorsV0; + use dpp::state_transition::batch_transition::batched_transition::document_transition::DocumentTransition; + use dpp::state_transition::batch_transition::batched_transition::BatchedTransitionRef; + use dpp::state_transition::StateTransition; + use drive::state_transition_action::batch::batched_transition::document_transition::document_create_transition_action::{ + DocumentCreateTransitionAction, DocumentCreateTransitionActionAccessorsV0, + }; + use drive::state_transition_action::batch::batched_transition::document_transition::DocumentTransitionAction; + use drive::state_transition_action::batch::batched_transition::BatchedTransitionAction; + use crate::execution::types::execution_operation::ValidationOperation; + use crate::execution::types::state_transition_execution_context::{ + StateTransitionExecutionContext, StateTransitionExecutionContextMethodsV0, + }; + use crate::execution::validation::state_transition::batch::action_validation::document::document_reference_validation::DocumentReferenceValidation; + use crate::platform_types::platform::PlatformStateRef; + use dpp::data_contract::document_type::property_constraints::DocumentSystemValues; + use dpp::tokens::gas_fees_paid_by::GasFeesPaidBy; + use dpp::fee::fee_result::FeeResult; + use dpp::validation::SimpleConsensusValidationResult; + use dpp::version::DefaultForPlatformVersion; + use drive::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::{ + DocumentBaseTransitionAction, DocumentBaseTransitionActionV0, + }; + use std::collections::BTreeMap; + + const CONTRACT_PATH: &str = "tests/supporting_files/contract/reference-validation/reference-validation-contract-where-typed-arrays.json"; + + /// The write was refused because the post fails the `where` entry whose + /// referring side is `referring`: a property for an equality, the + /// condition as written for the others (code 40127). + fn assert_mismatch(result: &StateTransitionsProcessingResult, referring: &str, because: &str) { + assert_matches!( + result.execution_results().as_slice(), + [StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError( + StateError::ReferencedDocumentPropertyMismatchError(mismatch) + ), + .. + }] if mismatch.referring_property() == referring, + "{because}: got {result:?}" + ); + } + + fn texts(texts: &[&str]) -> Value { + Value::Array( + texts + .iter() + .map(|text| Value::Text(text.to_string())) + .collect(), + ) + } + + impl Setup { + /// A post holding exactly `properties`, created. + async fn post(&mut self, properties: &[(&str, Value)]) -> Identifier { + let (post, result) = self.create("post", properties).await; + assert_successful(&result, "the post is created"); + post.id() + } + + /// A document of `type_name` naming the post `post_id`, beside + /// `properties`. + async fn create_on( + &mut self, + type_name: &str, + post_id: Identifier, + properties: &[(&str, Value)], + ) -> StateTransitionsProcessingResult { + let mut all = vec![("postId", Value::Identifier(post_id.to_buffer()))]; + all.extend(properties.iter().cloned()); + self.create(type_name, &all).await.1 + } + } + + /// A like echoes its post's tags exactly: the same tags in the same + /// order. Reordered, fewer, more or none differ. + #[tokio::test] + async fn should_agree_on_a_list_with_the_same_elements_in_the_same_order() { + let mut setup = Setup::new(CONTRACT_PATH, 6101); + let post = setup.post(&[("tags", texts(&["dash", "news"]))]).await; + + let result = setup + .create_on("like", post, &[("tags", texts(&["dash", "news"]))]) + .await; + assert_successful(&result, "the same tags in the same order agree"); + + for (tags, because) in [ + ( + Some(texts(&["news", "dash"])), + "the order of the elements counts", + ), + (Some(texts(&["dash"])), "a missing element differs"), + ( + Some(texts(&["dash", "news", "tech"])), + "an extra element differs", + ), + (None, "a list left out is empty, and the post's is not"), + ] { + let properties: Vec<(&str, Value)> = + tags.into_iter().map(|tags| ("tags", tags)).collect(); + let result = setup.create_on("like", post, &properties).await; + assert_mismatch(&result, "tags", because); + } + } + + /// A list left out is the empty one: a like leaving out its tags agrees + /// with a post that has none, and so does an empty list, either way round. + #[tokio::test] + async fn should_agree_on_lists_left_out_or_empty() { + let mut setup = Setup::new(CONTRACT_PATH, 6102); + let untagged = setup.post(&[("label", "plain".into())]).await; + let result = setup.create_on("like", untagged, &[]).await; + assert_successful(&result, "two lists left out agree"); + let result = setup + .create_on("like", untagged, &[("tags", texts(&[]))]) + .await; + assert_successful(&result, "an empty list agrees with one left out"); + + let empty = setup.post(&[("tags", texts(&[]))]).await; + let result = setup.create_on("like", empty, &[]).await; + assert_successful(&result, "a list left out agrees with an empty one"); + + let result = setup + .create_on("like", untagged, &[("tags", texts(&["dash"]))]) + .await; + assert_mismatch(&result, "tags", "a tag the post does not have differs"); + } + + /// Elements compare by value, not by how a transition carries them: an + /// identifier equals its 32 bytes, and an integer is the same at any width. + #[tokio::test] + async fn should_compare_list_elements_by_value() { + let mut setup = Setup::new(CONTRACT_PATH, 6103); + let alice = Identifier::new([1; 32]); + let bob = Identifier::new([2; 32]); + let post = setup + .post(&[ + ( + "members", + Value::Array(vec![ + Value::Identifier(alice.to_buffer()), + Value::Identifier(bob.to_buffer()), + ]), + ), + ("scores", Value::Array(vec![Value::U16(3), Value::U16(700)])), + ]) + .await; + + let members = Value::Array(vec![ + Value::Bytes(alice.to_buffer().to_vec()), + Value::Bytes(bob.to_buffer().to_vec()), + ]); + let scores = Value::Array(vec![Value::U64(3), Value::U64(700)]); + let nonce = setup.next_nonce(); + let (_, create) = create_transition( + &setup.contract, + "copy", + &[ + ("postId", Value::Identifier(post.to_buffer())), + ("members", members.clone()), + ("scores", scores.clone()), + ], + setup.owner, + &setup.key, + nonce, + &setup.signer, + &mut setup.rng, + PlatformVersion::latest(), + ) + .await; + // The node decodes the transition and checks the references on its + // action's data, which still holds the elements as they were carried + let data = create_action_data(&setup, &create); + assert_eq!(data.get("members"), Some(&members)); + assert_eq!(data.get("scores"), Some(&scores)); + let result = process_and_commit( + &setup.platform, + &setup.platform_state, + &create, + PlatformVersion::latest(), + ); + assert_successful(&result, "equal values agree however they are carried"); + + let result = setup + .create_on( + "copy", + post, + &[ + ( + "members", + Value::Array(vec![ + Value::Identifier(alice.to_buffer()), + Value::Identifier(bob.to_buffer()), + ]), + ), + ("scores", Value::Array(vec![Value::U64(700), Value::U64(3)])), + ], + ) + .await; + assert_mismatch(&result, "scores", "the order of the scores counts"); + } + + /// The data of the create action the node builds from `create` once + /// decoded from its bytes as a block's transition is, which the reference + /// check reads. + fn create_action_data(setup: &Setup, create: &StateTransition) -> BTreeMap { + let bytes = create + .serialize_to_bytes() + .expect("expected the transition to serialize"); + let StateTransition::Batch(batch) = + StateTransition::deserialize_from_bytes_untrusted_exact_in_version( + &bytes, + PlatformVersion::latest(), + ) + .expect("expected it to decode as the node does") + else { + panic!("expected a batch transition"); + }; + let Some(BatchedTransitionRef::Document(DocumentTransition::Create(create))) = + batch.first_transition() + else { + panic!("expected a document create"); + }; + let (_, contract_fetch_info) = setup + .platform + .drive + .get_contract_with_fetch_info_and_fee( + setup.contract.id().to_buffer(), + None, + false, + None, + PlatformVersion::latest(), + ) + .expect("expected to fetch the contract"); + let contract_fetch_info = contract_fetch_info.expect("the contract is in state"); + let (result, _) = + DocumentCreateTransitionAction::try_from_document_borrowed_create_transition_with_contract_lookup( + &setup.platform.drive, + setup.owner, + None, + create, + &BlockInfo::default(), + 0, + |_| Ok(contract_fetch_info.clone()), + PlatformVersion::latest(), + ) + .expect("expected the transformer to run"); + match result.into_data().expect("expected an action") { + BatchedTransitionAction::DocumentAction(DocumentTransitionAction::CreateAction( + action, + )) => action.data().clone(), + other => panic!("expected a create action, got {other:?}"), + } + } + + /// Validates the references of a document of `type_name` holding `data`, + /// written by the setup's identity, under `platform_version`, as a create's + /// state validation does, and returns the result with what it billed. + fn validate_references( + setup: &Setup, + type_name: &str, + data: &[(&str, Value)], + platform_version: &PlatformVersion, + ) -> ( + SimpleConsensusValidationResult, + StateTransitionExecutionContext, + ) { + let (_, contract_fetch_info) = setup + .platform + .drive + .get_contract_with_fetch_info_and_fee( + setup.contract.id().to_buffer(), + None, + false, + None, + platform_version, + ) + .expect("expected to fetch the contract"); + let base = DocumentBaseTransitionAction::V0(DocumentBaseTransitionActionV0 { + id: Identifier::from([0xAA; 32]), + identity_contract_nonce: 1, + document_type_name: type_name.to_string(), + data_contract: contract_fetch_info.expect("the contract is in state"), + token_cost: None, + shielded_token_payment: None, + gas_fees_paid_by: GasFeesPaidBy::default(), + contract_gas_fees_paid_by: GasFeesPaidBy::default(), + declared_action_fee: None, + }); + let platform_state = setup.platform.state.load(); + let platform_ref = PlatformStateRef { + drive: &setup.platform.drive, + state: &platform_state, + config: &setup.platform.config, + }; + let data: BTreeMap = data + .iter() + .map(|(property, value)| (property.to_string(), value.clone())) + .collect(); + let mut execution_context = + StateTransitionExecutionContext::default_for_platform_version(platform_version) + .expect("expected an execution context"); + let result = base + .validate_document_references( + &data, + setup.owner, + Some(setup.owner), + &DocumentSystemValues::default(), + None, + None, + &platform_ref, + &BlockInfo::default(), + &mut Vec::new(), + None, + &mut BTreeMap::new(), + None, + &mut execution_context, + platform_version, + ) + .expect("expected the references to be validated"); + (result, execution_context) + } + + /// The processing fee of everything the validation billed. + fn processing_fee(execution_context: &StateTransitionExecutionContext) -> u64 { + let mut fee = FeeResult::default(); + ValidationOperation::add_many_to_fee_result( + execution_context.operations_slice(), + &mut fee, + PlatformVersion::latest(), + ) + .expect("expected the operations to sum"); + fee.processing_fee + } + + /// Every list element an entry over lists reads is billed before it is + /// compared, on both sides, the refused write too: an equality reads both + /// lists, a `containsAll` the post's and the document's, a `contains` the + /// post's. + #[tokio::test] + async fn should_bill_every_list_element_a_where_entry_reads() { + let mut setup = Setup::new(CONTRACT_PATH, 6108); + let post = setup + .post(&[("tags", texts(&["dash", "news", "tech"]))]) + .await; + let post_id = ("postId", Value::Identifier(post.to_buffer())); + let per_element = |credits: u64| { + let mut platform_version = PlatformVersion::latest().clone(); + platform_version + .fee_version + .document_reference + .where_list_credits_per_element = credits; + platform_version + }; + let (free, seven) = (per_element(0), per_element(7)); + + for (type_name, data, holds, elements) in [ + ("like", ("tags", texts(&["dash", "news", "tech"])), true, 6), + ("subset", ("tags", texts(&["news", "dash"])), true, 5), + ("subset", ("tags", texts(&["sport"])), false, 4), + ("tagged", ("tag", Value::Text("tech".to_string())), true, 3), + ] { + let data = [post_id.clone(), data]; + let (result, billed) = validate_references(&setup, type_name, &data, &seven); + assert_eq!(result.is_valid(), holds, "{type_name}: {result:?}"); + let (_, unbilled) = validate_references(&setup, type_name, &data, &free); + assert_eq!( + processing_fee(&billed) - processing_fee(&unbilled), + 7 * elements, + "{type_name} bills {elements} elements" + ); + } + } + + /// An integer list may be carried as whole floats, which compare with the + /// post's integers as a scalar equality compares them: 1.0 is 1, `-0.0` is + /// not 0. A `containsAll` of many repeated floats looks each up, billed by + /// the elements read. + #[tokio::test] + async fn should_compare_floats_in_an_integer_list_as_scalars() { + let mut setup = Setup::new(CONTRACT_PATH, 6109); + let post = setup + .post(&[("scores", Value::Array(vec![Value::U16(0), Value::U16(1)]))]) + .await; + let post_id = ("postId", Value::Identifier(post.to_buffer())); + + let ones = Value::Array(vec![Value::Float(1.0); 1024]); + let (result, _) = validate_references( + &setup, + "scoreSubset", + &[post_id.clone(), ("scores", ones)], + PlatformVersion::latest(), + ); + assert!(result.is_valid(), "1.0 is the integer 1: {result:?}"); + + let negative_zero = Value::Array(vec![Value::Float(-0.0)]); + let (result, _) = validate_references( + &setup, + "scoreSubset", + &[post_id, ("scores", negative_zero)], + PlatformVersion::latest(), + ); + assert_matches!( + result.errors.as_slice(), + [ConsensusError::StateError(StateError::ReferencedDocumentPropertyMismatchError(e))] + if e.referring_property() == "{ \"containsAll\": \"scores\" }", + "-0.0 is not the integer 0" + ); + } + + /// A `contains` holds when the post's list holds the referring value; a + /// value left out is in no list, and a list left out holds nothing. + #[tokio::test] + async fn should_hold_contains_for_a_listed_value_only() { + let mut setup = Setup::new(CONTRACT_PATH, 6104); + let post = setup.post(&[("tags", texts(&["dash", "news"]))]).await; + let condition = "{ \"contains\": \"tag\" }"; + + let result = setup + .create_on("tagged", post, &[("tag", "news".into())]) + .await; + assert_successful(&result, "a listed tag is contained"); + + let result = setup + .create_on("tagged", post, &[("tag", "tech".into())]) + .await; + assert_mismatch(&result, condition, "an unlisted tag is not contained"); + + let result = setup.create_on("tagged", post, &[]).await; + assert_mismatch(&result, condition, "a tag left out is in no list"); + + let untagged = setup.post(&[("label", "plain".into())]).await; + let result = setup + .create_on("tagged", untagged, &[("tag", "dash".into())]) + .await; + assert_mismatch(&result, condition, "a list left out holds nothing"); + } + + /// `contains` of `"$ownerId"` is a writer gate: only a listed member + /// writes a note on the post. + #[tokio::test] + async fn should_let_only_a_listed_member_write() { + let mut setup = Setup::new(CONTRACT_PATH, 6105); + let writer = setup.owner; + let condition = "{ \"contains\": \"$ownerId\" }"; + let with_writer = setup + .post(&[( + "members", + Value::Array(vec![ + Value::Identifier([9; 32]), + Value::Identifier(writer.to_buffer()), + ]), + )]) + .await; + let result = setup + .create_on("memberNote", with_writer, &[("text", "hi".into())]) + .await; + assert_successful(&result, "a listed member writes"); + + let without_writer = setup + .post(&[("members", Value::Array(vec![Value::Identifier([9; 32])]))]) + .await; + let result = setup + .create_on("memberNote", without_writer, &[("text", "hi".into())]) + .await; + assert_mismatch(&result, condition, "an unlisted writer is refused"); + + let no_members = setup.post(&[("label", "plain".into())]).await; + let result = setup + .create_on("memberNote", no_members, &[("text", "hi".into())]) + .await; + assert_mismatch(&result, condition, "a post without members admits no one"); + } + + /// A `containsAll` holds when the post's list holds every element of the + /// referring list, in any order; every list holds all of an empty one or + /// of one left out. + #[tokio::test] + async fn should_hold_contains_all_for_every_element_in_any_order() { + let mut setup = Setup::new(CONTRACT_PATH, 6106); + let post = setup + .post(&[("tags", texts(&["dash", "news", "tech"]))]) + .await; + let condition = "{ \"containsAll\": \"tags\" }"; + + for (tags, because) in [ + ( + Some(texts(&["tech", "dash"])), + "a subset in another order is held", + ), + (Some(texts(&["news", "news"])), "a repeated element is held"), + (Some(texts(&[])), "every list holds all of an empty one"), + (None, "a list left out is empty"), + ] { + let properties: Vec<(&str, Value)> = + tags.into_iter().map(|tags| ("tags", tags)).collect(); + let result = setup.create_on("subset", post, &properties).await; + assert_successful(&result, because); + } + + let result = setup + .create_on("subset", post, &[("tags", texts(&["dash", "sport"]))]) + .await; + assert_mismatch(&result, condition, "an element the post lacks is not held"); + + let untagged = setup.post(&[("label", "plain".into())]).await; + let result = setup.create_on("subset", untagged, &[]).await; + assert_successful(&result, "a list left out holds all of another left out"); + let result = setup + .create_on("subset", untagged, &[("tags", texts(&["dash"]))]) + .await; + assert_mismatch(&result, condition, "a list left out holds no element"); + } + + /// A replace changing the like's tags asks about the pair again, and the + /// changed list must still be its post's. + #[tokio::test] + async fn should_check_a_changed_list_again_on_a_replace() { + let mut setup = Setup::new(CONTRACT_PATH, 6107); + let post = setup.post(&[("tags", texts(&["dash", "news"]))]).await; + let (mut like, result) = setup + .create( + "like", + &[ + ("postId", Value::Identifier(post.to_buffer())), + ("tags", texts(&["dash", "news"])), + ], + ) + .await; + assert_successful(&result, "the like is created"); + + like.set("note", "first".into()); + let result = setup.replace("like", &mut like).await; + assert_successful(&result, "a replace keeping the tags agrees"); + + like.set("tags", texts(&["news", "dash"])); + let result = setup.replace("like", &mut like).await; + assert_mismatch(&result, "tags", "reordered tags no longer agree"); + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests.rs index 62528c3a1ce..d752589b996 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests.rs @@ -85,6 +85,7 @@ mod referenced_rules; mod restore_constraints; mod retraction; mod seated_team; +mod typed_array_where; const DATA_CONTRACT_NOT_PRESENT: u32 = 10400; const CONTRACT_MODERATION_SELF_TARGET: u32 = 10901; @@ -2301,6 +2302,49 @@ fn post_schema(deletable_by_moderators: bool) -> Value { }) } +/// A creation by `actor` of a document of `document_type_name` holding exactly +/// `properties` +async fn create_with( + setup: &Setup, + actor: &Actor, + document_type_name: &str, + properties: Vec<(&str, Value)>, +) -> (Document, StateTransition) { + setup + .create_document_of_type_with(actor, document_type_name, |document| { + document.properties_mut().clear(); + for (property, value) in properties { + document.set(property, value); + } + }) + .await +} + +/// The replacement of `document` by `actor`, its owner, its revision bumped +async fn replacement( + actor: &Actor, + contract: &DataContract, + document_type_name: &str, + document: &mut Document, +) -> StateTransition { + document.increment_revision().expect("revision increments"); + BatchTransition::new_document_replacement_transition_from_document( + document.clone(), + contract + .document_type_for_name(document_type_name) + .expect("expected the document type"), + &actor.key, + actor.contract_nonce(), + 0, + None, + &actor.signer, + PlatformVersion::latest(), + None, + ) + .await + .expect("expected to build the replacement") +} + fn add_document_type(contract: &mut DataContract, name: &str, schema: Value) { contract .set_document_schema(name, schema, true, &mut vec![], PlatformVersion::latest()) diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests/referenced_rules.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests/referenced_rules.rs index be8bcac7e0a..15e99a9ba85 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests/referenced_rules.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests/referenced_rules.rs @@ -251,49 +251,6 @@ fn id_value(id: Identifier) -> Value { Value::Identifier(id.to_buffer()) } -/// A creation by `actor` of a document of `document_type_name` holding exactly -/// `properties` -async fn create_with( - setup: &Setup, - actor: &Actor, - document_type_name: &str, - properties: Vec<(&str, Value)>, -) -> (Document, StateTransition) { - setup - .create_document_of_type_with(actor, document_type_name, |document| { - document.properties_mut().clear(); - for (property, value) in properties { - document.set(property, value); - } - }) - .await -} - -/// The replacement of `document` by `actor`, its owner, its revision bumped -async fn replacement( - actor: &Actor, - contract: &DataContract, - document_type_name: &str, - document: &mut Document, -) -> StateTransition { - document.increment_revision().expect("revision increments"); - BatchTransition::new_document_replacement_transition_from_document( - document.clone(), - contract - .document_type_for_name(document_type_name) - .expect("expected the document type"), - &actor.key, - actor.contract_nonce(), - 0, - None, - &actor.signer, - PlatformVersion::latest(), - None, - ) - .await - .expect("expected to build the replacement") -} - fn assert_rule_broken(execution: &StateTransitionExecutionResult, rule: &str) { assert!( matches!( diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests/typed_array_where.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests/typed_array_where.rs new file mode 100644 index 00000000000..c291abff943 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/contract_user_moderation/tests/typed_array_where.rs @@ -0,0 +1,200 @@ +//! `where` entries over typed arrays through a `moderatedDocument` reference +//! (protocol version 14). Once a moderator removes the post an echo names, a +//! replace of the echo checks the entries it asks about against the post's +//! removal record, which keeps the post's `tags` and `members`: the equality of +//! the echo's `tags` with the post's when the replace changes them, and the +//! `contains` of the writer among the post's `members` on every replace. + +use super::*; + +const TAGGED_POST: &str = "taggedPost"; +const ECHO: &str = "echo"; + +fn string_list(position: u64) -> Value { + platform_value!({ + "type": "array", + "items": { "type": "string", "minLength": 1, "maxLength": 63 }, + "maxItems": 4, + "position": position, + }) +} + +/// A post only moderators take down, each removal keeping its tags and its +/// members +fn tagged_post_schema() -> Value { + platform_value!({ + "type": "object", + "documentsMutable": false, + "canBeDeleted": false, + "moderatorAbilities": { "delete": true, "deleteKeepsFields": ["tags", "members"] }, + "properties": { + "tags": string_list(0), + "members": { + "type": "array", + "items": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + }, + "maxItems": 4, + "position": 1, + }, + }, + "additionalProperties": false, + }) +} + +/// An echo of a post's tags, written only by one of its members +fn echo_schema() -> Value { + platform_value!({ + "type": "object", + "documentsMutable": true, + "canBeDeleted": false, + "properties": { + "postId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0, + "refersTo": { + "type": "moderatedDocument", + "documentType": TAGGED_POST, + "where": { + "tags": "tags", + "members": { "contains": "$ownerId" }, + }, + }, + }, + "tags": string_list(1), + "note": { "type": "string", "maxLength": 50, "position": 2 }, + }, + "required": ["postId"], + "additionalProperties": false, + }) +} + +async fn setup() -> Setup { + Setup::new_at_with( + Some(moderators_without_lists()), + PlatformVersion::latest(), + |contract| { + add_document_type(contract, TAGGED_POST, tagged_post_schema()); + add_document_type(contract, ECHO, echo_schema()); + }, + ) + .await +} + +fn texts(texts: &[&str]) -> Value { + Value::Array( + texts + .iter() + .map(|text| Value::Text(text.to_string())) + .collect(), + ) +} + +fn assert_where_refused(execution: &StateTransitionExecutionResult, referring: &str) { + assert!( + matches!( + execution, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError( + StateError::ReferencedDocumentPropertyMismatchError(e) + ), + .. + } if e.referring_property() == referring + ), + "expected the where entry {referring} to refuse the write, got {execution:?}" + ); +} + +/// The record keeps the post's tags and members: an echo leaving its tags +/// alone is asked only whether its writer is still a member, and one changing +/// them must still echo the tags the record kept, in their order. A list the +/// post held none of is kept as the empty one. +#[tokio::test] +async fn should_compare_lists_with_a_removal_record() { + let setup = setup().await; + let transaction = setup.platform.drive.grove.start_transaction(); + let member = Value::Array(vec![Value::Identifier(setup.stranger.id().to_buffer())]); + let (post, create_post) = create_with( + &setup, + &setup.user, + TAGGED_POST, + vec![ + ("tags", texts(&["dash", "news"])), + ("members", member.clone()), + ], + ) + .await; + assert_success(&setup.process(&create_post, &transaction)); + let (untagged, create_untagged) = + create_with(&setup, &setup.user, TAGGED_POST, vec![("members", member)]).await; + assert_success(&setup.process(&create_untagged, &transaction)); + + let (_, outsider) = create_with( + &setup, + &setup.user, + ECHO, + vec![ + ("postId", Value::Identifier(post.id().to_buffer())), + ("tags", texts(&["dash", "news"])), + ], + ) + .await; + assert_where_refused( + &setup.process(&outsider, &transaction), + "{ \"contains\": \"$ownerId\" }", + ); + let (mut echo, create_echo) = create_with( + &setup, + &setup.stranger, + ECHO, + vec![ + ("postId", Value::Identifier(post.id().to_buffer())), + ("tags", texts(&["dash", "news"])), + ], + ) + .await; + assert_success(&setup.process(&create_echo, &transaction)); + let (mut untagged_echo, create_untagged_echo) = create_with( + &setup, + &setup.stranger, + ECHO, + vec![("postId", Value::Identifier(untagged.id().to_buffer()))], + ) + .await; + assert_success(&setup.process(&create_untagged_echo, &transaction)); + + for removed in [post.id(), untagged.id()] { + let remove = setup + .moderate(&setup.moderator, delete_action(TAGGED_POST, removed)) + .await; + assert_success(&setup.process(&remove, &transaction)); + } + + // The writer is asked about on every replace, and is still a kept member + echo.set("note", "still here".into()); + let edit = replacement(&setup.stranger, &setup.contract, ECHO, &mut echo).await; + assert_success(&setup.process(&edit, &transaction)); + + // Changed tags are compared with the kept ones, order and all + let mut reordered = echo.clone(); + reordered.set("tags", texts(&["news", "dash"])); + let refused = replacement(&setup.stranger, &setup.contract, ECHO, &mut reordered).await; + assert_where_refused(&setup.process(&refused, &transaction), "tags"); + + // The untagged post's record keeps no tags, the empty list an empty one equals + untagged_echo.set("tags", texts(&[])); + let emptied = replacement(&setup.stranger, &setup.contract, ECHO, &mut untagged_echo).await; + assert_success(&setup.process(&emptied, &transaction)); + let mut tagged = untagged_echo.clone(); + tagged.set("tags", texts(&["dash"])); + let refused = replacement(&setup.stranger, &setup.contract, ECHO, &mut tagged).await; + assert_where_refused(&setup.process(&refused, &transaction), "tags"); +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs index 33c4e7dcc80..14a96fe1fb9 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_common/data_contract_reference_validation/v0/mod.rs @@ -1,17 +1,18 @@ use dpp::block::block_info::BlockInfo; use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::data_contract::document_type::accessors::{DocumentTypeV0Getters, DocumentTypeV2Getters}; +use dpp::data_contract::document_type::array::TypedArrayProperty; use dpp::data_contract::document_type::class_methods::{ referenced_peer_kind_mismatch, referenced_reads_error, where_condition_literal_error, }; use dpp::data_contract::document_type::property_constraints::referenced_fields; use dpp::data_contract::document_type::{ is_path_listed, is_referenced_system_agreement_property, - is_referring_system_agreement_property, is_transient, DocumentProperty, + is_referring_system_agreement_property, is_transient, typed_array_property, DocumentProperty, DocumentPropertyReferenceTarget, DocumentPropertyType, DocumentReferenceDeclaration, DocumentReferenceKind, DocumentTypeRef, KeyReferenceIdentityProperty, PreallocatedKeySource, - PropertyReference, ReferenceHolder, WhereCondition, WhereLiteralKind, WhereNotEqualBound, - WhereOrderingBound, MAX_INDEX_SIZE, + PropertyReference, ReferenceHolder, WhereCondition, WhereContainsBound, WhereLiteralKind, + WhereNotEqualBound, WhereOrderingBound, MAX_INDEX_SIZE, }; use dpp::data_contract::schema::DataContractSchemaMethodsV0; use dpp::data_contract::DataContract; @@ -170,6 +171,112 @@ fn preallocated_index_keyed_by( .map(|index| index.name.clone()) } +/// Why the elements of the referenced typed array `referenced` hold no value a +/// `where` entry over lists compares, `None` when they do: an integer, a +/// boolean, a string, an identifier or a byte array, the kinds a literal reads +/// as ([`WhereLiteralKind::of`]). A float element is refused, as a float is +/// for every other entry that is not an equality. +fn list_element_error(referenced: &TypedArrayProperty) -> Option<&'static str> { + WhereLiteralKind::of(&referenced.item_type) + .is_none() + .then_some("the referenced typed array holds floats, which a where entry does not compare") +} + +/// Why the typed array `referring_property` of the declaring `document_type`, +/// `referring`, can not be compared as a list with the referenced typed array +/// `referenced` (an equality, or the bound of a `containsAll`), `None` when it +/// can: it is stored, as the referenced one is (a list is compared with a list +/// the referring document keeps), and its elements are of the referenced +/// list's value kind, one a `where` entry compares. +fn list_pair_error( + document_type: DocumentTypeRef, + referring_property: &str, + referring: &TypedArrayProperty, + referenced: &TypedArrayProperty, +) -> Option<&'static str> { + if is_transient(document_type, referring_property) { + return Some( + "the referring typed array is transient or inside a transient object: a list is \ + compared with a list the referring document stores", + ); + } + if let Some(reason) = list_element_error(referenced) { + return Some(reason); + } + (!same_value_kind(&referring.item_type, &referenced.item_type)).then_some( + "the elements of the two typed arrays must share one value kind: lists of different \ + kinds would never compare equal", + ) +} + +/// Why a `contains` or a `containsAll` keyed by `referenced_property` of +/// `referenced_document_type`, on the reference carried by +/// `reference_property` of the declaring `document_type`, can not hold, `None` +/// when it can (and for every other condition). The referenced side is a typed +/// array whose elements a `where` entry compares ([`list_element_error`]). A +/// `contains` is bounded by a property of the declaring type holding one value +/// of the element kind, or by the writer among identifier elements; a +/// `containsAll` by a typed array of the declaring type it can be compared +/// with ([`list_pair_error`]). Neither is bounded by the reference's own +/// value, which found the document. +fn list_condition_error( + document_type: DocumentTypeRef, + reference_property: Option<&str>, + referenced_document_type: DocumentTypeRef, + referenced_property: &str, + condition: &WhereCondition, +) -> Option<&'static str> { + let Some(referenced) = typed_array_property( + referenced_document_type.flattened_properties(), + referenced_property, + ) else { + return Some( + "contains and containsAll read the elements of a typed array, and the referenced \ + property is none", + ); + }; + if let Some(reason) = list_element_error(referenced) { + return Some(reason); + } + let referring_property = match condition { + WhereCondition::Contains(WhereContainsBound::Writer) => { + return (WhereLiteralKind::of(&referenced.item_type) + != Some(WhereLiteralKind::Identifier)) + .then_some( + "$ownerId, the writer, is an identifier, so the referenced typed array must hold \ + identifiers", + ); + } + WhereCondition::Contains(WhereContainsBound::Property(path)) + | WhereCondition::ContainsAll(path) => path, + _ => return None, + }; + if Some(referring_property.as_str()) == reference_property { + return Some("the referring property cannot be the reference property itself"); + } + let Some(referring) = document_type.flattened_properties().get(referring_property) else { + return Some("the declaring document type does not define the referring property"); + }; + match (condition, &referring.property_type) { + (WhereCondition::ContainsAll(_), DocumentPropertyType::TypedArray(referring_list)) => { + list_pair_error( + document_type, + referring_property, + referring_list, + referenced, + ) + } + (WhereCondition::ContainsAll(_), _) => Some( + "containsAll reads a typed array of the declaring document type, whose every \ + element the referenced typed array must hold", + ), + (_, referring_type) => (!same_value_kind(referring_type, &referenced.item_type)).then_some( + "contains reads one value of the kind the referenced typed array's elements \ + are", + ), + } +} + /// Checks every reference declaration of the given contract that carries /// declaration content. /// @@ -768,7 +875,9 @@ fn validate_reference_target_declaration_v0( // `where` declarations: both sides must exist, be // plain values (not containers), and share one value kind — a // cross-kind equality could never be satisfied and would brick - // every create of the declaring document type. The referenced + // every create of the declaring document type. Two stored typed + // arrays of one element kind, not floats, may instead be paired, and + // are equal as lists; a typed array pairs with nothing else. The referenced // side may instead be one of the referenced document's // `$ownerId` and `$creatorId` system identifiers, which then // must face an identifier on the referring side; `$creatorId` @@ -867,17 +976,32 @@ fn validate_reference_target_declaration_v0( "agreement properties must be plain values, not object containers", )); } - // The write-time check compares single values, which a list is - // not, so an agreement on one would never hold - if matches!(referring_type, DocumentPropertyType::TypedArray(_)) - || matches!( - referenced.property_type, - DocumentPropertyType::TypedArray(_) - ) - { - return Ok(invalid( - "agreement properties must be single values, not typed arrays", - )); + // Two typed arrays are equal as lists, element by element, which no + // single value is. No index is keyed through a pair of lists: the + // preallocation bindings and `summableOffCountIndex` derivations bind + // no typed array, so the tree key check below is not theirs + match (referring_type, &referenced.property_type) { + ( + DocumentPropertyType::TypedArray(referring_list), + DocumentPropertyType::TypedArray(referenced_list), + ) => { + if let Some(reason) = list_pair_error( + document_type, + referring_property, + referring_list, + referenced_list, + ) { + return Ok(invalid(reason)); + } + continue; + } + (DocumentPropertyType::TypedArray(_), _) | (_, DocumentPropertyType::TypedArray(_)) => { + return Ok(invalid( + "a typed array is equal only to a typed array, compared as a list: a \ + single value never is one", + )); + } + _ => {} } if !same_value_kind(referring_type, &referenced.property_type) { return Ok(invalid( @@ -926,13 +1050,26 @@ fn validate_reference_target_declaration_v0( // the reference's own value, the writer an identifier; an ordering compares // two integers, its bound an integer property of the declaring type, an // integer some value of the referenced property is on the right side of, or - // a time or height that type records. Through a `moderatedDocument` - // reference, an entry asked about on every replace compares only what every - // removal record holds, as a writer gate does: a time or height bound - // changes without a write, and once a moderator removes the document only - // its record is left. Inert before protocol version 14, which alone reaches - // this module (contract create and update state validation 1), and the only - // one whose parse reads such an entry + // a time or height that type records. A `contains` or a `containsAll` + // compares the elements of a typed array instead ([`list_condition_error`]), + // and the `const`, `in`, `notEqual` and orderings read no list. Through a + // `moderatedDocument` reference, an entry asked about on every replace + // compares only what every removal record holds, as a writer gate does: a + // time or height bound or the writer changes without a write, and once a + // moderator removes the document only its record is left. Inert before + // protocol version 14, which alone reaches this module (contract create and + // update state validation 1), and the only one whose parse reads such an + // entry + let unkept_on_removal = |condition: &WhereCondition, referenced_property: &str| { + (kind == DocumentReferenceKind::Moderated + && condition.is_rechecked_on_every_replace() + && !kept_by_every_record(referenced_document_type, referenced_property)) + .then_some( + "the entry is checked on every replace, and once a moderator removes the document \ + only its record is left, which keeps its $id, its $ownerId and the fields its type \ + lists under deleteKeepsFields: compare one of those", + ) + }; for (referenced_property, condition) in where_conditions { let invalid = |reason: &str| { SimpleConsensusValidationResult::new_with_error( @@ -953,6 +1090,20 @@ fn validate_reference_target_declaration_v0( )? { return Ok(invalid(reason)); } + if condition.reads_a_list() { + let reason = list_condition_error( + document_type, + reference_property, + referenced_document_type, + referenced_property, + condition, + ) + .or_else(|| unkept_on_removal(condition, referenced_property)); + if let Some(reason) = reason { + return Ok(invalid(reason)); + } + continue; + } let referenced_type = referenced_document_type .flattened_properties() .get(referenced_property) @@ -1052,16 +1203,15 @@ fn validate_reference_target_declaration_v0( WhereOrderingBound::Constant(_) => {} } } + WhereCondition::Contains(_) | WhereCondition::ContainsAll(_) => { + return Err(Error::Execution(ExecutionError::CorruptedCodeExecution( + "a contains or a containsAll is checked against the elements of its list \ + before the conditions comparing one value", + ))); + } } - if kind == DocumentReferenceKind::Moderated - && condition.is_rechecked_on_every_replace() - && !kept_by_every_record(referenced_document_type, referenced_property) - { - return Ok(invalid( - "the entry is checked on every replace, and once a moderator removes the \ - document only its record is left, which keeps its $id, its $ownerId and the \ - fields its type lists under deleteKeepsFields: compare one of those", - )); + if let Some(reason) = unkept_on_removal(condition, referenced_property) { + return Ok(invalid(reason)); } } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs index 0c15483ea79..ce19f3c81c4 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs @@ -5850,6 +5850,7 @@ mod tests { use dpp::platform_value::string_encoding::Encoding; use dpp::tests::json_document::json_document_to_json_value; use drive::util::test_helpers::setup_contract; + use serde_json::json; const FOREIGN_CONTRACT_PATH: &str = "tests/supporting_files/contract/reference-validation/reference-validation-contract-permanent-doc-foreign.json"; @@ -5881,6 +5882,18 @@ mod tests { fixture_path: &str, foreign_contract_path: &str, config: DataContractConfig, + ) -> StateTransitionExecutionResult { + let contract_json = json_document_to_json_value(fixture_path) + .expect("expected to read the fixture contract"); + run_contract_json_create(contract_json, foreign_contract_path, config).await + } + + /// [`run_contract_create_with_foreign_and_config`] for the contract + /// `contract_json`, such as a fixture changed by the test. + async fn run_contract_json_create( + mut contract_json: serde_json::Value, + foreign_contract_path: &str, + config: DataContractConfig, ) -> StateTransitionExecutionResult { let platform_version = PlatformVersion::latest(); let mut platform = TestPlatformBuilder::new() @@ -5903,8 +5916,6 @@ mod tests { // The config goes in before the parse, since a type may use what it // declares (moderation) - let mut contract_json = json_document_to_json_value(fixture_path) - .expect("expected to read the fixture contract"); contract_json["config"] = serde_json::to_value(&config).expect("expected the config to serialize"); let data_contract = DataContract::from_json( @@ -6322,17 +6333,40 @@ mod tests { ); } - /// An agreement is checked at write time by comparing single values, - /// which a typed array is not, so one between two typed arrays of the - /// same element type is refused at registration rather than refusing - /// every write that carries them. + /// Two typed arrays of one element kind are equal as lists, element by + /// element, so an agreement between them registers. #[tokio::test] - async fn should_reject_agreement_on_typed_array_properties() { + async fn should_register_an_agreement_between_typed_arrays_of_one_element_kind() { let result = run_contract_create( "tests/supporting_files/contract/reference-validation/reference-validation-contract-agreement-typed-array.json", ) .await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. } + ); + } + + const WHERE_TYPED_ARRAYS_CONTRACT_PATH: &str = "tests/supporting_files/contract/reference-validation/reference-validation-contract-where-typed-arrays.json"; + + /// Creates the typed array `where` fixture after `change`, under the + /// default config, or `config`. + async fn run_where_typed_arrays_create( + change: impl FnOnce(&mut serde_json::Value), + config: Option, + ) -> StateTransitionExecutionResult { + let mut contract_json = json_document_to_json_value(WHERE_TYPED_ARRAYS_CONTRACT_PATH) + .expect("expected to read the fixture contract"); + change(&mut contract_json); + let config = config.unwrap_or_else(|| { + DataContractConfig::default_for_version(PlatformVersion::latest()) + .expect("default config available") + }); + run_contract_json_create(contract_json, FOREIGN_CONTRACT_PATH, config).await + } + + fn assert_where_entry_refused(result: &StateTransitionExecutionResult, reason: &str) { assert_matches!( result, StateTransitionExecutionResult::PaidConsensusError { @@ -6340,7 +6374,200 @@ mod tests { StateError::ReferencedDocumentPropertyAgreementInvalidError(error) ), .. - } if error.reason().contains("not typed arrays") + } if error.reason().contains(reason), + "expected the where entry refused for {reason:?}, got {result:?}" + ); + } + + /// The `where` of a document type's `postId` reference. + fn post_reference_where<'a>( + contract_json: &'a mut serde_json::Value, + document_type: &str, + ) -> &'a mut serde_json::Value { + &mut contract_json["documentSchemas"][document_type]["properties"]["postId"]["refersTo"] + ["where"] + } + + /// An equality between two typed arrays, a `contains` of a property + /// and of the writer, and a `containsAll` register. + #[tokio::test] + async fn should_register_where_entries_over_typed_arrays() { + let result = run_where_typed_arrays_create(|_| {}, None).await; + + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. } + ); + } + + /// Lists compare element by element, so the elements must be of one + /// value kind a `where` entry compares, and a typed array is equal to + /// nothing else. + #[tokio::test] + async fn should_reject_lists_a_where_equality_can_not_compare() { + let result = run_where_typed_arrays_create( + |contract| { + contract["documentSchemas"]["like"]["properties"]["tags"]["items"] = + json!({ "type": "integer", "minimum": 0, "maximum": 10 }); + }, + None, + ) + .await; + assert_where_entry_refused(&result, "share one value kind"); + + let result = run_where_typed_arrays_create( + |contract| { + *post_reference_where(contract, "like") = json!({ "tags": "note" }); + }, + None, + ) + .await; + assert_where_entry_refused(&result, "equal only to a typed array"); + + let result = run_where_typed_arrays_create( + |contract| { + let ratings = |position: u32| { + json!({ + "type": "array", + "items": { "type": "number" }, + "maxItems": 4, + "position": position + }) + }; + contract["documentSchemas"]["post"]["properties"]["ratings"] = ratings(4); + contract["documentSchemas"]["like"]["properties"]["ratings"] = ratings(3); + *post_reference_where(contract, "like") = json!({ "ratings": "ratings" }); + }, + None, + ) + .await; + assert_where_entry_refused(&result, "floats"); + + // The list compared is one the referring document stores + let result = run_where_typed_arrays_create( + |contract| { + contract["documentSchemas"]["like"]["transient"] = json!(["tags"]); + }, + None, + ) + .await; + assert_where_entry_refused(&result, "transient"); + } + + /// A `contains` reads a list and is bounded by one value of its + /// element kind (the writer for identifiers), never by the reference + /// itself; a `containsAll` is bounded by a list of the same kind; and + /// the conditions comparing one value read no list. + #[tokio::test] + async fn should_reject_list_conditions_that_can_not_hold() { + for (document_type, where_value, reason) in [ + ( + "tagged", + json!({ "tags": { "contains": "count" } }), + "contains reads one value", + ), + ( + "tagged", + json!({ "label": { "contains": "tag" } }), + "the referenced property is none", + ), + ( + "tagged", + json!({ "members": { "contains": "postId" } }), + "the reference property itself", + ), + ( + "tagged", + json!({ "tags": { "contains": "missing" } }), + "does not define the referring property", + ), + ( + "memberNote", + json!({ "tags": { "contains": "$ownerId" } }), + "must hold identifiers", + ), + ( + "subset", + json!({ "tags": { "containsAll": "postId" } }), + "the reference property itself", + ), + ( + "tagged", + json!({ "tags": { "containsAll": "tag" } }), + "containsAll reads a typed array", + ), + ( + "copy", + json!({ "tags": { "containsAll": "members" } }), + "share one value kind", + ), + ( + "tagged", + json!({ "tags": { "const": "news" } }), + "holds no single value", + ), + ( + "tagged", + json!({ "tags": { "notEqual": "tag" } }), + "holds no single value", + ), + ] { + let result = run_where_typed_arrays_create( + |contract| { + *post_reference_where(contract, document_type) = where_value.clone(); + }, + None, + ) + .await; + assert_where_entry_refused(&result, reason); + } + } + + /// Through a `moderatedDocument` reference, a `contains` of the writer + /// is asked about on every replace, so once a moderator removes the + /// post it reads the removal record, which must keep the list. + #[tokio::test] + async fn should_reject_a_moderated_contains_of_the_writer_the_record_does_not_keep() { + let moderated = |kept: serde_json::Value| { + move |contract: &mut serde_json::Value| { + let post = &mut contract["documentSchemas"]["post"]; + post["moderatorAbilities"] = + json!({ "delete": true, "deleteKeepsFields": kept }); + contract["documentSchemas"]["memberNote"]["properties"]["postId"]["refersTo"] + ["type"] = json!("moderatedDocument"); + // The other references stay permanent, into another post type + let mut permanent_post = contract["documentSchemas"]["post"].clone(); + if let Some(schema) = permanent_post.as_object_mut() { + schema.remove("moderatorAbilities"); + } + contract["documentSchemas"]["permanentPost"] = permanent_post; + for document_type in ["like", "copy", "tagged", "subset", "scoreSubset"] { + contract["documentSchemas"][document_type]["properties"]["postId"] + ["refersTo"]["documentType"] = json!("permanentPost"); + } + } + }; + let config = || { + Some( + DataContractConfig::default_for_version(PlatformVersion::latest()) + .expect("default config available") + .with_moderation(Some(ContractModerationConfig { + banlist: false, + suspensions: false, + moderators: ContractModerators::ContractOwner, + warnings: false, + })), + ) + }; + + let result = run_where_typed_arrays_create(moderated(json!(["tags"])), config()).await; + assert_where_entry_refused(&result, "checked on every replace"); + + let result = + run_where_typed_arrays_create(moderated(json!(["members"])), config()).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. } ); } diff --git a/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-where-typed-arrays.json b/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-where-typed-arrays.json new file mode 100644 index 00000000000..c078213b088 --- /dev/null +++ b/packages/rs-drive-abci/tests/supporting_files/contract/reference-validation/reference-validation-contract-where-typed-arrays.json @@ -0,0 +1,268 @@ +{ + "$formatVersion": "1", + "id": "8Bqs6itzfoDXzmgQibYZQABbqYsXmawVf7SKe3mKDQVg", + "ownerId": "2b994p95akyNFKtkDnDvBRUotDbkH54MHwGbhQLr5gcU", + "version": 1, + "documentSchemas": { + "post": { + "type": "object", + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "tags": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "maxLength": 63 + }, + "maxItems": 4, + "uniqueItems": true, + "position": 0 + }, + "members": { + "type": "array", + "items": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier" + }, + "maxItems": 4, + "position": 1 + }, + "scores": { + "type": "array", + "items": { + "type": "integer", + "minimum": 0, + "maximum": 1000 + }, + "maxItems": 4, + "position": 2 + }, + "label": { + "type": "string", + "maxLength": 63, + "position": 3 + } + }, + "additionalProperties": false + }, + "like": { + "type": "object", + "documentsMutable": true, + "properties": { + "postId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0, + "refersTo": { + "type": "permanentDocument", + "documentType": "post", + "where": { + "tags": "tags" + } + } + }, + "tags": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "maxLength": 63 + }, + "maxItems": 4, + "uniqueItems": true, + "position": 1 + }, + "note": { + "type": "string", + "maxLength": 63, + "position": 2 + } + }, + "required": ["postId"], + "additionalProperties": false + }, + "copy": { + "type": "object", + "documentsMutable": false, + "properties": { + "postId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0, + "refersTo": { + "type": "permanentDocument", + "documentType": "post", + "where": { + "members": "members", + "scores": "scores" + } + } + }, + "members": { + "type": "array", + "items": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier" + }, + "maxItems": 4, + "position": 1 + }, + "scores": { + "type": "array", + "items": { + "type": "integer", + "minimum": 0, + "maximum": 1000 + }, + "maxItems": 4, + "position": 2 + } + }, + "required": ["postId"], + "additionalProperties": false + }, + "tagged": { + "type": "object", + "documentsMutable": false, + "properties": { + "postId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0, + "refersTo": { + "type": "permanentDocument", + "documentType": "post", + "where": { + "tags": { "contains": "tag" } + } + } + }, + "tag": { + "type": "string", + "minLength": 1, + "maxLength": 63, + "position": 1 + }, + "count": { + "type": "integer", + "minimum": 0, + "maximum": 1000, + "position": 2 + } + }, + "required": ["postId"], + "additionalProperties": false + }, + "memberNote": { + "type": "object", + "documentsMutable": false, + "properties": { + "postId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0, + "refersTo": { + "type": "permanentDocument", + "documentType": "post", + "where": { + "members": { "contains": "$ownerId" } + } + } + }, + "text": { + "type": "string", + "maxLength": 63, + "position": 1 + } + }, + "required": ["postId"], + "additionalProperties": false + }, + "scoreSubset": { + "type": "object", + "documentsMutable": false, + "properties": { + "postId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0, + "refersTo": { + "type": "permanentDocument", + "documentType": "post", + "where": { + "scores": { "containsAll": "scores" } + } + } + }, + "scores": { + "type": "array", + "items": { + "type": "integer", + "minimum": 0, + "maximum": 1000 + }, + "maxItems": 4, + "position": 1 + } + }, + "required": ["postId"], + "additionalProperties": false + }, + "subset": { + "type": "object", + "documentsMutable": false, + "properties": { + "postId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "contentMediaType": "application/x.dash.dpp.identifier", + "position": 0, + "refersTo": { + "type": "permanentDocument", + "documentType": "post", + "where": { + "tags": { "containsAll": "tags" } + } + } + }, + "tags": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "maxLength": 63 + }, + "maxItems": 4, + "position": 1 + } + }, + "required": ["postId"], + "additionalProperties": false + } + } +} diff --git a/packages/rs-platform-version/src/version/fee/document_reference/mod.rs b/packages/rs-platform-version/src/version/fee/document_reference/mod.rs index f37291af39a..049a1ab8a7c 100644 --- a/packages/rs-platform-version/src/version/fee/document_reference/mod.rs +++ b/packages/rs-platform-version/src/version/fee/document_reference/mod.rs @@ -14,11 +14,17 @@ pub mod v1; /// with: a signer who could not pay for a full scan is kept out of the mempool rather than /// refused unpaid in a block, with its nonce left to replay. /// -/// Every earlier schedule carries the same group; it is unread there because `chooseFirst` -/// does not parse before protocol version 14. +/// A `where` entry comparing typed arrays (an equality between two of them, `contains` or +/// `containsAll`) is billed by the list elements it reads, before it compares them. +/// +/// Every earlier schedule carries the same group; it is unread there because neither +/// `chooseFirst` nor a `where` entry parses before protocol version 14. #[derive(Clone, Debug, Encode, Decode, Default, PartialEq, Eq)] pub struct FeeDocumentReferenceVersion { /// The credits per competitor a `chooseFirst` may read that bound what its scan bills, and /// what check tx reserves for it. pub choose_first_credits_per_candidate: u64, + /// The credits per list element a `where` entry comparing typed arrays reads, on both + /// sides, against a document in state or a moderated document's removal record. + pub where_list_credits_per_element: u64, } diff --git a/packages/rs-platform-version/src/version/fee/document_reference/v1.rs b/packages/rs-platform-version/src/version/fee/document_reference/v1.rs index db0fa92dd77..ed14fed36a0 100644 --- a/packages/rs-platform-version/src/version/fee/document_reference/v1.rs +++ b/packages/rs-platform-version/src/version/fee/document_reference/v1.rs @@ -8,7 +8,12 @@ use crate::version::fee::document_reference::FeeDocumentReferenceVersion; /// so 100,000 covers it twice over. At the limit of 1,000 competitors the budget is 100,000,000 /// credits (0.001 DASH), which a signer must hold beyond the create's estimated fee to be /// admitted, and which a reveal is charged only as far as its scan actually costs. +/// +/// A list element a `where` entry reads costs a comparison, or a set insertion and a lookup +/// for `containsAll`: about a BLAKE3 block's work, which `blake3_per_block` bills at 300, so +/// 100 an element covers it. Two lists of 1,024 elements bill 204,800 credits. pub const FEE_DOCUMENT_REFERENCE_VERSION1: FeeDocumentReferenceVersion = FeeDocumentReferenceVersion { choose_first_credits_per_candidate: 100_000, + where_list_credits_per_element: 100, }; diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index 6d6029db145..6237dfc97d6 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -2576,6 +2576,47 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// `worst_case_sum_item_cost_size` 1). Applied fees are unchanged. Inert /// before this version: `GROVE_V1` to `GROVE_V3` keep the old sizing. /// +/// 94. **`where` entries over typed arrays**: a `refersTo` `where` (63) may +/// compare a referenced typed array as a list. An equality between two +/// typed arrays (`"tags": "tags"`, kept in `property_agreement`) holds for +/// the same elements in the same order; `{ "contains": b }` +/// (`WhereCondition::Contains`, appended) for a list holding an element +/// equal to `b`, a referring property of the element kind or `"$ownerId"` +/// (a writer gate on identifier elements, asked about on every replace); +/// `{ "containsAll": b }` (`WhereCondition::ContainsAll`, appended) for a +/// list holding every element of the referring typed array `b`. Elements +/// compare as two single values do (`where_lists_equal`, by +/// `Value::same_scalar_data`): an identifier equals its 32 bytes and an +/// integer is the same at any width; a list left out is the +/// empty one, so two left out are equal and every list holds all of one +/// left out, while a `contains` value left out is held by no list. Meta-schema +/// v3 and parser generation 3 (in place) read both keys, refusing them on +/// `$ownerId`, `$creatorId` and `$id` and a `containsAll` of the writer. +/// Contract create and update state validation (data contract reference +/// validation 0, in place) admits a typed array pair, until now refused +/// with 40126, when both sides are stored typed arrays whose elements share +/// one value kind other than a float, and refuses with 40126: a typed array +/// paired with anything else, a `contains` or `containsAll` of a property +/// that is no typed array, of float elements, bounded by the reference +/// property itself or by a value of another kind (the writer only among +/// identifiers), a `moderatedDocument` `contains` of the writer whose list +/// no removal record keeps. `const`, `in`, `notEqual` and the orderings +/// still refuse a typed array. Document reference validation 0 (in place) +/// checks them against the document found or the removal record, refusing +/// with 40127, and refuses an entry whose referring path is no object on +/// the way as a mismatch. Before comparing, it bills the list elements +/// such an entry reads, both lists for an equality and a `containsAll`, the +/// referenced one for a `contains`, at +/// `FEE_VERSION3.document_reference.where_list_credits_per_element` (100 +/// credits, a field added in place to `FEE_DOCUMENT_REFERENCE_VERSION1`, +/// which every schedule carries and only this version reads). `Index::preallocation_bindings` and +/// `Index::count_index_derivations` bind no index property that is a typed +/// array, so no preallocated or `summableOffCountIndex` index is keyed +/// through a pair of lists; inert while an indexOnly type, the only one +/// declaring either, indexes no typed array. Inert before this version: +/// only parser generation 3 reads a `where`, and this module and that +/// validation are reached from this version only. +/// /// The app-connect system contract (`SystemDataContract::AppConnect`, schema v1) /// carries only the wallet's `loginKeyResponse`: a flat indexOnly entry keyed by /// the app's ephemeral key hash and the responding identity, with the wallet's diff --git a/packages/wasm-dpp2/src/data_contract/document_type_reference.rs b/packages/wasm-dpp2/src/data_contract/document_type_reference.rs index 60a577ec2ea..353543802a5 100644 --- a/packages/wasm-dpp2/src/data_contract/document_type_reference.rs +++ b/packages/wasm-dpp2/src/data_contract/document_type_reference.rs @@ -18,8 +18,8 @@ use crate::utils::define_own_property; use dpp::data_contract::document_type::{ DocumentPropertyReferenceTarget, DocumentReferenceLookup, DocumentTypeRef, IdentityKeyReferenceRequirements, KeyIdReference, LookupHashKey, LookupKeyParam, - LookupKeySource, PropertyReference, WhereCondition, WhereLiteral, WhereNotEqualBound, - WhereOrderingBound, + LookupKeySource, PropertyReference, WhereCondition, WhereContainsBound, WhereLiteral, + WhereNotEqualBound, WhereOrderingBound, }; use dpp::document::property_names::{ID, OWNER_ID}; use dpp::prelude::Identifier; @@ -355,7 +355,17 @@ export type DocumentReferenceWhere = Record WasmDppResul } values.into() } + WhereCondition::Contains(WhereContainsBound::Property(property)) + | WhereCondition::ContainsAll(property) => JsValue::from_str(property), + WhereCondition::Contains(WhereContainsBound::Writer) => JsValue::from_str(OWNER_ID), }; let object = Object::new(); set_field(&object, condition.wire_name(), &value, path)?; diff --git a/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts b/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts index 047c0ecec66..8542f6f05bc 100644 --- a/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts +++ b/packages/wasm-dpp2/tests/unit/DocumentPropertyReference.spec.ts @@ -1049,6 +1049,99 @@ describe('DataContract — refersTo declarations (v14)', () => { }); }); + describe('where over typed arrays', () => { + const tags = (position: number) => ({ + type: 'array', + items: { type: 'string', minLength: 1, maxLength: 63 }, + maxItems: 4, + uniqueItems: true, + position, + }); + + /** + * A like copying its post's tags, and a note naming one of them, written + * only by one of the post's members. + */ + const listSchemas = { + post: { + type: 'object', + canBeDeleted: false, + documentsMutable: false, + properties: { + tags: tags(0), + members: { + type: 'array', + items: { + type: 'array', + byteArray: true, + minItems: 32, + maxItems: 32, + contentMediaType: 'application/x.dash.dpp.identifier', + }, + maxItems: 4, + position: 1, + }, + }, + additionalProperties: false, + }, + like: { + type: 'object', + properties: { + postId: identifierProperty(0, { + type: 'permanentDocument', + documentType: 'post', + where: { tags: 'tags' }, + }), + tags: tags(1), + }, + required: ['postId'], + additionalProperties: false, + }, + note: { + type: 'object', + properties: { + postId: identifierProperty(0, { + type: 'permanentDocument', + documentType: 'post', + where: { + members: { contains: '$ownerId' }, + tags: { containsAll: 'tags' }, + }, + }), + tags: tags(1), + }, + required: ['postId'], + additionalProperties: false, + }, + }; + + function buildListContract() { + return new wasm.DataContract({ + ownerId, + identityNonce: BigInt(4), + schemas: listSchemas, + definitions: null, + fullValidation: true, + platformVersion: new PlatformVersion(14), + }); + } + + it('should carry an equality between typed arrays as an equality', () => { + const [postId] = buildListContract().documentTypeReferences('like') as Reference[]; + + expect(postId.where).to.deep.equal({ tags: 'tags' }); + }); + + it('should carry contains and containsAll as the schema writes them', () => { + const [postId] = buildListContract().documentTypeReferences('note') as Reference[]; + + expect(postId.where).to.deep.equal({ + members: { contains: '$ownerId' }, + tags: { containsAll: 'tags' }, + }); + }); + }); + describe('ownerRefersTo and creatorRefersTo', () => { /** * A `resignation` may only be written by the owner of a join request for