Allow enabling versioning on buckets with a GCP location constraint

[maeldonn]

Not handled yet

Gap	Impact	What it would take
Object versioning on the GCS bucket is a hard requirement: without it, the delete issued behind a delete marker destroys the data instead of archiving it	Silent data loss	Block writes on a non-versioned bucket, instead of only reporting it in the healthcheck
GCS lifecycle rules on noncurrent versions (isLive: false, daysSinceNoncurrentTime, numNewerVersions) can reap generations that S3 still serves as current	Deferred and silent data loss, with no error at write time	Document the precondition, and consider detecting such rules at configuration time
GCS never promotes a noncurrent generation, so the backend state diverges from the S3 view after an undelete or a version delete	Confusing during support, and the root cause of the lifecycle exposure above	Optional: promote by copy, accepting the new generation, the non-atomicity and the storage class reset

[bert-e]

Hello maeldonn,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/bypass_source_branch_lineage	Bypass the cross-branch contamination check	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request.
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[bert-e]

Incorrect fix version

The Fix Version/s in issue CLDSRV-984 contains:

• None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

• 9.4.3

Please check the Fix Version/s of CLDSRV-984, or the target
branch of this pull request.

[codecov]

Codecov Report

❌ Patch coverage is 88.13559% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.45%. Comparing base (ff75a60) to head (cb95846).
⚠️ Report is 6 commits behind head on development/9.5.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
lib/api/bucketPutVersioning.js	87.71%	7 Missing ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
lib/api/apiUtils/object/createAndStoreObject.js	86.33% <100.00%> (-0.09%)	⬇️
lib/api/objectCopy.js	89.74% <100.00%> (ø)
lib/api/bucketPutVersioning.js	90.00% <87.71%> (-0.11%)	⬇️

... and 1 file with indirect coverage changes

@@                 Coverage Diff                 @@
##           development/9.5    #6266      +/-   ##
===================================================
- Coverage            86.50%   86.45%   -0.05%     
===================================================
  Files                  212      212              
  Lines                14578    14576       -2     
===================================================
- Hits                 12610    12601       -9     
- Misses                1968     1975       +7     

Flag	Coverage Δ
checksums-disabled-tests	35.29% <10.16%> (-0.01%)	⬇️
file-ft-tests	69.99% <81.35%> (-0.06%)	⬇️
file-ft-tests-null-compat	70.47% <81.35%> (-0.06%)	⬇️
kmip-ft-tests	28.05% <10.16%> (-0.01%)	⬇️
mongo-v0-ft-tests	71.08% <81.35%> (-0.01%)	⬇️
mongo-v1-ft-tests	71.08% <81.35%> (-0.04%)	⬇️
multiple-backend	36.07% <64.40%> (-0.01%)	⬇️
s3c-ft-tests-v0	65.01% <81.35%> (-0.01%)	⬇️
s3c-ft-tests-v0-null-compat	65.06% <81.35%> (-0.01%)	⬇️
s3c-ft-tests-v1	64.99% <81.35%> (-0.01%)	⬇️
sur-tests	36.60% <64.40%> (-0.01%)	⬇️
sur-tests-inflights	39.40% <64.40%> (-0.04%)	⬇️
unit	74.17% <81.35%> (-0.01%)	⬇️
utapi-v2-tests	35.28% <64.40%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[francoisferrand]

S3 has a suspended versioning state, with nullversion ids. GCS versioning is only on or off

Suspended versioning is really just a name : in our implementation it differs (so in particular performance may not be the same), but functionally suspended is about the same as off?
i.e. is suspended not just a different name for no versioning (no versioning → versioning ←→ suspended versioning), so we could map both no & suspended versioning on GCP's disabled versioning?

Object versioning on the GCS bucket is a hard requirement: without it, the delete issued behind a delete marker destroys the data instead of archiving it

• not sure what you mean, there is no "archiving" in S3...
• if using GCP versioning is not practical, another option would be to "flatten" the versions: i.e. store each version side-by-side, as <objKey>.<scalVersionId>

Block writes on a non-versioned bucket, instead of only reporting it in the healthcheck

If there a GCP "extension" to make the putObject/deleteObject conditional on versioning? Otherwise this may be very costly....

GCS lifecycle rules on noncurrent versions (isLive: false, daysSinceNoncurrentTime, numNewerVersions) can reap generations that S3 still serves as current

• Should be part of the documentation, and may be verified when creating location and/or enabling versioning (though none of these are foolproof, and we still risk a data loss)
• Or a reason to switch to "emulating" versions by flattening them....

GCS never promotes a noncurrent generation, so the backend state diverges from the S3 view after an undelete or a version delete

That is not really a problem for us, right: The issue is more about what the user would see in GCP's bucket?

All in all, I don't think we should merge this -and esp. in 9.4- but take the time to make a (short) design in Citadel : so we can align on the requirements and limits of the solution (i.e. do we want to keep the same "view" on GCP, or is this really just data storage, accessed only through the product ? what to do with GCP-side lifecycle rules ? ...)

[bert-e]

Incorrect fix version

The Fix Version/s in issue CLDSRV-984 contains:

• 9.4.3

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

• 9.5.0

Please check the Fix Version/s of CLDSRV-984, or the target
branch of this pull request.

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[maeldonn]

but functionally suspended is about the same as off?

Checked and you're right, nothing to do. I removed the line from the table.

there is no "archiving" in S3

GCS calls noncurrent versions "archived" (no delete markers, no promotion back to live, and lifecycle rules can target them via isLive: false / daysSinceNoncurrentTime / numNewerVersions)

another option would be to "flatten" the versions

Possible, and it doesn't even need the S3 version id: the backend key isn't recomputed on read, put returns it and it's stored per-version in location[0].key, so any unique suffix minted at put time works. Objects already written keep resolving at their plain key, so no migration either.

The cost is elsewhere: every path that recomputes the key via _createAwsKey instead of reading it from MD (whole MPU lifecycle, copyObject/uploadPartCopy, and toObjectGetInfo which the backbeat routes use with only objectKey + bucket), plus _shouldSkipDelete, which skips the old-version delete assuming overwrites are in-place. Real work, but a design-doc call rather than a blocker.

If there a GCP "extension" to make the putObject/deleteObject conditional on versioning?

No, GCS preconditions cover generation/metageneration only, nothing on versioning state. We already fail after the fact (no x-goog-generation -> missingVerIdInternalError), but too late, and delete markers are exempted.

Should be part of the documentation, and may be verified when creating location and/or enabling versioning

Agreed on documenting it. Verifying is doable but not free. We talk to GCS through the S3 interop API, so the AWS SDK is what sends our requests. It can call ?lifecycle, but GCS answers with its own XML format, which the SDK doesn't know how to read, so we'd have to parse that response ourselves. It's the same kind of adaptation code we already have in GcpApis/* for the other places where GCS and S3 differ. And checking only at location creation wouldn't help much, since rules can be added later and we'd never notice. Better to read it in the healthcheck we already run every 60s, together with the versioning state.

The issue is more about what the user would see in GCP's bucket?

Right, not a problem while the bucket is only used as a Scality backend: we always read a specific generation, never the live one. It only bites if the bucket is used without us in front (direct reads, OOB, and GCS lifecycle above all). So we need to pin down the supported usage and document it. Same drift on the AWS backend, so it's the mirrored-bucket model rather than GCS-specific.