feat(storage): support storage_class in AsyncAppendableObjectWriter - #18289
feat(storage): support storage_class in AsyncAppendableObjectWriter#18289chandra-siri wants to merge 8 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for specifying a storage_class (specifically 'STANDARD' or 'RAPID') when asynchronously writing or appending to GCS objects. The feedback suggests importing the _SUPPORTED_STORAGE_CLASSES constant to avoid duplication, normalizing the storage_class input to uppercase for robustness, and updating the unit tests to verify this case-insensitive behavior.
Add support for specifying storage_class when writing objects via AsyncAppendableObjectWriter. - Add storage_class parameter to AsyncAppendableObjectWriter.__init__ and propagate it to _AsyncWriteObjectStream on open(). - Add storage_class parameter to _AsyncWriteObjectStream.__init__ and pass it to _storage_v2.Object creation. - Add unit tests covering initialization and open stream requests.
acb948c to
e2eab99
Compare
| RCU_SYSTEM_TESTS = os.getenv("RUN_RCU_SYSTEM_TESTS") == "True" | ||
| _RCU_BUCKET = os.getenv("RCU_BUCKET") | ||
| bucket_for_testing = ( | ||
| _ZONAL_BUCKET if os.getenv("RUN_ZONAL_SYSTEM_TESTS") else _RCU_BUCKET |
There was a problem hiding this comment.
I can see everywhere else we are explicitly checking True string, is it required here as well?
There was a problem hiding this comment.
yes, good catch. it should be - _ZONAL_BUCKET if os.getenv("RUN_ZONAL_SYSTEM_TESTS")==True else _RCU_BUCKET
but below is also makes sense and more readable.
_RCU_BUCKET if RCU_SYSTEM_TESTS else _ZONAL_BUCKET
| ) | ||
|
|
||
| async def create_async_grpc_client(attempt_direct_path=True): | ||
| async def create_async_grpc_client(attempt_direct_path=True, preprod=False): |
There was a problem hiding this comment.
are we exposing preprod testing setup?
There was a problem hiding this comment.
yes. any concerns ?
2 participants
Description
Adds support for specifying
storage_classwhen writing appendable objects viaAsyncAppendableObjectWriter, and updates system test configuration and zonal tests to support RCU buckets and preprod environments.Changes
Core Library:
storage_classparameter (defaulting toNone) inAsyncAppendableObjectWriter.__init__and propagated it to_AsyncWriteObjectStreamduringopen().storage_classparameter in_AsyncWriteObjectStream.__init__and set it on the_storage_v2.Objectresource when opening the stream.Unit Tests:
test_async_appendable_object_writer.pyandtest_async_write_object_stream.pycoveringstorage_classinitialization and stream opening.System Tests:
RUN_SYSTEM_TESTS_ON_PREPRODfixture support inconftest.py.RUN_RCU_SYSTEM_TESTS,RCU_BUCKET) alongside zonal bucket tests intest_zonal.py.AsyncAppendableObjectWriterinstantiations in zonal system tests to passstorage_class="RAPID" if RCU_SYSTEM_TESTS else Noneand usebucket_for_testing.skipifmarkers on tests for features not yet supported in SDK under RCU (cross-region, write from blob, KMS, contexts).