[🍒] Ignore 409 Conflict on bucket creation retry in BigQuery plugin for release/0.24

[vishwasvaidya-cloudsufi]

[🍒]

🍒 cherrypick

PR : (#1624)

Summary

Fixes PLUGIN-1787 / b/539473185 where BigQuery Source intermittently fails with 409 Conflict: Your previous request to create the named bucket succeeded and you already own it when the GCS client retries temporary staging bucket creation after transient network timeouts.

Changes

• Refactored bucket creation in BigQuerySourceUtils.java into a shared createBucket helper for both auto-generated and user-configured buckets.
• Caught StorageException and ignored 409 Conflict so the pipeline proceeds normally when the bucket already exists.
• Non-409 errors (e.g. 403 Forbidden) continue to fail as expected.

Testing

• Unit Tests: Updated BigQuerySourceUtilsTest following Google Unit Testing Best Practices (AAA pattern, methodName_stateUnderTest_expectedBehavior, and Assert.assertThrows).
  • getOrCreateBucket_nullBucketConflict409_returnsBucketAndEnablesDelete: Verifies 409 retry response returns the bucket and keeps delete enabled.
  • getOrCreateBucket_nullBucketNon409Error_throwsProgramFailureException: Verifies non-409 exceptions are properly propagated.
• Validated end-to-end on CDAP instance with pipeline bq-409-conflict-resolution (completed successfully).

[gemini-code-assist]

Code Review

This pull request refactors the bucket creation logic in BigQuerySourceUtils by extracting it into a private helper method, adding null-safety checks for the dataset ID, and introducing debug logging for 409 conflicts. It also adds corresponding unit tests to cover 409 conflict and non-409 error scenarios. The review feedback suggests replacing the magic number 409 with HttpURLConnection.HTTP_CONFLICT for better readability, and adding an additional unit test to cover the 409 conflict scenario when a user-provided bucket name is used.