Repository navigation
fix(oracle): respect version-specific identifier limits - #45801
shubhamjrd4559-sudo wants to merge 4 commits into
Conversation
| if ( | ||
| cls.max_column_name_length | ||
| and len(label_mutated) > cls.max_column_name_length | ||
| max_column_name_length | ||
| and len(label_mutated) > max_column_name_length | ||
| ): |
There was a problem hiding this comment.
Suggestion: Oracle enforces this legacy limit in bytes, but len() counts Unicode characters; multibyte labels under 30 characters pass through and can still fail with ORA-00972.
Assessment: 🟠 Major · 🔁 Occurrence: Rarely · 🏷️ Logic error
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** superset/db_engine_specs/base.py
**Line:** 2719:2722
**Comment:**
*Logic Error: Oracle enforces this legacy limit in bytes, but `len()` counts Unicode characters; multibyte labels under 30 characters pass through and can still fail with ORA-00972.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| and len(label_mutated) > max_column_name_length | ||
| ): | ||
| label_mutated = cls._truncate_label(label) | ||
| label_mutated = cls._truncate_label(label, max_column_name_length) |
There was a problem hiding this comment.
Suggestion: BigQueryEngineSpec and DatastoreEngineSpec override _truncate_label with one parameter, so over-limit labels now raise TypeError when this call passes two.
Assessment: 🟠 Major · 🔁 Occurrence: Sometimes · 🏷️ Api mismatch
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** superset/db_engine_specs/base.py
**Line:** 2723:2723
**Comment:**
*Api Mismatch: `BigQueryEngineSpec` and `DatastoreEngineSpec` override `_truncate_label` with one parameter, so over-limit labels now raise `TypeError` when this call passes two.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fixThere was a problem hiding this comment.
✅ CodeAnt verified this suggestion was addressed in subsequent commits and marked this thread resolved as of ef31e0d.
The truncation call now passes only label, matching the one-parameter overrides.
If that's not right, unresolve this thread and CodeAnt will leave it open.
There was a problem hiding this comment.
✅ CodeAnt verified this suggestion was addressed in subsequent commits and marked this thread resolved as of 83f9385.
The truncation call passes only label, matching the one-parameter _truncate_label overrides.
If that's not right, unresolve this thread and CodeAnt will leave it open.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #45801 +/- ##
=======================================
Coverage 84.75% 84.76%
=======================================
Files 3071 3071
Lines 206037 206064 +27
Branches 48506 48509 +3
=======================================
+ Hits 174625 174668 +43
+ Misses 28109 28098 -11
+ Partials 3303 3298 -5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
rebenitez1802
left a comment
There was a problem hiding this comment.
Request changes: aliases now fit Oracle's 30-byte limit before 12.2, and the SQL for 12.2+ and other engines is unchanged. But the PR has the same two regressions that were raised on #45675 (same #45758 item, opened earlier, and it already fixes both), and its unit tests fail once it's merged with current master. Details are inline.
| # Get the corresponding column from the subquery | ||
| subq_col_name = db_engine_spec.make_label_compatible( | ||
| col_name + "__" | ||
| col_name + "__", database=self.database | ||
| ) | ||
| # Reference the column from the already-created aliased subquery | ||
| subq_col = subq_alias.c[subq_col_name] |
There was a problem hiding this comment.
🔴 High — "Group remaining as Others" crashes when Oracle hashes a series label
make_sqla_column_compatible sets sqla_col.key = label_expected, so subq_alias.c is keyed by col_name + "__", not by the hashed alias. With a series column of 29 bytes or more on Oracle 11.2, this lookup therefore raises KeyError: '7e0a3e56f6524e25280e79b853efa5' while building the query. Users get a 500, and "View query" breaks too. The same happens when a database's Version is set to 12c; on a 12.2+ server set that way, this query works on master and crashes on this branch. Looking the column up by its key fixes it, and the 30-byte aliases stay intact:
| # Get the corresponding column from the subquery | |
| subq_col_name = db_engine_spec.make_label_compatible( | |
| col_name + "__" | |
| col_name + "__", database=self.database | |
| ) | |
| # Reference the column from the already-created aliased subquery | |
| subq_col = subq_alias.c[subq_col_name] | |
| # The subquery keys its columns by the expected label, which | |
| # differs from the SQL alias when the engine shortened it | |
| subq_col = subq_alias.c[col_name + "__"] |
| @classmethod | ||
| def get_max_column_name_length(cls, database: Database) -> int | None: | ||
| """Use Oracle's 30-byte identifier limit before release 12.2.""" | ||
| server_version = cls.get_server_version(database) |
There was a problem hiding this comment.
🔴 High — A failed version lookup is retried once per label
make_label_compatible now asks for the server version for every label (get_max_column_name_length → get_server_version), and a failed lookup isn't remembered. With no Version set and the database unreachable, building one chart query tries to connect 17–39 times, logging a traceback each time; master tries none while building the query. python-oracledb's default connect timeout is 20 s, so one chart can hang for about 340 s, well past the web server timeout. Fix: remember the failure for the current request (for example a set on flask.g, as #45675 does), so a chart makes at most one attempt and the next request still retries.
| db_engine_spec = self.db_engine_spec | ||
| series_column_labels = [ | ||
| db_engine_spec.make_label_compatible(column) | ||
| db_engine_spec.make_label_compatible(column, database=self.database) |
There was a problem hiding this comment.
🔴 High — Unit tests fail once merged with master
#45681 landed after this PR's CI run. It added test_cache_key_collects_series_metric_for_normalized_labels, which mocks make_label_compatible with side_effect=lambda name: .... Merged with current master, both of its parametrizations fail with TypeError: <lambda>() got an unexpected keyword argument 'database'; they pass on master. The branch merges without conflicts and the CI here is green, so this would only show up on master. Fix: merge master and change the mock in tests/unit_tests/connectors/sqla/virtual_dataset_connections_test.py to lambda name, database=None: ....
ef31e0d to
83f9385
Compare
|
Hi @rebenitez1802, thanks for the detailed review. I rebased this PR on the latest master and addressed the reported issues: the shortened series alias lookup, repeated Oracle version lookup failures within a request, and the test mock’s database argument. I’ve added regression tests, and CI is running on the updated commit. I also noticed #45675 covers the same issue—would you prefer I keep this PR open for re-review or close it in favor of #45675? |
SUMMARY
Oracle versions before 12.2 support identifiers up to 30 bytes, while Oracle 12.2 and later support up to 128. Superset currently applies the 128-character engine-spec limit to every Oracle version, which can produce ORA-00972 when generated chart aliases exceed the legacy limit.
This change makes the maximum alias length database-aware, uses 30 for Oracle releases before 12.2 and preserves the 128 limit for newer releases, and passes the database context through label generation. Other engine specs keep their existing class-level limit.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable.
TESTING INSTRUCTIONS
pytest tests/unit_tests/db_engine_specs/test_oracle.py -q.ADDITIONAL INFORMATION