Skip to content

match unsafe element names by local name and namespace, not name() - #218

Open
itecz26 wants to merge 2 commits into
scrapy:masterfrom
itecz26:unsafe-name-local-name
Open

itecz26 wants to merge 2 commits into
scrapy:masterfrom
itecz26:unsafe-name-local-name

Conversation

@itecz26

@itecz26 itecz26 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

an element name that is_safe_name rejects, a non-ascii one like título or an escaped one like di[v, is translated to a name() comparison. name() is the qualified name as written in the document, so a bare x-ä matches <x-ä> inside xmlns="urn:d" while the node test emitted for an ascii name only matches no-namespace elements, which is what the docs promise for both, and :not(x-ä) wrongly excludes those elements. with a prefix, ns|x-ä compares the document prefix as a string instead of resolving ns through the xpath prefix mapping like the ns:f node test does, so it misses <p:x-ä> bound to the same uri and matches ns:x-ä bound to another one; [ns|data-ä] in xpath_attrib has the same prefix problem. test local-name() and the namespace instead, mirroring the |* translation, and keep name() only for a prefix that cannot be a node test itself. the xpath expectations in test_translation change accordingly, and test_select_with_default_namespace and test_select_with_namespace gain selects that fail on master.

An element name that is_safe_name() rejects (non-ASCII, or escaped
characters) was emitted as a name() comparison. name() is the qualified
name as written in the document, so a bare "x-ä" matched elements in a
default namespace, unlike the node test emitted for an ASCII name, and
"ns|x-ä" compared the document prefix instead of resolving "ns" through
the XPath prefix mapping. Test local-name() and the namespace instead,
and do the same for a prefixed attribute name in xpath_attrib().
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (0a0c197) to head (f2ac28d).

Additional details and impacted files
@@            Coverage Diff            @@
##            master      #218   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            3         3           
  Lines         1019      1031   +12     
  Branches       170       174    +4     
=========================================
+ Hits          1019      1031   +12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AdrianAtZyte

Copy link
Copy Markdown
Contributor

Looks good to me, thanks.

  • I think unprefixed attribute names should get the same treatment: [p\:ö] still compares name(), so it matches a namespaced p:ö attribute, which is the same issue this fixes for elements. attribute::*[local-name() = … and namespace-uri() = ''] would make both consistent.
  • Please, flatten the xpath_attrib() branches into a single if/elif chain (no namespace / unsafe namespace / safe name / unsafe name), the nested else makes it hard to follow.
  • The new comment in xpath_element() could be a single sentence on what the code does.

@itecz26

itecz26 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

good catch on the unprefixed case. unprefixed names unusable in a name test now go through attribute::*[local-name() = ... and namespace-uri() = ''] as well, with a translation check for [p:ref] and a select check that [n:ö] doesn't match a namespaced n:ö attribute. xpath_attrib() is a flat if/elif chain now (wildcard / no namespace / unsafe prefix / safe name / unsafe name), and the xpath_element() comment is down to one sentence. also added a translation assert for the unsafe-prefix attribute branch since it lost its coverage once split out.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants