Skip to content

reject nested :has() in parse_simple_selector - #220

Closed
itecz26 wants to merge 1 commit into
scrapy:masterfrom
itecz26:has-reject-nested
Closed

itecz26 wants to merge 1 commit into
scrapy:masterfrom
itecz26:has-reject-nested

Conversation

@itecz26

@itecz26 itecz26 commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

nested :has() is accepted and its XPath blows up per level

selectors 4 says :has() can't be nested (browsers reject a:has(b:has(c))), but the parser takes it directly, after a combinator in the argument, or through :is()/:where()/:not(). each level becomes a predicate nested in the previous one, and libxml2 re-evaluates it for every node the outer axis reaches, so one short selector can stall anything that runs user-supplied selectors on fetched pages (parsel/scrapy-based scraping APIs). on lxml, i:has(~ i:has(~ i:has(~ x))) (28 chars) takes 1.5 s on a flat list of 300 <i/>, and *:has(*:has(*:has(*:has(x)))) (29 chars) takes 20 s on a 200-deep document; every extra level multiplies that again. these now raise SelectorSyntaxError("Got nested :has()") by threading the existing inside_relative_selector flag through the whole :has() argument; :scope was already rejected at all those positions so nothing else changes.

@codecov

codecov Bot commented Oct 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (0a0c197) to head (b7ed396).

Additional details and impacted files
@@            Coverage Diff            @@
##            master      #220   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            3         3           
  Lines         1019      1021    +2     
  Branches       170       171    +1     
=========================================
+ Hits          1019      1021    +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AdrianAtZyte

Copy link
Copy Markdown
Contributor

I am not sure about this one.

Given nested has() is invalid in the standard, supporting it does not conflict with the standard, and it works as one would expect. I think we should support extensions like that where possible.

@itecz26

itecz26 commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

fair point, the spec rule doesn't mean cssselect has to reject it. I checked master and nested :has() does match correctly: 23 shapes, including through :is()/:where()/:not(), against a brute-force matcher on random trees, no mismatches.

so all this PR really buys is cutting off the per-level cost. that could also be handled the way #219 did for :not(), by chaining steps instead of nesting predicates: flattening the 200-deep example by hand takes it from 22 s to 0.17 s. it wouldn't cover :has(a, b) at each level though, since XPath 1.0 can't branch mid-path. if you'd rather keep the extension as it is, I'm happy to close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants