Surgical Django cache purge

[skaphan]

Summary

• Replaces cache.clear() with surgical purge that only deletes cache entries for the affected slug's URLs
• Uses thread-local middleware to capture request host for accurate cache key reconstruction
• Catches DisallowedHost instead of temporarily mutating settings.ALLOWED_HOSTS (addresses review concern from Fix server-side cache to work with conditional GET #608)
• Comprehensive test verifies all URL variants (including query-string vistypes) are purged

Follow-up optimization to #608. This is a nice-to-have, not critical — #608's cache.clear() is correct and safe.

Test plan

• test_purge_vis_cache_clears_all_cached_urls — populates cache for base, embedded, and query-string URLs, purges by slug, verifies all gone
• test_purge_django_cache_tries_www_variant — verifies www. domain variant is included in purge domains

🤖 Generated with Claude Code

The safest fix is to remove the unused assignment while preserving behavior.

• General approach: if a variable is assigned and never used, delete the assignment unless the right-hand side has required side effects.
• Here: reverse('visualize', args=(config.slug,)) is side-effect free in this context, and the resulting path is unused, so remove that line.
• File/region: visualizer/tests/testSimple.py, inside test_purge_django_cache_tries_www_variant, remove line with path = ....
• No imports/methods/definitions are needed.

[artoonie]

This PR sat stale for a while because I had concerns about accuracy and it was hard to test outside of prod.

Since then, Claude has come a long way.

Modern Claude has taken another pass at this and finds the following:

  The bug. The purge reconstructs each page's cache key with a bare HttpRequest() in _make_cache_request. Django's base HttpRequest._get_scheme() returns the literal string "http" and ignores the wsgi.url_scheme
  value the PR sets. Only the WSGI subclass reads that value. So every synthetic key is built from http://host/path. Production requests arrive as https: the live page's og:url meta tag, which the app builds
  from request.is_secure(), reads https://www.rcvis.com/..., so the real cache keys are https://.... Nothing matches, cache.delete deletes nothing, and no error is raised. The DisallowedHost catch adds a second
  way to skip silently.

  What that means for your requirement. After an update, Cloudflare is purged correctly, re-fetches from the origin, and the origin serves the stale page out of Django's file cache. Cloudflare then caches the
  stale page again. Today cache.clear() makes that impossible. With this PR, the stale window is the Django cache lifetime, 600 seconds on main. That is a regression in exactly the scenario you care about.

  Evidence. I ran the PR's own tests plus a probe against its branch under Django 4.2:

  ┌───────────────────────────────────────────────────────────────────────────┬─────────┐
  │                                 Scenario                                  │ Purged? │
  ├───────────────────────────────────────────────────────────────────────────┼─────────┤
  │ PR's test shape: example.com over http, purge outside a request           │ yes     │
  ├───────────────────────────────────────────────────────────────────────────┼─────────┤
  │ Production shape: viewer via www.rcvis.com over https, save from a worker │ no      │
  ├───────────────────────────────────────────────────────────────────────────┼─────────┤
  │ Same, but origin saw plain http                                           │ yes     │
  ├───────────────────────────────────────────────────────────────────────────┼─────────┤
  │ Listed URL with an extra query string such as ?utm_source=x               │ no      │
  └───────────────────────────────────────────────────────────────────────────┴─────────┘

  The PR's tests pass only because the Django test client speaks http. Its CI on Heroku is also failing, though I couldn't see the log.

  Secondary concerns.
  - The purge depends on a hardcoded list of 17 URL variants. Any URL outside it, including any query string, stays stale until expiry. The test docstring acknowledges this.
  - Under Django 5.2.16 the PR's main test fails, because it relies on the old no-cache responses being cached, which 5.1+ refuses. It also conflicts with cache-updates, which deletes visualizer/middleware.py.
  - The thread-local request middleware is workable but is machinery in the hot path for a nice-to-have.

  Is the benefit worth it? With Cloudflare absorbing viewers and the Django cache now living 300 seconds, cache.clear() on a save costs at most one extra graph recompute per page on its next origin miss. For
  live voting with saves every minute across ten pages, that is about ten recomputes per minute instead of one. The PR description itself calls this a nice-to-have. My recommendation is to close both #609 and
  #635 and keep cache.clear().

  If you do want surgical purging later, the reliable design is to record the real cache keys per slug at store time and delete those, rather than reconstructing them from guessed host, scheme, and URL lists. A
  minimal fix of the current approach would be building the synthetic request with RequestFactory and secure=True, plus a production-shaped test: https, www host, Site domain set, and no current request.

I have a draft of middleware here, but I need to review it in more detail before I'm confident in its correctness: #644

[skaphan]

Apologies, that appears to have been not very well conceived. I'd blame it on Claude but I should have been more vigilant at the time. Shel

…

On Sep 11, 2026, at 8:49 PM, Armin Samii ***@***.***> wrote: artoonie left a comment (artoonie/rcvis#609) <#609 (comment)> This PR sat stale for a while because I had concerns about accuracy and it was hard to test outside of prod. Since then, Claude has come a long way. Modern Claude has taken another pass at this and finds the following: The bug. The purge reconstructs each page's cache key with a bare HttpRequest() in _make_cache_request. Django's base HttpRequest._get_scheme() returns the literal string "http" and ignores the wsgi.url_scheme value the PR sets. Only the WSGI subclass reads that value. So every synthetic key is built from http://host/path. Production requests arrive as https: the live page's og:url meta tag, which the app builds from request.is_secure(), reads https://www.rcvis.com/..., so the real cache keys are https://.... Nothing matches, cache.delete deletes nothing, and no error is raised. The DisallowedHost catch adds a second way to skip silently. What that means for your requirement. After an update, Cloudflare is purged correctly, re-fetches from the origin, and the origin serves the stale page out of Django's file cache. Cloudflare then caches the stale page again. Today cache.clear() makes that impossible. With this PR, the stale window is the Django cache lifetime, 600 seconds on main. That is a regression in exactly the scenario you care about. Evidence. I ran the PR's own tests plus a probe against its branch under Django 4.2: ┌───────────────────────────────────────────────────────────────────────────┬─────────┐ │ Scenario │ Purged? │ ├───────────────────────────────────────────────────────────────────────────┼─────────┤ │ PR's test shape: example.com over http, purge outside a request │ yes │ ├───────────────────────────────────────────────────────────────────────────┼─────────┤ │ Production shape: viewer via www.rcvis.com over https, save from a worker │ no │ ├───────────────────────────────────────────────────────────────────────────┼─────────┤ │ Same, but origin saw plain http │ yes │ ├───────────────────────────────────────────────────────────────────────────┼─────────┤ │ Listed URL with an extra query string such as ?utm_source=x │ no │ └───────────────────────────────────────────────────────────────────────────┴─────────┘ The PR's tests pass only because the Django test client speaks http. Its CI on Heroku is also failing, though I couldn't see the log. Secondary concerns. - The purge depends on a hardcoded list of 17 URL variants. Any URL outside it, including any query string, stays stale until expiry. The test docstring acknowledges this. - Under Django 5.2.16 the PR's main test fails, because it relies on the old no-cache responses being cached, which 5.1+ refuses. It also conflicts with cache-updates, which deletes visualizer/middleware.py. - The thread-local request middleware is workable but is machinery in the hot path for a nice-to-have. Is the benefit worth it? With Cloudflare absorbing viewers and the Django cache now living 300 seconds, cache.clear() on a save costs at most one extra graph recompute per page on its next origin miss. For live voting with saves every minute across ten pages, that is about ten recomputes per minute instead of one. The PR description itself calls this a nice-to-have. My recommendation is to close both #609 and #635 and keep cache.clear(). If you do want surgical purging later, the reliable design is to record the real cache keys per slug at store time and delete those, rather than reconstructing them from guessed host, scheme, and URL lists. A minimal fix of the current approach would be building the synthetic request with RequestFactory and secure=True, plus a production-shaped test: https, www host, Site domain set, and no current request. I have a draft of middleware here, but I need to review it in more detail before I'm confident in its correctness: #644 <#644> — Reply to this email directly, view it on GitHub <#609?email_source=notifications&email_token=AABCNY7YYTOAUI62FKMRAFT5OTBTNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNRUGMZDMNJQG442M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-5643265079>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABCNY6HPGPLGVBHRYPOHLT5OTBTNAVCNFSNUABFKJSXA33TNF2G64TZHMYTMMZRGM4DGMBUHNEXG43VMU5TIMBVHAZDGNBUGUZ2C5QC>. Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AABCNY65UFASSIUG5B7H5CD5OTBTNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNRUGMZDMNJQG442M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KUZTPN52GK4S7NFXXG> and Android <https://github.com/notifications/mobile/android/AABCNYYXU4WZKNQISF3Z47T5OTBTNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKNRUGMZDMNJQG442M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>. Download it today! You are receiving this because you authored the thread.