ls() keeps a deleted bucket's cached listing and ignores the version of a versionId path #1351

Part of #1326. Found while reviewing #1347 (#1330).

Summary

Two defects of S3FileSystem.ls() (and AioS3FileSystem._ls(), which runs it) predate #1347; that PR did not change them. Both were reproduced at 4a0cadb with the stubbed client of tests/pyathena/filesystem/test_s3.py (_make_fs, _serve_keys).

1. A refreshed listing of a deleted bucket keeps the cached listing

Behavior

fs.dircache[("bucket", "/")] = [<file bucket/old>]   # listed before the bucket was deleted
fs.ls("s3://bucket", refresh=True)                    # FileNotFoundError (NoSuchBucket)
fs.ls("s3://bucket")                                  # ['bucket/old']

The refreshed listing finds the bucket missing, but the next listing without refresh returns the deleted entries from the cache. A refreshed lookup that finds a path missing is expected to evict its entries, as _head_object() and the HeadBucket lookup of info() do.

Cause

_ls_dirs() (pyathena/filesystem/s3.py:467) evicts (path, delimiter) only after a complete listing (s3.py:522). The FileNotFoundError raised by core.list_objects() (s3.py:508) leaves the method before that, so the cached listing stays. The same applies to the recursive listing (delimiter="") that find() caches.

2. ls() of a version-qualified key path lists its children without checking the version

Behavior

With key/child present and no version v1 of key:

fs.ls("s3://bucket/key?versionId=v1", refresh=True)   # ['bucket/key/child'], one ListObjectsV2 request
fs.info("s3://bucket/key?versionId=v1", refresh=True) # FileNotFoundError

A path with a ?versionId= suffix refers to the object version (docs/filesystem.md), and info() looks the version up. ls() instead lists the directory key/, and returns its children whether or not the version exists. If the version exists, it is not listed either.

Cause

ls() (s3.py:573) lists key/ with _ls_dirs() first and looks the versioned path up with _head_object() (s3.py:575) only when that listing is empty.

Proposal

1. _ls_dirs() evicts the cached listing of the path when the listing raises FileNotFoundError, then re-raises.
2. ls() of a key path with a ?versionId= suffix looks the version up with _head_object() and returns that object, or raises FileNotFoundError, without listing the directory.

Validation

Unit tests with the stubbed client for both scenarios: the listing after the refreshed FileNotFoundError requests S3 again instead of returning the cached entries, and ls() of a version-qualified path returns the version or raises, matching info().