You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Measure the filesystem suite against s3fs on c8gn.2xlarge #1318
Run the extended filesystem suite on one c8gn.2xlarge Spot host in us-west-2 and record a history entry comparing S3FileSystem, AioS3FileSystem, and s3fs, with a recommendation for #1315.
Selection
Single object (Add s3fs as an API of the filesystem benchmark suite #1316): small, medium, large with read (both variants) and write (both variants); xlarge with the in-memory write and copy. Workers 8 and 40 (s3fs max_concurrency), pools 10 and 40, block size 8 MiB, five trials, as in the 4.2.0 run.
The PyAthena cases run in the benchmark environment and the s3fs cases in the s3fs environment of the same checkout; the entry records both environments' library versions.
The host's baseline is 25 Gbps; the 4.2.0 run reached 1,074 MiB/s for 6 GiB writes, so the host is not the limit for the single-object cases.
Deliverables
A benchmarks/history/ entry with the environment, selection, run record, findings, limits, and the result tables, following the existing entries.
Activity
Name the scales and compare matching rates in the s3fs results row (G…
Scope block sizes, download parallelism, and thread counts in the s3f…
Merge pull request #1321 from pyathena-dev/docs/1318-s3fs-comparison