Skip to content

Measure the filesystem suite against s3fs on c8gn.2xlarge #1318

Closed

Description

@laughingman7743

Part of #1315. Blocked by #1316 and #1317.

Summary

Run the extended filesystem suite on one c8gn.2xlarge Spot host in us-west-2 and record a history entry comparing S3FileSystem, AioS3FileSystem, and s3fs, with a recommendation for #1315.

Selection

The host's baseline is 25 Gbps; the 4.2.0 run reached 1,074 MiB/s for 6 GiB writes, so the host is not the limit for the single-object cases.

Deliverables

  1. A benchmarks/history/ entry with the environment, selection, run record, findings, limits, and the result tables, following the existing entries.
  2. A comment on Benchmark the S3 filesystem against s3fs and decide on a standalone package #1315 with the findings against its expected outcomes and decision criteria, and the improvement issues the results justify (for example parallel batch operations for the synchronous filesystem).

Cost

About 20 to 30 minutes of one Spot host (about $0.10 per hour) and S3 requests below $1; the same order as the 4.2.0 filesystem run.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions