aws s3 の ec2へのmount

mount-s3をオプションなしで実行した場合、ファイルの削除や上書きができない為、「--allow-delete」「--allow-overwrite」ありでマウントしています。

$ export AWS_ACCESS_KEY_ID=ないしょ
$ export AWS_SECRET_ACCESS_KEY=ないしょ

$ mkdir mount_s3
$ mount-s3 --allow-delete --allow-overwrite end0tknrnstest mount_s3

mount-s3経由でアップロード - 1MB x 1,000コ → 約7分

[ec2-user@ip-172-31-5-210 tmp]$ date; cp -r org_dummy_files/* mount_s3/; date
Wed Aug 21 07:14:01 AM UTC 2024
Wed Aug 21 07:21:12 AM UTC 2024

mount-s3経由でダウンロード - 1MB x 1,000コ → 約2分

$ date; cp mount_s3/* new_dummy_files/; date
Wed Aug 21 07:22:35 AM UTC 2024
Wed Aug 21 07:24:15 AM UTC 2024

mount-s3経由で削除 - 1MB x 1,000コ → 約1分

$ date; rm *; date
Wed Aug 21 07:24:43 AM UTC 2024
Wed Aug 21 07:25:41 AM UTC 2024

その他 - 今回使用したmount-s3のバージョンとヘルプ

[ec2-user@ip-172-31-5-210 mount_s3]$ mount-s3 --version
mount-s3 1.8.0
[ec2-user@ip-172-31-5-210 mount_s3]$ mount-s3 --help
Mountpoint for Amazon S3

Usage: mount-s3 [OPTIONS] <BUCKET_NAME> <DIRECTORY>

Arguments:
  <BUCKET_NAME>  Name of bucket to mount
  <DIRECTORY>    Directory to mount the bucket at

Options:
  -f, --foreground  Run as foreground process
  -h, --help        Print help
  -V, --version     Print version

Bucket options:
      --prefix <PREFIX>
          Prefix inside the bucket to mount, ending in '/' [default: mount the entire bucket]
      --region <REGION>
          AWS region of the bucket [default: auto-detect region]
      --endpoint-url <ENDPOINT_URL>
          S3 endpoint URL [default: auto-detect endpoint]
      --force-path-style
          Force path-style addressing
      --transfer-acceleration
          Use S3 Transfer Acceleration when accessing S3. This must be enabled on the bucket.
      --dual-stack
          Use dual-stack endpoints when accessing S3
      --requester-pays
          Set the 'x-amz-request-payer' to 'requester' on S3 requests
      --bucket-type <BUCKET_TYPE>
          Type of S3 bucket to use [default: inferred from bucket name] [possible values: general-purpose, directory]
      --storage-class <STORAGE_CLASS>
          Set the storage class for new objects
      --expected-bucket-owner <AWS_ACCOUNT_ID>
          Account ID of the expected bucket owner. If the bucket is owned by a different account, S3 requests fail with an access denied error.
      --sse <SSE>
          Server-side encryption algorithm to use when uploading new objects [possible values: aws:kms, aws:kms:dsse, AES256]
      --sse-kms-key-id <AWS_KMS_KEY_ARN>
          AWS Key Management Service (KMS) key ARN to use with KMS server-side encryption when uploading new objects. Key ID, Alias and Alias ARN are all not supported.
      --upload-checksums <ALGORITHM>
          Checksum algorithm to use for S3 uploads [default: crc32c] [possible values: crc32c, off]

AWS credentials options:
      --no-sign-request    Do not sign requests. Credentials will not be loaded if this argument is provided.
      --profile <PROFILE>  Use a specific profile from your credential file.

Mount options:
      --read-only              Mount file system in read-only mode
      --allow-delete           Allow delete operations on file system
      --allow-overwrite        Allow overwrite operations on file system
      --auto-unmount           Automatically unmount on exit
      --allow-root             Allow root user to access file system
      --allow-other            Allow other users, including root, to access file system
      --uid <UID>              Owner UID [default: current user's UID]
      --gid <GID>              Owner GID [default: current user's GID]
      --dir-mode <DIR_MODE>    Directory permissions [default: 0755]
      --file-mode <FILE_MODE>  File permissions [default: 0644]

Client options:
      --maximum-throughput-gbps <N>  Maximum throughput in Gbps [default: auto-detected on EC2 instances, 10 Gbps elsewhere]
      --max-threads <N>              Maximum number of FUSE daemon threads [default: 16]
      --part-size <SIZE>             Part size for multi-part GET and PUT in bytes [default: 8388608]
      --read-part-size <SIZE>        Part size for GET in bytes [default: 8388608]
      --write-part-size <SIZE>       Part size for multi-part PUT in bytes [default: 8388608]

Logging options:
  -l, --log-directory <DIRECTORY>  Write log files to a directory [default: logs written to syslog]
      --log-metrics                Enable logging of summarized performance metrics
  -d, --debug                      Enable debug logging for Mountpoint
      --debug-crt                  Enable debug logging for AWS Common Runtime
      --no-log                     Disable all logging. You will still see stdout messages.

Caching options:
      --cache <DIRECTORY>
          Enable caching of object content to the given directory and set metadata TTL to 60 seconds
      --metadata-ttl <SECONDS|indefinite|minimal>
          Time-to-live (TTL) for cached metadata in seconds [default: minimal, or 60 seconds if --cache is set]
      --max-cache-size <MiB>
          Maximum size of the cache directory in MiB [default: preserve 5% of available space]

Advanced options:
      --user-agent-prefix <PREFIX>  Configure a string to be prepended to the 'User-Agent' HTTP request header for all S3 requests