I asked someone at AWS about some tips to speed up AWS S3 transfer speeds

My name is Ito and I am an infrastructure engineer

Amazon S3is
an extremely scalable system boasting 99.99% availability.
(Incidentally, its robustness is 99.9999999%.)

Some people use S3 for static sites, while
others use it for storage.

What really matters is the transfer speed

Today I would like to introduce a "trick to speed up transfer speeds" that someone from AWS mentioned!

S3 is not a directory-file structure

Before discussing techniques to speed up transfers...
files to three data centers in the same region at the time of upload
copies

In S3, there are things called buckets and objects, and we tend to think of folders as buckets and files as objects, but that's not actually the case

I quote

The underlying technology of Amazon S3 is simply a KVS (Key-Value Data Store). For example, let's say we have a folder structure (as we perceive it) like this: (For the purposes of this entry, we will simply assume that bar.txt contains the text "bar" and baz.txt contains the text "baz".)
(Root)
└ foo/
└ bar.txt
However, this is just how we perceive it; from S3's perspective, it simply holds the following information. In S3, the / basically has no special meaning.
Key (Full path name) Value (Content of the file)
foo/bar.txt bar

Reference site:Shattering the illusion of "folders" in Amazon S3 and revealing their true nature | Developers.IO

Although Amazon S3 supports buckets and objects, Amazon S3 does not have a hierarchy, although prefixes and delimiters in object key names can imply hierarchy in the Amazon S3 console and AWS SDKs, introducing the concept of folders

Reference site:Object keys and metadata - Amazon Simple Storage Service

Using buckets and objects makes them seem like folders and files, but it seems they are only created as concepts

Since it's key-value data, retrieving the data simply involves a search.
Also, if you use similar bucket names, the data will be stored in the same data center before being copied, which
tends to slow down the transfer speed.

Prepend the bucket name with a hash value

By adding a hash value of a few characters to the bucket name, you can prevent data from being written to the same data center

• test01
• test02
• test03

Instead,

• abctest01
• yjctest02
• ckttest03

That's what it's like

However, you can't just use any bucket name;
there are restrictions on naming conventions, so be careful.

• Bucket names must be between 3 and 63 characters long
• Specify a bucket name as a single label or a series of labels, with adjacent labels separated by a single period. Bucket names can contain lowercase letters, numbers, and hyphens (-). Each label must start and end with a lowercase letter or number
• Bucket names cannot be in the format of an IP address (for example, 192.168.5.4)
• When using virtual hosted buckets with SSL, SSL wildcard certificates only match buckets that do not contain a period. To avoid this issue, use HTTP or write your own certificate validation logic. We recommend that you do not use periods (".") in bucket names

Source:Bucket constraints and limitations - Amazon Simple Storage Service

This means that various types of data are written to different data centers within the same region, which
is expected to be faster than writing them to the same data center.

I wonder how many characters were required at the beginning...I think it was at least three characters

Well, have a good S3 life!