Skip to main content

This operation is a Vault extension to the Amazon S3 API. The operation enables S3 clients to configure on a per-bucket basis how to store and protect S3 objects that are larger than a specified size threshold. The options are:

  • Store such objects in the HyperStore File System, where they are protected by replication.

  • Use erasure coding to store and protect such objects.

This S3 API extension also enables clients to specify the size threshold. S3 objects larger than this threshold will be stored according to the selected storage type. Objects at or below this size threshold will be stored in the Cassandra database and protected by replication.

Note

Support for this S3 API extension can be enabled or disabled via the system configuration property mts.properties: vault.s3.hsconfiguration.enabled. It is disabled by default. If disabled, requests to PUT, GET, or DELETE a bucket storage type will be rejected with an HTTP 403 error response. When the API extension is disabled, handling of large S3 objects is determined by system configuration (common.csv:hyperstore_size_threshold and common.csv:hyperstore_large_storage).

If this S3 API extension is enabled, then support for the x-gmt-hyperstore extension header to PUT Object, POST Object, and Initiate Multipart Upload is also enabled. With this extension header, S3 clients can override a bucket-level storage type configuration on a per-object basis. For example, a PUT Object request with the x-gmt-hyperstore header could direct the object to the Vault File System even if the object’s parent bucket is configured for erasure coding. Therefore bucket-level configuration is best regarded as setting a default for how to store and protect large objects in the bucket, with the caveat that the default may be overridden on a per-object basis.

In the case of large S3 objects for which no storage type configuration has been specified either at the per-object level or the per-bucket level, handling is determined by system configuration (based on the settings mentioned in the first part of this Note).

Connect with Us on LinkedIn

Follow Us on LinkedIn to find out what is currently going on and link with our Consultants!