Object Storage – Part 1

What Is Object Storage?

Object Storage is the storage that is designed to handle large amounts of unstructured data. This is the data that has no structure & largely consists of a mix of emails, images, audio files, text files, IoT data, videos & so on.

Since, this kind of data is continually generated in massive amounts on social platforms, also from the IoT devices. We need a storage system that can handle this data influx efficiently & economically.

Object storage is also the preferred storage model for data archiving and taking data backups as it offers dynamic scalability, unlike any other storage models. It can easily handle petabyte, exabyte scale data on an ongoing basis.

Now let’s understand how does object storage work?

How Does Object Storage Work?

Just like block storage, every data object in an object storage system contains a unique identifier for easy accessibility. Objects also contain metadata attached to them.

Attaching metadata with objects helps with the implementation of data policies, data protection, validation of the authenticity of the content, running business analytics and so on.

This metadata can also be customized based on the business requirements. For instance, we can customize the metadata of an image to add information such as from what device it was captured, the people or the objects in the image, date & location of the image, image category, filters applied on the image and so on.

Once this meta information is added to the images, they can be easily located & retrieved based on the meta-information such as fetching images belonging to a certain category or those captured by a certain camera & so on.

Storage of meta information of this level is not possible with other storage types such as block and file. The data stored with block and file storage contains very basic meta-information.

Storing data in objects helps with the performance big time when dealing with petabyte, exabyte scale data. The objects stored are further aggregated into object pools and are spread across the clusters & regions for scalability, high availability & disaster recovery. This is the reason object storage is widely used by businesses running their workloads on the cloud.

Illustration 1.70 - Distributed Object Storage In The Cloud

Accessing Object Store Data

Object store data is accessed over the web via REST APIs. The data is mostly stored in virtual machines running on commodity bare metal servers. Developers use the APIs provided by the cloud providers to read and write data in the object store managed by the cloud.

The provider is responsible for making the data redundant, setting up disaster recovery & so on. Fundamentally, the cloud storage provides all the features that a cloud typically provides for a workload running on the cloud such as high availability, scalability, elasticity, durability, security, a distributed environment facilitating storage of massive amounts of data, pay for what you use pricing model and so on.

Let’s continue this discussion in the next lesson.