Hierarchical storage management (HSM) is a data storage technique that automatically moves data between high-cost and low-cost storage media. HSM systems exist because high-speed storage devices, such as solid state drive arrays, are more expensive (per byte stored) than slower devices, such as hard disk drives, optical discs and magnetic tape drives. While it would be ideal to have all data available on high-speed devices all the time, this is prohibitively expensive for many organizations. Instead, HSM systems store the bulk of the enterprise’s data on slower devices, and then copy data to faster disk drives when needed. In effect, HSM turns the fast disk drives into caches for the slower mass storage devices. The HSM system monitors the way data is used and makes best guesses as to which data can safely be moved to slower devices and which data should stay on the fast devices.
HSM may also be used where more robust storage is available for long-term archiving, but this is slow to access. This may be as simple as an off-site backup, for protection against a building fire.
HSM is a long-established concept, dating back to the beginnings of commercial data processing. The techniques used though have changed significantly as new technology becomes available, for both storage and for long-distance communication of large data sets. The scale of measures such as ‘size’ and ‘access time’ have changed dramatically. Despite this, many of the underlying concepts keep returning to favor years later, although at much larger or faster scales.
Cloud Storage Tiering Implementation
The CloudTier Storage Tiering provides a simple storage solution to integrate the cloud as a second-tier storage, to reduce the cost of the storage, to enhance the data security, to scale the cloud storage unlimitedly. CloudTier Storage Tiering SDK uses the on-premises storage as a tier 0 storage (hot storage), uses the public cloud storage as a tier 1 storage (clod storage). The hot storage only stores the hot data which is highly volatile and time-sensitive and which needs to be available in the shortest possible time. The cold storage only stores the cold data (archived data) which isn’t performance-sensitive, rarely accessed and will be available with high latency.
Integrating Cloud Storage as a Second Tier
CloudTier Storage Tiering SDK was implemented with tiered storage file system filter driver. A file system filter driver intercepts requests targeted at a file system or another file system filter driver. With the ClooudTier Storage Tiering, you can connect the cloud storage as a second tier, seamless as a local on-premise storage, your application can use the cloud storage just like the local storage without any changes.
Cloud Archiving Solution for Unstructured Data
Cloud archiving is the process of moving data to secondary storage in the cloud. Potential benefits of cloud archiving include lower costs and easier access. Through the use of Hierarchical Storage Management, you can define criteria to archive less frequently used data from local disk to the cloud. The criteria can be based on inactivity limit, size, or even days used per month. Once the data meets the criteria, it is then migrated to the cloud.
After the file was migrated to the cloud, you can replace the file with a stub file, no longer taking up that much-needed space. A stub file looks and acts like a regular file, it has the same file attributes with the original file (file size, creation time, last write time, last access time), it also keeps the original file’s security. A stub file can embed with your custom tag data. A stub file doesn’t take the storage space as below figure.
Seamless Integration of Cloud and On-premises Applications
CloudTier Cloud Storage Connect service can connect an on-premises software appliance with cloud-based storage to integrate your existing on-premises applications with the remote cloud storage infrastructure in a seamless, secure, and transparent fashion. There are no interruption to migrate your on-premises files to the remote cloud storage, don’t need to change your existing applications and infrastructure.
- Set up file cloud migration policies based on the file type, file size, file attributes.
- Create stub file based on the policies after the file was migrated to the cloud storage, it can free up the space from on-premises storage.
- Transparent the cloud storage access by reading the stub file for your local application.
- Transparent moving data back from remote cloud storage to the local, re-hydrate the stub file for the recent access file based on the policies.