If you were to ask the average enterprise data center administrator how long their organization save its data, the quick answer would be “forever.” It’s been that way for years, in fact decades. Knowingly throwing away data is something that these people normally don’t do unless they know for sure it’s useless – like test data. Nor do they have the authority to decide what to delete in some cases. Plus, making those decisions at higher levels within the enterprise and then implementing them is often more costly than just consigning the data to tape vaults and saving it for who knows how long.
One would think that under the we-save-everything-forever policy, whether formalized or not, there would be demand for storage alternatives to tape that are more space efficient like optical disk and holographic film. But optical storage companies have come and gone and holographic has yet to achieve commercial viability despite product availability. Past vendors of optical failed because they could no longer sell to enterprise data center buyers as well as other types of customers. Demand for optical crashed in the early 2000s as disk advanced as a backup and archive target coupled with more or less steady demand for tape as advances continued to be made in its recording density.
There are a number of reasons why these alternatives have been consigned to the storage industry’s dust bin. But, I’m not going to explore them here because there’s now a chance that optical at least could get pulled out, brushed off, and pushed forward again. Enter Facebook. Reports surfaced a few months ago that had Facebook implementing Blu-ray optical storage technology. In fact, one report included quotes from skeptical industry analysts (including this one) that pointed to dead bodies littering the optical storage landscape.
So if optical is to rise from the ashes, it has to have a different set of buyers. Those could well come from Internet data centers and cloud services providers. Consider LinkedIn LNKD +3.09% that now generates revenue from two sources. LinkedIn sells data to potential employers looking for talented and qualified employees and to individual users who buy LinkedIn’s premium services. Certainly most of LinkedIn’s users will continue to maintain and update their accounts as long as they’re productive members of the workforce. That could be decades. And LinkedIn has every incentive to want to continue to generate revenue from this increasingly rich source of resumes and data.
Cloud services providers have similar incentives. They know that data creates “stickiness.” The more data that a customer stores in the cloud, the less likely that customer will move that data somewhere else, sometimes even in the face of escalating cost.
It is therefore no surprise that Facebook would try optical as an alternative to both disk (comparatively high cost) and tape (media deterioration over time) as a way to preserve data for at least thirty years. For both, that data is a source of revenue–and an increasingly rich one as time goes on. So they have an incetive to find very high density storage that can also preserve data for long periods of time at relatively low cost. Indeed, the person most responsible for the decision to go optical at Facebook has now broken off and formed his own optical storage startup.
For cloud service providers, the challenge is a bit more nuanced. Their customers assume that as long as they pay the monthly bills, the cloud will preserve this data. For the service provider, access patterns will vary. Some data will be accessed frequently, some rarely if at all. Yet, the service provider must maintain access to its customer’s data over what could be years. For them, optical represents a low performance but high capacity storage tier. And because it’s off line, it is an “infinite” storage resource like tape. However, unlike tape, it does not need to be refreshed as often by copying data from old to new media because it doesn’t deteriorate at the same rate as tape.
Finally, the growing interest from Chief Analytics Officers in mining the enterprise’s historical data for big data analytics projects could give new life to those saved-forever archives. More data results in making predictions with greater accuracy. So while there may be no value now to the enterprise in its oceans of archival data stored in vaults, there could perceived value be next year, giving even the enterprise a reason to revisit optical storage.
