As a storage analyst, I often get asked the same question, “What is the best array?” My answer has evolved slightly over the years, but now I use a very direct and clear answer that belies the complexity in this process. My answer is quite simply, “Well, I’m an engineer, so ‘It depends.’”
It depends upon your current environment, your planned environment in 3 – 5 years, your relative IT expertise, your size, your industry and of course the mix of applications you are running. In fact, even this list of items is too simple to accurately capture what considerations are needed to accurately answer this question. Usually when I give this answer, people smile and nod their head after realizing that this is probably the correct answer, but not the one they wanted. It is of course an understandable question; I have asked similar questions of other experts, not yet knowing what I don’t know.
As a storage industry analyst, I have been involved in many decision-making processes and I can tell you that there are more considerations than you may imagine. We have literally recommended nearly every product we cover, which is over 100 different storage products. Why would we recommend so many products? Quite simply because each client has different needs.
By now you are probably thinking: well of course it depends, so what is the point of this blog? The point is quite simply this; if you ever read a report that claims to rank order anything, the reviewer is simplifying things. This may be appropriate, if the considerations have been explained explicitly. If there are enough parameters to clearly explain a segment of the market, then a rank ordering can certainly make sense. These ideas apply to nearly everything, from the stock market with ETF and bond options to automobiles, etc. However, if the parameters are not clearly spelled out, than the so-called “analysis” is probably useless.
I recently read a blog from a former colleague who works for a storage vendor. They wrote about this very problem, taking issue with the way a rank-ordered list was created and the lack of analysis used for the list. What prompted his blog? Apparently an analyst firm published a rank ordering of storage products, with one proclaimed to be “The Best”. However, the analysis was based on a list of features, without regard to the workload, environment or any other considerations. The blogger argued that not only was this process unfair, but it was also misleading and therefore doing a disservice to consumers and vendors alike. I completely agree.
These rank-ordered lists may be interesting, but they are worthless and may even be damaging for the very reasons I have explained. Most IT consumers are savvy enough to realize that reports like these have little value. However, some are fooled; others will use the report as leverage to obtain the outcome they want. They may do so by showing how their favorite product is at the top of the list, or perhaps a product they want to disqualify from an evaluation because is it not at the top of the list. This happens on both sides of the vendor and IT consumer divide, with both knowing how to use an “independent report” to justify their decision.
There are no simple answers; deciding upon the best product for your environment takes quite a bit of analysis. Independent firms can help with this process and we frequently do this type of work with our IT customers. This process typically takes several conversations, with research and analysis from both the IT client and our analysts. The decision of a best fit is a process, not just picking an item from a report.
There are truly independent reports on products, but good reports always narrow the criteria and look at a specific aspect. For example, we may analyze a few products for small enterprise environments running virtualized Oracle database workloads. We may further look at the expertise of the IT staff and then provide arguments as to why one product has advantages for this market. Please understand that this is completely different than a blind ordering of “best to worst” products.
In other instances we are involved with testing or benchmarking products, often head to head with one product versus a competitor. Again, this is different because, in defining the test case, the workload is well defined. This workload may not apply to other environments and that is OK; no valid test report would claim it does. The only claim that can be made is that for a specific test and set of conditions, one product out-performed another. This result can then be used as one data point in the decision making process.
So, the next time you are tempted to ask, “What is the best storage array?”, please remember the answer is always… “It depends.”
