The secret to keeping your outsourcing initiative on track is having a governance model in place that not only specifies but also, constantly monitors and measures how effectively your provider is delivering the outsourced services model, without it you simply cannot assure or improve the quality of a service.

But how can you track the delivery of outsourced services when a portion of your services are in the Cloud?  

If your outsourced services are a combination of traditionally provided services alongside cloud-based services, then the monitoring tools you’ve already invested in to collect and analyze performance may not work with the cloud provider.  On the other hand, you may have made trade-offs for your cloud services, perhaps trading performance for cost, or very high quality for flexibility, in which case you may not want to measure the cloud services by the same yardstick as traditional services.

Also, by its very nature of being a more widespread service, you need to factor in measures of the cloud services that are more universal, rather than just based on the service that you are receiving.  For instance, if there is a security breach impacting other clients of the cloud service, it changes your risk profile even though it may not have impacted you directly.  So, the reputation of the service becomes important to you.  However, there currently is no universally accepted, unbiased measure to rank cloud services, other than the provider’s own claims.  But that is like a restaurant claiming their food is the best, with no opinion taken from its patrons.

So, it is welcome news that Carnegie Mellon University has taken on an initiative to develop a set of qualitative and quantitative measures which, in business terms, will give CIOs a standardized method for measuring the levels of their cloud services.
 
This initiative will lead to the development of a Service Management Index (SMI), which may, over time, become the industry’s adopted standard.  The SMI will be customizable, so that you can set it for the attributes that are important to you, and obtain a score for the cloud service.  It will also have a standard API definition for aggregating SLA or other data feeds from your existing monitoring tools, allowing you to compute and compare performance scores on an ongoing basis.

And with that you will finally be able to answer the question, “How do I measure success in the cloud?”

(Reference: Carnegie Mellon University Press Release)