By Christopher Ball, Senior Professional Services Consultant, MasterControl Inc.
Most companies understand that the collection of data can be valuable but often don’t really rationalize what they want to collect. Why collect data for “A” rather than data for “B”? Wouldn’t it be better to collect it all?
The problem is there is so much data available, much of it with an automatic collection option, that it’s easy to accumulate it. The downside of blindly taking this approach is data saturation. There’s so much to look at that nothing is done to analyze it. Because the data isn’t analyzed, no action is taken based on the findings. Effectively, the company drowns in data at best or worse still, makes random and or variable assumptions about the interpretation of the data. As a result, the company could make flawed and maybe fatal business decisions because “the data indicated …..”
The intention of this article it to put some simple steps into the armory of your business so that you won’t waste valuable resources collecting data you won’t use, need, or want. Instead, collect data that fits with your business direction and strategy---data you can analyze appropriately and use to make balanced informed decisions. You’ll see that having data and analysis that business management can trust has value.
Before getting into some dos and don’ts for collecting and using metrics data, it is worthwhile to have a definition for metrics. Unfortunately there is not just one definition. An Internet search reveals many similar definitions to those below. The definition you select should be based on the needs and focus of your business.
- Metrics can be defined as a collection of measurements that provide information about a product and or process. (Jeffery Gainer – Dec 2008)
- A business metric is any type of measurement used to gauge some quantifiable component of a company’s performance such as ROI. (whatis.com / Search CRM)
- Standards of measurement by which efficiency, performance, progress, or quality of a plan, process, or product can be assessed. (businessdictionary.com)
- Metrics are statements that reflect a definable and measurable situation or process. (Christopher Ball – Sept 2004)
StrategyThe collection and use of metrics needs to be approached with focus and should address a perceived need.
Essentially there are two types of metrics. High level metrics tend to monitor performance and to provide feedback against a given business goal or objective, such as year to date or quarterly performance against sales targets. These primarily allow a business the opportunity to get a warning message if performance is not as anticipated and possibly look at steps to understand and rectify the situation before it is too late.
This is important for any business but can be critical for a business whose performance and value is assessed by the stock market, for example, where a short-term loss of performance can make a business vulnerable to takeover due to a fall in the stock value. The other type of metric is purpose-driven metrics which are designed to evaluate the impact of a change or develop a baseline of data so that an assessment of possible changes and the result of those changes can be analyzed easily and effectively.
A metrics process is a cyclical process which must be aligned with the business objectives of the company, as illustrated below.
The key to developing solid metrics is based on definition and resources:
- Definition of the business objective the metric is aligned with including any assumptions taken.
- Definition of the metrics and data to be collected, including the purpose and any assumptions made.
- Upfront determination on how the data will be analyzed and assumptions made.
- Allocate appropriate resources to each step of this process. Scant resource allocation will result in inadequate data or analysis and can lead to critical failures in the business decision-making process.
Dos and don’ts of metrics collection and use
- Have a plan – collect metrics data for a business-oriented purpose. Never collect metrics without a purpose.
- Carefully define each metric statement. Establish guidelines that reject poorly-defined metrics.
- Document how the data will be gathered.
- Don’t collect metrics that you or your company cannot use to affect change, in the event that the data indicates change is needed.
- Do not create goals simply to match the metrics you may already have. Unless your metrics are linked to a business objective, this is a waste of resources.
- Don’t change your metrics yearly. Be consistent in your data collection. Use the data collected to identify trends. If new data indicates a change is needed, then consistency of data collection is the best way to evaluate if the adjustment made actually achieved the anticipated improvement.
- Remember data collection is only part of the process! The data has to be analyzed, evaluated and then used to deliver value.
- Allocate resources to the whole process.
- Document any assumptions at all of the previous stages.
Chris Ball is a senior professional services consultant at MasterControl. Prior to MasterControl, Chris had a diverse career in quality assurance and quality improvement in GLP, GCP GMP and ISO environments in the agrochemical, contract research medical device and pharmaceutical industry sectors.
CAPA Training 1 (38:22)
View QMS 3-Minute Video (3:00)