} -->
Back to Top

Are Your Metrics Meaningful?


(updated 2020)


Data is everywhere. You can literally create a data set out of anything. For example, “On average, how long does it take for you to brush your teeth in the morning?" Measure the average time it takes for one month, and you've got yourself a dataset. Now let's pair this with another set of data, such as, “On average, how long does it take you to brush your teeth, and style your hair, in the morning?” This study may seem frivolous, but it could also be very valuable if you're planning to join a carpool to work, and you need to determine how early you need to wake up, or your partner just moved in, and you need to truncate your morning routine, etc.

What makes a data point meaningful is 1) the subject matter is relevant to your objective, 2) only strong correlations to other data are considered, and 3) measurement occurs across a significant period of time, options, and scenarios. So, if you need to report back to your dentist an estimate of the time that you spend brushing your teeth, the data collected and considered will differ significantly from data collected because you need to shorten the time that it takes for you to get to work in the morning. If you only need information so that your dentist can assess whether or not you're spending enough quality time with your teeth, why waste time collecting data about the amount of time you spend styling your hair? In this case, the secondary information is irrelevant as it has no bearing on the objective, at all. The following steps will help ensure our metrics are meaningful:

  1. The purpose, objective, assumption must be made first, before brainstorming trackable variables.
  2. Track each variable for a sufficient period time, over sufficiently varied situations.
  3. Record all data as a simple a data point occurring at a specific date, time, event, etc.
  4. Ensure objectivity: Artificially skewed information defeats the purpose. If you cannot observe the data objectively, remove yourself from the process and assign the task to someone else.
  5. Note any strong correlation, which is a correlation that can be confirmed at a confidence level of at least 95%. 
  6. Identify the primary driver of the correlation.
  7. Consider the data in relation to the organization’s business cycle and note any other possible drivers, causes, or effects for each variable and relationship.
  8. Note any changes in correlations over time.
  9. Complete an assessment of the correlation and their drivers, e.g. the cause, the effect, the reasons for shifts, and the nature of shifts.

Organizations are only as good as their leadership and the translation of their message into effective actions. When organizations take care to ensure that proper leadership is in place across all levels of the organization, organizational effectiveness is realized. 

The Key to Sound Decision Making


But wait, we’re far from done. Metrics are often defined as “Parameters or measures of quantitative assessment used for measurement, comparison or to track performance or production.” This is true. Metrics provide a basis for comparing performance against prior performance, a target goal, or competition. However, this is an incomplete definition as quantitative data alone rarely gives the whole picture. Usually, you also need qualitative data to provide increased granularity and accuracy in the results. For example, if we can capture the variability in the time spent brushing my teeth, e.g. short, long, soft, hard, then my dentist can better assess the quality of the time that I spend brushing my teeth.

Next, you have to consider that perhaps there is more than one qualitative factor that affects a particular variable, and finally, perhaps one factor impacts the variable to a greater degree than the others. Thus, we capture the factor’s weight, e.g. 0.5x, 1x, 3x. 

Once the quantitative and qualitative factors are validated, basic algebraic equations can be developed to track and monitor performance. It's not rocket science; however, it requires time, attention to detail, and a scientific approach. Otherwise, you may end up with meaningless data that doesn't help you to effectively measure and monitor the organization's performance.