Business Efficiency: Measurement System Analysis
Measuring the wrong data when you are looking at business efficiency defeats the whole point. Measurement System Analysis (MSA) keeps your measurements honest, accurate, and expected.
Are you measuring the right data for efficiency?
Make sure you’re measuring the right data in your business or organization by running through this checklist:
Is my data fresh? If you want to know how customers feel about your product line today, customer satisfaction surveys from last year — or perhaps even last month — aren’t relevant. All your data should have a timestamp so you know you’re working with the freshest, most relevant material.
Is my data objective? It’s all too easy to go into a collection or measuring project with preconceived notions of what that data will tell you. This can skew the data you end up collecting, especially when it comes to information like sentiment that isn’t strictly mathematical on its face.
For example, if you believe that your employees are as happy as can be, you may feel you can collect feedback directly in one-on-one meetings and overlook the added benefit an anonymous survey may provide.
How precise is my data? If you’re measuring widget sizes by visually approximating their length, you can’t turn around and report accuracy to three decimal points. Your initial measurement units and tools affect the precision of your final numbers.
Is my data complete? As I touched on earlier, if you’re not measuring all the data points you have on a particular metric, your results will be skewed accordingly. For example, only surveying customers who call in with support requests would likely paint a more negative picture than if you surveyed a random sampling of your entire customer base.
Am I always measuring the same thing? For example, if you sell cases of wine, does a quantity of “1” mean one case or one bottle? If you operate in multiple countries, are you always measuring in the same currency? Is an order date the day the customer submitted the order or the day you start processing the order? Defining these terms up front can help eliminate lots of confusion down the road.
Are my data gatherers honest and committed? Assigning the wrong people to collect data can also skew your data. For example, an employee who feels her job is threatened by the outcome of a data collection project may not be as thorough as one who understands the importance of accurate information for the future health of the company.
Similarly, an intern may simply not understand that skipping over a file here and there or fudging an unclear answer can have far-reaching effects on your business.
Testing your data collectors
The process of making sure everyone measures in the same way is called gauge repeatability and reproducibility, also known as gauge R&R. The process is a statistical tool that allows you to measure the amount of variation caused by the measurement process, considering both the tool used for measuring and the people performing the measurements.
To carry out the gauge R&R, you test your measurement operators by having them measure the samples multiple times to see how close the results come to the standard measurements and how close the operators are in the measuring process.
You also need one expert operator to perform sample measurements under ideal conditions first — these are your standard measurements. After the operators measure all the samples, you’re ready to make your calculations:
Calculate the variation for each operator by counting the number of samples for which the operator gets the same measurement value and dividing this number by the number of samples.
Calculate the variation for each operator versus the standard by counting the number of samples for which the operator gets the standard measurement value and dividing this number by the number of samples.
Calculate the variation between the operators by counting the number of samples for which the operators get the same measurement value and dividing this number by the number of samples.
Calculate the variation between the operators versus the standard by counting the number of samples for which the operators get the same standard measurement value and dividing this number by the number of samples.