Understanding Variation

Data Comparisons

‘A simple comparison between the current figure and some previous value cannot fully capture and convery the behavior of any time series.’ ‘Comparisons of this sort are built on the implicit assumption that last year was normal’

Limited comparison is the primary way that data presentations fail to provide context for a value.

Monthly management metrics

Monthly management metrics often compare against a plan, a previous value, and perhaps an average. These comparisons may be limited and provide contradictory messages.

Managing a company by means of the monthly report is like trying to drive a car by watching the yellow line in the rear-view mirror

Myron Tribus

Presenting Data in Context

Both the current and previous values are subject to variation so comparing them is of limited value.

Shewhart’s Rules for Presentation of Data

1

Data should always be presented in such a way that preserves the evidence in the data for all the predictions that might be made from these data

• Tables should accompany graphs
• A table cannot convey the full picture
• the context for the data should be fully described
  • context-less data can be distorted

2

Whenever an average, range, or histogram is used to summarize data, the summary should not mislead the user into taking any action that the user would take if the data was presented in a time series.

• Since all data occur in time, virtually all data will have a time-order

The first principle for understanding data

No data have meaning apart from their context

• Stop reporting comparisons between pairs of values except as part of a broader comparison
• Start using graphs to present current values in context

Second principle for understanding data

While every data set contains noise, some data sets may contain signals. Therefore, before you can detect a signal within any given data set, you must filter out the noise.

Errors interpreting data

• Interpreting routine variation as signal (meaningful departure from the past)
• Failing to detect a signal when present

Knowledge is orderly an cumulative

• Data → Analysis → Interpretation
• ****If experience is the basis for interpreting the data then the interpretation is only as good as past experience****
• Interpretation patterns
  • Comparison to specification is the comparison to a plan or goal set by the business
    • Types of specification:
      • Facts of life: known to be true and not subjective
      • Planning: predictions and planning
      • Arbitrary numerical targets
    • Specifications are the Voice of the Customer
  • Comparison to average - an average value is roughly about the mid point of a data set, and therefore half the time you are expected to be above and half the time below.
• Voice of the system or Voice of the process are the metrics generated by monitoring the process
• To change the system you must:
  • Listen to the voice of the system
  • Understanding how Stocks and Flows
  • Change the inputs or the process to achieve the results
• Variation undermines simple and limited comparisons
• An unpredictable process demonstrates exceptional and routine variation

Process Behavior Chart