Measuring quality is most difficult.
One reason is that there is no universally accepted standard for quality. There
are Five categories of approach to quality. These are:
1. Transcendent Approach: a
subjective feeling of “goodness”.
2. Product-based Approach: measured
by attributes of the product
3. data-based Approach: conformance
to the specifications
4. value-based Approach: “goodness”
for the price
5. User-based Approach: the
capacity to satisfy the customer
6. Scientific Approach :
DMAIC Approach
One should note that the categories
are not mutually exclusive. In particular, no matter which definition is used,
quality is always ultimately defined by the customer (i.e. user-based). Let’s
look at these categories and see how they apply to knowledge work.
Transcendent Quality Approach
Transcendent quality measures are just
means for capturing subjective opinions. The most common tool used is the
rating scale. For example, cake mixes are tested by submitting their products
to a panel which rates the taste of the cake on a scale from one to five, with
five being the best possible. Knowledge workers sometimes use examine ratings
in a similar manner. When an attribute is actually subjective, like taste, the
transcendent cannot be challenged. In areas where other measures are possible,
the more objective measures are generally preferable. Even so, the transcendent
opinion of the customer is the most important measure of one’s quality. A
useful area for transcendent measures of quality is in the appraisal of
individual performance. Dr. Deming uses the annual appraisals for several
reasons. However, appraisal systems will probably be with us for a while, and
the use of transcendent measures may be one way to make them work. My
recommendation is to use general categories (e.g. shows initiative) scored by
the subjective opinion of the employee’s supervisor, on the assumption that the
supervisor’s transcendent quality judgment of the employee is likely to be an
accurate measure (He will know quality work when ne sees it). Even when using
more objective quality definitions, the transcendent can be useful as a “sanity
check”. If a measured quality value “feels” too nigh or too low, perhaps your
intuition is calling for you to reevaluate your selection of measures.
Product-based Quality Approach
Product-based quality is measured
by the amount of some desired ingredient or attribute. For example, the speed
of a computer. In Knowledge work, one desired attribute may be innovation. The
difference is, of course, that it is much easier to measure speed. Since innovation
and other intangible features are desired not for themselves, but for their
impact on the product, measurable units such as speed will reflect the quality
of knowledge work once the work is transitioned into hardware or software.
Under such circumstances, system parameters can be measured to establish the
quality of the underlying knowledge work. One would select the only most
meaningful measures. To be effective as quality measures, however, the measured
values must be referenced to some benchmarks. For example, the speed of a
computer is useless for quality evaluation unless the analyst knows what other
machines deliver. A problem with attribute measures is that trade-offs may not
be recognized. Speed may be enhanced at the expense of payload which may or may
not be an improvement overall. One way to evaluate this is the use of all-encompassing
measures such as “systems effectiveness,” defined as a function of a system’s
availability, dependability and capability against a specified threat. In the
simplest case, availability is the probability of a system being operable when
needed, dependability the probability that it will remain operable for the
length of a mission and capability the conditional probability that, if
operating, it will successfully complete the mission. For this simple case: System
Effectiveness = (Availability)x(Dependability)x(Capability) An approach between
the measurement of a few selected parameters and the calculation of system
effectiveness is the use of indexes. Indexes are artificial, but supposedly not
arbitrary, groupings of measures into an overall single measure. Examples are
the consumer price index and the index of leading economic indicators.
Similarly, a quality index can be created by identifying parameters of interest,
establishing measures, weighing the measures and combining them into one. As a
simple example, Robert Gunning invented a “fog index” for evaluating
understandability of text, calculated by computing the average sentence length,
adding this to the number of words of three syllables or more in 100 words,
then multiplying by 0.4. Though Gunning
claims his index corresponds roughly with the number of years of schooling a
person would require to read the text 4ith reasonable ease, an index figure is
generally not meaningful in absolute terms, out, rather, useful for showing
trends. The more tangible the product, the better product-based measures work. However,
in knowledge work the product is often intangible, such as a set of recommendations,
so product parameters cannot be measured. One alternative is to use even more
indirect measures so long as they also correlate with the attributes desired.
For example, a large number of patents held should indicate an innovative
agency. Some other measures might be the ratio of in-house to contracted work,
numbers of papers published, resources spent on education and training
activities, advanced degrees earned, name requests for consulting committees
received, and the amount of national/international professional activity among
the knowledge workers. These are measures of the laboratory climate or
environment favoring quality knowledge work. One could also measure the climate
opposing quality in knowledge work. Common measures indirectly showing
unfavorable climates include absenteeism, turnover percentage, average sick
days taken per employee, etc. Poor environments could perhaps be more directly
measured by the number of approvals required to do work, the ratio of overhead
to productive activity, the length of time required to obtain a part or a piece
of test equipment, etc. These could be labeled “Hassle Indexes.”
No comments:
Post a Comment