We expect complex systems like an airplane, a nuclear powerplant, or a LNG plant to practically never fail.  Yet systems are becoming increasingly complex, and the more components there are in a system, the more reliable each component must be, to the point where, at the element level, defects become impractical to measure within the time and resources available.

Additionally, in future, our expectations will increase for complex systems durability, reliability, total cost of ownership, and return on investment, as energy and raw materials increase in cost.

Ultraquality is defined as a level of quality so demanding that it is impractical to measure defects, much less certify the system prior to use.  It is a limiting case of quality driven to an extreme, a state beyond acceptable quality limits (AQLs) and statistical quality control.

One example of ultraquality is commercial aircraft failure rates.  Complexity is increasing: the Boeing 767 has 190k software lines of code, whereas the Boeing 777 has 4 million lines of code, and the Boeing 787 about 14 million lines of code.  The allowable failure rate of the flight control system continues to be one in 10 billion hours, which is not testable, yet the number of failures to date is consistent with this order of magnitude.

Another example of ultraquality is a modern microprocessor, which has the same per chip defect rates despite the number and complexity of operations have increased by factors of thousands.  The corresponding failure rate per individual operation is now so low to be almost unmeasurable.

What are the best practices to achieve ultraquality in complex systems?

Meier and Rechtin make a strong case that while analytical techniques like Six Sigma and Robust Engineering Design will get you close, the addition of heuristic methods will get you over the top.  This includes using a zero defects approach not only in manufacturing, but also design, engineering, assembly, test, operation, maintenance, adaptation, and retirement – the complete lifecycle.

There are many examples how analytical techniques alone underestimate failure; for example the nuclear industry analysis of core damage frequency is off by an order of magnitude in reality.

A sample of applicable heuristics include:

• Everyone in the production line is a customer and a supplier [also extended to each person in the development team – engineering, supply, etc.]
• The Five Why’s
• Some of the worst failures are system failures
• Fault avoidance is preferable to fault tolerance in system designs
• The number of defects remaining in a system after a given level of test or review  (design review, unit test, system test, etc.) is proportional to the number found during that test or review.
• Testing can indicate the absence of defects in a system only when: (1) The test intensity is known from other systems to find a high percentage of defects, and (2) Few or no defects are discovered in the system under test.

[pie chart courtesy Boeing.  FBW = Fly By Wire]

There is a lot more material on “how-to” in the works of Meier and Rechtin, Juran, and Phadke.

Ultraquality requires ultraquality throughout all the development processes, and by extension throughout the delivering organization.  That is, certify a lack of defects in the final product by insisting on a lack of defects anywhere in the development process.  Developing both the processes and organization to achieve this state is possible, is being done in some organizations, and allows for superior business performance.

There are many examples how organizations lack ultraquality in their processes or organization.  General Motors is under heavy criticism these days following the Valukas report, which exposes the poor organization and development practices.  This is anecdotally impacting the GM dealers and turning them into ghost towns.

So back to the tagline: is your complex development project on track for ultraquality implementation?