Category Archives: Uncategorized

A more useful Requirements Process Maturity Model

Posted on by

A useful diagnostic tool to help determine problem areas and areas for improvements are maturity models.  They can be used by both the client and consultant to determine the current level of performance.  The target level of performance doesn’t need level 5 (highest capability) for everything, as that is likely too expensive or difficult to achieve, or not necessarily needed.

One of the best ways of improving technology and product development is for your organization be good at developing and managing requirements.  About 70% of problems in technology and product development come from requirements and system interaction errors, and fixing these problems at the final acceptance test or in the customer’s hands costs about 100 times more than fixing them in the requirements development and management phases of the project.  Basically build the right thing, build things right, and find problems early.

For requirements development and management, there are a few maturity models published, but I have found them too specific to an industry (like for business analysts in the software industry), cover only certain aspects, or don’t cover integration, training, or culture well enough.  So I’ve developed the above model based on similar models from consulting houses, CMMI, Six Sigma, Model Based Systems Engineering, PLM, and my own background.  I think this can apply to all kinds of systems, from hardware-oriented (manufacturing, construction), software-oriented, or combinations of both.

Requirements Process Maturity Model

(click for full size)

Using this tool can then help structure the problem, ask the right questions and prioritize opportunities. Where does your organization stack up?

If you have comments or questions on the model, or have ideas for improvements, please contact me.

 

 

 

 

System Level Website Failures – Technical, Process, and Organization

Posted on by

In BC, we recently had a windstorm that knocked out power in the province for over 700,000 people, some for 4 days.  One of the most difficult parts of the outage was that the BC Hydro Website that provides outage updates also went out at the same time. This made it very difficult for people without power decide on what to do, where to go, what to do with the food in the refrigerator, etc. and made for many unhappy customers.

Many critical websites are complex systems, and fail more often than desired.  A good example was the failure of the ObamaCare’s HealthCare.gov website launch where there were serious technical problems at the rollout, which has subsequently taken about 6 months to fix the major issues. On launch day, as soon as the website hit about 2,000 simultaneous users, the website performance became unusable, which was an issue since on the first day, 250,000 simultaneous users tried to get access to the website. There are many other problems with the Healthcare.gov, as that project had large budget overruns, with $1.7 Billion dollars spent, which is about 10 times more than budget and what it should have cost. There are also lasting data and security problems with the website and internal database.

healthcare.gov-crash-1

The majority of the root causes of the Healthcare.gov failure were systems-level failures in all three major dimensions of any complex system delivery: technical, process, and organization.

  1. Technical: The system design used an outdated 1990’s database server model that doesn’t scale well with many concurrent users, as opposed to using a more typical e-commerce server model that can scale with users.
  2. Process: The system development process used a waterfall approach to build most of the website and then test it, vs. an agile approach where you test the important parts all throughout the development process.  Additionally there was very little testing during the development.  They were even off by a factor of five on the concurrent user requirement.
  3. Organization: The organizational system of the Government and the Contractor were poor with too many delays, last minute changes, poor subcontracting, poor reporting, and poor coordination.

BC Hydro is conducting a root case investigation of their website failure.  Perhaps the root cause was a simple and isolated issue, but I am interested to hear when the investigation is done on whether the failure had similar systems-level causes like the HealthCare.gov launch failure. For any complex interrelated technical, process and organizational complex problem, the Systems Approach is the best way to develop a solution that satisfies the overall needs and meets the expected behaviours of the system.

The Power of Systems Engineering

Posted on by

The Problem

Complexity is increasing everywhere, with increased software, connectivity, public policy issues, and development cycle acceleration. The trend is to add features and functionality, often implemented in software. Software size and complexity are growing exponentially, with the marriage of hardware and software enabling systems-of-systems – with examples being portable phones or airliner cockpits.

New integration problems result from combining rapid technological advancement and obsolescence, increasingly complex hardware and software evolution, and a migration to increasingly software based systems. Yet an increase in software leads to increased interfaces and an increase in integration problems. Software interfaces are not as “transparent” as mechanical interfaces and go beyond inputs and outputs. Additionally complex system interfaces are crossing multiple suppliers, both hardware and software. Overall the trend is that it is not going to get better, it is only going to get worse as software lines of code increase, and integration, verification and validation efforts also increase.

Success is getting harder in the “New Normal”.

  • 50% of product launches fail to live up to company expectations
  • 33% of new products fail to provide a satisfactory return
  • 70% of resources spent on new launches are allocated to products that are not successful in the market
  • 80% of projects cost 20% more person-hours to launch than initially forecast

Problems that arise from unmanaged complexity are no longer affordable. For example there were 18 million vehicle recalls in the US in 2012, which is more recalls than vehicles sold, and each recall costs $100/vehicle/recall leading to $1.8 billion in direct costs!

The Solution

The Systems Engineering Approach is the most effective way to manage complexity and change. Reducing the risk associated with new systems or modifications to complex systems is one of the primary goals of the systems engineer. Solving problems early in the development cycle saves enormous costs and time in the later phases of development. Costs and schedule overruns lessen with increasing systems engineering effort.

Better to Find Problems Early!

Things that go wrong include:

  • Weak or non-existent basis to requirements
  • Inadequate costing and time-scale estimation
  • Weak control of suppliers and subcontractors
  • Integration problems
  • Inadequate test and acceptance strategy

The project who’s turn to be “it”:

  • Scope problems
  • No overall orchestrated cracking of the whip
  • No idea of costs at the outset
  • No system-wide vision
  • An ‘odds and sods’ approach
  • Bitten by unproven technology

Hidden benefits:

  • Risks that didn’t materialize
  • Rework that didn’t need to be done
  • Customer complaints that didn’t occur
  • Product deficiencies that are circumvented

Engineering roles are changing – what an engineer does and is expected of an engineer now includes broader market, financial and social issues.

It is difficult to train good systems engineers and foster “systems thinking” and get organizations to apply the “systems approach”. The systems approach has both an art and science component to it, and like similar disciplines like medicine, it can be taught by expert practitioners, typically from industries that successfully develop complex systems.

Whether you are developing a product or a process, a well implemented systems approach produces superior performance.

Insight and Heuristics in System Architecting

Posted on by

 One insight is worth a thousand analyses

iron man simul

-Engineering and Art: Iron Man 3

Systems Architecting is as much art as it is science.  The best book on this subject is from Maier and Rechtin, and I highly recommend it.

ArtArchitecting

-Maier and Rechtin, The Art of Systems Architecting, second edition, CRC Press, 2000

One of the best section of the book deals with using the method of Heuristics in architecting.  Insight, or the ability to structure a complex situation in a way that greatly increases one’s understanding of it, is strongly guided by lessons learned from one’s own or others’ experiences and observations.  Given enough lessons, their meaning can be codified into “heuristics”.  Heuristics are an essential complement to analytics.

As in the previous post, where the system engineer is to consider the whole and apply wisdom, Maier and Rechtin also promote the use of wisdom but they note that “Wisdom does not come easy”

  • Success comes from wisdom
  • Wisdom comes from experience
  • Experience comes from mistakes

While required mistakes can come from the profession as a whole, or from predecessors, it also highlights the importance of systems engineering education from those skilled in the art.

Examples of heuristics are:

  1. Don’t assume that the original statement of the problem is necessarily the best, or even the right one
  2. In partitioning, choose the elements so that they are as independent as possible; that is elements of low external complexity and high internal complexity
  3. Simplify. Simplify. Simplify.
  4. Build in and maintain options as long as possible in the design and implementation of complex systems.  You will need them.
  5. In introducing technological and social change, how you do it is often more important than what you do
  6. If the politics don’t fly, the hardware never will.
  7. Four questions, the Four Whos, need to be answered as a selfconsistent set if a system is to succeed economically; namely, who benefits?, who pays? and, as appropriate, who loses?
  8. Relationships among the elements are what give systems their added value
  9. Sometimes it is necessary to expand the concept in order to simplify the problem.
  10. The greatest leverage in architecting is at the interfaces.

-taken from Maier and Rechtin, The Art of Systems Architecting, second edition, CRC Press, 2000

 Heuristics are tools, and must be used with judgement.  The ones presented in the book are trusted and time-tested.  They may not apply specifically to your complex systems architecting work, though I think you will find most of them do.