What I Think About
Subscribe To The Salient Voice Journal
Main | Crowdsourcing The Destruction of Software Patents »

The U.S. Government is Developing A Useful Software Engineering Management Case Study

Obamacare’s tech rollout has crashed. As a software CEO with plenty of software background and absolutely no insight into the actual Obamacare project, I’m regardless noticing a couple quick lessons peeking out from the media noise. They largely point to some failures in project governance. So in that spirit, here’s a couple tests from which to assess the software projects that are presented to you. Actually these tests derive from very old lessons in building software systems … this is just a new chance to air them out.

1. Dates without technical analysis. It would appear that lawmakers set policy and software delivery dates without a functional scope or design of the software system. Further, over the first couple years of the software effort, lawmakers were still changing policy, the functional requirements, and other externalities regarding the system. You can’t set believable goals for engineering-dependent policies without a sufficiently precise analysis of the software engineering scope and effort. Obvious.  So management vision has once again has met engineering realities. And once again engineering realities have won. This mistake has been the end of more than a few MBA's and strategy consultants (and now lawyers). 


2. Change without costs. The project was in flux but the dates were not. Functional requirements can and do change. Software design is by its nature iterative learning and ongoing adaptation. This is unremarkable. But there are always costs to changes. These costs show up as (pick one or more): mid-project feature and software usability compromises (also known as unhappy software users), software performance and scalability problems from quick fixing internal design assumptions (no time to start over!), quality problems from increased fragmentation of software design and its testing (or from the minimization of testing to meet delivery deadlines) and productivity reductions caused by all this software refactoring and regression retesting. If your system requirements are “evolving” and costs and dates are not moving, you should worry because one, more, or all of the above are in your future.


3. Resources growth is not acceleration. Massive resources don’t necessarily yield proportional increases in software development outcomes. This Obamacare dev effort has been remarkably expensive and resource intensive. Here is something that has been long understood in software development. Fredrick Brooks defined it. He developed the idea of The Mythical Man Month (The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition (2nd Edition). Metaphorically it states as untrue the following: if one woman can make a baby in 9 months, then nine women should be able to make a baby in one month. On a more practical level, the deployment of people on your software projects at some point will increase your coordination costs without accelerating the project (it can even slow it down). The trick is how to figure out that tipping point. There was an optimal resource model and deliverables time frame by which to build this health care software platform. I wonder what it was.


4. Complex dynamic systems and the number and diversity of stakeholders. It’s hard to model economic systems in software. These systems and their actors (users) continuously evolve and their actions vary widely.  But software cannot have infinite and changing design use cases. Even if your software could be all things to all people, this won't all be in Release 1.0. So you must constrain the software. It must do less. But alas, if you do less, the software will naturally constrain choices within the economic system and thus make it less dynamic, less innovative, and, as a consequence, less adopted - if not actually avoided and circumvented. You must whittle down the core stakeholders and use cases to the critical core for a first phase project. This health care systems project has done the opposite with all care to all people. It's a governmental version of the 1990’s mega-ERP software projects. This era saw massive corporate spending on meta enterprise resource planning systems and related process reengineering (along with lots of consultants of course!). Then one day it became apparent that a focus on integrations and middleware between disparate smaller best-of-breed systems was better than deployment and adaptation (constraint?) to large monolithic platforms ... and these systems started to become diluted. An interesting software design question: if you build software that constrains users, will they stay constrained? 


5. Staged Rollouts. They just threw the big red switch to "on" for zillions of people. Very brave. In the real world, you stage rollouts of software to mitigate risks. If your development team goes for an all or nothing deployment, it's important to remember that one bug delivered to 310 million peoples is, well, millions of customer support phone calls (and then lots and lots and lots of marketing to overcome the ill will). Your simple management rule is this: the cost of bugs grow exponentially after they get deployed to customers. Look carefully at the scope of your team's testing and the open bugs before software rollouts. And regardless, stage software rollouts to an initially small but expanding group of customers, while incrementally fixing and extending the software features.

6. Scalability and performance. It’s clear that server capacity planning and volume stress testing was insufficient for Obamacare (or alternatively and less generously the planning was sufficient, the problems were indicated, and it was rolled it out anyway). More generally, when a system cannot handle user transaction volume, there are several possible reasons; the first involves the technology and tool stack used to build the software, the second is the software and data base design itself, and the third is network and server environment. Working backwards, money solves network and server problem easily, then software re-engineering will slowly (but at some expense) improve performance (especially if structural software rewrites are required), and a bad software development tech stack decision is a disaster and points to a redo (bad choices include wrong programming languages, wrong data base technology, tool sets, etc).


I’m sure there’s more. What’s interesting about these tests is that any world class software engineering manager knows them. So the troubles mean poor choices in hiring for project governance, or weak organizational design such that project management could not control the processes and resources (and therefore the outcomes), or weak development methods and processes, or political management and goals simply overruling the dev teams. Taxpayer dollars are helping write a great software project management case study.



PrintView Printer Friendly Version

EmailEmail Article to Friend