Thursday, November 14, 2013

Why is broken? Blame the government procurement process

"Over the past 10 years, 94 percent of large federal information technology projects were unsuccessful, according to a study by the research firm the Standish Group. More than half of them were either delayed or over budget, or fell short of user expectations. More than 41 percent failed completely."
        - From the Nov. 8 issue of The Week, full story in Computerworld.
The Standish Group study received a considerable amount of media attention, and deservedly so. Its database of 3,555 government projects with labor costs of at least $10 million revealed only 6.4 percent were successful. Fifty-two percent of the large projects were "challenged" (over budget, behind schedule or didn't meet user expectations) and 41 percent were failures (abandoned or restarted).  
"They didn't have a chance in hell," said Jim Johnson, founder and chairman of Standish, of There was no way they were going to get this right - they only had a six percent chance." - Computerworld
I am not aware of a similar study of California government technology projects, but suspect the figures might come out about the same. (And in all fairness, a study of private technology projects might yield similar findings). 

When President Barack Obama promised to find out why the Affordable Care Act website failed miserably, the answer came to me in about two seconds: the government procurement process deserves a good share of the blame. 

It takes years to plan and execute a major governmental technology project. Staff turn over, technology evolves and expires, administrations change hands. It is not flexible or timely in any way, and yet that is exactly what technology projects need to be if they are to succeed.

Here are a few recent large-scale California government technology upgrade failures:
In each instance, tens or hundreds of millions of dollars were spent before the plug was finally pulled. And those agencies continue to serve the public with poor, outdated technology.

The large-scale California government technology project I have been tracking is VoteCal, a new statewide voter registration database to replace the one built in 1995. 

When VoteCal becomes operational, we will have statewide voter lookup tools so all California voters can easily check their registration or ballot status online - conveniences enjoyed by voters in most other states. We will be able to register 17 year olds to vote, and implement Election Day registration. Our registration records will be more accurate and California's embarrassingly low rate of registration (45th in the nation) could increase due to the greater efficiency and accuracy technology improvements can bring.

VoteCal has been in development since 2006 and already failed once. I'm hoping the Secretary of State and Department of General Services will succeed in keeping it on track this time around and so far the updates indicate progress is being made. But the new system is still four years out. It is not scheduled to be in operation until 2017. It's hard to imagine the technology they are planning for today will still be state-of-the-art by 2017 (and that assumes the project is not further delayed). 

Whether the government procurement process itself will ever be modernized to embrace rather than inhibit qualities like flexibility and timeliness remains to be seen. Some excellent suggestions recently appeared in Politico and the New York Times, which highlights the "agile software approach." The Times also reminds us that government technology successes include the creation of the Internet itself, which in turn has brought about online access to a whole host of government services (most recently, and successfully, California online voter registration) whose availability would otherwise be severely limited.

Governments are not like private businesses - we are not governed by autocrats who can push and abuse their employees the way some titans reportedly do. But we must find a way to update our procurement process so government agencies can put technology to uses that best serve, rather than inflame the public.