To make sure it's done right, we compiled six time-tested precepts that will guarantee the pilot produces useful results.
Avoid the Piecemeal
For example, about a decade ago, Minnesota embarked on a $100 million license and registration system. Joel Alter, a director in the state's legislative auditor office, reports that although each component of the system was tested independently, "you have to do end-to-end testing to make sure they work together. That wasn't done."
Unfortunately, the system as a whole failed. The state is continuing to use the previous system while it begins work on an alternative. It has little to show for the $100 million spent.
Keep an Eye on the Big Picture
Pilot programs can be used to help begin major initiatives with minimal expense. Public administrators can be risk adverse, notes Bill Leighty, former chief of staff to Gov. Mark Warner. Pilots, he says, allow incremental "decision-making, which can be far more appealing to decision-makers, as they are far less risky to their careers than big bold programs."
Spread the Wealth
Pilots should be staged so that they are reflect all the populations and regions that might use the program.
While a pilot program limited to Utah's densely populated Salt Lake City area may prove to be successful, for instance, there's no reason to have confidence that it will work well in sparsely populated portions of the state if it isn't piloted there as well. "Pilots fail because they fail to take into account the differentiation between where the pilot was done and full application," says Leighty.
A subsection of this point has to do with who is chosen to run the pilot. One particular hazard is that the departments or the communities that are used for pilots may simply be the first ones to raise their hands. As a result, they may not have the infrastructure or environment in place to give the pilot a fair chance, and so a perfectly good notion is discarded.
Get Buy-In from Lawmakers
If legislators aren't informed and onboard before the test run, a pilot's findings can be overwhelmed by political considerations. "The analysis doesn't always carry the day at the legislature," says Larry Jacobs, director for the Study of Politics and Governance at the University of Minnesota.
This point isn't a surprise to us, given the frequency with which good solid information doesn't go far to inform policy decisions. This is especially true when the program or policy may be politically unpopular.
Be Scalable
Those in charge of running the pilot need to make sure that -- if it works well -- the project can grow to cover the entirety of an entity. That is, if you just have a pilot without a plan for scaling it, the program is unlikely to be widely utilized.
Go With Piloting Experience
Several experts we talked to told us that sometimes the people who are running the pilot haven't done so before. There is a science to it. Although individuals may be well-intentioned, they may be unqualified to run a pilot effectively.
