In mid-2009, President Obama asserted "...government should be seeking out creative, results-oriented programs ... and helping them replicate their efforts across America." His first budget reflects this commitment.
The White House Office of Social Innovation expects a $50 million appropriation to leverage an additional $150 million in public and private funds for expansion of promising approaches in health, education, child and youth development, and economic opportunity.
I have been working with a group of public officials and social entrepreneurs at Harvard on just the questions raised by Feather in this column. The White House social innovation fund clearly targets programs that work. But true venture funding will not have a 100 percent success rate, and the more it incorporates new innovations the higher the rate of fatality. It is hard to imagine the Congressional hearing that compliments an agency for its efforts in funding failure. Feather offers good advice for applicants and public officials on how to get this balance right.
- Stephen Goldsmith
For the time being, the focus is on federal funds, but to reach scale state and local funding will be in play. One of the first places project proponents will look for new money, or "repurposed" money from other operating programs, will be state and local program managers and budget chiefs. Indeed, the substitutive element is already on the table. Peter Orzag, Office of Management and Budget director, emphasized in June that we want "to make sure that we invest in what works, do not waste taxpayer dollars on programs that do not work or are duplicative, and improve performance across the board."
On the surface, expanding successful evidence-based programs is compelling. Many of the nation's expenditures aimed at addressing persistent social ills, after all, fund programs which have not been evaluated for their benefits and long term cost-effectiveness. The numbers suggest we have yet to find reliable combinations of services to stem or reverse the socio-economic problems plaguing low-wealth communities. Given the leverage that experimental money and federal attention may provide, what could have greater appeal than directing scarce public or private dollars with greater confidence that they will achieve what is intended?
Of course, it's not that simple. Public managers intrigued by the possibilities of new program models should start with some critical thinking about how to judge the results of pilot demonstrations and to assess the implications of expansion. Philanthropic and non-profit advocates should anticipate and support the perspectives that public managers bring to these initiatives.
Here's a simple framework that may help.
Results. Is there quantifiable evidence that an approach or program has produced results? Is the program cost-effective? For example, does the front-end investment have pay-off in reduced public downstream costs? If cost-effectiveness is established in terms of, for example, improved health or school performance, how durable is the improvement? How rigorous is the evaluation? Was the evaluation conducted or supervised by an independent analyst? If the evaluation does not meet the gold standard of control group comparison, is there other evidence to suggest the positive results can really be tied to the investment?
Replicability. Often, as programs expand to new sites, administrators change the intervention, dilute the resources or skip important program "glue," such as program monitoring, quality assurance or staff training. Strong results arise from a combination of features in a pilot, relating to both content and style, and understanding which factors are most critical is essential to replication. How much did the program's success depend, for example, on a charismatic leader who was able to align diverse interests into a cohesive program operation?
The first question to ask in terms of replicability is whether there are examples of replications "by the blueprint," which have produced comparable results in differing sites, particularly sites similar to the expansion target. The second question to ask is whether it's possible to truly replicate a program at all - considering availability of resources, nature of interventions and administrative infrastructure. Customizing a program to fit a local situation may seem like good program design, but doing so may abandon the very elements that make the program work.
Scalability and Sustainability. Here's where the "demonstration" rubber hits the "budget reality" road. Two important considerations should guide public managers' thinking about downstream implications.
Many demonstrations are a complex mix of money and programs, or changed operations of existing programming and money. Good managers need to parse those elements and think ahead about further implementation. Ask the question, "If this program were a big success, what would we do about it?" Lay the groundwork early for obtaining a better understanding of the value of the program and building a constituency to support it.
A good public manager will seek to manage two downstream challenges - finding money to continue, and driving significant behavioral change deep into daily operations.
First, a replication or expansion that builds on a lot of new money will likely hit a brick wall - no matter how successful the results - when the money runs out. There's no reason to immediately borrow trouble by pitting ongoing operations against a new idea, but in an era of scarce resources the choices will have to be made.
Second, a demonstration that requires substantial reworking of daily operations or bargaining unit agreements presents different challenges going to scale. Grudging agreement to change may be relatively easy to achieve for pilots, but broad application will definitely produce pushback if constituency-building for reform hasn't been part of the plan.
Infusions of new money supporting fresh ideas can spark catalytic change. But if public executives want to use the leverage to produce systemic results, they need pay close attention to the blueprints and the tomorrows.
