States have long resisted being compared to one another by Washington, just as localities have resisted comparison by their states. Years ago, a major city in my state complained to the state's senior environmental official about an annual report that ranked communities according to their recycling rates. The reason for its concern? The city was embarrassed by its low position on the list. Rather than help the city increase its recycling rate or fix unfair aspects of the measurement system, the state simply stopped issuing the annual report.
Resistance to comparison is understandable, but unfortunate and self-defeating. When done well, comparison is a powerful tool for improving performance. Its greatest power comes not, as often assumed, from triggering competitive instincts to fight for the top spot or to avoid the bottom. Instead, comparison's chief value lies in its ability to illuminate government actions that work well and merit replication, as well as those that do not and are in need of attention. Useful comparison does not require unfairly comparing the big and the small, the rural and the urban, or the rich and the poor. When care is taken in the design, collection and analysis of comparative data, it enables all levels of government to learn from natural experiments taking place every day in states and localities across the country, despite their differences. It builds shared knowledge that improves the effectiveness and efficiency of government programs.
Consider one way in which the U.S. National Highway Traffic Safety Administration uses comparative state data. Every year, NHTSA gathers and studies traffic fatality data from all 50 states. One year, NHTSA noticed a significant drop in California automobile fatality levels not evident in other states. Puzzled, it contacted the state to ask why. California explained that it had adopted a "primary enforcement" law giving police authority to pull cars over to check safety-belt use. NHTSA began encouraging primary enforcement laws in others states, first to observe whether the beneficial effect would be reproduced, and when it was, to promote primary enforcement law adoption by all states. Today, half the states have primary enforcement laws. With annual comparison and analysis, NHTSA helps states learn from and apply the lessons of each other's experience.
Comparison helps identify and confirm not only successes, but also problem areas. Part of NHTSA's work is to study the natural experiments taking place in the states. When Kentucky and Louisiana relaxed their motorcycle helmet laws, NHTSA commissioned a study to measure how changes in their motorcycle fatality and injury rates compared to changes in states with unchanged laws. Following repeal of the helmet law, the motorcycle fatality rate increased 37.5 percent in Kentucky and 75 percent in Louisiana, compared to an 18 percent average increase for all states in the same period. The injury rate in Kentucky and Louisiana went up 17 and 21 percent, respectively, compared to a 3 percent decline nationally. NHTSA disseminates this analysis to encourage stronger motorcycle helmet laws in all states. It also provides marketing material to support local advocates pressing for more protective helmet laws.
Comparison is useful not just across jurisdictions, but also as a handy management tool within a single government. A permit-issuing agency, for example, can learn by comparing the permit response times of its field offices. If response times slow across all offices at the same time, it suggests that factors outside the offices may have caused the slowdown. If delays increase only in one office, however, it suggests an office-specific problem needing attention. Smart governments do not compare in order to penalize poor performers but to get early warnings of problems and to fix those problems before they worsen.
Despite its ability to produce useful insights, only a few brave communities enthusiastically embrace comparison. Most ferociously resist it. They fear showing up at the bottom of comparative listings or falling from their prior-year position, especially if the local media is watching. A few actually don't like being at the top, knowing that they have nowhere to go but down and fearing they will alienate goal-sharing allies whose cooperation they need. In addition, all communities resent being compared unfairly to communities with notably different characteristics that affect outcomes.
Does comparison have to be embarrassing? Not terribly, if comparison is designed primarily to support learning. Several comparison techniques keep the focus on learning. Compare trends to find divergent ones, as NHTSA does. Look for significant changes in outcomes evident in one or a few communities but not in others, and follow up to see if the observed variance can be explained by the adoption or cessation of a specific government practice. Also, compare multiple dimensions of performance, but avoid combining multiple dimensions of performance into a single numeric index for ranking or grading. Indexed comparisons may get a headline, but provide little insight about what or how to improve performance. And compare government performance of preventable and commendable causal factors that affect target outcomes, such as the effect of alcohol and seat-belt use on fatality rates. Comparison of causal factors spotlights opportunities for performance gain.
To avoid comparing apples and oranges, facilitate peer comparison. The nonprofit Education Trust maintains a Web site that makes it easy for the public to compare schools of similar wealth and racial composition to search for the "positive deviants," schools that outperform their demographic peers. This comparative analysis focuses follow-up studies to search for effective teaching practices and conditions.
Because all federal agencies are required by law to measure outcomes, comparison is likely to happen eventually for any state and local program supported with federal funds. Anticipate that likelihood. Work with peers and other levels of government to create comparative measurement systems that advance learning. Press for state and federal agencies to create measurement systems that support learning, not reward and punishment, as their primary objective.
It is tempting to resist comparison, especially when proposed by another level of government. Resistance, unfortunately, supports ignorance.
