Governing Magazine/June 1997

FEATURE: PERFORMANCE MEASUREMENT

PERFORMANCE AND PAIN

As performance measurement sweeps through state and local government,
a lot of managers are finding themselves having to do a lot of
explaining. To advocates of performance measures, that's just fine.

By Jonathan Walters

Unflappability is clearly one of the principal reasons Charlie T.
Deane rose through the ranks to become chief of the Prince William
County, Virginia, police department. With a soft drawl, clear eyes and
a no-nonsense demeanor, he's not somebody you'd peg as easily rattled.
 But on a recent spring day, the chief was a little ticked off, and
for a pretty good reason. His department had come in for some very
public criticism over its "clearance rates" for crime.
 It seems that a performance audit of the county by one of the Big Six
accounting firms had turned up the fact that when it comes to clearing
cases, Prince William County's performance just doesn't stack up very
well when compared with a sample group of other jurisdictions
nationwide. The local newspapers got copies of the report and
proceeded to happily hammer Deane's department for what seemed to be
comparatively ineffectual work.
 The problem, though, is that Deane's department does good work. In
fact, given its resources, it does as good a job or better than just
about all the jurisdictions with which it was compared, and it has the
data to prove it. Unhappily for Deane, the reason for the big gap in
crimes cleared on paper has less to do with his department's
performance on the street than with how different jurisdictions
calculate clearance rates. Some consider cases cleared even when
there's no arrest and conviction but they feel confident they knew who
pulled the job; others consider an arrest enough to call a case
cleared, even if the defendant beats the rap. Prince William County is
much stingier: It only clears cases where there's been an arrest and a
conviction.
 Deane had to explain all of this at a hearing before his county board
of supervisors, who were naturally very curious about the chief's
story in the wake of all the bad press. In the end they were
satisfied, but Deane is still smarting from the experience. Being
sloppily compared to other jurisdictions was one of the things that
worried him most about performance measures to begin with. "My opinion
is that performance measures have real value as an internal management
tool," says Deane, "but those kinds of statistics shouldn't be used to
make comparisons."
 In spite of such legitimate reservations, upper-level managers and
elected officials everywhere are jumping onto the performance
measurement bandwagon, the downside be damned. If Deane has any doubts
about that, he need only walk down the hall to where his newly hired
boss, county manager Henry B. Ewert, sits. "I think it's good to be
uncomfortable," says Ewert. "I think it's healthy for the chief to
have to address the issues raised by that study."
 Ewert's is an increasingly common attitude nationwide, and is one of
the more fundamental reasons why performance measurement is gaining
momentum so rapidly. While Deane might have the sympathy of hundreds
of other public managers nationwide, his lament is not likely to
change a fundamental fact: The whole area of performance measurement--
eyed suspiciously just a few years ago as merely another reinventing
government-inspired, consultant-driven public management fad--is
looking much less faddish these days and much more like a public
administration juggernaut. If performance measures are tough to do
well, if they fail utterly in some cases to illuminate anything like
cause and effect, even if they sometimes invite unfair comparison--
well, that is just life, as uncomfortable as it might sometimes make
things for managers such as Deane.
 In fact, for years Jay Fountain, research director for the
Governmental Accounting Standards Board, has been coming back with
"Oh, just deal with it!" when confronted with all of the dog-ate-my-
homework reasons why governments couldn't possibly implement
performance measurement programs. What surprises Fountain is how many
people now seem to agree with him. Of 900 jurisdictions responding to
a recent GASB survey, around 400 said they are currently using
performance measures; another 300 said they plan to use performance
measures in the near future. "It's really kind of amazing the degree
to which this is bubbling," says Fountain.
 The interest and activity is timely, seeing as how GASB, which is
responsible for establishing all accounting rules for state and local
government, is very likely to mandate "service efforts and
accomplishments reporting" (the term of accounting art for measuring
what governments are doing to achieve certain public goals and how
close they're coming to succeeding) sometime shortly after the turn of
the century. Which means that even more jurisdictions than the
multitudes getting into it now are going to have learn all about
results-based government.
 That proposition is certainly less daunting for novice jurisdictions
these days because of the wide variety of models that are now out
there to copy; governments don't have to gin up performance
measurement efforts from scratch as they did just a few years ago. But
if novice jurisdictions are looking for some words of wisdom from such
performance measurement veterans as the state of Oregon--considered
one of the real performance measurement meccas in the country--the
words are now decidedly more sober than they were just a few years
ago, when Oregon officials were a bit drunk on their progress [see
"The Benchmarking Craze," GOVERNING, April 1994]. "I view part of my
role as trying to explain to people just how challenging all of this
is to do," says Jeff Tryens, the new executive director of the Oregon
Progress Board, which has the substantial job of continuing to
shepherd one of the nation's most ambitious efforts in performance
measurement through its natural progression from paper to practice.
 Challenging or not, states and localities are piling on. A recent set
of case studies pulled together by the American Society of Public
Administration's Task Force on Government Accomplishments and
Accountability lists not only Oregon and the perennial award-winner
for best-managed city, Phoenix, but also such newcomers as Catawba
County, North Carolina, and the state of Iowa. A consortium of
governments currently involved in the International City/County
Management Association's Comparative Performance Measurement
Consortium includes jurisdictions ranging from Long Beach, California,
to Hamilton County, Ohio, to Richmond, Virginia. Indeed, the practice
is catching on everywhere, from management bellwether states such as
Minnesota to cities as obscure as Leominster, Massachusetts.
 Even the federal government appears to be more serious about
performance measures, which although mandated by the 1993 Government
Performance and Results Act still seem like a dubious proposition
given the amorphous nature of many federal agencies' missions. On
Capitol Hill, infamous for its indifference to management issues, the
law's requirement that all major federal agencies have strategic plans
in place by this coming September is actually getting some attention.

But while action on performance measures is clearly bubbling across
the country, real progress continues to be the purview of a relatively
small number of jurisdictions. In fact, beginners might envy Charlie
Deane for being far enough down the road so that at least his problems
involve how measures are used, rather than how to develop and
implement them in the first place. Novices are all still wrestling
with the basics: What is performance measurement? How do we get
started? Where might it take us? How can it help? How can it hurt? And
what is the difference between an input, an output and an outcome--
never mind an interim outcome--anyway?
 In dealing with those questions, some jurisdictions are proceeding
cautiously; others have enrolled in crash courses, usually inspired by
the precipitous acts of blithe politicians. The South Carolina
Legislature, for example, decreed last year that the state's higher
education system would be on a 100 percent performance-based budget
footing by 1999. The decree has many in the education community in
South Carolina more than a little concerned. No state has ever tried
it before. "Some performance-based funding for public colleges and
universities makes sense," says Joseph C. Burke, director of the
Center for Effective Public Higher Education at the Rockefeller
Institute of Government, which is studying the use of performance
measures in public higher education, "but there's no way that such
complex institutions can operate with 100 percent of the budget up for
grabs every year."
 It's true that using the performance measurement process to bring
some reason and science to the messy business of governmental
budgeting is the logical next step. But jurisdictions that have been
at the performance measurement game for a while are finding that there
is nothing simple about making the jump to performance-based
budgeting. The budgeting process, after all, is essentially a
political one, driven as often as not by crisis, whimsy and pressure
from all points.
 Even veterans admit that just measuring performance--let alone using
those measurements in crafting budgets--is a work in progress. "We're
in year six of this, and we still regard ourselves as toddlers," says
Ara Merjanian, point man for performance budgeting in the Texas
governor's office. Merjanian is eloquent on the subject of just how
much work is involved in creating good measures, in making them an
integral part of an executive agency's budget document, and then in
trying to tune the legislature in to the meaning of performance
measures for the purposes of both program evaluation and budget
decision-making. "In the short run, this sort of information can make
a big difference in how you run individual agencies," says Merjanian,
"but it takes a long time for high-level decision makers, including
the legislature, to have confidence in the information they're
getting."
 In spite of the difficulty, a number of jurisdictions are using
results to drive budgeting. In Prince William County, for example,
budget director Craig S. Gerhart says the budget of one of the
county's drug rehabilitation programs was cut this year because of
poor participation rates. Performance data indicated that 75 percent
of its clients were dropping out before completing the program. While
participation rates aren't technically "outcomes," by the strict
definition of performance measures, county budgeteers were comfortable
making the logical leap: If 75 percent of clients are dropping out
before they even finish the program, the outcome of getting people off
drugs is probably not being well served. And so the decision wasn't
very hard to make, says Gerhart. "We said, `We're not interested in
buying that level of service. Let's put our money in another program.'
"

Performance budgeting is even rarer at the state level, where it's
obviously harder to make a direct connection between resource
allocation and street-level results. At this stage in the evolution of
performance measures, it is mostly a matter of getting legislators
comfortable with seeing dollars tied to outcomes. Adjusting
investments based on desired results is only just beginning to occur,
and only in a handful of places.
 In Texas, for example, the state Commission for the Blind has
calculated the cost of moving clients to self-sufficiency. Based on
that calculation, the commission now has some idea how much of an
appropriation it ought to be getting from year to year, depending on
its caseload. It is using that data to make its case to the
legislature. In Oregon, the experience has been similar. The Oregon
Department of Transportation's budget now moves through the
legislative budgeting process in days instead of weeks because of the
extensive amount of performance data ODOT includes in its budget
request.
 But while being able to illustrate in great detail for legislators
what previous appropriations bought is one handy aspect of doing solid
performance measures, it is hardly the holy grail, says Marv Weidner,
director of policy and strategic planning in the Iowa Department of
Management. The next step toward a performance-based budget would be
for lawmakers to begin adjusting appropriation levels depending on the
outcomes they're after. "Just changing the budgeting system is not the
goal. Why go down this painful path unless you want to focus on socio-
economic change? Using the budget to bend the trend lines is how we
want decisions to be made in Iowa."
 Budgeting being an inherently political act, probably the best that
any government can expect, though, is that performance statistics at
least inform debate. And while states such as Oregon and Texas are on
the cusp of that level of performance budgeting, much more common are
states that are either still struggling just to get measures in place,
or to make the measures they have in place even remotely meaningful to
the budget process. Mississippi, which embarked on its performance
budgeting effort in 1994, is still in its five-agency pilot stage, and
is still a year from having core data by which to evaluate
performance. Florida, which has made considerable progress in
developing measures, is nowhere near integrating performance data into
the budget.
 "It's just a lot easier to use measures for management purposes than
`We're going to give you money for outcomes,' " says Karen Sanford,
executive director of the Florida Commission on Governmental
Accountability to the People, a citizen board that oversees the
state's performance measurement effort. "That would mean giving up
some control, and legislators don't want to do that."
 Indeed, finding jurisdictions that use performance measures as a
management tool is a relative snap. Take, for example, Long Beach,
California, which in the early 1990s was actually considering
eliminating its police department and contracting the job out to the
Los Angeles County Sheriff's Department. The reason was pretty simple:
Between 1983 and 1990, crime in Long Beach increased 30 percent
overall. Violent crime doubled. Morale in the department was so bad
that it was having a hard time recruiting and keeping personnel, which
meant that even as the population of Long Beach was increasing, the
Long Beach police force was shrinking.
 Rather than turn the whole job over to the county, however, the city
council voted 5-4 to offer its department a performance-based
reprieve. And so, in conjunction with the city manager's office, the
department embarked on an ambitious performance measurement plan to
refocus the police on 14 broad goals, such as "improve community
access to the police department," "improve patrol response time,"
"improve services that support patrol and investigations" and "reduce
illegal gang activity." Using the goals, the department came up with
about 100 separate measures by which to gauge progress toward those
goals, measures that include factors like response times, those pesky
clearance rates and citizen surveys (which are rapidly becoming a
staple of aggressive performance measurement programs). Using the
goals and the measures, the department developed a much more
preemptive and community-based policing strategy, while also dealing
with the department's morale problem, including getting police
officers a 10 percent raise.
 Again, the issue of cause and effect is always debatable, but in the
five years since the department started to closely track its
performance--1991 to 1995--violent crime in Long Beach fell 38
percent, property crime dropped 19 percent, and "gang-related
incidents" fell nearly 40 percent. At the same time, the percentage of
citizens who say they feel safe in their neighborhoods went from 50
percent in 1993--the year the city began surveying citizens--to 64
percent in 1995. Scott Bryant, a consultant and former director of
strategic planning for Long Beach, argues that if those numbers aren't
convincing enough, consider the fact that while Long Beach's violent
crime rate was falling 38 percent, the average decline nationally was
only 10 percent. "I know some will say that it's demographics or the
economy, but only 12 cities had greater rates of decline than Long
Beach from '91 to '95," Bryant says.
 Probably the most amazing thing about the performance measurement
trend is that even managers in jurisdictions that some might regard as
highly unlikely candidates for the technique are getting into the game
in policy areas considered to be among the toughest in which to
develop good measures and anything like a correlation between cause
and effect.
 Brian J. Wing, acting head of New York State's sprawling Department
of Social Services, is as good an example as any of the new breed of
manager, determined to bring some accountability to a system that in
his view was devoted in previous years to doing little more than
writing checks and punishing local governments for not following the
rules. Wing has very little patience for anyone who wants to cavil
over the legitimacy of performance measures as a general matter, and
he is entirely dismissive of the notion that performance measurement
is especially hard to do in the human services area. "In fact, we had
a lot of the data already in hand, it's just that we had no way to get
it out of the computer," he says. Wing's lament is actually fairly
common among public sector managers. Many say that they have plenty of
good data already collected, but their information technology systems
were never set up to gather, disaggregate and then disseminate it
centrally. And so part of the performance measurement effort in his
department has been to bring both the department's technology, along
with staff, into the new world of results-based government.
 Wing and his top staff have been pushing performance measures with
such singular determination for a couple of years now that his
department is currently producing quarterly performance reports broken
out on a county-by-county basis in areas from "percent of human
resource cases closed due to earnings" to "percent of out-of-wedlock
children with paternity established."
 Not everybody is wild about Wing's work, though, particularly those
county officials whose performance comes up short in the quarterly
reports. The complaint--a corollary to Charlie Deane's lament in
Prince William County--is that some counties that look like laggards
actually have some pretty good socio-economic and even geographic
reasons why they might be having a hard time jacking up "percent of
human resource cases closed due to earnings." Wing is understanding of
that, and he allows all his counties to include explanatory notes in
each quarterly report. But he's not going to stop gathering or
publishing the data. In fact, he doesn't seem to mind at all that the
monthly reports regularly cause a bit of discomfiture among his
counties. "I actually think the competition is good for everybody," he
says.
 While not everybody would agree with Wing, he might actually now have
an unlikely convert in the form of Charlie Deane, the Prince William
County police chief. That's because in Deane's latest budget go-round,
he was able to illustrate how short-staffed he was compared with his
compatriots in other counties. That was enough to persuade the county
council to up his annual manpower allotment. But the cash comes with a
caveat: In return for the fresh troops, Deane will be held to a new
and higher level of performance.

----------------------------------------------------------------------

Copyright 1997, Congressional Quarterly, Inc. Reproduction in any form
without the written permission of the publisher is prohibited.
Governing, City & State and Governing.com are registered trademarks of
Congressional Quarterly, Inc.
http://governing.com