Governing Magazine/April 1998

GUIDE TO TECHNOLOGY

PERFORMANCE MEASUREMENT'S ESSENTIAL INGREDIENT

By Christopher Swope

For a decade, New York City's parks and recreation department sent
inspectors into parks to rate the performance of its maintenance
teams. The inspection crews compiled massive reports covering all
1,500 city parks. The task was so overwhelming that the report could
only come out three times a year, and by the time it came out, the
data was stale and too mountainous to be useful.
 Three years ago, the department overhauled the inspection program and
bought four hand-held computers. The computers fit in the palm of one
hand, and show a menu of the 12 parks features, from lawn condition to
graffiti, that inspectors rate as either acceptable or unacceptable.
Inspectors enter information through a touch-screen keypad and can add
additional comments through a handwriting feature. At the end of the
day, the data is uploaded straight into the database that the
department uses to churn out performance reports.
 The computers created the obvious benefit of automating the data-
entry process, which used to require transcribing handwritten notes
from inspectors' notebooks. But more important, by speeding the data-
collection process, the technology allowed a complete revamping of the
inspection program. The department ditched the voluminous triennial
reports in favor of bi-weekly performance reports on roughly 115 parks
at a time. With fresher, more workable data, Parks Commissioner Henry
J. Stern began putting pressure on managers to take more immediate
action. "With a report from just the last two weeks," says Joshua
Anderson, senior adviser to Stern, "the borough parks commissioner can
easily track down who was responsible for that site on the day of the
inspection. There's more accountability."
 Once attention focused on the performance data, conditions in the
parks began improving. In the summer of 1995, 44 percent of the parks
were rated as acceptable for overall condition. By this winter, the
rating had almost doubled, to 83 percent.
 What New York's parks department is doing is applying information
technology to performance measurement, the public management technique
that has become a juggernaut in state and local government. Government
officials traditionally have thought in terms of the resources that
went into government programs, but the growing tide of performance
measurement instead emphasizes what comes out. An environmental agency
that once was measured by the amount of money and manpower given it,
for example, might now be judged by how clean the air and water are.
 Just about any kind of information technology that might come to
mind, from the simple computer spreadsheet to complex geographic
information systems, is being put to use in hundreds of efforts to
remake governments using the performance measurement approach.
Performance measurement requires crunching data about government in
new and unusual ways; computers make this new focus on results
possible. "Technology is an important part of why performance
measurement will work, if it works," says Jeff Tryens, executive
director of the Oregon Progress Board.

LIVING NUMBERS
Technology is proving to be an essential ingredient in performance
measurement because it helps in three primary ways: in collecting
performance data, in processing or analyzing it, and in disseminating
it to decision makers or the public. And technology can support
performance measurement in delivering any number of government
services.
 One of the most heralded examples comes from law enforcement, where a
growing number of cities are using GIS-based systems to track the
prime performance indicator for police: crime. The computer systems--
New York City's pioneering version is known as "Comstat"--overlay
crime data on a map to graphically show where police efforts are and
are not working; the data can then be used as a basis for
redistributing police manpower.
 The graphical presentation in the case of Comstat and similar systems
in use by police around the country is important because it puts the
performance measures in a useful form. Tables of statistics are only
good until the eyes of the person reading them glaze over. But with
GIS, or even with a graphics-capable spreadsheet, the numbers come
alive as trends. They become a way to visualize performance.
 "Technology is most valuable in the display and communication of
performance information," says Dick Tracy, a performance measurement
expert in the Portland, Oregon, auditor's office. The auditor is now
using simple desktop publishing software to produce a tabloid
newspaper called The City Review, and it is full of performance
graphs, maps and pie charts for a citizen audience.
 The need for visualization comes not just from the blandness of raw
numbers but from the multitude of them. In a given state, there might
be dozens of performance indicators for a particular program, hundreds
for an agency and thousands for the entire government. Making sense of
that morass of data requires either good technological support or long
hours of flipping through thick reports.
 Just as important to making performance measurement work is
timeliness of the data, and technology can make loads of difference
here through automation. Part of the idea behind measuring results is
that decision makers have flexibility in choosing a rapid response.
But that flexibility is worthless if the data is out of date. In the
Comstat example, information on where crime concentrated six weeks ago
would not help police craft an effective response today. Any way that
governments can find to collect and process performance data faster--
giving police laptop computers to file crime reports from the scene,
for instance--is a boost to performance measurement.
 On the other end, it is essential that decision makers have easy and
direct access to where that data is stored. "Time is very important in
performance measurement," says Michael Lombardo, manager of the
performance measurement program at the International City/County
Management Association. "Access to the most recent information is key.
The best case is when you have direct access from your desktop, rather
than having to call somebody else and have them put it together for
you."
 Part of the challenge in integrating technology with performance
measurement is to align government computer systems with the new
results-oriented thinking. Generally, systems are still geared toward
measuring transactions (how many building inspections were done, for
example) rather than outcomes (such as whether building conditions
improved).
 This challenge to measure results is especially pressing today as
states and counties update their information systems for welfare
reform. The federal welfare law changed the focus of welfare from
getting checks to anyone eligible to getting people off the rolls.
Many states are considering adopting outcome indicators for welfare
programs, and Washington will be giving states bonuses according to
broad performance measures, such as success of former recipients in
the work force. The new school of thought is requiring millions of
dollars' worth of system overhauls and countless programmer hours.

SIMPLE TOOLS
Despite all that spending and effort, perhaps no single technological
tool is more useful to performance measurement at this point than
spreadsheet software. At the program or even agency level, it is often
all that is needed to calculate, sort and analyze performance
information.
 It gets more difficult, however, when governments try to assemble
large quantities of data across disparate programs or agencies to get
a broader sense of outcomes. This is especially important in places
moving toward performance-based budgeting, which ties funding
decisions across agencies and programs to outcomes. That requires a
government-wide view of performance data, and officials are finding
that while technology is up to the task in general, their own needs
are very specific and vendors are not yet producing the kind of
software they want.
 "The application of information systems to performance measurement is
still pretty primitive, compared to what's out there for other areas,"
says Enrique Balaguer, a policy analyst in Boston's budget office. "In
part that's because no one has been doing performance measurement long
enough, and it doesn't yet have the status of, say, auditing or
accounting, where big firms jump in to develop tools."
 Boston cobbled together a solution for its own performance
measurement system by adapting the budgeting software it was already
using. "Our budgeting database has fairly strong capabilities in
dealing with budget line items," Balaguer says. "We tricked it into
thinking that performance measures were also line items."
 A database query can bring back the whole menu of customer service
measures across 52 city departments on one screen, Balaguer says. A
manager could ask, for example, "Where did we meet the promised level
of service?" and the system gives the answer quickly. Side-by-side
performance comparisons are made easily. The mayor's office and budget
office use the system for making budget decisions, and departments use
their own slice of data for program management.
 While the jerry-built system works pretty well, Balaguer says, it
does have technical limitations. It is not as flexible as he would
like it to be, and does not allow analysis of the raw data used to
create the performance measures.
 In Louisiana, the creation of a performance measurement database was
one of the requirements of Act 1465, the state's performance budgeting
law passed last year. State agencies have begun populating the
database with the first rounds of performance reports, marking the
first time that the information has been available across government
in a searchable, electronic form. It is expected to be widely used by
budget and legislative staff, who will evaluate the data in assessing
rewards and penalties based on agency or program performance. Auditors
and agency managers will also use the database, which runs on the same
software that agencies already use for the government's finance
system.
 Each quarter, agencies are expected to report detailed information on
their performance indicators, including prior-year performance, a
target performance level for the current quarter given appropriation
levels, and actual performance for the current quarter. The database
calculates variances between target performance and actual, and tracks
revisions to performance standards made throughout the year.
 Programs are coded with keywords, which is important for tracking
performance of services that fall under several agencies' purview. "We
have umpteen zillion literacy programs in different agencies, but to
compare how each of those entities is doing is difficult," says
management analyst Carolyn Lane of the Office of Planning and Budget.
"With the database, we can key in `literacy,' and up will come all of
the literacy programs in the state. For budgeting, it's a great tool
to look across jurisdictions, or to look historically at how these
programs performed."
 Reporting this information into the database is about to get a whole
lot easier. During the first year of operation, agencies are sending
in the performance data in spreadsheet form on disks. Starting in
July, however, agencies will use a new Internet-based reporting tool
that will allow them to enter data directly into the database from the
World Wide Web.

ROLE OF THE WEB
The Web is also proving itself useful to performance measurement as a
means of disseminating information on government performance, both to
decision makers and to the public.
 In January, Florida unveiled a painstakingly thorough Web site called
the Florida Government Accountability Report, also known as FGAR. Run
by the legislature's research arm, the Office of Program Policy
Analysis and Government Accountability, or OPPAGA, the site is a
virtual encyclopedia of Florida government, and is loaded with
performance measures. It is intended for legislators and staff as a
comprehensive source of agency and program information to be used as
the state gradually phases in performance-based budgeting.
 The site (www.oppaga.state.fl.us) is searchable via a handful of
different methods--by agency or by broad policy areas, for example.
One can also search by performance ratings, meaning that a search
could, for example, turn up all poorly performing programs rated as
"needs major modifications," or all highly rated programs rated "meets
expectations."
 All programs that are currently budgeted by performance have
performance data included with their profiles. Profiles include
performance targets set by the legislature for the current year, and
where available, historical data for recent years' performance levels.
Also shown is information on benchmarks relevant to the program
published by the Florida Commission on Government Accountability to
the People, a citizen board.
 The key feature of FGAR, however, is that the performance information
is put into a larger program context. Links to performance data appear
on the same page as budget information that lets the user know how
much money and how many employees the program has to work with.
Changes in performance might be explained in the links to pages that
describe recent legislative changes to the program. Each profile also
includes links to the sections of Florida code that describe the
program's statutory authority, as well as descriptions of recent
government reports and audits on the program. Finally, each report
includes contact information for the OPPAGA person and the agency
person responsible for the program.
 "There are huge advantages to the Web environment," says Karen
Stanford, executive director of the accountability commission. "Often,
legislative staff or executive-branch people are working at midnight
to get a report out, and they need specific information about a
program. You would have to go to the library to get all the
information on FGAR, but now all that information is online."
 Oregon, which has probably more performance measurement experience
than any state in the country, is also finding the Web a useful tool
to report performance data to a more public audience.
 The Oregon Department of Transportation is experimenting with putting
piles of performance data on the Web
(www.odot.state.or.us/fsb/offprod/tmr/index.htm). The site breaks down
layer upon layer of detailed graphs covering performance measures from
maintenance to safety to customer satisfaction surveys. The data are
broken down by five Oregon regions, so taxpayers checking in can see
how transportation's performance in their region stacks up against
others.

FISHY DATA
Oregon officials are hoping the gee-whiz quality of another Web site
can help rouse energy around a massive cross-agency effort to restore
the state's salmon population. In this case, improving the broad
outcome measure--more salmon--is requiring the involvement of 11 state
agencies.
 The effort is being monitored on a new site at www.oregon-
plan.org/progress.html. Layers of graphs show how the quality of
salmon habitat is changing in different Oregon watersheds over time.
Graphs provided by different agencies provide possible explanations:
The transportation department, for example, posts information on how
many stream-choking culverts it has yet to replace in each watershed.
Likewise, Oregon police provide performance information on compliance
with sport-fishing regulations. The information is updated frequently,
so improvements or declines can be tracked over time.
 Getting all the players assembled on the project hasn't been easy,
says the progress board's executive director, Jeff Tryens, but he
hopes the graphical, easy-to-use format of the Web will stimulate
interest in the salmon project. "We have such a cool platform for
providing the information that people want to be on the Web," he says.
"If we can make it up-to-the-minute and readily available to lots of
people, we'll have hundreds of junior policy analysts out there saying
recovering the salmon population is what we really need to do."


THE BUMPINESS INDEX
One might never expect a smooth ride from a New York City taxicab, but
the city is hoping to keep its streets from sharing the blame for
that. Technology is making it possible for the city to apply the
techniques of performance measurement to track how well the
transportation department is maintaining streets.
 Engineers have long been able to measure road smoothness on highways,
but city streets have posed a problem for the high-tech sensors: All
the stopping and starting, along with constantly varying speeds,
hindered the equipment, which was designed to be driven by car at a
constant pace. After a long search, researchers from a foundation
called the Fund for the City of New York found that equipment used to
measure smoothness on airplane runways was more adaptable to city
streets and gave it a try. Attached to both sides of a car is a device
known as a profilometer, which uses a laser to measure the distance
from the car to the street. Another device, known as an accelerometer,
controls for the movement of the car. Together, the machines help
create an accurate profile of the road, a sort of bumpiness index that
looks something like an EKG readout.
 While the technology can produce exacting measurements, the
performance data needed a useful human context. People in their cars
feel bumps, not graphs. "We tried to find a way to objectively, fairly
and accurately replicate what the public feels," says Barbara Cohn,
the Fund's vice president. "We rode people over the same roads for
consistency between what they thought was rough and what the machines
showed."
 Cohn says the data from a 700-mile street sample ultimately will be
broken down by each of the city's five boroughs and 59 community
districts. That will allow the transportation department to compare
its paving and maintenance performance around the city, and to see
where its resources are needed most.

----------------------------------------------------------------------

Copyright 1998, Congressional Quarterly, Inc. Reproduction in any form
without the written permission of the publisher is prohibited.
Governing, City & State and Governing.com are registered trademarks of
Congressional Quarterly, Inc.
http://governing.com