Performance Auditing: Beyond 'Gotcha!'

It's been about five years since Washington State Auditor Brian Sonntag announced the launch of his ambitious effort to bring a new style of performance auditing to the state. The change came on the heels of an unprecedented voter ballot initiative -- I 900 -- that created a dedicated stream of cash solely for the purpose of doing state and local government performance audits.

As envisioned by Sonntag, these wouldn't be the kind of audits where government goes in looking for (if not outright assuming) trouble. Rather, they would be based on both government and citizen input that would steer the audits toward areas of high citizen interest and performance improvement in areas ranging from education to transportation. Government entities could even invite the auditor in to help look at ways to bolster efficiency and effectiveness.

The Washington State initiative sent ripples through the nation's public-sector auditing community, and for a very simple reason: Performance auditing is an evolving and somewhat controversial practice. There are plenty of those in the profession who still believe that an auditor's job should solely be a relentless focus on tracking dollars in and dollars out. That same faction believes just as strongly that auditors have no business wandering into program or policy evaluation or making judgments about whether government is actually operating efficiently and effectively.

Another reason that performance auditing in Washington State generated an unusual amount of interest is because the ballot initiative that created the performance-audit bonanza was co-sponsored by Tim Eyman, a government-spending gadfly who has made his name successfully supporting ballot initiatives aimed at shrinking government revenues. That he would back a measure aimed at investigating ways to help government run better was encouraging, even to some of his harshest critics.

"It's the only Eyman initiative that I ever supported," state Representative Mark Miloscia, the Washington legislature's most prominent and outspoken performance measurement and management guru, said during an interview several years ago.

The overarching questions -- not just in Washington State, but for those watching from afar -- have been whether Sonntag would be able to convince skeptics that his performance audits added value, and whether those audits could be done in a way that mixed necessary scrutiny with less of an adversarial flavor. Miloscia, for one, says that his initial enthusiasm has been tempered somewhat by concerns about just those two issues.

Indeed, five years out, there are still no definitive answers to those questions. Sonntag is credited with using his newfound auditing arsenal to do some very good work that has clearly improved the way some public entities function. His audit of the Port of Seattle led to changes in the port's management of overall operations, especially in how it accounts for assets and manages construction projects.

On the other hand, many local officials complain that they have had to endure audits of very dubious value. For example, having been invited by the city of Yakima to do a top-to-bottom evaluation of the city's operations, auditors came back with a suggestion that the primary focus should be on fleet management. "We had set up a task force and worked on this for a year," says city councilwoman Kathy Coffey. "And what came out after all the meetings and all the talking was that fleet management was the only area they were going to look at."

As for the promise of a more cooperative, collaborative approach to audits -- an approach that many at the national level argue is very important to broader acceptance of the practice -- that seems not to have fully materialized either. "It was all very Kumbaya at the beginning," says one official, who asked not to be named. "But that's not how it's happening."

One high-level audit conducted by Sonntag's shop, for example, was aimed at assessing the responsiveness of local governments to requests under the state's open-records law. "It was a total sting operation," one official says, "that just set everyone on edge." A better approach, this same official suggests, would simply have been to work with cities and counties on how to improve their response systems. Or, better yet, rather than spending money trying to catch governments failing at open-records compliance, the auditors might have spent more of their time actually trying to help local government make improvements in key functional areas, such as public works and public safety.

Sonntag says he's well aware of the criticism, but stands by both the methods and targets of his effort, and notes that significant changes have occurred in the wake of audits in areas ranging from open government to procurement. Nobody particularly likes it when an auditor comes knocking, says Sonntag, "so there's always going to be some sour grapes and whining."

One official who doesn't think that he's merely whining is Dave Curry, head of finance for Educational Service District 105 in Yakima. "I am a performance-audit advocate if they are properly prepared and if they stay on scope," Curry says. His concept of a performance audit is rigorous and straightforward: "First, you go in and see if this agency is still accomplishing that vital service that we thought was so important back in 1850. Next, you see if it's performing that service efficiently and effectively."

What Curry claims the auditor discovered in Yakima was a very high-functioning organization doing exactly what it was supposed to be doing. Not satisfied to pat the city on the back and clear out of town, he says, "they started looking for stuff." Curry's overall impression: "I was pretty disappointed in this process. It took a couple months and hundreds of hours of work on our part."

The findings from Yakima do suggest that the auditor found little wrong with the school district's operations, and had to dig a little to come up with any significant signs of trouble at all. The handful of audit recommendations that came out included the suggestion that the district consider outsourcing courier services and develop an "inquiry-based science program."

But these days, Sonntag has higher-level critics to worry about. The most prominent is Representative Miloscia, the early supporter of performance audits. He now thinks the whole effort -- while still full of potential -- has fallen short. Not necessarily because of anything particular to I 900 itself, but rather because of the way Sonntag has carried out the mission.

"Generally, his audits tend to be more 'gotcha' than focused on process improvements," says Miloscia, who recently was named chairman of the House Auditing and Oversight Committee. "So we're trying to get him to change from the old-style audits, which are really focused more on legal compliance kinds of things, to more of a teaching instrument. What we want to see is these audits help with system learning and process improvements, and he's not doing that yet."

Sonntag is blunt in his reply to Miloscia's criticism. The language in I 900 is clear, he says: The auditor is supposed to focus first on cost savings by looking at services that could be reduced, eliminated or privatized; whether statutory or regulatory changes are needed in order to improve performance; how well an entity tracks performance; and finally, its capacity to identify best practices. "That's the law we have to follow," Sonntag says, "and it overrides Mark's desire for his management agenda."

In response to those who criticize Sonntag's strategy and tactics, Mike Bailey, the auditor for Redmond, Washington, notes that I 900 was passed, in part, with the help of people who want the state auditor to ferret out incompetence, waste, fraud and abuse. "So Brian has to deal with that element of his support." Meanwhile, Bailey contends, some of the toughest audits have yielded the best results. "The audit of the Port of Seattle is a good example, where some good things came from that audit and there was even a change in leadership."

While many critics of the current auditing style agree that evaluations such as the one at the Port of Seattle were worthwhile, they still speculate that the spirit of Tim Eyman is a bit too prominent in the whole process. They continue to insist that the exercise focuses more on catching governments in the act than on proactively working to improve key government operations.

Another critique of the audits is the that most of them have been farmed out to private-sector auditors, companies that may or may not have a good grasp of how the public sector operates. While Lloyd Tyler, chief financial officer for the city of Vancouver, Washington, says his city's experience with I 900 audits went O.K., he adds that during an audit of the collection and use of "impact fees" (fees charged to developers to offset potential negative impacts of large-scale development), his office had to explain to the contracted auditor all the ways in which Washington's approach to impact fees differed from that of other states. "There was a little bit of a learning curve there," Tyler recalls.

"It's true, we are bringing in folks new to state and local government operations," Sonntag replies, "but that's part of the audit process, too, to learn how a particular entity operates."

The bottom line is that pursuing high-profile, tough investigative audits while engaging in more collaborative, good-government focused performance reviews is a balancing act that's simply difficult to manage at times.

One person in the state, however, is determined that this balancing act will get some scrutiny. Representative Miloscia's committee already has scheduled four hearings into the I 900 audits -- the way they're conducted and what they're really accomplishing. The stakes are high, since the legislature has the authority to order the auditor where and how to spend I 900 money. The legislature has exercised this power in the past and may be more willing to do so in the future.

Note: This online version has been corrected to reflect that Tim Eyman is not a radio talk-show host and that he was a co-sponsor of the performance-auditing ballot measure.

These articles are part of a continuing series on public performance measurement focusing on citizen involvement. Support has been provided by the Alfred P. Sloan Foundation. Although the foundation may assist some of the programs described in these articles, it had no control or influence over the editorial content, and no one at Sloan read the material prior to publication. All reporting and editing was done independently by Governing staff.