Mailbag: Survey numbers require context

The residents of Newport Beach are far happier than residents of other cities, and not just because of Newport’s natural attractions. They are more satisfied with crime-prevention; the quality of city services; safety when walking in the neighborhood, by day or night; city communications with the public and service from city employees; walking and biking trails; the quality, safety and maintenance of city parks; the promptness of police response to emergencies; the management of traffic flow; enforcement of traffic laws; and planning for growth.

These are the findings of the second Newport Beach Residents Survey (the first was in 2008). You will not learn them anywhere else, because neither the council, city manager or, disappointingly, even the survey company, has been able to get its head around the numbers. Here’s why. If I told you 67% of the Newport’s residents are satisfied with “management of traffic flows” what would that tell you? Nothing in itself. But when I add that for other Californian cities in 2010 it is 52%, or 15 points lower, you will get the point. It takes a comparison to make sense of the numbers, and neither the city nor the survey company has taken that step, even though other cities were surveyed and the data are available.

The survey company collected and tabulated the data for multiple cities, a cookie-cutter process utilizing a standard questionnaire and computerized tabulation, then dumped many pages of raw numbers and colored charts with no reasoned conclusions on the city, where they lie inert and unexplained on the city’s website. The survey company’s dazzling slideshow presentation to the council, which I attended, was nothing more than a commentary on the raw data.

There is a huge leap from having data to learning from it. To quote from my Forum article of May 8, 2008 on the first survey, “If someone tells you the home team scored 20 in last night’s game you’ll wait for the rest of the sentence, because whether 20 is good news or bad news depends of course on what the opposition scored.”

The surveys will have cost $44,000 and counting. That’s not a lot to pay for useful research, but it’s a lot of money to waste, and wasted it will be if not interpreted correctly. Residents are owed a full interpretation because they paid for the survey and it’s about them. I have shared these concerns with council members and City Manager Dave Kiff. The mayor and council members are not required to have specialist knowledge but city staff should know the basics of opinion research and the difference between data and information, or know where to obtain it, when purchasing survey services. This is something to bear in mind when it comes to big bucks, such as a new civic center.

Since the 2010 survey doesn’t indicate any areas requiring improvement other than, mildly, the city’s web page, there seems no need to hide information. But maybe the absence of intellectual rigor provides cover for anything the city wants to do under the guise of reaching out.

Tom Moulson

Corona del Mar