A little over a year ago we wrote about the challenges of developing a fair and reliable placarding system for restaurants (read it now!). You see, not all placarding systems work the same way. We did some research to find out what other jurisdictions do, and we identified some areas for improvement as we develop our own methodology:

• Because good food safety needs to be practiced every day, we want our placards to communicate how well a restaurant has performed over a range of time, not just a single inspection. Imagine assessing a student’s academic abilities based on a single quiz. A grade in a class, and a college GPA, usually reflects multiple assessments rather than just one.
• We also want to communicate a scale of performance. Restaurant customers want to know more than if a restaurant passes or fails – they want to know the degree to which they pass or fail and how often that happens. It’s the difference between passing a class and performing at the top of the class.

By addressing these goals, we will be able to help consumers predict how well a restaurant is practicing food safety on any given day.

These goals make sense, but this approach is the first of its kind, and it has required (and will require) a lot of work to place it on a firm evidence base. Here’s what we’ve done, what we’re doing next, and what you can do to stay informed while we build a great system.

Work we’ve done so far.

We cleaned up our legacy database of restaurant inspection reports. Painstaking work.

We figured out how many inspections were required to accurately predict how well a restaurant will perform on any given day. This required pretty comprehensive statistical analyses, and luckily, we got to work closely with Stanford Professor Dan Ho, who is a preeminent scholar of government data disclosure and administrative law. (The likely answer is around 4-5 routine inspections.)

We figured out what kind of information best predicts how well a restaurant will perform on any given day. As part of that statistical analysis, we analyzed whether the “grade” should include red and blue points, or just red (critical) points. Remember, our goal is to provide information that helps diners predict the food safety practices of the restaurant they’re about to patronize. (The answer is: a grade should be based on red (critical) violations.)

We worked with Prof. Ho to train our inspectors to be as consistent as possible. Our 55 inspectors, who conduct some 30,000 inspections every year, are some of the most highly trained and experienced food safety experts in the business today. They have environmental health science backgrounds, have to pass a very rigorous national exam, and many have on-the-ground retail food operation experience. The question of consistency is: if Inspector A conducts an inspection resulting in 25 points, how likely will it be that Inspector B would also give the same 25 points?

The reality is, there is always some degree of variation across how an inspector inspects. This is not unique to food inspections – in fact it is widely documented across all kinds of areas, from the regulatory space (like inspections) to the classroom space (like with teachers).  With that in mind, we work toward increased consistency.

So, to get there, we occasionally ask inspectors to perform inspections side by side. Through this peer-review process, our inspectors are able to learn how they review restaurants compared to each other. When they identify discrepancies in practice, they work together to reach consensus on protocol, and the accuracy and consistency of inspections improve substantially.

Work we’re doing now and over the course of the year.

We’re improving our website. Some people believe (and we have to agree) that our website could be more user-friendly. We’re working to overhaul the entire site this year, and part of that includes making the restaurant inspection portion more intuitive. Through a very helpful stakeholder engagement process, we identified a list of recommendations for how to make it better. We are currently working with our IT department to determine  timeline for implementation. It will be an iterative process – when we are able to make an improvement, we’ll do it.

We’re working on the design of the restaurant placards. It is very important that our placards convey the right message and that the message be understood by any restaurant patron. Given the diverse population of King County, designing a meaningful and easy-to-use placard is no short order. We are working with a talented and experienced graphic designer who will help us arrive at a solution that works for everyone, regardless of language or culture.

We’re getting feedback on the design of the restaurant placards. This is part B of our placard design process, and it is absolutely critical. Once we have produced designs, we will be holding focus groups with stakeholders and community members to make sure what we post in restaurant windows works. We hope to pilot our placards before the end of this year.

Ways you can stay informed until we finish.

We made all of our inspection data available online. This means that the data can be casually perused, but it also means that third-party developers and companies can use the data to create apps and other tools. For instance, Yelp could feature inspection histories on its website, or someone could create an app that shows restaurant histories in a given area. We can’t endorse these sorts of projects, but we are happy to make the data available for anyone to use. Check it out here.

We made our restaurant inspection history searchable by restaurant name and address. On our website, anyone can search for a restaurant or an address and find its full inspection history. Wondering about that restaurant down the street? The answers are at your fingertips. Search now.

We know that you want more, and we are anxious to give it to you. But, we don’t want to give you a shoddy product. As we promised you a year ago, we’re not going to slap a grade on a restaurant’s window without making sure it’s based on rigorous evidence. Right now, we hope these extensive online tools can empower you as a consumer, and we look forward to piloting our new placarding system with you soon.