How to build a zombie detector: Identifying software quality problems
How do we measure and track the quality of the open source software we rely on? When a project is in trouble, how do we know? What have past approaches to this question gotten right, where have they gone astray, and what’s still a mystery?
Predicting and measuring software quality is a subject of substantial academic research but not always discussed with the people who actually make software. To help close that gap, we sifted hundreds of research articles that proposed, assessed, and sometimes verified different metrics of software quality. In this talk, I’ll summarize the current state of the software quality field, and explore the map of known approaches: from by-hand inspections through repository mining and machine learning. What can these approaches tell us? I’ll share both the best, most-validated metrics we found and the weakest, least-proven ones.
Finally, I’ll share a proposed measurement strategy, drawn from work in multiple disciplines – informed by practical experience as well as computer science, information systems, and social science. I’ll share our preliminary results from a project to measure the relative health of thousands of open source projects, and what that might mean for the future of our shared digital infrastructure.
After hearing these approaches to measuring quality and some of the results of using them, I hope you’ll offer your feedback: what zombies have you seen, slain, or run away from? Do the academic approaches I’ve described make sense here among the living – in the everyday experiences of development and systems administration?
Presenters
Kaylea Champion, University of Washington
Kaylea Champion is a graduate student in Communication at the University of Washington with a background in systems administration and support. Her research seeks to understand how people collaborate to build amazing things, like GNU/Linux and Wikipedia. In her free time, she enjoys tromping through the hills, cooking for a crowd, and smashing goblins.