Scientists build to study,
engineers study to build.
Syllabus Schedule Papers Project My home page
Last update
December, 2014
Software Engineering Experimentation
Spring 2015

Note: The papers on this page are from the Spring 2012 offering of this course. The list will be updated before the course starts in January.


The papers on this list are available online. I have created direct links from this web page when practical. GMU students can access papers from off-campus through the university's digital subscriptions by going to the library's database portal.

Publishers digital libraries:
 ACM   IEEE   Wiley   Springer   Elsevier 

Note: Papers with a "local copy" are password protected to avoid copyright violations. The password will be given out in class.


    I. Empirical Methods Overview

  1. Norman Fenton, Shari Lawrence Pfleeger and Robert L. Glass, "Science and Substance: A Challenge to Software Engineers", IEEE Software, 11(4):86-95, July 1994. IEEE local copy
  2. Walter F. Tichy, "Hints for Reviewing Empirical Work in Software Engineering", Empirical Software Engineering, 5(4):309-312, December 2000. EMSE Home local copy
  3. Andreas Zendler, "A Preliminary Software Engineering Theory as Investigated by Published Experiments", Empirical Software Engineering, 6(2):161-180, June 2001. EMSE Home local copy
  4. Warren Harrison, "Editorial: Open Source and Empirical Software Engineering", Empirical Software Engineering, 6(3):193-194, September 2001. EMSE Home local copy
  5. Forrest Shull, Manoel G. Mendoncça, Victor Basili, et al. "Knowledge-Sharing Issues in Experimental Software Engineering", Empirical Software Engineering, 9(1-2):111-137, March 2004. EMSE Home local copy
  6. Per Runeson, and Martin Höst, "Guidelines for conducting and reporting case study research in software engineering", Empirical Software Engineering, 14(2):131-164, April 2009. EMSE Home local copy
  7. Forrest Shull, Jeffrey Carver, Sira Vegas and Natalia Juristo, "The role of replications in Empirical Software Engineering", Empirical Software Engineering, 13(2):211-218, April 2008. EMSE Home local copy
  8. Barbara Kitchenham, "The Role of Replications in Empirical Software Engineering—A Word of Warning", Empirical Software Engineering, 13(2):219-221, April 2008. EMSE Home local copy
  9. Jeff Offutt, "Editorial: Standards for reviewing papers", Journal of Software Testing, Verification and Reliability, 17(3):135-136, September 2007. STVR local copy
  10. Jeff Offutt, "Editorial: Plagiarism Is For Losers", Journal of Software Testing, Verification and Reliability, 25(1), January 2015. local copy

    II. Metrics and Complexity

  11. Burak Turhan, Tim Menzies, Ayse B. Bener and Justin Di Stefano, "On the Relative Value of Cross-Company and Within-Company Data for Defect Prediction", 14(5):540-578, October 2009. EMSE SpringerLink local copy
  12. Denys Poshyvanyk, Andrian Marcus, Rudolf Ferenc and Tibor Gyimóthy, "Using Information Retrieval Based Coupling Measures for Impact Analysis", 14(1):5-32, February 2009. EMSE SpringerLink local copy
  13. Lionel Briand and J. Wuest, "Empirical Studies of Quality Models in Object-Oriented Systems", Advances in Computers, vol. 56, 2002, Academic Press. Briand's homepage
  14. Norman Fenton and Niclas Ohlsson, "Quantitative Analysis of Faults and Failures in a Complex Software System", IEEE Transactions on Software Engineering, 26(8):797-814, August 2000. PDF local copy

    III. Testing

  15. Mark Hennessy and James F. Power, "Analysing the Effectiveness of Rule-Coverage as a Reduction Criterion for Test Suites of Grammar-Based Software", 13(4):343-368, August 2008. EMSE SpringerLink local copy
  16. Yu-Seung Ma, Jeff Offutt and Yong Rae Kwon, "MuJava: An Automated Class Mutation System", Journal of Software Testing, Verification and Reliability, 15(2):97-133, June 2005. PDF local copy
  17. Mats Grindal, Birgitta Lindstrom, Jeff Offutt and Sten F. Andler, "An Evaluation of Combination Strategies for Test Case Selection", Empirical Software Engineering, 11(4):583-611, December 2006. EMSE Home local copy

    IV. Security

  18. Toan Huynh and James Miller, "An Empirical Investigation Into Open Source Web Applications’ Implementation Vulnerabilities", 15(5):556-576, October 2010. EMSE SpringerLink local copy

    V. Reliability

  19. Carina Andersson, "A Replicated Empirical Study of a Selection Method for Software Reliability Growth Models", Empirical Software Engineering, 12(2):161-182, April 2007. EMSE Home local copy

    VI. Maintenance and Evolution

  20. Daryl Posnett, Christian Bird and Prem Dévanbu, "An Empirical Study on the Influence of Pattern Roles on Change-Proneness", 16(3):396-423, June 2011. EMSE SpringerLink local copy
  21. Stephen R. Schach, Bo Jin, Liguo Yu, Gillian Z. Heller and Jeff Offutt, "Determining the Distribution of Maintenance Categories: Survey versus Measurement", Kluwer's Empirical Software Engineering, 8(4):351-365, December 2003. online EMSE Home local copy

    VII. Design and Modeling

  22. Anne Martens, Heiko Koziolek, Lutz Prechelt and Ralf Reussner, "From monolithic to component-based performance evaluation of software architectures: A series of experiments analysing accuracy and effort", 16(5):587-622, October 2011. EMSE SpringerLink local copy
  23. John C. Knight and Nancy G. Leveson, "An Experimental Evaluation of the Assumption of Independence in Multiversion Programming", IEEE Transactions on Software Engineering, SE-12(1):96-109, January 1986. NEC Research Index (CiteSeer) local copy
    A followup paper on the controversy that ensued.
  24. Wojciech James Dzidek, Erik Arisholm and Lionel C. Briand, "A Realistic Empirical Evaluation of the Costs and Benefits of UML in Software Maintenance", IEEE Transactions on Software Engineering, 34(3):407-432, May/June 2008. local copy


  25. Richard J. Miara, Joyce A. Musselman, Juan A. Navarro and Ben Shneiderman, "Program Indentation and Comprehensibility", Communications of the ACM, 26(11):861-867, November 1983. ACM digital library ACM digital library page local copy

    IX. Requirements

  26. Carol K. Gonzales and Gondy Leroy, "Eliciting User Requirements using appreciative Inquiry", 16(6):733-772, December 2011. EMSE SpringerLink local copy

    X. Process and Management

  27. Mats Grindal, Jeff Offutt and Jonas Mellin, "On the Testing Maturity of Software Producing Organizations", Testing: Academia & Industry Conference – Practice And Research Techniques, TAIC PART 2006, pages 171-180, August 2006 Windsor, UK. local version
  28. M. Pikkarainen, J. Haikara, O. Salo, P. Abrahamsson and J. Still, "The Impact of Agile Practices on Communication in Software Development", 13(3):303-337, June 2008. EMSE SpringerLink local copy


© Jeff Offutt, 2014, all rights reserved. This document is made available for use by students in software engineering experimentation. Copying, distribution or other use of this document without express permission of the author is forbidden. You may create links to pages in this web site, but may not copy all or part of the text without permission of the author.