CS672 - Computer System Performance Evaluation - Spring 2017

George Mason University
DEPARTMENT OF COMPUTER SCIENCE
Tuesdays 4:30pm to 7:10pm Innovation Hall 134
Professor Daniel A. Menasce'
Course office hours Tuesdays 3:00pm to 4:00 PM or by appt; e-mail at all times
Phone: 703-993-1537
E-mail: menasce at gmu dot edu (please prefix the subject of your message with CS672)



CS 672 Home Page
 

DESCRIPTION :

CS672 is a 3-credit course with CS571 as prerequisite. It introduces the main concepts and techniques needed to plan the capacity of computer systems, predict their future performance under different configurations, and design new applications that meet performance requirements. The course is mainly based on the use of analytic queuing network models of computer systems. These techniques are applied to study the performance of centralized, distributed, parallel, client/server systems, Web server and e-commerce site performance. The course also discusses performance measuring tools for operating systems such as Unix and Windows. The course provides the students with hands-on experience in performance evaluation through a project. The concept and applications of software performance engineering are also covered.

Back to the top



 
 

GRADING POLICY:

Grades are based on a midterm, a take-home final, and one project. Grades will be numerical on the scale 0-100. Your final numerical grade, G, is computed as:
G = 0.35* Midterm + 0.30 * Final + 0.35 * Project.

The following table is used to convert your numerical grade G to a letter grade:

letter grade
[97,100]  A+
[93,97)  A
[88,93)  A-
[85,88)  B+
[81,85)  B
[77,81)  B-
[65, 77)  C
< 65  F

There is no curving. No extra credit assignments will be given after the semester is over to increase grades. The instructor may decide to give an extra-credit assignment during the semester. Everyone will be given the opportunity to do the extra-credit assignment. Several homeworks will be assigned during the semester but will not considered for the final grade. However, students who turn in all homeworks in time and get most of them correct may receive an upgrade in their letter grade if their numerical grade G is very close to the boundary between two letter grades.

Back to the top


SYLLABUS:
Week 1:  Computer System Lifecycle and From Systems to Descriptive Models
Week 2: Quantifying Performance Models
Weeks 3-4: Performance Engineering Methodology and Case Study I: A Database Service
Week 5: Multiclass Open Queuing Network Models
Weeks 6-7: Case Study II: A Web Server and Multiclass Closed Queuing Network Models
Week 8: Midterm
Week 9: Case Study III: A Data Center
Week 10: Markov Models
Week 11: Case Study IV: An E-Business Service and Case Study V: A Help-Desk Service
Week 12: Queuing Networks with Load Dependent Devices.
Week 13: Non-Product Form Queuing Network Models
Back "to the top


READINGS:

Required readings:

  1. D. A. Menascé, V. Almeida, and Larry W. Dowdy, Performance by Design: Computer Capacity Planning by Example, Prentice Hall, 2004.
  2. Slides by D. A. Menascé.
Other recommended books:
  1. D. A. Menascé and V. Almeida, Capacity Planning for Web Services: Metrics, Models, and Methods, Prentice Hall, 2002.
  2. Performance Modeling and Design of Computer Systems: Queueing Theory in Action, Mor Harchol-Balter, 2013, Cambridge University Press.
  3. Introduction to Computer System Performance Evaluation, K. Kant, McGraw Hill, 1992.
  4. Probability, Stochastic Processes, and Queuing Theory, Randolph Nelson, Springer Verlag, 1995.
  5. Quantitative System Performance, E. Lazowska, J. Zahorjan, G. Graham, and K. Sevcik, Prentice Hall, 1984.
  6. The Benchmark Handbook for Database and Transaction Processing Systems, ed. Jim Gray, Morgan Kauffman, 1991.
  7. Optimizing Oracle Performance: A Practitioner's Guide to Optimizing Response Time, Cary Milsap, 2003, O'Reilly.
  8. Configuration and Capacity Planning for Solaris Servers, Brian Wong, Prentice Hall, 1997.
  9. Windows 2000 Performance Guide, M. Friedman and O. Pentakalos, O'Reilly, 2002.
  10. Sun Performance and Tuning, second edition, Adrian Cockcroft and Richard Pettit,Prentice Hall, 1998.
  11. Simulation Modeling and Analysis, A. M. Law and W. D. Kelton, 3rd edition, McGraw Hill,200.
  12. Simulation with Arena, W. David Kelton, Randall P. Sadowski, Deborah A. Sadowski, McGraw Hill, 1998.
Other sources of relevant material:
  1. Proceedings of the ACM Sigmetrics Conference.
  2. Proceedings of the Computer Measurement Group (CMG) Conference Proceedings.
Back to the top


RESOURCES

  • Transaction Processing Performance Council: organization that defines and publishes transaction processing and database benchmarks known as the TPC benchmarks.
  • The Standard Performance Evaluation Corporation: organization that establishes, maintains, and endorses a standardized set of benchmark metrics for performance evaluation known as SPEC ratings. They are also responsible for the SPECWeb benchmark for Web servers.
  • WebStone Benchmark: a benchmark, orginally developed by SGI, used to assess Web servers.
  • Keynote:a service that measures end-user performance for E-commerce sites.
  • The WEBBIB Project: an online bibliography of distributed systems topics related to the WWW.
  • ACM Digital Library: look for the Proceedings of the ACM Sigmetrics conferences, as well as papers in other ACM conferences and journals.
  • IEEE/IEE Eletronic Library: look for papers in the IEEE Tr. Computers, IEEE Tr. Software Engineering, IEEE Transactions on Parallel and Distributed Systems, Conference Proceedings, as well as in various IEEE magazines such as IEEE Internet Computing.
  • Performance Evaluation Journal: Go to the Science Direct (Elsevier) link on this page and select the Performance Evaluation journal.

  • Back to the top
     
     


    PROJECT:

    Students will work individuallyn a project and will make a presentation (a final exhibition) to the whole class. Each student has to turn in a written report describing the performance problem being studied, the model built, the data collection procedures used, the various numerical results, and conclusions obtained. In order for projects to complete successfully, it is very important to plan all activities and to follow a schedule. Starting to work on a project a few days before it is due will very likely cause problems to its successful completion.

    Back to the top


    IMPORTANT DATES :
    First Day of Classes  January 24, 2017. 
    Project Proposal Due March 7, 2017.
    Spring break - No classes March 14, 2017.
    Midterm  March 21, 2017. 
    Final exam available  May 2, 2017.
    Last day of classes May 2, 2017.
    Project presentations  May 2, 2017.
    Final exam due date:  May 10, 2017.
    Back to the top


    IMPORTANT RULES :

    No collaboration is allowed among students in any of the individual exams. Students are allowed to discuss with other students the solution of homework assignments.

    Back to the top


    Last updated: December 17, 2016.