Software Engineering Experimentation
We are entitled to
our own opinions,
but not our own facts.
GMU: SWE 763 (3 credits) ...
Linköping: SAS 435 (6 hp credit) ...
Skövde: (7.5 ECTS credits) ...
Collaboration empowered with Piazza
|Professor: ||Jeff Offutt |
|Email: || offutt ++++ gmu.edu |
|URL: || http://www.cs.gmu.edu/~offutt/ |
|GMU Office: || ENGR 4430, (+1) 703-993-1654 |
|Class Hours: || T 4:30-7:10, East 201 ||
Software Engineering Experimentation
is being taught Spring 2012 as part of a
unique and innovative international collaborative learning opportunity.
Lectures and discussions will be asynchronous
and the class will virtually merge students from three universities,
one in the US and two in Sweden.
- COURSE STRUCTURE:
This course will have three parts:
(1) The professor will lecture on concepts of designing experiments and
the peculiar complexities of experimenting in computing;
(2) Students and professor will read and discuss experimental research papers
from the literature;
(3) Students will carry out a semester-long experimental project,
write a paper, and present the results conference-style.
- ASYNCHRONOUS DELIVERY:
Most of this class will be taught asynchronously.
The first meeting will be face-to-face in classrooms.
The lectures in part 1 will be recorded and deployed online.
The paper discussions in part 2 will be carried out on the online discussion board
The students' papers in part 3 will be posted online and
the final presentations will be in person.
- INTERNATIONAL VIRTUAL CLASSROOM:
We will leverage the online educational opportunities to
virtually merge three groups of students from
George Mason University,
Skövde University, and
All groups will use the same discussion board
see the same lectures,
read and discuss the same papers on the same schedule.
Student presentations will be synchronized with the professor
and broadcast to the other groups via DE technology.
All lectures and discussions will be in English.
- OVERVIEW OF CONTENTS:
This course will take a horizontal look at
a several different research areas in software engineering,
with the view being on empirical validation.
Most research in computing and almost all research in software engineering
must be validated empirically.
This course discusses how to design and carry out experiments in computing,
with a focus on
defining experimental hypotheses,
choosing appropriate dependent and independent variables,
and designing an experiment that will answer the right question without bias.
Lectures and papers.
- Recorded lectures will be posted online.
- A collection of papers.
Most are available on the web;
others will be distributed as the course progresses.
- DISCUSSION BOARD USE:
Software Engineering Experimentation will use the
Piazza bulletin board
for lectures and discussions.
Information for accessing our class will be provided during the first meeting.
Lectures will be posted on
and joining the discussions is mandatory.
- COURSE LEVEL:
This course is
ideal as an introductory course to computing or software engineering research
for PhD students,
appropriate for advanced MS students in
all computing disciplines
(software engineering, computer science, information security, information systems, etc.).
- LEARNING OBJECTIVES:
On completion of the course
students will able to design and carry out experiments
in ways appropriate for a given problem, and will
have skills in analyzing and presenting experimental data.
- Understanding of the scientific process; particularly using the experimental method.
- Knowledge of how empirical studies are carried out
in computing and software engineering.
- Understanding when experimentation is required in software engineering,
and what kinds of problems can be solved using experimentation.
- Knowledge for how to control variables and eliminate bias in experimentation.
- LEARNING PRACTICE:
This course will examine and critique experimental techniques to
evaluate software engineering techniques and processes.
We will read and discuss papers in the current literature,
specifically focusing on the experiments as opposed to
the background or the implications of the results.
Specific topics to be explored include,
but are not limited to,
software quality and testing, human computer interaction, software design,
maintenance, and software development techniques.
The first set of papers are about experimentation in general,
and the rest present experimental results.
Each student must submit a simple summary and evaluation of half of
the first set of papers.
Your summary and evaluation should be about a page long.
Due dates for the summaries are
given on the schedule page
Friday evening the week the paper is scheduled.
Each experimental result paper is assigned to a specific week
on the schedule page.
Two people will be assigned to write and post an evaluation
for each remaining paper.
The summaries should:
(1) describe the problem in general terms,
(2) paraphrase the experimental hypothesis,
(3) summarize and critique the design,
(4) discuss the conduct of the experiment,
(5) explain whether the hypothesis was proved or disproved, and
(6) critique the presentation of the paper.
These summaries will be posted on the bulletin board
on Mondays by close of business
Sweden Fairfax (6:00 PM EST, or midnight in Sweden).
One student will be designated as "dissenter" for each paper
and is required to disagree with the posted summaries and
provide reasonable arguments.
The dissenter must post the counter-arguments on Tuesday by
close of business midnight in Sweden.
All students must join in the discussion for each paper throughout the week.
All postings, including dissents, must be courteous and professional!
Each student will design, conduct, write up and present a small-scale experiment.
A paper will be delivered and the results presented orally to the class.
I will suggest a number of projects or students can develop their own
related to their research topic.
I strongly recommend that GMU
students seek help on writing from the
GMU writing center,
especially if English is not your first language.
are especially helpful.
Numerous papers from past semesters have been published in refereed conferences
and led to PhD dissertations.
- GRADING STANDARDS:
The course has four major components to the grading.
these percentages will be used to compute a score from 1 to 100,
which will then be used to compute a letter grade (A+, A, A-, B+, B, B-, C, F).
Grading policies at Linköping
are still to be determined.
I. (25%) Participation via the bulletin board is required.
All students are expected to participate energetically in the discussions.
Discussion postings must have substantial content.
Postings such as "I agree with Mats" will not earn credit.
Postings such as "Mats thought the number of subjects were too few,
but I believe five is enough for this study because ..."
Students who do not participate in the discussions on a weekly basis
will not receive credit for this portion of the grade.
Shyness and lack of English proficiency are not valid excuses.
II. (20%) Each student will
post summaries and evaluations as described above under PAPERS.
I will assign papers to students.
You can "bid" on papers by sending me an email
and I will try to honor all requests.
Depending on the numbers, you will only have two or three papers,
and it is imperative that you post your summaries on time.
III. (50%) The experimental project will constitute half the grade,
and both the paper and the presentation will be graded.
Each student will write and post a short description of the proposed experiment
early in the semester and the class will make suggestions for improvements.
IV. (5%) A draft of each paper will be reviewed by three people;
the professor and two classmates.
We will assign papers for reviews late in the semester.
- 1 ECTS: Seminar (Participation, part I above)
- 2.5 ECTS: Written (Summaries and reviews, parts II and IV above)
- 4 ECTS: Project (paper and presentation, part III above)
Please note that the seminar and written portions are prerequisites
for presenting the project.
- NECESSARY BACKGROUND:
A strong background in software will be necessary.
Although many of the papers will be quite specific in topic,
extensive background in these topics will not be necessary–we will extract the experimental
techniques and procedures from the papers.
Although experimental data must be analyzed appropriately,
simple analysis usually suffices for this type of research,
so this course is not a statistical methods course.
- CS 700 (for GMU students):
SWE 763 is a perfect complement to CS 700.
CS 700 focuses on data analysis (statistics),
while SWE 763 focuses on how to gather data that will answer a hypothesis.
© Jeff Offutt, 2012, all rights reserved.
This document is made available for use by students in
software engineering experimentation.
Copying, distribution or other use of this document
without express permission of the author is forbidden.
You may create links to pages in this web site,
but may not copy all or part of the text without permission
of the author.