Professor Harry Wechsler

Department of Computer Science

George Mason University

Fairfax, VA 22030

e-mail : wechsler@cs.gmu.edu

web : http://cs.gmu.edu/~wechsler/

(703) 993-1533 (office)

(703) 993-1530 (sec)

(703)993-1710 (fax)

GEORGE MASON UNIVERSITY

SPRING '2003

 IT 844 --- PATTERN RECOGNITION

Class Information

001 36056 R 4:30 p.m. – 7:10 p.m. ST2 430A

Office Hours

R 3:00 p.m. - 4:00 p.m. or by appointment (SITE II - Rm. 461)

Textbook

1. C. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.

References

1. V. Cherkassky and F. Mulier, Learning from Data : Concepts, Theory, and Methods, Wiley, 1999.

2. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines, Cambridge University Press, 2001.

3. R. Duda, P. Hart and D. Stork, Pattern Classification, Wiley, 2002.

4. S. Haykin, Neural Networks, Prentice-Hall, 1999.

5. B. Scholkopf and A. J. Smola, Learning with Kernels : Support Vector Machines, Regularization,

Optimization, and Beyond, MIT Press, 2002.

6. V. Vapnik, Statistical Learning Theory, Wiley, 1998.

Course Description

The course covers the Bayesian and Statistical Pattern Recognition (SPR),

the Neural Network (NN), and the Statistical Learning Theory (SLT) approaches

for Pattern Recognition (PR) Topics include Bayes’ theorem,

density approximation, multilayer networks and BackPropagation (BP) learning,

pre-processing and feature extraction, data and dimensionality reduction, functional approximation

and adaptive kernel methods, clustering and self-organization, decision trees, committee machines,

structural risk minimization (SRM), model selection, support vector machines (SVM),

support vector regression (SVR) and support vector clustering (SVC),

evolutionary computation and genetic algorithms, and fuzzy systems.

Experimental design, applications and performance evaluation are

emphasized throughout the course.

Grading

1.Homework: 15 %

2.Class Presentation: 15 %

3.MIDTERM EXAM -- March 20 : 20%

4.PROJECT: (4.1) Literature Survey and (4.2) Experimental Results à 50 %

(item #2 and #4.1 can be related)

Term Project
Topic and Scope for the project to be agreed with the instructor before Spring break.

Tentative Schedule

January 23	Introduction
January 30	Bayes’ theorem, decision boundaries, risk minimization, discriminant analysis and its relation to connectionist5 learning, naïve Bayes.
February 6	Density approximation : Maximum-Likelihood (ML) and maximum-entropy estimates.
February 13	Mixture models : Expectation-Maximization (EM), Radial Basis Functions (RBFs), Probabilistic Neural Networks (PNN), Non-Parametric Estimation (histograms, kernel-based methods and Parzen windows, k-nearest neighbors <k-nn>), and Performance Evaluation.
February 20	Fuzzy systems; Learning and Estimation (LS estimation and regression, Mean-Square Error <MSE> and the orthogonality principle, stochastic approximation and Robbins-Monroe algorithm, Least-Mean Squares <LMS> / Widrow-Hoff / delta-rule algorithm and Adaline, relationships between {MSE, RBF, and LMS}, Associative Memories uisng Hebbian learning and the Pseudo-Inverse Approach, regression is the optimal MSE estimate for neural networks learning, MSE estimates a-posteriori class probabilities, and bias-variance trade-offs}
February 27	Brain Modelling and Connectionism, Functional Approximation, Linear Learning Machines and the Perceptron, MultiLayer Networks and Backpropagation Learning, Dual Forms Learning and Support Vectors.
March 6	Data & Dimensionality reduction and Feature Extraction : PCA (Principal Component Analysis), LDA (Linear Discriminant Analysis), ICA (Independent Component Analysis), and LFA (Linear Feature Analysis).
March 13 (make-up class held during Spring break)	Evolutionary computation, Genetic Algorithms (GAs), Co-Evolution, Biometrics and Face Recognition. REVIEW for MIDTERM
March 20	MIDTERM EXAM – closed books ! bring examination books !
March 27	Clustering, ISODATA and k-means clustering (vs. EM), Self-Organization Maps (SOM) and Learning Vector Quantization (LVQ).
April 3	Decision Trees (DT), committee machines and mixtures of experts, Active Learning.
April 10 - 17	Regularization, generalization, VC-dimension, Statistical Learning Theory (SLT), structural risk minimization (SRM) and model selection, kernel-induced feature space (Mercer theorem), optimization theory, Lagrangian theory, Karush-Kuhn-Tucker (KKT) Theorem, duality, and kernel methods.
April 24	Support Vector Machines (SVM), support vector regression (SVR) and support vector clustering (SVC) (1-class classification).
May 1 - 8	FINAL PROJECT PRESENTATION