CSE176 Introduction to Machine Learning (Fall semester 2015)

Instructor

Miguel Á. Carreira-Perpiñán Professor Electrical Engineering and Computer Science School of Engineering University of California, Merced mcarreira-perpinan-[at]-ucmerced.edu; 209-2284545 Office: 217, Science & Engineering Building 2

Office hours: Mondays/Wednesdays 2:45-3:45pm (SE2-217).

TA: Guoxiang Zhang, gzhang8-[at]-ucmerced.edu. TA hours: Wednesdays 11am-12pm (AOA142).

Lectures: Mondays/Wednesdays 1:30-2:45pm (COB114).

Lab class: Fridays 1:30-4:20pm (Linux Lab, SE1-138).

Course web page: http://faculty.ucmerced.edu/mcarreira-perpinan/teaching/CSE176

Course description

Survey of techniques for the development and analysis of software that learns from experience. Specific topics include: supervised learning (classification, regression); unsupervised learning (clustering, dimensionality reduction); reinforcement learning; computational learning theory. Specific techniques include: Bayesian methods, mixture models, decision trees, instance-based methods, neural networks, kernel machines, ensembles, and others.

Prerequisites: CSE31, MATH24, MATH32. Essentially, you need to know the fundamentals of linear algebra, multivariate calculus and probability, and have good programming skills.

Textbook

Required textbook (get the errata and additional errata):

E. Alpaydin: Introduction to Machine Learning, 3rd ed. MIT Press, 2014.

The companion site for the book has additional materials (lecture slides, errata, etc.).

Other books recommended as additional reading at undergraduate level:

R. O. Duda, P. E. Hart and D. G. Stork: Pattern Classification, 2nd ed. Wiley, 2001.
P. A. Flach: Machine Learning. The Art and Science of Algorithms that Make Sense of Data. Cambridge University Press, 2012. Companion site for the book.
G. James, D. Witten, T. Hastie and R. Tibshirani: An Introduction to Statistical Learning, with Applications in R. Springer, 2013. Companion site for the book.
S. Marsland: Machine Learning: An Algorithmic Perspective, 2nd ed. Chapman and Hall/CRC Press, 2014. Companion site for the book.
T. M. Mitchell: Machine Learning. McGraw-Hill, 1997. Companion site for the book.
S. Russell and P. Norvig: Artificial Intelligence: A Modern Approach, 3rd ed. Prentice-Hall, 2010. Companion site for the book.

If you want to refresh your knowledge of linear algebra, multivariate calculus or probability, the following are helpful (any edition or similar book is fine):

Gilbert Strang: Introduction to Linear Algebra. Wellesley-Cambridge Press.
Murray Spiegel: Schaum's Outline of Vector Analysis. McGraw-Hill.
Murray Spiegel: Schaum's Outline of Theory and Problems of Probability and Statistics. McGraw-Hill.

The textbook has an appendix on probability. Some of the other books have similar appendices on linear algebra, multivariate calculus or probability.

Syllabus and required textbook reading

Syllabus

Before each class, you should have read the corresponding part of the textbook and the notes. I will teach the material in the order below (which is more or less the order in the book).

Textbook reading (table of contents):

Ch. 1 Introduction: all.
Exercises: 1-3, 5, 7.
Ch. 2 Supervised learning: all except sections 2.2-2.3.
Exercises: 1-7, 10-11.
Ch. 3 Bayesian decision theory: all.
Exercises: 1-4, 7-9, 11.
Ch. 4 Parametric methods: all.
Exercises: 1-10.
Ch. 5 Multivariate methods: all.
Exercises: 1-9.
Ch. 6 Dimensionality reduction: sections 6.1-6.3, 6.7-6.8.
Exercises: 1-3, 5, 8.
Ch. 7 Clustering: all except sections 7.5-7.7 and the derivation between eqs. (7.7) and (7.11).
Exercises: 2, 6-11.
Ch. 8 Nonparametric methods: all.
Exercises: 2-4, 8, 10-11.
Ch. 9 Decision trees: sections 9.1-9.4.
Exercises: 1-2.
Ch. 10 Linear discrimination: all except section 10.9.
Exercises: 2-5.
Ch. 11 Multilayer perceptrons: all except sections 11.10, 11.12.
Exercises: 1-5, 7, 12.
Ch. 12 Local models: sections 12.3, 12.5.
Exercises: 1-4.
Ch. 13 Kernel machines: sections 13.1-13.3, 13.5-13.6, 13.9-13.10.
Exercises: 6.
Ch. 14 Graphical models: sections 14.1-3, 14.6-14.7.
Exercises: 1-5.
Ch. 15 Hidden Markov models: sections 15.1-4.
Exercises: 1-3.
Ch. 16 Bayesian estimation: nothing.
Ch. 17 Combining multiple learners: all except section 17.8, 17.10.2 and 17.11.
Exercises: 1.
Ch. 18 Reinforcement learning: nothing.
Ch. 19 Design and analysis of machine learning experiments: sections 19.1, 19.7.
Exercises: 1.

Handouts

The notes to accompany the textbook (bring the corresponding part to each class):
Carreira-Perpiñán, M. Á. (2015): CSE176 Introduction to Machine Learning: Lecture notes. University of California, Merced, 2015.

Labs

Lab 1: a tour of machine learning and visualization.
Lab 2: PCA and LDA.
Lab 3: clustering algorithms.
Lab 4: nonparametric methods.
Lab 5: (stochastic) gradient descent for linear models.
Lab 6: multilayer perceptrons.
Lab 7: RBF networks and polynomials.
Lab 8: support vector machines.
Lab 9: deep learning.
Lab 10: a) discrete Markov processes; b) ensemble learning.

Course grading

Final exam (33%): in-class, closed-book, consisting of problems and conceptual questions. It will cover the entire course.
Lab assignments (66%): these consist of programming selected algorithms in Matlab, exploring their behavior with different datasets and writing a report analyzing the results.

While I encourage you to discuss your work with other students, the homeworks, lab assignments, project and exams must be the result of your own work without collaboration. See the Academic Dishonesty Statement and the UC Merced Academic Honesty Policy.

Grade curves (exams): final.

Matlab tutorials

Matlab tutorials: if you have never used Matlab, there are many online tutorials, for example:

Also see Matlab courses from Engineering Service Learning at UC Merced.