`
Miguel Á. Carreira-Perpiñán
Professor
Electrical Engineering and Computer Science
School of Engineering
University of California, Merced
mcarreira-perpinan-[at]-ucmerced.edu; 209-2284545
Office: 217, Science & Engineering Building 2
`

Office hours: Mondays/Wednesdays 2:45-3:45pm (SE2-217).

TA: Guoxiang Zhang, gzhang8-[at]-ucmerced.edu. TA hours: Wednesdays 11am-12pm (AOA142).

Lectures: Mondays/Wednesdays 1:30-2:45pm (COB114).

Lab class: Fridays 1:30-4:20pm (Linux Lab, SE1-138).

Course web page: `http://faculty.ucmerced.edu/mcarreira-perpinan/teaching/CSE176`

Survey of techniques for the development and analysis of software that learns from experience. Specific topics include: supervised learning (classification, regression); unsupervised learning (clustering, dimensionality reduction); reinforcement learning; computational learning theory. Specific techniques include: Bayesian methods, mixture models, decision trees, instance-based methods, neural networks, kernel machines, ensembles, and others.

Prerequisites: CSE31, MATH24, MATH32. Essentially, you need to know the fundamentals of linear algebra, multivariate calculus and probability, and have good programming skills.

Required textbook (get the errata and additional errata):

- E. Alpaydin:
*Introduction to Machine Learning*, 3rd ed. MIT Press, 2014.

The companion site for the book has additional materials (lecture slides, errata, etc.).

Other books recommended as additional reading at undergraduate level:

- R. O. Duda, P. E. Hart and D. G. Stork:
*Pattern Classification*, 2nd ed. Wiley, 2001. - P. A. Flach:
*Machine Learning. The Art and Science of Algorithms that Make Sense of Data*. Cambridge University Press, 2012. Companion site for the book. - G. James, D. Witten, T. Hastie and R. Tibshirani:
*An Introduction to Statistical Learning, with Applications in R*. Springer, 2013. Companion site for the book. - S. Marsland:
*Machine Learning: An Algorithmic Perspective*, 2nd ed. Chapman and Hall/CRC Press, 2014. Companion site for the book. - T. M. Mitchell:
*Machine Learning*. McGraw-Hill, 1997. Companion site for the book. - S. Russell and P. Norvig:
*Artificial Intelligence: A Modern Approach*, 3rd ed. Prentice-Hall, 2010. Companion site for the book.

If you want to refresh your knowledge of linear algebra, multivariate calculus or probability, the following are helpful (any edition or similar book is fine):

- Gilbert Strang:
*Introduction to Linear Algebra*. Wellesley-Cambridge Press. - Murray Spiegel:
*Schaum's Outline of Vector Analysis*. McGraw-Hill. - Murray Spiegel:
*Schaum's Outline of Theory and Problems of Probability and Statistics*. McGraw-Hill.

The textbook has an appendix on probability. Some of the other books have similar appendices on linear algebra, multivariate calculus or probability.

Before each class, you should have read the corresponding part of the textbook and the notes. I will teach the material in the order below (which is more or less the order in the book).

Textbook reading (table of contents):

- Ch. 1
*Introduction*: all.

Exercises: 1-3, 5, 7. - Ch. 2
*Supervised learning*: all except sections 2.2-2.3.

Exercises: 1-7, 10-11. - Ch. 3
*Bayesian decision theory*: all.

Exercises: 1-4, 7-9, 11. - Ch. 4
*Parametric methods*: all.

Exercises: 1-10. - Ch. 5
*Multivariate methods*: all.

Exercises: 1-9. - Ch. 6
*Dimensionality reduction*: sections 6.1-6.3, 6.7-6.8.

Exercises: 1-3, 5, 8. - Ch. 7
*Clustering*: all except sections 7.5-7.7 and the derivation between eqs. (7.7) and (7.11).

Exercises: 2, 6-11. - Ch. 8
*Nonparametric methods*: all.

Exercises: 2-4, 8, 10-11. - Ch. 9
*Decision trees*: sections 9.1-9.4.

Exercises: 1-2. - Ch. 10
*Linear discrimination*: all except section 10.9.

Exercises: 2-5. - Ch. 11
*Multilayer perceptrons*: all except sections 11.10, 11.12.

Exercises: 1-5, 7, 12. - Ch. 12
*Local models*: sections 12.3, 12.5.

Exercises: 1-4. - Ch. 13
*Kernel machines*: sections 13.1-13.3, 13.5-13.6, 13.9-13.10.

Exercises: 6. - Ch. 14
*Graphical models*: sections 14.1-3, 14.6-14.7.

Exercises: 1-5. - Ch. 15
*Hidden Markov models*: sections 15.1-4.

Exercises: 1-3. - Ch. 16
*Bayesian estimation*: nothing. - Ch. 17
*Combining multiple learners*: all except section 17.8, 17.10.2 and 17.11.

Exercises: 1. - Ch. 18
*Reinforcement learning*: nothing.

- Ch. 19
*Design and analysis of machine learning experiments*: sections 19.1, 19.7.

Exercises: 1.

The notes to accompany the textbook (bring the corresponding part to each class):

Carreira-Perpiñán, M. Á. (2015): CSE176 Introduction to Machine Learning: Lecture notes. University of California, Merced, 2015.

- Lab 1: a tour of machine learning and visualization.
- Lab 2: PCA and LDA.
- Lab 3: clustering algorithms.
- Lab 4: nonparametric methods.
- Lab 5: (stochastic) gradient descent for linear models.
- Lab 6: multilayer perceptrons.
- Lab 7: RBF networks and polynomials.
- Lab 8: support vector machines.
- Lab 9: deep learning.
- Lab 10: a) discrete Markov processes; b) ensemble learning.

- Final exam (33%): in-class, closed-book, consisting of problems and conceptual questions. It will cover the entire course.
- Lab assignments (66%): these consist of programming selected algorithms in Matlab, exploring their behavior with different datasets and writing a report analyzing the results.

While I encourage you to discuss your work with other students, the homeworks, lab assignments, project and exams must be the result of your own work without collaboration. See the Academic Dishonesty Statement and the UC Merced Academic Honesty Policy.

Grade curves (exams): final.

Matlab tutorials: if you have never used Matlab, there are many online tutorials, for example:

Also see Matlab courses from Engineering Service Learning at UC Merced.

- Machine learning in the news:
- Science: Artificial Intelligence special issue, Jul. 17, 2015.
- Nature: Machine Intelligence supplement, May 28, 2015.
- SIAM News: Data Science, Jul. 15, 2015.
- New York Times: Big Data, Deep Learning, Machine Learning.

- Which algorithms are more popular?
- Top 10 algorithms in data mining (2008 paper in the journal
*Knowledge and Information Systems*). - Most popular algorithms for analytics/data mining (KDnuggets Poll 2011).

- Top 10 algorithms in data mining (2008 paper in the journal
- Eigenfaces and Fisherfaces.
- The MNIST dataset of handwritten digits, including the classification error achieved by different classifiers.
- Generating pseudorandom numbers:
- With a table: A Million Random Digits with 100,000 Normal Deviates.
- ...or, more easily, with Matlab.

- Deep learning software:

Miguel A. Carreira-Perpinan Last modified: Wed Jun 22 20:45:09 PDT 2016