CS 472/572         Machine Learning

Course Description

Machine learning is about building predictive or descriptive models automatically from data. There are two central challenges in machine learning. The first is generalizing from data to future situations. A model that performs very well on past data may nonetheless perform very poorly on unseen examples, a phenomenon known as "overfitting". The second challenge is building models efficiently, especially in cases where the dataset is very large and the patterns are complex. We will cover standard methodologies, models, and algorithms for machine learning. Specific topics include decision trees, instance-based learning, linear classifiers, probabilistic classifiers, support vector machines, deep learning, and learning theory.

Instructor

Thien Huu Nguyen, thien@cs.uoregon.edu

Lectures

Prerequisites

• CIS 315 - Intermediate Algorithms

Textbooks and Readings

• CIML:

  Hal Daume III, A Course in Machine Learning 0.99, 2015 (Online!)

• Mitchell:

  Tom Mitchell, Machine Learning, 1997.

• Murphy:

  Kevin Murphy, Machine Learning: A Probabilistic Perspective, 2012.

• ESL:

  Hastie, Tibshirani, and Friedman, The Elements of Statistical Learning, 2009 (Online!)

• DL:

  Ian Goodfellow and Yoshua Bengio and Aaron Courville, Deep Learning, 2016 (Online!)

• Domingos:

  Pedro Domingos’ video lectures on YouTube

Major Topics

• Machine learning introduction, decision trees: 2 lectures
• Inductive learning and nearest neighbor: 2 lectures
• Perceptron, linear regression, logistic regression: 2 lectures
• Kernel methods, SVMs: 2 lectures
• Neural Networks: 2 lectures
• Deep Learning : 4 lectures
• Convolutional Neural Networks, Recurrent Neural Networks, Transformers: 4 lectures

Expected Learning Outcomes

This course covers standard methodologies, models, and algorithms for machine learning. Specific topics include decision trees, instance-based learning, linear classifiers, probabilistic classifiers, support vector machines, model ensembles, and learning theory. Especially, we will spend a large amount of time for deep learning, the recent approach to machine learning that has achieved very high performance for tasks in different application domains (e.g., computer vision, natural language processing).

Upon successful completion of the course, students will be able to:

• demonstrate facility with the cycle of machine learning model development, including training, developing, and testing;
• demonstrate facility with the fundamental concepts and problems in machine learning, including (hyper-)parameters, loss functions, overfitting, underfitting, cross-validation, regularization, evaluation measures;
• demonstrate facility with different types of learning, i.e., supervised learning, semi-supervised learning, and unsupervised learning;
• demonstrate facility with traditional machine learning methods, i.e., decision trees, nearest neighbor, perceptron, linear and logistic regression, and support vector machines;
• demonstrate facility with the basic concepts in deep learning, i.e., feed-forward networks, activation functions, computational graphs, back-propagation, and (stochastic) gradient decent;
• demonstrate facility with advanced deep learning models, including convolutional neural networks, recurrent neural networks, residual models, transformers, and their applications to different types of data;

Acquired Skills

Upon successful completion of the course, students will have acquired the following skills:

• be able to identify machine learning problems and design appropriate features and models to solve those problems;
• be able to leverage available machine learning frameworks (pytorch, scikit-learn) to implement models and apply them for data of interested domains;
• be able to analyze the operation/outputs of basic machine learning models for debugging and performance improvement purposes.

Tentative Schedule

Assignments

• Assignment 1 (written): Link (posted on April 8), due date: April 17 at 11:59pm.

• Assignment 2 (programming): Link (posted on April 18), due date: May 3 at 11:59pm.

• Assignment 3 (programming): Link (posted on May 7), due date: May 20 at 11:59pm.

Final Projects

• Final projects involve implementing and evaluating models for some ML problems. You will need to submit a proposal and a report for your project.
  We will also collect your code. Working on the project early is highly recommended. Good projects can lead to paper submission at conferences afterward.
  
  More detailed instructions for the final project and report: Link
  
  
• Sample project report: Link

• Final project proposal Due: May 9 (11:59 pm)

• Final paper and code due: Finals week (June 10, 11:59pm)

• Helpful links

  • Kaggle ML competitions

  • Stanford ML projects (also look at the previous years)

  • Stanford ML project ideas

  • UCI ML repository

  • Datasets for ML problems in Hugging Face

  • scikit-learn

  • Weka

Supplementary Materials

• Pedro Domingos’ Machine Learning course on Coursera

• CMU 10-701, Tom Mitchell

• You are encouraged to explore deep learning with PyTorch in this class.

Course Requirements and Grading

This course will be taught in-person. Please use Piazza and Canvas for communication and discussion.

Grading will be based on the following criteria:

Grading Scale