Machine Learning, 10-701 and 15-781, 2004

Ziv Bar-Joseph and Andrew W. Moore

School of Computer Science, Carnegie Mellon University

Fall 2004 (Starting Tue Sep 14th)

It is hard to imagine anything more fascinating than automated systems that improve their own performance. The study of learning from data is commercially and scientifically important. This course is designed to give a graduate-level student a thorough grounding in the methodologies, technologies, mathematics and algorithms currently needed by people who do research in learning and data mining or who may need to apply learning or data mining techniques to a target problem.

The topics of the course draw from classical statistics, from machine learning, from data mining, from Bayesian statistics and from statistical algorithmics.

Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate.

Class lectures: Tuesdays & Thursdays 10:30am-11:50am, Wean Hall 5409 starting on Tuesday September 14th, 2004

Review sessions: Tuesdays 4:30pm-6:00pm Wean Hall 4623 starting on Tuesday September 14th, 2004 (details)

Instructors:

• Ziv Bar-Joseph, Wean Hall 4107, x8-5196, Office hours: Tuesday 12:00pm - 1:00pm
• Andrew Moore, NSH 3117, x8-7599, Office hours: by appointment through Kristen Schrauder, kristens@cs.cmu.edu

Teaching Assistants:

• Dave Ferguson, NSH 2211, x8-3978, Office hours: Thursdays 3:30pm-5:00pm
• Yanjun Qi, Wean Hall 3124, x8-9859, Office hours: Fridays 11:30am-1:00pm
• Maxim Likhachev, Wean Hall 8130, x8-3077, Office hours: Tuesdays 2:00pm-3:30pm

Textbook:

• Primary recommended (but optional) textbook: Machine Learning, Tom Mitchell.
• Additional recommended (optional) textbook: Pattern Classification (2nd Edition), Duda, Hart and Stork.
• Additional recommended (optional) textbook: Neural Networks for Pattern Recognition, Chris Bishop.

Course Website (this page):

• http://www.cs.cmu.edu/~awm/15781
• Corrections or comments to Dave.

Grading:

• Final grades will be based on midterm (20%), homework (4 assignments, 30%), final project (20%), and final exam (30%)

Policy on collaboration

Homeworks will be done individually: each student must hand in their own answers. It is acceptable, however, for students to collaborate in figuring out answers and helping each other solve the problems. We will be assuming that, as participants in a graduate course, you will be taking the responsibility to make sure you personally understand the solution to any work arising from such collaboration.

• Homework is worth full credit at the beginning of class on the due date.
• It is worth half credit for the next 48 hours.
• It is worth zero credit after that.
• You must turn in all of the 4 homeworks, even if for zero credit, in order to pass the course.

Homework assignments

• HW1: Probability, PAC learning, Density Estimation. Out Sep 16th, due 10:30am Thursday September 30 (duration 2 weeks).
• Homework 1 (PDF)
• Homework 1 Solution (PDF)
• For questions and clarifications, contact Max or Yanjun.
• HW2. MLE, EM, Regression. Out September 30, due 10:30am Thursday Oct 14th (duration 2 weeks).
• Homework 2 (PDF) (Updated version !!!)
• Matlab and data files for hw2
• For questions and clarifications, contact Max or Yanjun.
• Sample Matlab Solution Code
• HW2 Solution
• HW3. Out Oct 14th, due 10:30am Oct 28th (duration 2 weeks).
• Homework 3 (PDF)
• Matlab data files for hw3
• K-Means PNG file from Homework
• For questions and clarifications, contact Dave or Max.
• Homework 3 Solutions (PDF)
• HW4. Out Oct 28th, due 10:30am Tuesday Nov 16th (duration 19 days).
• Homework 4 (PDF)
• Solutions (PDF)
• Sine data (question 2)
• Figure 1 (EPS)
• Figure 2 (EPS)
• Example kNN data
• kNN Testing Code
• For questions and clarifications, contact Dave or Yanjun.

Projects

• Project Proposals due 10:30am Tuesday November 16th
• Candidate Projects
• Project Presentation Schedule:
  • 12:00pm - 1:30pm Thursday December 9th. NSH A507
  • 2:30pm - 4:00pm Thursday December 9th. NSH 1505
  • 10:30am - 12:00pm Friday December 10th. NSH A507
  • 3:00pm - 4:30pm Friday December 10th. NSH 1505
• Some data sets might be useful to you:
• UCI ML Repository (Various)
• UCI KDD Repository (Various)
• 20 News groups (Text)

Lecture schedule (and online slides when available)

Review sessions

Exam Schedule

• Note: both the midterm and the final will be open book.
• Midterm: Tuesday Nov 2nd, 10:30am-11:50am, WeH 4623 (in class).
  • Midterm exam solutions
• Final: Sunday, December 12th, 6:00pm-9:00pm, NSH 1305
  • Final exam solutions

Additional Resources

Here are some example questions here for studying for the midterm/final. Note that these are exams from earlier years, and contain some topics that will not appear in this year's final. And some topics will appear this year that do not appear in the following examples.

• The 2001 midterm exam
• The 2001 final exam
• Solutions to the 2001 final exam
• The 2002 midterm exam
• Solutions to the 2002 midterm exam
• Additional examples of midtermlike questions (PS) or (PDF)
• Solutions to the additional examples (PS) or (PDF)
• Final Exam 2002 (some figs missing)
• Answers for final Exam 2002
• The 2003 midterm exam
• Solutions to the 2003 midterm exam
• Final Exam 2003
• Answers for final Exam 2003

Note to people outside CMU

Feel free to use the slides and materials available online here. Please email the instructors with any corrections or improvements. Additional slides and software are available at the Machine Learning textbook homepage and at Andrew Moore's tutorials page.