Math 490, Mathematics of Machine Learning, Fall 2018

Professor:Dr. Kay Kirkpatrick
Contact:231 Illini Hall, kkirkpat(at)illinois.edu
TA:Michael Livesay, mlivesa2(at)illinois.edu
Course site:https://faculty.math.illinois.edu/~kkirkpat/490fall2018.html
Lectures: MWF 2:00-2:50pm in 341 Altgeld Hall
Office hours: Mondays and Fridays, 3:00-3:50, or by appointment. I would be happy to answer your questions in my office anytime as long as I'm not otherwise engaged, and before and after class are good times to catch me either in my office or in the classroom.
Textbook: The main text will be Understanding Machine Learning: From Theory to Algorithms, 1st Edition by Shai Shalev-Shwartz and Shai Ben-David: https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf
Grading policy: Homework: 40% of the course grade
Two Midterms: 15% each. Please let me know as soon as possible if you need any accommodation.
Final Project: 30%, on a topic of your choice related to the course. See HW#3 for options. Our assigned final exam time is 7:00-10:00pm, Friday, December 14.
Inclusion: I am committed to affirming the identities, realities and voices of all students, especially those from historically marginalized or underrepresented backgrounds. I value the use of person-centered language, specified gender pronouns, and respect for everyone. Please contact me to request disability accommodations. You should also know that I'm a mandatory reporter.


Homework (due Fridays in class or to my email address or my mailbox in AH 250 by the end of class): You are encouraged to work together on the homework, but I ask that you write up your own solutions and turn them in separately.

Late homework will not be graded, so I will drop your two lowest homework scores.

HW #1 is due the second Friday in class: info sheet handed out, also available here.

HW #2 is due Fri 9/14 in class: available here. Solutions here.

HW #3 is due Fri 9/21 in class: Textbook problems: p. 30 #3.6, p. 35 #4.2, p. 41 #5.1. PLUS: Email me (with copy to the TA) about your final project, indicating (a) whether you'll pick a talk, paper, poster or webpage, (b) which topics you are considering, (c) what is your motivation, and (d) 2 or 3 references you will work from--books, articles, and technical websites are all fine. Talks will be 15 minutes each, subject to time constraints. Papers will be 3-10 pages. Webpage will be 2-5 screens long, possibly with an interactive simulation that you program. Solutions here.

HW #4 is due Fri 9/28 in class: available here. PLUS: read "How to give a good colloquium" by John McCarthy, and email both prof and TA a few sentences describing a) a piece of anyone's speaking advice that you have used in the past and how well it worked, and b) a piece of McCarthy's advice that you will try in the future. Solutions here.

Announcement: the first exam will be October 12, covering all of the course material (lectures and HWs) up to and including the Friday before the exam, Oct 5.

HW #5 is due Fri 10/5 in class: Before you do the other HW items, read John Lee's essay "Some Remarks on Writing Mathematical Proofs" and put his advice into practice for the rest of the HW in at least 2 specific ways that you flag for the grader with citations. The remaining HW problems will be from the textbook: Ch. 6: #6.1, #6.4, #6.8, and Ch. 9: #9.1 and #9.6. Solutions here.

NO HW Fri 10/12; instead there was Exam 1.

HW #6 due Fri 10/18 in class, textbook problems: #10.2, #12.1, and #12.3 Solutions here.

HW #7 due Fri 10/19 in class, from the textbook: #10.3, #12.2, and after reading the writing blogpost Five common writing mistakes, email Kay and Michael with a two-sentence summary of what you learned from the blog. Announcement: Kay will be giving a talk at Beckman on 10/26 at noon, and lunch will be provided there (Beckman 1005). Attendance is non-mandatory. Solutions here.

HW #8 due Fri 11/2 to Kay (or her email with subject line "[Math 490] HW#8 ..." or her mailbox in AH 250 or her door-box at IH 231) by 3pm: Proposal for final project.
1. Identify your topic and your thesis statement (see link).
2. Think about your audience: your classmates, not just me; people in STEM, not just in your field. Answer two or more of the following questions: Why should your audience care? What do you want them to take away from your project? How can you clarify the benefits of your project to your audience?
3. Specifics: What kinds of audiovisual aids will you be choosing to use? Your project should have at least one item of visual interest (picture, simulation, etc.), and at least one item of technical interest (theorem, algorithm, etc.). Which two or three definitions or key ideas will you introduce to your audience? What is a good example (think n=2) that illustrates the main point of your project? Can you find a story that's related to your topic?
4. Describe the main messages of at least 3 references that you are using, i.e., what are their thesis statements?

HW#9: textbook problems #12.4, #13.1, #13.3, #14.1. Due on paper to Kay or Michael by the end of class on Friday 11/9. Solutions here. Plus read one or two of the following on technical communication that is relevant to your final project, and write a two- or three-sentence summary/analysis of what you took away from your reading in an email to Kay with subject line: [Math 490] HW#9 reading by [author]...
"How to Talk Mathematics" by Paul Halmos
Doumont's downloadable booklet on slide design for scientific talks
"Slides are not all evil" by Jean-luc Doumont.
"How to give a good 20-minute math talk" by William Ross
"The Science of Scientific Writing" by Gopen and Swan
"How to Write Mathematics" by Halmos.

HW#10 due Nov 16 by the end of class: textbook problems #14.2, #15.1. Solutions here. Plus this revising exercise: trimming words. Option 1): Take an old email of yours to someone important that was too long (more than 2-3 paragraphs of 2-3 lines each), and trim it down without losing key information. You may fictionalize/redact names, etc., for privacy. Include word counts before and after: the after count should be no more than 85% of the before count. Option 2): Revise two slides of a talk, maybe yours or someone else's, according to the principles that you've learned. This may include finding or drawing a picture to illustrate the main point of the slide, or making the wording more efficient to reduce word-wrap. The result should look something like this example, with four slides: two originals and two improved versions (hand-drawn is fine).


Extra credit opportunity: a book review due after Thanksgiving, based on a book that will help you with your final project. You can pick the book in class on Mon 11/12, Wed 11/14, or Fri 11/16. For this extra credit, you should read (at least) part of the book, summarize its main points, find a nice quotation from it, and provide a recommendation of who should read it. Please also return the book by the final :)



HW#11 preview, due Nov 30: book problems 15.3 and 16.3 due by the end of class Friday. Solutions here. Plus: First draft of your final project due by the end of the week. For a paper, you should have at least 1.5 pages of text (12 pt font; 1 to 1.5 spacing), about half a page outlining the remainder, a figure, and citations. For a talk, you should have at least 8 finished slides, plus the remainder of the talk outlined (e.g., headlines only). If you turn your first draft in as hard-copy, I can mark it up with specific suggestions; if you turn it in electronically, I will reply by email with more general comments/suggestions. If you'd like a particular kind of feedback (e.g., if you hand in hard-copy but you only want general suggestions), please let me know.

Other resources, especially for editing help:
UIUC writing center: http://www.cws.illinois.edu/workshop/
Purdue Online Writing Lab: https://owl.purdue.edu/owl/purdue_owl.html

NO HW due on December 7. Instead there is Exam 2 in class.

HW#12 preview due 12/12: Read this review of Noble's Algorithms of Oppression, and write a two sentence summary/analysis.



Revising advice from Stephen B. Heard, The Scientist's Guide to Writing, p. 198ff.
1. Read for self-revision at the time of day that you think least clearly.
2. Change your font to something unfamiliar or strange-looking.
3. Read your draft out loud.
4. Remind yourself to read like a reader, not like the writer.
5. Check for unclear pronoun antecedents: this, that, these, and those.
6. Does your topic sentence of each paragraph cohere with the rest of the paragraph?
7. Do you have transitions between paragraphs and sections? How are the different topics related?



All final projects are due via email by the final exam time, Friday, Dec. 14 at 7pm (for speakers, this means emailing final slides before 7pm). Attendance/participation will be graded during the final exam period, estimated to run from 7pm to 9pm on Dec. 14, in the usual classroom.

Departures from "traditional" machine learning will be encouraged for the final project, and you have a choice of paper (3-10 pages), talk (15 minutes), poster (regular size), or webpage (3-6 screens) Here's an example of a project webpage: https://faculty.math.illinois.edu/~kkirkpat/percolation.html

Final project ideas: summary/analysis of anything by Alan Turing, Sarah M. Brown (http://sarahmbrown.org/research/), Timnit Gebru (http://ai.stanford.edu/~tgebru/), Sanmi Koyejo (http://sanmi.cs.illinois.edu/publications.html), Giuseppe Longo (https://www.di.ens.fr/users/longo/download.html).
Bengio's The Consciousness Prior (https://arxiv.org/pdf/1709.08568.pdf).
Neuronal Synchrony in Complex-Valued Deep Networks (https://arxiv.org/abs/1312.6115v5).
Quantum computing and quantum information: Peter Shor's famous paper, Deutsch and Marletto.
Biological phenomena that don't fit the DNA=code metaphor, e.g., Denis Noble's work.
Ethics and algorithms: Cathy O'Neil's book Weapons of Math Destruction; Safiya Umoja Noble's book Algorithms of Oppression.
Reproducibility crises in psychology and ML/AI.
Spurious correlations in Big Data: Calude and Longo, 2016.
Backpropagation vs. neural back-feed of information to pre-synaptic neuron.
Causality and causal inference: Judea Pearl's work.


Syllabus

This is an advanced course on the mathematics of machine learning, and some probability/statistics and programming are prerequisites for this course. Machine learning is a growing field at the intersection of probability, statistics, optimization, and computer science, which aims to develop algorithms for making predictions based on data. This course will cover foundational models and mathematics for machine learning, including statistical learning theory and neural networks, with a project component.

Topics: We will be covering most of the topics in Chapters 2, 3, 4, 5, 6, 9, 10, 12, 13, 14, 15, 16, 19, and 20. We will skip most of Chapters 7, 8, 11, 17, and 18. At the end of the semester, time permitting, we will cover some of my recent research. The final exam period will probably be spent on talks by your classmates, so please plan to be here then.


Why work on your communication skills?

"It usually takes me more than three weeks to prepare a good impromptu speech." --Mark Twain

I think that success in your career (any career) depends in part on how well you communicate your ideas and persuade other people, so I am giving you a chance to learn and practice good writing or presenting skills. Some of the homework assignments will lead up to the final project, for which you will have a choice of topic (related to the course) and of communication format (paper or talk). The homework will be graded partly on clarity, brevity, and coherence. This is a great opportunity to improve your writing or presenting skills, in order to make your ideas more clear and persuasive--and to succeed.

"I am sorry I have had to write you such a long letter, but I did not have time to write you a short one." --Blaise Pascal

Emergency information link and the new one.


Some more resources for writing and speaking:

Su: Good Math Writing
Halmos: How to Write Mathematics
Gopen and Swan: The Science of Scientific Writing
Williams: Style: The Basics of Clarity and Grace (book, any edition), Longman.


Bruce Reznick's list of resources
Gallian: How to Give a Good Talk
Shaw: Making Good Talks Into Great Ones
Gallo: Public-Speaking Lessons from TED Talks
Lerman: Math job talk advice
Steele: Speaking tips organized in categories that includes this great but little-known tip about graphs on slides


Fun picture of queues.