6.S951: Modern Mathematical Statistics, fall 2024

Overview

Mathematical statistics provides a formal language for reasoning about data and uncertainty. This course introduces the basic framework of statistical decision theory and learning theory. We focus on two themes. First, we will develop results for statistical inference  —  procedures that have mathematical guarantees about the degree of certainty in the output. Such results involve both mathematical derivations as well as conceptual considerations about what of outputs and guarantees are practically informative. We discuss causality, multiple hypothesis testing, nonparametric and semiparametric statistics, and results for model misspecification.

The second theme is statistical optimality  —  what are the fundamental limits of certainty and precision given a limited amount of data, and which procedures achieve this? Here, we discuss concepts such as sufficiency, Bayes and minimax optimality of statistical procedures, with applications to optimal estimation, hypothesis testing, and prediction. In particular, we discuss the asymptotic optimality of maximum likelihood and information-theoretic lower bounds for estimation.

This is a graduate course targeted at students interested in statistical research, with an emphasis on proofs and fundamental understanding. We assume previous exposure to undergraduate statistics, along with a strong undergraduate background in linear algebra, probability, and real analysis. This course together with IDS.160[J] (which is taught in the spring) form a two-part graduate curriculum in statistical theory. Neither course strictly depends on the other.

Time and location

• T/Th 9:30am - 11:00am

• 36-156

Outline

• Intro to statistical inference, the central limit theorem and concentration inequalities

• Conformal prediction and boostrap (valid inference with ML algorithms)

• Statistical decision theory and learning theory framework

• Minimax and Bayes optimality

• Optimal hypothesis testing and Neyman–Pearson

• Information-theoretic minimax lower bounds

• Theory of regression and generalized linear models

• M-estimation and maximum likelihood

• Behavior of M-estimators and maximum likelihood with misspecified models

• Optimality of maximum likelihood

• Empirical Bayes and the James-Stein estimator

• Multiple testing and the false discovery rate

• Semiparametric inference and causality

This course is not about any one set of mathematical techniques, but is instead about understanding foundational statistical behavior. We will study this with exact calculations, asymptotic approximations, and concentration inequalities (finite-sample bounds).

Schedule

The course is split into four units.

• Optimality I. This unit introduces statistical decision theory and notions of statistical optimality (in particular, Bayes optimality and minimax optimality). We show how the framework can be used to describe estimation, prediction, and hypothesis testing.

• Inference I This unit discusses statistical inference  —  procedures that output a rigorous notion of confidence/uncertainty. We focus on confidence intervals and related outputs for now, giving algorithms such as bootstrap and conformal prediction that yield confidence intervals. We also discuss the relationship between confidence intervals and statistical testing.

• Advanced Inference. This unit covers advanced topics in statistical inference: predictive inferenence and conformal prediction, multiple hypothesis testing, and empirical Bayes.

• Asymptotic Inference and Optimality. This unit uses asymptotic calculations to analyze the behavior of estimators. This enables us to (i) establish (asymptotically) optimal estimators, (ii) give approximate confidence intervals, and (iii) to understand behavior under model misspecification.

Homework

• Homework 1 is released on Canvas, due Thursday, September 19.

• Homework 2 is released on Canvas, due Friday, September 27.

• Homework 3 is released on Canvas, due Friday, October 4.

• Homework 4 is released on Canvas, due Friday, October 11.

• Homework 5 is released on Canvas, due Sunday, October 20.

• Homework 6 is released on Canvas, due Sunday, October 27.

• Homework 7 is released on Canvas, due Friday, Nov 8.

• Homework 8 is released on Canvas, due Monday, Nov 18.

• Homework 8 is released on Canvas, due Wednesday, Nov 25.

Course materials

This course will use course notes and selections from Asymptotic Statistics (“vdV”) as the primary course material. All of Statistics (“Wass”), Theory of Point Estimation (“TPE”), and Testing Statistical Hypotheses (“TSH”) will be used as supplementary texts. All are available online through the MIT library.

Logistics and grading

Assignments are submitted via Gradescope, and we have a Piazza discussion board.

The course will have weekly problem sets, a final exam, and a midterm exam. Enrolled students must submit latexed scribe notes for one lecture.

Final course grades are computed as 35% homework score + 35% max(midterm, final) + 25% min(midterm, final) + 5% scribe notes score.

Your lowest two problem set scores will be dropped. Your third lowest problem score can be dropped if you earned a score of least 50%. The point is that your grade is not sensitive to missing an assignment or having a poor score.

Office hours: see Piazza

Scribe notes

Scribe due 24 hours after the lecture for full credit (50% credit if within 48 hours), and these are graded. Please submit these on Gradescope.

• scribe sign-up

• scribe note template