1 L01.1 Lecture Overview 2 L01.2 Sample Space 3 L01.3 Sample Space Examples 4 L01.4 Probability Axioms 5 L01.5 Simple Properties of Probabilities 6 L01.6 More Properties of Probabilities 7 L01.7 A Discrete Example 8 L01.8 A Continuous Example 9 L01.9 Countable Additivity 10 L01.10 Interpretations & Uses of Probabilities 11 S01.0 Mathematical Background Overview 12 S01.1 Sets 13 S01.2 De Morgan's Laws 14 S01.3 Sequences and their Limits 15 S01.4 When Does a Sequence Converge 16 S01.5 Infinite Series 17 S01.6 The Geometric Series 18 S01.7 About the Order of Summation in Series with Multiple Indices 19 S01.8 Countable and Uncountable Sets 20 S01.9 Proof That a Set of Real Numbers is Uncountable 21 S01.10 Bonferroni's Inequality 22 L02.1 Lecture Overview 23 L02.2 Conditional Probabilities 24 L02.3 A Die Roll Example 25 L02.4 Conditional Probabilities Obey the Same Axioms 26 L02.5 A Radar Example and Three Basic Tools 27 L02.6 The Multiplication Rule 28 L02.7 Total Probability Theorem 29 L02.8 Bayes' Rule 30 L03.1 Lecture Overview 31 L03.2 A Coin Tossing Example 32 L03.3 Independence of Two Events 33 L03.4 Independence of Event Complements 34 L03.5 Conditional Independence 35 L03.6 Independence Versus Conditional Independence 36 L03.7 Independence of a Collection of Events 37 L03.8 Independence Versus Pairwise Independence 38 L03.9 Reliability 39 L03.10 The King's Sibling 40 L04.1 Lecture Overview 41 L04.2 The Counting Principle 42 L04.3 Die Roll Example 43 L04.4 Combinations 44 L04.5 Binomial Probabilities 45 L04.6 A Coin Tossing Example 46 L04.7 Partitions 47 L04.8 Each Person Gets An Ace 48 L04.9 Multinomial Probabilities 49 L05.1 Lecture Overview 50 L05.2 Definition of Random Variables 51 L05.3 Probability Mass Functions 52 L05.4 Bernoulli & Indicator Random Variables 53 L05.5 Uniform Random Variables 54 L05.6 Binomial Random Variables 55 L05.7 Geometric Random Variables 56 L05.8 Expectation 57 L05.9 Elementary Properties of Expectation 58 L05.10 The Expected Value Rule 59 L05.11 Linearity of Expectations 60 S05.1 Supplement: Functions 61 L06.1 Lecture Overview 62 L06.2 Variance 63 L06.3 The Variance of the Bernoulli & The Uniform 64 L06.4 Conditional PMFs & Expectations Given an Event 65 L06.5 Total Expectation Theorem 66 L06.6 Geometric PMF Memorylessness & Expectation 67 L06.7 Joint PMFs and the Expected Value Rule 68 L06.8 Linearity of Expectations & The Mean of the Binomial 69 L07.1 Lecture Overview 70 L07.2 Conditional PMFs 71 L07.3 Conditional Expectation & the Total Expectation Theorem 72 L07.4 Independence of Random Variables 73 L07.5 Example 74 L07.6 Independence & Expectations 75 L07.7 Independence, Variances & the Binomial Variance 76 L07.8 The Hat Problem 77 S07.1 The Inclusion-Exclusion Formula 78 S07.2 The Variance of the Geometric 79 S07.3 Independence of Random Variables Versus Independence of Events 80 L08.1 Lecture Overview 81 L08.2 Probability Density Functions 82 L08.3 Uniform & Piecewise Constant PDFs 83 L08.4 Means & Variances 84 L08.5 Mean & Variance of the Uniform 85 L08.6 Exponential Random Variables 86 L08.7 Cumulative Distribution Functions 87 L08.8 Normal Random Variables 88 L08.9 Calculation of Normal Probabilities 89 L09.1 Lecture Overview 90 L09.2 Conditioning A Continuous Random Variable on an Event 91 L09.3 Conditioning Example 92 L09.4 Memorylessness of the Exponential PDF 93 L09.5 Total Probability & Expectation Theorems 94 L09.6 Mixed Random Variables 95 L09.7 Joint PDFs 96 L09.8 From The Joint to the Marginal 97 L09.9 Continuous Analogs of Various Properties 98 L09.10 Joint CDFs 99 S09.1 Buffon's Needle & Monte Carlo Simulation 100 L10.1 Lecture Overview 101 L10.2 Conditional PDFs 102 L10.3 Comments on Conditional PDFs 103 L10.4 Total Probability & Total Expectation Theorems 104 L10.5 Independence 105 L10.6 Stick-Breaking Example 106 L10.7 Independent Normals 107 L10.8 Bayes Rule Variations 108 L10.9 Mixed Bayes Rule 109 L10.10 Detection of a Binary Signal 110 L10.11 Inference of the Bias of a Coin 111 L11.1 Lecture Overview 112 L11.2 The PMF of a Function of a Discrete Random Variable 113 L11.3 A Linear Function of a Continuous Random Variable 114 L11.4 A Linear Function of a Normal Random Variable 115 L11.5 The PDF of a General Function 116 L11.6 The Monotonic Case 117 L11.7 The Intuition for the Monotonic Case 118 L11.8 A Nonmonotonic Example 119 L11.9 The PDF of a Function of Multiple Random Variables 120 S11.1 Simulation 121 L12.1 Lecture Overview 122 L12.2 The Sum of Independent Discrete Random Variables 123 L12.3 The Sum of Independent Continuous Random Variables 124 L12.4 The Sum of Independent Normal Random Variables 125 L12.5 Covariance 126 L12.6 Covariance Properties 127 L12.7 The Variance of the Sum of Random Variables 128 L12.8 The Correlation Coefficient 129 L12.9 Proof of Key Properties of the Correlation Coefficient 130 L12.10 Interpreting the Correlation Coefficient 131 L12.11 Correlations Matter 132 L13.1 Lecture Overview 133 L13.2 Conditional Expectation as a Random Variable 134 L13.3 The Law of Iterated Expectations 135 L13.4 Stick-Breaking Revisited 136 L13.5 Forecast Revisions 137 L13.6 The Conditional Variance 138 L13.7 Derivation of the Law of Total Variance 139 L13.8 A Simple Example 140 L13.9 Section Means and Variances 141 L13.10 Mean of the Sum of a Random Number of Random Variables 142 L13.11 Variance of the Sum of a Random Number of Random Variables 143 S13.1 Conditional Expectation Properties 144 L14.1 Lecture Overview 145 L14.2 Overview of Some Application Domains 146 L14.3 Types of Inference Problems 147 L14.4 The Bayesian Inference Framework 148 L14.5 Discrete Parameter, Discrete Observation 149 L14.6 Discrete Parameter, Continuous Observation 150 L14.7 Continuous Parameter, Continuous Observation 151 L14.8 Inferring the Unknown Bias of a Coin and the Beta Distribution 152 L14.9 Inferring the Unknown Bias of a Coin - Point Estimates 153 L14.10 Summary 154 S14.1 The Beta Formula 155 L15.1 Lecture Overview 156 L15.2 Recognizing Normal PDFs 157 L15.3 Estimating a Normal Random Variable in the Presence of Additive Noise 158 L15.4 The Case of Multiple Observations 159 L15.5 The Mean Squared Error 160 L15.6 Multiple Parameters; Trajectory Estimation 161 L15.7 Linear Normal Models 162 L15.8 Trajectory Estimation Illustration 163 L16.1 Lecture Overview 164 L16.2 LMS Estimation in the Absence of Observations 165 L16.3 LMS Estimation of One Random Variable Based on Another 166 L16.4 LMS Performance Evaluation 167 L16.5 Example: The LMS Estimate 168 L16.6 Example Continued: LMS Performance Evaluation 169 L16.7 LMS Estimation with Multiple Observations or Unknowns 170 L16.8 Properties of the LMS Estimation Error 171 L17.1 Lecture Overview 172 L17.2 LLMS Formulation 173 L17.3 Solution to the LLMS Problem 174 L17.4 Remarks on the LLMS Solution and on the Error Variance 175 L17.5 LLMS Example 176 L17.6 LLMS for Inferring the Parameter of a Coin 177 L17.7 LLMS with Multiple Observations 178 L17.8 The Simplest LLMS Example with Multiple Observations 179 L17.9 The Representation of the Data Matters in LLMS 180 L18.1 Lecture Overview 181 L18.2 The Markov Inequality 182 L18.3 The Chebyshev Inequality 183 L18.4 The Weak Law of Large Numbers 184 L18.5 Polling 185 L18.6 Convergence in Probability 186 L18.7 Convergence in Probability Examples 187 L18.8 Related Topics 188 S18.1 Convergence in Probability of the Sum of Two Random Variables 189 S18.2 Jensen's Inequality 190 S18.3 Hoeffding's Inequality 191 L19.1 Lecture Overview 192 L19.2 The Central Limit Theorem 193 L19.3 Discussion of the CLT 194 L19.4 Illustration of the CLT 195 L19.5 CLT Examples 196 L19.6 Normal Approximation to the Binomial 197 L19.7 Polling Revisited 198 L20.1 Lecture Overview 199 L20.2 Overview of the Classical Statistical Framework 200 L20.3 The Sample Mean and Some Terminology 201 L20.4 On the Mean Squared Error of an Estimator 202 L20.5 Confidence Intervals 203 L20.6 Confidence Intervals for the Estimation of the Mean 204 L20.7 Confidence Intervals for the Mean, When the Variance is Unknown 205 L20.8 Other Natural Estimators 206 L20.9 Maximum Likelihood Estimation 207 L20.10 Maximum Likelihood Estimation Examples 208 L21.1 Lecture Overview 209 L21.2 The Bernoulli Process 210 L21.3 Stochastic Processes 211 L21.4 Review of Known Properties of the Bernoulli Process 212 L21.5 The Fresh Start Property 213 L21.6 Example: The Distribution of a Busy Period 214 L21.7 The Time of the K-th Arrival 215 L21.8 Merging of Bernoulli Processes 216 L21.9 Splitting a Bernoulli Process 217 L21.10 The Poisson Approximation to the Binomial 218 L22.1 Lecture Overview 219 L22.2 Definition of the Poisson Process 220 L22.3 Applications of the Poisson Process 221 L22.4 The Poisson PMF for the Number of Arrivals 222 L22.5 The Mean and Variance of the Number of Arrivals 223 L22.6 A Simple Example 224 L22.7 Time of the K-th Arrival 225 L22.8 The Fresh Start Property and Its Implications 226 L22.9 Summary of Results 227 L22.10 An Example 228 L23.1 Lecture Overview 229 L23.2 The Sum of Independent Poisson Random Variables 230 L23.3 Merging Independent Poisson Processes 231 L23.4 Where is an Arrival of the Merged Process Coming From? 232 L23.5 The Time Until the First (or last) Lightbulb Burns Out 233 L23.6 Splitting a Poisson Process 234 L23.7 Random Incidence in the Poisson Process 235 L23.8 Random Incidence in a Non-Poisson Process 236 L23.9 Different Sampling Methods can Give Different Results 237 S23.1 Poisson Versus Normal Approximations to the Binomial 238 S23.2 Poisson Arrivals During an Exponential Interval 239 L24.1 Lecture Overview 240 L24.2 Introduction to Markov Processes 241 L24.3 Checkout Counter Example 242 L24.4 Discrete-Time Finite-State Markov Chains 243 L24.5 N-Step Transition Probabilities 244 L24.6 A Numerical Example - Part I 245 L24.7 Generic Convergence Questions 246 L24.8 Recurrent and Transient States 247 L25.1 Brief Introduction (RES.6-012 Introduction to Probability) 248 L25.2 Lecture Overview 249 L25.3 Markov Chain Review 250 L25.4 The Probability of a Path 251 L25.5 Recurrent and Transient States: Review 252 L25.6 Periodic States 253 L25.7 Steady-State Probabilities and Convergence 254 L25.8 A Numerical Example - Part II 255 L25.9 Visit Frequency Interpretation of Steady-State Probabilities 256 L25.10 Birth-Death Processes - Part I 257 L25.11 Birth-Death Processes - Part II 258 L26.1 Brief Introduction (RES.6-012 Introduction to Probability) 259 L26.2 Lecture Overview 260 L26.3 Review of Steady-State Behavior 261 L26.4 A Numerical Example - Part III 262 L26.5 Design of a Phone System 263 L26.6 Absorption Probabilities 264 L26.7 Expected Time to Absorption 265 L26.8 Mean First Passage Time 266 L26.9 Gambler's Ruin