Page 2 :
Random Variables, If S is a sample space with a probability measure and X is a realvalued function defined over the elements of S, then X is called, a random variable (or stochastic variable)., , Example:, •X is the random variable corresponding to the temperature of, the room at time t., • x is the measured temperature of the room at time t., 2

Page 3 :
Random Variable, Mathematically, a random variable (r.v.) X is a real-valued function, X(ω) over the sample space Ω of a random experiment, i.e., X : Ω → R, Random Variable 𝑋, Sample Space, Ω, Real Number Line, , Randomness comes from the fact that outcomes are random (X(ω) is a, deterministic function), , A random variable is called discrete if its range is either finite or, countably infinite. e.g. Two rolls of a die. The sum of the two, rolls. The number of sixes in the two rolls., But Continuous random variables defined over continuous sample, spaces will be taken.

Page 4 :
Examples for Random Variable, 1., , Flip a coin n times. Here Ω = {H, T }n. Define the random, , variable, X ∈ {0, 1, 2, . . . , n} to be the number of heads, 2. Roll a 4-sided die twice., (a) Define the random variable X as the maximum of the two rolls (X ∈ {1, 2, 3,, 4}), (b) Define the random variable Y to be the sum of the outcomes of the two, rolls (Y ∈ {2, 3, ··· , 8}), (c), , Define the random variable Z to be 0 if the sum of the two rolls is, odd and 1 if it is even, , 3. Flipping a coin until the first heads occurs: Ω = {H, TH, TTH,, , TTTH, . . .} Define the random variable, X ∈ {1, 2... .} to be the number of flips until the first heads

Page 5 :
Why Random Variable?, Why do we need random variables?, ◦ Random variable can represent the gain or loss in a random, experiment, e.g., stock market, ◦ Random variable can represent a measurement over a random, experiment, e.g., noise voltage on a resistor, , • In most applications we care more about these costs/measurements, than the underlying probability space, • Very often we work directly with random variables without knowing, (or caring to know) the underlying probability space

Page 6 :
Random Variables, , Example 1, Suppose that a coin is tossed twice so that the sample space is S = {HH, HT ,TH,T T }. Let, X represent the number of heads that can come up. With each sample point we can associate a, number for X as shown in the table:, , Event, Probability, x, , HH, , HT, , TH, , TT, , 2, , 1, , 1, , 0, , 1, 4, , 1, 4, , 1, 4, , 1, 4, , Thus, for example, in the case of HH (i.e., 2 heads), X = 2 while for TH (1 head), X = 1. It follows, that X is a random variable., , Also, we can write P(X = 2) =, P(X = 0)= 41 ., , 1,, 4, , P(X = 1)=, , 1, 4, , + 14=, , 1,, 2, , and

Page 7 :
Random Variables, , Example 2, A balanced coin is tossed four times. List the elements of the sample space, that are presumed to be equally likely, as this is what we mean by a coin being, balanced, and the corresponding values x of the random variable X , the total, number of heads., , Solution. If H and T stand for heads and tails, the results are as shown in the, following table:

Page 8 :
HHHH, HHHT, HHTH, HTHH, THHH, HHTT, HTHT, HTTH, , 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, , 4, 3, 3, 3, 3, 2, 2, 2, , THHT, THTH, TTHH, HTTT, THTT, TTHT, TTTH, TTTT, , 1 , P(X = 1) = 4 ,, Thus, we can write P(X = 0) = 16, 16, 6, 4, 1., P(X = 2)= 16, P(X = 3)= 16 and P(X = 4)= 16, , 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, 1, 16, , 2, 2, 2, 1, 1, 1, 1, 0

Page 9 :
Example 3, Two socks are selected at random and removed in succession from a drawer, containing five brown socks and three green socks. List the elements of the, sample space, the corresponding probabilities, and the corresponding values, x of the random variable X is the number of brown socks selected., , Solution. If B and G stand for brown and green, then we have following, probabilities, 5 4 20, 5 3 15, P (B B ) = · =, ,P (BG ) = · =, ,, 8 7 56, 8 7 56, P (GB ) = 3 ·5 = 15 , and P (GG ) = 3 ·2 = 6 ,, 8 7 56, 8 7, 56, and the results are shown in the following table:, Elements of Sample Space, BB, BG, Probability, 20/56 15/56, x, 2, 1, , GB, 15/56, 1, , GG, 6/56, 0

Page 10 :
Discrete Random Variables, , Many situations can be adequately modeled using discrete, random variables:, , • Number of failed hard-drives in a RAID system, •Number of (dijken) in a redundant water protection system, • Number of customers attempting to access a webserver, •Number of times you need to call an airline company until you get, someone competent on the other side…

Page 11 :
Probability Distributions, The probability distribution of a random variable X is a, description of the probabilities associated with each possible, outcome of X., For a discrete random variable X this is actually quite simple, as, X can only take a finite/countable number of possible values., , Definition: Probability Mass Function (p.m.f.), , 𝑝𝑋 (𝑥), Sample space, Ω, , Event 𝑋 = 𝑥, , 4

Page 12 :
Probability Distributions, If the space of possible outcomes is countably infinite, the, previous definition can be easily generalized:, , Definition: Probability Mass Function (p.m.f.)

Page 13 :
Example 4, Check whether the function given by, f (x) =, , x +2, for x = 1,2,3,4,5, 25, , can serve as the probability distribution of a discrete random variable., 3, 25, , Solution. Substituting the different values of x , we get f (1) =, 4 , f (3) = 5 , f (4) = 6 , and f (5) = 7 . Since these, f (2) = 25, 25, 25, 25, values are all nonnegative, the first condition of Theorem is satisfied, and, since, f(1)+f(2)+f(3)+f(4)+f(5) =, 3, 25, , 3, 25, , 4, , 5, , 6, , 7, , + 25 + 25 + 25 + 25 = 1, , The second conditions of Theorem is satisfied Thus, the given function an serve as, the probability distribution of a random variable having the range {1,2,3,4,5}.

Page 14 :
Example 5, Suppose that a pair of fair dice are to be tossed, and let the random, variable X denote the sum of the points. Obtain the probability, distribution for X ., , Solution. The random variable X is the sum of the coordinates for each, point. Thus for (3, 2) we have X = 5. Using the fact that all 36 sample, points are equally probable, so that each sample point has probability 1/36., First Die, , Second, Die, , The one is red is the sum

Page 16 :
Figure 1 : Probability Bar Chart

Page 17 :
Example, Consider a situation in which a user can make one of three, possible requests to a GUI. We can view this as a random, experiment with the following sample space:, We can identify each request with a number. For instance, identify Print, Save and Cancel with 0,1, and 2, respectively. Let, X be the random variable corresponding to the random, experiment above., , It’s very easy to check this is a valid probability mass function.

Page 18 :
Example, Let X be the number of times you need to send a packet over a, network to ensure it was received., , Let’s check this is a valid probability mass function

Page 19 :
Example

Page 20 :
Cumulative Distribution Function, For any random variable the possible outcomes of a random, variable are real numbers. It is then often useful to describe the, probability distribution using a different function…, , Definition: Cumulative Distribution Function (c.d.f.), , This seemingly simple object is extremely powerful, as all the, information contained in the probability mass function is also, contained in the c.d.f., (furthermore we will see that this object is also properly, defined for any random variable, unlike the p.m.f.)

Page 21 :
Cumulative Distribution Function, Properties:, , IMPORTANT: note that the c.d.f. is defined for any real, number, and not only for the values X can take !!!, For discrete r.v.’s this object looks a bit funky…

Page 22 :
C.D.F. Example, , Let’s plot the p.m.f. and c.d.f.:, , -1, , 1.0, , 1.0, , 0.9, , 0.9, , 0.8, , 0.8, , 0.7, , 0.7, , 0.6, , 0.6, , 0.5, , 0.5, , 0.4, , 0.4, , 0.3, , 0.3, , 0.2, , 0.2, , 0.1, , 0.1, , 0, , 1, , 2, , 3, , 4, , 5, , -1, , 0, , 1, , 2, , 3, , 4, , 5

Page 23 :
C.D.F. Example, 1.0, 0.9, 0.8, 0.7, , 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, , -1, , A – 0.3, C – 0.5, , 0, , B – 0.4, D – 0.6, , 1, , 2, , 3, , 4, , 5, , 12

Page 24 :
Random Variables, , Discrete Probability Distributions, , Continuous Random Variables, , Example, Find the cumulative distribution function of the total of heads obtained in, four tosses of a balanced coin., 4 , f (2) = 6 , f (3) =, Solution. Given f (0) = 116 , f (1) = 16, 16, f (4) = 116from Example, it follows that, , F (0) = f (0) =, , 1, ,, 16, , 5, F (1) = f (0) + f (1) =, ,, 16, , 11, F (2) = f (0) + f (1) + f (2) =, ,, 16, , 15, F (3) = f (0) + f (1) + f (2) + f (3) =, ,, 16, F (4) = f (0) + f(1) + f (2) + f (3) + f (4) = 1., , 4,, 16, , and

Page 25 :
Random Variables, , Discrete Probability Distributions, , Continuous Random Variables, , Hence, the distribution function is given by, 1, , F (x) =, , ⎪161, ⎪, ⎨, 5, 16, 11, ⎪16, 15, ⎪16, , 1, , for x < 0,, for 0 ≤ x, for 1 ≤ x, for 2 ≤ x, for 3 ≤ x, for x ≥ 4., , <1,, <2,, <3,, <4,, , Observe that this distribution function is defined not only for the values, taken on by the given random variable, but for all real numbers, For instance, we can write F (1.7) =, , 5, 16, , and F (100) =1,, , although the probabilities of getting at most 1.7 heads or at most 100 heads, in four tosses of a balanced coin may not be of any real significance.

Page 26 :
Example, Find the cumulative distribution function of the random variable X of, Example 3 and plot its graph., Solution. Based on the probabilities given in the following table, Elements of Sample Space, BB, BG, Probability, 20/56 15/56, x, 2, 1, we can write f (0) =, , 6,, 56, , f (1) =, , 15 +, 56, , 15 = 30 ,, 56, 56, , and f (2) =, , that, , F (0) = f (0) =, , 6, ,, 56, , GB, GG, 15/56 6/56, 1, 0, , 36, F (1) = f (0) + f(1) =, ,, 56, F (2) = f (0) + f (1) + f (2) = 1., , 20 ,, 56, , so

Page 27 :
Hence, the distribution function of X is given by, ⎧, 1 for x < 0,, ⎪, ⎨ 6 for 0 ≤ x <1,, F (x) = 56, 36 for 1 ≤ x <2,, ⎪56, 1 for x ≥ 2., F (x), 1, , 36/56, , 6/56, 0, , 1, , 2, , x

Page 28 :
Example, Find the distribution function of the random variable that has the, probability distribution, x, f (x) =, for x = 1,2,3,4,5., 15, Solution. Since f (1) =, 5 ,then, f (5) = 15, , 1,, 15, , f (2) =, , 0, 1, ⎪15, , F (x) =, , 15, 6, ⎪15, 10, ⎪15, , ⎪, 1, , 2,, 15, , f (3) =, , for x < 1,, for 1 ≤ x, for 2 ≤ x, for 3 ≤ x, for 4 ≤ x, for x ≥ 5, , 3,, 15, , <2,, <3,, <4,, <5,, , f (4) =, , 4 ,, 15, , and

Page 29 :
Example, If X has the cumulative distribution function F (1) = 0.25, F (2) = 0.61,, F (3) = 0.83, and F (4) = 1 for x = 1,2,3,4, find the probability, distribution of X ., Solution. We have, f (1) =, f (2) =, f (3) =, f (4) =, , F (1) = 0.25,, F (2) − F (1) = 0.61 − 0.25 = 0.36,, F (3) − F (2) = 0.83 − 0.61 = 0.22,, F (4) − F (3) = 1 − 0.83 = 0.17.

Page 30 :
Example, If X has the distribution function, ⎧, 1 for x < −1,, ⎪1, ⎨, ⎪4 for − 1 ≤ x < 1,, F (x) =, for 1 ≤ x <3,, 3, ⎪, 4, for 3 ≤ x <5,, ⎪, ⎩, 1 for x ≥ 5., find, 1, 2, 3, , 4, 5, , P(X ≤ 3), P(X = 3), P(X < 3);, P(X ≥ 1);, P(−0.4 < X < 4);, , P(X = 5);, the probability distribution of X .

Page 32 :
Mean and Variance, , There are certain “summaries” of the distribution of a random, variable that can give a lot of information about it. These have, also easy intuitive interpretations., , Definition: Mean and Variance (discrete r.v.)

Page 33 :
Mean and Variance, The mean is a weighted average of the possible values of X,, where the weights are their corresponding probabilities., Therefore the mean describes the “center” of the distribution., In other words X takes values “around” its mean…, The variance measures the dispersion of X around the mean. If, this value is large means X varies a lot., , Definition: Standard deviation, , 14

Page 34 :
Probability Distributions, Mean and Variance for, Discrete Random Variables, The probability distribution of a discrete random variable is defined as a, function that specifies the probability associated with each possible, outcome the random variable can assume., p(x) ≥ 0 for all values of x, p(x) = 1,  The mean, or expected value, of a discrete random variable is, ,  = E( x) =  xp( x)., ,  The variance of a discrete random variable x is, ,  2 = E[( x −  )2 ] =  ( x −  )2 p( x).

Page 35 :
Example, The number of surface defects on the CD-RW of a certain, manufacturer is well modeled by a random variable X with the, following distribution:, , 35

Page 36 :
Example, , 36

Page 37 :
Example, Suppose you collect happy-meal* toys, and are missing only one, (out of 5 possible toys) – the TeslaCoil. Assume that in each, meal you buy the probability of getting this toy is 0.2, and, these events are independent from one another., •Suppose you go to two times to the restaurant. Let X be, random variable corresponds to the number TeslaCoils you have, after the two meals., , 20, * - happy meal is a trademark of McDonald's corporation, and I have no affiliation with them whatsoever…

Page 38 :
Properties of the Mean and Variance, Properties:, , WARNING!!!

Page 39 :
Example, You are playing slot-machine game in a casino downtown. The, machine gives a prize of €100 with probability 0.02. To play you, need to pay €5…, , €, €2, , €

Page 40 :
Independence of Random Variables, Earlier we talked about independent events. Actually the concept, of independence extends naturally to random variables, and plays, a crucial role both in probability and statistics., , Definition: Independent Random Variables, , Definition: Independence of Multiple Random Variables, , 23

Page 41 :
Independence of Random Variables, Example: Let X denote the number of students in the classroom, today and Y denote the outcome of flipping a coin., Are X and Y independent?, Example: Let X be defined as before, and Y be a random, variable taking the value 1 if today is sunny and zero otherwise., , Are X and Y independent?, Example: Let X be defined as before, and Y be number of, female students in the classroom., , Are X and Y independent?

Page 42 :
Properties of Independent R.V.s, Later in the course we’ll show the following result, but we will, start using it right now., , Proposition:, , This result will be very useful to us, since many common, distributions can be described as the sum of independent, random variables!!

Page 43 :
Multiple Random Variables, Sometimes we must deal with multiple random variables, simultaneously, Example: Let X and Y denote the blood pressure and heart rate, of a randomly chosen TU/e student (during an exam)., , These two quantities are likely to be related, and describing the, probability distribution of X and Y separately will not capture the, relation between the two…, Therefore we need a joint description of the distribution. Let’s, focus first on discrete random variables.

Page 44 :
Joint Probability Mass Functions, Theorem 3: Joint Probability Mass Function, , This is quite similar to what we had before, but now we are, jointly describing the two random variables., Let’s see an example…

Page 50 :
Example, Suppose you want to study the relation between the number of, bars in your mobile, and the quality of the call. You collect data, over time and come up with the following stochastic model:, , 1, , 2, , 3, , 4, , 0, , 0, , 0, , 0, , 0, , 1, , 0.10, , 0.05, , 0, , 0, , 2, , 0.05, , 0.10, , 0.04, , 0.01, , 3, , 0, , 0.05, , 0.15, , 0.15, , 4, , 0, , 0, , 0.05, , 0.25, , It’s easy to check that this table describes a valid joint, probability mass function

Page 51 :
Conditional Probability Distribution, What information does one variable carry about the other?, In our example we might be interested to know about the call, quality when we have 4 bars…, , Definition: Conditional Probability Mass Function, , 7

Page 52 :
Example, , 1, , 2, , 3, , 4, , 0, , 0, , 0, , 0, , 0, , 0, , 1, , 0.10, , 0.05, , 0, , 0, , 0.15, , 2, , 0.05, , 0.10, , 0.04, , 0.01, , 0.20, , 3, , 0, , 0.05, , 0.15, , 0.15, , 0.35, , 4, , 0, , 0, , 0.05, , 0.25, , 0.30, , 0.15, , 0.20, , 0.24, , 0.41, , 8

Page 53 :
Properties, Note that the conditional probability mass function is actually a, proper mass function therefore, , We need to be careful when, avoid that situation., , . In this course we’ll

Page 54 :
Independence of Random V.’s, There are situations were knowing the value of X doesn’t tell, something about Y and vice-versa. This brings up to notion of, independence of random variables, that we used quite frequently, in the course (using the def. in iv) below)., , Definition: Independence of Random Variables, , 10

Page 56 :
Example, , 1, , 2, , 3, , 4, , 0, , 0, , 0, , 0, , 0, , 0, , 1, , 0.10, , 0.05, , 0, , 0, , 0.15, , 2, , 0.05, , 0.10, , 0.04, , 0.01, , 0.20, , 3, , 0, , 0.05, , 0.15, , 0.15, , 0.35, , 4, , 0, , 0, , 0.05, , 0.25, , 0.30, , 0.15, , 0.20, , 0.24, , 0.41

Page 57 :
Means and Variances for Two R.V.’s, If you have two random variables described by a joint, distribution you can always compute the mean, or expected value,, of a function. For simplicity consider only discrete r.v.’s:, , Proposition: Mean of a Function of Two Random Variables, , (this is a consequence of the law of the unconscious statistician), 57

Page 58 :
Means and Variances for Two R.V.’s, This definition seems to make perfect sense. For instance, , We also see that the mean of the sum is the sum of the means!!!, , 58

Page 62 :
Covariance, How about the variance of the sum?, , Definition: Covariance, , 29

Page 63 :
Covariance, Dependence of realizations of 2(or more) different RVs., Covariance is a measure of how much two random variables vary together. It’s similar, to variance, but where variance tells you how a single variable varies, co variance tells, you how two variables vary together., , The covariance is a measure of linear relationship between random, variables. If one of the variables is easy to predict as a linear, function of the other then the covariance is going to be non-zero., Let’s summarize what we just showed in a simple lemma:, , Lemma:, , 30

Page 65 :
Covariance –, It is the relationship between a pair of random variables, where change in one variable causes change in another, variable., It can take any value between -infinity to +infinity, where the, negative value represents the negative relationship whereas a, positive value represents the positive relationship., It is used for the linear relationship between variables., It gives the direction of relationship between variables.

Page 66 :
Properties of Covariance, Cov(X,Y)>0 : A positive covariance means that large values of one, variable are associated with big values from the other (right )., Cov(X,Y)<0 : A negative covariance means that large values of one, variable are associated with small values of the other one (left)., , Cov(X,Y)=0 : Then X and Y may be either independent or not., if X and Y are independent r.vs, then Cov(X,Y)=0

Page 67 :
Correlation Coefficient, It is useful to normalize the covariance, and define the, , Definition: Correlation Coefficient, , 67

Page 69 :
Problems with Interpretation, A large covariance can mean a strong relationship between variables., However, you can’t compare variances over data sets with different scales (like pounds and inches). A weak, covariance in one data set may be a strong one in a different data set with different scales., The main problem with interpretation is that the wide range of results that it takes on makes it hard to, interpret. For example, your data set could return a value of 3, or 3,000. This wide range of values is cause by, a simple fact; The larger the X and Y values, the larger the covariance. A value of 300 tells us that the, variables are correlated, but unlike the correlation coefficient, that number doesn’t tell us exactly how strong, that relationship is. The problem can be fixed by dividing the covariance by the standard deviation to get the, correlation coefficient., Corr(X,Y) = Cov(X,Y) / σXσY, , Advantages of the Correlation Coefficient, The Correlation Coefficient has several advantages over covariance for determining strengths of, relationships:, Covariance can take on practically any number while a correlation is limited: -1 to +1., Because of it’s numerical limitations, correlation is more useful for determining how strong the relationship is, between the two variables., Correlation does not have units. Covariance always has units, Correlation isn’t affected by changes in the center (i.e. mean) or scale of the variables

Page 70 :
Covariance versus Correlation –, Covariance is a measure of, how much two random, variables vary together, , Correlation is a statistical, measure that indicates how, strongly two variables are, related., , involve the relationship, between two variables or, data sets, , involve the relationship, between multiple variables as, well, , Lie between -infinity and, +infinity, , Lie between -1 and +1, , Measure of correlation, , Scaled version of covariance, , provide direction of, relationship, , provide direction and, strength of relationship, , dependent on scale of, variable, , independent on scale of, variable, , have dimensions, , dimensionless

Page 71 :
Example, Recall our previous example, , 1, , 2, , 3, , 4, , 0, , 0, , 0, , 0, , 0, , 0, , 1, , 0.10, , 0.05, , 0, , 0, , 0.15, , 2, , 0.05, , 0.10, , 0.04, , 0.01, , 0.20, , 3, , 0, , 0.05, , 0.15, , 0.15, , 0.35, , 4, , 0, , 0, , 0.05, , 0.25, , 0.30, , 0.15, , 0.20, , 0.24, , 0.41, 71

Page 72 :
Example, , 0, 1, 2, 3, 4, , 1, , 2, , 3, , 4, , 0, 0.10, 0.05, 0, 0, , 0, 0.05, 0.10, 0.05, 0, , 0, 0, 0.04, 0.15, 0.05, , 0, 0, 0.01, 0.15, 0.25, , 0.15, , 0.20, , 0.24, , 0.41, , 0, 0.15, 0.20, 0.35, 0.30, , 72

Page 73 :
Example, , 0, 1, 2, 3, 4, , 1, , 2, , 3, , 4, , 0, 0.10, 0.05, 0, 0, , 0, 0.05, 0.10, 0.05, 0, , 0, 0, 0.04, 0.15, 0.05, , 0, 0, 0.01, 0.15, 0.25, , 0.15, , 0.20, , 0.24, , 0.41, , 0, 0.15, 0.20, 0.35, 0.30, , As the correlation is significantly different than zero this, means that Y can be well-predicted from X using a linear, relationship…, , 73

Page 74 :
Correlation of Independent R.V’s, Proposition:, , 74

Page 75 :
Uncorrelation IS NOT EQUIVALENT to Independence, It’s important to note that the implication goes only in one, direction:, , 75

Page 76 :
Specific Discrete Distributions, In what follows we are going to study and learn about some, specific discrete distributions, namely:, , • Bernoulli Distribution, • Binomial Distribution, • Geometric and Negative Binomial Distributions, • Hypergeometric Distribution, • Poisson Distribution, All these are very important in a number of scenarios, fitting, many practical situations one might encounter.