Page 2 :
Introduction, Why Probability?, • Many systems are very complex (e.g. an operating system), • It is not easy to model complex systems, • Sometimes we cannot predict how users are going to behave, , 2
Page 3 :
6
Page 4 :
Introduction, Why Probability?, • Many systems are very complex (e.g. an operating system), • It is not easy to model complex systems, • Sometimes we cannot predict how users are going to behave, THE OUTCOME OF SOME EXPERIMENTS, IS “UNPREDICTIBLE” !!!, What can we do with probabilistic models?, •Use randomness to model various events that can, happen., •Use stochastic models to describe how users/clients, interact with a system (e.g. TCP/IP, queuing models), • Quantify the reliability of a system, , 7
Page 5 :
Relation to Statistics, Probability and Statistics are NOT the same thing!!!, • Probability provides the foundation of statistics, •Statistics is concerned with making inference from data,, assumed to be collected according to a probabilistic model, Probabilistic Model, , Sample, , Population (World), , Statistics – Inference about the, population using the sample, , 8
Page 7 :
Sample Spaces and Events, Definition: Random Experiment, , Examples:, • Time to reboot a server (in seconds), • Measuring the clock frequency in a motherboard (in GHz), • Age (in terms of number of birthdays) of a person, • Measuring the temperature evolution in a GPU while, performing a benchmark task (in deg. Celsius), , 7
Page 8 :
Sample Space, Definition: Sample Space, , Examples:, • Time to reboot a server (in seconds), • Measuring the clock frequency in a motherboard (in GHz), • Age (in terms of number of birthdays) of a person, • Measuring the temperature evolution in a GPU while, performing a benchmark task (in deg. Celsius), , 8
Page 9 :
Discrete and Continuous Sample Spaces, , Examples:, • Time to reboot a server, • Measuring the clock frequency in a motherboard, • Age (in terms of number of birthdays) of a person, •Measuring the temperature evolution in a GPU while, performing a benchmark task, 9
Page 10 :
More Examples, The choice between discrete or continuous sample spaces, might depend on the particular objective of the study…, • Messaging in a communication system:, •Three time sensitive messages are sent. These will either, be on time, or delayed, What is the sample space?, , 10
Page 11 :
Events, Definition: Events, , 11
Page 12 :
Events, Example:, , Events are just sets contained in the sample space !!!, , 12
Page 13 :
Set Operations, , Definition: Mutually Exclusive Events (Disjoint Events), , 18
Page 14 :
Venn Diagrams, , 19
Page 15 :
Counting Techniques, Sometimes we need to count the number of possible outcomes., So far we seen only easy cases, but things can get complicated, pretty quickly…, Example 1: Options in a car, • Manual or automatic transmission, • With or without air conditioning, • Three different stereo systems, • Four possible exterior colors, , How many possibilities are there?, • A - 11, • B - 32, • C - 48, • D - 54, , 20
Page 16 :
Permutations, Example: 4 friends (Alice, Bob, Eve and Robert) place 4 pieces, of paper with their names in a hat. After shuffling them each, one of them takes one piece., What are the possible outcomes of this experiment?, , In this case there are 4x3x2x1=24 possibilities, Sinterklaas Challenge: If you have n friends how many, possible outcomes exist such that no one takes their own name, out of the hat???, (if n=4 the answer is 9), , 21
Page 17 :
Permutations, Example: A motherboard has 8 slots, where we want to place 4, different cards. How many possibilities are there?, , Why?, •Take card one and place it in one of the 8 positions (8, possibilities)., • Take card 2 and place it in one of the 7 remaining positions, • ... and so on, , 17
Page 18 :
Combinations, Example: A motherboard has 8 memory slots, and we want to, put 4 identical memory cards in it., , Why?, •Same as before, but now the order of the cards doesn’t, matter. So, there are 4! possible ordering of the 4 cards all, yield the same outcome., , 18
Page 19 :
Sampling With or Without Replacement, Example: I ask you to take five cards out of a deck, one by one, without putting them back (sampling without replacement), Example: I ask you to take a card out of the deck and look at, it, then replace it, I shuffle the deck and ask you to take a, card again, and so on, five times (sampling with replacement), , Which type of sampling is more adequate depends on the, context (but typically sampling with replacement is easier, to study), 19
Page 20 :
Interpretation of Probability, The concept of probability is a very abstract one:, •Idea: quantify the “likelihood” or “chance” of the outcome, of an experiment…, • One can have a subjective interpretation of probability:, , This might not be very satisfying, as different, people will assign different values for the, probability of an event…, , 20
Page 21 :
Axioms of Probability, We will restrict ourselves to discrete sample spaces for now, to, avoid some technical difficulties…, By taking an axiomatic approach we can make sure everything is, well defined, if we take the axioms for granted…, , 21
Page 22 :
Axioms of Probability, Example:, , Random experiment – Flipping a coin, , If we have a fair coin then, In general, if you have N possible outcomes:, , 22
Page 23 :
Basic Properties, Just from the axioms we can deduce a number of simple, but, useful properties:, , Lemma:, , 23
Page 24 :
Addition Rules, Actually, the probability of an event is just a way to measure it !!, - you can think of it as the generalized volume of the set., , Lemma:, If the events are mutually exclusive then (and only then!):, 24
Page 25 :
Proof:, , We can obviously generalize all this to multiple events…, , 25
Page 26 :
Conditional Probability, This is a very important concept. In most cases events “share”, information, and so we want to see how can we take this into, account, , Example: taking a card out of a shuffled deck, , 26
Page 27 :
Conditional Probability, Definition: Conditional Probability, , 27
Page 28 :
An Illustrative Toyish Example, Sammy has only four types of pairs of socks in his, drawer:, • Red or Blue socks, • Striped or plain, Striped \ Plain, Red, , 13, , 16, , Blue, , 5, , 8, , 28
Page 29 :
An Illustrative Toyish Example, Sammy has only four types of pairs of socks in his, drawer:, • Red or Blue socks, • Striped or plain, Striped \ Plain, Red, , 13, , 16, , 42, , Striped, , 29, , Plain, 24, , 18, , Blue, , 5, , 8, , 13, , 18, , 24, , 42, , Red, 13, , Blue, 5, , Red, , Blue, , 16, , 8, , 29
Page 30 :
An Illustrative Toyish Example, Sammy takes a pair of socks from his drawer at random (he, is equally likely to take any of the pairs):, • What is the probability that it is a red pair?, •Given he took a pair of red socks, what is the probability, these have stripes?, , Striped \ Plain, Red, , 13, , 16, , 29, , Blue, , 5, , 8, , 13, , 18, , 24, , 42, , 36
Page 31 :
An Illustrative Toyish Example, , 42, , Striped, , Plain, 24, , 18, , Red, 13, , Blue, 5, , Red, , Blue, , 16, , 8, , Striped, 18, , Red, 13, , Blue, 5, 31
Page 32 :
Multiplication Rules, Multiplication Rule:, , Total Probability Rule:, , 32
Page 34 :
Total Probability Rule for Multiple Events, , 34
Page 35 :
Example, In a class with 3/5 women and 2/5 men, 25% of the women are business majors., Find the probability that a student chosen from the class at random is a female, business major., , Define the relevant events: W = the student is a woman, B = the student is a business major, , Express the given information and question in probability notation:, “class with 3/5 women” P ( W ) = 3/ 5 = 0.60, “25% of the women are business majors” is the same as saying “the probability a, student is a business major, given the student is a woman is 0.25” P ( B W ) = 0.25, “probability that a student chosen from the class at random is a female business, major” is the same as saying “probability student is a woman and a business, major” P (W and B ), Use the multiplication rule to answer the question:, P (W and B ) = P (W ) P ( B W ) = ( 0.60 )( 0.25) = 0.15
Page 36 :
Example, A box contains 5 red balls and 9 green balls. Two balls are drawn in, succession without replacement. That is, the first ball is selected and, its color is noted but it is not replaced, then a second ball is selected., What is the probability that:, a., b., c., d., , the first ball is green and the second ball is green?, the first ball is green and the second ball is red?, the first ball is red and the second ball is green?, the first ball is red and the second ball is red?
Page 37 :
Decision tree
Page 38 :
Solution, Using the decision tree, we see that:, a. the probability the first ball is green and the second ball is green =, P ( G1 and G 2 ) =, , 36, 91, , b. the probability the first ball is green and the second ball is red =, P ( G1 and R2 ) =, , 45, 182, , c. the probability the first ball is red and the second ball is green =, 45, P ( R1 and G 2 ) =, 182, 10, d. the probability the first ball is red and the second ball is red = P ( R1 and R2 ) =, 91
Page 39 :
Independence of Events, Sometimes two events might not be “related”:, •Given that the outcome of the experiment is in A might not, affect the probability that this same outcome is also in B…, , Example: Take a card out of a shuffled deck, , This means that B doesn’t give any probabilistic, information about A !!!, , 40
Page 40 :
Definition of Independence, Definition: Independence, , Independence is a truly probabilistic concept, and not a set, relationship !!!, , Challenge: Can you construct a probability model and three events, 41, that are pairwise independent but not jointly independent?
Page 41 :
Properties, Lemma:, , Proof: left as exercise…, , 42
Page 42 :
Independence is IMPORTANT !!!, Independence is a key assumption in most probability models,, e.g.:, •We assume each memory chip from a certain manufacturer, fails independently from one another, •Processes arriving to a server from different users are, assume to arrive independently from one another, Example: There are three elevators in the MetaForum, but one is, currently broken. The other two fail independently with probability, 0.2 and 0.1 respectively., What is the probability I cannot take one of the elevators to get, to the 7th floor?, , A – 0.1, , B – 0.2, , C – 0.3, , D – 0.02, , 43
Page 43 :
A Simple Example, We can think of this setting as a circuit:, Elevator 1, Elevator 2, , 43
Page 44 :
Bayes’ Rule, In many situations in practice what we can measure (estimate) are, conditional probabilities. It is rather useful to be able to relate, various conditional probabilities to one another., Reverend Thomas Bayes (1702-1761):, , Despite it’s simplicity, this is a very powerful, result, and forms the basis of many inference, procedures used nowadays (e.g. both the, GPS system and the communication encoding, used in cellphones rely on Bayes’ rule)., , 46
Page 45 :
Bayes’ Rule, , 47
Page 46 :
A Real Life Example, The probability that one of these women has breast cancer is 0.8, percent. If a woman has breast cancer, the probability is 90, percent that she will have a positive mammogram. If a woman does, not have breast cancer, the probability is 7 percent that she will, still have a positive mammogram., Imagine a woman who has a positive mammogram. What is the, probability that she actually has breast cancer?, The doctors answer for this question ranged from 1 to 99%., What is the correct answer ???, Enter your answer as a percentage, (e.g. 47% corresponds to 47), 46
Page 47 :
A Real Life Example, The probability that one of these women has breast cancer is 0.8, percent. If a woman has breast cancer, the probability is 90, percent that she will have a positive mammogram. If a woman does, not have breast cancer, the probability is 7 percent that she will, still have a positive mammogram., Imagine a woman who has a positive mammogram. What is the, probability that she actually has breast cancer?, , 47
Page 48 :
A Real Life Example, , So a woman between 40 and 50 with a positive, mammogram has only about 9.5% chance of actually, having breast cancer…, , 51
Page 49 :
Example, • Suppose 50% of the people in a particular population have high blood, pressure. Of the people with high blood pressure, suppose 75% smoke;, whereas, only 50% of the people without high blood pressure actually, smoke., What percent of the people who smoke have high blood pressure?, • Define Events & Express in probability, • P(high BP) = 0.5, • P(smoke | high BP) = 0.75, • P(smoke | without high BP) = 0.5, • Find P(high BP | smoke) ?
Page 50 :
Solution, Using Bayes Rule
Page 51 :
Example, An automobile dealer has kept records on the customers who visited his, showroom. Forty percent of the people who visited his dealership were, women. Furthermore, his records show that 37% of the women who, visited his dealership purchased an automobile, while 21% of the men who, visited his dealership purchased an automobile., a. What is the probability that a customer entering the showroom will, buy an automobile?, b. Suppose a customer visited the showroom and purchased a car., What is the probability that the customer was a woman?, c. Suppose a customer visited the showroom but did not purchase a, car. What is the probability that the customer was a man?
Page 52 :
Solution, Define the events:, , Express the given information and question in probability notation:, “Forty percent of the people who visited his dealership were women”, P ( A1 ) = 0.40, , this statement also tells us that 60% of the customers must be men P ( A2 ) = 0.60, “37% of the women who visited his dealership purchased an automobile”, P ( B A1 ) = 0.37, , “21% of the men who visited his dealership purchased an automobile”, P ( B A2 ) = 0.21, , “What is the probability that a customer entering the showroom will buy an, automobile?” P ( B ) = ?
Page 53 :
Create a tree diagram:
Page 54 :
Solution to part b:, “Suppose a customer visited the showroom and purchased a car. What is the, probability that the customer was a woman?”, Express the question in probability notation:, We can rewrite the question as, “What is the probability that the customer was a, woman, given that the customer purchased an automobile.” That is, we want to, find P ( A1 B ), We can use Bayes’ Theorem to help us compute this conditional probability., Bayes’ Theorem (Two-Event Case):, , P ( A1 B ) =, , P ( A1 ) P ( B A1 ), , P ( A1 ) P ( B A1 ) + P ( A2 ) P ( B A2 ), , =, , P ( A1 and B ), P ( A1 and B ), =, P ( A1 and B ) + P ( A2 and B ), P ( B), , where A1 and A2 are mutually exclusive events with P ( A1 ) + P ( A2 ) = 1, and B is any event with P ( B ) 0, , Note: the denominator is determined by the Law of Total Probability
Page 55 :
Solution to part b (continued):, Use Bayes’ Theorem and your tree diagram to answer the question:, , P ( A1 B ) =, , P ( A1 ) P ( B A1 ), , P ( A1 ) P ( B A2 ) + P ( A2 ) P ( B A2 ), , =, , P ( A1 and B ), 0.148, 0.148, =, =, 0.540, P ( A1 and B ) + P ( A2 and B ) 0.148 + 0.126 0.274, , The probabilities needed for the computation are easily obtained from our tree, diagram., We already found P ( A1 and B ) + P ( A2 and B ) , which is P ( B ) , for part a.) of this, example and P ( A1 and B ) is obtained by following the tree diagram path → A1 → B ,, the product of the corresponding probabilities is 0.148., Solution to part c:, “Suppose a customer visited the showroom but did not purchase a car. What is the, probability that the customer was a man?
Page 56 :
Solution to part c (continued):, Express the question in probability notation:, We can rewrite the question as, “What is the probability that the customer was a, man, given that the customer did not purchase an automobile.” That is, we want to, find P ( A2 BC ), Use Bayes’ Theorem and your tree diagram to answer the question:, , (, , ), , P A2 BC =, , P ( A2 ) P ( BC A2 ), , P ( A2 ) P ( BC A2 ) + P ( A1 ) P ( BC A1 ), , =, , 0.474, 0.474, =, 0.653, 0.474 + 0.252 0.726, , Again, the probabilities needed for the computation are easily obtained from our, tree diagram.
Page 57 :
Additional Notes:, The probabilities P ( A1 ) and P ( A2 ) are called prior probabilities because they are, initial or prior probability estimates for specific events of interest. When we, obtain new information about the events we can update the prior probability values, by calculating revised probabilities, referred to as posterior probabilities. The, conditional probabilities P ( A1 B ) , P ( A2 B ) , P ( A1 BC ) , and P ( A2 BC ) are posterior, probabilities. Bayes’ Theorem enables us to compute these posterior probabilities.
Page 59 :
A Paradoxical Example, Example: Real experimental data for two different treatments of, kidney stones is shown below. Their success was studied when, patients had both small and large stones in four groups of patients:, , Treatment A, , Treatment B, , Successful, , Stone, size, , Yes, , No, , Small, , 81, , 6, , Large, , 192, , 71, , Successful, , Stone, size, , Yes, , No, , Small, , 234, , 36, , Large, , 55, , 25, , Which treatment is better?, , 52
Page 60 :
Successful, , Treatment A, Stone, size, , Yes, , No, , Small, , 81, , 6, , Large, , 192, , 71, , Let’s compute the overall success probability of treatment A:, , Let’s also compute conditional success probabilities:, , 60
Page 61 :
Successful, , Treatment B, Stone, size, , Yes, , No, , Small, , 234, , 36, , Large, , 55, , 25, , Let’s compute the overall success probability of treatment B:, , Let’s also compute conditional success probabilities:, , 55 / 80= 0.68, 61
Page 62 :
A Paradoxical Example, In terms of the overall success rate we have, , So treatment B seems to be better overall…, But, For each class of patients we have, , 55 / 80= 0.68, , So for patients with small stones treatment A is better, and for, patients with large stones treatment A is also better…, , Where’s the catch?, , 55
Page 63 :
Successful, , The Two Studies, Few patients with small Many patients with, large stones, stones, , Stone, size, , Yes, , No, , Small, , 81, , 6, , Large, , 192, , 71, , This is known as Simpson’s paradox, and continues to, confuse many people, even nowadays…, Successful, , Stone, size, , Yes, , No, , Small, , 234, , 36, , Large, , 55, , 25, , Few patients with large, Many patients with stones, small stones, , 56
Page 64 :
Random Variables, Often it is useful to consider random variables. These map the, outcomes of a random experiment to a number., , Example:, •X is the random variable corresponding to the temperature of, the room at time t., • x is the measured temperature of the room at time t., 64
Page 65 :
Random Variables, , Next in the course:, •We are going to study both discrete and continuous random, variables, and learn about probabilistic settings that are useful, to model real-world situations…, , 65
