Introduction to Probability - The Science of Uncertainty

Probability models and axioms

• Quantitative description: it is a description in terms of numbers

• A probabilistic model is a quantitative description of a situation, a phenomenon, or an experiment whose outcome is uncertain.

• Creating a model involves the following steps:
  1. Define the sample space: Describe the possible outcomes of the experiment (E.g. flip of a coin, roll a dice, etc.), composed of
     • Sample spaces are sets which can be discrete, finite, infinite, continuous (recorded with infinite precision)...
     • List (set) of the possible outcomes ($\Omega$), the elements of the set must be:
       • Mutually Exclusive: At end of experiment there can only be one of the outcomes of the set
       • Collectively Exhaustive: Together all the elements of the set exhaust all the possibilities, at the end of the experiment you should be able to select one from the set
       • At the 'right' Granularity includes only the different outcomes in what we are considering relevant and not different in irrelevant outcomes. E.g. in the flip a coin example the sample space (set) would be Heads=0 or Tails=1, instead of Heads & Rain=0, Heads & No Rain=1, Tails & Rain=2, Tails & No Rain=3, whether is raining it's irrelevant, we only include what is relevant
  2. Specify a probability law which assigns probabilities to outcomes or to collection of outcomes, the probability law specifies which outcomes are more likely to occur and which are less likely
     • Probability is usually assigned to events (an event is a subset of the sample space). We assign probabilities to the various subsets of the sample space.
     • Probabilities are given between 0 (practically cannot happen) and 1 (practically certain event will happen)
     • Probabilities have certain axioms or basic properties (axioms: unprovable rule or principle accepted as true) for example probabilities cannot be negative.
     • Axioms of probability:
       • Non-negativity: Probabilities are positive $P(A) \ge 0$
       • Normalization: We are certain event Omega will occur $P(\Omega) = 1$
       • Additivity The probability that the outcome of the experiments falls in Event A or B is equal to the sum of the probabilities of these two sets.

• There are probabilistic experiments that can be described in stages (E.g two rolls of a tetrahedral die), these type of experiments can be represented using a sequential tree or description

Properties of Probabilities

• if $A$ is subset of $B$ the probability that event $B$ occurs ($P(B)$) must be greater or equal than the probability that event $A$ occurs

$$A \subset B \Rightarrow P(A) \le P(B)$$

$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$

• Discrete Uniform Probability Law
  • A finite sample space consist of $n$ equally likely elements
  • The probability that event $A$ occurs which can be set of $k$ outcomes is $P(A) = \dfrac{k}{n}$

• Probability calculations steps:
  • Specify sample space (values of the outcomes)
  • Specify probability law
  • Identify the probability of an event of interest
  • Calculate

References

• Introduction to Probability - The Science of Uncertainty edX Course