Basics of probability (I)

Reference (of all figures and content): Introduction to Probability, 2nd edition [Blitzstein and Hwang, 2019]

Photo: Unsplash

Probability

Probability, by definition, is the number of desired outcomes (the event) divided by the number of all possible outcomes (all possible events).

Let’s assume the probability with the naïve definition:

Figure 1

Example:
Event: Flip a coin for two times and the probability of one Head and one Tail
All possible outcomes: {H,H}, {T,H}, {H,T}, {T,T} → |S| = 4
Desired outcomes: {T,H}, {H,T} →|A| =2
Probability of the event: 2/4 = 0.5

Sample space and events

The above example has two highlights: event (A) and sample space (S).

Figure 2

Note that
1) “An event is a subset of the sample space”, and that
2) the probability of the event is the result of the size of the event divided by the size of the sample space.

Set theory is very helpful in probability

Set theory

A set is a collection of “objects” — be it numbers, people, or other sets

Example:
•{1,3,5,7,…}: the set of all odd numbers
•R: the set of all real numbers
•{(x,y)∈R²:x²+y²≤1}: a set can be described with conditions; x and y which are the subsets of the square of the real numbers ”such that” x²+y²≤1

Empty set

Definition: The smallest set; the set with no elements.
Notation: ∅ or {}
p.s. {∅} ≠{}: the former has an empty set as the element

Subsets

Definition: If A and B are sets, then A is a subset of B if every element of A is also an element of B.
Notation: A ⊆ B

It is always true that ∅ and A are subsets of A.
To prove that A = B, prove that each is a subset of the other

Unions, intersection, and complements

This principle allows the creation of new sets

Definition: For sets A and B
•Unions: the set of all objects that are in A or B
•Intersections: the set of all objects that are in A and B
•Complements: the complement of set A à the set of objects that that are not in A

Notation:
•Unions: A ∪ B
•Intersections: A ∩ B
•Complements: A^c

Figure 3

Some other properties:
•A and B are disjoint if A ∩ B = ∅
•De Morgan’s law

Figure 4

Partitions

Definition: a partition of a set is a collection of disjoint subsets whose union is the entire set
Notation: A collection of subsets A_1,A_2, A_3, …, A_n of a set S is a partition of S if A_1∪A_2∪A_3∪… ∪ A_n = S and A_i∩A_j = ∅ for all i ≠j.

Cardinality

Definition: the size or cardinality of a set
Notation: |A| if A is finite

The inclusion-exclusion theorem
|A ∪ B| = |A| + |B| — |A ∩ B|

Two sets A and B are said to have the same cardinality if they can be put into one-to-one correspondence à holds true for both finite and infinite

Figure 5

An infinite set is countably infinite if it has the same cardinality as the set of all positive integers (one-to-one applies)
→ A set is countable if finite or countably infinite
→ Not all infinite sets are countable or have the same size!

Python practice for “Unions, intersection, and complements”

We flip a coin for 3 times → the number of possible outcomes: 2³ = 8
We encode Head as 1 and Tail as 0.

S = [[0,0,0], [0,0,1], [0,1,0], [1,0,0], [0,1,1], [1,0,1], [1,1,0], [1,1,1]]

Question 1: find at least one Head

A = [] ## create an empty set for event
for i in S:
    if i.count(1) >=1:
        A.append(i)
print("the cardinality of the event A:", len(A))
print("event A", A)

Question 2: find at least two consecutive Heads

A = [] ## create an empty set for event
for i in S:
    for p in i:
        if i[p] == 1 and i[p+1] == 1:
            A.append(i)
print("the cardinality of the event A:", len(A))
print("event A", A)