Probability: Defined as a chance, or possibility with which an event occurs.
There are two interpretations based on which the probability of an event can be calculated, let’s look at the two interpretations closely:
Frequentist Probability: In this interpretation, we estimate the probability of an event by observing how many times the event has occured in a large number of repeated, identical trials. For example, a person tosses a coin 100 times and observes Head 53 times, then according to frequentist interpretation, the estimated probability of head will be
$$ \hat{P}(H)=\frac{53}{100}=0.53. $$Bayesian Probability: Unlike frequentist, in this interpretation, we update our estimated probability based on new evidences. For example, suppose a disease affects 1% of the population. Initially, we believe that the probability that a randomly selected person has the disease is:
$$ P(\text{Disease}) = 0.01 $$Now suppose the person tests positive for the disease. This is new evidence, so we update our belief about the probability that the person has the disease:
$$ P(\text{Disease}\mid\text{Positive Test}) $$