Skip to main content
Peyman Salimi
فاEN

Probability, likelihood, and Bayes: reasoning under uncertainty

From conditional probability to likelihood and Bayes' theorem, build foundations that recur across statistics and machine learning.

Mathematics for AIIntermediate
Explore more lessons

Learning goal

Distinguish probability from likelihood, write a simple likelihood, and interpret Bayes' theorem with a numerical example.

Prerequisites

  • Basic probability
  • Basic algebra
  • Exponential function

Conditional probability

When B is known, the probability of A is described with conditional probability.

Conditional probability
P(A\mid B)=\frac{P(A\cap B)}{P(B)}

Bayes' theorem

Bayes' theorem
P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}

Bayes provides a way to update an initial belief using observed evidence.

How is likelihood different from probability?

For likelihood, observed data are fixed while the model parameter varies. Likelihood is therefore not the probability of a parameter; it compares how well different parameter values explain the observed data.

Likelihood
L(\theta\mid x)=p(x\mid\theta)
Maximum likelihood
\hat\theta=\arg\max_\theta L(\theta\mid x)

Bernoulli example

Coin likelihood
L(p)=p^k(1-p)^{n-k}
Bernoulli MLE
\hat p=\frac{k}{n}

If a coin is flipped n times and produces k heads, the maximum-likelihood estimate for p is k/n.

Connection to machine learning

Probability and likelihood appear in classification, estimation, and statistical losses. Understanding them helps explain model outputs and the statistical assumptions behind them.

Exercises

  • For k=7 and n=10, compute the Bernoulli MLE.
  • Explain the difference between P(A|B) and P(B|A).
  • Give an example of Bayes updating a belief in diagnosis or decision-making.

Related project

HowDeepLearningWorks

Learning and optimization concepts are developed through a technical learning path.

Open source

Related solutions

AI & Machine LearningModeling, Algorithms & Optimization

Working on an AI problem under uncertainty?

A problem can move from data and statistical assumptions through modeling and implementation.

Start a conversation