21. Probabilistic Inference I

Probabilities in Artificial Intelligence

With a joint probability table, recording the tally of crossed events occurrence will allow us to measure the probabilities of each event happening, conditional or unconditional probabilities, independence of events, etc.

The problem with such table is that as the number of variables increase, the number of rows in the table grows exponentially.

Reminders of probabilities formulas

Basic axioms of probability

  • 0 ≤ P(a) ≤  1
  • P(True) = 1 ; P(False) = 0
  • P(a+b) = P(a) + P(b) – P(a,b)

Basic definitions of probability

  • P(a/b) = P(a,b) / P(b)
  • P(a,b) = P(a/b) P(b)
  • P(a/b,c) = P(a/b,c) P(b,c) = P(a/b,c)P(b/c)P(c)

Chain rule of probability

By generalizing the previous formula, we obtain the following chain rule:

Chain rule of probability

Independence

Independent events

P(a/b) = P(a) if a and b are independent

Conditional independence

If a and b are independent

  • P(a/b+z) = P(a/z)
  • P(a+b/z) = P(a/z)P(b/z)

Belief nets

Causal relations between events can be represented in nets. These models highlight that any event is only dependent from its parents and descendants. Recording the probabilities at each node, the number of table and rows is significantly smaller than a general table of all events tallies.