Probabilities in Artificial Intelligence
With a joint probability table, recording the tally of crossed events occurrence will allow us to measure the probabilities of each event happening, conditional or unconditional probabilities, independence of events, etc.
The problem with such table is that as the number of variables increase, the number of rows in the table grows exponentially.
Reminders of probabilities formulas
Basic axioms of probability
- 0 ≤ P(a) ≤ 1
- P(True) = 1 ; P(False) = 0
- P(a+b) = P(a) + P(b) – P(a,b)
Basic definitions of probability
- P(a/b) = P(a,b) / P(b)
- P(a,b) = P(a/b) P(b)
- P(a/b,c) = P(a/b,c) P(b,c) = P(a/b,c)P(b/c)P(c)
Chain rule of probability
By generalizing the previous formula, we obtain the following chain rule:
Independence
Independent events
P(a/b) = P(a) if a and b are independent
Conditional independence
If a and b are independent
- P(a/b+z) = P(a/z)
- P(a+b/z) = P(a/z)P(b/z)
Belief nets
Causal relations between events can be represented in nets. These models highlight that any event is only dependent from its parents and descendants. Recording the probabilities at each node, the number of table and rows is significantly smaller than a general table of all events tallies.