Baye's Theorem is named after an 18th-Century Presbyterian minister Thomas Bayes. The theorem tries to understand the probability of a belief or hypothesis in light of new evidence. The theory can be broken down like this: "In the face of uncertainty, a Bayesian asks three questions: How confident am I in the truth of my initial belief? On the assumption that my original belief is true, how confident am I that the new evidence is accurate? And whether or not my original belief is true, how confident am I that the new evidence is accurate?"
Honestly, it kind of feels like the Work of Byron Katie.
Now for the theorem: the posterior probability of a hypothesis is equal to the product of (a) the prior probability of the hypothesis and (b) the conditional probability of the evidence given the hypothesis, divided by (c) the probability of the new evidence.
Basically the theorem helps to answer the question of what does new evidence do to your prior belief? Mathematically, it often changes it. Philosophically - well, that's another matter.