Bayes' Theorem: Let A and B be two events such that P(B) > 0. Then we can recall that the conditional probability of A given B is:
or
Hence, if we are given that event B occurred, the relevant sample space is reduced to B {P(B)=1 because we know B is true} and conditional probability becomes a probability measure on B. Therefore,
The probability of A given B = probability of B given A X prior probability of A / prior probability of B. This is Baye’s Rule.
Bayes’ Theorem shows the relationship between a conditional probability and its inverse i.e. it allows us to make an inference from the probability of a hypothesis given the evidence to the probability of that evidence given the hypothesis and vice versa
For multiple events: Given k represents a mutually exclusive and exhaustive event such for B1, B1, . . . Bk, such that P(B1) + P(B2) + … + P(Bk) = 1, where P (B) > 0 and an observed event A, then
Naïve Bayes Classifier Algorithm o Naïve Bayes algorithm is a supervised learning algorithm, which is based on Bayes theorem and used for solving classification problems.
o It is mainly used in text classification that includes a high-dimensional training dataset.
o Some popular examples of Naïve Bayes Algorithm are spam filtration, Sentimental analysis, and classifying articles.
Bayes' Theorem: o Bayes' theorem is also known as Bayes' Rule or Bayes' law, which is used to determine the probability of a hypothesis with prior knowledge. It depends on the conditional probability.
o The formula for Bayes' theorem is given as:
P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B. P(B|A) is Likelihood probability: Probability of the evidence given that the probability of a hypothesis is true. P(A) is Prior Probability: Probability of hypothesis before observing the evidence. P(B) is Marginal Probability
Working of Naïve Bayes' Classifier:
Working of Naïve Bayes' Classifier can be understood with the help of the below example: Suppose we have a dataset of weather conditions and corresponding target variable "Play".
So using this dataset we need to decide that whether we should play or not on a particular day according to the weather conditions. So to solve this problem,
we need to follow the below steps:
- Convert the given dataset into frequency tables.
- Generate Likelihood table by finding the probabilities of given features.
- Now, use Bayes theorem to calculate the posterior probability.
Advantages of Naïve Bayes Classifier: o Naïve Bayes is one of the fast and easy ML algorithms to predict a class of datasets. o It can be used for Binary as well as Multi-class Classifications. o It is the most popular choice for text classification problems.
Disadvantages of Naïve Bayes Classifier: o Naive Bayes assumes that all features are independent or unrelated, so it cannot learn the relationship between features.
Applications of Naïve Bayes Classifier: o It is used for Credit Scoring. o It is used in medical data classification. o It can be used in real-time predictions because Naïve Bayes Classifier is an eager learner. o It is used in Text classification such as Spam filtering and Sentiment analysis.