![]() ![]() Mathematical Explanation:įeel free to skip this, if not interested. To convey all states the coin can take i.e. When everything is equally likely (at probability 1/2), the entropy is highest because you don’t know what’s going to happen. In the case of a coin, thus, the maximum entropy will be log₂2 = 1 bit. In the general case, the maximum value of entropy for any system is log₂c. We see that Entropy becomes maximum at probability 1/2. How? The numerator of a probability value tells the number of times the event happens, that’s how.įor the next part, let us look at the formula of entropy in a new light: The pᵢ part of the Entropy’s formula tells us about the importance of the event in the whole system. How do you evaluate a part of the system for the randomness it will contribute to the system? We need to evaluate the parts of the system and see their effect in summation. In order for us to get a sense of the whole system, we need to come up with a way that tells us a global picture of the whole system. Probability gives a local picture of the whole system ![]() Thus, what probability can give us is a local picture, a limited picture of the whole system. The definition of probability talks about a single event, not the whole system. What is probability? The likelihood of an event to occur. Let’s revisit the definition of probability. There is one pertinent thing to notice, the word ‘system’ in the layman definition and the occurrence of probability term two times though differently. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |