Entropy of a Message Using Random Variables
Entropy of a Message Using Random Variables
Using the second law of thermodynamics, it is possible to use random variables to calculate the information entropy (or Shannon entropy) of a message, which is a measure of the amount of information in the message. The probabilities that and occur in the message are and .
A
B
P(A)
P(B)
H(X)
X
A
B
P(A)