Useful identities and inequalities in information theory are derived and. Entropy and information theory stanford ee stanford university. The lemma follows easily from the elementary inequality for real numbers ln x. In the years since the first edition of the book, information theory. We shall often use the shorthand pdf for the probability density func. Inequality matters stanford graduate school of education. Inequality and economic growth joseph stiglitz introduction in the middle of the twentieth century, it came to be believed that a rising tide lifts all boats. At the time, there was some evidence behind that claim.
The notion of entropy, which is fundamental to the whole topic of. This upper bound is tight if and only if x is distributed uniformly on x. This book is devoted to the theory of probabilistic information measures and. The first part of the monograph introduces classical concentration inequalities for. Concentration of measure inequalities in information theory. Information theory, pattern recognition, and neural. Gibbs inequality is probably the most important inequality in this book. Gender inequality united nations development programme. Foundations and trends in communications and information theory, vol. Cover and thomas wrote a book on information theory 49 ten years ago. Regardless of ones socioeconomic class, there are systematic gender differences in material wellbeing, although the degree of inequality varies across countries and over time. The notion of entropy, which is fundamental to the whole topic of this book, is introduced here.
Indeed, various aspects of social inequality race, gender, class, sexual orientation, and immigrant statushave been the subject of protest. In particular, if xk has probability density function pdf p, then hxk elog 1 pxk. Grant foundation 2014 inequality matters 1 talk of inequality, particularly economic inequality, in the public sphere is commonplace in twentyfirst century america. Indepth treatment of these concepts and information theory in general can be gathered from many of the excellent text books in this subject, most notably, 3,4,5.
198 655 1195 1579 601 502 67 476 1263 1530 1036 1118 78 159 497 91 173 1021 232 889 1400 448 510 38 338 1556 380 932 972 1361 1439 727 244 253 1252 1126 85 1032 284