jensen inequality

Secant line of a convex function lies above the function.

In the context of probability, and especially entropy, it has an extension related to expectation.

\begin{equation} f(ax_1 + (1-a)x_2) \leq a(f(x_1)) + (1 - a)f(x_2) \end{equation}
\begin{equation} p(E[X]) \leq E[p(X)] \\ \end{equation}

And the difference is known as the Jensen gap.

Back to index