jensen inequality

Secant line of a convex function lies above the function.

In the context of probability, and especially entropy, it has an extension related to expectation.

f(ax1+(1a)x2)a(f(x1))+(1a)f(x2)
p(E[X])E[p(X)]

And the difference is known as the Jensen gap.

Back to index