Marginalize and condition

To forget a variable, add or integrate it out. To learn its value, slice and renormalize.

For discrete variables, pX(x)=Σ over y of pXY(x,y). For continuous variables, replace the sum by an integral.

Conditioning gives p(x|y)=p(x,y)/pY(y) when the denominator is positive.

The product rule p(x,y)=p(x|y)p(y) factors a joint law in either direction.

A joint table assigns probability to every pair. Summing across a row or column forgets the other coordinate and creates a marginal; selecting one row or column and renormalizing creates a conditional distribution.

Marginalization adds mutually exclusive ways to obtain x. Conditioning divides the desired joint cell by the total mass in the known slice so the new reduced world sums to one.

The product rule rebuilds the joint from a marginal and a conditional. This factorization is the foundation of probability trees, graphical models, sequence models, and Bayesian updates.

Conditional expectation and dependence

E[X|Y] is the best mean-squared prediction of X after seeing Y.

Conditional expectation is itself a random variable: its value changes with Y.

The tower property E[E[X|Y]]=E[X] says that averaging all conditional means restores the overall mean.

The law of total variance splits uncertainty into average remaining noise plus variation in conditional means.

Conditional expectation is a function of the information Y: each possible y gets its own best mean-squared prediction of X. Averaging those predictions over Y returns the unconditional mean by the tower property.

Total variance splits uncertainty into noise remaining inside Y-groups and variation between their conditional means. The pieces answer whether uncertainty comes mainly from unexplained individual variation or predictable group differences.

A demand forecast conditions orders on weekday and weather, averages within each state, then combines states using their forecast probabilities. This works because conditional expectation uses available information optimally under squared error, while omitted events and distribution shift still limit accuracy.

Practical deep dive for this lesson

Use total expectation and variance in a mixture

Condition on a meaningful group, calculate within each group, then combine to reveal both internal noise and differences between groups.

Suppose demand depends on weather. E[X|weather] gives each weather-specific forecast; weighting those means by weather probabilities gives overall expected demand through the tower property.

Total variance adds average within-weather variance and variance among weather-specific means. Even perfectly stable demand inside each weather state can have large overall variance when the state means differ.

This decomposition guides action: better within-state models reduce the first component, while better prediction of the state helps explain the second. Missing states or changing conditional laws still cause forecast error.