Local Smoothing
Definition
Local smoothing is a family of methods that estimate a regression function by fitting simple models locally around each input value.
The target function is:
\[m(x) = E[Y \mid X=x]\]The estimate is:
\[\hat m(x)\]Main Idea
Instead of fitting one global model to all data, local smoothing fits or averages only the observations near the point of interest.
This allows the fitted curve to adapt to different parts of the data.
Local Average
The simplest local smoother is a local average:
\[\hat m(x) = \frac{1}{|S_x|}\sum_{i \in S_x}y_i\]where $S_x$ is the set of observations near $x$.
Local Linear Regression
A more advanced smoother fits a small linear regression near $x$:
\[y_i \approx a + b(x_i - x)\]The prediction at $x$ is the fitted intercept:
\[\hat m(x) = \hat a\]Why Local Linear Smoothing Is Useful
Local linear smoothing can reduce boundary bias compared with simple local averaging.
It can follow curved patterns while still using simple linear models locally.
Smoothing Parameter
Local smoothing depends on a neighborhood size or bandwidth.
Small neighborhood:
- Less smooth.
- More sensitive to noise.
- Lower bias.
- Higher variance.
Large neighborhood:
- More smooth.
- Less sensitive to noise.
- Higher bias.
- Lower variance.
Basket Size Example
Suppose we plot:
- $X$ = actual final basket size.
- $Y$ = prediction error.
A local smoother can show where the model starts to systematically overpredict or underpredict.
For example, if the smoothed error curve drops below zero for large baskets, the model is underestimating large baskets.
Relation to Kernel Regression
Kernel Regression is a type of local smoothing.
It uses kernel weights to decide how much nearby observations influence the prediction.
Relation to Polynomial Regression
Polynomial regression fits one global curve.
Local smoothing fits many small local curves.
| Method | Model scope | Flexibility |
|---|---|---|
| Polynomial regression | Global | Medium |
| Local smoothing | Local | High |
Strengths
- Reveals nonlinear trends.
- Good for visualization.
- Useful for residual diagnostics.
- Does not require a fixed global model shape.
Weaknesses
- Choice of smoothing parameter matters.
- Less interpretable than a single formula.
- Can behave poorly in sparse regions.
- Mainly useful in low-dimensional settings.
Exercises
- Explain the difference between a global model and a local model.
- Why can local smoothing reveal systematic prediction errors?
- In a residual plot, what does it mean if the local smoother drops far below zero?