A scatter plot does not come with a line attached. Ordinary least squares chooses the line that minimizes the sum of squared residuals. Each residual is a vertical distance: the observed outcome minus the prediction for that same value of xx.

From distances to a fitting criterion

In yi=β0+β1xi+εiy_i=\beta_0+\beta_1x_i+\varepsilon_i, predictions are y^i=β^0+β^1xi\hat y_i=\hat\beta_0+\hat\beta_1x_i and residuals are ei=yi−y^ie_i=y_i-\hat y_i. Provided xx varies, the solution is:

β^1=∑i(xi−xˉ)(yi−yˉ)∑i(xi−xˉ)2,β^0=yˉ−β^1xˉ.\begin{gathered} \hat\beta_1=\frac{\sum_i(x_i-\bar x)(y_i-\bar y)}{\sum_i(x_i-\bar x)^2},\\[0.5em] \hat\beta_0=\bar y-\hat\beta_1\bar x. \end{gathered}

Squaring prevents positive and negative errors from cancelling, while giving large deviations more influence. The drawn squares visualize this penalty; some may be clipped by the plot boundary.

Draw a sample. Fit a line.Synthetic data · n ≤ 250

Click inside the plot or enter coordinates below. Salary is in arbitrary units; the example is not a real earnings estimate.

SSR manual / current line
—
SSR minimum (OLS)
—
Excess above the minimum
—

Each witness adds one point to the current sample and activates OLS. Reload the sample between the two experiments for a fair comparison. The sample concentrates x between 8 and 12 to make leverage at the edge visible.

n
0
β̂₀
—
β̂₁
—
R²
—
Residual standard error
—

Residuals, squares and the table follow the current line. Coefficients with hats, R² and residual standard error refer to the OLS minimum. That minimum changes when the sample changes.

Observations and current-line residuals
xᵢyᵢb₀ + b₁xᵢeᵢeᵢ²

Try a line before minimizing

The sliders choose an arbitrary intercept b0b_0 and slope b1b_1. The line, residuals, squares and table follow those values while the counter recomputes:

SSRmanual(b0,b1)=∑i[yi−(b0+b1xi)]2.\mathrm{SSR}_{\mathrm{manual}}(b_0,b_1)=\sum_i\left[y_i-(b_0+b_1x_i)\right]^2.

Try lowering it by hand, then press Fit with OLS (Minimum). The button moves to the analytical coefficients without rounding them to the displayed values. Excess above the minimum becomes zero. This is not a local minimum found by trial and error: no other straight line can achieve a lower SSR for that same sample. Writing Sxx=∑i(xi−xˉ)2S_{xx}=\sum_i(x_i-\bar x)^2, the reason is:

SSR(b0,b1)=SSROLS+n(b0+b1xˉ−yˉ)2+Sxx(b1−β^1)2.\begin{aligned} \mathrm{SSR}(b_0,b_1)={}&\mathrm{SSR}_{\mathrm{OLS}}\\ &+n(b_0+b_1\bar x-\bar y)^2\\ &+S_{xx}(b_1-\hat\beta_1)^2. \end{aligned}

The minimum is unique when Sxx>0S_{xx}>0. After fitting, adding observations keeps OLS active; moving either slider switches back to manual mode. The minimum always belongs to the current sample: comparing SSR across different sample sizes is not the same optimization problem.

A large residual is not the same as high influence

Reload the sample before each witness experiment for a fair comparison. Sample education values are concentrated between 8 and 12 years to make the contrast visible:

The panel explicitly distinguishes residuals before and after refitting. A point exactly on the old line would not move it, however extreme its x value: high leverage means potential influence, not guaranteed influence. Leverage is calculated in the augmented sample:

hi=1n+(xi−xˉ)2Sxx.h_i=\frac{1}{n}+\frac{(x_i-\bar x)^2}{S_{xx}}.

The dashed teal line preserves the previous OLS fit and the outlined point identifies the added observation. Diagnostics compare OLS before and after the addition, even if you subsequently explore another line manually.

What does a good fit mean?

With an intercept, define SSR=∑iei2\mathrm{SSR}=\sum_i e_i^2 and SST=∑i(yi−yˉ)2\mathrm{SST}=\sum_i(y_i-\bar y)^2. SSR here means the residual sum of squares, not the explained sum:

R2=1−SSRSST,σ^=SSRn−2.\begin{gathered} R^2=1-\frac{\mathrm{SSR}}{\mathrm{SST}},\\[0.5em] \hat\sigma=\sqrt{\frac{\mathrm{SSR}}{n-2}}. \end{gathered}

The panel reports the residual standard error, in outcome units, rather than the standard error of the estimated slope. Two points leave no residual degrees of freedom; constant outcomes make R2R^2 undefined; constant xx values leave the slope unidentified. Those cases display a dash.

Geometry for fitting, assumptions for inference

Computing the OLS line is an algebraic and geometric operation: normality, homoskedasticity and error independence are not required. A unique solution does require variation in xx, or a full-rank design matrix in multiple regression.

Probability assumptions justify estimator properties and inferential procedures. In the classical Gauss–Markov formulation, exogeneity and homoskedastic, uncorrelated errors, together with full rank, make OLS the best linear unbiased estimator. Normality is not required, and uncorrelated errors need not be independent. Normality is added for exact classical finite-sample tt and FF tests and confidence intervals. Asymptotic inference and robust covariance estimators allow other conditions without changing the line that minimizes SSR.

Methodological conclusion. Add an extreme observation and watch the fitted line move. High R2R^2 describes an in-sample fit: it establishes neither causation nor out-of-sample accuracy and does not solve omitted variable bias. The education and salary example is entirely synthetic.

Reference: NIST, least squares and residual standard deviation.

Further reading: Penn State, influential points and leverage and Marco Taboga, Gauss–Markov theorem.