Least Squares Principle

The Least Squares Principle determines model parameters by minimizing the sum of squared residuals between observed values and model predictions: arg min_theta Σ_i (y_i - f(x_i; theta))^2. It defines a loss function that yields closed-form estimators in linear models and coincides with the maximum-likelihood estimator under i.i.d. Gaussian additive noise.