Econ 103 Homework 2 Manu Navjeevan August 15, 2022 Single Li

Econ 103 Homework 2 Manu Navjeevan August 15, 2022 Single Linear Regression Theory Review

Recall that we define our parameters of interest β0 and β1 as the parameters governing the “line of best fit” between Y and X: β0, β1 = arg min_{b0,b1} E[(Y - b0 - b1X)^2]. Once we define these parameters, we define the regression error term as ε = Y - β0 - β1X, which then generates the linear model Y = β0 + β1X + ε.

(a) Using the first order conditions for β0 and β1 (setting the derivatives of the right hand side of the minimization problem to zero), show why E[ε] = 0 and E[X] = 0.

(b) Using the definition of β0 and β1 as the line of best fit parameters, provide an intuitive explanation for why E[ε] = 0.

Paper For Above instruction

The principles of simple linear regression involve estimating the parameters β0 and β1 that best characterize the relationship between the dependent variable Y and the independent variable X. The objective function minimizes the expected squared differences between observed and predicted values, leading to the least squares estimates. The assumption that E[ε] = 0 is fundamental because, in the classical linear regression model, the residuals (or errors) are expected to average out to zero across the population, implying no systematic bias in the predictions.

By applying the first order conditions for the parameters, which involve setting the derivatives of the sum of squared residuals with respect to β0 and β1 to zero, it can be demonstrated that the expected residuals are mean-zero. Specifically, the condition for β0 leads to E[ε] = 0, while the condition for β1 relates to the covariance between X and ε, which is zero under classical assumptions, implying the errors are uncorrelated with the regressors.

Intuitively, since the least squares method chooses coefficients β0 and β1 that minimize the sum of squared residuals, the residuals are projected orthogonally to the space spanned by the regressors. This orthogonality ensures the residuals have a mean of zero, and the estimation process effectively centers the residuals around zero, reflecting the best linear approximation to the data.

In hypothesis testing and constructing confidence intervals, the focus is on assessing whether the estimated parameters significantly differ from hypothesized values. For example, testing whether β1 equals zero involves calculating test statistics based on the estimated coefficient and its standard error and comparing these to critical values from the normal or t-distributions, depending on the sample size and assumptions.

The standard error of the estimate quantifies the variability of the estimator. For large samples, the distribution of the scaled difference (√n(β̂1 - β1)) approximates a normal distribution with mean zero and variance σ^2_{β1}, facilitating hypothesis tests and confidence interval construction.

Furthermore, the interpretation of the estimated slope β̂1 depends on the units of X and Y; it indicates the expected change in Y associated with a one-unit increase in X, all else constant. Statistical tests examine whether this effect is statistically meaningful, with the p-value indicating the probability of observing the data under the null hypothesis.

The confidence intervals provide a range of plausible values for the true parameter. If the interval excludes zero, it lends evidence against the null hypothesis, implying a statistically significant relationship between X and Y.

In empirical applications, assumptions such as random sampling, homoskedasticity (constant variance of errors), and the rank condition (variability in X) are crucial for valid inference. Violations of these assumptions can lead to biased estimates and invalid inferences.

References

• Greene, W. H. (2018). Econometric Analysis (8th ed.). Pearson.
• Wooldridge, J. M. (2016). Introductory Econometrics: A Modern Approach (6th ed.). Cengage Learning.
• Verbeek, M. (2017). A Guide to Modern Econometrics (5th ed.). Wiley.
• Stock, J. H., & Watson, M. W. (2019). Introduction to Econometrics (4th ed.). Pearson.
• Hayashi, F. (2000). Econometrics. Princeton University Press.
• Angrist, J. D., & Pischke, J. S. (2009). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press.
• Pratt, J. W. (1981). Comments on some variances of estimators in linear regression. Journal of the American Statistical Association, 76(375), 848-849.
• Davies, R. B. (1987). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika, 74(1), 33-43.
• Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics: Methods and Applications. Cambridge University Press.
• Kennedy, P. (2008). A Guide to Econometrics (6th ed.). Wiley.