Avoiding mistakes when you do econometric analysis depends on your ability to apply knowledge you acquired before and during your econometrics class. Following is a rundown of common pitfalls to help you improve your application of econometric analysis.
Failing to use your common sense and knowledge of economic theory
One of the characteristics that differentiate applied research in econometrics from other applications of statistical analysis is the use of economic theory and common sense to motivate the connection between the independent and dependent variables.
In econometrics, you should be able to make a strong case for the independent variables (Xs) causing changes in the dependent variable (Y). You need sound theory and good common sense to justify your approach. Doing so allows you to provide a sensible interpretation of your results in addition to the typical measures of statistical significance and fit.
Asking the wrong questions first
Getting obsessed with the technical details of estimating econometric models can be easy. However, you should always take a step back and ask yourself why you’re doing what you’re doing. Why will others find my topic interesting and important?
Ignoring the work and contributions of others
Failing to connect your work with that of others who have examined your research question or something closely related to it is a serious mistake. Understanding how others have dealt with similar issues can help you figure out which model to use, may yield refinements in your work, and allows readers to better understand the relevance of your topic.
In your literature review, focus on papers or segments of papers that are directly related to your work. Summarize the approach, data, and findings of other researchers. Finally, be clear about how your work fits in with what’s already been done by others, what’s been improved, and/or how new dimensions of the topic have been explored.
Failing to familiarize yourself with the data
Students often assume that the data they’re working with is complete for all variables and that the reported information is accurate. You can reduce your chances of getting unwelcome surprises in your results by doing some exploratory work that includes descriptive statistics, line charts (for time-series data), frequency distributions, and even listings of some individual data values.
A number of undesirable outcomes can result from failing to get familiar with your analysis data. These three examples are perhaps the most common:
Variables you thought were measured continuously are actually in categories or groups.
Measurements that you believed were real values are actually missing values.
Data values that appear perfectly legitimate are actually censored values.
Making it too complicated
The art of econometrics lies in finding the appropriate specification or functional form to model your particular outcome of interest. In many cases, however, theory can be vague about the specific elements of a model’s specification.
Given the uncertainty of choosing the “perfect” specification, many applied econometricians make the mistake of overspecifying their models (meaning they include numerous irrelevant variables) or favor complicated estimation methods over more straightforward techniques. It can result in undesirable estimator properties and difficulty interpreting the meaning of the results.
Being inflexible to real-world complications
The solutions or predictions derived by using economic theories use logical deduction and/or mathematical proof that usually rely on the ceteris paribus (all else constant) assumption.
The data you use to test economic hypotheses, however, are derived from a world where agents (individuals, firms, or what have you) are engaged with their surrounding environment in ways that aren’t likely to satisfy the ceteris paribus assumption because many of the variables defining their specific circumstances vary considerably from one observation to another.
Looking the other way when you see bizarre results
Most econometric research projects contain estimation results for numerous variations of related models. You want to focus on your primary variables of interest (core variables), but make sure you examine all of your results.
That means don’t ignore unreasonable results (mostly insignificant estimates, coefficients with the wrong sign, and magnitudes that are too large) and proceed to reporting and interpretation. If some results don’t pass a common-sense test, then the statistical tests are likely to be meaningless and may even indicate that you’ve made a mistake with your variables, the estimation technique, or both.
Obsessing over measures of fit and statistical significance
After you estimate an econometric model, focus your attention and guide the reader (if you’re writing a research paper) to the results that are most relevant in addressing your research question.
The importance of your results shouldn’t be determined on the basis of fit (R-squared values) or statistical significance alone. Sure, statistically insignificant coefficients suggest that your independent variable isn’t likely to affect your dependent variable. However, if the lack of a relationship is new or unexpected, this finding may be significant!
Forgetting about economic significance
You can use measures of statistical significance to determine which variables aren’t likely to have an effect on the dependent variable, but you can’t use them to determine which variables have a relevant effect.
After you’ve established that a variable is statistically significant, don’t forget to focus your attention on the coefficient. Sometimes variables can have coefficients that are highly statistically significant even though no economic significance is associated with the result.
The most important element in the discussion of your results is the evaluation of statistical significance and magnitude for the primary variables of interest. If a variable has a statistically significant coefficient but the magnitude is too small to be of any importance, then you should be clear about its lack of economic significance.
Assuming your results are robust
In most cases, economic theory allows for a considerable amount of flexibility in determining the exact specification of the econometric model. You’ll want to see if minor adjustments change your results.
Don’t assume that only one econometric model can apply to your research question and that the results won’t change with reasonable modifications to your specification. You want to perform robustness (or sensitivity) analysis to show that your model estimates aren’t sensitive (are robust) to slight variations in specification.