Exploratory Data Analysis
The goal of exploratory data analysis (EDA) is to analyze and investigate the main characteristics of datasets, often using data visualization methods like plots. My first step for EDA is to look at the dataset and understand it at a high level. How large is it and what are the variables? Are there missing values? Are there a lot of missing values of specific variables? Are each of the variables continuous, discrete, or categorical? After getting this basic understanding, I want to look at some plots. If the data is taken at different times, I would look at plots of variables vs. time to look for signs of seasonality or a general positive or negative trend over time. I also will look at basic summary statistics, like mean and variance, and plots that allow me to see the shape of the distribution of variables, like histograms and bar plots. Then, I can look for significant correlation between variables using a correlation matrix. Finally, I can pull together and summarize what I have learned from the data. The learnings from the EDA can inform whether assumptions are reasonably being met for statistical analysis techniques I may want to use for further analysis, and the EDA also helps me identify interesting relationships between variables that I may want to investigate further.