10.0 Bivariate Statistics
Now that we understand univariate statistics, we can extend these concepts to bivariate statistics, which involve the analysis of two variables simultaneously. Bivariate statistics help us understand the relationship between two variables and how they interact with each other.
We will begin by focusing on numeric-to-numeric (n2n) bivariate statistics, which involve two numeric variables. In this chapter we will focus on correlation.
Understanding Correlation
Correlation ≠ Causation
When we discuss correlation, we are referring to the relationship between two numeric variables. In other words, how much of a change in variable a is associated with a change in variable b. It is crucial to remember that correlation does not imply causation; just because two variables are correlated does not mean that one causes the other.
For instance, we might consider the relationship between ice cream sales and drowning incidents. While there may be a positive correlation between the two (both tend to increase during the summer months), it does not mean that ice cream sales cause drowning incidents. Instead, a third variable, such as hot weather, influences both. Similarly, we might consider the relationship between home runs in baseball and full moons. There may be a correlation between the two, but it is unlikely that one causes the other.
In other circumstances, two variables may be correlated due to causative forces. For example, there is a strong positive correlation between smoking and lung cancer. In this case, smoking is a causative factor for lung cancer. So we need to always understand the context of the data and the variables involved.
At its core, correlation measures the strength and direction of a linear relationship between two numeric variables. In other words, it quantifies how much the movement of one variable depends on the movement of another variable. The variable that depends on another is often referred to as the dependent variable, while the variable that influences it is called the independent variable. In our example of smoking and lung cancer, lung cancer is the dependent variable, while smoking is the independent variable. No matter what, people are smoking not smoking; they are not smoking because of lung cancer. On the other hand, lung cancer rates may change based on smoking rates. Therefore, we say that lung cancer depends on smoking, and our correlation will measure to what degree to which lung cancer rates increases or decreases as smoking rates increase or decrease.
Correlation is typically measured using the Pearson correlation coefficient, which quantifies the linear relationship between two variables. The correlation coefficient, often denoted as r, ranges from -1 to +1. A positive correlation (r > 0) indicates that as one variable increases, the other variable also tends to increase. A negative correlation (r < 0) indicates that as one variable increases, the other variable tends to decrease. A correlation of zero (r = 0) indicates no linear relationship between the variables.
We can quantify the strength of the correlation based on the absolute value of r:
- 0.0 to 0.3: Weak correlation
- 0.3 to 0.7: Moderate correlation
- 0.7 to 1.0: Strong correlation
In other words, the closer the absolute value of r is to 1, the stronger the linear relationship between the two variables.
The r value can be incredibly helpful because it tells us the direction of the relationship (positive or negative), but it does not tell us the proportion of variance explained between the two variables. Luckily, we can easily derive this information from the correlation coefficient itself. If we simply need to square the r value, we get a new statistic called R-squared (r²), also known as the coefficient of determination. It explains the proportion of variance in the dependent variable that can be explained by the independent variable. In other words, it tells us how much of the change in variable b can be explained by the change in variable a.