(This post was last updated on 2021-04-12.)
This post mainly discusses how to convert between partial eta squared and Cohen’s d for within-subject designs (with two conditions). (For the conversion for between-subject designs, please refer to this website or this website.)
Here I make a simple simulated dataset for the within-subject designs:
# load library
library(tidyverse)
library(afex)
library(effectsize)
# simulate data
set.seed(42)
nSubj <- 20
# parts of the covariance matrix
C <- matrix(0.5, nrow = 2, ncol = 2)
diag(C) <- 1
# simulate data for the two conditions
simu_dv <- mvtnorm::rmvnorm(nSubj, mean = c(14, 10), sigma = C*6^2)
colnames(simu_dv) <- c("cond1", "cond2")
simu_long <- simu_dv %>%
as_tibble(.name_repair="check_unique") %>%
mutate(Subject = 1:nSubj) %>%
pivot_longer(c(cond1, cond2), names_to = "IV", values_to = "DV")
simu_long
## # A tibble: 40 x 3
## Subject IV DV
## <int> <chr> <dbl>
## 1 1 cond1 21.1
## 2 1 cond2 8.86
## 3 2 cond1 17.1
## 4 2 cond2 14.2
## 5 3 cond1 16.2
## 6 3 cond2 10.0
## 7 4 cond1 22.6
## 8 4 cond2 11.8
## 9 5 cond1 25.6
## 10 5 cond2 12.8
## # … with 30 more rows
in which, twenty participants completed tasks in two conditions. There are forty data points in total.
1 Paired sample t-test vs. one sample t-test and Cohen’s d
To analyze data with within-subject design, we usually use paired sample t-test. Alternatively, we can choose one sample t-test after we calculate the differences between the pairs and further test if those differences are significantly different from zero.
1.1 Paired sample t-test
t.test(DV ~ IV,
data = simu_long,
paired = TRUE)
##
## Paired t-test
##
## data: DV by IV
## t = 3.4454, df = 19, p-value = 0.002711
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 2.225084 9.112426
## sample estimates:
## mean of the differences
## 5.668755
cohens_d(DV ~ IV,
data = simu_long,
paired = TRUE)
## Cohen's d | 95% CI
## ------------------------
## 0.77 | [0.27, 1.30]
1.2 One sample t-test
simu_one <- simu_long %>%
pivot_wider(id_cols = Subject, values_from = DV, names_from = IV) %>%
mutate(DV = cond1 - cond2) %>%
select(Subject, DV)
# one sample t-test
t.test(simu_one$DV)
##
## One Sample t-test
##
## data: simu_one$DV
## t = 3.4454, df = 19, p-value = 0.002711
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
## 2.225084 9.112426
## sample estimates:
## mean of x
## 5.668755
(cohen_simu <- cohens_d(simu_one$DV))
## Cohen's d | 95% CI
## ------------------------
## 0.77 | [0.27, 1.30]
Please note that the results of paired sample t-test and those of one sample t-test are exactly the same, including the effect sizes (i.e., Cohen’s d).
1.3 Formula for Cohen’s d
The Cohen’s d for one sample t-test is relatively simple:
\[\begin{align} \tag1 \text{Cohen's}\; d = \frac{mean(data)}{sd(data)} \end{align}\]
where mean
denotes the mean and sd
denotes the standard deviation.
Following this equation, Cohen’s d for the one sample t-test is mean(simu_one$DV)/sd(simu_one$DV)
, i.e., 0.77, which is the same as the effect size calculated by cohens_d()
.
1.4 Formula for t-value
The equation for t-value (in one sample t-test) is \[\begin{align} \tag2 t = \frac{mean(data)}{se(data)} = \frac{mean(data)}{\frac{sd(data)}{\sqrt{N}}} \end{align}\]
where mean
denotes the mean, se
denotes standard error, sd
denotes the standard deviation, and N
denotes the number of subjects.
Following this equation, t-value in that one sample t-test is mean(simu_one$DV)/(sd(simu_one$DV)/sqrt(nSubj))
, i.e., 3.445, which is the same as that calculated by t.test
.
1.5 Relationships between Cohen’s d and t-value
With equation (1) and (2), we can obtain the relationship between Cohen's d
and t
:
\[\begin{align}
\tag3
t &= \frac{mean(data)}{sd(data)} \times \sqrt{N} \\
&= \text{Cohen's}\;d \times \sqrt{N}
\end{align}\]
2 Repeated-mesuare ANOVA and partial eta squared
aov_simu <- aov_4(DV ~ IV + (IV | Subject),
data = simu_long)
aov_simu
## Anova Table (Type 3 tests)
##
## Response: DV
## Effect df MSE F ges p.value
## 1 IV 1, 19 27.07 11.87 ** .136 .003
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1
(eta_simu <- eta_squared(aov_simu, ci=.95))
## Parameter | Eta2 (partial) | 95% CI
## -----------------------------------------
## IV | 0.38 | [0.07, 0.63]
2.1 Formula for partial eta squared (based on F-value)
According to these resources (website 1, website 2 and others), partial eta squared can be computed from F-value:
\[\begin{align} \tag4 \text{partial}\ \eta^2 = \frac{F \times df_{\text{effect}}}{F \times df_{\text{effect}} + df_{\text{error}}} \end{align}\]
Since there are only two conditions for paired samples, \[\begin{align} \tag5 df_{\text{effect}} = 1\\ df_{\text{error}} = N-1 \end{align}\]
Thus, \[\begin{align} \tag6 \text{partial}\ \eta^2 = \frac{F}{F + N-1} \end{align}\]
where F
is the F-value and N
is the number of subjects.
Following equation 6, partial eta squared for the effect is as_tibble(aov_simu$anova_table)$F/(as_tibble(aov_simu$anova_table)$F + as_tibble(aov_simu$anova_table)[1,2])
, i.e., 0.384532, which is the same as the effect size calculated by eta_squared
.
3 Conversion between Cohen’s d and partial eta squared
In the statistics analysis for the same dataset, \[\tag7 F = t^2 \]
where F
denotes the F-value in ANOVA and t
denotes the t-value in t-test. (For derivation see here or here)
3.1 From Cohen’s d to partial eta squared
Put equations 3, 6 and 7 together: \[\begin{align} t &= \text{Cohen's}\;d \times \sqrt{N}\\ \\ \text{partial}\:\eta^2 & = \frac{F}{F + N-1} \\ & = \frac{t^2}{t^2 + N - 1} \\ \tag8 & = \frac{{\text{Cohen's}\;d}^2 \times N}{{\text{Cohen's}\;d}^2 \times N + N -1} \end{align}\]
Therefore, the equation for conversion from Cohen’s d to partial eta squared is: \[\begin{align} \tag9 \text{partial}\;\eta^2 = \frac{\text{Cohen's}\;d^2 \times N}{{\text{Cohen's}\;d}^2 \times N + N -1} \end{align}\]
With formula 9, we can calculate partial eta squared from Cohen’s d via cohen_simu$Cohens_d^2 * nSubj /(cohen_simu$Cohens_d^2 * nSubj + nSubj - 1)
, i.e. 0.38, which is the same as the effect size calculated by eta_squared
.
3.2 From partial eta squared to Cohen’s d
The equation for conversion from partial eta squared to Cohen’s d can be obtained from equation 9: \[\begin{align} \text{partial}\:\eta^2 & = \frac{{\text{Cohen's}\;d}^2 \times N}{{\text{Cohen's}\;d}^2 \times N + N -1}\\ ({\text{Cohen's}\;d}^2 \times N + N -1) \times \text{partial}\:\eta^2 & = {\text{Cohen's}\;d}^2 \times N \\ \\ {\text{Cohen's}\;d}^2 \times N \times (\text{partial}\:\eta^2 - 1) & = - (N-1) \times \text{partial}\:\eta^2 \\ \\ {\text{Cohen's}\;d}^2 & = \frac{- (N-1) \times \text{partial}\:\eta^2}{N \times (\text{partial}\:\eta^2 - 1)} \\ & = \frac{(N-1)}{N} \times \frac{\text{partial}\:\eta^2}{(1 - \text{partial}\:\eta^2)} \\ \\ \text{Cohen's}\;d &= \sqrt(\frac{(N-1)}{N} \times \frac{\text{partial}\:\eta^2}{(1 - \text{partial}\:\eta^2)}) \tag{10} \end{align}\]
With formula 10, we can calculate the Cohen’s d from partial eta squared via sqrt((nSubj-1)/nSubj*eta_simu$Eta2_partial/(1-eta_simu$Eta2_partial))
, i.e., 0.77, which is the same as the effect size calculated by cohens_d
.
4 Some words at the end
I cannot guarantee these conversions are applicable in all situations, but at least they work well for the cases similar to the simulated data. To make it simple, I guess as long as the data can be analyzed with one sample t-test (e.g., paired sample t-test, or even the interaction between two independent variables with within-subject design), these conversions should work.
The initial motivation for me to look into the conversion between Cohen’s d and partial eta squared is my trying to run equivalence tests. I just start to learn running equivalence tests and notice that an interval of raw values or an interval of Cohen’s d needs to be set for using TOSTER package in R. However, what I got from a meta-analysis paper is an interval of partial eta squared.
Hope these conversions make sense and I really appreciate if you could let me know if there is any error in this post.
5 Update
Now I added the two functions of converting between partial eta squared and Cohen’s d in gist. The example usage is as following:
# load the functions
devtools::source_gist("https://gist.github.com/HaiyangJin/3334e4d6588cbfe36b69c1bf2540c2ea")
cohen_simu$Cohens_d
## [1] 0.770416
# convert partial eta squared to Cohen's d
pes2d(eta_simu$Eta2_partial, nSubj) # 0.38
## [1] 0.770416
eta_simu$Eta2_partial
## [1] 0.384532
# convert Cohen's d to partial eta square
d2pes(cohen_simu$Cohens_d, nSubj) # 0.77
## [1] 0.384532