Publication Date

2013-06-06

Availability

Open access

Embargo Period

2013-06-05

Degree Type

Dissertation

Degree Name

Doctor of Philosophy (PHD)

Department

Educational and Psychological Studies (Education)

Date of Defense

2013-04-15

First Committee Member

Nicholas D. Myers

Second Committee Member

Soyeon Ahn

Third Committee Member

Batya Elbaum

Fourth Committee Member

Jaime Maerten-Rivera

Fifth Committee Member

Etiony Aldarondo

Abstract

Previous research has demonstrated that DIF methods that do not account for multilevel data structure could result in too frequent rejection of the null hypothesis (i.e., no DIF) when the intraclass correlation coefficient (ρ) of the studied item was the same as ρ of the total score. The current study extended previous research by comparing the performance of DIF methods when ρ of the studied item was less than ρ of the total score, a condition that may be observed with considerable frequency in practice. The performance of two frequently used simple DIF methods that do not account for multilevel data structure, the Mantel-Haenszel test (MH) and Logistic Regression (LR), was compared to a less frequently used complex DIF method that does account for multilevel data structure, Hierarchical Logistic Regression (HLR). HLR and LR performed equivalently in terms of significance tests under most generated conditions, and MH was conservative across all conditions. Effect size estimates of HLR, LR and MH were more accurate and consistent under the Rasch model than under the 2 parameter item response theory model. The results of the current study provide evidence to help researchers further understand the comparative performance between complex and simple modeling for DIF detection under multilevel data structure.

Keywords

DIF; HLR; LR; MH; Multilevel data

Download

Share

COinS