DOI

https://doi.org/10.25772/1ARG-1C48

Defense Date

2012

Document Type

Dissertation

Degree Name

Doctor of Philosophy

Department

Business Administration

First Advisor

Anson Seers

Abstract

This research outlines a conceptual framework and data analysis process to examine multisource feedback (MSF) rater group differences from a leadership assessment survey, after testing the measures for equivalence. MSF gathers and compares ratings from supervisors, peer, followers and self and is the predominant leadership assessment tool in the United States. The results of MSF determine significant professional outcomes such as leadership development opportunities, promotions and compensation. An underlying belief behind the extensive use of MSF is that each rater group has a different set of implicit leadership theories (ILTs) they use when assessing the leader, and therefore each group is able to contribute unique insight. If this is true, research findings would find rater group consistency in leadership assessment outcomes, but they do not. A review of group comparison research reveals that most empirical MSF studies fail to perform preliminary data exploration, employ consistent models or adequately test for measurement equivalence (ME); yet industry standards strongly suggest exploratory methods whenever data sets undergo changes, and misspecified models cause biased results. Finally, ME testing is critical to ascertain if rater groups have similar conceptualizations of the factors and items in an MSF survey. If conceptual ME is not established, substantive group comparisons cannot be made. This study draws on the extant MSF, ILT and ME literature and analyzes rater group data from a large, application-based MSF leadership database. After exploring the data and running the requisite MI tests, I found that the measures upheld measurement invariance and were suitable for group comparison. Additional MI tests for substantive hypotheses support found that significant mean differences did exist among certain rater groups and dimensions, but only direct report and peer groups were consistently significantly different in all four dimensions (analytical, interpersonal, courageous and leadership effectiveness). Additionally, the interpersonal dimension was the most highly correlated with leadership effectiveness in all five rater groups. The overall findings of this study address the importance of MSF data exploration, offer alternative explanations to the disparate leadership MSF research findings to date and question the application use of MSF tools in their current form.

Rights

© The Author

Is Part Of

VCU University Archives

Is Part Of

VCU Theses and Dissertations

Date of Submission

May 2012

Share

COinS