Thursday, 16 January 2014

Looking back on JAN: Likert or Rasch? Nothing is more applicable than good theory

Roger Watson, Editor-in-Chief

Sometimes a particular paper ‘turns your head’; it has a great influence on you, and the significance of the paper remains clear for years.  In my research career, such a paper is the one by van Alphen et al. (1994) titled: Likert or Rasch? Nothing is more applicable than good theory.  One of the co-authors - Ruud Halfens - featured in a previousblog post on the LPZ project.

The paper is about scaling and looks at the use, in questionnaires, of either simple Likert scaling or Rasch scaling.  In Likert scales items are assumed to have the same weight and the total score of a respondent on a set of items is a measure of where they lie on the latent trait being measured by the questionnaire.  This is, essentially, classical test theory.  In Rasch scaling, items are not considered to be equally weighted, they vary in ‘difficulty’ and, instead, are ordered along the latent trait.  The total score on a set of items is a function of the ordering of the items and how the respondents score on the items.  As stated by van Alphen et al. (1994, p. 196): ‘With Rasch both respondents and items are scaled on the same continuum’.  As also stated (p. 196): ‘The rationale behind this procedure is item response theory’.  It is worth noting that all of the above is contained in the abstract; in 203 words Alphen et al. (1994) virtually summarise a whole field of theory regarding measurement with questionnaires.  It has taken me 181 words to summarise their abstract.

When I read this paper I encountered all of the following concepts for the first time:

·         classical test theory
·         item response theory
·         item difficulty
·         unidimensionality
·         local stochastic independence
·         the item characteristic curve

I recommend this paper to anyone who wants to know what item response theory is and how it contrasts with classical test theory.  Moreover - and this is where the paper has had such a profound influence on my own research career - the basic tenets of item response theory are explained: unidimenensionality and local stochastic independence.  These are not easy concepts to understand; they are related, and Alphen et al. (1994, p.199) describe unidimensionality beautifully as meaning: ‘that dependence among items can be accounted for by only one latent trait’.  Their explanation of local stochastic independence is not quite as clear, being based on dependence in the entire population but independence for subpopulations.  The usual explanation of local stochastic independence is more simply rendered along the lines of the response to one item in a questionnaire being independent of other, previous, responses to other items and that the scores on the items are only a function of the respondent’s position on the latent trait.  Nevertheless, despite the slightly convoluted description of local stochastic independence, this was the first time I had encountered the concept and it has haunted me ever since.  I say ‘haunted’ because I am a frequent user of one form of item response theory (Mokken scaling) which is based on the assumption of local stochastic independence.  However, the assumption of local stochastic independence of items in questionnaires remains largely that: an assumption; it is very hard to prove.

An item characteristic curve
The item characteristic curve was a revelation to me in terms of representing how questionnaire items behave. 
Before meeting the item characteristic curve I had never considered the relationship between an item score and the total score on a latent trait in terms of probability.  The concept was not immediately clear to me but I am now so familiar with item characteristic curves that most of my research work is concerned with analysing them.

Therefore, in terms of looking back on JAN, I am incredibly grateful that Alphen et al. (1994) took the trouble to write this methodological paper and to submit it to the journal.  I am also grateful that it was published and it remains worth reading.



Reference
van Alphen A, Halfens R Hasman A, Imbos T (1994) Likert or Rasch? Nothing is more applicable than good theory Journal of Advanced Nursing 20, 196-201


No comments:

Post a Comment