Re-Thinking the Place of Semiotics in Psychology... 131Osbourne continues: "We have ways of understanding measurement that allow us to
create ever more sophisticated quantification of human attributes and behaviors. Truly, this is
a wonderful time to be a quantitative researcher" (p. 2). The failure to test assumptions, he
says, "troubles me, and I hope it troubles you" (p. 3). Finally:
The world doesn't need another journal promulgating 20th century thinking, genuflecting
at the altar of p < 0.05. I challenge us to challenge tradition. Shrug off the shackles of
20th century methodology and thinking, and the next time you sit down to examine your
hard-earned data, challenge yourself to implement one new methodology that represents
best practice. Use Rasch measurement or IRT rather than averaging items to form scale
scores. Calculate p(rep) in addition to power and p. Use HLM to study change over time, or
use propensity scores to create more sound comparison groups. Use meta-analysis to
leverage the findings of dozens of studies rather than merely adding one more to the
literature. Choose just one best practice, and use it. (p. 3, emphasis in original).I have quoted substantially from this paper in order to illustrate the overwhelming irony
in its message. The failure to test assumptions does indeed ―trouble" me, and it ought, as
Osbourne urges, to trouble any scientific psychologist. But which assumptions? Some of our
assumptions about quantitative psychology rest on other, deeper, assumptions. If we fail to
recognise and examine those deeper assumptions, and if those assumptions turn out to be
indefensible, then ―a great deal of psychological research might well rest on philosophical
quicksand" (Green, 1992, p. 292). In Osbourne's enthusiastic rallying of the quantitative
troops, there is no awareness of the widespread failure to recognise, let alone test, the basic
assumption underlying psychological measurement in general. Nor is there any apparent
awareness that the "promising signs" at the dawn of the 21st century were already presented
and discussed nearly a century earlier, sometimes even by those who invented the
techniques.^20 There is no awareness of the invalidity at the heart of psychology's concept of
psychometric test validity (Michell, 2009). There is no acknowledgment that the altars of
effect sizes, confidence intervals, Rasch measurements, etc. are no more deserving of
genuflection than that of p < 0.05. If this paper is not some quantitative psychologist‘s
attempt to replicate the Sokal hoax, then it is worrying indeed.^21 With respect to semiotics, the
danger of the whole quantitative enterprise is that meaning phenomena either remain
neglected or become distorted and misrepresented (quantified) via the ubiquitous mainstream
incantations of quantification.
The second preparatory step offered by realism is to allow mainstream psychology to
extricate qualitative methods and, hence, semiotic phenomena, from their typical antirealist
and antiscientific ideological and metatheoretical contexts, and thereby to solve the major
acknowledged problem within the qualitative movement. The flip-side of psychology‘s
obsession with quantification is its prejudice against qualitative methods. Yet this is also at
odds with psychology‘s explicit commitments to realism and science. It will be recalled that a
thoroughgoing realist approach gives the lie to the science/meaning divide and, along with it,
(^20) For example, Neyman & Pearson (1933) state quite clearly that, in terms of the nature of scientific evidence,
there is nothing to be gained in a single piece of research by performing a statistical analysis and computing a
single p-value. That is, there is no point at all to how we routinely perform statistical analyses in single
experiments (cf. also Halpin & Stam, 2006).
(^21) The fact that the author has recently published a book entitled Best Practices in Quantitative Methods (Osbourne,
2008) suggests that this paper is not a hoax.