8.6 Test-Takers’Cognitive Behavior in Phase 1 and Phase
2 Across Their Different Competence Levels
The following arguments attempt to address the research question on the differences
of test-takers’cognitive processes evidence by TAP data and retelling protocols and
bring forth the corresponding implications on the nature and measurement of lec-
ture comprehension.
8.6.1 Argument 1
There is an urgent need for the qualitative analysis in research upon the test-taking
process. The use of frequency data needs to be integrated with qualitative analysis
(Cohen and Upton 2007). Barkaoui et al. (2013: 321) listed some limitations of
their research into test-takers’strategic behaviors in the speaking tasks of TOEFL
iBT test, from which one was that they only considered frequencies (percentages) of
reported strategic behaviors without considering the sequencing of those strategies.
Meanwhile, the number and quality of strategies with different score levels were not
compared and analyzed. Therefore, they called for research“beyond simple fre-
quency counting”(Barkaoui 2013: 321). In another word, we, testers can not be
satisfied with answers to simply“what”questions; instead, we need to examine who
uses each cognitive process or strategy, why, where, when, and how they use them
as well as the effectiveness of these cognitive processes in different contexts. If we
move forward, this type of in-depth research can help us better understand how task
characteristics and test-takers’performance interact with each other with the use of
cognitive processes or strategies as“a mediating tool”.
The present research aims to probe into participants’cognitive processes in
completing an academic listening test with qualitative analysis of their verbal
protocols. With the high interaction with task targets, individual test-takers’cog-
nitive processes can be indicators of their corresponding academic listening
competence.
8.6.2 Argument 2
Test-takers’cognitive depth varies toward the same gap. An obvious difference is
that test-takers of different listening competence approach the same gap at different
cognitive depth. The successful listeners actively employed strategies to contex-
tualize their notes and processed the information semantically while less successful
listeners would resort to lower cognitive processes such as focusing on certain
words without progressing to the next stage, such as interpreting or elaborating on
the given piece of information, etc. It doesn’t mean that successful listeners don’t
8.6 Test-Takers’Cognitive Behavior... 141