Testing Lecture Comprehension Through Listening-to-summarize Cloze Tasks

between metal processes language tests elicit and those happen in non-test condi-
tions would be out of the question. The current project contributes to the explo-
ration of candidates’ cognitive processes they employ in doing the TEM 8
Mini-lecture task, which could in turn help assess the cognitive validity of it.
It is assumed that the cognitive operation sequence in the test-taking phase
projects a linear pattern based on the questionnaire data and the following is the
assumed sequence model of the test-taking phase:

Decoding input Taking notes Discourse representation Filling the gap

The same main factors are extracted from both teachers’and students’ques-
tionnaire survey, i.e., decoding input, macro and micro discourse construction and
note-taking and hence according to either teachers or students, these prominent
factors can generalize the skills to be tested by an academic listening test and can
impact test-takers’test performance. In the assumed linear sequence, test-takers
shouldfirst decode the information from an academic lecture, immediately take
down relevant notes and ultimately form a discourse representation of the academic
lecture. But this is merely a simplified presumed model. What really happens could
be a different version.
According to all the codes of the Phase 1 Think-aloud Protocols, participants’
cognitive processes involved in dealing with the TEM 8 Mini-lecture Gap-filling
task have become clearer. Actually, after decoding the input of the mini-lecture in
the testing condition, the test-takers’attention is rather focused on making sense of
the notes and contextualizing their notes in the gaps in order to complete the
comprehension task. Meanwhile, they are also constructing the discourse repre-
sentation of the mini-lecture since the gap-filling task is given in an outline format
that highly condenses the content of the lecture. In this case, test-takers would
simultaneously be more inclined to make sense of the given outline while searching
for suitable words in their notes tofill in the blanks.
The table of nodes list with sources and references (see Table6.13) shows that
all the 16 participants employed the decoding process, meaning-building process
andfinally made their decisions. Only two of them did not report anything related to
the monitoring process. In terms of frequency, cognitive operations used in the
decoding process top others and the meaning-building cognitive operations go in
the middle, while monitoring processes are mentioned least frequently. On the

Table 6.13 Nodes list for
Phase 1-Tesk-taking

Nodes Sources References

Cognitive processes in Phase 1 16 642

Decoding + selective attention 16 322

Meaning + discourse construction 16 124

Monitoring process 14 67

Decision-making 16 129

6.6 Results of the Test-Taking TAPs 91