Testing Lecture Comprehension Through Listening-to-summarize Cloze Tasks

difficult as answers are not directly stated in the mini-lecture. Therefore, it becomes
crucially important to observe how the same inference is perceived by test-takers
and whether inferences intended by test-developers are also reached by the
test-takers.
But, what on earth these gaps are really targeting still remains unsettled until it is
supported by substantial empirical data at the end of this chapter.

6.2.4.2 Scoring Criteria of TEM 8 Mini-Lecture and Gap-Filling
Tasks

In order to guarantee the reliability of the test scores of both versions, only one
experienced rater rated all the participants’fulfilled TEM 8 mini-lecture tasks and
two tasks were used as the research version in the current study, i.e., the
mini-lecture task of the year 2010 and 2013 respectively (see Appendix 4 and 5).
The mini-lecture and gap-filling task section in TEM 8 equals 10 points, occupying
10% of the total score. The rater participated in the TEM 8 annual rating of
subjective sections for at least three consecutive years and had clear knowledge of
the scoring rubrics. Since it is explicitly stated in the instruction“Make sure the
word(s) youfill in is (are) both grammatically and semantically acceptable”,
answers that fail to meet this standard will result in deduction of points. For
example, excluding those totally irrelevant answers that resulted in deduction of the
whole point, there were also some common mistakes that failed to meet the
semantic or grammatical criteria and resulted in deduction of half a point, such as
spelling and grammatical mistakes, semantically incomplete answers, etc.

6.3 Qualitative Data Collection...............................

In a more detailed manner, qualitative data collection aiming to elicit participants’
cognitive processes involved in test-taking and retelling follows the following
sequence (see Table6.8).

Table 6.7 Levels of
cognitive process elicited by
gap types of TEM 8

Gap type

1

GT

2

GT

3

GT

4

Input decoding √√√√

Lexical search √√√√

Parsing √√√√

Meaning

construction

√√

Discourse

Construction

√

6.2 Participants in the Qualitative Part of the Research 75