Testing Lecture Comprehension Through Listening-to-summarize Cloze Tasks

lecture; items 10, 15, 24 and 25 emphasizing cohesion and coherence, e.g., the
cohesive devices used in an academic lecture and the logical connection between
different points. In another word, all of these items contain prominent words such as
macro structure, main points, key words, etc. that suggest the holistic compre-
hension of an academic lecture. Items representative of Factor 1 all contain a
common feature which can be interpreted as creating macro discourse representa-
tion. Meanwhile, interpretation of Factor 1 echoes the above descriptive statistics of
the teacher questionnaire data that demonstrate teachers have placed their main
concern on the attainment of the discourse-level comprehension of an academic
lecture because a lecture is a scientifically organized discourse which needs the
macro understanding of its key points as the prerequisite.
For Factor 2, since items 31–35 heavily loaded on Factor 2 are all closely related
to the note-taking skill, it is easy to name Factor 2 as taking effective notes, which is
crucially important not only in the test setting but also in real lecture halls.
For Factor 3, items 1, 2 and 3 focused on the acoustic input of a lecture are the
most heavily loaded factors, which can be considered representative of Factor 3
while items 5 and 6 deal with inferring meaning of new words. Item 4 concerns the
ability to follow the lecturer’s speed. Therefore, the representative ones fall into the
decoding process of listening comprehension and hence Factor 3 can be named as
decoding acoustic/visual input. Less related items are item 7 indicating the grasp of
the literal meaning of a lecture and items 14, 20 and 21 suggesting the skill to infer
the connotation from the relevant lecture content.
For Factor 4, the most highly related items include items 23 and 27 (above 0.7).
Items 22 and 23 clearly indicate the skill to identify details of an academic lecture
through micro discourse markers; items 26 and 27 deal with the communicative
functions suggested by micro discourse markers; items 28, 29 and 30 are concerned
with the skill of making detailed inferences. As a whole, this factor is geared toward
micro understanding of a lecture, i.e. grasping details.
As afinal remark, the items with cross-loadings, such as items 13, 24 and 25, are
related to more than one factor, though these items actually have very clear indi-
cations. Item 13 emphasizes the skill to infer the main idea via the gap-filling task
itself, and items 24 and 25 both suggest identification of cohesion. These items
might not contribute a lot to the four-factor structure that is processed so far, but
they are still part of the big latent construct of TEM 8 Mini-lecture comprehension.

Table 5.9 Main factors extracted from teacher questionnaire

Factor Factor 1

Creating macro

discourse

representation

Factor 2

Taking

effective

notes

Factor 3

Decoding

acoustic/visual

input

Factor 4

Creating micro

discourse

representation

Items Items 8, 9, 10, 11,

12, 15, 16, 17, 18,

19

Items 31,

32, 33, 34,

35

Items 1, 2, 3, 4,

5, 6, 7, 14, 20,

21

Items 22, 23, 26,

27, 28, 29, 30

Cross-loadings Items 13, 24, 25

5.5 Results of Questionnaire Survey 55