Testing Lecture Comprehension Through Listening-to-summarize Cloze Tasks

The holistic task demand covering both teachers’and students’perceptions shall be
triangulated with the cognitive processes elicited from TAP data.

5.3 Design of Questionnaire.................................

Both teachers’and students’form of questionnaire covers three parts: the personal
information and the cognitive preparation at the beginning and the skill part as the
main body. As listed above, the cognitive preparation part is different in the two
forms of questionnaire, because teachers who participated in the TEM 8 human
rating section are all familiar with the test and a clear description of its academic
listening subtest mainly serves as a prompt to elicit their perception of the skills that
should be tested as part of the academic listening construct. Students on the other
hand are required tofill in their questionnaires after they have completed a simu-
lation test using one of the previous years’actual TEM 8 papers in the classroom
setting; therefore, their general impression as the cognitive preparation before they
proceed to the main part can help clarify their understanding of the test. The skill
part as the main body of the questionnaire remains the same across the two
versions.
The main body, i.e. the academic listening skill part of the questionnaire is
largely based upon Richard’s (1983) academic skill checklist and compensated by
Munby’s taxonomy of language skills (Munby 1978). More specifically, the
questionnaire is designed to elicit participants’perception on the skills tested by a
high-stakes academic listening test-TEM 8 Mini-lecture and Gap-filling task, which

Table 5.1 Questionnaire participants

Teacher questionnaire Student questionnaire

Participants 187 valid teacher questionnaires.

176 raters from over 50

universities and colleges

participated in 2013 TEM8

human rating section and

submitted their digital forms of

questionnaire; 3 Ph.D. students in

the same doctoral program and 8

college teachers handed in their

printed form of questionnaire

Altogether 230 students

submitted their printed form of

questionnaire among which 204

were valid. All the participants

were given a simulation testfirst

to make sure they must acquire

due knowledge of the task that the

questionnaire is focused upon

Representativeness Respondents cover most districts

from mainland China and nearly

80% of them have teaching

experience of over 10 years

Over 80% of the seniors of the

English Major in the college

responded to the questionnaire

and all of them are familiar with

the task in question

44 5 Exploring Task Demands of TEM 8 Mini-Lecture...