Testing Lecture Comprehension Through Listening-to-summarize Cloze Tasks

6.3.1 Pilot Study of the Qualitative Part of the Research.......

A pilot study was carried out two months prior to the main study and two junior
students participated in it. There are four main aims of the pilot study: a. check the
condition of the physical environment where TAP and interview data are to be
recorded; b. check the procedure of the main study; c. decide whether prompts are
needed and what prompts are appropriate in the main study; d. enrich and validate
the questionnaire for the quantitative part of the project.
During the pilot study, the two participants were given a full-length TEM 8
mini-lecture for their verbalization of thoughts. They were encouraged to think
aloud during the task-tackling process no matter English or Chinese was used.
Afterward, based on the pilot studyfindings, there were several changes to be made
in the main study:

1. Facilities: the pilot study was conducted in an office with occasional noise from
   the classroom nearby; therefore, the main study was moved to a new isolated
   office which was completely quite without any noise to disturb the participants
   and interfere with the recording. In additional, two digital voice recorders were
   prepared for the main study in case that one might fail in the recording process.
   In the pilot study, the participant and researcher sat fact to face and then the
   participants felt a bit nervous facing a teacher during the test. Then, in the main
   study, the researcher and participants sat in an angle that they could see each
   other but not directly face each other and they sat with distance, in order to
   eliminate any discomfort or nervousness imposed on the participants.


2. Instructions: the researcher used Chinese to clarify instruments and what par-
   ticipants are required to do during the main study. In the pilot study, the
   instructions were given in English and participants were not very sure what they
   should exactly do, so in the main study, all the instructions were given in
   Chinese to make sure that participants fully understand what were required from

Table 6.8 Three phases of the qualitative part of the study

Phase

number

Task Description of the task

Phase 1 Task-tackling Students take notes while listening to the TEM 8 mini-lecture

and then do the gap-filling task afterward. They are required to

report why theyfill in the blanks with certain words and the

researcher will give them a prompt if they stop over 10 s

Phase 2 Retelling After the participants havefinished thefirst phase, they need to

listen to the same TEM 8 mini-lecture again for the second time.

The researcher will pause the recording after each move of the

lecture discourse and students are required to retell what they

have heard immediately during the interval

Phase 3 Interview A brief follow-up interview is conducted right after participants

havefinished their retelling

76 6 Employing the Think-Alound Method...