Testing Lecture Comprehension Through Listening-to-summarize Cloze Tasks

(National Geographic (Little) Kids) #1

Third, after a short break, the research version of the main test was conducted.
Participants heard the research version of the TEM 8 mini-lecture for thefirst time.
While listening, they took notes on the important points. Afterward, they were
given the answer sheet with the gap-filling task and asked to complete the task after
the mini-lecture. They could refer to their notes while doing the gap-filling task. All
the participants were fully engaged in thinking aloud during the whole task-tackling
process and they were encouraged to verbalize the heeded thoughts controlled by
the central processor (Ericsson and Simon 1993). If there appeared a pause longer
then 10 s, brief and clear probing would be given, e.g.“Would you please go on?”
and if the researcher was not clear about the participant’s reported thought, further
probing such as“what does it mean?”,“can you please say it again?”or“what are
the reasons behind your answer?”, etc. to elicit as much valid information as
possible. It is important to note here all the participants were only required to think
aloud right on the spot rather than offering further interpretation of their thoughts or
any comment on what they should have performed in the test condition or any
assumption of a good test-taker’s strategies. As a result, all the participants chose to
use Chinese to verbalize their thoughts during thefirst phase of the research version
test.
Then, after participants had completed the gap-filling task, they heard the
mini-lecture for the second time, but the researcher would pause in between, with
the average audio length of approximately 15–30 s and during the intervals, par-
ticipants were required to retell what they had already heard. In the retelling phase,
all the participants were required to use English to retell what they heard. Certain
prompts were given to the participants if they suddenly became nervous or they
asked for prompts of their own accord. As retelling is by nature the retrospection of
what the participants already heard, it might not be categorized as concurrent verbal
reports, but both concurrent and immediate retrospective verbal reports are con-
sidered a valid means to obtain individuals’thinking processes (Faerch and Kasper
1987; Ericsson and Simon 1993; Brown and Rodgers 2002). Participants’imme-
diate retrospective TAP data in the retelling phase later yielded rich information that
facilitated understanding of how they parsed complicated sentences and constructed
the discourse representation of the academic lecture.
Finally, a very brief semi-structured follow-up interview was conducted which
only lasted 15–20 min with the major aim to clarity vague part of the recordings
and ask about participants’perception of the test, including difficulties of the test
and how they cope with those difficulties. Please refer to Fig.6.2for the overall
procedures.


78 6 Employing the Think-Alound Method...

Free download pdf