Testing Lecture Comprehension Through Listening-to-summarize Cloze Tasks

The following coding scheme (Table6.10) is an inventory of cognitive opera-
tions in Phase 2-retelling:
There are also two levels in coding retelling protocols. Thefirst level is the basic
cognitive processes test-takers underwent while they were required to retell the
lecture. The second level of coding mainly emerges in summarization of the dis-
course. So, the individual codes should be the second level of this coding scheme
while coding summarizing protocols.
Since the following-up interview is very brief with the main purpose to clarify
the TAPs as well as elicit participants’true perception of the test, the coding scheme
(see Table6.11) for the interview part is also simplified to one dimension—main
themes arising from what they have said.

Table 6.9 (continued)

Cognitive processes Definition Protocols

Monitoring

Assessing

comprehension (AC)

Reflect on the word/words to be

filled in the blank; evaluate one’s

comprehension

But I am not very sure if I should put

the word here

Checking quantity

(CQ)

Check the number of words to be

filled in the blank according to the

word limit

Since the word limit is three words,

I’d betterfill in the original words“I

don’t care”

Checking

grammatical

acceptability (CGA)

Check whether the word/words is/are

grammatically acceptable iffilled in

the blank

But because I judge it should be a

noun in the blank, I’d better write

“verbal communication”here

Checking semantic

coherence (CSC)

Check whether the word/words is/are

semantically acceptable iffilled in

the blank

It’s not something toward behavior.

It should be something toward

difficulty or failure, it’s more suitable

Decision-making

(DM)

Pinning down a word/words tofill in

the blank

The second is reflect on information

Test-wiseness

strategies

(independent of

cognitive processes)

Skipping difficult

items (SDI)

Skip the difficult blanks and focus on

the easier onesfirst

The seventh one, I cannot do it. I’d

better skip it

Guessing randomly

(GR)

Fill in a word without any rationale Just randomly choose a word, put

“information”here

Predicting the tester’s

intention (PTI)

Predict the tester’s intention andfill

in the word/words based on the

prediction

Initially, I want to write down

“college learning”, then I thought it

won’t be that simple

6.5 Transcription of TAPs, Retelling Protocols and Interviews 85