6.5.2 Coding Scheme..................................
The crucial criterion one should bear in mind while developing a coding scheme,
according to Green (1998), is that the coding scheme should capture as much of the
information within the verbal protocols as possible. Ericsson and Simon (1993)
once drew a conclusion that little could be learned about cognition from studies
largely focused on task-independent processes after they reviewed a large number
of studies of this category. That is to say, a valid coding scheme should be rooted in
the specific task and help the researcher garner useful information from the con-
textualized verbal protocols. Therefore, a good coding scheme must achieve a
balance between specificity and generalizability (Green 1998), and neither a too
general scheme unable to catch enough cognitive behavior nor a too specific one
only able to describe individual cognitive activity could work.
In the current study, the coding scheme is initiated and developed based on the
literature review across academic English language skills, listening comprehension,
language testing and learning, and more importantly, on the previous conclusions
drawn from the academic listening skill questionnaire survey. We also used the
grounded theory approach of data analysis (Glaser and Strauss 1967), so the cat-
egories were adapted from the previous taxonomies and rather emerged from the
data of both pilot and main studies.
The coding scheme of thefirst phase, the test-taking phase covers the main
cognitive processes such as: selective attention, meaning construction, monitoring,
and decision-making, and the test-wiseness strategies independent of the normal
test-taking cognitive processes. There are also subcategories contained in each
general one. From time to time, there emerged new cognitive operations from the
data during the coding process and they would be added to the appropriate cate-
gories with definitions and concrete examples.
All the data were coded to the level of subcategories, but in the data analysis,
only general categories were used in order to reduce dimensions of variables.
The following examples show how the verbal protocols and interview data were
coded:
a. balance the coding. The coding scheme is refined to capture both representative
cognitive processes of more than two test-takers and cover all the verbal pro-
tocol data related to all the 16 participants across two years’tasks. For example,
atfirst, checking grammatical acceptability as one of the monitoring processes
was further divided into checking spelling, checking part of speech, checking
concord, etc. Later, few references were coded to the detailed nodes and then
these smaller nodes were cancelled, so grammatical acceptability detained as the
only sub-node in this category.
b. split segments. Split the initial one segment into two in order to avoid double
coding and mixed coding. For example, the following excerpt was atfirst
regarded as one segment because it was said in a short time influency and can
be regarded as one thinking unit. But in the coding process, we found that the
first small clause mentioned the participant’s awareness of the main idea of the
6.5 Transcription of TAPs, Retelling Protocols and Interviews 81