43gathered information of how your target audience defines the construct. You’ve
updated your definition based on that feedback. So now you have a list of compo-
nents that make up your construct, and you should develop (or refine) a question for
every single one of them.
It’s easy to write a bad question. Writing a good question requires work. Bad
questions will yield bad data, and, for this reason, it is important to scrutinize every
part of every question you write.
To paraphrase the late Jacques Barzun, the writing of a sentence isn’t finished
until its meaning is clear, until it means just one thing. The same rule should be
applied when writing assessment questions: a question isn’t finished until its answer
means just one thing. If a question is unclear or confusing, then so will be the data
you collect.
Consider the following survey item:
My peers are conscientious and caring:
- Always
- Sometimes
- Never
There is nothing wrong with the grammar of this item. It seems clear enough –
one could imagine it placed in a survey on workplace morale. And yet, in nine
words, there are no fewer than six serious errors.
There are many rules to writing assessment items. Not all of them can be covered
here, but we can cover common and completely avoidable errors.
Avoid conjunctions: No word has a simpler meaning than and. Yet and is perhaps
the most problematic word in the example item above. In assessment jargon, this
is called a “double-barreled” question, because it asks the respondent to consider
two ideas at once: “my peers are conscientious” and “my peers are caring.” This
creates confusion. If, for example, a person’s peers are always conscientious and
sometimes caring, the correct answer is unclear.
The simplest way to avoid double-barreled items is to break the question into sepa-
rate items. Two items – “My peers are conscientious” and “My peers are car-
ing” – work far better than the confusing single item.
When writing a survey question, you must scrutinize every word. Conjunctions
are easy to spot. It’s more difficult to check one’s vocabulary: are you using words
that your readers understand?
Avoid big words, rare words, and jargon: If your reader doesn’t know what a word
means, then you can’t know what her answer means. Consider our example item:
“my peers are conscientious.” One word should stand out – conscientious. You may
know what the word means, but is it a word that your readers know so well that they
would be comfortable using it themselves? If your answer to that question is any-
thing other than “definitely yes,” then you need to find another, simpler word.
4 Measurement in Education: A Primer on Designing Assessments