The second step in test planning is determining the format and length of the test. The format is based on the different types of items to be included in the test. The construction of valid and good test items is a skill just like effective teaching. Some rules are to be followed and some techniques are to be used to construct good test items. Test items can be used to assess student’s ability to recognize concepts or to recall
concepts. Generally there are two types of objective test items:-
i. Select type.
ii. Supply type.
Select Type Items
Matching Items
According to W. Wiersma and S.G. Jurs (1990), the matching items consist of two parallel columns. The column on the left contains the questions to be answered, termed premises; the column on the right, the answers, termed responses. The student is asked to associate each premise with a response to form a matching pair. For example
According to W. Wiersma and S.G. Jurs (1990) in some matching exercises the number of premises and responses are the same, termed a balanced or perfect matching exercise. In others, the number and responses may be different.
Advantages
The chief advantage of matching exercises is that a good deal of factual information can be tested in minimal time, making the tests compact and efficient. They are especially well suited to who, what, when and where types of subject matter. Further students frequently find the tests fun to take because they have puzzle qualities to them.
Disadvantages
The principal difficulty with matching exercises is that teachers often find that the subject matter is insufficient in quantity or not well suited for matching terms. An exercise should be confined to homogeneous items containing one type of subject matter (for instance, authors-novels; inventions inventors; major events-dates terms – definitions; rules examples and the like). Where unlike clusters of questions are used to adopt but poorly informed student can often recognize the ill-fitting items by their irrelevant and extraneous nature (for instance, in a list of authors the inclusion of the names of capital cities).
Student identifies connected items from two lists. It is Useful for assessing the ability to discriminate, categorize, and association amongst similar concepts.
Suggestions for Writing Matching Items
Here are some suggestions for writing matching items:
i. Keep both the list of descriptions and the list of options fairly short and homogeneous – they should both fit on the same page. Title the lists to ensure homogeneity and arrange the descriptions and options in some logical order. If this is impossible, you’re probably including too wide a variety in the exercise. Try constructing two or more exercises.
ii. Make sure that all the options are plausible distracters for each description to ensure homogeneity of lists.
iii. The list of descriptions on the left side should contain the longer phrases or statements, whereas the options on the right side should consist of short phrases, words or symbols.
iv. Each description in the list should be numbered (each is an item), and the list of options should be identified by letter.
v. Include more options than descriptions. If the option list is longer than the description list, it is harder for students to eliminate options. If the option list is shorter, some options must be used more than once. Always include some options that do not match any of the descriptions, or some that match more than one, or both.
vi. In the directions, specify the basis for matching and whether options can be used more than once.
B. Multiple Choice Questions (MCQ’s)
Norman E. Grounlund (1990) writes that the multiple choice question is probably the most popular as well as the most widely applicable and effective type of objective test. Student selects a single response from a list of options. It can be used effectively for any level of course outcome. It consists of two parts: the stem, which states the problem and a list of three to five alternatives, one of which is the correct (key) answer and the others are distracters (“foils” or incorrect options that draw the less knowledgeable pupil away from the correct response).
The stem may be stated as a direct question or as an incomplete statement. For example:
Direct question
Which is the capital city of Pakistan? -------- (Stem)
A. Lahore. -------------------------------------- (Distracter)
B. Karachi. ------------------------------------- (Distracter)
C. Islamabad. ---------------------------------- (Key)
D. Peshawar. ----------------------------------- (Distracter)
Incomplete Statement
The capital city of Pakistan is
A. Lahore.
B. Karachi.
C. Islamabad.
D. Peshawar.
RULES FOR WRITING MULTIPLE-CHOICE QUESTIONS
1. Use Plausible Distracters (wrong-response options)
Only list plausible distracters, even if the number of options per question changes
Write the options so they are homogeneous in content
Use answers given in previous open-ended exams to provide realistic distracters
2. Use a Question Format
Experts encourage multiple-choice items to be prepared as questions (rather than incomplete statements)
Incomplete Statement Format:
The capital of AJK is in-----------------.
Direct Question Format:
In which of the following cities is the capital of AJK?
3. Emphasize Higher-Level Thinking
Use memory-plus application questions. These questions require students to recall principles, rules or facts in a real life context.
The key to prepare memory-plus application questions is to place the concept in a life situation or context that requires the student to first recall the facts and then apply or transfer the application of those facts into a situation.
Seek support from others who have experience writing higher-level thinking multiple-choice questions.
EXAMPLES:
Memory Only Example (Less Effective)
Which description best characterizes whole foods?
a. orange juice
b. toast
c. bran cereal
d. grapefruit
Memory-Plus Application Example (More Effective)
Sana’s breakfast this morning included one glass of orange juice (from Concentrate), one slice of toast, a small bowl of bran cereal and a grapefruit. What “whole food” did Sana eat for breakfast?
a. orange juice
b. toast
c. bran cereal
d. grapefruit
Memory-Plus Application Example
Ability to Interpret Cause-and-Effect Relationships Example
Why does investing money in common stock protect against loss of assets during inflation?
a. It pays higher rates of interest during inflation.
b. It provides a steady but dependable income despite economic conditions.
c. It is protected by the Federal Reserve System.
d. It increases in value as the value of a business increases.
Ability to Justify Methods and Procedures Example
Why is adequate lighting necessary in a balanced aquarium?
a. Fish need light to see their food.
b. Fish take in oxygen in the dark.
c. Plants expel carbon dioxide in the dark.
d. Plants grow too rapidly in the dark.
4. Keep Option Lengths Similar
Avoid making your correct answer the long or short answer
5. Balance the Placement of the Correct Answer
Correct answers are usually the second and third option
6. Be Grammatically Correct
Use simple, precise and unambiguous wording
Students will be more likely to select the correct answer by finding the grammatically correct option
7. Avoid Clues to the Correct Answer
Avoid answering one question in the test by giving the answer somewhere else in the test
Have the test reviewed by someone who can find mistakes, clues, grammar and punctuation problems before you administer the exam to students
Avoid extremes – never, always, only
Avoid nonsense words and unreasonable statements
8. Avoid Negative Questions
31 of 35 testing experts recommend avoiding negative questions
Students may be able to find an incorrect answer without knowing the correct answer
9. Use Only One Correct Option (Or be sure the best option is clearly the best option)
The item should include one and only one correct or clearly best answer
With one correct answer, alternatives should be mutually exclusive and not overlapping
Using MC with questions containing more than one right answer lowers discrimination between students
10. Give Clear Instructions
Such as:
Questions 1 - 10 are multiple-choice questions designed to assess your ability to remember or recall basic and foundational pieces of knowledge related to this course.
Please read each question carefully before reading the answer options. When you have a clear idea of the question, find your answer and mark your selection on the answer sheet. Please do not make any marks on this exam.
Questions 11 – 20 are multiple-choice questions designed to assess your ability to think critically about the subject.
Please read each question carefully before reading the answer options.
Be aware that some questions may seem to have more than one right answer, but you are to look for the one that makes the most sense and is the most correct.
When you have a clear idea of the question, find your answer and mark your selection on the answer sheet.
You may justify any answer you choose by writing your justification on the blank paper provided.
11. Use Only a Single, Clearly-Defined Problem and Include the Main Idea in the Question
Students must know what the problem is without having to read the response options
12. Avoid “All the Above” Option
Students merely need to recognize two correct options to get the answer correct
13. Avoid the “None of the Above” Option
You will never know if students know the correct answer
14. Don’t Use MCQ When Other Item Types Are More Appropriate
Limited distracters or assessing problem-solving and creativity
Advantages
The chief advantage of the multiple-choice question according to N.E. Gronlund (1990) is its versatility. For instance, it is capable of being applied to a wide range of subject areas. In contrast to short answer items limit the writer to those content areas that are capable of being stated in one or two words, multiple choice item necessary bound to homogeneous items containing one type of subject matter as are matching items, and a multiple choice question greatly reduces the opportunity for a student to guess the correct answer from one choice in two with a true – false items to one in four or five, there by increasing the reliability of the test. Further, since a multiple – choice item contains plausible incorrect or less correct alternative, it permits the test constructor to tine tune the discriminations (the degree or homogeneity of the responses) and control the difficulty level of the test.
Disadvantages
N.E. Gronlund (1990) writes that multiple-choice items are difficult to construct. Suitable distracters are often hard to come by and the teacher is tempted to fill the void with a “junk” response. The effect of narrowing the range of options will available to the test wise student. They are also exceedingly time consuming to fashion, one hour per question being by no means the exception. Finally they generally take student longer to complete (especially items containing fine discrimination) than do other types of objective question.
Suggestions for Writing MCQ’s Items
Here are some guidelines for writing multiple-choice tests:
I. The stem of the item should clearly formulate a problem. Include as much of the item as possible, keeping the response options as short as possible. However, include only the material needed to make the problem clear and specific. Be concise – don’t add extraneous information.
II. Be sure that there is one and only one correct or clearly best answer.
III. Be sure wrong answer choices (distracters) are plausible. Eliminate unintentional grammatical clues, and keep the length and form of all the answer choices equal. Rotate the position of the correct answer from item to item randomly.
IV. Use negation questions or statements only if the knowledge being tested requires it. In most cases it is more important for the student to know what a specific item of information is rather than what it is not.
V. Include from three to five options (two to four distracters plus one correct answer) to optimize testing for knowledge rather than encouraging guessing. It is not necessary to provide additional distracters from an item simply to maintain the same number of distracters for each item. This usually leads to poorly constructed distracters that add nothing to test validity and reliability.
VI. To increase the difficulty of a multiple-choice item, increase the similarity of content among the options.
VII. Use the option “none of the above” sparingly and only when the keyed answer can be classified unequivocally as right or wrong.
VII. Avoid using “all of the above”. It is usually the correct answer and makes the item too easy for students with partial information.
II. Supply Type Items
A. Completion Items
Like true-false items, completion items are relatively easy to write. Perhaps the first tests classroom teachers’ construct and students take completion tests. Like items of all other formats, though, there are good and poor completion items. Student fills in one or more blanks in a statement. These are also known as “Gap-Fillers.” Most effective for assessing knowledge and comprehension learning outcomes but can be written for higher level outcomes. e.g.
The capital city of Pakistan is -----------------.
Suggestions for Writing Completion or Supply Items
Here are our suggestions for writing completion or supply items:
I. If at all possible, items should require a single-word answer or a brief and definite statement. Avoid statements that are so indefinite that they may be logically answered by several terms.
a. Poor item:
Motorway (M1) opened for traffic in ____________.
b. Better item:
Motorway (M1) opened for traffic in the year______.
II. Be sure the question or statement poses a problem to the examinee. A direct question is often more desirable than an incomplete statement because it provides more structure.
III. Be sure the answer that the student is required to produce is factually correct. Be sure the language used in the question is precise and accurate in relation to the subject matter area being tested.
IV. Omit only key words; don’t eliminate so many elements that the sense of the content is impaired.
c. Poor item:
The ____________ type of test item is usually more _________ than the _____ type.
d. Better item:
The supply type of test item is usually graded less objectively than the _________ type.
V. Word the statement such that the blank is near the end of the sentence rather than near the beginning. This will prevent awkward sentences.
VI. If the problem requires a numerical answer, indicate the units in which it is to be expressed.
B. Short Answer
Student supplies a response to a question that might consistent of a single word or phrase. Most effective for assessing knowledge and comprehension learning outcomes but can be written for higher level outcomes. Short answer items are of two types.
Simple direct questions
Who was the first president of the Pakistan?
Completion items
The name of the first president of Pakistan is ___________.
The items can be answered by a work, phrase, number or symbol. Short-answer tests are a cross between essay and objective tests. The student must supply the answer as with an essay question but in a highly abbreviated form as with an objective question.
Advantages
Norman E. Gronlund (1990) writes that short-answer items have a number of advantages.
They reduce the likelihood that a student will guess the correct answer
They are relatively easy for a teacher to construct.
They are will adapted to mathematics, the sciences, and foreign languages where specific types of knowledge are tested (The formula for ordinary table salt is ________).
They are consistent with the Socratic question and answer format frequently employed in the elementary grades in teaching basic skills.
Disadvantages
According to Norman E. Grounlund (1990) there are also a number of disadvantages with short-answer items.
They are limited to content areas in which a student’s knowledge can be adequately portrayed by one or two words.
They are more difficult to score than other types of objective-item tests since students invariably come up with unanticipated answers that are totally or partially correct.
Short answer items usually provide little opportunity for students to synthesize, evaluate and apply information.
i. Select type.
ii. Supply type.
Select Type Items
Matching Items
According to W. Wiersma and S.G. Jurs (1990), the matching items consist of two parallel columns. The column on the left contains the questions to be answered, termed premises; the column on the right, the answers, termed responses. The student is asked to associate each premise with a response to form a matching pair. For example
According to W. Wiersma and S.G. Jurs (1990) in some matching exercises the number of premises and responses are the same, termed a balanced or perfect matching exercise. In others, the number and responses may be different.
Advantages
The chief advantage of matching exercises is that a good deal of factual information can be tested in minimal time, making the tests compact and efficient. They are especially well suited to who, what, when and where types of subject matter. Further students frequently find the tests fun to take because they have puzzle qualities to them.
Disadvantages
The principal difficulty with matching exercises is that teachers often find that the subject matter is insufficient in quantity or not well suited for matching terms. An exercise should be confined to homogeneous items containing one type of subject matter (for instance, authors-novels; inventions inventors; major events-dates terms – definitions; rules examples and the like). Where unlike clusters of questions are used to adopt but poorly informed student can often recognize the ill-fitting items by their irrelevant and extraneous nature (for instance, in a list of authors the inclusion of the names of capital cities).
Student identifies connected items from two lists. It is Useful for assessing the ability to discriminate, categorize, and association amongst similar concepts.
Suggestions for Writing Matching Items
Here are some suggestions for writing matching items:
i. Keep both the list of descriptions and the list of options fairly short and homogeneous – they should both fit on the same page. Title the lists to ensure homogeneity and arrange the descriptions and options in some logical order. If this is impossible, you’re probably including too wide a variety in the exercise. Try constructing two or more exercises.
ii. Make sure that all the options are plausible distracters for each description to ensure homogeneity of lists.
iii. The list of descriptions on the left side should contain the longer phrases or statements, whereas the options on the right side should consist of short phrases, words or symbols.
iv. Each description in the list should be numbered (each is an item), and the list of options should be identified by letter.
v. Include more options than descriptions. If the option list is longer than the description list, it is harder for students to eliminate options. If the option list is shorter, some options must be used more than once. Always include some options that do not match any of the descriptions, or some that match more than one, or both.
vi. In the directions, specify the basis for matching and whether options can be used more than once.
B. Multiple Choice Questions (MCQ’s)
Norman E. Grounlund (1990) writes that the multiple choice question is probably the most popular as well as the most widely applicable and effective type of objective test. Student selects a single response from a list of options. It can be used effectively for any level of course outcome. It consists of two parts: the stem, which states the problem and a list of three to five alternatives, one of which is the correct (key) answer and the others are distracters (“foils” or incorrect options that draw the less knowledgeable pupil away from the correct response).
The stem may be stated as a direct question or as an incomplete statement. For example:
Direct question
Which is the capital city of Pakistan? -------- (Stem)
A. Lahore. -------------------------------------- (Distracter)
B. Karachi. ------------------------------------- (Distracter)
C. Islamabad. ---------------------------------- (Key)
D. Peshawar. ----------------------------------- (Distracter)
Incomplete Statement
The capital city of Pakistan is
A. Lahore.
B. Karachi.
C. Islamabad.
D. Peshawar.
RULES FOR WRITING MULTIPLE-CHOICE QUESTIONS
1. Use Plausible Distracters (wrong-response options)
Only list plausible distracters, even if the number of options per question changes
Write the options so they are homogeneous in content
Use answers given in previous open-ended exams to provide realistic distracters
2. Use a Question Format
Experts encourage multiple-choice items to be prepared as questions (rather than incomplete statements)
Incomplete Statement Format:
The capital of AJK is in-----------------.
Direct Question Format:
In which of the following cities is the capital of AJK?
3. Emphasize Higher-Level Thinking
Use memory-plus application questions. These questions require students to recall principles, rules or facts in a real life context.
The key to prepare memory-plus application questions is to place the concept in a life situation or context that requires the student to first recall the facts and then apply or transfer the application of those facts into a situation.
Seek support from others who have experience writing higher-level thinking multiple-choice questions.
EXAMPLES:
Memory Only Example (Less Effective)
Which description best characterizes whole foods?
a. orange juice
b. toast
c. bran cereal
d. grapefruit
Memory-Plus Application Example (More Effective)
Sana’s breakfast this morning included one glass of orange juice (from Concentrate), one slice of toast, a small bowl of bran cereal and a grapefruit. What “whole food” did Sana eat for breakfast?
a. orange juice
b. toast
c. bran cereal
d. grapefruit
Memory-Plus Application Example
Ability to Interpret Cause-and-Effect Relationships Example
Why does investing money in common stock protect against loss of assets during inflation?
a. It pays higher rates of interest during inflation.
b. It provides a steady but dependable income despite economic conditions.
c. It is protected by the Federal Reserve System.
d. It increases in value as the value of a business increases.
Ability to Justify Methods and Procedures Example
Why is adequate lighting necessary in a balanced aquarium?
a. Fish need light to see their food.
b. Fish take in oxygen in the dark.
c. Plants expel carbon dioxide in the dark.
d. Plants grow too rapidly in the dark.
4. Keep Option Lengths Similar
Avoid making your correct answer the long or short answer
5. Balance the Placement of the Correct Answer
Correct answers are usually the second and third option
6. Be Grammatically Correct
Use simple, precise and unambiguous wording
Students will be more likely to select the correct answer by finding the grammatically correct option
7. Avoid Clues to the Correct Answer
Avoid answering one question in the test by giving the answer somewhere else in the test
Have the test reviewed by someone who can find mistakes, clues, grammar and punctuation problems before you administer the exam to students
Avoid extremes – never, always, only
Avoid nonsense words and unreasonable statements
8. Avoid Negative Questions
31 of 35 testing experts recommend avoiding negative questions
Students may be able to find an incorrect answer without knowing the correct answer
9. Use Only One Correct Option (Or be sure the best option is clearly the best option)
The item should include one and only one correct or clearly best answer
With one correct answer, alternatives should be mutually exclusive and not overlapping
Using MC with questions containing more than one right answer lowers discrimination between students
10. Give Clear Instructions
Such as:
Questions 1 - 10 are multiple-choice questions designed to assess your ability to remember or recall basic and foundational pieces of knowledge related to this course.
Please read each question carefully before reading the answer options. When you have a clear idea of the question, find your answer and mark your selection on the answer sheet. Please do not make any marks on this exam.
Questions 11 – 20 are multiple-choice questions designed to assess your ability to think critically about the subject.
Please read each question carefully before reading the answer options.
Be aware that some questions may seem to have more than one right answer, but you are to look for the one that makes the most sense and is the most correct.
When you have a clear idea of the question, find your answer and mark your selection on the answer sheet.
You may justify any answer you choose by writing your justification on the blank paper provided.
11. Use Only a Single, Clearly-Defined Problem and Include the Main Idea in the Question
Students must know what the problem is without having to read the response options
12. Avoid “All the Above” Option
Students merely need to recognize two correct options to get the answer correct
13. Avoid the “None of the Above” Option
You will never know if students know the correct answer
14. Don’t Use MCQ When Other Item Types Are More Appropriate
Limited distracters or assessing problem-solving and creativity
Advantages
The chief advantage of the multiple-choice question according to N.E. Gronlund (1990) is its versatility. For instance, it is capable of being applied to a wide range of subject areas. In contrast to short answer items limit the writer to those content areas that are capable of being stated in one or two words, multiple choice item necessary bound to homogeneous items containing one type of subject matter as are matching items, and a multiple choice question greatly reduces the opportunity for a student to guess the correct answer from one choice in two with a true – false items to one in four or five, there by increasing the reliability of the test. Further, since a multiple – choice item contains plausible incorrect or less correct alternative, it permits the test constructor to tine tune the discriminations (the degree or homogeneity of the responses) and control the difficulty level of the test.
Disadvantages
N.E. Gronlund (1990) writes that multiple-choice items are difficult to construct. Suitable distracters are often hard to come by and the teacher is tempted to fill the void with a “junk” response. The effect of narrowing the range of options will available to the test wise student. They are also exceedingly time consuming to fashion, one hour per question being by no means the exception. Finally they generally take student longer to complete (especially items containing fine discrimination) than do other types of objective question.
Suggestions for Writing MCQ’s Items
Here are some guidelines for writing multiple-choice tests:
I. The stem of the item should clearly formulate a problem. Include as much of the item as possible, keeping the response options as short as possible. However, include only the material needed to make the problem clear and specific. Be concise – don’t add extraneous information.
II. Be sure that there is one and only one correct or clearly best answer.
III. Be sure wrong answer choices (distracters) are plausible. Eliminate unintentional grammatical clues, and keep the length and form of all the answer choices equal. Rotate the position of the correct answer from item to item randomly.
IV. Use negation questions or statements only if the knowledge being tested requires it. In most cases it is more important for the student to know what a specific item of information is rather than what it is not.
V. Include from three to five options (two to four distracters plus one correct answer) to optimize testing for knowledge rather than encouraging guessing. It is not necessary to provide additional distracters from an item simply to maintain the same number of distracters for each item. This usually leads to poorly constructed distracters that add nothing to test validity and reliability.
VI. To increase the difficulty of a multiple-choice item, increase the similarity of content among the options.
VII. Use the option “none of the above” sparingly and only when the keyed answer can be classified unequivocally as right or wrong.
VII. Avoid using “all of the above”. It is usually the correct answer and makes the item too easy for students with partial information.
II. Supply Type Items
A. Completion Items
Like true-false items, completion items are relatively easy to write. Perhaps the first tests classroom teachers’ construct and students take completion tests. Like items of all other formats, though, there are good and poor completion items. Student fills in one or more blanks in a statement. These are also known as “Gap-Fillers.” Most effective for assessing knowledge and comprehension learning outcomes but can be written for higher level outcomes. e.g.
The capital city of Pakistan is -----------------.
Suggestions for Writing Completion or Supply Items
Here are our suggestions for writing completion or supply items:
I. If at all possible, items should require a single-word answer or a brief and definite statement. Avoid statements that are so indefinite that they may be logically answered by several terms.
a. Poor item:
Motorway (M1) opened for traffic in ____________.
b. Better item:
Motorway (M1) opened for traffic in the year______.
II. Be sure the question or statement poses a problem to the examinee. A direct question is often more desirable than an incomplete statement because it provides more structure.
III. Be sure the answer that the student is required to produce is factually correct. Be sure the language used in the question is precise and accurate in relation to the subject matter area being tested.
IV. Omit only key words; don’t eliminate so many elements that the sense of the content is impaired.
c. Poor item:
The ____________ type of test item is usually more _________ than the _____ type.
d. Better item:
The supply type of test item is usually graded less objectively than the _________ type.
V. Word the statement such that the blank is near the end of the sentence rather than near the beginning. This will prevent awkward sentences.
VI. If the problem requires a numerical answer, indicate the units in which it is to be expressed.
B. Short Answer
Student supplies a response to a question that might consistent of a single word or phrase. Most effective for assessing knowledge and comprehension learning outcomes but can be written for higher level outcomes. Short answer items are of two types.
Simple direct questions
Who was the first president of the Pakistan?
Completion items
The name of the first president of Pakistan is ___________.
The items can be answered by a work, phrase, number or symbol. Short-answer tests are a cross between essay and objective tests. The student must supply the answer as with an essay question but in a highly abbreviated form as with an objective question.
Advantages
Norman E. Gronlund (1990) writes that short-answer items have a number of advantages.
They reduce the likelihood that a student will guess the correct answer
They are relatively easy for a teacher to construct.
They are will adapted to mathematics, the sciences, and foreign languages where specific types of knowledge are tested (The formula for ordinary table salt is ________).
They are consistent with the Socratic question and answer format frequently employed in the elementary grades in teaching basic skills.
Disadvantages
According to Norman E. Grounlund (1990) there are also a number of disadvantages with short-answer items.
They are limited to content areas in which a student’s knowledge can be adequately portrayed by one or two words.
They are more difficult to score than other types of objective-item tests since students invariably come up with unanticipated answers that are totally or partially correct.
Short answer items usually provide little opportunity for students to synthesize, evaluate and apply information.
No comments:
Post a Comment