Notice: Adaptive Testing
Notice: Adaptive Testing
To all CNE and CNI certification candidates:
In November 1991, the Novell Testing Program will begin using
computerized adaptive testing technology. The first two adaptive
tests are the DOS/Microhardware Combined Test (50-15) and the
Networking Technologies Test (50-80 and 50-81). Eventually,
adaptive testing will be used to deliver most of Novell's
certification tests. Although adaptive testing has been used by
various government agencies, professional organizations and school
districts for more than a decade, it is still an unfamiliar concept
to most people. This memo is intended to explain adaptive testing,
point out its merits and describe how adaptive testing will affect
Novell tests.
Adaptive Testing vs. Traditional Testing
Traditional paper-and-pencil or computerized tests deliver the same
set of questions to every examinee, regardless of the examinee's
level of knowledge or proficiency. Your score on a traditional
test depends on the number of questions you answer correctly. If
you know a great deal about a topic, you can expect to answer more
questions correctly than someone who knows less. Traditional tests
have a long history and have proven very useful; nevertheless, a
traditional test is not the most efficient way to measure your
knowledge or proficiency.
Imagine that you are a teacher giving an oral exam to one of your
students. You might begin by asking a question of moderate
difficulty. If the student answers correctly, you follow with a
more difficult question; if he or she answers incorrectly, you make
the next question easier. You might continue asking questions
adapting the difficulty of subsequent questions based on the
student's answers to earlier questions. Within a short time you
would probably have a good idea of the student's level of knowledge
and you would not spend time asking questions you knew were too
easy or too difficult. Your judgment about the student's level of
knowledge would probably not be based on the number of correct
answers, but on the difficulty of the questions the student could
answer correctly compared to those he or she could not.
An adaptive test works something like a good oral exam. It begins
by giving you a question of moderate difficulty. After you
respond, the test scores your response and estimates your level of
ability. It then looks for a question whose difficulty matches
your presumed ability. The test presents that question, you
respond, the program scores your response and revises its estimate
of your ability based on the two questions given thus far. This
process continues, with the program revising its estimate of your
ability following each response. The estimate becomes more and
more accurate with each question. The accuracy of the ability
estimate is also calculated following each question. The test
concludes when a) your ability is estimated with sufficient
accuracy, b) the program is at least 95 percent confident that your
ability score lies somewhere above the passing score, or c) the
maximum number of questions is given.
Your Adaptive Test Score
Because an adaptive tests tailors itself to your ability, it is
possible that someone who is less proficient than you would answer
a similar number of questions correctly. Thus, counting the number
of correct answers is not a good way to score an adaptive test.
Adaptive test scores are based on the difficulty of the questions
you answer correctly and incorrectly. In other words, the score is
based on your ability, rather than simply the number of items you
get right.
Novell has adopted an ability score scale that has a minimum value
of 200 and a maximum value of 800. All test scores will fall
somewhere on this scale. For each test, a passing score will be
established somewhere on the scale. You will pass the test if your
ability score is equal to or greater than the passing score.
Section Scores
In the past, Novell tests have reported the percentage of questions
answered correctly from each section of the test. This information
has been useful, helping examinees understand their relative
strengths and weaknesses.
Novell adaptive tests will provide section scores as well; however,
section scores will be calculated differently than they are for
traditional tests because lower-ability examinees often answer as
many items correctly as higher-ability examinees. When your
adaptive test is finished, your final ability score will be used to
estimate the number of questions you would have answered correctly,
by section, if you were given all the questions in the adaptive
test's item pool. These estimated section scores, while only
estimates, should still be useful.
The Advantages of Adaptive Testing
The two main advantages of adaptive testing over traditional
testing are accuracy and efficiency. Questions that are too easy
or too difficult provide little information about your true
ability. By minimizing the number of such questions, adaptive
testing can accurately measure your ability with fewer questions,
saving you valuable time. Adaptive tests are as accurate as longer
traditional tests for people of average ability; for people of
higher and lower ability, adaptive tests are usually even more
accurate.
Adaptive tests not only save you time, they can also save you from
the boredom or intimidation of questions that are too easy or too
hard. You should find that most of the items you receive are
moderately challenging, helping to make the testing session more
interesting and less stressful.
To all CNE and CNI certification candidates:
In November 1991, the Novell Testing Program will begin using
computerized adaptive testing technology. The first two adaptive
tests are the DOS/Microhardware Combined Test (50-15) and the
Networking Technologies Test (50-80 and 50-81). Eventually,
adaptive testing will be used to deliver most of Novell's
certification tests. Although adaptive testing has been used by
various government agencies, professional organizations and school
districts for more than a decade, it is still an unfamiliar concept
to most people. This memo is intended to explain adaptive testing,
point out its merits and describe how adaptive testing will affect
Novell tests.
Adaptive Testing vs. Traditional Testing
Traditional paper-and-pencil or computerized tests deliver the same
set of questions to every examinee, regardless of the examinee's
level of knowledge or proficiency. Your score on a traditional
test depends on the number of questions you answer correctly. If
you know a great deal about a topic, you can expect to answer more
questions correctly than someone who knows less. Traditional tests
have a long history and have proven very useful; nevertheless, a
traditional test is not the most efficient way to measure your
knowledge or proficiency.
Imagine that you are a teacher giving an oral exam to one of your
students. You might begin by asking a question of moderate
difficulty. If the student answers correctly, you follow with a
more difficult question; if he or she answers incorrectly, you make
the next question easier. You might continue asking questions
adapting the difficulty of subsequent questions based on the
student's answers to earlier questions. Within a short time you
would probably have a good idea of the student's level of knowledge
and you would not spend time asking questions you knew were too
easy or too difficult. Your judgment about the student's level of
knowledge would probably not be based on the number of correct
answers, but on the difficulty of the questions the student could
answer correctly compared to those he or she could not.
An adaptive test works something like a good oral exam. It begins
by giving you a question of moderate difficulty. After you
respond, the test scores your response and estimates your level of
ability. It then looks for a question whose difficulty matches
your presumed ability. The test presents that question, you
respond, the program scores your response and revises its estimate
of your ability based on the two questions given thus far. This
process continues, with the program revising its estimate of your
ability following each response. The estimate becomes more and
more accurate with each question. The accuracy of the ability
estimate is also calculated following each question. The test
concludes when a) your ability is estimated with sufficient
accuracy, b) the program is at least 95 percent confident that your
ability score lies somewhere above the passing score, or c) the
maximum number of questions is given.
Your Adaptive Test Score
Because an adaptive tests tailors itself to your ability, it is
possible that someone who is less proficient than you would answer
a similar number of questions correctly. Thus, counting the number
of correct answers is not a good way to score an adaptive test.
Adaptive test scores are based on the difficulty of the questions
you answer correctly and incorrectly. In other words, the score is
based on your ability, rather than simply the number of items you
get right.
Novell has adopted an ability score scale that has a minimum value
of 200 and a maximum value of 800. All test scores will fall
somewhere on this scale. For each test, a passing score will be
established somewhere on the scale. You will pass the test if your
ability score is equal to or greater than the passing score.
Section Scores
In the past, Novell tests have reported the percentage of questions
answered correctly from each section of the test. This information
has been useful, helping examinees understand their relative
strengths and weaknesses.
Novell adaptive tests will provide section scores as well; however,
section scores will be calculated differently than they are for
traditional tests because lower-ability examinees often answer as
many items correctly as higher-ability examinees. When your
adaptive test is finished, your final ability score will be used to
estimate the number of questions you would have answered correctly,
by section, if you were given all the questions in the adaptive
test's item pool. These estimated section scores, while only
estimates, should still be useful.
The Advantages of Adaptive Testing
The two main advantages of adaptive testing over traditional
testing are accuracy and efficiency. Questions that are too easy
or too difficult provide little information about your true
ability. By minimizing the number of such questions, adaptive
testing can accurately measure your ability with fewer questions,
saving you valuable time. Adaptive tests are as accurate as longer
traditional tests for people of average ability; for people of
higher and lower ability, adaptive tests are usually even more
accurate.
Adaptive tests not only save you time, they can also save you from
the boredom or intimidation of questions that are too easy or too
hard. You should find that most of the items you receive are
moderately challenging, helping to make the testing session more
interesting and less stressful.
Comments
Post a Comment