COMPUTER ADAPTIVE LANGUAGE TESTING (CALT) PROJECT

Idea and context. Against a background of deep transformations at the military and political stage of the state, which is accompanied by a great number of reforms concerning the implementation of NATO standards into all fields of the activities of the Armed Forces of Ukraine, advanced foreign language proficiency of military personnel becomes a topical task. Such a tendency caused the increase in the number of language testing according to NATO STANAG 6001 standard, which in its turn required the improvement of the abovementioned procedure. The implementation of the CALT project should help to solve this task.

The CALT project was initiated at the National Defence University of Ukraine named after Ivan Cherniakhovskyi in 2019, as the scientific project. Further development and support of the CALT project ware executed under the auspices of the NATO DEEP Ukraine. The kick-off meeting of the international format took place in February 2019 in Krakow. During the development process, the representatives from the University of Warmia and Mazury in Olsztyn, Polish Naval Academy in Gdynia, PfP Consortium ADL WG, Rakovski National Academy in Sofia, and NATO DEEP ADL Portal participated. Final coordination meeting took place on-line, in January 2021, as the NATO DEEP eAcademy event.

Principle of the CALT method/algorithm. CALT Method /Algorithm of test items formation is formed following the feedback principle when in case of the correct answer the next test item proposed is of a higher complexity level, and the wrong answer shows the necessity to propose easier test item comparing to the one answered incorrectly and vice versa. It follows that different candidates tested by CALT most likely answer the tasks of the language test of a different complexity level. Such an approach makes an essential difference between CALT and the traditional one, in which the main result index is “the number of correct answers”, unlike the adaptive testing. Thus, CALT allows giving the test-takers an essentially less number of test items keeping the diagnosing goal of the language test in general.

CALT advantages. The main utility of CALT is that it is shorter in time because the computer tests adapt to the proficiency level of the test-takers, and the test items which are above or below competence are not proposed, which allows spending less time for getting the precise information on the test taker's competence level. In its turn, reduction of time for language testing increases the motivation of the test-takers to pass the test, as they concentrate on the items of their complexity level. It allows the test-taker not to be bored answering easy questions and to feel frustrated passing too difficult test items. CALT makes it possible to show the results on the monitor after finishing the test that facilitates the work of persons holding the test who don’t need to spend time for works checking and assessment. It should be mentioned that the high-reliability coefficient of CALT provides the low probability of standard error in measurement (also defined as the standard deviation of error assessments). The CALT software supports the test developers as well through providing them a high level of security of the test results, opportunity to have immediate report feedback after each testing session and flexible navigation system. CALT software provides the distractors analysis and statistics report as well as allows conducting the items moderation sessions in an easy and effective way.