Three years have passed since the last IJSA Special Issue dealing with personal computer issues and computer-based testing (Bartram, 1995). Since that time general advances in personal computer (PC) hardware and software have, again, reaffirmed Grosch's Law—that with twice the money, one can buy (at least) four times the computational power of the previous technological generation (see Knight and Cerveny, 1976). In spite of faster microprocessors and more flexible and graphically-powerful software, however, progress in computer-based testing (CBT) has, generally speaking, not kept pace with such technological progress. Three reasons for this impasse will be offered below.

First, the pace of PC technological developments may be outstripping the ability of the CBT professional to keep up with such change. It seems just yesterday that the 'Pentium' wave crashed upon our technology shore; now experts are telling us that application-centric user interface (UI) paradigms are giving way to documentation-centric ones (Schlack, 1997), newer Java technology will enable programmers to more easily access databases on the Web (Linthicum, 1998), and speech recognition software has made sufficient advances to warrant its trial use in effecting voice-controlled documentation (Kay, 1998). All of these technological developments may very well help us in effecting what Bartram (1997) has termed 'distance assessment' which would represent a new rung on the CBT technological ladder. As promising as such developments may be, however, they first need to be understood and mastered as tools by the CBT specialist before they can be effectively and responsibly implemented for assessment purposes — a formidable challenge requiring a combination of skills more often found in an interdisciplinary team of experts versed in computer programming, psychometrics, user interface design issues and user psychology. Such resources, skills and competencies are not to be found everywhere.

Second, CBT is costly. Although CBT proponents often extoll the cost-reduction benefits of such technology, such extollings need to be taken with a grain of salt. For example, adaptive testing (AT) is often cited as being an ideal candidate for economical CBT as it can potentially result in considerably reduced testing time. Research shows, however, that AT doesn't always deliver what it promises. For example, AT can actually result in longer testing times, and because of what has been termed as 'technologically-linked presentation modality' effects (Kubinger, 1989), can result in consistently lower test performance as compared to conventional paper-testing — suggesting that AT is susceptible to certain particular validity problematics (see Booth, 1991). Going beyond time-saving issues and aspects of validity, however, additional costs of AT technology need to be calculated regarding actual operation cost — such as item pool development, updating of normative data, incrementing of the item pool, possible outside consultant fees etc. And given any form of CBT, regardless of whether one is using AT or not, one must also add the costs of HW and SW purchase, system maintenance, possible network down-time (regarding group-testing setups) and general system management. And if CBT involves few subjects (which, in private practice, is often the case, with only stand-alone testing being conducted), economy of scale of PC technology can be even less favorable: Diskettes get misplaced, clients may arrive late for testing, and why-didn't-we-order-toner-for-the-printer-last-week? woes may tarnish one's CBT cost-benefit ratio even more.

Third, CBT may have an image problem. In this regard, one must always consider the potential impact of a technology upon its user (here: the job applicant or person being tested) and the organization within which the technology is being used. In being enthusiastic about CBT one can easily overlook the fact that others may not be as enamored of such technology. For example, and to continue using AT as an illustrative technology, as a subset of CBT technology having special value for specific testing purposes under very specific testing conditions, the value of AT as an assessment alternative is, psychometrically established (see Booth, 1991). Its face validity, however, regarding its assessment legitimacy, so to speak, has been seriously challenged. Alan Bugbee (1996), for example, Director of the American College which has had extensive experience in CBT, refers to a voicing of concern as to whether or not AT is a proper method for all large-scale testing (National Council, 1991; as referenced by Bugbee), and also cites literature questioning the 'political acceptability' of AT (Jacobson, 1994; as referenced by Bugbee).

 To summarize: CBT will make more rapid and solid advances as a basic and applied discipline of psychometrics if

1. CBT research and development is effected via interdisciplinary teamwork involving experts in psychometrics, human psychology, UI design, system programming and, of course, test content experts. The days where one could plan, implement and deliver a complete CBT system single-handedly are a technological shadow of the past.

2. Purchasers and users of CBT technology need to be realistically aware of the effort and cost involved in the development of solidly-designed CBT systems, and that the true cost of implementing technology goes far beyond its initial purchase. CBT can definitely offer newer avenues of diagnostic approach as compared to more conventional (i.e., paper-pencil) technology. Eventually, however, we must find ways to reduce the cost of such technology regarding its development and use. Otherwise, CBT will remain primarily the domain of organizations having the infrastructure, personnel and budgets needed to support such systems (such as federal employment agencies and the military). In this regard, it might well be that newer technology such as the Internet (and/or intranet concept) may provide a platform for a democratization of CBT.

3. CBT professionals need to take the concerns of technology-critical others seriously. For example, union officials may be concerned regarding data security respective to test scores, testing officials may be worried about test security, psychologists may fear that such technology is making their learned skills obsolete. In this regard, the CBT professional must convincingly demonstrate that the use of such technology can be controlled and used competently and ethically; further, it is not intended on replacing solid, competent diagnostic thinking. Indeed, CBT is only a diagnostic alternative from which the contemporary psychodiagnostician can choose. Ultimately, however, the eventual success of CBT will be based upon its ability to help us help people and organizations in effecting efficient and economical valid psychodiagnostics. This Special Issue addresses these issues. (Booth, 1998, pp. 57-58.)

Booth, J. F. (1998). Guest Editorial. Special Issue: Uses of PC technology in selection and assessment. International Journal of Selection and Assessment , 6 (2), 57-60

Cited Literature

Bartram, D. (1995). Guest Editorial: Special Issue: Computer-based testing. International Journal of Selection and Assessment, 3, (2), 73-74.

Bartram, Dave. (1997). Distance Assessment: Psychological assessment through the internet. Proceedings of the British Psychological Society, Occupational Psychology Conference, Blackpool.

Booth, J. F. (1991). The key to valid computer-based testing: The User Interface. Revue européenne de Psychologie Appliquée (European Review of Applied Psychology), 41 (4), 281-293.

Bugbee, A. C. (1996). The equivalence of paper-and-pencil and computer-based testing. Journal of Research on Computing in Education, 28, (3), 282-299.

Jacobson, R. L. (1994). Computerized testing runs into trouble. The Chronicle of Higher Education, (August 3), A16-A17.

Kay, R. (1998). Do you hear what I say? Byte, 1, 115-116.

Knight, K.E.and Cerveny, R. P. (1976) Grosch's Law. In A. Ralston and C. L.Meek (Eds.) Encyclopedia of Computer Science. (First edition). New York: Van Nostrand, 599.

Kubinger, K.D. (1989) Aktueller Stand und kritische Würdigung der Probabilistischen Testtheorie. [Present status and critical evaluation of probabilistic test theory.] In Kubinger, K.D. (Hg) Moderne Testtheorie. Weinheim: Beltz.

Linthicum, D.S. (/1998). Java evolves. Byte , 1, 60.

National Council of State boards of Nursing. (1991). Collected works on the legal aspects of computerized adaptive testing. Chicago: Author.

Montgomery, J., (1998). The next Windows. Byte, 1 , 56.

Schlack, M. (1997). Why interfaces matter. Byte, 7, 12.

Tober, B. The quality factor. (1997). Byte, 4, 15-22