Proceedings of the workshop "Intelligent Educational Systems on the World Wide Web",
8th World Conference of the AIED Society, Kobe, Japan, 18-22 August 1997

Abstract. Education and training are multi-billion dollar concerns: the U.S. Department of Defenses shrinking budgets and increasingly diverse missions dictate that knowledge can be imparted in a far more effective manner than is currently available. Network-based training and intelligent tutoring mechanisms address not only these concerns, but also accommodate the militaries increasingly dispersed, reserve personnel. This paper presents our work on developing authoring tools and materials for intelligent tutoring systems.

This paper is organized in the following manner. First, we present the training domain and the motivations for developing our tools. Next, we present an architectural overview of the system. We then delve into findings of developing the system and some results of experimenting with the system. Finally, we illustrate the future directions of our work.

Introduction

The U.S. military is particularly interested in distance learning to address its multi-billion per year training expenses. Distance learning and WWW-based solutions facilitate the training of reserve personnel, personnel in remote locations, troops being transported to remote locations, and personnel stationed in inaccessible locations (e.g., undersea). These problems are analogous to educating students on subjects that cannot be found on-site; few universities, for example, maintain degreed professionals to teach all the language courses that are offered (e.g., Japanese). The National Guard compromising 52% of the U.S. military personnel require the same training as full-time military personnel, but have less time for rehearsal, less intermittent feedback, and more expensive training costs. The use of intelligent tutoring systems -- to increase the effectiveness of training by tailoring it to students' backgrounds and knowledge -- enables making the most of increasingly tighter training budgets. 98% of students receiving individualized instruction outperform those receiving computer based training [Cannuso96]. Network-based training dramatically reduces the cost of both developing and disseminating training.

World Wide Web (WWW)-based training and intelligent tutoring systems (ITS) are an excellent marriage of advanced technologies. WWW browsers overcome many of the shortcomings of traditional CBT by affording platform independent, easily updated training materials; existing CBT materials (developed using commercial authoring software) often work only on Windows platoforms configured uniquely for the training materials.

Intelligent tutoring systems often meld products of different areas of research to provide tailored instruction addressing the particular shortcomings of each student's performance (and hence knowledge). ITS in the literature regularly affect or are affected by research in areas such as natural language processing [LP96], planning [HJ94], knowledge acquisition [BM96], animated figure presentation [SL96], and simulated agents [EW96]. The asynchronous nature of ITS make them often ideal candidates for use over the WWW, since it facilitates offering instruction in the domain to anywhere, to anyone, at any time. However, a number of shortcoming in current ITS technology impedes their widespread use.

1. Intelligent tutoring systems are often very bandwidth intensive.
2. The cost of developing the multimedia required for many ITS applications can be prohibitive.
3. The cost of developing the knowledge base can be costly. Large knowledge bases are often required for supporting ITS.
4. The large knowledge base required by such systems is difficult to develop. This issue is closely related to issue (3). Since many tools are often made by AI specialists for AI specialists, widespread acceptance and use is understandably small.

Issue (2) is currently being actively examined by the multimedia reuse and multimedia understanding community [KL96]. A large number of very talented researchers are addressing issue (3); DARPA's High Performance Knowledge Base initiative should stimulate great progress in the area. While a number of projects have addressed issues (1) and (4), including TOPKAT [CRFL95] and Protégé II [SM96], it is in these areas that our work has made the most progress [GM97].

Architecture and Components

The overall goal of our system is to provide a huge advance in the state-of-the-art of intelligent tutoring systems; over a two-year period, we are creating authoring tools to dramatically reduce the skill level (knowledge) required by domain experts to author such systems [GC97]. The automated generation of remediation is the less movel portion of our work in intelligent tutoring systems. The more novel, critical goal of our system is to adapt training materials to address the needs of individual students (Figure 1). Such tailoring is particularly critical for multi-national, multi-force efforts; a soldier in the Italien Army may need to work with a fighter pilot from the Israelli Air Force for air defense. The training materials themselves are contained in an entirely object-oriented manner to facilitate such tailoring by components of our architecture (part of which is shown in Figure 2).

The training materials are represented as a collection of knoweldge bases. These knowledge bases are orthogonal to the domain knowledge bases. The individual objects are stored as frames using WebExpert ™. The tool is a sophisticated, WWW-based implementation of NASA's C Language Integrated Production System (CLIPS). However, the tool has been enhanced with to provide database support, creating graphics and animation, etc., for better World Wide Web interaction as well as backward-chaining (for goal-directed reasoning). WebExpert is ideal for implementing reasoning systems over the Intenret.

The instructional designer adapts existing classes of instructional materials to suite his own needs. For example, one can specialize the general purpose mathematical exercise class to create a class of electrical cicruit word problems (Figure 3). Each of the classes has corresponding rule bases and methods for their use in actual instructional materials: the exam generator has modules for creating questions, grading questions, and checking for similarty of of answers among students (i.e., cheating).

Materials are adapted to individual students through their profiles and planning software. As individual lesson components are selected, many of them are adapted to students based upon constraints attached to the materials. A student's performance with using the materials, in turn, updates his profile.

Security is also a key concern in our system. Because significant decisions can be made based-upon the training (e.g., promotion), we must authenticate users to (1) discourage cheating and (2) prohibit a student from "trashing" another. Hence, a significant part of our effort has gone to ensuring that state information cannot be readily examined or modified by students.

Much of our work can be examined from http://www.netaxs.com/~intelli. This link will shortly have a more formal name, but the longer form willl, of course, still work.

Results

Many of the components are still being constructed; system integration and testing are scheduled to begin during the 1997-98 academic year. However, a number of interesting items have been glean simply from component testing.

(defclass Math-Exercise-Page     ; Defining a mathematical exercise
       (is-a Exercise-Page)   ; This type of page inherits from an exercise
   (pattern-match reactive)   ; The exercises can be pattern-matched
   (role concrete)                ; And one can instantiate math exercises
   (slot page-type (default math-exercise)  ; The page's type
      (create-accessor read-write) (visibility public)) 
   (slot link-next-when-done                    ; Where to go when the student
      (create-accessor read-write) (visibility public)) ; finishes the exercise
   (multislot formula     ; The formulae used on the page
      (create-accessor read-write))
   (multislot response-to-correct   ; How to repsond to right answers
      (create-accessor read-write))
   (multislot response-to-incorrect ; How to responde to wrong answers
      (create-accessor read-write))
   (message-handler draw)   ;The page can be "drawn" in HTML
)

One of the most obvious areas for controversy is the utility of dynamically created lesson materials: materials must be created dynamically to facilitate tutoring at any level more sophisticated than a decision tree. However, dynamically created materials pose a number of challenges that have largely been overlooked in current technology.

1. Dynamically created materials are not "searchable" by current WWW "spider" technology. Hence, our pages and demonstrations are difficult to have cataloged by sites such as Lycos, Excite, etc.
2. The time required for creating materials is proportional to the amount of search (planning) required by the system. This factor has led us to dramatically complicate the models shown above in order to utilize hierarchy to combat these problems. However, a great deal of work is still required to tailor materials more efficiently.
3. Dynamic materials must also be inhibited at times. A particularly acute instance of this involves testing. Given that a student has started an examination, methods required to ensure that he tries the same questions only once, with a limited time to ascertain their answers. Some approaches used include static exams, files of questions, files of answers, etc. However, side effects also pose problems given the stateless nature of the WWW. Browser differences, communications problems, etc., further exacerbate these issues.

Conclusions and Future Work

Network-based training and education offers a number of important benefits. However, the wider audience enabled by remote instruction also poses difficulties with its wider audience. Intelligent tutoring systems offering tailored instruction should greatly improve the utility of such instruction by tailoring the instruction to each student receiving instruction. Previous research has repeatedly shown that tailoring and remediation improve student retention of knowledge.

This paper has only really superficially described our system for brevity's sake: our contract is charged with providing a "revolutionary advance" in intelligent tutoring systems. As such the future direction of our work involves advanced user interfaces embedding human factors knowledge bases. We invite interested readers to contact us and examine our World Wide Web site for demonstrations and further details.

References

[BF96] Baffes, P. and Mooney, R., 1996. A Novel Application of Theory Refinement to Student Modeling, in Proc. of the Thirteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park, CA, 1996, 403-408.

[CRFL95] Y. Chen-Burger, D. Robertson, J. Fraser, C. Lissoni, KBST: A Support Tool for Business Modelling in BSDM, Applications and Innovations in Expert Systems, Proceedings of the 15th Annual Conference of the British Computer Society Specialist Group on Expert Systems (SGES), Cambridge, UK, December 1995

[EW96] R. Elliot III and B. Woolf, A Simulation-based Tutor that Reasons about Multiple Agents, in Proc. Thirteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park, CA , 1996. Pp. 409-415.

[GM97] D. Goldstein and J. Montgomery, Lowering Lifecycle Costs of Training via Networks, in Proc. of the 19^th Interservice/Industry Training Systems and Education Conference, American Defense Preparedness Association, Orlando, FL, December, 1997.

[GC97] D. Goldstein and H. Clark, Hybrid Tutoring: A New Paradigm for Instruction, in Proc. of the 19^th Interservice/Industry Training Systems and Education Conference, American Defense Preparedness Association, Orlando, FL, December, 1997.

[HJ94] Hill, Randall W., Jr., and W. Lewis Johnson, "Situated Plan Attribution for Intelligent Tutoring." Paper presented to the 12^th National Conference on Artificial Intelligence, Seattle,August 1994, pp. 499-505.

[KL96] C. Knoblock and A. Levy, Information Gathering and Integration, Thirteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park, CA, 1996.

[Merrill96] D. Merrill, Instructional Transaction Theory: Instructional Design based on Knowledge Objects, Educational Technology, May/June 1996, pp. 30-37.

[MS91] Merrill, David and J.M. Spector. "Designing an Advanced Instructional Design Advisor: Transaction Shell Theory, Volume 6." MEI Associates, Inc., Lexington MA, December 1991.

[SL96] B. Stone and J. Lester, Dynamically Sequencing an Animated Pedagogical Agent, in Proc. Thirteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park, CA , 1996, pp. 424 - 432.

[SM96] Y. Shahar and H. Molina, Knowledge-Based Spatiotemporal Abstraction, KSL-96-12, Stanford University, Palo Alto, 1996.