Steven Ritter
Department of Psychology
Carnegie Mellon University
Pittsburgh, PA 15213
USA
Phone : (412) 268-3498
Fax: (412) 268 2844
E-mail:
sritter@cmu.edu
Abstract: PAT Online is a variant of the PAT Algebra Tutor, a model-tracing tutor being used in several middle, high school and college classes. This paper discusses the applicability of desktop-based architectures to tutors deployed on the World-Wide Web. We concentrate on technical aspects involved in implementing a model-tracing tutor on the World-wide Web.
The World-Wide Web is becoming an increasingly important mechanism for delivering educational software. The rapid rise of the World-Wide Web has left many educational software developers in the difficult position of reconsidering architectures that were developed with the assumption of a stand-alone computer or local area network. As internet-based systems become more prevalent, educational software developers need to consider ways in which they can preserve the educational effectiveness shown with more traditional architectures while taking advantage of the benefits of moving to an internet-based architecture. In this paper, I describe our experience in moving a version of the PAT Algebra Tutor from a stand-alone system to an internet-based system. Our emphasis in this process was on reusing as much of the work that had already been done.
The PAT tutor contains an expert system capable of solving the problems that are posed to students. As students take steps to complete the problem (for example, by filling in cells in the spreadsheet), the tutor considers whether or not those steps are consistent with a direct solution to the problem. If so, the tutor remains silent, and the student proceeds. If the student's action is not recognized as being on some solution path, the tutor checks to see if the step is consistent with a common misconception (or "bug"). In such cases, the tutor is able to provide instruction tailored to that bug. Since the tutor is tracking the student's solution at each step, the tutor is able to give help associated with the student's solution path. The expert system operating within the tutor is referred to as the "cognitive model." Each rule in the cognitive model represents a component of the skill required to perform the task. In this way, rules can be mapped to cognitive skills as well as student actions. Thus, each student action provides a bit of information about the student's state of knowledge. Anderson and Corbett (1992) describe the specific algorithm used to update the tutor's estimate of the student skill, based on the student's actions. The tutor's assessment of a student's knowledge is available to the student at all times, through a graph known as the "skillometer."
Previous work with tutoring systems of this type (e.g. Anderson, Conrad and Corbett, 1993) have shown that learning takes place at the level of these cognitive skills, and the tutor measures the progress of students with reference to these underlying skills. The process of modeling student knowledge and solutions in this manner is known as "model tracing" (Anderson, Boyle, Corbett, & Lewis, 1990).
The PAT tutor was developed using the Tutor Development Kit (TDK; Anderson and Pelletier, 1991), which is a Lisp-based system that includes facilities for developing the cognitive model, defining the user interfaces and specifying a mapping between the interfaces and the cognitive model. The TDK also provides facilities ordering student exercises, and for storing information about students between sessions. All of these facilities are provided in a tightly-integrated manner within the Lisp environment.
The plug-in architecture defines a strict communication protocol between a tool (which corresponds to the set of objects that the user can manipulate) and a tutor (which is responsible for the analysis of student actions and the generation of feedback). A "translator" element is responsible for mediating the communication between the tool and the tutor agent. A "curriculum manager" component is responsible for storing student records and for sequencing student activities. The plug-in component architecture is illustrated in Figure 1.
Based on the plug-in architecture, we developed a translator, tutoring agent and curriculum manager in C++ and used AppleEvents to handle the messaging between components. By modifying the translator slightly, we were able to link the same tutoring agent and curriculum manager to three functionally-equivalent tools: a spreadsheet tool built with the SK8 programming language, a spreadsheet written in C++ and Microsoft Excel. The translator ensures that any differences between the tools are not seen by the tutoring agent. Modification of the translator in this fashion is not a difficult task. Ritter and Blessing (in press) describe a visual authoring tool for the translator component of this system.
Figure 1: Plug-in component architecture
In the PAT Online system, the "tool" is an HTML form, and this form input needs to be interpreted by a Common Gateway Interface (CGI) program. The CGI program basically parses information about the values that the user has entered into the form and passes it on to the translator. The translator, in this system, has a more difficult translation task than in other plug-in-based systems. The tutor's commands to the translator consist of directions to modify the state of the interface. For example, the tutor might direct the interface to display a particular bit of help text. In other systems, this command would directly result in a message of some form to the interface. In an HTML form-based system, the entire interface must be described in a block of HTML commands. The translator, then, must construct the interface corresponding to all of the commands given by the tutor in a particular cycle.
Figure 2: PAT Online Architecture
In the classroom version of the tutor, students can ask for help on any available action. To do this, they select an item (for example, a cell in the spreadsheet) and then push the "help" key. The tutor detects which cell is currently selected, and it uses that information to direct the help. When HTML form information is submitted, no information about the selected cell is available. Thus, to allow students to ask for help on any item, we added help buttons next to each of the cells in the spreadsheet. Information about which help button was selected is transmitted to the tutor, which then gives help on the appropriate item.
Use of an HTML form allowed us to add formatting and images to PAT Online which are not available in the classroom version of the tutor. This is all handled through the translator. When the translator receives a request to start a problem, it looks for an HTML file with that problem name. The HTML file can contain all of the usual HTML commands, allowing for the inclusion of all kinds of images and formatting. The translator appends commands to this file to build the current view of the spreadsheet. Sometime, help commands wish to illustrate the connection between aspects of the problem statement and the desired answer. For example, if the student asks for help in entering a unit of measure, the word "year" in the text might be highlighted. In these cases, the tutor agent describes "text pointers" which are simply character ranges in the text to highlight. The HTML file representing the problem contains comment lines indicating the beginning and end of crucial segments of the problem statement (the introduction, each of the questions and the conclusion). With this information, the translator is able to find the correct segments of the text to be highlighted and to add HTML commands to perform the highlighting before passing the file to the user.
Another interesting interface design decision involved the display of "bug" messages. In the plug-in architecture, these messages are "attached" to particular elements in the interface. In the Excel version of the tutor, for example, a message relating to the student's answer for cell R1C1 would be placed in a "note" for that cell. In Excel, cells with attached notes appear with a red square. The user can view the note by selecting the cell and calling up the "notes" window. Similar mechanisms were used in the other spreadsheet tools. The PAT Online design needed to respond to several design considerations. First, since users can complete more than one cell before getting feedback, they may receive more than one bug message. Second, there is no element in HTML which would allow us the kind of interaction we used with Excel. Our solution was to present bug messages as "footnotes." When a cell has a bug message attached, a number is drawn next to it. At the bottom of the screen, that number appears along with the bug text. This is a nice illustration of how the high-level "attach message" command defined by the plug-in architecture can be implemented in very different ways, depending on the requirements and capabilities of the tool.
Figure 3: The PAT Online tutor (full-size)
In one version of PAT Online, the skillometer was generated by sending AppleScript commands to Microsoft Excel running on the server. These commands insert the appropriate data, direct Excel to draw a graph, copy that graph out of Excel and save it as a GIF file (using the Clip2GIF program to change the graphics format). While this procedure was serviceable, it slows the response time considerably, so we have added the ability to use a simple Java-based skillometer.
More specifically, when a user logs on to the PAT Online system, the translator directs the tutor agent to create a tutor object with the student's name. The name is encoded in the HTML forms as a hidden field, so that any input from a form can be identified with that student. The input is then forwarded to the appropriate tutor object, which is directed to model-trace the student's action(s).
This solution also helps get around some of the problems introduced by the web's "stateless" nature. The server stores tutor objects for all active students, and the tutor objects maintain the state. After some period of inactivity (usually set to 1/2 hour), tutor objects "expire." Expiration involves storing information about the student in a file on disk (though the student record server) and clearing the tutor object out of the tutor application's memory. If the student returns at a later time, records can be read out of the student file and into memory, restoring the last state of the student.
In order to allow interaction with other tutors on the World-Wide Web (see Brusilovsky, Ritter and Schwarz, 1997) and to allow easier access to student records, the student record server was designed as a CGI program in itself. This way, an HTML form system separate from PAT Online itself can access student records. Similarly, other systems can access the CGI by sending it commands mimicking form input. The student record server thus has two APIs: one through the CGI mechanism (used by forms and remote tutors) and one through AppleEvents (used by the PAT Online tutor).
The PAT Online system provides a good illustration of the flexibility of the plug-in tutor architecture. With only minor modifications, this architecture allowed us to redirect the tutoring knowledge contained in a tutor agent designed to run in a non-networked environment and use in on the World-Wide Web. Still, the system would benefit from more precise and targeted interaction. The use of Java-based programs for the interfaces would allow the kind of interaction achieved in the classroom version of the tutor, but the system would still be hampered by the potential delay in continually sending messages between tool and translator across the internet. A more satisfactory but longer-term solution would be to re-write all of the components of the tutor, with the exception of the student record server, in Java. This would allow quick interaction between all segments of the system as well as centralized record storage. We are currently at work on such a system.
Anderson, J. R., Conrad, F. G. and Corbett, A. T. (1993). The Lisp tutor and skill acquisition. In J. R. Anderson, (Ed.), Rules of the Mind (pp. 143-164). Hillsdale, NJ: Lawrence Erlbaum Associates.
Anderson, J.R. and Corbett, A.T. (1993). Tutoring of cognitive skill. In J.R. Anderson, Rules of the Mind (pp. 235-255). Hillsdale, NJ: Erlbaum.
Anderson, J. R., & Pelletier, R. (1991). A development system for model-tracing tutors. In Proceedings of the International Conference of the Learning Sciences (pp. 1-8). Evanston, IL.
Brusilovsky, P., Ritter, S. and Schwarz, E. (1997). Distributed intelligent tutoring on the web. Proceedings of the Eighth World Conference on Artificial Intelligence in Education. Kobe, Japan.
Corbett, A. T. and Anderson, J. R. (1992). Student modeling and mastery learning in a computer-based programming tutor. In C. Frasson, G. Gauthier and G. McCalla (Eds.), Intelligent Tutoring Systems: Second international conference proceedings (pp. 413-420). New York: Springer-Verlag.
Koedinger, K. R., Anderson, J. R.., Hadley, W. H.., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8.
National Council of Teachers of Mathematics (1989). Curriculum and Evaluation Standards for School Mathematics. Reston, VA: The Council.
Ritter, S. and Blessing, S. B. (in Press). Authoring tools for component-based learning environments. Journal of the Learning Sciences, to appear.
Ritter, S. and Koedinger, K. R. (1996). An architecture for plug-in tutor agents. Journal of Artificial Intelligence in Education, 7, 315-347.