Proceedings of the workshop "Intelligent Educational Systems on the World Wide Web",
8th World Conference of the AIED Society, Kobe, Japan, 18-22 August 1997

PAT Online: A Model-tracing tutor on the World-wide Web

Steven Ritter
Department of Psychology
Carnegie Mellon University
Pittsburgh, PA 15213 USA
Phone : (412) 268-3498
Fax: (412) 268 2844

Abstract: PAT Online is a variant of the PAT Algebra Tutor, a model-tracing tutor being used in several middle, high school and college classes. This paper discusses the applicability of desktop-based architectures to tutors deployed on the World-Wide Web. We concentrate on technical aspects involved in implementing a model-tracing tutor on the World-wide Web.

1 Introduction

The World-Wide Web is becoming an increasingly important mechanism for delivering educational software. The rapid rise of the World-Wide Web has left many educational software developers in the difficult position of reconsidering architectures that were developed with the assumption of a stand-alone computer or local area network. As internet-based systems become more prevalent, educational software developers need to consider ways in which they can preserve the educational effectiveness shown with more traditional architectures while taking advantage of the benefits of moving to an internet-based architecture. In this paper, I describe our experience in moving a version of the PAT Algebra Tutor from a stand-alone system to an internet-based system. Our emphasis in this process was on reusing as much of the work that had already been done.

2 The PAT Algebra Tutor

The PAT Online tutor is a variant of the PAT algebra 1 tutor described in Koedinger, Anderson, Hadley and Mark (1995). In the 1996/97 school year, this tutor is in daily use in 22 high schools, 4 middle schools and 2 college classrooms. Typically, the tutor is used in conjunction with a class curriculum designed to reflect the recommendations of the National Council of Teachers of Mathematics (NCTM; 1989). These guidelines emphasize the importance of students being able to understand and appreciate the fundamental mathematics underlying real-world situations. They expect students to be able to understand the mathematical equivalence between different representations and to be able to map between them. That is, students are expected to understand the relationship between a problem situation, an equation, a graph and a table of values. These mappings are emphasized both within the tutor and in classwork. In a typical exercise in the tutor, students are asked to create a spreadsheet and graph representing the problem situation and to create algebraic expressions and solve equations relating to the problem situation.

The PAT tutor contains an expert system capable of solving the problems that are posed to students. As students take steps to complete the problem (for example, by filling in cells in the spreadsheet), the tutor considers whether or not those steps are consistent with a direct solution to the problem. If so, the tutor remains silent, and the student proceeds. If the student's action is not recognized as being on some solution path, the tutor checks to see if the step is consistent with a common misconception (or "bug"). In such cases, the tutor is able to provide instruction tailored to that bug. Since the tutor is tracking the student's solution at each step, the tutor is able to give help associated with the student's solution path. The expert system operating within the tutor is referred to as the "cognitive model." Each rule in the cognitive model represents a component of the skill required to perform the task. In this way, rules can be mapped to cognitive skills as well as student actions. Thus, each student action provides a bit of information about the student's state of knowledge. Anderson and Corbett (1992) describe the specific algorithm used to update the tutor's estimate of the student skill, based on the student's actions. The tutor's assessment of a student's knowledge is available to the student at all times, through a graph known as the "skillometer."

Previous work with tutoring systems of this type (e.g. Anderson, Conrad and Corbett, 1993) have shown that learning takes place at the level of these cognitive skills, and the tutor measures the progress of students with reference to these underlying skills. The process of modeling student knowledge and solutions in this manner is known as "model tracing" (Anderson, Boyle, Corbett, & Lewis, 1990).

The PAT tutor was developed using the Tutor Development Kit (TDK; Anderson and Pelletier, 1991), which is a Lisp-based system that includes facilities for developing the cognitive model, defining the user interfaces and specifying a mapping between the interfaces and the cognitive model. The TDK also provides facilities ordering student exercises, and for storing information about students between sessions. All of these facilities are provided in a tightly-integrated manner within the Lisp environment.

3 The Plug-in Tutor Architecture

More recently, we have been reconsidering this architecture. Although the tightly integrated architecture of the PAT tutor enforces consistency between the components (a user action can result in a direct change to the working memory representation, for example), it can be difficult to incorporate different components into this architecture. In particular, we wished to be able to integrate commercial, off-the-shelf tools into our system. This would provide students with the ability to work with industry-standard software (for example, Microsoft Excel) while still gaining the advantages of the tutor's intelligent guidance. Another consideration was externalizing storage of student records, so that those records could more easily be shared with other systems. These considerations led to a component-based system model which we call the "plug-in" tutor architecture (Ritter and Koedinger, 1996).

The plug-in architecture defines a strict communication protocol between a tool (which corresponds to the set of objects that the user can manipulate) and a tutor (which is responsible for the analysis of student actions and the generation of feedback). A "translator" element is responsible for mediating the communication between the tool and the tutor agent. A "curriculum manager" component is responsible for storing student records and for sequencing student activities. The plug-in component architecture is illustrated in Figure 1.

Based on the plug-in architecture, we developed a translator, tutoring agent and curriculum manager in C++ and used AppleEvents to handle the messaging between components. By modifying the translator slightly, we were able to link the same tutoring agent and curriculum manager to three functionally-equivalent tools: a spreadsheet tool built with the SK8 programming language, a spreadsheet written in C++ and Microsoft Excel. The translator ensures that any differences between the tools are not seen by the tutoring agent. Modification of the translator in this fashion is not a difficult task. Ritter and Blessing (in press) describe a visual authoring tool for the translator component of this system.

Figure 1: Plug-in component architecture

4 The PAT Online Architecture

The tutoring agent in the PAT tutor represents a substantial amount of effort, so one design consideration in PAT Online was to reuse as much of that component as possible. Our design was to use an HTML form as the client system, with the rest of the system running on the server (see Figure 2). This required substantial modifications to the translator component, but allowed us to use the existing tutoring agent in PAT Online without modification. Just as the translator shelters the tutoring agent from differences in the various spreadsheet tools, it shelters the tutoring agent from considerations of whether the tool is running on the same machine as the tutoring agent or somewhere else on the network.

The Tutor CGI

In the basic plug-in architecture, the translator is necessary to ensure that tool and tutor agent can communicate, even though they make "speak" different languages. This is especially important when working with commercial, off-the-shelf tools (like Microsoft Excel), since we can have little control over the language that such a tool speaks. In systems that are designed from scratch to work together, it is possible to design the system in a way that the translator is unnecessary.

In the PAT Online system, the "tool" is an HTML form, and this form input needs to be interpreted by a Common Gateway Interface (CGI) program. The CGI program basically parses information about the values that the user has entered into the form and passes it on to the translator. The translator, in this system, has a more difficult translation task than in other plug-in-based systems. The tutor's commands to the translator consist of directions to modify the state of the interface. For example, the tutor might direct the interface to display a particular bit of help text. In other systems, this command would directly result in a message of some form to the interface. In an HTML form-based system, the entire interface must be described in a block of HTML commands. The translator, then, must construct the interface corresponding to all of the commands given by the tutor in a particular cycle.

Figure 2: PAT Online Architecture

5 The PAT Online Interface

The PAT Online tutor is illustrated in Figure 3. The use of an HTML form in place of the spreadsheet substantially changes the user experience. In the classroom version of the tutor, students receive immediate feedback on every action they take. If they enter an incorrect value into a cell, the system will beep and display that value in a different font to point out their error. HTML forms do not allow this kind of tightly coupled interaction. The server receives information about student actions only when the student submits that information (through the use of a "submit" button). To accommodate this mode of interaction, we allow students to enter values for as many cells as they would like before submitting. The tutoring agent then evaluates all of those actions and presents feedback on all of them.

In the classroom version of the tutor, students can ask for help on any available action. To do this, they select an item (for example, a cell in the spreadsheet) and then push the "help" key. The tutor detects which cell is currently selected, and it uses that information to direct the help. When HTML form information is submitted, no information about the selected cell is available. Thus, to allow students to ask for help on any item, we added help buttons next to each of the cells in the spreadsheet. Information about which help button was selected is transmitted to the tutor, which then gives help on the appropriate item.

Use of an HTML form allowed us to add formatting and images to PAT Online which are not available in the classroom version of the tutor. This is all handled through the translator. When the translator receives a request to start a problem, it looks for an HTML file with that problem name. The HTML file can contain all of the usual HTML commands, allowing for the inclusion of all kinds of images and formatting. The translator appends commands to this file to build the current view of the spreadsheet. Sometime, help commands wish to illustrate the connection between aspects of the problem statement and the desired answer. For example, if the student asks for help in entering a unit of measure, the word "year" in the text might be highlighted. In these cases, the tutor agent describes "text pointers" which are simply character ranges in the text to highlight. The HTML file representing the problem contains comment lines indicating the beginning and end of crucial segments of the problem statement (the introduction, each of the questions and the conclusion). With this information, the translator is able to find the correct segments of the text to be highlighted and to add HTML commands to perform the highlighting before passing the file to the user.

Another interesting interface design decision involved the display of "bug" messages. In the plug-in architecture, these messages are "attached" to particular elements in the interface. In the Excel version of the tutor, for example, a message relating to the student's answer for cell R1C1 would be placed in a "note" for that cell. In Excel, cells with attached notes appear with a red square. The user can view the note by selecting the cell and calling up the "notes" window. Similar mechanisms were used in the other spreadsheet tools. The PAT Online design needed to respond to several design considerations. First, since users can complete more than one cell before getting feedback, they may receive more than one bug message. Second, there is no element in HTML which would allow us the kind of interaction we used with Excel. Our solution was to present bug messages as "footnotes." When a cell has a bug message attached, a number is drawn next to it. At the bottom of the screen, that number appears along with the bug text. This is a nice illustration of how the high-level "attach message" command defined by the plug-in architecture can be implemented in very different ways, depending on the requirements and capabilities of the tool.

Figure 3: The PAT Online tutor (full-size)

In one version of PAT Online, the skillometer was generated by sending AppleScript commands to Microsoft Excel running on the server. These commands insert the appropriate data, direct Excel to draw a graph, copy that graph out of Excel and save it as a GIF file (using the Clip2GIF program to change the graphics format). While this procedure was serviceable, it slows the response time considerably, so we have added the ability to use a simple Java-based skillometer.

6 Handling Multiple Users

In the classroom version of the PAT tutor, a tutor is assigned to each student. PAT Online needed to have the ability to tutor multiple students. The design of the plug-in architecture assumed a single student for each tutor as well, so we needed to reconsider that design. It turns out that there is a fairly simple modification to the design that accommodates the needed changes. Instead of thinking of the tutoring agent as a program or process running on the server, we just needed to redefine it as an object running within that process. The translator then specifies a specific object within the tutor agent program, and a particular user's actions are directed to that object.

More specifically, when a user logs on to the PAT Online system, the translator directs the tutor agent to create a tutor object with the student's name. The name is encoded in the HTML forms as a hidden field, so that any input from a form can be identified with that student. The input is then forwarded to the appropriate tutor object, which is directed to model-trace the student's action(s).

This solution also helps get around some of the problems introduced by the web's "stateless" nature. The server stores tutor objects for all active students, and the tutor objects maintain the state. After some period of inactivity (usually set to 1/2 hour), tutor objects "expire." Expiration involves storing information about the student in a file on disk (though the student record server) and clearing the tutor object out of the tutor application's memory. If the student returns at a later time, records can be read out of the student file and into memory, restoring the last state of the student.

7 Access to student records

In order to allow interaction with other tutors on the World-Wide Web (see Brusilovsky, Ritter and Schwarz, 1997) and to allow easier access to student records, the student record server was designed as a CGI program in itself. This way, an HTML form system separate from PAT Online itself can access student records. Similarly, other systems can access the CGI by sending it commands mimicking form input. The student record server thus has two APIs: one through the CGI mechanism (used by forms and remote tutors) and one through AppleEvents (used by the PAT Online tutor).

8 Conclusion

The PAT Online system provides a good illustration of the flexibility of the plug-in tutor architecture. With only minor modifications, this architecture allowed us to redirect the tutoring knowledge contained in a tutor agent designed to run in a non-networked environment and use in on the World-Wide Web. Still, the system would benefit from more precise and targeted interaction. The use of Java-based programs for the interfaces would allow the kind of interaction achieved in the classroom version of the tutor, but the system would still be hampered by the potential delay in continually sending messages between tool and translator across the internet. A more satisfactory but longer-term solution would be to re-write all of the components of the tutor, with the exception of the student record server, in Java. This would allow quick interaction between all segments of the system as well as centralized record storage. We are currently at work on such a system.


Anderson, J. R., Boyle, C. F., Corbett, A., & Lewis, M. (1990). Cognitive modeling and intelligent tutoring. Artificial Intelligence, 42, 7-49.

Anderson, J. R., Conrad, F. G. and Corbett, A. T. (1993). The Lisp tutor and skill acquisition. In J. R. Anderson, (Ed.), Rules of the Mind (pp. 143-164). Hillsdale, NJ: Lawrence Erlbaum Associates.

Anderson, J.R. and Corbett, A.T. (1993). Tutoring of cognitive skill. In J.R. Anderson, Rules of the Mind (pp. 235-255). Hillsdale, NJ: Erlbaum.

Anderson, J. R., & Pelletier, R. (1991). A development system for model-tracing tutors. In Proceedings of the International Conference of the Learning Sciences (pp. 1-8). Evanston, IL.

Brusilovsky, P., Ritter, S. and Schwarz, E. (1997). Distributed intelligent tutoring on the web. Proceedings of the Eighth World Conference on Artificial Intelligence in Education. Kobe, Japan.

Corbett, A. T. and Anderson, J. R. (1992). Student modeling and mastery learning in a computer-based programming tutor. In C. Frasson, G. Gauthier and G. McCalla (Eds.), Intelligent Tutoring Systems: Second international conference proceedings (pp. 413-420). New York: Springer-Verlag.

Koedinger, K. R., Anderson, J. R.., Hadley, W. H.., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8.

National Council of Teachers of Mathematics (1989). Curriculum and Evaluation Standards for School Mathematics. Reston, VA: The Council.

Ritter, S. and Blessing, S. B. (in Press). Authoring tools for component-based learning environments. Journal of the Learning Sciences, to appear.

Ritter, S. and Koedinger, K. R. (1996). An architecture for plug-in tutor agents. Journal of Artificial Intelligence in Education, 7, 315-347.