Best practice in special education requires the use of various
assessment techniques to guide instructional procedures. For many
reasons, however, appropriate classroom assessment for students with
severe and multiple disabilities is difficult. In a comprehensive study
of assessment practices of special education staff serving students with
severe disabilities in five school districts, Sigafoos, Cole, and
McQuarter (1987) reported that special educators (a) lacked training in
the assessment of students with severe disabilities and (b) were limited
by the unavailability of appropriate assessment tools and were therefore
overly dependent on non-referenced instruments for directing treatment.
Dependence on norm-referenced tests for directing treatment for
students with severe and multiple disabilities is problematic. Most
norm-referenced assessment instruments have not been validated for this
population or for designing, implementing, and monitoring instruction.
Classroom teachers need to use assessment techniques that are designed
to guide instructional practice (Epps & Tindal, 1987; Evens, 1991).
Appropriate assessment tools require a high degree of "treatment
validity" Hofmeister, 1986; Hofmeister & Preston, 1981);
however, because students with severe and multiple disabilities have
diverse curricular needs, the design of such assessment instruments is
difficult (Brown & Snell, 1993).
Computer-based expert system technology can provide effective
assessment tools for assisting teachers in meeting the varied
instructional needs of students with severe and multiple disabilities.
Historically, expert system technology has provided field-based
practitioners with the expertise of a specialist in response to their
specific problems (Stefik et al., 1983). The classic example of an
expert system is Mycin (Davis, Buchanan, & Shortliffe, 1975). Mycin
allowed physicians to use its knowledge and their description of
symptoms to guide diagnosis. Expert systems have also been used to
provide expert-level assistance in solving educational problems.
In 1983, the Utah State University (USU) Center for Persons with
Disabilities (CPD) initiated a programmatic line of research to apply
expert system technology and procedures to problems in special
education. CPD projects developed and validated 16 major systems. These
included a variety of training applications, (Ferrara, Althouse,
Findlay, & Elwell, 1992; Prater, 1987; Thornburg, 1991),
applications that addressed procedural issues in special education
(Baer, Likins, Casdorph, Althouse, & Ferrara, 1989; Hoehle, 1993;
Parry, 1982), and expert systems that supported assessment.
EXPERT SYSTEMS FOR ASSESSMENT
The assessment systems developed by the CPD and other researchers
can be divided into two general categories. One type of expert system
provides advice regarding eligibility for special education services
(Baer, Ferrara, & Althouse, 1991; Ferrara, Hofmeister, &
Althouse, 1983; Giere, Ferrara, & Gustafson, 1991). The second class
of expert system helps teachers develop and implement practical
treatment programs (Fuchs, Fuchs, Hamlett, & Steckler, 1991;
Greenwood et al., 1993; Malouf & Pilato, 1991).
Studies describing the performance of both types of expert systems
have suggested that these systems can enhance the special education
decision-making process and lead to improved student outcomes. With the
use of a well-designed, validated expert system, nonspecialists have
achieved "near-expert" levels of performance (Martindale,
Ferrara, & Campbell, 1987; Parry & Hofmeister, 1986; Speziale,
THE SMH.PAL EXPERT SYSTEM
SMH.PAL was designed to assist educators in identifying appropriate
treatments for students with severe and multiple disabilities
(Hofmeister, Althouse, Likins, & Jenson, 1992). SMH.PAL consists of
two applications. The first is a large database containing research
literature that describes effective treatment of selected problem
behaviors. The second is an expert system designed to help educators
match presenting challenging behavior topography with appropriate,
The overall design of SMH.PAL called for the completion of two
program elements, a database and a knowledge base (KB).
The system's development required 3 years' work by an
eight-member team of programmers, knowledge engineers, data-entry
personnel, and knowledge experts. In addition, SMH.PAL employed many
data structures and computer routines that were developed on earlier CPD
expert system projects and modified for use in this application.
Finally, the program could not have been completed without access to a
literature review conducted by Jenson and Reavis (1984).
Database Development. Producing SMH.PAL's database involved
four activities: (a) selecting challenging behaviors, (b) selecting
research articles, (c) developing an instrument for collecting data
describing selected articles, and (d) systematically using the
instrument to code article data.
To determine which behaviors the system should address, system
developers asked Utah educators who taught students with severe
disabilities to identify the most common and severe behavior problems
exhibited by their students. The teachers first listed all challenging
behaviors that had occurred in their classrooms and then selected the 5-
10 most frequently occurring behaviors and the 5-10 most severe
behaviors. A total of 110 teachers responded to the survey. Figure 1
shows the 20 most frequently identified behaviors.
We conducted a literature search for each teacher-identified
problem behavior. The review identified 1,023 potentially useful
articles published between 1974 and 1990. About half (512) of these
articles were selected for careful examination, and a panel of experts
selected 354 for inclusion in the database. Because many articles
contained data describing the treatment of more than one challenging
behavior, SMH.PAL had a functional database of 655 studies.
We then designed, revised, and tested a uniform data-recording
form. Project staff used the form to describe (a) reference information;
(b) subject characteristics; (c) study specifics, including research
design, setting, length of treatment, intervention, functional analysis,
follow-up, and generalization; and (d) overall treatment efficacy.
Knowledge Base (KB) Development. Expert system KBs are developed by
gathering, integrating, and structuring the procedures and knowledge of
experts (see Giere, 1989; Weiss & Kulikowski, 1984). Programmers
worked with behavior management specialists to develop the
program's KB. SMH.PAL's KB embodies a model for matching
case-specific behaviors to research articles. The KB contains rules,
facts, and questions that are used to select and recommend articles from
the program's database.
We developed four interactive components for the program's KB.
The first component contains a set of rules and facts that analyze
student behavior, search the database, and create a list of articles
that describe potentially useful interventions.
The next major KB component contains rules that use classroom and
student data to prune the list. This section of the KB was designed to
remove articles describing treatments that appear to be inappropriate.
For example, if teachers are known to have little training and
experience, this part of the KB will remove articles describing complex
and difficult procedures. Figure 2 lists factors used by the KB to mark
articles for deletion.
The third KB component was designed to rank remaining articles from
most to least useful. KB rules and facts used for this function can be
grouped into five clusters: (a) facts describing the intrusiveness of
the study's primary intervention, (b) facts describing the degree
of replication the treatment has within the literature base, (c) rules
that infer probable treatment efficacy, (d) facts containing a rating of
the quality of the study procedures, and (e) rules that evaluate
closeness of the match between student/environmental characteristics and
Designers of the fourth and final KB component sought to create
computer code that would write a case-specific rationale for
recommending each article and an explanation of how rank was determined.
These explanations are based on the conclusions and results generated
when other KB components are invoked.
The Completed System. At present, SMH.PAL consists of a research
literature database containing information on 368 studies addressing 20
behaviors; extensive database and report programming; and a KB that
contains 406 rules, questions, and facts. Because of the symbolic nature
of expert system rules, the same rules may handle hundreds of
situations. Therefore, this seemingly small set of rules takes the place
of thousands of case-specific rules. The program currently operates in a
640K MS-DOS environment and requires about 1 MB of disk space to store
the program files.
Running a Consultation. An SMH.PAL consultation (see Figure 3)
begins with the selection of one of the 20 teacher-identified problem
behaviors. The system then uses the knowledge base and literature base
to identify studies associated with the behavior. After compiling a list
of articles describing potentially useful treatments, SMH.PAL asks a
case-specific series of questions. These questions allow the expert
system to consider student, environmental, and treatment characteristics
in an effort to find the best match" and determine the suitability
of the interventions described in the listed articles.
System Output. System-generated reports contain a summary of each
article in the ranked list. The summary information for each article
contains (a) standard reference information in American Psychological
Association style, (b) a list of the treatment(s) described in the
article, (c) any exclusions found for the use of study interventions,
(d) a list of the information used to rank the study, and (e) references
to other studies in the information base with the same intervention.
Figure 4 shows one page of an SMH.PAL-generated report.
The development of the SMH.PAL system has generated several
The SMH.PAL System. This expert system provides immediate,
informed, and selective access to descriptions of hundreds of relevant,
high-quality interventions. The report contains a listing of articles
and associated cautions ranked in order of appropriateness to help users
identify studies and treatments that best match specific circumstances.
The system can be updated with additional studies or expanded to address
additional problem behaviors. The system can be reproduced and
disseminated to provide consistent, near-expert-level decision making.
Unlike some assessment processes and instruments that are not
tightly linked to treatment validity data, SMH.PAL is integrated with
and dependent on research literature on effective interventions. The use
of a research literature database allows the system to represent expert
knowledge of treatment experiences and efficacy, thus supporting the
system's content validity.
The program is not intended to serve as a general-purpose,
computer-based, literature service, such as ERIC. Rather, SMH.PAL
focuses on a limited number of challenging problems faced by classroom
teachers serving students with severe and multiple disabilities. As a
result of its narrow focus, the program has several advantages over
general-literature database services.
The value of most general-purpose databases is clearly dependent on
the user's experience and ability to develop efficient search
schemes. As a result, inexperienced or unlucky users often sort through
numerous position papers, proceedings, and books before finding useful
citations. The artificial intelligence (AI) techniques used by SMH.PAL
prompt users to develop article selection criteria that are used to
access citations in the research literature database. As a result of
these differences, SMH.PAL is able to help inexperienced users quickly
find appropriate articles.
Even the most experienced ERIC user can only generate reports that
provide indexing data and abstracts. These abstracts often do not
contain data that teachers can use to determine applicability of a
study. Users of SMH.PAL receive annotated summaries that are both
treatment and student specific.
Operationalized Treatment-Selection Procedures. The rules and facts
in SMH.PAL's knowledge base clarify and operationalize a systematic
approach to literature selection. These rules are a formal
representation of abstract concepts associated with the selection of
literature-based information for classroom intervention. While
conducting knowledge-engineering activities, the team created rules by
synthesizing, collecting, sorting, discarding, interpreting, and
applying information from various sources. Previous to this effort, few,
if any, of the KB's components could have been found in any single
source. With the development and publication of the system, these
procedures and their underlying assumptions can be examined and
validated or critiqued in a meaningful and replicable way. This
synthesis of the decision-making expertise represents a substantive
contribution to knowledge and has value independent of the expert
CPD continues to support the programmatic line of AI research, and
funding has been obtained to refine and expand SMH. PAL. In addition,
training materials will be developed to assist users in the appropriate
implementation of suggested interventions. Activities also include the
development of mentor-supported implementation procedures and the
collection of data assessing the effectiveness of the system in
improving student outcomes.
Baer, R., Feffara, J. M., & Althouse, R. B. (1991). Class.BD
[Computer program]. Logan: Utah State University. Baer, R., Likins, M.,
Casdorph, T., Althouse, R. B., & Ferrara, J.M. (1989). Referral
consultant: an expert system for making special education referral
decisions. [Computer program]. Logan: Utah State University. Brown, F.,
& Snell, M.F. (1993). Meaningful assessment. In M. F. Snell (Ed.),
Instruction of students with severe disabilities (pp. 61-98). New York:
Macmillan. Davis, R., Buchanan, & Shortliffe, E. (1975). Production
rules as a representation for knowledge-based consultant programs
(Report STAN-CS-57-519 memo-AIUM-2260). Palo Alto, CA: Stanford
University Computer Science Department. Epps, S., & Tindal, G.
(1987). The effectiveness of differential programming in serving
students with mild handicaps: Placement options and instructional
programming. In M. C. Wang, M. C. Reynolds, & H. J. Walberg (Eds.),
Handbook of special education: Research and practice (pp. 213-247). New
York: Pergamon Press. Evens, I. M. (1991). Testing and diagnosis: A
review and evaluation. In L. H. Meyer, C. A. Peck, & L. Brown
(Eds.), Critical issues in the lives of people with severe disabilities
(pp. 25-44). Baltimore: Paul H. Brookes. Ferrara, J. M., Althouse, R.
B., Findlay, P., & Elwell, K. (1992). LD Trainer II [Computer
program]. Logan: Utah State University. Ferrara, J. M., Hofmeister A.
M., & Althouse, R. B. (1983). Class. LD [Computer program]. Logan:
Utah State University. Fuchs, L.S., Fuchs, D., Hamlett, C. L., &
Steckler, P. M. (1991). Effects of curriculum-based measurement and
consultation on teacher planning and student achievement in mathematics
operations. American Education Research Journal, 28(3), 617-641. Giere,
S. S. (1989). A method for knowledge engineering in clinical
decisiomnaking. Dissertation Abstracts International, 51(1), 113.
University Microfilms No. ADG90-15017) Giere, S. S., Ferrara, J. M.,
& Gustafson, K. (1991). Expert system development for knowledge
clarification. (Special Education Programs [ED/OSERS] Grant
#H092K80072). Washington, DC: U.S. Department of Education. Greenwood,
C. R., Finney, R., Terry, B., Arreaga-Mayer, C., Carta, J. J., Delquadri
J., Walker, D., Innocenti, M., Lignugaris-Kraft, J., Harper, G. F.,
& Clifton, R. (1993). Monitoring, improving, and maintaining quality
implementation of the classwide peer tutoring program using behavioral
and computer technology. Education and Treatment of Children, 16(1),
19-47. Hoehle, R. L. (1993). The development of an expert system to
evaluate the individualized education program components of student
records. Dissertation Abstracts International, 54(08), 2981. (University
Microfilms No. ADG94-02300) Hofmeister, A. M. (1986). Formative
evaluation in the development and validation of expert systems in
education. Computational Intelligence, 2(2), 65-67. Hofmeister, A. M.,
Althouse, R. B., Likins, M., & Jenson, W. R. (1992). SMH.PAL
[Computer program]. Logan: Utah State University. Hofmeister, A. M.,
& Preston, C. N. (1981). Educational assessment of special education
populations. In T. Oakland (Ed.), Nonbiased assessment (pp. 77-96).
Minneapolis: University of Minnesota. Jenson, W. R., & Reavis, K.
(1984). The prescriptive checklist and interventions manual for
SMH/autistic children: An outcome based education. The Special Educator,
5, 7-8. Malouf, D. B., & Pilato, V. H. (1991). The SNAP System for
Inservice Training of Regular Educators. (Final Project Report). College
Park: University of Maryland-College Park, Institution for the Study of
Exceptional Children and Youth. (ERIC Document Reproduction Service No
ED339181) Martindale, E. S., Feffara, J. M., & Campbell, B. (1987).
A preliminary report on the performance of CLASS.LD. Computers in Human
Behavior, 3, 263-272. Parry, J. D. (1982). The development and
validation of an expert system for reviewing a special education
practice. Dissertation Abstracts Intemational, 47, 2123. (University
Microfilms No. ADG86-19424) Parry, J. D., & Hofmeister, A.M. (1986).
Development and validation of an expert system for special educators.
Learning Disability Quarterly, 9(2), 124-132. Prater, M. A. (1987).
Expert system technology and concept instruction: Training educators to
accurately classify leaming disabled students. Dissertation Abstracts
Intemational, 48(07), 1736. (University Microfilms No. ADG87-19469)
Sigafoos, J., Cole, D. A., & McQuarter. (1987). Current practices in
the assessment of students with severe handicaps. The Journal of the
Association for Persons with Severe Handicaps, 12(4), 264-273. Speziale,
M. J. (1991). Comparing the diagnostic/prescriptive performance of
expert systems to the performance of elementary educators, special
educators, and remedial reading specialists. Dissertation Abstracts
International, 52(09), 325 1. (University Microfilms No. ADG92-07027)
Stefik, M., Aikins, J., Blazer, R., Benoit, J., Birnbaum, L.,
Hayes-Roth, F., & Sacerdoti, E. (1983). The architecture of expert
systems. In F. Hayes-Roth, D. A. Waterman, & D. B. Lenat (Eds.),
Building expert systems (pp. 89-126). Reading, MA: Addison-Wesley.
Thornburg, M. S. (1991). The development and validation of a system for
the knowledge-based tutoring of special education rules and regulations.
Dissertabtion abstracts international, 52(02), 505. (University
Microfilms. No. ADG91-20595) Weiss, S. M., & Kulikowski, C. A.
(1994). A practical guide to designing expert, systems. Totowana, NJ:
Rowman & Allantkeld.