Sign up

SMH.PAL: an expert system for identifying treatment procedures for students with severe disabilities.
Abstract:
A variety of difficulties are associated with the development and implementation of assessment for students with severe and multiple disabilities. This article describes an assessment approach, SMH.PAL, that employs expert system technology to help teachers identify interventions that may be effectively used with students exhibiting selected problem behaviors. The system matches behavior topography with studies describing treatment approaches and then generates case-specific, annotated descriptions of potentially useful articles. In addition, the system models a reasoned approach to searching intervention literature.

Subject:
Disabled students (Education)
Special education (Activity programs)
Computerized testing (Usage)
Authors:
Hofmeister, Alan M.
Althouse, R. Brad
Likins, Marilyn
Morgan, Daniel P.
Ferrara, Joseph M.
Jenson, William R.
Rollins, Evelyn
Pub Date:
10/01/1994
Publication:
Name: Exceptional Children Publisher: Council for Exceptional Children Audience: Academic; Professional Format: Magazine/Journal Subject: Education; Family and marriage Copyright: COPYRIGHT 1994 Council for Exceptional Children ISSN: 0014-4029
Issue:
Date: Oct-Nov, 1994 Source Volume: v61 Source Issue: n2
Accession Number:
15824013
Full Text:
Best practice in special education requires the use of various assessment techniques to guide instructional procedures. For many reasons, however, appropriate classroom assessment for students with severe and multiple disabilities is difficult. In a comprehensive study of assessment practices of special education staff serving students with severe disabilities in five school districts, Sigafoos, Cole, and McQuarter (1987) reported that special educators (a) lacked training in the assessment of students with severe disabilities and (b) were limited by the unavailability of appropriate assessment tools and were therefore overly dependent on non-referenced instruments for directing treatment.

Dependence on norm-referenced tests for directing treatment for students with severe and multiple disabilities is problematic. Most norm-referenced assessment instruments have not been validated for this population or for designing, implementing, and monitoring instruction. Classroom teachers need to use assessment techniques that are designed to guide instructional practice (Epps & Tindal, 1987; Evens, 1991). Appropriate assessment tools require a high degree of "treatment validity" Hofmeister, 1986; Hofmeister & Preston, 1981); however, because students with severe and multiple disabilities have diverse curricular needs, the design of such assessment instruments is difficult (Brown & Snell, 1993).

Computer-based expert system technology can provide effective assessment tools for assisting teachers in meeting the varied instructional needs of students with severe and multiple disabilities. Historically, expert system technology has provided field-based practitioners with the expertise of a specialist in response to their specific problems (Stefik et al., 1983). The classic example of an expert system is Mycin (Davis, Buchanan, & Shortliffe, 1975). Mycin allowed physicians to use its knowledge and their description of symptoms to guide diagnosis. Expert systems have also been used to provide expert-level assistance in solving educational problems.

In 1983, the Utah State University (USU) Center for Persons with Disabilities (CPD) initiated a programmatic line of research to apply expert system technology and procedures to problems in special education. CPD projects developed and validated 16 major systems. These included a variety of training applications, (Ferrara, Althouse, Findlay, & Elwell, 1992; Prater, 1987; Thornburg, 1991), applications that addressed procedural issues in special education (Baer, Likins, Casdorph, Althouse, & Ferrara, 1989; Hoehle, 1993; Parry, 1982), and expert systems that supported assessment.

EXPERT SYSTEMS FOR ASSESSMENT

The assessment systems developed by the CPD and other researchers can be divided into two general categories. One type of expert system provides advice regarding eligibility for special education services (Baer, Ferrara, & Althouse, 1991; Ferrara, Hofmeister, & Althouse, 1983; Giere, Ferrara, & Gustafson, 1991). The second class of expert system helps teachers develop and implement practical treatment programs (Fuchs, Fuchs, Hamlett, & Steckler, 1991; Greenwood et al., 1993; Malouf & Pilato, 1991).

Studies describing the performance of both types of expert systems have suggested that these systems can enhance the special education decision-making process and lead to improved student outcomes. With the use of a well-designed, validated expert system, nonspecialists have achieved "near-expert" levels of performance (Martindale, Ferrara, & Campbell, 1987; Parry & Hofmeister, 1986; Speziale, 1991).

THE SMH.PAL EXPERT SYSTEM

SMH.PAL was designed to assist educators in identifying appropriate treatments for students with severe and multiple disabilities (Hofmeister, Althouse, Likins, & Jenson, 1992). SMH.PAL consists of two applications. The first is a large database containing research literature that describes effective treatment of selected problem behaviors. The second is an expert system designed to help educators match presenting challenging behavior topography with appropriate, research-supported procedures.

System Development

The overall design of SMH.PAL called for the completion of two program elements, a database and a knowledge base (KB).

The system's development required 3 years' work by an eight-member team of programmers, knowledge engineers, data-entry personnel, and knowledge experts. In addition, SMH.PAL employed many data structures and computer routines that were developed on earlier CPD expert system projects and modified for use in this application. Finally, the program could not have been completed without access to a literature review conducted by Jenson and Reavis (1984).

Database Development. Producing SMH.PAL's database involved four activities: (a) selecting challenging behaviors, (b) selecting research articles, (c) developing an instrument for collecting data describing selected articles, and (d) systematically using the instrument to code article data.

To determine which behaviors the system should address, system developers asked Utah educators who taught students with severe disabilities to identify the most common and severe behavior problems exhibited by their students. The teachers first listed all challenging behaviors that had occurred in their classrooms and then selected the 5- 10 most frequently occurring behaviors and the 5-10 most severe behaviors. A total of 110 teachers responded to the survey. Figure 1 shows the 20 most frequently identified behaviors.

We conducted a literature search for each teacher-identified problem behavior. The review identified 1,023 potentially useful articles published between 1974 and 1990. About half (512) of these articles were selected for careful examination, and a panel of experts selected 354 for inclusion in the database. Because many articles contained data describing the treatment of more than one challenging behavior, SMH.PAL had a functional database of 655 studies.

We then designed, revised, and tested a uniform data-recording form. Project staff used the form to describe (a) reference information; (b) subject characteristics; (c) study specifics, including research design, setting, length of treatment, intervention, functional analysis, follow-up, and generalization; and (d) overall treatment efficacy.

Knowledge Base (KB) Development. Expert system KBs are developed by gathering, integrating, and structuring the procedures and knowledge of experts (see Giere, 1989; Weiss & Kulikowski, 1984). Programmers worked with behavior management specialists to develop the program's KB. SMH.PAL's KB embodies a model for matching case-specific behaviors to research articles. The KB contains rules, facts, and questions that are used to select and recommend articles from the program's database.

We developed four interactive components for the program's KB. The first component contains a set of rules and facts that analyze student behavior, search the database, and create a list of articles that describe potentially useful interventions.

The next major KB component contains rules that use classroom and student data to prune the list. This section of the KB was designed to remove articles describing treatments that appear to be inappropriate. For example, if teachers are known to have little training and experience, this part of the KB will remove articles describing complex and difficult procedures. Figure 2 lists factors used by the KB to mark articles for deletion.

The third KB component was designed to rank remaining articles from most to least useful. KB rules and facts used for this function can be grouped into five clusters: (a) facts describing the intrusiveness of the study's primary intervention, (b) facts describing the degree of replication the treatment has within the literature base, (c) rules that infer probable treatment efficacy, (d) facts containing a rating of the quality of the study procedures, and (e) rules that evaluate closeness of the match between student/environmental characteristics and study characteristics.

Designers of the fourth and final KB component sought to create computer code that would write a case-specific rationale for recommending each article and an explanation of how rank was determined. These explanations are based on the conclusions and results generated when other KB components are invoked.

The Completed System. At present, SMH.PAL consists of a research literature database containing information on 368 studies addressing 20 behaviors; extensive database and report programming; and a KB that contains 406 rules, questions, and facts. Because of the symbolic nature of expert system rules, the same rules may handle hundreds of situations. Therefore, this seemingly small set of rules takes the place of thousands of case-specific rules. The program currently operates in a 640K MS-DOS environment and requires about 1 MB of disk space to store the program files.

Running a Consultation. An SMH.PAL consultation (see Figure 3) begins with the selection of one of the 20 teacher-identified problem behaviors. The system then uses the knowledge base and literature base to identify studies associated with the behavior. After compiling a list of articles describing potentially useful treatments, SMH.PAL asks a case-specific series of questions. These questions allow the expert system to consider student, environmental, and treatment characteristics in an effort to find the best match" and determine the suitability of the interventions described in the listed articles.

System Output. System-generated reports contain a summary of each article in the ranked list. The summary information for each article contains (a) standard reference information in American Psychological Association style, (b) a list of the treatment(s) described in the article, (c) any exclusions found for the use of study interventions, (d) a list of the information used to rank the study, and (e) references to other studies in the information base with the same intervention. Figure 4 shows one page of an SMH.PAL-generated report.

Benefits

The development of the SMH.PAL system has generated several important products.

The SMH.PAL System. This expert system provides immediate, informed, and selective access to descriptions of hundreds of relevant, high-quality interventions. The report contains a listing of articles and associated cautions ranked in order of appropriateness to help users identify studies and treatments that best match specific circumstances. The system can be updated with additional studies or expanded to address additional problem behaviors. The system can be reproduced and disseminated to provide consistent, near-expert-level decision making.

Unlike some assessment processes and instruments that are not tightly linked to treatment validity data, SMH.PAL is integrated with and dependent on research literature on effective interventions. The use of a research literature database allows the system to represent expert knowledge of treatment experiences and efficacy, thus supporting the system's content validity.

The program is not intended to serve as a general-purpose, computer-based, literature service, such as ERIC. Rather, SMH.PAL focuses on a limited number of challenging problems faced by classroom teachers serving students with severe and multiple disabilities. As a result of its narrow focus, the program has several advantages over general-literature database services.

The value of most general-purpose databases is clearly dependent on the user's experience and ability to develop efficient search schemes. As a result, inexperienced or unlucky users often sort through numerous position papers, proceedings, and books before finding useful citations. The artificial intelligence (AI) techniques used by SMH.PAL prompt users to develop article selection criteria that are used to access citations in the research literature database. As a result of these differences, SMH.PAL is able to help inexperienced users quickly find appropriate articles.

Even the most experienced ERIC user can only generate reports that provide indexing data and abstracts. These abstracts often do not contain data that teachers can use to determine applicability of a study. Users of SMH.PAL receive annotated summaries that are both treatment and student specific.

Operationalized Treatment-Selection Procedures. The rules and facts in SMH.PAL's knowledge base clarify and operationalize a systematic approach to literature selection. These rules are a formal representation of abstract concepts associated with the selection of literature-based information for classroom intervention. While conducting knowledge-engineering activities, the team created rules by synthesizing, collecting, sorting, discarding, interpreting, and applying information from various sources. Previous to this effort, few, if any, of the KB's components could have been found in any single source. With the development and publication of the system, these procedures and their underlying assumptions can be examined and validated or critiqued in a meaningful and replicable way. This synthesis of the decision-making expertise represents a substantive contribution to knowledge and has value independent of the expert system.

Future Work

CPD continues to support the programmatic line of AI research, and funding has been obtained to refine and expand SMH. PAL. In addition, training materials will be developed to assist users in the appropriate implementation of suggested interventions. Activities also include the development of mentor-supported implementation procedures and the collection of data assessing the effectiveness of the system in improving student outcomes.

REFERENCES

Baer, R., Feffara, J. M., & Althouse, R. B. (1991). Class.BD [Computer program]. Logan: Utah State University. Baer, R., Likins, M., Casdorph, T., Althouse, R. B., & Ferrara, J.M. (1989). Referral consultant: an expert system for making special education referral decisions. [Computer program]. Logan: Utah State University. Brown, F., & Snell, M.F. (1993). Meaningful assessment. In M. F. Snell (Ed.), Instruction of students with severe disabilities (pp. 61-98). New York: Macmillan. Davis, R., Buchanan, & Shortliffe, E. (1975). Production rules as a representation for knowledge-based consultant programs (Report STAN-CS-57-519 memo-AIUM-2260). Palo Alto, CA: Stanford University Computer Science Department. Epps, S., & Tindal, G. (1987). The effectiveness of differential programming in serving students with mild handicaps: Placement options and instructional programming. In M. C. Wang, M. C. Reynolds, & H. J. Walberg (Eds.), Handbook of special education: Research and practice (pp. 213-247). New York: Pergamon Press. Evens, I. M. (1991). Testing and diagnosis: A review and evaluation. In L. H. Meyer, C. A. Peck, & L. Brown (Eds.), Critical issues in the lives of people with severe disabilities (pp. 25-44). Baltimore: Paul H. Brookes. Ferrara, J. M., Althouse, R. B., Findlay, P., & Elwell, K. (1992). LD Trainer II [Computer program]. Logan: Utah State University. Ferrara, J. M., Hofmeister A. M., & Althouse, R. B. (1983). Class. LD [Computer program]. Logan: Utah State University. Fuchs, L.S., Fuchs, D., Hamlett, C. L., & Steckler, P. M. (1991). Effects of curriculum-based measurement and consultation on teacher planning and student achievement in mathematics operations. American Education Research Journal, 28(3), 617-641. Giere, S. S. (1989). A method for knowledge engineering in clinical decisiomnaking. Dissertation Abstracts International, 51(1), 113. University Microfilms No. ADG90-15017) Giere, S. S., Ferrara, J. M., & Gustafson, K. (1991). Expert system development for knowledge clarification. (Special Education Programs [ED/OSERS] Grant #H092K80072). Washington, DC: U.S. Department of Education. Greenwood, C. R., Finney, R., Terry, B., Arreaga-Mayer, C., Carta, J. J., Delquadri J., Walker, D., Innocenti, M., Lignugaris-Kraft, J., Harper, G. F., & Clifton, R. (1993). Monitoring, improving, and maintaining quality implementation of the classwide peer tutoring program using behavioral and computer technology. Education and Treatment of Children, 16(1), 19-47. Hoehle, R. L. (1993). The development of an expert system to evaluate the individualized education program components of student records. Dissertation Abstracts International, 54(08), 2981. (University Microfilms No. ADG94-02300) Hofmeister, A. M. (1986). Formative evaluation in the development and validation of expert systems in education. Computational Intelligence, 2(2), 65-67. Hofmeister, A. M., Althouse, R. B., Likins, M., & Jenson, W. R. (1992). SMH.PAL [Computer program]. Logan: Utah State University. Hofmeister, A. M., & Preston, C. N. (1981). Educational assessment of special education populations. In T. Oakland (Ed.), Nonbiased assessment (pp. 77-96). Minneapolis: University of Minnesota. Jenson, W. R., & Reavis, K. (1984). The prescriptive checklist and interventions manual for SMH/autistic children: An outcome based education. The Special Educator, 5, 7-8. Malouf, D. B., & Pilato, V. H. (1991). The SNAP System for Inservice Training of Regular Educators. (Final Project Report). College Park: University of Maryland-College Park, Institution for the Study of Exceptional Children and Youth. (ERIC Document Reproduction Service No ED339181) Martindale, E. S., Feffara, J. M., & Campbell, B. (1987). A preliminary report on the performance of CLASS.LD. Computers in Human Behavior, 3, 263-272. Parry, J. D. (1982). The development and validation of an expert system for reviewing a special education practice. Dissertation Abstracts Intemational, 47, 2123. (University Microfilms No. ADG86-19424) Parry, J. D., & Hofmeister, A.M. (1986). Development and validation of an expert system for special educators. Learning Disability Quarterly, 9(2), 124-132. Prater, M. A. (1987). Expert system technology and concept instruction: Training educators to accurately classify leaming disabled students. Dissertation Abstracts Intemational, 48(07), 1736. (University Microfilms No. ADG87-19469) Sigafoos, J., Cole, D. A., & McQuarter. (1987). Current practices in the assessment of students with severe handicaps. The Journal of the Association for Persons with Severe Handicaps, 12(4), 264-273. Speziale, M. J. (1991). Comparing the diagnostic/prescriptive performance of expert systems to the performance of elementary educators, special educators, and remedial reading specialists. Dissertation Abstracts International, 52(09), 325 1. (University Microfilms No. ADG92-07027) Stefik, M., Aikins, J., Blazer, R., Benoit, J., Birnbaum, L., Hayes-Roth, F., & Sacerdoti, E. (1983). The architecture of expert systems. In F. Hayes-Roth, D. A. Waterman, & D. B. Lenat (Eds.), Building expert systems (pp. 89-126). Reading, MA: Addison-Wesley. Thornburg, M. S. (1991). The development and validation of a system for the knowledge-based tutoring of special education rules and regulations. Dissertabtion abstracts international, 52(02), 505. (University Microfilms. No. ADG91-20595) Weiss, S. M., & Kulikowski, C. A. (1994). A practical guide to designing expert, systems. Totowana, NJ: Rowman & Allantkeld.
Gale Copyright:
Copyright 1994 Gale, Cengage Learning. All rights reserved.