Automated Essay Scoring: A Cross-disciplinary Perspective ハードカバー – 2003/1/12
Kindle 端末は必要ありません。無料 Kindle アプリのいずれかをダウンロードすると、スマートフォン、タブレットPCで Kindle 本をお読みいただけます。
This new volume is the first to focus entirely on automated essay scoring and evaluation. It is intended to provide a comprehensive overview of the evolution and state-of-the-art of automated essay scoring and evaluation technology across several disciplines, including education, testing and measurement, cognitive science, computer science, and computational linguistics.
The development of this technology has led to many questions and concerns. Automated Essay Scoring attempts to address some of these questions including:
*How can automated scoring and evaluation supplement classroom instruction?
*How does the technology actually work?
*Can it improve students' writing?
*How reliable is the technology?
*How can these computing methods be used to develop evaluation tools?
*What are the state-of the-art essay evaluation technologies and automated scoring systems?
Divided into four parts, the first part reviews the teaching of writing and how computers can contribute to it. Part II analyzes actual automated essay scorers including e-raterTM, Intellimetric, and the Intelligent Essay Assessor. The third part analyzes related psychometric issues, and the final part reviews innovations in the field.
This book is ideal for researchers and advanced students interested in automated essay scoring from the fields of testing and measurement, education, cognitive science, language, and computational linguistics.
Jill C. Burstein
Mark D. Shermis
Automated Essay Scoring is organized into five major sections: (I) Teaching of Writing, (II) Psychometric Issues in Performance Assessment, (III) Automated Essay Scorers, (IV) Psychometric Issues in Automated Essay Scoring, and (V) Current Innovation in Automated Essay Evaluation. Section III contains a detailed discussion of the leading essay scorers on the market: Project Essay Grade, Intellimetric, e-rater, and the Intelligent Essay Assessor. There is also a chapter on Bayesian text classification. Curiously, though, no mention is made of BETSY-the Bayesian Essay Test Scoring sYstem, which, to my knowledge, is the most fully developed essay scorer using Bayes Theorem.
The authors and editors have done a remarkably good job of presenting their respective grading engines in a fair and balanced manner without bashing competing technologies. That said, it is clear at times where the authors have placed their allegiance. For example, Jill Burstein, one of the editors of the book, is employed by ETS and penned several chapters in which she touts the benefits of e-rater. Nothing wrong with that, though, as she does it without tearing down her competitors.
I have only a few closely related criticisms of this book. The first is that the subtitle is somewhat of a misnomer. The vast majority of the book appears to be written by, for, and to psychometricians or people with substantial psychometric training. It therefore can hardly be called "cross disciplinary." As a writing instructor, I had hoped to find much more discussion written from a composition teacher's point of view. I was disappointed, however, to find that only two of the book's thirteen chapters (chapters 1 and 3) made any attempts to address the concerns that writing instructors have regarding these technologies. This seems ironic since writing teachers comprise the population most likely to be affected by the proliferation of AEETs.
As a whole, teachers of writing and researchers in the field of rhetoric and composition have not responded favorably to the idea of having computers teach and assess writing. Some see it as a return to philosophical positivism or an abandonment of the postmodern values on which the "New English" was built. Regardless of one's position on the issue, such concerns deserve attention, and I was disappointed not to see them addressed in a volume as distinctive and comprehensive as this.
Be that as it may, Lawrence Erlbaum has a deserved reputation for publishing high quality academic material, and this volume is certainly fitting to that reputation. Despite its shortcomings, Automated Essay Scoring: A Cross-Disciplinary Perspective is a work of scholarship no one pursuing research in this field should do without.