Newsletters > October 2000
D is for DiagnosticWith California’s increased emphasis on high stakes testing, there has been an increased call for MDTP tests to be used for other than diagnostic purposes. The following article discusses some reasons those uses are not appropriate
Why are MDTP tests called diagnostic? They are designed to inform California teachers and their students about the extent to which specific mathematical ideas have been learned. To do this well, the tests must be properly administered and taken. When this happens, MDTP student reports indicate how well individual students understand and can apply the mathematics of critical topic area. Further, since MDTP teacher reports contain class summaries not only by topics but also for each test question, teachers are better able to understand their class's strengths and weaknesses. Common student misconceptions often lead to many of the incorrect responses on MDTP tests. When MDTP's readiness tests are given early in a course, MDTP reports can help the teacher determine what review is needed for the class as a whole and also help individual students address weaknesses in their background knowledge. When the tests are given later in a course, the results can suggest topics and concepts that need further attention. MDTP results can help teachers implement teaching strategies that effectively address common student misunderstandings.
To make the interpretations of MDTP reports as accurate as possible, the tests should be clearly presented to the students as diagnostic. The students should understand that their goal is to answer as many questions correctly as they can, but only when they think that they have solved the problem posed in the question. Since the purpose of the test is to indicate what students can and cannot do, guessing answers would create less than accurate results.
High Stakes Testing Strategies
MDTP tests are designed to be "low stakes" tests. "High stakes" tests are those that can have serious positive or negative consequences for students, teachers, or schools. These tests are often designed to ensure that students have met course or curriculum objectives. Obvious examples of high stakes tests for students are final exams or any tests that can have a major impact on a student's course grade. Another obvious example is California's forthcoming High School Exit Examination. Examples of high stakes tests for teachers and districts include tests that are used to evaluate the performance of teachers and schools, such as the Stanford Achievement Test 9 and its augmentation. Indeed, any tests that are used for accountability are high stakes.
The goal for students and schools in high stakes testing is to obtain the highest possible score. In such situations, guessing may be both appropriate and advisable. Even if scores are adjusted for guessing (which is NOT done in MDTP score reports), it may be advisable to guess on multiple-choice tests when the student can confidently eliminate some of the choices. When obtaining the highest possible score is the goal, it becomes reasonable for teachers to help their students learn appropriate test-taking strategies. Since guessing is a reasonable strategy for some high stakes tests, teachers often are expected to teach students various guessing techniques for use in those situations.
It is worthwhile to consider the effects of guessing on the data provided by MDTP reports. When guessing is minimized by students understanding the diagnostic purpose of a test, the item analyses MDTP provides are accurate indications of how the students solved the questions--either correctly or using a common error that teachers can usually identify. On the other hand, if a substantial proportion of the students guessed, then both aspects of the item analysis are likely to be much less accurate.
For example, suppose that half the students marked the correct choice for a question, all the students who did not know how to solve the question guessed, the guesses were randomly distributed, and there were four choices (as on MDTP's AR tests). Letting K be the proportion of students who knew how to answer the question and assuming that one-fourth of the students who do not know guessed the correct response, K satisfies the equation
K + .25(1 - K) = .50 .
This shows that when half the students responded correctly to a four-choice item, only one-third of the students actually knew how to solve the problem. So, when students treat a diagnostic test as a high stakes test, the test data provide significantly less information about what students can do and what errors they make.
The over-simplified situation described in the preceding example is unlikely to occur when students answer MDTP questions. Each choice of an incorrect response usually reflects a conceptual misunderstanding or a computational error or a failure to completely solve a problem. It is very likely that most incorrect responses for the overwhelming majority of students taking MDTP tests result from incorrect solutions rather than from guessing. Even so, the preceding analysis indicates the undesirable effects of guessing on diagnostic reports. Further, those effects are more likely when students are encouraged to guess than when they are encouraged to only answer questions that they think they have solved.
MDTP Tests Are Not High Stakes Tests
MDTP strongly discourages the use of its tests as final examinations in courses or for school accountability measures. We have seen that the information that MDTP reports for diagnostic use is less reliable when students do not understand the diagnostic nature of the test. MDTP tests were not designed as high stakes tests. They are not comprehensive, necessarily failing to cover all the topics of a course or curriculum in order to provide diagnostic information on some critical topics. Each MDTP test is limited to a few topics; data support the opinion of many experts that knowledge of these topics is essential to learning more mathematics. All questions on MDTP tests are multiple-choice. MDTP tests are not secure. All of these features of MDTP tests limit their suitability for high stakes uses
Final course examinations need to be written by the course instructors. An essential component of a good course final examination is testing how well students can apply and extend what they learned in the course. Only the instructors are familiar with their students' academic experiences, and so only the instructors can design and write tests that do this. Another important goal of mathematics courses is helping students communicate their mathematical reasoning. Good course final examinations test the effectiveness of written presentations of solving problems and understanding mathematics. This cannot be done using only multiple-choice questions.
High stakes tests need to be reasonably (in some cases extremely) secure. MDTP tests are not secure. Indeed, part of the value of diagnostic tests lies in teachers having full access to the questions and their students' responses. This conflicts with most security requirements for high stakes tests. To maintain security and increasing demands for public disclosure of high stakes tests, many such tests are now completely revised each year (or even more frequently). Instead of making massive and rapid revisions, MDTP carefully develops high quality questions and tests that can be used for many years.
Summary
The different purposes of tests significantly affect test design, test-taking strategies, and the interpretation of the test results. Low-stakes diagnostic tests, such as MDTP's, can provide good information to guide teachers and students toward strengthening the students' opportunities to learn. High stakes tests, such as course final examinations and program accountability tests, provide other measures of how well students can do in different situations. MDTP urges teachers and others to use its tests appropriately, taking full advantage of the information that can be obtained to find strong and weak points in students' knowledge and in mathematics courses.
New Written Response Materials to be Released
Nine new written response items will be released this fall. Four will be at the Geometry Readiness (GR) level, two at the Second-Year Algebra Readiness (SR) level, two at the Mathematical Analysis readiness (MR) level and one at the Calculus readiness (CR) level. If you have already received a written response binder with the initially released items and returned the response form that is in the front, you will automatically receive these updated materials to insert in your binder soon. If you would like to use these materials in your classroom and do not have a binder yet, please call your site director and ask for one.
We are pleased with the favorable responses we have had from teachers who have attended a workshop on how to use and score these materials. Some teachers have remarked that these activities complement their curriculum and believe they represent the kind of challenging work it is important for their students to have the opportunity to undertake.
We intend to field-test the next group of items this school year. We will need teachers to participate with their classes and for this we depend upon our users. So far they have never let us down. These items will be at the GR level and higher. The protocol requires the item to be given to the class and scored by the teacher using the materials provided. The teacher must also complete an evaluation form about the item and submit samples of students' work at each scoring level. If you would be willing to participate in this round of field-testing, please notify your site director or contact directly Barbara Wells at UCLA (310) 206-8360 or bgwells@ucla.edu.
New MDTP Site Opens at CSU San Bernardino
A new site will open soon at CSU San Bernardino to better serve schools in that region. Specifically, this site is slated to serve San Bernardino County and parts of Riverside County. Until now those schools that were aware of MDTP have been probably using the Los Angeles, Fullerton, or San Diego sites-depending upon their proximity or convenience. John Sarli, a professor of mathematics at CSU San Bernardino, is the director of MDTP's newest site. John has been a member of the MDTP Workgroup for several years and has taken on this new assignment because he believes that he can contribute to the mathematics achievement of students in these schools through his work as site director. If you are a MDTP services user located near CSU San Bernardino, he is eager to meet you and your colleagues. Please let him know how he can best meet the needs of you and your students
MDTP Scoring Sites
| Berkeley Robert Mattison (510) 642-0752 Fax: (510) 642-6726 Counties: San Francisco Bay area and all counties not served by other sites. Chico Jack Ladwig (530) 898-6367 Counties: Butte, Colusa, Glenn, Lassen, Modoc, Nevada, Plumas, Shasta, Sierra, Siskiyou, Tehama, Trinity, and Yuba Davis Phil Knox (530) 752-2021 Fax: (530) 752-7706 Counties: Alpine, Amador, Calaveras, El Dorado, Placer, Sacramento, San Joaquin, Solano, Sutter, and Yolo Fresno Peter Tannenbaum (559) 278-4029 Counties: Fresno, Kern, Madera, Tulare and surrounding areas Fullerton David Pagni (714) 278-2671 Fax: (714) 278-3972 Counties: Orange and schools in Los Angeles and Riverside near Fullerton |
Los Angeles Barbara Wells (310) 206-8360 Fax: (310) 206-5369 E-mail: bgwells@ucla.edu Counties: Los Angeles and Ventura (except for schools that are near Fullerton) San Bernardino John Sarli (909) 880-7670 Fax: (909) 880-7119 E-mail: sarli@math.csusb.edu Counties: Riverside (Riverside City and NW of Riverside City) and San Bernardino San Diego Richard Pilgrim (858) 534-3298 Fax: (858) 534-1011 E-mail: jforsythe@ucsd.edu Counties: Imperial, San Diego, and schools not close to Fullerton or San Bernardino in Riverside county San Luis Obispo Steve Agronsky (805) 756-1683 Fax: (805) 756-6537 Counties: San Luis Obispo, Santa Barbara, and southern Monterey Santa Cruz Teel Blackman (831) 459-2400 Fax: (831) 459-3260 Counties: Santa Cruz and northern Monterey |
Next Teachers Institute Planned
A plethora of professional development activities designed for mathematics teachers was scheduled within the span of two short months this past summer. We believe these circumstances contributed to numerous conflicts that resulted in the MDTP Teacher lnstitute—also scheduled during the summer—having to be cancelled. In order to avoid the reoccurrence of this situation, we are advertising the next institute earlier and will be leaving the exact dates open for negotiation with accepted applicants.
The first institute, held at UCLA during the summer of 1999, was quite successful as attested to by the presentations made by its participants in such places as Bakersfield, CSU San Marcos, and throughout the San Diego, Chico, Sacramento, and Los Angeles areas. We hope that another cohort of experienced MDTP users will be interested in attending the next institute to learn more about MDTP and how to present workshops about effective uses of our services.
This will be a four-day residential institute for which a stipend of $300 is awarded. A speaker's fee and travel expenses will be paid for any subsequent presentations given at MDTP's request. If you think you would be interested in attending the next institute, contact your site director for an application and return it by the end of December stating when you will be available during the summer of 2001.
Upcoming Users' Conferences
UC Davis
Thursday, November 9, 2000
4:15-7:30 PM at the University Club Conference Center
For more information contact: Trish Ramos
Telephone: (530) 752-2015
E-mail: plramos@ucdavis.edu
UCLA
Saturday, January 20, 2001
8:30AM 1:30PM at UCLA
For more information contact: John Hoover
Telephone: (310) 825-8030
E-mail: hoover@gseis.ucla.edu
CSU Fullerton
Thursday, January 25, 2001
For more information contact: David Pagni
Telephone: (714) 278.2671
E-mail: dpagni@fullerton.edu
UC San Diego
Thursday, March 29, 2001
3:30 PM in the Warren Lecture Hall Courtyard
For more information contact: Jean Forsythe
E-mail: jforsythe@ucsd.edu
New Tests to be Released
MDTP will usher in the year 2001 with three new tests. For the first time there will be two tests that were designed based upon the common content of two of the integrated curricula in widespread use in California. A revised version of the algebra readiness test that prohibits calculator use will also be released concurrently.
The Integrated Second Year Readiness Test is appropriate at the end of the first course in an integrated sequence or at the beginning of its second course. Its topics are:
* Linear Equations and Inequalities
* Polynomials, including Quadratic Equations
* Exponents and Square Roots; Scientific Notation, Rational Expressions
* Graphical Representation
* Informal Geometry and Logic
The Integrated Third Year Readiness Test is appropriate at the end of the second course in an integrated sequence or at the beginning of its third course. Its topics are:
* Linear Equations and Inequalities
* Polynomials, including Quadratic Equations
* Exponents and Square Roots; Scientific Notation, Rational Expressions
* Data Analysis and Graphical Representation
* Geometric Applications Geometric Concepts
The revised Algebra Readiness test has five fewer items and has added a new topic, "Data Analysis, Probability, & Statistics." Its topics are:
* Data Analysis, Probability, & Statistics
* Decimals, their Operations & Applications; Percent
* Simple Equations and Operations with Literal Symbols
* Exponents and Square Roots; Scientific Notation
* Fractions and their Applications
* Measurement of Geometric Objects
* Graphical Representation
* Integers, their Operations & Applications
Users can be assured that their current versions of the algebra readiness test will still be scored by their sites. Should you wish to review copies of any of these tests, please request them from your site director.
News from MDTP Scoring Site
From CSU Chico
During the first half of 2001, Chico site director Jack Ladwig will be available to travel to individual schools and districts in Northern California to do on-site presentations and workshops for those interested in learning the effective classroom use of MDTP tests and materials. To schedule a visit to your school, contact him at (530) 898-6367.
Jack is looking forward to seeing Northern California teachers at the conference workshops he will be giving this fall. These presentations will address using MDTP test results to analyze student strengths and weaknesses as well as using written response items to enhance student learning.
The workshops will be given at:
· The Mount Lassen Math Council Conference, Saturday, October 7, at Chico HS;
· The California Math Council Far North Conference, Saturday, October 28, at Arcata HS;
· The California Math Council - North Conference, Saturday, December 2, at Asilomar.
From CSU Fullerton
We just completed our 14th year of serving Orange County schools. Last year we scored tests for about 340 schools in Orange and parts of Riverside, Los Angeles, and San Bernardino counties. MDTP has been playing a role in helping schools with their summer programs as well as special summer programs at Cal State Fullerton that serve minority students (AMP Upward Bound, Bridge, GEAR UP, MISS).
We are in the process of preparing for the fall usage of MDTP which includes updating our database and restocking tests and answer sheets. Last spring we hosted two workshops for training teachers to use the Written Response materials and we anticipate hosting one or more of those workshops in the fall/spring of 2000-2001.
From CSUP San Luis Obispo
There will be new faces at our site because after serving 15 years as the director of the Cal Poly, San Luis Obispo MDTP Site, Martin Lang is retiring in December and Steven Agronsky has been appointed to be the new director. Dr. Agronsky is a professor of Mathematics at Cal Poly. To ensure a smooth transition, Professor Lang has agreed to work with Professor Agronsky this fall. Together they hope to maintain the cooperative relationships established over the years with teachers and administrators of the schools in this area. The San Luis Obispo site serves all schools in the counties of San Luis Obispo, Santa Barbara, and the southern part of Monterey. It is their goal to continue to provide personalized service to area schools and they invite teachers and administrators who are interested in the free diagnostic testing service to contact them for more information at (805) 756-2445.
From UC Davis
Our upcoming users conference will be held at the University Club Conference Center on Thursday, November 9, 2000 from 4:15-7:30 p.m. Refreshments and dinner will be included.
From UCLA
If you use our site for your MDTP services, we hope you will consider spending the day with us at our users' conference on January 20, 2001. It is a great opportunity for users in this region to develop new professional alliances and exchange ideas between schools. We also believe that you will be glad you came because this year's guest speaker, Guershon Harel, a prominent mathematics educator, promises to be both stimulating and challenging. His presentation will address three main questions: (1) What are the factors that influence mathematics learning and what are the conditions that must be fulfilled for mathematics learning to occur? (2) What is the role of the teacher in motivating students to learn mathematics? (3) How can we help students develop ways of thinking effectively in mathematics?
From UC Santa Cruz
As a result of Michelle Dohl, the former coordinator at the UC Santa Cruz site, having moved into the position of Mathematics Department Office Manager, there will be new faces at this site. Michelle is doing an excellent job and we congratulate her on a new adventure in her life. Teel Blackman has replaced her as MDTP Coordinator and Alessandra Alvares is the new MDTP Assistant. Both of them look forward to continuing to provide services to the schools in this region.
From UC San Diego
The University of California, San Diego site continues to serve schools in the San Diego, Imperial, and parts of Riverside counties. We did not get much summer break due to helping many schools in the region with summer school programs. This year looks to be busier than ever with more schools participating in testing more students, especially those in the higher level high school courses. If you wish to request an MDTP workshop please do so soon to avoid scheduling conflicts. We will once again have our March mini-conference and workshops on Thursday evening, March 29th.
Produced by the
MDTP Relations with Schools Office
1329 Moore Hall, Box 951521
Los Angeles, CA 90095-1521
Relations with Schools Coordinator: Barbara Wells
This newsletter has been provided with the support of the Regents of the University of California, The Trustees of the California State University, and the California Academic Partnership Program.
Send questions, comments, and suggestions regarding theMathematics Diagnostic Testing Project to: mdtp@ucsd.edu.
Copyright © 1999-2009, CSU/UC Mathematics Diagnostic Testing Project.