Broad, Bob. What We Really Value: Beyond Rubrics in Teaching and Assessing Writing. Logan: Utah State UP, 2003.
Broad introduces the practice of dynamic criteria mapping (DCM) as an inquiry-driven alternative to static, traditional rubrics that have a normative rather than descriptive function, not even addressing many of the things are taught in writing classes (therefore not a valid assessment). His book is a case study of the use of DCM at “City University,” a university with 4000 students in a 3-course English sequence that is assessed through portfolios, graded collectively by 3-teacher teams. Instead of starting with certain textual features to check off, DCM asks teachers and assessors to describe what they see in a text (good and bad.) Together, the instructors find synonyms and antonyms for what they notice, categorize similar ones, and create a visual map that illustrates the values about good writing that the program’s teachers hold collectively. This method, though time-consuming and messy, better articulates the complex processes and ideas that students are showing in their writing. The process is locally, site-baed: though the method of DCM can be used, individual maps cannot be transported across institutions or even across years; it should be a conversation about values that happens continually.
“We can now face the truth equipped with tools (qualitative inquiry) and attitudes (hermeneutics) that help us tap the energy of apparant chaos without being consumed by it. We can embrace the life of things” (137).
“In their rush toward clarity, simplicity, brevity, and authority, traditional scoring guides make substantial knowledge claims based on inadequate research” (3)
“In pursuit of their normative and formative purposes, traditional rubrics surrender thier descriptive and informative potential: responsiveness, detail, and complexity in accounting for how writing is actually evaluated” (2).
“The age of the rubric has passed” (4)
Vinland map – not appropriate now
move to validity(not the same as reliability)
the DCM finds textual criteria and contextual criteria (things not found in text but have an impact on assessing, before DCM these have not been visible)
benefits of DCM: 1. student learning (shows writing is more complex, they have a better understanding of what they’re doing well and what teachers are looking for); 2. professional development and community; 3. program development and evaluation; 4. more valid assessment; 5. better relations with the public (values are made public, written down)
drawbacks? time-consuming and needs constant reflection and revisiting
must happen in communal writing assessment so there will be debate, disagreement, and discussion of values.
once the values are visible, you can start having conversations about whether you should value what you do.
a search for truth through hermeneutics, not psychometrics