Revolution Lullabye

April 15, 2009

Broad, What We Really Value

Broad, Bob. What We Really Value: Beyond Rubrics in Teaching and Assessing Writing. Logan: Utah State UP, 2003.

Broad introduces the practice of dynamic criteria mapping (DCM) as an inquiry-driven alternative to static, traditional rubrics that have a normative rather than descriptive function, not even addressing many of the things are taught in writing classes (therefore not a valid assessment). His book is a case study of the use of DCM at “City University,” a university with 4000 students in a 3-course English sequence that is assessed through portfolios, graded collectively by 3-teacher teams. Instead of starting with certain textual features to check off, DCM asks teachers and assessors to describe what they see in a text (good and bad.) Together, the instructors find synonyms and antonyms for what they notice, categorize similar ones, and create a visual map that illustrates the values about good writing that the program’s teachers hold collectively. This method, though time-consuming and messy, better articulates the complex processes and ideas that students are showing in their writing. The process is locally, site-baed: though the method of DCM can be used, individual maps cannot be transported across institutions or even across years; it should be a conversation about values that happens continually.

Quotable Quotes

“We can now face the truth equipped with tools (qualitative inquiry) and attitudes (hermeneutics) that help us tap the energy of apparant chaos without being consumed by it. We can embrace the life of things” (137).

“In their rush toward clarity, simplicity, brevity, and authority, traditional scoring guides make substantial knowledge claims based on inadequate research” (3)

“In pursuit of their normative and formative purposes, traditional rubrics surrender thier descriptive and informative potential: responsiveness, detail, and complexity in accounting for how writing is actually evaluated” (2).

“The age of the rubric has passed” (4)

Notable Notes

Vinland map – not appropriate now

move to validity(not the same as reliability)

the DCM finds textual criteria and contextual criteria (things not found in text but have an impact on assessing, before DCM these have not been visible)

benefits of DCM: 1. student learning (shows writing is more complex, they have a better understanding of what they’re doing well and  what teachers are looking for); 2. professional development and community; 3. program development and evaluation; 4. more valid assessment; 5. better relations with the public (values are made public, written down)

drawbacks? time-consuming and needs constant reflection and revisiting

must happen in communal writing assessment so there will be debate, disagreement, and discussion of values.

once the values are visible, you can start having conversations about whether you should value what you do.

a search for truth through hermeneutics, not psychometrics

Huot, (Re)Articulating Writing Assessment

Huot, Brian. (Re)Articulating Writing Assessment for Teaching and Learning. Logan, Utah: Utah State UP, 2002.

Assessment needs to be rearticulated by composition and rhetoric scholars as an important, necessary part of writing scholarship and teaching. Huot addresses assessment in a different way in each chapter (highlighting its connection to student response, teaching students self-assessment, need to create a field of writing assessment, and a history of writing assessment practices), but all of his studies and discussion point to central principles for his new theory and practice of writing assessment. Assessment must be site-based, locally controlled, context-sensitive, rhetorically-based, and accessible (to students, public, teachers, adminstrators.) Composition and rhetoric scholars and teachers are doing themselves no favors by abdicating assessment to education or to self-appointed writing assessment specialists; assessment is an issue that must be taken up by every WPA and teacher.

Quotable Quotes

“Instead of envisioning assessment as a way to enforce certain culturally positioned standards and refuse entrance to certain people and groups of people, we need to use our assessments to aid the learning environment for both teachers and students” (8).

“People who write well have hte ability to assess their own writing, and if we are to teach students to write successfully, then we have to teach them to assess their own writing” (10)

Notable Notes

assessment is articulating what we value; it marks our identities as teachers, programs, and a field; how do our judgments get articulated into our assessments?

Chapter 2 – need to connect comp/rhet with K-12 assessment to create  a writing assessment subfield, pooling knowledge and methods, talk about validity

Chapter 3 – need to teach students how to assess their own writing; writing as reflective judgment; use portfolios to full advantage

Chapter 4 – history of assessment practices

Chapter 5 – teacher response to student writing (draw on Phelps) and the contraint inherent in the act of reading

Chapter 6 – writing assessment is treated like a technology. It needs to be reimagined as research. This changes the role and activity of the assessors (151)

Chapter 7 – the practice of writing assessment needs to be reflective, conscious, theoretical, and instructive. Assessment can be social action, something that the field claims again, led by WPAs and teachers. (175)

movement away from objective rubric-like assessments, more based on community questions, inquiry, research, and practice

technocentric argument (Hawshier) – the tool of the assessment should not drive the practice

Blog at WordPress.com.