Revolution Lullabye

October 14, 2013

McLaughlin and Moore, Integrating Critical Thinking into the Assessment of College Writing

McLaughlin, Frost and Miriam Moore. “Integrating Critical Thinking into the Assessment of College Writing.” Teaching English in the Two-Year College 40.2 (December 2012): 145-162.

McLaughlin and Moore explain their study of how to assess critical thinking in college student essays. They developed a writing rubric intended to assess student writing across the disciplines, and then asked participants at the March 2011 Symposium on Thinking and Writing at the College Level to use the rubric to evaluate two student papers (both essays were written in response to a prompt that asked the student to define a term.) The results of the assessment surprised McLaughlin and Moore, as they assumed that one of the student essays was markedly stronger than the other. What they found was that the evaluators (80% of whom taught first-year writing in a variety of contexts) valued different attributes in student writing. McLaughlin and Moore argue that it is simpler to assess student writing based on attributes like “correctness” or “voice” instead of characteristics that point to critical thinking, like thoughtfulness, logical development, and consideration of alternative perspectives. They contend that the writing tasks students are given in K-12, which emphasize creative writing and the development of a strong, emotive voice, are distinctly different from the careful, reasoned academic writing (a very specific voice) that is hallmark of “college-level writing” and which is expected in first-year composition writing tasks.

Notable Notes

based the construction of their critical thinking in writing rubric (CTWR) on other rubrics designed by other institutions (Washington State University) and Bloom’s Revised Taxonomy (147)

categories of the CTWR: Focus, Logic (both of these first two categories contain language that incorporates elements of critical thinking), Content, Style, Correctness, Research (150).

keywords that point to critical thinking in these first two rubric categories: thoughtful, interpret evidence, draws warranted conclusions, analyzes alternative perspectives, evaluates when appropriate (150).

overemphasis on the construction of voice (155) – emotional voice (pathos) can mislead a reader where there is no logical, critical thought

college-level writing is mostly expository – requires a “drier” academic voice (156).

personal narrative v. critical analysis – writing tasks students are given in high school, college

the difficulty of capturing elements of critical thinking in a rubric – rubrics simplify writing, often assess what’s easy to assess instead of what’s the most important element (146-147).

Quotable Quotes

“College-level writing, it seems, values the well-reasoned point over its dramatic rendering. Perhaps reasoning, then, is a salient feature of college-level writing. Whether it is as important in high school writing is certainly worth examining in greater detail in the future” (157).

“In conclusion, the assessment of critical thinking takes time and often complicates the act of writing assessment.  Sometimes the most highly detalied and interesting student writing is not the product of complicated thinking but rather of strong feeling. Yet voice is not a substitute for thinking, though it can certainly enhance the expression of thought” (157).

“Without open-minded thinking as a basis of approaching the writing task – the thinking that prompts the writer to consider alternative approaches and possible outcomes – the writer may not achieve the level of reasoning that we expect in freshman writing. This thoughtful, fair-minded approach with its resulting careful reasoning, often expressed in a clear but neutral tone, may well be one of the distinguishing features of ‘college-level’ thinking and writing” (158).

 

April 15, 2009

Broad, What We Really Value

Broad, Bob. What We Really Value: Beyond Rubrics in Teaching and Assessing Writing. Logan: Utah State UP, 2003.

Broad introduces the practice of dynamic criteria mapping (DCM) as an inquiry-driven alternative to static, traditional rubrics that have a normative rather than descriptive function, not even addressing many of the things are taught in writing classes (therefore not a valid assessment). His book is a case study of the use of DCM at “City University,” a university with 4000 students in a 3-course English sequence that is assessed through portfolios, graded collectively by 3-teacher teams. Instead of starting with certain textual features to check off, DCM asks teachers and assessors to describe what they see in a text (good and bad.) Together, the instructors find synonyms and antonyms for what they notice, categorize similar ones, and create a visual map that illustrates the values about good writing that the program’s teachers hold collectively. This method, though time-consuming and messy, better articulates the complex processes and ideas that students are showing in their writing. The process is locally, site-baed: though the method of DCM can be used, individual maps cannot be transported across institutions or even across years; it should be a conversation about values that happens continually.

Quotable Quotes

“We can now face the truth equipped with tools (qualitative inquiry) and attitudes (hermeneutics) that help us tap the energy of apparant chaos without being consumed by it. We can embrace the life of things” (137).

“In their rush toward clarity, simplicity, brevity, and authority, traditional scoring guides make substantial knowledge claims based on inadequate research” (3)

“In pursuit of their normative and formative purposes, traditional rubrics surrender thier descriptive and informative potential: responsiveness, detail, and complexity in accounting for how writing is actually evaluated” (2).

“The age of the rubric has passed” (4)

Notable Notes

Vinland map – not appropriate now

move to validity(not the same as reliability)

the DCM finds textual criteria and contextual criteria (things not found in text but have an impact on assessing, before DCM these have not been visible)

benefits of DCM: 1. student learning (shows writing is more complex, they have a better understanding of what they’re doing well and  what teachers are looking for); 2. professional development and community; 3. program development and evaluation; 4. more valid assessment; 5. better relations with the public (values are made public, written down)

drawbacks? time-consuming and needs constant reflection and revisiting

must happen in communal writing assessment so there will be debate, disagreement, and discussion of values.

once the values are visible, you can start having conversations about whether you should value what you do.

a search for truth through hermeneutics, not psychometrics

Create a free website or blog at WordPress.com.