Revolution Lullabye

October 4, 2013

Gere et al, Local Assessment: Using Genre Analysis to Validate Directed Self-Placement

Gere, Anne Ruggles, et al. “Local Assessment: Using Genre Analysis to Validate Directed Self-Placement.” College Composition and Communication 64.4 (June 2013): 605-633.

Gere et al describe the revised Directed Self-Placement (DSP) system used by the Sweetland Center for Writing at the University of Michigan, arguing that the locally-developed and administered assessment achieves validity based on a study of placement essay that uses rhetorical move analysis and corpus-based text analysis.

The study of students’ placement essays shows that there are key textual and rhetorical differences between the essays written by students who self-selected into the FYW program instead of the credit-bearing PREP preparatory program. By coding the introductory paragraphs of the placement essays, the researchers determined both what constituted a “prototypical” introduction to an academic essay that articulated an argumentative stance in response to a text and what rhetorical and linguistic strategies are used by undergraduate FYW writers (as opposed to those writers less prepared for “college-level” writing.)

This study shows the benefits of using research and methodologies from linguistics in order to develop and evaluate local writing assessments. This essay also helps articulate more precisely what it means to say that undergraduate students are “good college writers” or have “rhetorical knowledge,” a goal stated in the Frameworks for Success in Postsecondary Writing document. In the end, this study also demonstrates what good local assessment looks like: a dynamic feedback loop that impacts instruction and a writing program’s definition of good writing.

 

Notable Notes

good argumentative writing has a “critical distance” that can be gleaned from the rhetorical and linguistic moves the student writer makes (623)

the revision of the DSP program in 2009 based on ten years of data (1998-2008). Their revision was based on three areas of research: research on writing prompts/assignments (resulted in giving students a reading and a specific prompt to create an academic argument, with explanations of what that means); research on rhetorical genre studies (influenced b Carolyn R. Miller’s ideas of genre as social action – genre not as fixed form but flexible and purposeful); text analysis methods used by ESP/linguistics, including corpus-based text analysis.)

attention to the “meso-level rhetorical actions” and the “micro-level linguistic resources” students bring to their writing (612).

three regularly occurring moves in text-based argument introductions: 1. establishing a background (not always there, so non-prototypical); 2. reviewing the article (either a Review-Summary or a Review-Evaluation); and 3. taking a stand (616). Gives examples from the student placement essays of these three rhetorical moves (617-619).

Used a software program (AntConc) to identify linguistic moves:

  1. “References to and citations from the source text
  2. Code glosses (e.g., in other words; in fact)
  3. Evidentials of deduction (e.g. therefore)
  4. Reporting verbs focused on processes of argumentation (e.g. argues, claims, asserts)
  5. Contrastive connectors (e.g. However, nevertheless) and denials (it is not...)
  6. Specific hedging devices associated with academic registers (e.g., perhaps, likely)…
  7. Self mentions (e.g. I and my), personalized stances (e.g. I agree)
  8. Boosters (e.g. clearly, certainly)”

FYW writers used more of #1-6 than PREP writers; FYW writers were less likely to use #7 and #8 (619-620)

PREP writers more likely to use “says, believes, thinks”; FYW writers more likely to use “argues, discusses, claims, asserts” (620) – reporting verbs

Sample coded FYW and PREP introduction in the appendix

tables of frequencies of certain linguistic features/moves (620-622).

push for genre-based pedagogies, teaching students to use genres as “guideposts” that help them solve rhetorical problems (625).

 

Quotable Quotes

“What our methods have helped us to do, however, is to tease out several linguistic features that, in this context, help to differentiate between students who are more and less at ease with projecting a novice academic stance” (623).

“By ‘meso-level rhetorical actions’ we mean the collections of communicative purposes in smaller sections of a text – larger than the sentence – that together construct the text’s overall pragmatic value as a message” (612).

“Often underconceptualized by those who create them, assignments play a significant role in students’ ability to perform well on a given writing task and therefore merit special attention in assessment” (610).

“Writing an evidence-based argument in response to a prompt like this requires not just arguing for one’s own opinion, but also identifying important propositions in the reading and then summarizing, analyzing, evaluating, and arguing for or against these propositions for using textual and other sources of evidence. Constructing such an argument also requires control of the necessary discursive resources for building an effective argumentative stance” (615).

“stance-taking” (615).

 

Advertisements

January 4, 2013

Adler-Kassner, The Companies We Keep

Adler-Kassner, Linda. “The Companies We Keep or The Companies We Would Like to Keep: Strategies and Tactics in Challenging Times.” WPA 36.1 (Fall/Winter 2012): 119-140.

In this article, based off of the author’s 2012 CWPA conference keynote address, Adler-Kassner calls on WPAs and writing studies scholars to be more proactive in the national conversations about what “college preparation” means (specifically what it means in terms of writing) and how that can and should be assessed.  WPAs need to articulate what it is that writing studies does (why the content of writing studies matters) and offer curricular and assessment strategies based on those basic writing studies principles.

Adler-Kassner points out that the conversations are already happening, and she describes five corporate organizations who are central in the drafting of education legislation and the construction and assessment of the Common Core State Standards.  These organizations are more powerful politically and financially than NCTE, MLA, and CWPA.  However, Adler-Kassner contends that this fact is not a reason why WPAs should give up. Rather, this is the time – while the Common Core is in its initial implementation – that WPAs need to work with K-12 educators to take ownership of writing curriculum and assessment.

Adler-Kassner points to the specific outcomes outlined by the DQP (the Degree Qualification Profile, developed by Lumina) to show that writing is cast as merely a skill – students are asked to produce forms of writing.  If writing is only seen as a tool, Adler-Kassner argues, then the discipline of writing studies is erased.  Adler-Kassner argues that WPAs need to emphasize the disciplinarity of writing studies in all writing classes, especially first-year writing classes, teaching students and other stakeholders the value of the central inquiries of the field.

Notable Notes

5 organizations that Adler-Kassner describes:

  • ALEC (American Legislative Executive Council)
  • VSA (Voluntary System of Accountability)
  • Lumina Foundation
  • DQP (Degree Qualification Profile)
  • Common Core State Standards

shift in the purpose of education to “college and career readiness,” a readiness achieved through emphasis of liberal-arts like skills (writing, communication, critical thinking.)  The ultimate purpose of 21st century education, as seen through these national discussions, is economic competition for employment (127-128).  Uses David Larabee’s analyses of public and higher education.

Her major three suggestions:

  1. “no vampires” – make writing courses focused on writing
  2. define what we think is college readiness (through documents like the Framework)
  3. build alliances with K-12 educators, even if we’re not thrilled with the standards they now must work with.

Quotable Quotes

Definition of writing studies:  “Writing Studies focuses on three things: 1. The roles that writers and writing perform in particular contexts; 2. The values reflected in writing and in those roles, and 3. The implications extending from relationships between roles, writing, and values” (131).

“This is because from a content-vacant, skills-oriented perspective, our discipline of Writing Studies is erased. Until we develop and act from principles about the meaning of what composition and writing studies is as a discipline, and then link what happens in composition courses – which exist within our discipline – to those principles, we are at the mercy of the companies seeking to keep our company. And to me, that’s a problem” (130).

“No vampires policy” – “Writing classes, especially first year classes, must absolutely and always be grounded in Writing Studies, must always be about the study of writing” (132).

“The key is to frame the study of writing wtihin the larger principle: that writing classes focus on the study of writing within particular contexts, the values reflected in that writing, and the implications of relationships between writing and values. Not vampires” (134).

“We must build alliances with colleagues who are immersed in efforts to implement the Common Core State Standards in Writing, especially K-12 colleagues, no matter how problematic we find those standards to be” (135). – if we don’t, there’s no chance of our voice being heard.  That’s the price we pay.

“I’ll begin, then, by updating the narrative that I’ve contended extends from documents like the Spellings Report. This narrative says that the purpose of postsecondary education is to prepare students for participation in the 21st century economy, but that faculty aren’t doing a good job with this preparation because we don’t understand what’s necessary for success.

“As I’ve said, answers to two key questions – what is meant by ‘preparation?’ And how should ‘how well’ be indicated? – are critical, because the responses provided to these questiosn will shape curriculum (and assessments)” (120).

September 7, 2012

Gallagher, The Trouble with Outcomes

Gallagher, Chris W. “The Trouble with Outcomes: Pragmatic Inquiry and Educational Aims.” College English 75.1 (September 2012): 42-60.

Gallagher uses Pragmatism philosophy to argue against outcome-based assessment, which he contends focuses on the ends of the educational experience, not the means, and argues for articulated assessment, which is an ongoing inquiry process that involves all stakeholders (teachers, students, program administrators) in determining what the hoped-for and actual consequences of an educational experience are, and evaluating and refining programs based on those.

Gallagher uses the CWPA’s “Outcomes Statement for First-Year Composition” and the Framework for Success in Postsecondary Writing to distinguish between outcomes and consequences. Outcomes, he argues, are fixed and rigid, explain what the conclusion of the educational experience should be, and are often handed down from administrators and professional groups to teachers and students.  Consequences, on the other hand, are more open to what happens, are emergent and developmental, emphasize potentiality, and can be surprising. Gallagher doesn’t demonize outcomes: he argues that the outcomes created by our field were constructed to prevent those not in the discipline from imposing their own outcomes, and he does explain how outcomes give programs a shared sense of common values and goals. However, Gallagher argues that when we read student texts through the lens of outcomes, we lose the student and the text: we search for what we want to find as evidence to meeting an outcome instead of reading what is there (and potentially there.)

Gallagher explains in the final section of his essay how he employed the Pragmatistic concept of atriculation to move the teachers, students, and administrators in his wriitng program away from setting pre-conceived outcomes and towards developing ways individual, internal goals for writing classess can be related to larger outcomes statements. The articulation processes he describes engage multiple people in the process, emphasize conversation that relate the program and classroom work to larger department/institutional/professionalal goals, happen in an ongoing, inquisitive process, and find ways to assess and track progress within the goals and also beyond, to the unintended consequences.

Notable Notes

teachers mapped their hopes/goals for writing classes, and those maps were used to discover areas of overlap or tension

John Dewey – pragmatism, inquiry-based learning and assessment.

need to find the spot between coherence and openness to opportunity, potentiality

Quotable Quotes

“Under these conditions, teachers and students merely receive the outcomes; they experience them as imposed, whether they were formulated by a distant regulatory body, a professional group, or some earlier incarnation of the local faculty” (45).

“If close attention to outcomes tends to narrow our view to what we wish to find, close attention to consequences broadens our view to include what we never thought to look for, opening us up (potentially!) to surprise and wonder” (48).

“Regardles of whether we find ourselves working (or choose to work) within the OA model, the challenge before us is to frame and use educational aims in ways that avoid the pernicious separation of means and ends, the rigidity of fixed ends, the narrow focus on predetermined results, and the imposition of external ends on faculty and students – while addressing institutional demands for assessment of student learning and maintaining program coherence.” (49).

Create a free website or blog at WordPress.com.