Revolution Lullabye

August 27, 2014

Newton, Value-added Modeling of Teacher Effectiveness

Newton, Xiaoxia A, et al. “Value-added Modeling of Teacher Effectiveness: Exploration of Stability across Models and Contexts.” Educational Policy Analysis Archives, 18 (23). 2010. Print.

Newton et al investigate measures of teacher effectiveness based on VAM (value-added modeling) to show that these measures, based on in large part on measured student learning gains, are not stable and can vary significantly across years, classes, and contexts. The study focused on 250 mathematics and ELA teachers and approximately 3500 students they taught at six high schools in the San Francisco Bay Area. The researchers argue that measures of teacher effectiveness based solely on student performance scores (those measures that don’t take into account student demographics and other differences) cannot be relied on to get a true understanding of a teacher’s effectiveness because so many other unstable variables impact those student test scores. Models of teacher evaluation that rely heavily on student performance scores can negatively impact teachers who teach in high-need areas. This is especially true with teachers who teach disadvantaged students or students with limited English proficiency.

Quotable Quotes

“Growing interest in tying student learning to educational accountability has stimulated unprecedented efforts to use high-stakes tests in the evaluation of individual teachers and schools. In the current policy climate, pupil learning is increasingly conceptualized as standardized test score gains, and methods to assess teacher effectiveness are increasingly grounded in what is broadly called value-added analysis. The inferences about individual teacher effects many policymakers would like to draw from such value-added analyses rely on very strong and often untestable statistical assumptions about the roles of schools, multiple teachers, student aptitudes and efforts, homes and families in producing measured student learning gains. These inferences also depend on sometimes problematic conceptualizations of learning embodied in assessments used to evaluate gains. Despite the statistical and measurement challenges, value-added models for estimating teacher effects have gained increasing attention among policy makers due to their conceptual and methodological appeal” (3).

Differences in teacher effectiveness in different classes: “An implicit assumption in the value-added literature is that measured teacher effects are stable across courses and time. Previous studies have found that this assumption is not generally met for estimates across different years. There has been less attention to the question of teacher effects across courses. One might expect that teacher effects could vary across courses for any number of reasons. For instance, a mathematics teacher might be better at teaching algebra than geometry, or an English teacher might be better at teaching literature than composition. Teachers may also be differentially adept at teaching new English learners, for example, or 2nd graders rather than 5th graders. It is also possible that, since tracking practices are common, especially at the secondary level, different classes might imply different student compositions, which can impact a teacher’s value-added rankings, as we saw in the previous section.” (12)

“the analyses suggested that teachers’ rankings were higher for courses with “high-track” students than for untracked classes” (13).

“These examples and our general findings highlight the challenge inherent in developing a value-added model that adequately captures teacher effectiveness, when teacher effectiveness itself is a variable with high levels of instability across contexts (i.e., types of courses, types of students, and year) as well as statistical models that make different assumptions about what exogenous influences should be controlled. Further, the contexts associated with instability are themselves highly relevant to the notion of teacher effectiveness” (16).

“The default assumption in the value-added literature is that teacher effects are a fixed construct that is independent of the context of teaching (e.g., types of courses, student demographic compositions in a class, and so on) and stable across time. Our empirical exploration of teacher effectiveness rankings across different courses and years suggested that this assumption is not consistent with reality. In particular, the fact that an individual student’s learning gain is heavily dependent upon who else is in his or her class, apart from the teacher, raises questions about our ability to isolate a teacher’s effect on an individual student’s learning, no matter how sophisticated the statistical model might be” (18).

“Our correlations indicate that even in the most complex models, a substantial portion of the variation in teacher rankings is attributable to selected student characteristics, which is troubling given the momentum gathering around VAM as a policy proposal. Even more troubling is the possibility that policies that rely primarily on student test score gains to evaluate teachers – especially when student characteristics are not taken into account at all (as in some widely used models) — could create disincentives for teachers to want to work with those students with the greatest needs” (18).

“Our conclusion is NOT that teachers do not matter. Rather, our findings suggest that we simply cannot measure precisely how much individual teachers contribute to student learning, given the other factors involved in the learning process, the current limitations of tests and methods, and the current state of our educational system” (20). 

Notable Notes

The problem of variables impacting the calculation of teacher effectiveness: the students’ background (socioeconomic, cultural, disability, language diversity), the effects of the school environment, how teachers perform year-to-year, the curriculum

VAM makes assumptions that schools, teachers, students, parents, curriculum, class sizes, school resources, and communities are similar.

The variables the researchers collected and measured included CST math or ELA scaled test scores, students’ prior test scores for both average and accelerated students, students’ race/ethnicity, gender, and ELL status, students’ parents’ educational level and participation in free or reduced school lunch, and individual school differences. Tries to look at the issue longitudinally by looking at student prior achievement (7). They were able to link students to teachers (8).

Advertisements

August 25, 2014

Popham, Tough Teacher Evaluation and Formative Assessment

Popham, W. James. “Tough Teacher Evaluation and Formative Assessment: Oil and Water?” Voices from the Middle 21.2 (December 2013): 10-14. Print.

Popham argues that teachers who commit to using formative assessment techniques in their classroom will have better student performance on the new Common Core student assessments, and therefore these teachers, whose evaluations increasingly depend on student performance on high-stakes assessments, will have better evaluations. Popham explains that although the high-stakes state and federal assessments seem to only value summative assessments, students and teachers who regularly do formative assessment do better on these tests. Popham’s article shows how federal policies, such as the 2009 “Race to the Top” initiative and the 2002 No Child Left Behind Act, not only change curriculum and testing but also change teacher evaluation. He explains how teacher evaluation criteria vary considerably from state to state and district to district. Althoguh there is great diversity in the measures and the relative weight of those measures used, Popham insists student performance scores on high-stakes assessments are going to continue being one of the most significant factors used to evaluate teacher performance and effectiveness.

Quotable Quotes

“In short, because students’ achievement will play such a prominent role in almost all states’ teacher-evaluation procedures, and because teachers who employ the formative-assessment process will almost always engender improved achievement in their students, this is precisely the moment when sensible teachers should learn to employ the formative-assessment process. The higher the stakes associated with a given teacher-evaluation system, the greater should be a teacher’s interest in becoming a skilled user of formative assessment. This is a classic “win-win” situation” (14).

Notable Notes

explains formative assessment – not a particular kind of assessment, but a process of using a few or occassional “checks” to determine how well students are learning and to adapt instruction based on that feedback. Describes it as a “means-ends approach” (11)

Analysis of the teacher-evaluation system: is it that simple, really?

March 6, 2014

Mayher, English Teacher Education as Literacy Teacher Education

Mayher, John S. “English Teacher Education as Literacy Teacher Education.” English Education 44.2 (January 2012): 180-187.

Mayher calls for English education programs to form alliances with colleagues and departments across their campuses in order to restructure English teacher education as literacy teacher education. Mayher argues that integration is necessary and overdue, especially in the context of the Common Core State Standards, which emphasize literacy education across the curriculum. Mayher points out that English education programs that focus primarily on literature are not preparing their students to teach all the students they will encounter in the 21st century American school system. Mayher calls on national organizations like CEE and NCTE to take the lead in conversations about literacy education, and part of taking this lead, he contends, is re-examining the assumptions that form our English teacher education programs.

Notable Notes

Mayher notes that there are at least four discrete teacher education disciplines that prepare students to teach literacy: secondary English education, early childhood/childhood education, TESOL, and special education. Added to that is the emergence of literacy education (K-12) programs (182).

the problem with specialization within national organizations – we’ve become fragmented, have different cultures, but we need to move beyond these cultural boundaries to work together to prepare teachers to meet the challenges of the modern American school system and CCSS. (184)

need to support beginning teachers after they graduate from teacher education programs – they still are not expert teachers and need ongoing mentoring and support. Teacher education programs need to reach out to the schools their students are placed in and give them more hands-on practice in how to teach – focus on “the transition to teaching” (186).

CCSS asks teachers from all subject areas to focus on literacy across the curriculum, something English teacher education programs should prepare their student to work towards in their schools (183).

English teacher education programs need “to be inclusive and therefore more than the traditional literature-centered subject English” (182).

The lit-centered English teacher education curriculum doesn’t prepare teachers to help students who are not proficient in the literacy skills they needed to master in the early grades (183).

Quotable Quotes

“The need for integration stems from the need to better serve the pupils our students will teach and to better prepare our students to do so” (182).

“Whatever one thinks of the ways being attempted to change schools, the fact is that the problems are real and we must play a central role in helping to solve them” (180).

“So if we are serious about building a teacher education culture that is responsive to the complex, interconnected, and integrated demands of K-12 teaching, we have to find ways to open up our curricular boxes and reconceive them across all the sub-specialties that touch on literacy education” (184).

“The common denominator here must be literacy teacher education for K-12 schools” (185).
“The Common Core, the NAEP, NCLB, and the state standards as well all recognize the centrality of literacy to the educational enterprise. What we must do is move into that center and claim the pedagogical high ground. Let’s change our name, our stance, and our capacity to collaborate with those who are also trying to help the children we serve. The time is now” (187).

October 13, 2013

Kroll, The End of the Community College Profession

Kroll, Keith. “The End of the Community College English Profession.” Teaching English in the Two-Year College 40.2 (December 2012): 118-129. Print.

Kroll argues that US community colleges have been overtaken by a neoliberal agenda, shifting the focus of education at community colleges from academics to vocational and career training. Kroll uses Giroux to define neoliberalism as both an economic and political movement that emphasizes private, corporate interests over the public good. Kroll contends that with this market-driven influence, community colleges will continue to prioritize the bottom line over what’s best for education, resulting in an increased reliance on contingent faculty and curriculum that responds to the needs of corporate America. He calls on faculty to teach critical literacy as a counternarrative in their own English and writing classrooms and to push back against this shift by taking on public intellectual roles.

Notable Notes

Courses are valued based on their perceived economic value

Large departments of contingent faculty overseen by a faculty manager (122)

Community colleges haven’t felt the pressure of professional guidelines on class size, etc. published by CCCC, MLA, NCTE (123)

Quotable Quotes

“The ‘grand experiment’ of the community college, as that of ‘Democracy’s college,’ is coming to an end. And with that ending comes the end of the community college’s academic function – that is, to provide an education– and concomitantly the community college English profession” (118).

“Neoliberalism imagines community college curricula as business-driven and focused on job (re)training; defines those who attend community colleges as economic entities:  ‘customers,’ ‘workers,’ and a ‘workforce’; and marks the end of a full-time faculty profession” (119)

“Consequently it has been rather easy for politicians, corporate America, and community college leaders to diminish or ignore the community college’s academic function while steadily turning them into job (re)training centers whose primary mission is to train workers to assume their subservient place in corporate America” (121).

August 23, 2012

Cambridge, Research and Policy: Antithetical or Complementary?

Cambridge, Barbara. “Research and Policy: Antithetical or Complementary?” WPA 35.1 (Fall/Winter 2011): 135-147.

Cambridge uses her experience meeting with the Secretary of Education Duncan with other NCTE representatives to reflect on the relationship between research and policy-making in the US and to call on compositionists and writing program administrators to use their expertise in writing teaching and learning to impact educational policy.

Cambridge resists the notion that legislators don’t listen or don’t care about writing research; she cites examples from studies done on the impact of research on policy to show that legislators and policy-makers are working in a complex political situation, so that their decisions do not always mesh with what we in the field think should happen. The research Cambridge cites does show that legislators do relies on “intermediaries”: trusted sources of research-based information on public policy issues.  Cambridge argues that writing program administrators and compositionists can become these intermediaries and impact public policy decisions if 1. we craft long-term, trusted relationships with policy makers; 2. become knowledgable about policy issues and build those into our research agenda so that we can give pertinent information in a timely manner so we become “go-to” experts; 3. teach public writing in our composition classroom so that our students have “rhetorical agency in the current political climate” (143); 4. argue for changes in tenure guidelines that count policy research and public policy work as research, not service; and 5. understand and communicate to others how important research is to policy work.

Notable Notes

Bogenschneider and Corbett study: how sound research affects policy: “allocations – altering how resources are distributed; tactics – altering how policies and programs are designed; solutions – altering how policies and programs are pursued; framework – altering how we basically think about certain social issues; salience – altering how much importance we give an issue; awareness – altering even how we think about doing policy” (qtd. 293) (146)

importance of narration in public writing and rhetoric. The Common Core has de-emphasized narration in favor of argumentation. Does that really prepare our students for being effective citizens in a democracy? What is lost? (144)

know your legislator = knowing your audience. It’s basic rhetorical strategy.

timeliness is key – research isn’t valuable unless it’s available as (or before) decisions are being made.

Quotable Quotes

“Unless colleges and universities wake up to the crisis in our political system; acknowledge their responsibility to address it in multiple ways, including figuring out how to generate and communicate research that applies to the system; and value those of its faculty members and administrators who develop expertise in that responsibility, colleges and universities are failing the society in which they operate.” (146).

 

December 29, 2011

Reid, “Preparing Writing Teachers”

Reid, E. Shelley. “Preparing Writing Teachers: A Case Study in Constructing a More Connected Future for CCCC and NCTE.” CCC 62.4 (June 2011): 687-703.

In the CCC Special Symposium on the NCTE/CCCC Relationship

Reid argues that the research, scholarship, and practice in the training of writing teachers, which she terms “writing pedagogy education,” can be fruitful ground for future collaborations between NCTE (focusing on K-12 English education) and CCCC (college composition and rhetoric.)  Reid claims that as a professional organization, CCCC has turned away from the practical issues of training teachers to teach writing.  She insists that scholarship on writing teacher preparation, instead being regulated to the margins of the field, as a solitary-institution specific practice or “sub-field” special interest group,  can bring together a variety of members of NCTE and CCCC in order to work on developing policy and practices for the training of writing teachers. In this way, Reid sees potential for a sub-field (writing pedagogy education) to revitalize larger disciplinary organizations.

Reid uses her own efforts to chair a SIG on the Education and Mentoring of TAs and Instructors in Composition and her work on the CCCC Committee on Preparing Teachers of Writing to show how difficult it was, with limited time and resources, to weave together local experiences of writing pedagogy education into a coherent, useful, and theorized whole about the preparation of teachers of writing. Reid calls on WPAs and those who train writing teachers to stop seeing themselves as “local practitioners” and rather, as part of a national, scholarly organization whose aim is to “articulate a larger vision” about writing pedagogy education (692-693). She argues that forums like SIGs and commissioned committees are not stable or sufficient enough to provide writing pedagogy education practicioners and researchers what they need: momentum and diversity of members. She suggests that CCCC follow NCTE’s lead and form a task force on writing pedagogy education, which could help create and support research grants, national studies, or online clearinghouses.

Reid points out specifically that “few studies of writing pedagogy education are data-driven, longitudinal, or inclusive of more than one program.” (692)

Notable Notes

Argues that scholarship in writing pedagogy education can address Patricia Stock’s 3rd question in what English education is: “(1) What is English? (2) How is English best taught and learned? and (3) How are teachers of English best prepared for their professional work?” (368, Stock “NCTE and the Preparation of Teachers of the English Language Arts,” 2010)

common problem in writing pedagogy education: the local: the isolation of individual writing programs, institution-specific needs and policies. No national network or conversation.

problems facing writing pedagogy education: How do you quantify teacher quality (tie in with national discussions on teacher tenure)? How can you measure writing learning as connected to teacher quality? How long does it take to develop good writing practices?  (692)

Move beyond the discussion of “what worked for us.” (692)

November 17, 2010

Little and Rose, A Home of Our Own

Little, Sherry Burgus, and Shirley K. Rose. “A Home of Our Own: Establishing a Department of Rhetoric and Writing Studies at San Diego State University.” WPA: Writing Program Administration 18.1-2 (Fall/Winter 1994): 16-28. Print.

Little and Rose describe how the stand-alone Department of Rhetoric and Writing were created at SDSU, explaining the changes that occured in the establishment of the new department, and argue that WPAs need more than good reasons for advocating for a separate writing program; they need to use rhetoric, good reasoning to argue for independence, which comes through an understanding of local institutional constraints, mission, and politics. They stress the importance of knowing university polity (organizational and governance structure); policy (principles and procedures for getting things done); and politics (who has power and sway, who are your allies.)

Notes and Quotes

go beyond the English department to the rest of the institution – get to know others in other departments.

Department of Rhetoric and Writing became independent at San Diego State University in May 1993 (Colgate, University of Arkansas at Little Rock, UT Austin all around the same time)

Little and Rose, rejecting the metaphor of divorce to describe the separation of composition from literature into independent departments, adopt Phelps’ metaphor of describing composition as a ‘grown child’ who needs a ‘home of her own’ as a separate and equal adult.

They give their responses and arguments to the following objections: 1. Writing has always been in English (just not historically or currently true) 2. study and teaching of writing is necessarily linked to the teaching and reading of literature 3. the writing program needs the English department for protection (placing it outside will strenghten it, showing connections to other disciplines besides just English 4. composition is not a legitimate discipline 5. English departments don’t marginalize composition (just look at the pervasive labor problem and assumption that no one wants to teach writing) 6. money isn’t an issue (it always is and composition is a very fiscally efficient and profitable enterprise) 7. loss of graduate TA lines in English 8. if English majors dry up, there won’t be composition classes for English faculty to teach

“Creating a separate writing department does not, then, separate reading from writing, but terminates the exclusive relationship between writing studies and literary studies” (20).

June 11, 2009

Schell and Stock, Moving a Mountain

Schell, Eileen E. and Pamela Lambert Stock. Moving a Mountain: Transforming the Role of Contingent Faculty in Composition Studies and Higher Education. Urbana: NCTE, 2001.

Schell and Stock have two main purposes for this collection of essays about contingent labor in composition: 1. to inform others in the field, especially WPAs, about the issues of contingent labor in composition teaching in the context of the changing 21st century university structure; and 2. to show the strategies some in the field are using to try to change the working conditions of contingent faculty (unionization, collective bargaining) with the hopes that these local changes can be the beginning of national policies. The collection consists of case studies from which guidelines can be extracted for working with contingent, non-tenure track faculty, including hiring practices, orientation, contracts, salaries and benefits, evalations, and professional development. Their collection concludes with essays that explain how non-tenure track faculty, who have become a staple labor force for the university, are instrumental to the 21st century university institutions want to become because of their willingness to take risks with new technology, to teach distance education online, and to engage in the scholarship of teaching.

Notable Notes

Schell’s essay – the 4 Cs: compensation, contracts, conditions, and coalition building. Turn to a “rhetoric of responsibility” between faculty, institutions, and students.

unions legitimize labor

advocate a proactive approach to the ethical problem of contingent labor

review of literature about contingent labor in the introduction, spans the 1980s (focus on social science and on the quality of teaching) through the 1990s (disciplinary attention and on working conditions, Wyoming Resolution)

lots of qualified people to fill non-tenure track contingent roles because of the explosion in MAs and PhDs

Create a free website or blog at WordPress.com.